I went to a talk of David Jordan who was explaining what factorization algebras are, and how they can be used to quantize character varieties. As far as I understood, a factorization structure on a manifold is some locally constant cosheaf of algebras in which the corestriction is an equivalence whenever the inclusion of open sets is an equivalence, or some such thing. The data required to define such a structure is just the E-whatever algebra it associates to a disk. Putting defects in, i.e considering this on stratified spaces, makes sense, and corresponds to bimodules of such algebras.

So one should ask: if the factorization structure one obtains from $O(G)$ is related in some particular way to the character variety, if we put in defects and specify singular support conditions (i.e. we allow our cosheaf of algebras to be constructible rather than locally constant, make sense of the notion of singular support, and demand that it lie in the legendrian), there should be some story like: from a parabolic structure on $G$, we can define a knotty factorization structure on a surface decorated with knots, and its knotty factorization homology should be related to the knotty character variety in the same way as the usual factorization homology is related to the usual character variety.