Passing DC to your amp and speakers: this video should explain very quickly why you don't want that to happen. Ever.
My suggestion to all of this is to actually use a...yep...Eurorack stereo mixer with proper VCAs, CV control over panning, AUX send/returns, and so forth. Yes, they cost money and yes, they take up space. But they're the right choice inasmuch as, in this build, they're the right tool for the job. If the idea here is to control the shifting from sound to sound in an even manner via modulation, sure, you could kludge together a bunch of VCAs and attenuators and so forth. Or just do this, plus an isolating output module to both attenuate the synth-level stereo feed and to provide isolation to keep from passing DC, prevent ground loop and noise issues, etc.
Let's look at this...right now, you have a Rosie as a "mixer" (it technically isn't...it's a crossfader with a cue line and headphone amp) and you're considering using the Quad VCA and two A-183-1s as VCA + attenuation (which, I note, does ZERO for DC issues and noise/ground loops between your mixer and the modular). This, totalled in hp, is 30 hp altogether. And in monetary terms, all of those are $408.
Without chucking the Quad VCA (you need that!), let's see what we can do. My suggestion would be to look at the Qu-bit Mixology, which is one of the cheaper and smaller full-CV stereo output mixers at 28 hp and $399. This gives you full CV over levels, AUX sends, and panning over four channels, plus stereo AUX return, metering, mute and solo switches. Adding a suitable and well-featured stereo output for this leads to Bastl's Ciao!, which offers balanced TRS 1/4" outputs, two stereo input pairs (the second can be used for a second signal chain, or for paralleling another stereo effect), headphone amp, clip indicators, and so on in 5 hp at $122(ish). So, $113 higher in price but MUCH more extensive in terms of features. And only 3 hp bigger in terms of "footprint". Frankly, the price and size difference vs. benefit...for me, at least...would point me more in the direction of the Mixology/Ciao! pair. It does everything necessary for not much more in about the same space.