Thank you for your input!
So the knob would still be an offset that is not quantized, but all quantized notes would still be semitones with respect to each other. That's well worth considering as the offset can be full range, or just 1V (another option limits the knob's deviation) so would be easy to handle.
I had thought of a different action for the knob when there is no CV input patched in. The knob (while quantizing option is on) becomes quantized to be useful (like a glissando) but now not so sure ;)