Thread: Sample Based Eurorack
Modular gear allows for very detailed levels of control by reducing inputs and outputs to absolutely basic levels of performance.
The use of samples or digitally recorded audio in modular is more of an adjunct than a method for sample playback. You'd have less expense and more success using something like an Akai MPC Live MkII. It will allow for you to load libraries of sounds as well as interface with the modular synthesis world.
I would work from that angle.