Well, first off, what constitutes a "proper sequencer"? Do you want something that does a full complement of CVs, gate/trigs, in multiple rows with typical end/start/stop/etc control? Or are you talking about just for drums, in which case a trigger pattern sequencer might make more sense. Or a digital full-on controller like a Squarp Hermod or Orthogonal's ER system? Each has pros and cons, but some are suited for one use, and others for...well, others.
Best suggestion I can make: don't freak out. Take your time, use MG's resources to get a better feel for what to do. This isn't a test, nor does the first build you do have to nail the idea you have. Building an instrument more or less from scratch (which is pretty much what modular synthesis is about) is not anything close to easy and at first glance, it's daunting as hell. Expect to refine and whittle away at design ideas for literally months until you hit something that feels right. And the best thing is that if you utterly and irredeemably fuck up your design here on MG, you just delete it and start over. No massive cash outlay and resulting buyer's remorse. Learn at a pace that works for you, and tweak away without worry.
The other suggestion is to start with something too big. If you think you need two rows of 104 hp, go with three, or three rows of 126 hp, even. Allow yourself space in which to screw up ably, then pare that result down, and you'll find that you actually get decent ideas from 'going too far' and then pulling back from that point. And study existing builds by experienced synthesists as well as classic modular systems that still inspire awe perhaps 50+ years on to get ideas of how to proceed. Just don't demand of yourself that this get done NOWNOWNOW...you'll just wind up banging your head against the display screen.