or go very low cost diy - beg/borrow/steal tools and an off cut of wood - there are some cheap ok diy psus out there too (frequency central for example)
and build up modules slowly over time - either diy or used or 'budget'
if you want to keep costs down don't go for expensive trendy modules (unless you can find a used bargain) go for simple building blocks - eg ladik and doepfer modules or similar in diy

"some of the best base-level info to remember can be found in Jim's sigfile" @Lugia

Utility modules are the dull polish that makes the shiny modules actually shine!!!

sound sources < sound modifiers < modulation sources < utilities