How budget are we talking? If I search around enough I can usually find a good number of designer scents for under twenty dollars. Particularly Jaques Bogart and Antonio Puig seem to sell their stuff very cheap. Also the Versace 'jeans' scents.
Cheaper than that?
Well, I've always liked Tabu, and I'm rather fond of Dana's Monsier Musk (pure leathery/soapy goodness) and English Leather, a classic 'dad's cologne,' with it's unique lime and leather scent. If you buy the big bottles these are insanely cheap. I have a 250ml bottle of English Leather (I'm really fond of the bottle too) which I paid $17 for. Then of course there's the various Jovans. Jovan Musk for men is pretty solid for the price. Another around this price is the reformulated Worth Pour Homme, while I've heard some praise for the reformulated Je Reviens although I can't go near it myself because I have too many memories attatched to the original.
I remember being bored and spraying one of Tabac's fragrances. Tabac Man Silver, and I thought it was wearable and reasonably pleasant which I found surprising because I was expecting to be horrified. Tabac Original EDC is also rather nice. Simmilarly there's a few Adidas scents which seem nice enough in a generic, sporty kind of way.
Often designer flops which get discontinued can become incredibly cheap. Then ironically they can suddenly become very expensive once the shelf stock has dryed up and only Ebay types and the like have them. Examples of this are Diesel Plus Plus (femenine and masculine) which can be had for $10 a bottle, and which - to me - are really quite nice.
On overstock I've picked up one ounce bottles of designer fragrances like Zino, Dirty English, Lanvin Vetyver, Cool Water Pure Pacific, and Nautica Voyage all for 10-15 dollars. Also on Overstock-Perfumes I sometimes find Roger & Gallet testers for $12. I bought two bottles of Gingembre that way.
Buying in large sizes can make budget buys out of scents which might not seem budget initially. For example Molinard's EDC comes in for me at under $30 for 250mls. That's 125mls for $15, cheap any way you look at it. Simmilarly Caron's Pour un Homme can be purchased in absolutely crazy bottle sizes which brings the $ per ml price down to a pittance. Azzaro also have some big bottles for Chrome and Pour Homme which brings the price down substantially. This can be a very inexpensive way to buy a scent you really like and wear often.
Halston scents seem to be very cheap, and I've heard friends say that Vicky Tiel's ulysses is quite good. Also something called Caution by Kraft. I'm tempted to get it because I dig the bottle design:
If you're interested I'll tell you where you can buy genuine vintage Worth Pour Homme HC for 13 dollars a bottle. Not joking, and I've bought it myself so I know it's legit. Deals like that bring me to another point. If you ever find a source like that it can be worth buying a bunch of bottles and trading them for other stuff.
In the end? My favourite way to find bargains is to go to a big website like overstock or a huge ebay seller like bestdealperfume and list all of their fragrances, then sort by price and start at the lowest. You'd be surprised what you can find.
As for 'dollar store' type fragrances the only one I've ever owned was something called Perfect Black. I believe it was supposed to be a copy of Drakkar Noir. I was on Holiday in a very small town with no fragrances, and ended up buying it and CKIN2U which was all they had which didn't make me gag. I actually wore it much more often than IN2U and have used the bottle, while IN2U languishes still in my wardrobe. I occasionally use it as air freshener.
Oh, I should note. It was 'Perfect Black,' which brought me to basenotes. I was looking up and notes, but couldn't find it. However it lead me to discover Mandarina Duck Pure Black (one of my all-time favourite modern scents) which I bought because the notes and the name were so cool. Plus I like ducks.
Edit - something for the ladies: Love's Baby Soft (yep, you can still get it XD)
Also: http://www.overstock.com/Health-Beau...t=Lowest+Price link to Overstock set to lowest price.
Edit again: also Fragonard Zizanie. Super cheap for what you get.