Here are a couple of places one might chance upon discontinued gems and other mainstream fragrances on the cheap:
High Street Centre, North Bridge Road (directly opposite The Parliament House)
They close early on Saturdays, and do not open on Sunday. The shop has three sections, with the third one located a few units away. Make sure you visit all three. The last time I was there I saw bottles of Yohji Homme, Shalimar Light, La Nuit.
Little India, Serangoon Road
They are open around the clock. They have three wings, with perfume sections in all three. The newest wing, the one with the wavy reflective facade, mostly have newer frags. The older wings may be better hunting grounds, i.e. Serangoon Plaza and the building next to it. In Serangoon Plaza, the perfume section is on the second floor. In the other wing, it is in between the CD/DVD section and the toiletries.
If you are on Orchard Road and have no time to go anywhere else; there are so many discounters here, it may not be a good idea to buy without comparing prices between the shops first.