Well, since the 1970's in the western countries at least, the rich have been getting richer faster, while the rest have either been stagnating or falling behind. When you're a seller, it's alot easier to target a smaller group of very rich people who are faithful, regular customers, who can indulge all of their wants, and charge them ever increasing prices, than to have to serve large numbers of people. Just imagein having a restaurant - if you made 1$ per plate, you'd have to have so many employees, and work so hard serving 100 customers to make $1000 dollars a day. But if you could make $1000 of one plate, your life would e so much easier - as long as you make sure that your plate is excellent. If you lose alot of lower end customers, who cares if the ones on the top are more than picking up the slack.
Look at the exploding number of fragrances that come out each year, vs previous decades, where a house *might* come out with one every year. That's because there are some people who can actually buy all those new releases, vs in the past.
Remember the 20/80 rule in business, that 20% of your customers account for 80% of your sales. As incomes get polarized, so do those numbers.
That said, niche perfumes would be expensive anyways (though maybe a bit less) because the materials are just expensive.