Would you still wear fragrances, and spend as much as you do now on them?
Seriously, my main goal isn't pleasing other people, but one of the main things I like about fragrance is that I can share them with other people. I send decants to my friends. And, I don't pursue compliments but I do enjoy them.
I would still probably own fragrances, but it probably wouldn't be as much of a priority. That, and there would be no fragrance community, further diminishing the point of even owning a collection.