Yes, I think this is always the question as someone gets further and further into fragrances. I have, after three years, found many scents that I am totally comfortable wearing in any circumstance (and I work in a hospital, so I have to be somewhat aware of that). But it is an odd thing, what people will accept. I recently started giving away my unwanted samples to a friend who is not that perfume aware. Oddly enough, she loved Le Labo Iris 39, which I would have thought was somewhat an "advanced" fragrance. She immediately fell in love. So who knows. I think anyone out there, if they are willing to step outside the department store comfort zone, can find things they like that are certainly not mainstream.
My go to scents are probably not necessarily mainstream, but I do find them unoffensive enough to turn to them time and time again. Many of the Chanel Exclusifs are included in that, so not really too out there, even though most people have not really smelled them or been exposed to them.
Wear what you like, as long as you are aware of the sometimes oddness of our hobby, and our increased exposure to different smells.
I consider myself and wearer, not a collector. If I keep a bottle, it is because I will wear it. Possibly under certain circumstances, but I will wear it.