Of course. I often wear perfumes that are not exactly loved by my husband or friends, just because I like them a lot. But if someone around me (colleagues, friends etc) is truly bothered or made uncomfortable by my perfume, I avoid that perfume around them.
We all consider smells to be an important aspect of our lives. Why not accept that they might be important for others too? There are always choices that can make ourselves happy and the others -well, at least, not miserable. Ideally, perfume appreciation should make us better people, not even more self involved and inconsiderate, in my opinion.