Luckily, it's never been an issue for me. Is it really an issue for you? What I mean is, lots of people don't "get" those of us who are into perfume, and frankly there isn't much we can do about that (although some of us are lucky and our SOs take an interest). I wouldn't really call that an issue, though.
If it's causing serious problems in your relationship, that's another story. Frankly, I believe in live and let live.