I have been hesitant to jump into the fray here (because I'm a pussy!).
Honestly, unless people know the fragrance, most people won't have any idea if what you are wearing is men's or women's unless you tell them. I was wearing Donna Karan Chaos at a picnic at my daughter's school about a month ago, sat down at a crowded picnic table and the lady next to me, exclaims, "who is wearing that wonderful scent?" People looked around, and it was evident it was me. She sniffed my wrist and exclaimed how good it was and how it suited me. She proclaimed it smelled like French perfume, "quality". Turns out, interestingly enough, that this was not some uninformed amateur but a woman who owned a perfume shop for over a decade here in town selling Creed and L'Artisan and Nicolai fragrances among others. She closed shop several years ago to retire (grandmother of one of my daughter's classmates), and I am interesting in seeing what she may have stashed away in her closets someday leftover from her old stock. The fact is, by far, most of the time, the only one who knows what you are wearing is YOU.
I wear fragrances marketed as women's about 2 or 3 days a week, being in the "If I like it, I will wear it" camp. On the other hand, I do have some sensitivity about certain situations and may avoid wearing things too floral or incense-y or Patchouli-y on office or on-call days. Why? I hadn't thought too much about it, but this thread has made me think. I do have a certain consciousness about drawing attention to myself at times and things that may play a role. Smelling like a woman has never come too much to mind; I got more compliments wearing O de Lancome than anything else this year and I wasn't ashamed to mention what it was to the ladies who complimented it. I think much of this sensitivity is an overread by those of us who pay WAY too much attention to fragrance in our day to day lives than the average man or woman. If even professionals and afficionados aren't sure the "gender" of your fragrance, why would "normal" people care?