I smell fragrance on people all the time! The lady that works in the local store is always wearing Miss Dior, a gent that is usually on the same bus as me in the morning seems to alternate between Rive Gauche and Kouros (always hope for a Rive Gauche day - haven't noticed a pattern yet...), an lady that lives a few doors away always smells of Youth Dew.
Most of the women in my city wear sweet fragrances (I notice a lot of Be Delicious) and the men usually wear Joop, JPG Le Male, or something citrus/aquatic (so much smells citrus/aquatic that I can never be sure), I very rarely smell anything oriental unless it's on an older woman unfortunately.
I can't fathom a reason why you don't smell fragrance from others. Perhaps they simply do not wear it! (What is wrong with them??! lol)