Where the reason for the connection is synesthesia -- sensory input from one sense being interpreted by the brain in terms of another sense -- don't think you'll be able to communicate colors reliably to others, as that is very individual.
Where the reason is a connection connected by habit and experience, where a person thinks about color at all from a smell, there is the obvious of for example the fragrance "orange" suggesting the color orange, what we call green fragrances suggesting green, etc. For your vanilla example the reason might be a visual association on your part between the smell and color of vanilla beans, or vanilla extract, both of which are very dark. But for me, for example, it's a sunny fragrance suggestive of very light colors, I suppose very light yellow. Probably not as a synesthesia, but perhaps from association with smelling it while seeing light colored foods. Though to some extent, it could be from effect on mood. Don't know.
Anyway, that would be an example of different perception between persons.