Check this out, Vladdy:
He's a BNer who did what you're saying - sucked it up and abandoned his previous plans, so that he could go into perfumery in school in France. He even learned French to do it. It's a tough road, though there are some other cases where people sort of dropped into perfumery accidentally with almost no effort.
I knew a couple of guys who went into the industry through my background, chemistry. They were natural product chemists who went into the big aromachemical houses after school. If you are more interested in simply working in the fragrance industry than perfumery per se, you could stay on your present course, but take every opportunity to work toward a specialty in fragrance-oriented Chem. E., and hope to move into that end of the industry down the line.
Also, remember that - just like in music - there are lots of professionals who are not stars but make a living and love their work. Hundreds or maybe thousands of regular Joes and Janes for every star. If you're willing to teach, lug your instrument to orchestra dates, sing clubs or play weddings, then in the perfume world, you could find yourself being a functional perfumer. And there's no shame in being an unknown perfumer. In a really great interview that I read, I learned who it is that Guerlain's Thierry Wasser admires: the functional perfumers who work on difficult chemical compositions!