I'd say, sniff and sniff and sniff, and for every perfume that you sniff, read the notes list. At first you'll say, "Yeah, galbanum, whatever", but at some point you'll have smelled and read "galbanum" enough times that you'll suddenly be able to pick it out. And the same for the other notes.
I don't think that you need to seek out single-note scents, but if you send off for samples rather than sniffing what you can find at the department store, you could get sets of samples around a theme - five sandalwood-focused scents this month, five "green" scents next month, and so on. You could also focus on scents that have at least one major note that you already know - rose, citrus, coffee, vanilla and so on. That could give you a base for picking out the other notes.
You can also go out of your way to sniff things in your daily life. For example, when you're at a grocery that sells fresh herbs, you could smell each kind. And then when you're home you could smell any jars of dried herbs that you have. Sniff the peel of a lemon compared to the flesh, and the same for an orange. If you eat a salad of mixed greens, try eating the different leaves individually, to get the different tastes, which are at least partly different smells. Sniff the eraser of a new pencil, the cellophane around a new CD, a new book, an old book, a fresh newspaper. Sniff everything.