This post has been edited to reflect current conditions as of 4/11
Yale Center for British Art doesn't carry any other fragrances-- only Penhaligon's. And not all of them. They don't have Sartorial-- for that, go to Saks in Greenwich.
Saks 5th Avenue Greenwich for Creed, Jo Malone, Bond No. 9, a very few of Annick Goutal. They no longer have Armani Privé. They have a number of the Penhaligon's in 100 ml sizes and they carry Chantecaille, Kiehl's and some others.
Edit: Current staff at Saks in Greenwich do not allow the decanting of samples any longer, but if they have manufacturer samples they will give you one or two.
Dovecote in Westport (on Rt. 1, opposite the YMCA) now only carries Kai.
Parfumerie Douglas is now Bluemercury2 but the stock (for now) is much the same as it was when owned by Parfumerie Douglas. They do not allow you to decant samples, unlike Bluemercury across the street.
L'Occitane at West Farms mall in Farmington, and on Main Street Westport and Greenwich Avenue Greenwich and in the same plaza as L.L. Bean store (Windsor?) The store in WestFarms closed.
Achorn's Pharmacy on Main Street, Westport, has some things one doesn't find everywhere, like Canoe, and Taylor of London's Worth a browse if you are in the area.
EDIT: As of May 2012 Achorn's Pharmacy is moving to a site on the Post Road.
Pacifica fragrances at Whole Foods (Westport and Greenwich; West Hartford store no longer carries them) and the museum store of the Peabody. Dava in Hamden has candles.
Sephora in various locations including Greenwich Avenue Greenwich, Danbury Fair mall and West Farms Mall.
Nordstrom (at the mall in Farmington) has Geir Ness, some Annick Goutals (butrarely the newest ones). They also carry Laura Mercier. No longer carry Armani Privé, but now they have a Jo Malone boutique in-store.
Recently we visited a shop that had some perfumes I'd never seen or smelled before: Parfums du Jour at Blue Back Square, West Hartford. She has Annick Goutals also. Google Blue Back Square for a map of the shops there and contact information. EDIT 11/26/11: I no longer see this store in the directory of Blue Back Square and have not been able to make phone contact. Assume the store is closed unless you can make contact.
Villa Fleur --independent perfume shop just outside WalMart at Hamden Village Fair Shopping Center, Dixwell Avenue. Has full size bottles and minis (including, occasionally, hard to find ones)
Flutterbye on the green in Guilford. Clothing, gifts, and a few scents including K. Hall and Pacifica.
Edit:The Bath Tub in Mystic has closed.
Edit: Foxglove in Madison, CT, is no more; there is now a cheese shop but no fragrances.[b]