Check farmer's markets. I get most of mine at a weekly Amish Market. Prices aren't bad and they have had just about everything I've ever looked for. They have a dedicated stall for spices and flavorings.
I get all my spices and such at Bulk Barn and casings at the local TSC store. Some not so standard spices , like curing salts and soy concentrate, i would order online, as these are not to be found around here.
I ended up ordering from americanspice.com and conyeagerspice.com
Found lots of cool things that I didn't even know existed. Got some ideas to try later.
keep posting your favorite places it all good help.
One thing I like is it was cheaper than driving to the store for me. With flat rate shipping it wasn't to bad. And I think the quality/freshness is better.
Most everything I buy, which isn't too much, comes from either Butchers & Packers, and I have bought once from American Spice company. They have the fullest collection of spices I know of, but they are proud of what they have.
My normal spices are McCormick. 40 years of consistent spices. Black Peppers and extracts from Watkins. Ask your grandma about Watkins black pepper, its still as good too!