Okay, I'm a *real* cheapass, so I'm not about to go out and buy any high-falootin' labware! Which leaves....what? Well so far I've been using glass jars, those work very well. You have to be real careful around heat because the soda-lime glass they're made of does not like shock. Boil water in it then add some cold water down the side and you're going to lose it I promise you. Microwaving even has its troubles, but is usually okay.
For larger containers that don't need to handle much heat, milk jugs are a good item. I like to cut the tops off so I have a handle and a big mouth to add stuff. The high density polyethylene (HDPE, recyclable #2) can stand boiling water without shrinking, melting or deforming - it does get soft, so be careful.
Now that you've got something to put it in, what do you want to make? And with what! Now in my area I have trouble finding damn near anything, for instance I have yet to see nitrate fertilizer. I haven't even seen copper sulfate (although I'll admit I haven't looked for it real hard ;). But there's still a good bit you can do with a few common OTC acids, bases, metals, salts, oxidizing and reducing agents. Pottery supplies also have a variety of chemicals, alkali fluxes, transition metal colorants, opacifiers and glass-formers, "acidic" refractories and glass-formers, and several chemicals related to making slip: flocculants and deflocculants mostly. Pottery supply warehouses are a bit sparser than hardware stores, but there ought to be one within 50 miles wherever you are.