removing tree stumps ...

This actually makes the solution easy. Buy some Ammonium Nitrate (AN) - you can order it online, buy it at the farm supply, etc. Alternatives would be similar Nitrogen-rich products like Potassium Nitrate (KN) or even cow manure (slower, but same process). Drill several large bore holes in the top of the stump, fill up the holes (wider and deeper holes are better) with the AN. Add water one time. This Nitrogen overdose is going to speed up nature's process of rotting the tree - it's going to be 8-12 weeks before you see any results, so this is not a fast option, but it is very effective. The side effect of using a nitrogen base is that this is the same stuff they use in fertilizer, so its not going to wipe out your garden in the process. You could end up with too much in the garden soil, but that usually isn't a problem and can be mitigated by using manure instead.

That being said, I like using AN because it is the most common oxidizer used for highly stable, but very explosive combinations that are legal to use on your own property (Federally- I have no idea what your local laws are) as long as you mix them the day you use them, and don't store them mixed. One common mixture is 5% Aluminum powder (buy the smallest you can find, such as 2 micron) and 95% AN (or 10%/90%) - this is relatively safe to handle (you can hit with a hammer, shake it like a shake-weight, etc. and you really need to mix it well), can be set off with a high caliber rifle (270, 30-06, etc.) from a distance, and will destroy the stump with even a small charge (i.e. 1Kg) appropriately placed if you get tired of the "fast rot" method described above. This is the same premise as the binary targets you buy at stores under brand names such as Tannerite, although they may have additional chemicals in theirs. AN is highly hygroscopic, meaning it will absorb water very quickly, even from the air, so keep it dry and in sealed bags - this is related to why it rots stumps and why it is used in cold packs (water+AN gets cold very quickly as the AN absorbs water).