Our food mostly gets spoiled by bacteria and / or fungi. There are 2 main reasons for sugar crystals not getting spoiled.
1- Lack of sufficient amount of water to sustain metabolic processes by microbes.
2- Even if sufficient amount of water is present , then due to high osmolality of the fluid formed, will cause plasmolysis or cell shrinkage.
Some technical data:
> Microorganisms differ widely in their ability to resist salt- or sugar-induced reductions of aw. Most disease-causing bacteria do not grow below 0.94 aw (roughly 10 percent sodium chloride concentration), whereas most molds that spoil foods grow at an aw as low as 0.80, corresponding to highly concentrated salt or sugar solutions.
Source: <