If you were to ask the average person whether sugar was a spice, they'd probably say no.
I googled "Is sugar a spice?" and the results that came up were illogical: "No, sugar is an ingredient" (as if other spices are not!); "Sweet is not a flavour" (just plain wrong); and "Sugar is not a spice because it is not grown as a plant. Sugar is simply glucose." The latter implies sugar springs from nothingness, when actually, it can be processed from many different plants.
The Oxford English Dictionary defines spices as "strongly flavoured or aromatic substances of vegetable origin obtained from tropical plants". Sugar has a strong flavour, is of vegetable origin and can be made from tropical plants.
Sounds like a spice to me.
If you want to argue otherwise because it's usually used in large quantities, I'd ask; when has ubiquity changed the definition of something?
White sugar is the most refined variety and, therefore, has the least character - it just adds sweetness (although there's nothing wrong with that!). It's made by extracting the juice out of a plant (such as sugar cane or sugar beets), boiling it to evaporate the liquid, then refining it to remove the impurities that colour it and affect the flavour.