Spell Totem charge generation

I wanted to play spell totem, but charge generation is just exhausting early. If I can't generate charges, why bother with skill? Why bother with skill, if I have to pick another skill I don't want to play to get through 20+ hours of campaign/farming to buy unique that allows me to generate charges? If I use some skill like devour corpse, I have to constantly keep checking if I got 1-2-or-3 charges and I can finally summon spell totem, when most enemies are already dead and my charges are likely to expire before I get to more enemies - not to mention bosses without charge generation. I can't keep track of both charge generation and enemies attacking me efficiently.

Wouldn't it be more sensible to use skill from the beginning and build your character around it, rather than rush entire campaign with different skill and respecing? What was the reasoning of having charges as requirement rather than pure bonus?

I gave up after 10 hours act 1-3.