Ancestral Warrior Totems Crashes Maps

I'm playing with 2 friends and I use Ancestral Warrior Totems with spirits enabled from the Warbringer Ascendancy. If a player joins the map while I am placing Ancestral totems, the instance crashes, logs us all out, and we lose the waystone that's active.

I have Sunder slotted in my ancestral totem. It may be an issue with how Ancestral totems load an extra entity(spirit) to use my skill gems slotted in the totem. I have tested it with 10 waystones in a row(some T1 stones so I didn't lose any valuable ones) and it has crashed every time without fail. It doesn't crash with my shockwave totems, only my ancestral spirit totems.