Raise Spectre weapon swap Corpse despawn bug(100% reproduction rate)

I've been having issues with my Perfect Forest Warriors and other Warlock bought corpses from the Wildwood disappearing in hideout seemingly without explanation. And from tinkering with my gems I thought about what was in my weapon swaps and a massive bug/issue is going on here.

Preface:
1) I'm using Replica Bitterdream so my Raise Spectre gem is on my weapon(weapon swapping without any other condition drops the Perfect corpses normally)
2) I have a Melee Splash setup for another gem, SRS, which I occasionally swap to Multistrike for bossing(Multistrike is located on my weapon swap)
3) So sometimes I weapon swap to slot in Multi, but if I ever do that, I ALWAYS go back to hideout BEFORE swapping weapons.
4) If I weapon swap, #1 happens and the corpses are dropped, I switch SRS supports and then swap back to Bitterdream mainhand and reraise the corpses.

The Bug:
1) I wanted to level a Raise Spectre of Transience gem for another build.
2) I slot it in my weapon swap
3) Weapon swapping now DELETES the corpses. It's not like some sort of triggered ability like carrion golem or an offering, they are just straight up gone into the void.
4) Weapon swapping back to the original Raise Spectre gem doesn't bring back the corpses.

I have a feeling I guess this has something to do with the game treating all spectres as from the current gem you have equipped or something? And if I weapon swapped to a different gem that wasnt the one that originally raised the corpses it deletes it or something.

Anyways, I lost like 30c so far from this bug, luckily its not much more but it means you CANNOT level another Spectre gem on weapon swap if you also have one on MH/OF.

Another way to circumvent this is I guess to slot Raise Spectre NOT in your MH/OH that way when you weapon swap they don't get dropped.

Still really annoying to deal with though as I thought the gems shouldn't have anything to do with each other(kinda like how you can swap gems for Animate Guardian and it still keeps all the same gear).

Sounds like issue I am having. Weapon swap - spectres are gone and no corpses on the ground. I do not use anything that can consume corpses, just marionetes for aura setup.
EDIT: Corpses drop after swap, ut if you leave area (your ho as an example) they are gone.