COE is bugged

When I use Cast on Elemental Ailment, it sometimes breaks the gem, meaning I can’t use it. It says “missing socketed spell,” even though I have a spell in it that worked fine before. I think this occurs when I remove some armor or something that has Spirit on it. I now have two broken COE gems that can’t be used. Please help or fix this.