Skills become grayed out after dying in multi-player xbox

So when playing multiplayer for some reason anytime I die it grays out all my skills and I can't use any of them and just become a brick. The only way to get your skills to work again is by leaving g the instance and going back to town which already is inconvenient. But when it comes to boss battles if you die once might aswell restart the whole fight because you will be completely useless until go can go back to town and you can't portal during boss fights. And this literally happens every single time I die in multiplayer