Disconnecting in maps

Having an issue on my Xbox series x. I keep getting disconnected while running maps, the odd thing is I will run a simple icmp ping to anything outside my network and my latency spikes and I start timing out as soon as the game forces a game pause on me. Then about 10-15 seconds later it disconnects me. This only happens when I'm playing POE 2, don't think its coincidental or my internet connection causing it because I sat here for 15 minutes watching my icmp pings and its been running smoothly with only a few minor latency spikes from time to time (perfectly normal). It looks like POE2 is somehow locking up my router until the game disconnects, then my pings stabilize again.