Possible latency spikes/connection issue?

So the thing is, whenever I play PoE, the latency jumps to 10k/20k/999k(i tried to save screenshots of biggest latency ever for Guinness records) the internet
connection also just "dies", browser stops working, all streams/youtube videos in background die, etc.

After some time it goes back up, in like a minute or two. And I tried speed test, I tried heavy Steam game downloads, everything goes fine right untill I open up PoE and log in.

Sometimes I lag out in HO, sometimes in map, sometimes even in only one specific part of map, like literally right now, I was booted out in bottom left corner of Atol (and I kept going there like an idiot due to uncompleted ritual).

I have no clue what could be causing it, but ISP said that all is good, which I know they will always say, but on other hand on PoE has this issue.
And yes, I tried different gates, my most used is Frankfurt, but neither London, Amsterdam, Milan, and even more distant ones work, internet still "dies".

If anyone has any clue or sollution, I am happy to try it.
If PC specs are needed(which is doubtful due to it being latency issue) I can post them.