Game freezes randomly, seeming to cause TDR

When playing PoE2, since around new years, I have started getting random freezes where the game stops, pauses (as in the game pauses) and then after a few seconds, the world goes black but the UI stays open and you can use the UI, then the cogwheels comes out and the game loads again (in the black screen) and then the game unpauses and continues.

From what I have figured out, its some sort of new TDR issue thats caused by PoE 2.

My hardware:

AMD 7950x Ryzen 9
Nvidia 4090
64 gb DDR5 Ram

...

I have tried the following with no change:

Put in a TdrDelay in my regedit for my graphics driver.
Increase cache size to 100 GB
Reinstall clean drivers
Reinstall confirmed stable driver (566.36)

This seems to be related to DX12, though I havent checked if it happens on vulcan since vulcans performance isnt as good and the only issue with DX12 I have is this TDR issue.

My computer is fully updated with all drivers.

My only theory is something was changed in a hotfix that suddenly started causing this.

I have the same issue, this happens randomly and does not depend instance.

AMD ryzen 7 5700X3D
Nvidia 3060 Ti
32gb ram
m2 ssd


Drivers updated etc.

same problem