Periodic freezes and game reload

My game will periodically freeze during which I cannot do anything. After something like 20-ish seconds it will unfreeze with a black screen and the GGG gears spinning in the level. It appears it is having to reload everything. After a bit it will finish loading and the game runs as normal.

While it is frozen the game does not seem to pause since after it unfreezes, even if I hit escape while it is frozen, I will have moved.

This has happened since 0.3.0 at least.

Specs:
Ryzen 7 9700X
RTX 4070
32 GB DDR5
Windows 10

Bug Report ID: #1150544749

I've been praying this is fixed in 0.4.0... looks like not
This is why I had to stop 0.3.0...

But now before I get to maps... the zones now take 1-2 minutes to load
Used to be a few seconds...