Can't Insert the Three Fates into the Trialmaster's Tower

I've beaten the chaos trials with 75+ ultimatum's multiple times (at least 10 times). I finally have all three fates and tried to click on the door to insert the fates, but nothing happens. I can't insert anything. When I think about it, I have NEVER been able to insert anything into the Trialmaster door. Any ideas?

Reference screenshot: https://imgur.com/a/ApjNF0H