Unable to Refund Allocated Keystone (Bulwark) After Using Oracle Ascendancy

Here are the steps where the issue occurred:

I used a From Nothing (Diamond) jewel at the Unwavering Stance Keystone location to allocate the Bulwark Keystone.

I then used the Oracle Ascendancy notable Entwined Realities to allocate two notables:

Warlord Berserker

Warlord Leader

After that, I refunded both notables (Warlord Berserker and Warlord Leader).

However, after refunding these two notables, I am unable to refund the Bulwark Keystone, which prevents me from removing the From Nothing jewel from the passive skill tree.