Show Recommended Gems Option Not Present

While using a gamepad/controller the option to uncheck show recommended gems isn't there. If i swap to mouse and keyboard I can uncheck the box, but when I go back to the controller all the gems are hidden again. Any idea if this is meant to be unavailable while using a controller?

I've scrubbed reddit and google to figure this out for about an hour and I didn't see anything about keyboard vs controller for this option displaying. Been through the options menu multiple times and can't find anything that would affect it. Also checked and unchecked anything that seemed related in any way.