Can’t pickup items

Since we have no mouse, we’re dependent on the game to auto-lock to enemies and to loot.
The issue is pretty simple. Whenever there are enemies around I cannot choose loot.

There needs to be a way to override the default behavior of the game of preferring to lock to enemies than to lock to loot.
In the Trials there are often situations where we are asked to pick some orbs but I cannot complete them because I cannot lock the item while enemies keep coming…

Xbox Series X.

Example: Trial of Chaos.
I get to the Soul Cores but I cannot pick them because there are always enemies spawning and stopping me to pick the soul cores…

Thanks to @Tagz from Forever Exiled Discord I got a good way forward.
By not having a skill assigned to the ‘A’ button it will allow that to only interact/loot.
I lost a skill button but it’s good enough :)

(Seems an intended feature and not a bug)