Armour Breaker skill unable to quickly rebreak armour

I am playing a thorns build (because I like off-meta and bad builds) and running into a problem where for one or two attacks after the Broken Armour debuff is removed via Exploit Weakness, any hits with the Armour Breaker skill do not contribute to breaking enemy armour.

My current setup is a quality Solar orb to remove broken armour and I even tried Overabundance so I had two pulsing at the same time and still had the same problem. Easily replicatable with the same setup and any fast attacking one hand mace.

Scavenged plating gives me a large amount of thorns and 70% more armor at full stacks, but the delay in rebreaking armour makes keeping full stacks on single target extremely difficult.