Meditation skill from Invoker ascendency can't be assigned with specific weapon set

There's no option to choose which weapon set to use when casting the skill. Is this a bug or working as intended?

I want to cast this skill with my second weapon set and my attack skills with my first weapon set. Right now I have to manually switch between weapon set with X key before each cast.