It is not an uncommon fault. Quite a few barrel and key sets were changed under warranty due to stiff locks and broken keys. They revised the key slightly to help fix the problem, but all they did in reality was shorten it!
You will need to remove the barrel from the pannier to extract the broken bit of key. Quite how you're going to do that with the pannier locked shut I don't know? Can't really help anymore than that as my Yamaha panniers were disposed of many years ago (before they broke)!
Replacement locks and keys were available from dealers. Whether they still are is another matter.
Have you read
this thread? Also
this.