If its the link bar that goes from the rear of the pannier frames and through the rear wheel space than it should be used. When ever the panniers are not on its not a big problem but when you put them on an load them up and hit the bumps on our lovely paved roads your panniers will bounce about the place putting more pressure on the locks and other stuff.
Mate had an Aprilia Pegaso that had a pannier system with no link bar and just so funny
seeing him hit a bump, the bike would go down and then the panniers would wobble and look as if thier coming off.