It's always been a dilemma , go lightweight which is great on dirt but 'twitchy' at speed on the tar, or a bit more weight, great at high speed but can be a bit more awkward on the dirt.
I loved my time on my old modied DRZ 400 E, but was starting to feel a bit more nervous venturing further afield, so upgraded to a Tenere which is a lot safer at highway speed.
You just got to be honest with yourself in deciding what you want to be doing with the bike most of the time.