Suffered the same thing, wind was catching me just under the peak of my lid so I got a Metal Mule screen � it's eased the problem considerably.

Still have buffeting probs across my shoulders from time to time though.

For info I'm 6'1" and wear a Hornet DS � other heights and lid combinations might be better, or worse