First and foremost.... DUCK!
Secondly, wear a hat.
Thirdly and more to the point. I'd shoot thru at least 500 or so rounds before I started worrying. After that, you have to ask yourself what changed between the first 100 rounds and now, if they are still hitting you in the face. Try it from a rest, see if they are still ejecting in that direction. If not, then the pistol is probably reacting to how you are holding it, not just randomly kicking them back into your cranium. Make some notes as to how you are holding the pistol, have someone take a picture, post it here and we can go from there. Good luck.