Please let us know what you are shooting. Nothing wrong with these. Were the shots in the middle from just 1, 5 shot group or a mix of two groups? I went through a time where I would shoot 5 shot groups with 4 at sub 3" but always have 1 flier out of the 5 and never the same number shot. Sometimes the first, sometimes the fifth or anyone in between. It was just a lack of concentration.
Realize, however, that if you are shooting for home or personal defense, your groups current are plenty tight. To me it's like golf, I played and practiced but one day something just clicked for me to get really good and shoot a round under 80. It took 10 years of shooting in the 80's and 90's to get there.
KEEP AT IT.