I'd say that if they moved because of recoil, then you can probably move them by hand. It's most likely that it is not the case.
Do you shoot often?
Did you start shooting faster than at first?
Did you change your stance?
Was there a girl present? (showoff hehe)
Do your wrists hurt the next morning?
I ask these things because...
If you don't shoot often, you will fatigue and shots will go to crap.
If you started shooting faster than at first you may not have had complete control over the pistol.
Changing your stance can change everything including point of impact.
Girls tend you make men do "macho" stuff, seriously. (hehe)
If your wrists hurt after shooting, hit the weights a little, get a racketball to squeeze during the day. It'll help alot.