I would guess (being a lefty too) that it's trigger finger related. It sounds to me like you are using too much trigger finger because of the force need to pull the double action trigger. Try it in single action mode and use more of the pad of your finger - then see if the group moves toward the center. That will at least give you and idea of whether it's sights or a trigger finger issue.
But if you decide to move the sights, move the rear sight in the direction you want the group to move - so in your case, move the rear sight to the left, to move the group left. It's going to be a slight move. I'd mark the center of the rear sight with a pencil and extend the line to the slide. Use the sight-pusher (I assume that's what you bought) to move the rear sight about the width of a sharp pencil line to start. Hope this helps.