Interesting chronography results found here: http://www.leverguns.com/articles/ballisticians.htm.
You'll see that there is a lot of variation from gun to gun, and that some of the 4" guns even shot as fast as the 6" guns with the same loads. Thus, I'd not concern myself with ballistics much. I'd choose based on ease of shooting versus portability. For me, the handier, more portable 4" gun would win out in a wilderness gun, or one that may be used for home protection. Either is fine for casual target shooting.