If we don't do stupid things, avoid stupid people, and use some common sense about where we go and when, the chances of needing a pistol are very slim. Which is not to say it never happens, like the recent mall murders or the doctor's family in Connecticut, but staying reasonably alert and avoiding places where crime is likely to occur goes much further toward keeping us safe than any weapon we can carry.
By the way, walking around the house with a loaded pistol in hand is probably not the best idea in the world, especially if you're going outside with it. Consider a holster if you want to keep the pistol ready all the time. A holstered weapon is a safe weapon, and also a far less threatening one.