Modern factory-made ammunition will last a long, long time, given cool, dry storage conditions.
However, when it has been loaded into a pistol, the story changes somewhat.

If the cartridges, or some of the cartridges, in a magazine have been cycled through the action (chambered), there is a possibility that doing so has pushed the cartridges' bullets a little deeper into their cases. If this is repeated sufficiently, the result of firing the cartridges may be excessive pressure and the remote chance of doing serious damage to gun or hand.
Therefore, it is best to occasionally fire-off the magazine-full of cartridges, some of which have been chambered more than once. Doing so should afford you some good practice time, and will assure you that your weapon and its ammunition continues to function well.

I agree.

I take pains not to cycle the same round all the time when loading or unloading, and when I start to lose track of which rounds have been cycled, I just go ahead and shoot the whole magazine.

+1 for Steve and Bisley, setback is a major issue, especially in .40 S&W. Another problem that could occur is from over oiling. I have had .22 rounds that went bad because oil had made it's way inside the cartridge somehow.

If you shoot your EDC as often as you should, you won't have any problems.