Once, probably no damage done. Do it several times, well, it's hard to know how much is TOO much. When a 1911-style gun's hammer slams forward without the slide/firing pin in place to stop it, the hammer strikes the frame at the thinnest point in the hammer's construction. This can stress it the hardened steel, causing a crack and eventual failure.
These photos are of a full-size 1911 hammer (Colt, I believe; it's been a few....decades) which broke-off when the shooter was cleaning his field-stripped weapon and accidentally pulled the trigger. I asked for the part (parts?) so I could use it as a training aid in weapon classes, and it still resides in my gun parts drawer.
"Placement is power" -- seen in an article by Stephen A. Camp
(RIP, Mr. Camp; you will be remembered, and missed)