The first blow to the knee knocks him sideways, and the second one knocks him to the ground in a tangle of fabric. His arms move up to defend his head, which leaves other parts unprotected and vulnerable to the pipe in Tyrion's hands. He waits for the blows to stop, then straightens himself up (while still seated on the ground; he doesn't trust his knees to hold his weight at the moment) and dusts himself off.

He takes a moment to catch his breath and when his reply comes it's measured, precise, betraying no emotions. He won't give this little man the satisfaction of knowing how much pain he inflicted.

"Doesn't a man even get a chance to defend himself?"

He doesn't even know from what. Things were definitely not in this state between him and Tyrion when he set out for the Eyrie.