What is gradient checkpointing tradeoff analysis?

Checkpointing reduces memory but increases compute due to recomputation.

It saves only selected activations and recomputes intermediate ones during backward pass. This reduces memory from O(n) to O(sqrt(n)) in deep networks but increases runtime overhead.