midPyTorch
What is gradient clipping in PyTorch?
Updated May 17, 2026
Short answer
Gradient clipping prevents exploding gradients by limiting their magnitude.
Deep explanation
It scales gradients if they exceed a threshold to stabilize training.
Real-world example
Used in RNN and transformer training.
Common mistakes
- Clipping after optimizer step instead of before.
Follow-up questions
- What is exploding gradient problem?
- Difference between norm and value clipping?