What is exploding gradient problem?
Updated May 15, 2026
Short answer
Exploding gradients occur when updates become excessively large.
Deep explanation
This destabilizes cost optimization, especially in deep networks.
Real-world example
Training deep RNNs without gradient clipping.
Common mistakes
- Ignoring gradient normalization.
Follow-up questions
- How to fix exploding gradients?
- What is vanishing gradient?