What is exploding gradient problem?
Updated May 16, 2026
Short answer
Exploding gradient occurs when gradients become excessively large during training.
Deep explanation
When gradients grow exponentially during backpropagation, parameter updates become unstable and training diverges. It often occurs in deep or recurrent networks.
Real-world example
RNN training becoming numerically unstable.
Common mistakes
- Ignoring gradient clipping techniques.
Follow-up questions
- What is gradient clipping?
- How is it different from vanishing gradient?