seniorGradient Descent
What is saddle escape in high-dimensional Gradient Descent?
Updated May 16, 2026
Short answer
Saddle escape refers to techniques used to move out of flat saddle regions in loss surfaces.
Deep explanation
In high-dimensional spaces, saddle points are more common than local minima. Gradient Descent can stall due to near-zero gradients. Methods like noise injection, momentum, and adaptive optimizers help escape these regions efficiently.
Real-world example
Training deep neural networks where optimization plateaus temporarily.
Common mistakes
- Confusing saddle points with convergence.
Follow-up questions
- Why are saddle points common in deep learning?
- Which optimizers help escape?