seniorGradient Descent
What is saddle point problem in Gradient Descent?
Updated May 16, 2026
Short answer
A saddle point is where gradient is zero but it is neither a minimum nor maximum.
Deep explanation
Saddle points are common in high-dimensional optimization. Gradient Descent can get stuck because gradients vanish, even though better directions exist. This is especially common in deep neural networks.
Real-world example
Deep learning loss surfaces with flat unstable regions.
Common mistakes
- Confusing saddle points with local minima.
Follow-up questions
- Why are saddle points problematic?
- How to escape saddle points?