seniorGradient Descent
What is loss surface geometry in Gradient Descent?
Updated May 16, 2026
Short answer
Loss surface geometry describes the shape of the optimization landscape.
Deep explanation
Loss surfaces in deep learning are high-dimensional landscapes with valleys, plateaus, saddle points, and sharp minima. Gradient Descent behavior depends heavily on this geometry, affecting convergence speed and generalization.
Real-world example
Neural network training dynamics depending on architecture.
Common mistakes
- Assuming convex-like geometry in deep networks.
Follow-up questions
- What are sharp vs flat minima?
- Why is geometry important?