seniorGradient Descent
What is loss landscape connectivity in Gradient Descent?
Updated May 16, 2026
Short answer
Loss landscape connectivity describes how different minima are connected in parameter space.
Deep explanation
Research shows that in over-parameterized neural networks, many minima are connected by low-loss paths. Gradient Descent may move within connected basins, explaining why different solutions often have similar performance.
Real-world example
Mode connectivity in deep learning loss surfaces.
Common mistakes
- Assuming minima are isolated points.
Follow-up questions
- What is mode connectivity?
- Why does it matter?