juniorGradient Descent
What is convergence in Gradient Descent?
Updated May 16, 2026
Short answer
Convergence occurs when updates become very small and the algorithm reaches a minimum.
Deep explanation
Gradient Descent converges when gradients approach zero or changes in cost function become negligible. It may converge to local or global minima depending on the function.
Real-world example
Model training stopping when validation loss stabilizes.
Common mistakes
- Assuming convergence always means global optimum.
Follow-up questions
- What affects convergence speed?
- Can GD fail to converge?