juniorGradient Descent
What is Gradient Descent?
Updated May 16, 2026
Short answer
Gradient Descent is an optimization algorithm that minimizes a function by iteratively moving in the direction of steepest descent.
Deep explanation
Gradient Descent is used to minimize a cost or loss function by computing its gradient (partial derivatives) and updating parameters in the opposite direction. It is foundational in machine learning for training models by reducing prediction error.
Real-world example
Training a linear regression model by minimizing mean squared error.
Common mistakes
- Using too large learning rates causing divergence.
Follow-up questions
- Why do we move opposite to gradient?
- What happens if learning rate is zero?