juniorGradient Descent
What is learning rate in Gradient Descent?
Updated May 16, 2026
Short answer
Learning rate controls the step size during parameter updates.
Deep explanation
The learning rate determines how big each update step is in Gradient Descent. A small learning rate leads to slow convergence, while a large one can overshoot minima or diverge.
Real-world example
Adjusting speed while driving toward a destination to avoid overshooting.
Common mistakes
- Using a fixed learning rate without tuning or scheduling.
Follow-up questions
- What is learning rate scheduling?
- What happens with high learning rate?