juniorGradient Descent
What is step size in Gradient Descent?
Updated May 16, 2026
Short answer
Step size is the amount by which parameters are updated in each iteration.
Deep explanation
Step size is controlled by learning rate and gradient magnitude. It determines how far the algorithm moves in parameter space per iteration.
Real-world example
Adjusting thermostat temperature gradually toward target.
Common mistakes
- Confusing step size with gradient magnitude.
Follow-up questions
- Can step size change dynamically?
- What happens with large steps?