What is learning rate decay?
Updated May 16, 2026
Short answer
Learning rate decay gradually reduces learning rate over time.
Deep explanation
Decay strategies help stabilize training by taking large steps early and smaller steps later, improving convergence to minima.
Real-world example
Deep neural network training stabilization.
Common mistakes
- Reducing learning rate too aggressively.
Follow-up questions
- What are decay strategies?
- Why decay helps?