seniorGradient Descent
What is the role of Lipschitz continuity in Gradient Descent?
Updated May 16, 2026
Short answer
Lipschitz continuity bounds how fast gradients can change.
Deep explanation
A function is Lipschitz smooth if its gradients do not change too abruptly. This ensures stable updates in Gradient Descent and helps derive convergence guarantees. The Lipschitz constant controls maximum safe learning rate.
Real-world example
Training stable neural networks with bounded gradients.
Common mistakes
- Using overly large learning rates ignoring smoothness.
Follow-up questions
- What is L in practice?
- Why is it important?