seniorGradient Descent
What is gradient flow (continuous-time Gradient Descent)?
Updated May 16, 2026
Short answer
Gradient flow is the continuous-time limit of Gradient Descent.
Deep explanation
When learning rate approaches zero, Gradient Descent becomes a differential equation called gradient flow: dw/dt = -∇f(w). This formulation allows analysis using dynamical systems theory and provides deeper insight into convergence behavior and stability.
Real-world example
Theoretical analysis of neural network training dynamics.
Common mistakes
- Thinking gradient flow is a separate algorithm rather than a limit.
Follow-up questions
- Why is it useful?
- What connects it to physics?