seniorGradient Descent
What is stochastic resonance in Gradient Descent?
Updated May 16, 2026
Short answer
Stochastic resonance describes how noise can improve optimization performance.
Deep explanation
In Gradient Descent, moderate noise can help the optimizer escape shallow minima and saddle points, improving convergence. This phenomenon is called stochastic resonance, where noise enhances system response rather than degrading it.
Real-world example
SGD outperforming batch GD in deep learning tasks.
Common mistakes
- Assuming noise is always harmful.
Follow-up questions
- What is optimal noise level?
- Where is it observed?