seniorGradient Descent
What is noise-driven escape in optimization?
Updated May 16, 2026
Short answer
Noise-driven escape uses stochasticity to move out of saddle points and local minima.
Deep explanation
In stochastic optimization, random noise from mini-batches or injected perturbations helps the optimizer escape flat or suboptimal regions. This is crucial in high-dimensional non-convex problems where deterministic gradients may stall.
Real-world example
Training deep networks where SGD noise prevents stagnation.
Common mistakes
- Trying to eliminate all noise in training.
Follow-up questions
- Is noise always beneficial?
- What controls noise level?