seniorPyTorch
What is gradient noise and how does it affect training?
Updated May 17, 2026
Short answer
Gradient noise is stochastic variation in gradient estimates from mini-batches.
Deep explanation
Because gradients are computed on subsets of data, they introduce noise which can help escape local minima but also slow convergence if excessive.
Unlock with a Pro subscription to view this section.
View pricingReal-world example
No real-world example available yet.
Unlock with a Pro subscription to view this section.
Upgrade to ProCommon mistakes
No common mistakes listed yet.
Unlock with a Pro subscription to view this section.
Upgrade to ProFollow-up questions
No follow-up questions available yet.
Unlock with a Pro subscription to view this section.
Upgrade to Pro