seniorGradient Descent
What is stochastic gradient Langevin dynamics (SGLD)?
Updated May 16, 2026
Short answer
SGLD adds Gaussian noise to SGD updates to approximate Bayesian posterior sampling.
Deep explanation
Stochastic Gradient Langevin Dynamics (SGLD) extends SGD by injecting calibrated Gaussian noise into parameter updates. This transforms optimization into a stochastic process that approximates sampling from the posterior distribution in Bayesian learning. Unlike SGD, which converges to a point estimate, SGLD explores the parameter space, capturing uncertainty in model parameters. It is particularly useful in Bayesian deep learning and uncertainty estimation.
Real-world example
Used in Bayesian neural networks to estimate prediction uncertainty in medical diagnosis systems.
Common mistakes
- Confusing SGLD with regular SGD with dropout noise
- SGLD noise is theoretically calibrated.
Follow-up questions
- How does SGLD relate to MCMC?
- Why does noise scale with learning rate?