seniorSupervised Learning
What is stochasticity in supervised learning and why is it important?
Updated May 17, 2026
Short answer
Stochasticity refers to randomness introduced in training processes to improve generalization and optimization.
Deep explanation
Stochasticity appears in supervised learning through random initialization, mini-batch sampling, and stochastic optimization methods like SGD. Instead of computing exact gradients over the full dataset, stochastic methods use random subsets, which introduces noise. This noise helps escape saddle points and poor local minima, leading to better generalization in high-dimensional optimization problems.
Real-world example
Training deep learning models where each epoch shuffles data to avoid learning order bias.
Common mistakes
- Assuming deterministic training is always better for stability.
Follow-up questions
- Why does randomness help optimization?
- What is deterministic vs stochastic training?