What is the mathematical decomposition of bias and variance in supervised learning?
Updated May 15, 2026
Short answer
The expected prediction error can be decomposed into bias², variance, and irreducible error.
Deep explanation
In supervised learning, the expected squared error at a point x is decomposed as: E[(y - f̂(x))²] = Bias² + Variance + Irreducible Error. Bias measures systematic error from wrong assumptions. Variance measures sensitivity to training data changes. Irreducible error is noise inherent in the data that no model can eliminate. This decomposition explains why improving one component often worsens another.
Real-world example
Predicting house prices where measurement errors (noise) cannot be removed even with perfect models.
Common mistakes
- Ignoring irreducible error and assuming models can achieve zero error.
Follow-up questions
- Why is bias squared instead of bias?
- Can variance be eliminated completely?