What is the bias-variance tradeoff in ensemble learning?
Updated May 16, 2026
Short answer
Bagging reduces variance, boosting reduces bias, and ensembles balance both to improve generalization.
Deep explanation
The bias-variance tradeoff explains prediction error as a combination of bias (systematic error) and variance (sensitivity to data fluctuations). Ensemble learning helps manage this tradeoff. Bagging methods like Random Forest reduce variance by averaging multiple models trained on different bootstrapped samples. Boosting methods like Gradient Boosting reduce bias by sequentially correcting errors. Stacking can balance both by learning optimal combinations of diverse models. The key insight is that ensembles improve generalization by stabilizing predictions or refining decision boundaries depending on the strategy.
Real-world example
In credit scoring, Random Forest reduces volatility in predictions, while boosting improves detection of subtle fraud patterns.
Common mistakes
- Assuming all ensemble methods reduce both bias and variance equally.
Follow-up questions
- Why does bagging reduce variance?
- Why can boosting increase overfitting?