juniorEnsemble Learning
What is the difference between bagging and boosting?
Updated May 16, 2026
Short answer
Bagging trains models independently in parallel, while boosting trains models sequentially to correct errors.
Deep explanation
Bagging (Bootstrap Aggregation) builds multiple models on different random subsets of data and averages their predictions to reduce variance. Boosting builds models sequentially, where each new model focuses on correcting errors made by previous models, reducing bias. Bagging is robust to overfitting, while boosting is more sensitive but often more accurate.
Real-world example
Random Forest uses bagging, while XGBoost uses boosting.
Common mistakes
- Thinking boosting and bagging are interchangeable.
Follow-up questions
- Which is more prone to overfitting?
- Why is bagging parallelizable?