What is XGBoost and why is it widely used in ensembles?

XGBoost is a highly optimized gradient boosting framework designed for speed, regularization, and scalability.

XGBoost (Extreme Gradient Boosting) improves traditional gradient boosting by introducing regularization (L1 and L2), parallel tree construction, missing value handling, and efficient memory usage. It uses second-order gradients (Hessian information) to improve optimization accuracy. It also includes pruning techniques and sparsity-aware split finding, making it extremely effective for structured/tabular data.