What is gradient boosting and how does it differ from AdaBoost?

Gradient boosting optimizes a loss function using gradients, while AdaBoost reweights misclassified samples.

AdaBoost adjusts sample weights based on classification errors, forcing subsequent models to focus on difficult cases. Gradient boosting, on the other hand, builds models that fit the negative gradient of a loss function, making it more flexible and general. Gradient boosting supports arbitrary differentiable loss functions, making it more powerful and widely used in modern systems like XGBoost and LightGBM.