What is entropy in probability and why is it important in ML?

Entropy measures uncertainty in a probability distribution and is widely used in ML loss functions.

Entropy quantifies unpredictability. Higher entropy means more uncertainty. It is central in decision trees, cross-entropy loss, and information theory-based ML models.