What is entropy in data mining and why is it important?

Entropy measures the uncertainty or impurity in a dataset.

Entropy comes from information theory and quantifies unpredictability in data. In classification problems, it measures how mixed class labels are. Higher entropy means more disorder. It is foundational in decision tree algorithms like ID3 and C4.5, where splits are chosen to reduce entropy and increase information gain. A perfectly pure dataset has zero entropy.