What is impurity reduction bias in Decision Trees?

Impurity reduction bias refers to decision trees favoring features with many unique values or high cardinality.

Decision Trees select splits based on impurity reduction metrics like Gini or entropy. Features with many unique values (e.g., ID-like features) can artificially create pure splits, even if they carry little predictive meaning. This introduces bias toward high-cardinality features. Gain ratio and regularization methods help mitigate this issue.