What is impurity-based vs permutation feature importance in Decision Trees?

Impurity-based importance is computed during training, while permutation importance measures performance drop after feature shuffling.

Impurity-based feature importance (used in Decision Trees and Random Forests) sums the reduction in Gini or entropy contributed by each feature across all splits. It is fast but biased toward high-cardinality or frequently used features. Permutation importance is model-agnostic: it measures how much model performance decreases when a feature’s values are randomly shuffled. This directly captures predictive dependency but is computationally expensive and sensitive to correlated features.…