What is feature importance in Decision Trees?
Updated May 16, 2026
Short answer
Feature importance measures how much each feature contributes to reducing impurity in a tree.
Deep explanation
Decision Trees compute feature importance by summing the impurity reduction (weighted by number of samples) contributed by each feature across all splits. Features that frequently split high in the tree or significantly reduce impurity are considered more important. However, this method can be biased toward high-cardinality features.
Real-world example
Used in finance to identify which factors (income, credit history) most influence loan approval.
Common mistakes
- Assuming feature importance implies causation.
Follow-up questions
- Is feature importance reliable?
- How can we improve interpretability?
- Do correlated features affect importance?