How do Decision Trees decide the best split among many candidate features?

Decision Trees evaluate all candidate features and thresholds, selecting the split that maximizes impurity reduction (information gain or Gini decrease).

At each node, the algorithm performs an exhaustive search over all features. For continuous features, it sorts values and evaluates midpoints between unique values as potential thresholds. For each candidate split, it computes impurity reduction (Gini, entropy, or variance reduction). The best split is the one that yields the highest weighted improvement in node purity. This greedy search is repeated recursively at every node, which is computationally expensive but effective in practice.