What is Random Forest, and how does it work in machine learning?

Updated Feb 20, 2026

Short answer

A Random Forest is an ensemble machine learning algorithm that combines many decision trees to make a stronger prediction. It works by training multiple trees on different random samples of data and features, then combining their outputs through voting for classification or averaging for regression. The key idea is that many diverse weak learners can create a more accurate and stable model.

Deep explanation

Random Forest is a supervised learning algorithm based on the idea of ensemble learning: instead of trusting one model, it combines many models to improve performance.

A single decision tree is easy to understand but can suffer from overfitting. It may learn noise or very specific patterns from the training data. Random Forest reduces this problem by creating a collection of different decision trees and combining their predictions.

How Random Forest works

The algorithm has two main sources of randomness:

Random data sampling (bagging)

Each tree is trained on a randomly selected sample of the original training data.
The sampling is done with replacement, meaning the same data point can appear multiple times in a tree's training set.
This technique is called bootstrap aggregation, or bagging.

Random feature selection

When a tree decides which feature to split on, it does not consider every feature.
Instead, it randomly selects a subset of features and chooses the best split from that subset.
This makes trees less similar to each other.

The overall process looks like this:

Rendering diagram…

Prediction process

After training, Random Forest combines all tree predictions:

For classification:

Each tree predicts a class.
The class with the most votes becomes the final prediction.

For regression:

Each tree predicts a numeric value.
The model averages all tree predictions.

Example:

TEXT

1Tree 1 → Cat
2Tree 2 → Dog
3Tree 3 → Cat
4Tree 4 → Cat
5
6Final prediction → Cat

Random Forest does not make one perfect tree; it creates many different trees whose combined decision is usually more reliable.

Why Random Forest works well

Decision trees have high variance: small changes in training data can create very different trees. Random Forest reduces this variance by averaging many trees.

The trees are intentionally made different because:

Different training samples create different learning experiences.
Different feature subsets prevent every tree from focusing on the same patterns.
Averaging reduces the impact of individual mistakes.

A useful intuition is asking several independent experts instead of relying on one expert. If their errors are not identical, combining their opinions usually produces a better answer.

Important hyperparameters

Common Random Forest settings include:

Parameter	Purpose	Effect
`n_estimators`	Number of trees	More trees usually improve stability but increase computation
`max_depth`	Maximum tree depth	Controls complexity and overfitting
`max_features`	Features considered per split	Controls tree diversity
`min_samples_split`	Minimum samples needed for a split	Prevents overly specific branches

A typical implementation using scikit-learn might look like:

random_forest_example.py

Python

1from sklearn.ensemble import RandomForestClassifier
2
3model = RandomForestClassifier(
4    n_estimators=100,
5    max_depth=10,
6    random_state=42
7)
8
9model.fit(X_train, y_train)
10
11prediction = model.predict(X_test)

Trade-offs compared with a single decision tree

Model	Advantage	Disadvantage
Decision Tree	Easy to interpret and visualize	More likely to overfit
Random Forest	Better accuracy and generalization	Less interpretable and uses more memory
Linear Model	Fast and simple	May miss complex relationships

Random Forest is often a strong baseline model because it handles many real-world datasets with little preprocessing.

It can work well with:

Numerical features
Categorical features after encoding
Non-linear relationships
Data with noisy patterns

However, it is not always the best choice. For very large datasets, gradient boosting methods may achieve better accuracy. For applications requiring simple explanations, a single tree or linear model may be preferred.

Rule of thumb: use Random Forest when you want a strong, reliable model quickly and interpretability is less important than predictive performance.

Real-world example

Imagine a bank wants to predict whether a customer is likely to default on a loan. A single decision tree might focus too heavily on one pattern, such as income level, and make unreliable decisions.

A Random Forest creates many trees. One tree may focus on income, another on credit history, another on loan amount, and another on payment behavior. The forest combines all these opinions to produce a final risk prediction.

loan_default_model.py