juniorPCA

What are principal components in PCA, and how are they calculated?

Updated Feb 20, 2026

Short answer

Principal components are new axes created by PCA that capture the maximum possible variance in data while being mutually uncorrelated. They are calculated by finding the eigenvectors of the data’s covariance matrix and ranking them by their eigenvalues, which represent how much variance each component explains.

Deep explanation

Principal components are the directions in feature space where the data varies the most. Instead of looking at original features independently, PCA creates new combinations of features that summarize the important patterns in the data.

The key idea is:

A principal component is a direction, not a feature. It is a weighted combination of the original features that captures variance.

For example, if a dataset contains height and weight, the first principal component might represent a general "body size" direction because both variables often increase together.

How principal components are calculated

The standard PCA process follows these steps:

  1. Center the data

Subtract the mean of each feature so every feature has an average value of zero.

TypeScript
centered_value = original_value - feature_mean
   ```


2. **Compute the covariance matrix**


   The covariance matrix shows how features change together.


   * Positive covariance: features increase together.
   * Negative covariance: one increases while the other decreases.
   * Near-zero covariance: little relationship.


3. **Find eigenvectors and eigenvalues**


   The covariance matrix is decomposed into:


   * **Eigenvectors** → directions of the principal components.
   * **Eigenvalues** → amount of variance captured by each direction.


   The eigenvector with the largest eigenvalue becomes the first principal component.


4. **Sort components by explained variance**


   Components are ranked from most important to least important.


   * `PC1` explains the most variance.
   * `PC2` explains the next highest amount.
   * Later components explain less.


5. **Project the data**


   The original data points are transformed onto the selected principal component directions.


   ```python title="pca-example.py"
   from sklearn.decomposition import PCA


   pca = PCA(n_components=2)
   reduced_data = pca.fit_transform(data)
   ```


The transformation can be represented as:

new_data = original_data × principal_component_directions

TypeScript
### Visual intuition


Imagine a cloud of points on a graph. The first principal component points along the longest spread of that cloud. The second principal component points in the next largest spread direction while staying perpendicular to the first.

mermaid flowchart TD A["Original feature space"]:::muted --> B["Center data around the mean"]:::accent B --> C["Calculate covariance matrix"]:::accent C --> D["Find eigenvectors and eigenvalues"]:::good D --> E["Select highest variance directions"]:::good E --> F["Project data onto components"]:::accent

TypeScript
### Why principal components are useful


`PCA` is mainly used for dimensionality reduction. A dataset may have hundreds of features, but many features contain overlapping information. `PCA` compresses this information into fewer dimensions.


Benefits include:


| Benefit              | Why it helps                                           |
| -------------------- | ------------------------------------------------------ |
| Lower dimensions     | Makes models faster and easier to visualize            |
| Removes correlation  | Creates independent component directions               |
| Reduces noise        | Small-variance components may represent noise          |
| Better visualization | Allows high-dimensional data to be plotted in 2D or 3D |


However, `PCA` does not always improve a machine learning model. It can also remove useful information, especially when low-variance features contain important signals.


> ==The first principal component is not the "best feature"; it is the direction that explains the most variation in the dataset.==
### Variance explained and choosing components


A common question is: how many components should we keep?


`PCA` provides an explained variance ratio:

explained_variance_ratio = component_variance / total_variance

TypeScript
A common approach is keeping enough components to explain around `90%` to `95%` of the total variance.
### PCA calculation using singular value decomposition


Although eigenvalue decomposition explains the mathematics, many implementations use `SVD` because it is numerically stable.


The centered data matrix can be decomposed as:

X = UΣVᵀ ```

Where:

  • Vᵀ contains the principal component directions.
  • Σ contains scaling values related to explained variance.
  • U contains the transformed coordinates.

In practice, libraries such as scikit-learn handle these calculations internally.

Real-world example

A company has customer data with age, income, purchase_frequency, and many other behavioral features. Many of these variables are correlated, making customer analysis difficult.

Using PCA, the company reduces 50 customer features into 3 principal components:

  • PC1: overall purchasing power
  • PC2: shopping frequency patterns
  • PC3: customer engagement behavior

The marketing team can visualize customer groups more easily and build faster models.

customer-pca.py
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA


scaled = StandardScaler().fit_transform(customer_data)


pca = PCA(n_components=3)
customer_profiles = pca.fit_transform(scaled)

The important detail is that the new components are not directly named features. They are mathematical combinations of the original customer attributes.

Standardize features before PCA when variables have different scales, because large-scale features can dominate the components.

Common mistakes

  • * **Confusing features** - PCA components are combinations of features, not renamed original columns. Interpret them by looking at their feature weights.
  • * **Skipping scaling** - Features measured in larger units can dominate the covariance matrix. Standardize data when scales differ.
  • * **Keeping every component** - Reducing dimensions requires selecting useful components, not simply applying PCA and keeping everything.
  • * **Assuming maximum accuracy** - PCA may remove useful information. Compare model performance with and without PCA.
  • * **Ignoring interpretability** - Components can be harder to explain because each one mixes multiple original variables.

Follow-up questions

  • Why are principal components always uncorrelated with each other?
  • What is the difference between eigenvalues and eigenvectors in PCA?
  • Why should data usually be standardized before applying PCA?
  • Can PCA be used for feature selection?
  • What happens if we keep only the first principal component?

More PCA interview questions

View all →