How does PCA behave when applied to categorical encoded features?

PCA can misinterpret encoded categorical variables as numeric structure, leading to misleading components.

When categorical variables are label-encoded or ordinal-encoded, PCA treats numeric values as having meaningful distances. This creates artificial variance patterns that do not represent real relationships. One-hot encoding is safer, but it increases sparsity and dimensionality. Even then, PCA may still produce components that mix unrelated categories.