How does PCA affect distance-based algorithms like KNN?

PCA can improve or degrade KNN performance by changing distance geometry and removing noise or useful features.

KNN relies heavily on distance metrics such as Euclidean distance. In high-dimensional spaces, distances become less meaningful due to the curse of dimensionality. PCA reduces dimensionality by projecting data onto directions of maximum variance, which can improve distance separation and remove noisy dimensions. However, if low-variance features contain discriminative information, PCA may reduce KNN accuracy.