How does k-nearest neighbors (KNN) illustrate bias-variance tradeoff?
Updated May 15, 2026
Short answer
Small k leads to low bias and high variance, while large k leads to high bias and low variance.
Deep explanation
KNN is a non-parametric model where predictions depend on nearby data points. When k is small, the model is highly sensitive to noise, leading to high variance. When k is large, predictions become smoother and less sensitive, increasing bias. Choosing k is a direct application of the bias-variance tradeoff.
Real-world example
Recommendation systems using KNN must balance personalization and stability.
Common mistakes
- Assuming higher k always improves accuracy.
Follow-up questions
- Why does KNN suffer in high dimensions?
- How do you choose k optimally?