seniorK-Nearest Neighbors
How can KNN be derived from a first-principles intuition of similarity?
Updated May 16, 2026
Short answer
KNN is derived from the idea that similar points in feature space are likely to share the same label.
Deep explanation
At its core, KNN assumes a smoothness prior: points close in feature space should have similar outputs. This is equivalent to local constant function approximation, where the label of a point is approximated by the empirical distribution of its neighborhood.
Real-world example
Predicting housing prices based on nearby houses with similar features.
Common mistakes
- Thinking KNN is purely geometric rather than assumption-driven.
Follow-up questions
- What assumption does KNN rely on?
- Why does similarity imply prediction?