The SVM closed the previous lesson with an idea in embryo: classifying by similarity to other points. K-Nearest Neighbors (K-NN) builds an entire algorithm on that idea and nothing else: to classify a new MercaFresh customer, it finds the K most similar historical customers and takes a vote — if most of them churned, it predicts churn. No equation, no coefficients, no real training. That radical simplicity makes it the best model for understanding what "similar" means in a feature space — and a perfect laboratory for two themes that run through all of ML: the choice of distance (and why the scaling from 03-05 is vital yet again) and the curse of dimensionality.

1. Lazy learning: a model that doesn't train
2. Distance metrics: Euclidean and Manhattan
3. Scaling is critical once again
4. Choosing K: small, large and odd
5. K-NN for classification and for regression
6. Implementation with scikit-learn: MercaFresh churn
7. Prediction cost and the curse of dimensionality
8. K-NN is supervised (and K-means is not)

Every model so far compresses the dataset into a few parameters during fit: linear regression into weights, the tree into rules, the SVM into support vectors. K-NN compresses nothing: its fit merely stores the training dataset (at most, indexing it for fast lookup). That's why it's called lazy learning: all the work is postponed until prediction time.

The prediction procedure, in full:

1. A new customer $\mathbf{x}$ arrives.
2. Compute the distance from $\mathbf{x}$ to every training customer.
3. Select the K closest ones (the "neighbors").
4. Classification: vote the majority class among the neighbors (or average their proportions to give a probability). Regression: average their values.

The implicit hypothesis is pure business common sense: customers with similar features behave similarly. If the five historical customers most similar to Marta churned, Marta is at risk. It's the same logic KNNImputer used in 03-02 to fill nulls with the neighbors' values — that imputer was, literally, a regression K-NN applied to the incomplete column.

"Closest" requires defining distance. The two main characters:

Practical difference: Euclidean squares the differences, so a single highly discrepant feature dominates the distance (like the MSE with outliers in 04-01); Manhattan spreads the weight linearly and is somewhat more robust to extreme differences in one coordinate. On tabular datasets like MercaFresh's, Euclidean is the standard starting point; trying Manhattan is a cheap experiment when some features have extreme values.

The third star appearance of 03-05, and the most dramatic. Compute the Euclidean distance between two unscaled customers:

• Customer A: recency 30 days, inactivity_ratio 0.10
• Customer B: recency 90 days, inactivity_ratio 0.95

$d = \sqrt{(90-30)^2 + (0.95-0.10)^2} = \sqrt{3600 + 0.72} \approx 60.006$

The inactivity_ratio — which radically distinguishes the two customers — contributes 0.01% of the distance. For unscaled K-NN, that feature doesn't exist: neighbors are chosen by recency alone. After a StandardScaler or RobustScaler, both features speak in comparable units and both get a say. Absolute rule: never K-NN without scaling — like the SVM (04-04), and unlike the tree (04-03).

K is the only essential hyperparameter, and its effect is a tug-of-war between flexibility and stability:

Picture it: with K=1, every atypical train customer — the loyal one with a churner's profile — creates a small island of wrong predictions around itself; a single mislabeled point contaminates its neighborhood. With K=25, that anecdote gets drowned out by the vote and the boundary smooths; but if K keeps growing, the vote includes ever less similar customers and the boundary loses the real detail. Choosing K systematically is done with cross-validation (06-03); the usual initial heuristic is $K \approx \sqrt{n}$, then explore around it.

Two concrete tips:

• Odd K in binary classification: avoids 2-2 ties in the vote (sklearn resolves them, but better not to have them).
• weights="distance": gives closer neighbors more voting power — useful when K is large and you don't want the neighbors at the edge of the neighborhood to weigh as much as the ones right next to the point.

Classification (churn) is the flagship case and the one in our example. The regression version is identical, swapping the vote for an average: to estimate a customer's monthly spend (the 04-01 problem), KNeighborsRegressor averages the spend of their K neighbors. It produces locally adaptive predictions without assuming linearity — but, like the regression tree (04-03), it doesn't extrapolate: the predicted spend will never leave the range of the neighbors' spend.

The same professional pattern as in 04-02 and 04-04: the 03-06 preprocessor (with its scaling) and the model, chained:

What you'll see when running the loop:

• K=1: train accuracy = 100% always (the nearest neighbor of a train point is itself). Test, clearly worse: the signature of overfitting.
• Intermediate K: the best test score — the neighborhood averages out the noise without diluting the pattern.
• K=201: both accuracies fall toward the majority-class proportion — the model barely looks at the customer anymore.

On top of that, predict_proba comes for free and reads directly: "8 of your 11 neighbors churned" is an argument the retention team understands — K-NN's interpretability lies not in coefficients or rules, but in being able to show the neighbors behind each prediction (kneighbors() returns them).

Inverted cost. K-NN flips the cost profile of every previous model:

For a production system that scores every customer on every visit (08-02), paying for the neighbor search on every prediction — and carrying the entire history in memory — can be prohibitive. Spatial indexes (algorithm="kd_tree"/"ball_tree") speed up the search with few dimensions, but lose effectiveness as dimensions grow... which connects to the deeper problem.

The curse of dimensionality. In spaces with many features, geometry betrays intuition: volume grows exponentially with dimension, points spread out, and the distances between all pairs become nearly equal — the "nearest" neighbor is barely closer than the farthest. When that happens, "similar" stops meaning anything and K-NN (and every distance-based method, the RBF SVM included) degenerates.

Rules of thumb: with the ~15 features of the churn dataset, K-NN breathes easily; with hundreds of features (vectorized text, genomics), it suffers. The cures: the feature selection of 03-06 (fewer dimensions, more signal) and dimensionality reduction with PCA, which we'll see in 05-03 precisely as the usual antidote before applying distance methods.

A mandatory clarification before module 5, because the shared letter K confuses everyone:

K-NN answers "will this customer churn?" using the labeled history; K-means will answer "what customer segments exist?" without anyone telling it what to look for. They share the notion of distance (and the obligation to scale), nothing more. We'll leave it here: K-means gets its own full lesson opening module 5.

• Forgetting the scaling. In K-NN it's not that the model performs worse: small-range features de facto vanish from the calculation. Always inside the Pipeline with the 03-06 preprocessor.
• Evaluating K=1 on the train set itself and celebrating the 100%. It's a mirage by construction (each point is its own neighbor). Every comparison of K must be done on held-out data — or better, with cross-validation (06-03).
• Using K-NN with dominant one-hot features. Many binary city_* columns can weigh as much as all the numeric features combined in the Euclidean distance. Watch the proportion of binary features, or weight/select (03-06).
• Deploying it without measuring prediction latency. It works beautifully in the notebook with 800 customers and crawls with 2 million. Before proposing it for production, time predict with the real volume.
• Tip: use K-NN as a probe of the problem. If with good scaling it doesn't clearly beat predicting the majority class, your features don't define a useful notion of "similar" — and that is a diagnosis about the data (go back to 03-06) that no sophisticated model will fix on its own.

Exercise 1. By hand: a new customer at (scaled_recency = 0.0, scaled_trend = 0.0). Candidate neighbors from the train set: A(0.1, 0.2, loyal), B(−0.3, 0.1, loyal), C(0.8, −0.9, churn), D(0.2, −0.1, churn), E(−1.5, 1.2, loyal). Compute the Euclidean distances, classify with K=3 and with K=5, and give the churn probability in each case.

Exercise 2. Repeat the K=3 classification of exercise 1 with Manhattan distance. Does any neighbor in the top 3 change? Does the prediction change?

Exercise 3. Explain why KNNImputer (03-02) needed the features to be on comparable scales, using what you learned in this lesson. Which feature of the MercaFresh dataset would have dominated the imputation had it not been scaled?

Exercise 1

Distances to the origin: A: $\sqrt{0.01+0.04}=0.224$; B: $\sqrt{0.09+0.01}=0.316$; C: $\sqrt{0.64+0.81}=1.204$; D: $\sqrt{0.04+0.01}=0.224$; E: $\sqrt{2.25+1.44}=1.921$.

• K=3: neighbors A (loyal), D (churn), B (loyal) → 2-1 vote → loyal, p(churn) = 1/3 ≈ 0.33.
• K=5: C (churn) and E (loyal) also come in → 3 loyal, 2 churn → loyal, p(churn) = 2/5 = 0.40.

Notice how the probability changes with K even though the label doesn't: the granularity of p is 1/K, another reason not to use a tiny K if you need fine-grained probabilities.

Exercise 2

Manhattan: A: 0.1+0.2=0.3; B: 0.3+0.1=0.4; C: 0.8+0.9=1.7; D: 0.2+0.1=0.3; E: 1.5+1.2=2.7. The top 3 is still {A, D, B} and the prediction is still loyal (2-1). Nothing changes in this case — the two metrics usually agree when the neighbors are clear-cut; they diverge mostly when some candidate owes its Euclidean closeness to offsetting one very discrepant coordinate with several very similar ones (the square is less forgiving than the absolute value).

Exercise 3

KNNImputer fills a customer's null with the average of that column across its K nearest neighbors — and "nearest" is decided by Euclidean distance over the remaining features. It's exactly this lesson's regression K-NN. Without scaling, the feature with the largest numeric range in the MercaFresh dataset — total_spend (hundreds or thousands of euros) or, failing that, recency_days (up to ~180) — would have monopolized the distance: the "neighbors" would simply be the customers with similar spend, ignoring trend, ratios and satisfaction, and the imputations would inherit that bias. That's why in 03-02 we imputed inside a flow that scales — and why the order of the preprocessor's branches matters.

K-NN has shown you ML at its most minimal: memorize the past and predict by resemblance. Along the way you've consolidated three cross-cutting ideas: distance is a design decision (Euclidean vs. Manhattan), without scaling there's no distance worth having (03-05 again), and K is the umpteenth dial in the trade-off between memorizing and generalizing (06-05). You've also seen its bills — expensive prediction, voracious memory, and the curse of dimensionality that will motivate the PCA of 05-03 — and the boundary with K-means is now clear: same surname, different families.

So far, all our classifiers decide by measuring — distances, margins, impurities. The next lesson takes up a completely different path we left open in 02-05: deciding by computing probabilities with Bayes' theorem, as our fraud detector did. Turning that theorem into a full classifier — fast, frugal and surprisingly effective — only requires one brazenly false assumption that works: the naivety of Naive Bayes.