The Project | About Us | Contribute | Donations | License

HOME

At the end of the previous lesson we were left with a fully numeric matrix, but with wildly different scales: one-hot columns worth 0 or 1 living alongside spends of up to €15,400. For many algorithms, that disparity is a serious problem: a column with big numbers dominates the calculations and the others become invisible — not because they matter less, but by pure accident of units. Normalizing and standardizing means putting all variables on comparable scales. In this lesson you'll understand why scale matters (and for which algorithms), master scikit-learn's three fundamental scalers — MinMaxScaler, StandardScaler and RobustScaler —, see their effect before and after, and learn the golden rule that prevents leakage: fit the scaler on training data only.

Why scale matters
Which algorithms need it and which don't
Min-Max scaling: bringing everything to [0, 1]
Z-score standardization: StandardScaler
RobustScaler: scaling with outliers in the house
Visual comparison before and after
Which scaler to use: decision table
Fit on train, transform on test

Why scale matters

Take two MercaFresh customers described by two variables: total spend in euros and tenure in years.

Customer A: €1,200 in spend, 1 year of tenure.
Customer B: €1,150 in spend, 9 years of tenure.

For a distance-based algorithm (like the K-NN we'll see in 04-05), the "difference" between them is:

import numpy as np

A = np.array([1200, 1])
B = np.array([1150, 9])
print(np.sqrt(((A - B) ** 2).sum()))   # 50.64

The distance comes out at 50.64... of which 50 come from the spend and only 0.64 from the tenure. The 8-year difference — enormous in terms of customer loyalty — gets crushed by a €50 difference in spend, which is pocket change. Tenure could be removed from the dataset and the result would barely change: the unit of measurement, not the real importance, is deciding which variable rules.

The same problem appears, with different mechanics, in algorithms that learn by gradient (logistic regression 04-02, SVM 04-04, neural networks 04-07): when variables have very different scales, the error surface becomes a long, narrow valley, and the optimization process zigzags slowly instead of descending straight. Scaling rounds out the valley and speeds up — and stabilizes — training. The details of each algorithm belong to module 4; here the message is enough: distances and gradients demand comparable scales.

Which algorithms need it and which don't

Family	Scale-sensitive?	Why
K-NN (04-05), K-means (05-01)	Yes, critical	They compute distances between rows
SVM (04-04)	Yes, critical	Distances to the hyperplane + optimization
Linear/logistic regression with gradient (04-01, 04-02)	Yes	Gradient convergence; comparability of coefficients
Neural networks (04-07)	Yes	Stable gradients
PCA (05-03)	Yes, critical	Large variance dominates the components
Decision trees (04-03), Random Forest, Gradient Boosting (07-02/03)	No	They only make splits like "spend > 500?"; the split is the same on any scale
Naive Bayes (04-06)	Generally no	Works with per-variable probabilities

This table will accompany you through the whole course: before training any model, ask yourself which row it falls into. And a reminder from lesson 03-02: KNNImputer also measures distances, so it too appreciates scaled data.

Min-Max scaling: bringing everything to [0, 1]

Min-Max normalization transforms each variable so its minimum becomes 0 and its maximum 1:

x_scaled = (x - min) / (max - min)

import pandas as pd
from sklearn.preprocessing import MinMaxScaler

customers = pd.DataFrame({
    "total_spend": [1250.5, 890.0, 2100.75, 310.2, 15400.0, 670.4, 980.1, 45.9],
    "tenure_years": [3, 1, 6, 2, 9, 4, 1, 5],
    "num_orders": [18, 12, 25, 4, 210, 9, 14, 6],
})

mms = MinMaxScaler()
scaled = pd.DataFrame(mms.fit_transform(customers), columns=customers.columns)
print(scaled.round(3))
print(mms.data_min_, mms.data_max_)   # what was learned in fit

Direct interpretation: 0 is "the customer with the least", 1 "the one with the most", 0.5 "halfway there". Pros and cons:

In favor: a bounded, predictable range ([0, 1]), intuitive interpretation, fits wherever a fixed range is required (for example, some neural networks or images with 0-255 pixels).
Against: extremely sensitive to outliers. Look at the total_spend column: the €15,400 customer takes the 1, and everyone else gets compressed between 0.00 and 0.13. The outlier has hijacked the scale and the remaining customers are nearly indistinguishable from one another.
Also, a future value larger than the training maximum will come out > 1: the [0, 1] range is only guaranteed for the data used in fit.

Z-score standardization: `StandardScaler`

Standardization transforms each variable by subtracting its mean and dividing by its standard deviation:

z = (x - mean) / std

Sound familiar? It's exactly the z-score from lesson 02-02: each value comes to be expressed as "number of standard deviations from the mean". A customer with z = +2 in spend is two deviations above the average spend; one with z = 0 is exactly average. What we used there as an oddity detector here becomes a common language for all variables: after standardizing, they all have mean 0 and deviation 1, and "being 2 deviations above" means the same in euros as in years.

from sklearn.preprocessing import StandardScaler

ss = StandardScaler()
standardized = pd.DataFrame(ss.fit_transform(customers), columns=customers.columns)
print(standardized.round(2))
print(standardized.mean().round(2))   # ~0 in every column
print(standardized.std().round(2))    # ~1 in every column
print(ss.mean_, ss.scale_)            # mean and deviation learned in fit

Properties:

It doesn't bound the range: typical values fall between -3 and +3 (the 68-95-99.7 rule from 02-02), but an outlier can come out at z = 6 without issue.
It doesn't change the shape of the distribution: if spend was skewed, standardized spend is still skewed. Scaling is not transforming (that was 03-03); they're complementary steps: first log1p fixes the shape, then StandardScaler adjusts the scale.
It's the default scaler in practice: robust "enough" for most cases and the one many algorithms assume.
It still uses the mean and deviation, so outliers disturb it (less than Min-Max, but they do): with the €15,400 customer inside, the mean and deviation of spend are inflated.

`RobustScaler`: scaling with outliers in the house

When outliers are a legitimate part of the dataset — and at MercaFresh they are: restaurant customers exist and we don't want to remove them — it's better to scale with statistics that ignore them. RobustScaler uses the median and the IQR (the interquartile range from 02-01) instead of the mean and deviation:

x_scaled = (x - median) / IQR

from sklearn.preprocessing import RobustScaler

rs = RobustScaler()
robust = pd.DataFrame(rs.fit_transform(customers), columns=customers.columns)
print(robust.round(2))
print(rs.center_, rs.scale_)   # median and IQR learned in fit

Since the median and IQR barely move even with extreme values present (you saw it in 02-01: they're robust statistics), the bulk of the customers ends up nicely spread out and the outlier simply comes out with a large value, without compressing everyone else. It's the natural choice when the cleaning in 03-01 concluded "these outliers are real and they stay".

Visual comparison before and after

Nothing convinces like seeing it. Let's compare the three scalers on spend, outlier included:

import matplotlib.pyplot as plt

spend = customers[["total_spend"]]
versions = {
    "Original (EUR)": spend.values.ravel(),
    "MinMaxScaler": MinMaxScaler().fit_transform(spend).ravel(),
    "StandardScaler": StandardScaler().fit_transform(spend).ravel(),
    "RobustScaler": RobustScaler().fit_transform(spend).ravel(),
}

fig, axes = plt.subplots(1, 4, figsize=(14, 3))
for ax, (title, values) in zip(axes, versions.items()):
    ax.boxplot(values, vert=True)
    ax.set_title(title, fontsize=10)
plt.tight_layout()
plt.show()

What the four boxplots show (the ones from 02-01, back at work):

Original: a tiny box at the bottom, the outlier far away at the top. Scale in the thousands.
MinMaxScaler: exactly the same silhouette, but compressed into [0, 1]: the entire box occupies 13% of the range. The outlier rules.
StandardScaler: same silhouette, centered at 0; the outlier comes out at z ≈ 2.6 and the rest sits huddled in a narrow band below the mean.
RobustScaler: the central box is nicely spread around 0 (from roughly -0.5 to 0.5, because the IQR now equals 1) and the outlier goes far away without bothering anyone.

Visual conclusion: scaling never changes the shape of the distribution, only its measuring stick; what distinguishes the scalers is which stretch of the data they use as the reference for that stick.

Which scaler to use: decision table

Situation	Recommended scaler	Reason
General case, no serious outliers	`StandardScaler`	De facto standard; many algorithms assume it
Legitimate outliers present	`RobustScaler`	Median and IQR don't get dragged along
A bounded [0, 1] range is required	`MinMaxScaler`	The only one that guarantees it (on train)
Already-binary variable (one-hot)	None	It's already in {0, 1}; scaling only hurts interpretability
Tree-based model (04-03, 07-02/03)	None needed	Splits are scale-invariant
Heavily skewed distribution	Transform first (03-03), scale after	Scaling doesn't fix the shape

And the connection with the workflow we've been assembling: the scaler is one more step in the numeric branch's Pipeline inside the ColumnTransformer — imputation → transformation → scaling, in that order, while the one-hot columns pass through untouched.

from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer

numeric_branch = Pipeline(steps=[
    ("impute", SimpleImputer(strategy="median")),
    ("scale", RobustScaler()),
])

Fit on train, transform on test

The rule that closes the lesson is the same one that appeared in 03-02 with imputation, now applied to scaling — and scalers are where this sin is committed most. Every scaler learns parameters in fit: minimums and maximums, means and deviations, medians and IQRs. If those parameters are computed on the full dataset before splitting into training and test (the split we'll formalize in 06-01), the test data will have left its fingerprint on the mean or the maximum: data leakage. The evaluation will come out a little better than it deserves, and no error or warning will alert you.

The correct protocol:

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

# 1) Split FIRST (details in 06-01)
X_train, X_test = train_test_split(customers, test_size=0.25, random_state=42)

# 2) Fit the scaler on training data ONLY
scaler = StandardScaler()
scaler.fit(X_train)

# 3) Transform both sets with WHAT WAS LEARNED ON TRAIN
X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test)

# Equivalent shortcut for train: scaler.fit_transform(X_train)

A consequence that surprises everyone the first time: the scaled test set will not have a mean of exactly 0 or a deviation of exactly 1 — it's been measured with train's stick, not its own. And that's how it should be: in production, each new customer will arrive one at a time and be scaled with the frozen training parameters. fit_transform on train, plain transform on test and in production: burn it in as a reflex. And if everything lives inside a Pipeline, the reflex comes built in: the pipeline only calls fit when you ask it to, on train.

Common Mistakes and Tips

Running fit_transform on the test set. It's the most common leakage mistake among beginners: on test (and in production), transform only. If your scaled test set has a perfect mean of 0, be suspicious.
Scaling a dataset with outliers using MinMaxScaler. The rest of the values get compressed into a corner of [0, 1]. With legitimate outliers, RobustScaler.
Expecting scaling to fix skewness. Scaling is a linear operation: the histogram's shape doesn't change. For the shape, the transformations from 03-03.
Scaling the one-hot columns. A standardized 0/1 turns into values like -0.58/1.72: you lose readability without gaining anything relevant. Leave the binary columns out of the scaling branch in the ColumnTransformer.
Scaling the target variable of a classification problem. Churn (0/1) is the label, not a feature: it doesn't get scaled.
Forgetting to save the scaler. The learned parameters (mean_, scale_...) are part of the model: in production you'll need the same object to scale each new customer (we'll come back to this in module 8).
Tip: print describe() of the scaled result. Means ~0 and deviations ~1 (or [0, 1] ranges) on train confirm everything worked; absurd values expose columns that shouldn't have been scaled.

Exercises

Exercise 1

Without running any code: MercaFresh delivery times are [38, 42, 44, 47, 52, 55, 61, 190] minutes (the 190 was an order with a broken-down van). Which scaler would you choose and why? What would happen to the rest of the values with MinMaxScaler?

Exercise 2

Standardize by hand (no scikit-learn, just NumPy) the variable tenure = [3, 1, 6, 2, 9, 4, 1, 5] and check that the result has mean 0 and deviation 1. Then verify that StandardScaler gives the same result.

Exercise 3

This code has a data leakage bug. Find it and fix it:

scaler = StandardScaler()
data_scaled = scaler.fit_transform(data)
X_train, X_test = train_test_split(data_scaled, test_size=0.3)

Solutions

Exercise 1

RobustScaler. The 190 is an outlier with a known cause (breakdown) but present in the data; the mean and deviation (StandardScaler) would be inflated by it. With MinMaxScaler it would be worse: 190 would take the 1 and the seven normal values would be compressed roughly between 0.00 and 0.15, nearly indistinguishable from one another. RobustScaler, using the median (49.5) and the IQR, spreads the normal values out nicely and lets the 190 land far away without distorting the rest. (A legitimate alternative from 03-01: treat the 190 as a one-off error and correct it before scaling.)

Exercise 2

import numpy as np
from sklearn.preprocessing import StandardScaler

x = np.array([3, 1, 6, 2, 9, 4, 1, 5], dtype=float)

z = (x - x.mean()) / x.std()        # NumPy's std() divides by n, like StandardScaler
print(z.round(3))
print(z.mean().round(10), z.std().round(10))   # 0.0 and 1.0

ss = StandardScaler()
z_sk = ss.fit_transform(x.reshape(-1, 1)).ravel()
print(np.allclose(z, z_sk))          # True

A fine detail: StandardScaler uses the population standard deviation (dividing by n), just like np.std() by default; pandas' sample version (Series.std(), which divides by n-1) would give slightly different values.

Exercise 3

The fit_transform runs on all the data before splitting: the learned mean and deviation contain information from the rows that will later be test. The fix:

X_train, X_test = train_test_split(data, test_size=0.3, random_state=42)

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)   # learns ONLY from train
X_test_scaled = scaler.transform(X_test)         # applies what was learned

Split first, then fit on train only, and transform test with train's parameters.

Conclusion

You now master the last purely mechanical piece of preprocessing: why distance- and gradient-based algorithms need comparable scales, how MinMaxScaler bounds to [0, 1] (but surrenders to outliers), how StandardScaler converts each variable to the z-scores you met in 02-02, how RobustScaler scales with median and IQR when outliers are legitimate, and the non-negotiable rule of always fitting the scaler on training data only. The MercaFresh churn dataset is clean, complete, transformed, encoded and scaled.

So is it finished? Technically yes; competitively no. So far we have repaired and adapted the columns that already existed. The quality leap comes from creating new columns that condense business knowledge: how long since the customer last bought, how much they're worth, whether their activity is growing or fading. That is feature engineering, the lesson that closes the module — and where the MercaFresh dataset will reach its final form.

Normalization and standardization

Contents

Why scale matters

Which algorithms need it and which don't

Min-Max scaling: bringing everything to [0, 1]

Z-score standardization: `StandardScaler`

`RobustScaler`: scaling with outliers in the house

Visual comparison before and after

Which scaler to use: decision table

Fit on train, transform on test

Common Mistakes and Tips

Exercises

Exercise 1

Exercise 2

Exercise 3

Solutions

Conclusion

Machine Learning Course

Module 1: Introduction to Machine Learning

Module 2: Foundations of Statistics and Probability

Module 3: Data Preprocessing

Module 4: Supervised Machine Learning Algorithms

Module 5: Unsupervised Machine Learning Algorithms

Module 6: Model Evaluation and Validation

Module 7: Advanced Techniques and Optimization

Module 8: Model Implementation and Deployment

Module 9: Hands-On Projects

Module 10: Additional Resources

Normalization and standardization

Contents

Why scale matters

Which algorithms need it and which don't

Min-Max scaling: bringing everything to [0, 1]

Z-score standardization: StandardScaler

RobustScaler: scaling with outliers in the house

Visual comparison before and after

Which scaler to use: decision table

Fit on train, transform on test

Common Mistakes and Tips

Exercises

Exercise 1

Exercise 2

Exercise 3

Solutions

Conclusion

Machine Learning Course

Module 1: Introduction to Machine Learning

Module 2: Foundations of Statistics and Probability

Module 3: Data Preprocessing

Module 4: Supervised Machine Learning Algorithms

Module 5: Unsupervised Machine Learning Algorithms

Module 6: Model Evaluation and Validation

Module 7: Advanced Techniques and Optimization

Module 8: Model Implementation and Deployment

Module 9: Hands-On Projects

Module 10: Additional Resources

Z-score standardization: `StandardScaler`

`RobustScaler`: scaling with outliers in the house