The Project | About Us | Contribute | Donations | License

HOME

In project 1 you worked with tabular data, scikit-learn's natural habitat. Now we change worlds: images. This project has a double goal. First, to find out how far the classical methods from module 4 can go with small images (the handwritten digits in load_digits); then, to make the jump to Fashion-MNIST — 70,000 photos of clothing items in 10 categories — and solve it with Keras, comparing a dense network (MLP) against a convolutional neural network (CNN), putting into practice everything you saw in 07-04 "Deep neural networks (Deep Learning)". By the end you will have your own criteria for answering the important question: when is classical enough, and when do you need deep learning?

Problem definition and target metric
Stage A: digits with classical methods
The limits of the classical approach
Stage B: Fashion-MNIST with Keras
Dense MLP: the neural baseline
Small CNN: convolutions in action
Overfitting in practice: dropout and early stopping
Final evaluation: confusion matrix over the 10 classes
Conclusions: classical or deep?

Problem definition and target metric

Problem: multiclass image classification — 10 digits first, 10 garments later.
Primary metric: accuracy. Unlike the fraud project waiting for you in 09-04, here the classes are perfectly balanced (10% each), so accuracy does not lie — it is the benign case we described in 06-02 "Evaluation metrics". We will pair it with the confusion matrix to see what gets confused with what.
Success criterion: on digits, beat 95% (it is an easy dataset); on Fashion-MNIST, beat 90%, which requires a CNN.

A grayscale image is a matrix of pixels (0 = black, 255 = white). To a classical model, each pixel is simply one more column: an 8×8 image is 64 features. That "translation" — flattening the image into a vector — is the key to stage A, and also its Achilles' heel.

Stage A: digits with classical methods

load_digits provides 1,797 handwritten digits of 8×8 pixels, in the same format as any tabular sklearn dataset:

import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split, cross_val_score

digits = load_digits()
X, y = digits.data, digits.target   # X: (1797, 64), values 0-16
print(X.shape, np.unique(y))

# Let's look at a few examples
fig, axes = plt.subplots(2, 8, figsize=(10, 3))
for ax, img, label in zip(axes.ravel(), digits.images, y):
    ax.imshow(img, cmap="gray_r")
    ax.set_title(label); ax.axis("off")
plt.tight_layout(); plt.show()

We split (stratifying, as always since 06-01 "Data splitting: training, validation and test") and try two classics with scaling in a Pipeline — the logistic regression from 04-02 "Logistic regression" and the SVM from 04-04 "Support Vector Machines (SVM)":

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, stratify=y, random_state=42
)

classics = {
    "Logistic": Pipeline([("scale", StandardScaler()),
                          ("m", LogisticRegression(max_iter=2000))]),
    "SVM (RBF)": Pipeline([("scale", StandardScaler()),
                           ("m", SVC(kernel="rbf", gamma="scale"))]),
}

for name, model in classics.items():
    scores = cross_val_score(model, X_train, y_train, cv=5)
    print(f"{name}: {scores.mean():.3f} ± {scores.std():.3f}")

Typical results: logistic ≈ 0.96 and SVM ≈ 0.98 cross-validated accuracy. With tiny, well-centered images, classical methods work wonderfully.

To understand why, visualize the dataset with the PCA from 05-03 "Principal Component Analysis (PCA)":

from sklearn.decomposition import PCA

pca = PCA(n_components=2)
X_2d = pca.fit_transform(StandardScaler().fit_transform(X_train))

plt.figure(figsize=(8, 6))
sc = plt.scatter(X_2d[:, 0], X_2d[:, 1], c=y_train, cmap="tab10", s=10, alpha=0.6)
plt.colorbar(sc, label="Digit")
plt.title("Digits projected to 2D with PCA")
plt.show()
print("Explained variance:", pca.explained_variance_ratio_.sum().round(3))

Even with only 2 components (≈ 22% of the variance), you can make out one cloud per digit: the classes are fairly separable even after flattening the image. That is the secret of the classical success here.

The limits of the classical approach

Why not carry on like this with any image? Because flattening destroys the spatial structure:

A pixel no longer "knows" who its neighbors were: column 13 and column 14 are adjacent in the image, but to the logistic regression they are two features as independent as MedInc and Latitude in project 1.
The model is not translation-invariant: the same digit shifted two pixels activates completely different columns and looks like "another" pattern.
Dimensionality explodes: 8×8 is 64 features; a modest 224×224 color photo is 150,528. With features that raw, the methods from module 4 drown.

With clean, centered 8×8 digits, these problems are barely noticeable. With 28×28 clothing photos they will start to weigh; with real photos, they would be fatal. This is exactly the motivation for the convolutions you saw in 07-04: processing the image while respecting its geometry.

Stage B: Fashion-MNIST with Keras

Fashion-MNIST is the modern replacement for the original MNIST: 60,000 training images and 10,000 test images, 28×28 in grayscale, with 10 garment types. It ships with Keras:

import tensorflow as tf
from tensorflow import keras

(X_train, y_train), (X_test, y_test) = keras.datasets.fashion_mnist.load_data()

classes = ["T-shirt/top", "Trouser", "Pullover", "Dress", "Coat",
           "Sandal", "Shirt", "Sneaker", "Bag", "Ankle boot"]
print(X_train.shape, X_test.shape)  # (60000, 28, 28) (10000, 28, 28)

Pixel normalization: values range from 0 to 255; networks train far better on small, homogeneous inputs (the same principle as 03-05 "Normalization and standardization", applied to images). We divide by 255 to bring them into [0, 1], and set aside a validation set to keep an eye on training:

X_train = X_train / 255.0
X_test = X_test / 255.0

# Validation: last 10,000 training images
X_val, y_val = X_train[50000:], y_train[50000:]
X_tr,  y_tr  = X_train[:50000], y_train[:50000]

Dense MLP: the neural baseline

We start with the architecture you already know from 04-07 "Neural networks" and 07-04: dense layers on top of the flattened image. It is our "no spatial structure" baseline:

mlp = keras.Sequential([
    keras.layers.Flatten(input_shape=(28, 28)),   # 784 features
    keras.layers.Dense(256, activation="relu"),
    keras.layers.Dense(128, activation="relu"),
    keras.layers.Dense(10, activation="softmax"), # 10 probabilities
])

mlp.compile(optimizer="adam",
            loss="sparse_categorical_crossentropy",
            metrics=["accuracy"])

hist_mlp = mlp.fit(X_tr, y_tr, epochs=20, batch_size=128,
                   validation_data=(X_val, y_val), verbose=2)

A reminder from 07-04: softmax turns the 10 outputs into probabilities that sum to 1, and sparse_categorical_crossentropy is the right loss when the labels are integers (0–9) rather than one-hot. Typical result: ≈ 0.89 validation accuracy, with the training curve pulling away from the validation curve in the final epochs — overfitting creeping in, as in 06-05 "Overfitting and underfitting".

Small CNN: convolutions in action

Now the alternative that respects the geometry. Picking up from 07-04: a Conv2D layer slides small filters (3×3) across the whole image, learning local detectors (edges, textures) that apply equally at any position — that gives translation invariance and far fewer parameters. MaxPooling2D halves the resolution by keeping the strongest activation in each 2×2 window, gaining robustness to small shifts:

cnn = keras.Sequential([
    keras.layers.Reshape((28, 28, 1), input_shape=(28, 28)),
    keras.layers.Conv2D(32, (3, 3), activation="relu"),  # 32 3x3 detectors
    keras.layers.MaxPooling2D((2, 2)),
    keras.layers.Conv2D(64, (3, 3), activation="relu"),  # combines patterns
    keras.layers.MaxPooling2D((2, 2)),
    keras.layers.Flatten(),
    keras.layers.Dense(128, activation="relu"),
    keras.layers.Dropout(0.3),
    keras.layers.Dense(10, activation="softmax"),
])
cnn.summary()

Read the summary(): the convolutional layers have just a few thousand parameters (the filters are shared across the whole image), whereas in the MLP the first dense layer alone had 784 × 256 ≈ 200,000. Fewer parameters doing more useful work: that is the convolutional idea.

Overfitting in practice: dropout and early stopping

Two defenses from 07-04, applied for real:

Dropout(0.3): at each training step it randomly switches off 30% of that layer's neurons, preventing the network from memorizing fragile co-adaptations.
Early stopping: it watches the validation loss and stops training when it stops improving, restoring the best weights — the practical equivalent of picking the optimal point on the curve from 06-05.

cnn.compile(optimizer="adam",
            loss="sparse_categorical_crossentropy",
            metrics=["accuracy"])

stopper = keras.callbacks.EarlyStopping(
    monitor="val_loss", patience=3, restore_best_weights=True
)

hist_cnn = cnn.fit(X_tr, y_tr, epochs=30, batch_size=128,
                   validation_data=(X_val, y_val),
                   callbacks=[stopper], verbose=2)

# Learning curves
plt.plot(hist_cnn.history["accuracy"], label="train")
plt.plot(hist_cnn.history["val_accuracy"], label="validation")
plt.xlabel("Epoch"); plt.ylabel("Accuracy"); plt.legend()
plt.title("CNN: learning curves"); plt.show()

Typical result: ≈ 0.91–0.92 on validation, with the curves much closer together than in the MLP. Early stopping usually halts training between epochs 10 and 15.

Final evaluation: confusion matrix over the 10 classes

As always, the test set is opened exactly once, with the model already decided:

from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay, accuracy_score

y_pred = cnn.predict(X_test).argmax(axis=1)
print("CNN test accuracy:", round(accuracy_score(y_test, y_pred), 4))

cm = confusion_matrix(y_test, y_pred)
disp = ConfusionMatrixDisplay(cm, display_labels=classes)
fig, ax = plt.subplots(figsize=(9, 9))
disp.plot(ax=ax, cmap="Blues", xticks_rotation=45, colorbar=False)
plt.title("Fashion-MNIST: confusion matrix (test)")
plt.tight_layout(); plt.show()

Model	Accuracy (validation)	Accuracy (test)	Approx. parameters
Dense MLP	≈ 0.89	≈ 0.88	≈ 235,000
Small CNN	≈ 0.92	≈ 0.91	≈ 225,000

The confusion matrix tells the interesting story: the errors are not spread at random. The dominant confusions are Shirt ↔ T-shirt/top ↔ Pullover ↔ Coat (all torso garments with similar silhouettes at 28×28 pixels) and, to a lesser extent, Sneaker ↔ Ankle boot ↔ Sandal. By contrast, Trouser and Bag are hardly ever confused with anything: their silhouettes are unmistakable. The network fails exactly where a human, squinting at a blurry 28×28 thumbnail, would also hesitate — a reasonable error is the sign of a model that has learned real patterns rather than noise.

Common Mistakes and Tips

Forgetting to normalize the pixels. With 0–255 values the network trains poorly or very slowly (out-of-proportion gradients). Dividing by 255 is one line that changes everything.
Normalizing train and test with different criteria. Here it is a constant (255), but if you used mean and standard deviation, they would be computed on the training set only — the same discipline as 03-05 and 06-01.
Evaluating on the test set every epoch. The test set is not the validation set: if you use it to decide when to stop or which architecture to pick, it stops measuring generalization. That is what validation_data is for.
Comparing networks on a single run. Random initialization shifts the result by a few tenths of a point; to claim the CNN beats the MLP you should repeat with several seeds (with tabular data you would use the CV from 06-03; with large networks you repeat training runs, because full CV is expensive).
Huge networks for small problems. The temptation to stack layers arrives before the need does. Start small, measure, and grow only if the validation asks for it.

Challenges to go further

Data augmentation. Generate variants of the training images (shifts, slight rotations, horizontal mirroring) to teach the network invariances the data does not show. Hint: look into the keras.layers.RandomTranslation and RandomFlip layers placed at the start of the model; with clothing, be careful with vertical mirroring (is an upside-down ankle boot still an ankle boot?).
Transfer learning. In real problems you almost never train a CNN from scratch: you start from a network pretrained on millions of photos (ResNet, MobileNet, available in keras.applications) and retrain only the last layer. Hint: it requires resizing the images to the chosen network's input size and converting them to 3 channels; with Fashion-MNIST the gain is modest, but the pattern is the one you will use at work.
Your own photos. Photograph a sneaker or a T-shirt against a light background, convert it to 28×28 grayscale with PIL, invert the contrast if needed (Fashion-MNIST has a black background) and feed it to the CNN. Hint: it will probably fail more than you expect — reflect on the drift between your photos and the training ones, the same phenomenon as in 08-03 "Model maintenance and monitoring".

Conclusion

This project has given you the practical answer to "classical or deep?": with tiny, centered images like the digits, an SVM with scaling flirts with 98% and you need nothing more; with images just slightly richer like Fashion-MNIST, flattening pixels starts costing points and convolutions — shared local filters, pooling, translation invariance — win them back with fewer parameters. You have also practiced the craft of training networks: normalizing pixels, watching the train/validation curves, curbing overfitting with dropout and early stopping, and reading the confusion matrix as a map of reasonable errors. The general rule you take with you: start with the simplest method that could work, and let the data justify every additional layer. In the next project you will apply that same rule to another kind of non-tabular data — text — where classical methods, given the right representation, still put up quite a fight.

Project 2: Image classification

Contents

Problem definition and target metric

Stage A: digits with classical methods

The limits of the classical approach

Stage B: Fashion-MNIST with Keras

Dense MLP: the neural baseline

Small CNN: convolutions in action

Overfitting in practice: dropout and early stopping

Final evaluation: confusion matrix over the 10 classes

Common Mistakes and Tips

Challenges to go further

Conclusion

Machine Learning Course

Module 1: Introduction to Machine Learning

Module 2: Foundations of Statistics and Probability

Module 3: Data Preprocessing

Module 4: Supervised Machine Learning Algorithms

Module 5: Unsupervised Machine Learning Algorithms

Module 6: Model Evaluation and Validation

Module 7: Advanced Techniques and Optimization

Module 8: Model Implementation and Deployment

Module 9: Hands-On Projects

Module 10: Additional Resources