You already know how to build Conv2D and MaxPooling2D layers, predict their output shapes, and count their parameters. The natural next question is: how do those pieces combine into networks that actually work? The good news is that you don't have to invent it from scratch: between 1998 and today, a series of celebrated architectures solved, one by one, the difficulties of training ever-deeper CNNs, and their design patterns have become the field's standard vocabulary. In this lesson we'll walk through that evolution — LeNet-5, AlexNet, VGG, GoogLeNet/Inception, ResNet, and the efficient mobile families — understanding the key idea behind each one, and we'll ground it by implementing a mini-VGG and a residual block in Keras. Knowing these architectures is not trivia: they are exactly the models TecnoMarket (and you) will reuse in practice.

1. LeNet-5: the pioneer (1998)
2. AlexNet: the awakening (2012)
3. VGG: the elegance of depth (2014)
4. GoogLeNet/Inception: parallel modules (2014)
5. ResNet: residual connections (2015)
6. Efficiency for mobile and edge: MobileNet and EfficientNet
7. Comparison table
8. Hands-on: a mini-VGG and a residual block in Keras

Long before the deep learning boom, Yann LeCun designed LeNet-5 to read handwritten digits on bank checks — the very MNIST problem you solved in 02-05, but fifteen years earlier and on 1990s hardware. Its structure will look familiar because it is exactly the anatomy we saw in 03-01:

• About 60,000 parameters in total: fewer than our dense network from 02-05 (~110,000), and yet more accurate on digits, because it respects the spatial structure.
• It established the canonical conv → pool → conv → pool → dense pattern we still use.
• Its limitation wasn't conceptual but of its era: without GPUs, without large datasets, and with sigmoid/tanh activations (with their vanishing gradient, as you saw in 02-03), it couldn't scale.

In 01-02 we told how AlexNet won ImageNet 2012, cutting the error so brutally that it reignited the whole field. Now you can understand what AlexNet was: in essence, a scaled-up LeNet plus three new ingredients.

• Structure: 5 convolutional layers + 3 dense ones, ~60 million parameters, 224×224×3 input (the very image from our parameter-explosion calculation in 03-01!).
• Ingredient 1: ReLU instead of sigmoid/tanh — much faster training and less vanishing (02-02).
• Ingredient 2: GPUs — it was trained on two consumer GPUs for about a week.
• Ingredient 3: massive data (1.2 million ImageNet images) and anti-overfitting tricks such as dropout and data augmentation (we'll systematize them in 05-04).
• Result: 15.3% top-5 error on ImageNet versus 26.2% for the runner-up. The modern era of deep learning starts here.

Its design lesson: scale (more data, more compute, more layers) + ReLU. But AlexNet was artisanal: 11×11, 5×5, and 3×3 filters mixed with no clear criterion. The next architecture brought order.

VGG (University of Oxford, 2014) is built on a single, radically simple design rule:

Always use 3×3 filters with same padding, stack them in blocks, and after each block do 2×2 max pooling while doubling the number of filters.

VGG-16 (16 weight layers) ends up as: blocks of 64, 128, 256, 512, and 512 filters, with the image shrinking 224 → 112 → 56 → 28 → 14 → 7.

Why 3×3 filters and not bigger ones? Two stacked 3×3 filters "see" a 5×5 area (the second one looks at a window of results that in turn looked at windows), and three stacked ones see 7×7. But let's compare parameters for C input and output channels:

The three 3×3 layers see the same thing with almost half the parameters and, on top of that, interleave three ReLUs instead of one: more non-linearity, more representational capacity. This observation crowned 3×3 as the standard (which is why we used it by default in 03-02).

The price: VGG-16 has 138 million parameters, of which ~102 million sit in the first dense layer after the Flatten (7×7×512 = 25,088 inputs × 4,096 neurons) — the pathology we already diagnosed when reading the summary() in 03-02, taken to the extreme.

That same year, Google attacked the question "what filter size do I use in each layer?" with a lateral answer: don't choose — use several at once. The Inception module applies, in parallel over the same input, 1×1, 3×3, and 5×5 convolutions plus a max pooling, and concatenates all the resulting maps along the channel axis: the network decides during training how much weight to give each branch.

Two ideas from this design became common heritage:

• The 1×1 convolution ("bottleneck"): a 1×1×C filter doesn't look at spatial neighbors, but it mixes and compresses channels (e.g. from 256 channels down to 64) before the expensive convolutions. It is a dense layer applied pixel by pixel over the channel vector, and it makes the module drastically cheaper.
• Global average pooling at the end: instead of Flatten + giant dense layers, each complete feature map is averaged down to a single number. GoogLeNet thus achieved 22 layers with only ~7 million parameters — 20 times fewer than VGG with better error.

With ReLU, GPUs, and good designs, you'd expect that stacking more layers would always help. But the experiments showed the opposite: beyond a certain depth (~20 layers), adding layers made even the training error worse. It wasn't overfitting (that would only worsen validation): it was an optimization problem — degradation. The gradient, after crossing dozens of layers multiplying derivatives (the chain rule and the "blame assignment" of 02-03), reached the early layers vanished, and the deep network couldn't even learn what the shallow one already knew.

ResNet (Microsoft Research, 2015) solved it with a minimal, brilliant change: the residual connection (skip connection). Instead of asking a block of layers to learn a complete transformation H(x), it is asked to learn only the correction F(x) on top of the identity, and the input is added directly to the output:

Why it works, in two readings:

• The learning reading: if a block contributes nothing useful, it just has to learn F(x) = 0 (weights at zero — easy) and the block becomes the identity: the 50-layer network can never be worse than the 20-layer one, because the surplus layers can "step aside". Learning a small correction is easier than learning an entire transformation.
• The gradient reading: when differentiating F(x) + x, the + x branch has derivative 1, so in backpropagation the gradient gets a direct highway to the early layers, without traversing (and without being shrunk by) all the intermediate multiplications. It is the architectural solution to the vanishing gradient we analyzed in 02-03.

With this, depth was unlocked: ResNet-152 (152 layers) won ImageNet 2015 with a top-5 error of 3.6%, below the human reference (~5%), and ResNet-50 remains one of computer vision's workhorses to this day. Residual connections have appeared in almost everything since, including transformers (05-05).

A brief mention, because you'll run into them constantly: not everything is about winning ImageNet. If TecnoMarket wanted its mobile app to classify the photo on the seller's phone (without uploading it to a server), it would need small, fast networks:

• MobileNet (2017): replaces the standard convolution with the depthwise separable convolution (first a spatial filter per channel, then a 1×1 conv that mixes channels), cutting compute ~8-9x with little accuracy loss. Ideal for mobile and edge devices.
• EfficientNet (2019): starts from an efficient base and defines a compound scaling rule (increasing depth, width, and input resolution together in balanced proportions), generating a B0...B7 family that spans from mobile to server with state-of-the-art accuracy at each compute level.

The trend the table tells: the error drops from 15% to under 2% in seven years, and not by inflating parameters (GoogLeNet and MobileNet reduce them), but through better architectural ideas.

We're not going to implement all of VGG-16 or ResNet-50 (training them from scratch here would make no sense, and we'll soon see why it isn't needed anyway). We're going to implement the patterns at a reduced scale, which is what truly transfers.

The VGG pattern in miniature for 64×64×3 TecnoMarket images, with the rule "two 3×3 same convs + pooling, doubling filters" and a modern ending with global average pooling instead of the gluttonous Flatten:

Details worth noticing:

• We're using Keras's functional API (keras.Input, layers applied as functions, keras.Model) instead of the Sequential from 02-05 and 03-02. For VGG it wouldn't matter, but it is essential for the residual block below, because Sequential only knows how to chain layers in a straight line, and a shortcut is not a straight line.
• Each vgg_block stacks two convs before the pooling: a 5×5 receptive field with 3×3+3×3 parameters and two ReLUs — VGG's argument.
• GlobalAveragePooling2D turns the 128 maps of 8×8 into a vector of 128 means: the dense output only needs 128×4+4 = 516 parameters. The whole model stays at ~300 K parameters, without the monstrous dense layer that in 03-02 concentrated 85% of the total.

The minimal ResNet pattern — two convs and the shortcut sum:

Reading the code line by line:

• shortcut = x: the skip connection is not a layer, it is simply keeping a reference to the input to use later.
• The second conv has no activation: first the shortcut is added and then the ReLU is applied. If we activated before summing, F(x) could only add positive values and the correction would lose half its range.
• layers.Add()([y, shortcut]) requires both tensors to have the same shape — which is why we use same padding and the same 64 filters the input carries. When a block changes the number of channels or reduces the size (stride 2), the shortcut is adapted with a 1×1 conv (Inception's bottleneck trick reused); real ResNets alternate both kinds of block.
• Stacking residual_block dozens of times is, literally, how a ResNet is built: each block adds its small correction and the gradient always has its highway back.

• Trying to train VGG-16 or ResNet-50 from scratch with little data. These networks were trained on 1.2 million images; with TecnoMarket's tens of thousands of photos (or fewer) they would overfit massively. The solution is to reuse them pretrained (see the conclusion).
• Confusing degradation with overfitting. Overfitting worsens validation while improving training; the degradation that motivated ResNet worsened training too. They are different ailments with different remedies (05-04 for the first, skip connections for the second).
• Forgetting that Add sums — it doesn't concatenate. layers.Add() sums element-wise (identical shapes, the ResNet pattern); layers.Concatenate() stacks channels (the Inception pattern). Choosing the wrong one changes the design completely.
• Putting the ReLU before the residual sum. The canonical order is conv → conv → add shortcut → ReLU. Activate earlier and you'll be restricting the correction F(x) to non-negative values.
• Memorizing architectures instead of ideas. In three years there will be another fashionable architecture; what endures is the repertoire: stacked 3×3s, 1×1 convs to compress channels, global average pooling, residual connections. Learn the vocabulary, not the encyclopedia.

Without looking at the table, match each idea to its architecture and order them chronologically: (a) modules with several convolutions in parallel, (b) shortcuts that add the input to the output, (c) only 3×3 filters stacked in blocks, (d) first big success with ReLU + GPU on ImageNet, (e) original conv→pool→dense pattern for digits.

A layer receives and produces 128 channels. Compare the parameters (ignore biases) of (a) a single 7×7 conv versus (b) three stacked 3×3 convs. What additional advantage, beyond the parameter reduction, do the three layers provide?

This block throws an error when built. Explain why and propose two different fixes:

Exercise 1: (e) LeNet-5, 1998 → (d) AlexNet, 2012 → (c) VGG, 2014 → (a) GoogLeNet/Inception, 2014 → (b) ResNet, 2015.

Exercise 2:

• (a) One 7×7 conv: 7 × 7 × 128 × 128 = 802,816 parameters.
• (b) Three 3×3 convs: 3 × (3 × 3 × 128 × 128) = 442,368 parameters — 45% fewer with the same 7×7 receptive field.
• Additional advantage: the three layers interleave three ReLUs instead of one, so the transformation is more non-linear and expressive (and the reasoning from 02-02 explains why that matters).

Exercise 3: The input x has 64 channels but F(x) produces 128; layers.Add() requires identical shapes and fails with (8, 8, 64) versus (8, 8, 128). Fixes: (1) use 64 filters in the block's two convs, so F(x) preserves the input's shape; (2) adapt the shortcut with a 1×1 conv of 128 filters — shortcut = layers.Conv2D(128, (1, 1), padding="same")(x) — which projects the 64 channels to 128 before the sum (this is what real ResNets do when changing stages).

You've covered twenty years of evolution in one lesson: LeNet fixed the conv→pool→dense pattern; AlexNet proved that scale + ReLU + GPU changed the rules; VGG distilled design down to blocks of 3×3 filters; Inception introduced parallel branches, 1×1 convs, and global average pooling; ResNet unlocked arbitrary depth with residual connections that give the gradient a highway against the vanishing of 02-03; and MobileNet/EfficientNet optimize the cost to reach the seller's phone. You've also implemented the two most reused patterns — the VGG block and the residual block — with Keras's functional API.

One observation remains, with enormous practical consequences: all these architectures are available already trained on ImageNet (in Keras, one line away: keras.applications.ResNet50(...)). Their early layers learned edge, texture, and shape detectors that work for any image — including TecnoMarket's product photos. Reusing that knowledge instead of training from scratch is called transfer learning, and it's the technique we'll study in 05-03 (and apply with fine-tuning in 07-05). Before we get there, in the next lesson we'll lift our gaze from pure classification: we'll see everything CNNs can do with images — localize, detect, segment, read, and search by similarity — and how each task fits into TecnoMarket's operations.