In 04-07 we built an MLP — a multilayer perceptron with one or two hidden layers — and left pending the acronyms that dominate the headlines: CNN, RNN, transformers. This lesson settles that debt. Deep learning is, in essence, the same neural network from 04-07 taken into depth: many stacked layers that learn hierarchical representations of the data. You'll understand exactly what depth contributes, why it only became practical about fifteen years ago, which concrete techniques make it trainable, which architecture fits each kind of data, and you'll train a deep network with Keras on MercaFresh's churn — to discover, honestly, when deep learning pays off and when the gradient boosting from the previous lesson remains the better choice.

1. From the MLP to the deep network: what depth contributes
2. Why it's possible now: GPUs, data and techniques
3. The techniques that make it trainable
4. Architectures and their domains: CNN, RNN/LSTM, transformers
5. Example with Keras: MercaFresh churn
6. Deep learning vs. gradient boosting on tabular data
7. What comes next

Recall the MLP of 04-07: layers of neurons where each one computes a linear combination of its inputs followed by a non-linear activation, trained with backpropagation. In theory, a single sufficiently wide hidden layer can approximate any function (the universal approximation theorem). So why stack more layers?

Because depth doesn't provide more capacity in the abstract, but a better organized capacity: a hierarchy of representations. Each layer builds concepts on top of the previous one's:

• In an image: layer 1 detects edges; layer 2 combines edges into textures and corners; layer 3, into parts (an eye, a wheel); layer 4, into whole objects.
• In text: characters → words → syntax → meaning.
• In audio: waveforms → phonemes → words.

This composition is exponentially more efficient than the brute force of one wide layer: it reuses intermediate concepts just as software reuses functions. And it has an enormous practical consequence that connects with 03-06: the network learns the feature engineering on its own. Where we hand-designed the RFM features for churn, a deep network on images discovers its own "visual RFM" layer by layer. That's why deep learning crushes on unstructured data (images, audio, text), where designing features by hand is nearly impossible.

In 01-02 we recounted the AI winters: neural networks have existed since the 1950s-80s, but for decades deep networks were untrainable in practice. Their resurrection (~2012, with AlexNet winning ImageNet) rested on three simultaneous legs:

No leg suffices alone: it's their historical coincidence that opened the current era. The first two are context; the third is engineering you need to know, and we unpack it next.

Training 50 layers with the "naive" backpropagation of 04-07 fails on two fronts: the gradients vanish as they traverse so many layers (the correction signal arrives at the first layers as nothing) and the millions of parameters overfit greedily. These are the tools that solved it:

Notice the pattern: none of these ideas is really foreign to you. Dropout is regularization (07-01) with an ensemble flavor (07-02); batch normalization is standardization (03-05); early stopping you already used in 07-03. Deep learning recombines the whole course's arsenal at another scale.

A "dense" network (every neuron connected to every other, like our MLP) ignores the structure of the data. The great architectures of deep learning are ways of building that structure into the network's design:

A 1000×1000-pixel image has 3 million inputs; a dense layer would need billions of weights. The CNN solves it with the convolution: instead of connecting each neuron to the whole image, a small filter (e.g. 3×3 weights) slides across the entire image, computing at each position the product with the local patch. The same filter — the same 9 weights — is reused at every position.

Two intuitions justify it: (1) a vertical-edge detector is useful in any corner of the image — sharing weights encodes that invariance and slashes the parameter count —; (2) stacking convolutions builds exactly the edges → textures → parts → objects hierarchy from section 1. Between convolutions comes pooling (keeping the maximum of each zone), which summarizes and grants tolerance to small shifts. CNNs dominate computer vision: image classification, object detection, medical imaging. At MercaFresh: automatically classifying the photos suppliers upload to the catalog — and it's the architecture we'll use in project 09-02.

For sequential data (time series, text), the recurrent network processes the elements in order while maintaining an internal state — a "memory" updated at every step. LSTMs (Long Short-Term Memory) add gates that decide what to remember and what to forget, fixing the inability of plain RNNs to retain long-range dependencies. Natural domain: forecasting daily demand for MercaFresh products, text (before 2018), audio, sensors.

The transformer (2017, "Attention Is All You Need") replaced recurrence with the attention mechanism: each element in the sequence decides, with learned weights, which other elements to "look at" to build its representation — in "the order that Ana canceled yesterday", the word "canceled" attends strongly to "order" and to "Ana". By eliminating step-by-step processing, training parallelizes massively, which made it possible to scale to models with billions of parameters trained on a good chunk of the internet. They are the foundation of the large language models and today's generative AI we mentioned when closing the history in 01-02 — GPT, Claude, Gemini are giant transformers. Studying them in detail is beyond this course; what matters is that you can place them.

So far we've used sklearn's MLPClassifier (04-07). For real deep networks, the standard is dedicated frameworks; we'll use Keras (TensorFlow's high-level API) for its clarity. The full framework catalog — TensorFlow, PyTorch and company — is covered in 08-01; here one is enough to get our hands on this lesson's techniques.

Reading the code, piece by piece:

• Sequential stacks layers in order; Dense is the fully connected layer of the MLP from 04-07. Three hidden layers (64→32→16) already make a modest "deep" network.
• compile sets the optimizer (Adam: the gradient descent of 04-01 with adaptive per-parameter steps) and the loss (binary cross-entropy is exactly the cost function of the logistic regression of 04-02: the sigmoid output of the last layer is a logistic regression over the learned representations).
• fit trains by epochs (full passes over the train set) in mini-batches of 32 examples, setting aside 20% for validation; the early-stopping callback watches that validation and restores the best weights, just as we did with boosting in 07-03.
• history.history stores the train/validation loss curves per epoch: plotting them with matplotlib reproduces the learning curves of 06-05 and is the first diagnostic for any network.

If you compare the result with the HistGradientBoosting of 07-03 on these same data, the network normally won't beat it (and takes longer). It's not a bug in the code: it's the lesson of the next section.

Time to be honest, because the marketing isn't. On tabular data — tables of customers, orders, RFM features: the daily bread of MercaFresh and of most companies — the empirical evidence is consistent: gradient boosting wins or ties almost always, with less effort.

The practical rule for MercaFresh and for your career:

• Does your data fit naturally in a pandas DataFrame? Start with gradient boosting (with the Random Forest of 07-02 as the baseline). Consider a network only if you have vast amounts of data and boosting falls short.
• Is your data images, audio or free text? Deep learning, no debate — and today, almost always, starting from a pretrained model rather than training from scratch (a technique, transfer learning, we'll touch on in the image project 09-02, where we'll finally build a CNN end to end).

With this lesson you have the complete map of deep learning at the informed-user level: you know what depth contributes, which techniques make it possible and which architecture fits each kind of data. Going deep into each architecture fills entire courses; in this course, the framework ecosystem (TensorFlow, PyTorch and their world) is cataloged in 08-01, and real hands-on practice with CNNs arrives in 09-02.

• Using deep learning by default on tabular data. The fashionable mistake. For MercaFresh's churn, a deep network takes longer, demands more tuning and will probably perform the same as or worse than LightGBM. Choose the tool for the data, not for the headline.
• Forgetting to scale the inputs. Unlike trees, networks are very sensitive to scale (gradient descent struggles with features of wildly different magnitudes). StandardScaler fitted on train only, always — the discipline of 03-05 and 06-01.
• Training with no validation set and no early stopping. A network with thousands of parameters memorizes the train set with relish; validation_split + EarlyStopping is the minimum seatbelt.
• Reading the training accuracy as performance. With dropout active, the train metric during fit is distorted (part of the network is switched off); always look at the val_loss/val_accuracy curves.
• Huge networks for tiny datasets. With 1,000 customers, a 4-layer network with hundreds of neurons is a flamethrower to light a candle; if you insist on a network, make it small and with generous dropout.
• Tip: for any network training run, always plot history.history["loss"] and ["val_loss"]. Divergence between the two = overfitting (more dropout, smaller network, stop earlier); both high and flat = underfitting or a badly chosen learning rate. It's the diagnosis of 06-05 applied epoch by epoch.

1. Training curves. Retrain the example's network without dropout (remove the Dropout layers) and without early stopping, with a fixed epochs=300. Plot loss and val_loss per epoch alongside those of the original network. Which pattern from 06-05 appears in the unregularized network, and how do dropout and early stopping correct it?
2. Depth vs. width. With roughly the same neuron budget (~112), compare three architectures on the scaled churn data: (a) a single layer of 112, (b) 64→32→16 (the example's), (c) 6 layers of ~19 neurons. Use early stopping and evaluate test accuracy with 5 different seeds each. Does depth win on this tabular problem? Why was that to be expected?
3. The definitive duel. Pit the example's network against HistGradientBoostingClassifier (07-03) on the same train/test: compare test F1 (use a 0.5 threshold on the network's probabilities), training time (time.perf_counter) and lines of preparation code. Write in three sentences the recommendation you'd make to MercaFresh's data team.

1. Without dropout or early stopping, loss falls monotonically while val_loss hits a minimum early and then climbs: the classic overfitting divergence (the same silhouette as the validation curve of 06-05 and as exercise 3 of 07-03). With dropout, both curves move forward closer together (the train set can't even be memorized, because each step sees a different subnetwork); early stopping additionally cuts off near the validation minimum and restores those weights. Regularization and early stopping attack the same symptom through complementary routes.
2. With 5 tabular features and 1,000 rows, the three architectures come out statistically tied (the differences between seeds exceed the differences between architectures); the deepest one is often the most unstable. It was to be expected: the hierarchy of representations shines when the data has compositional structure (pixels→edges→objects); five RFM features already are the representation — there is no hierarchy to learn. Depth pays on unstructured data, not here.
3. Typical result: similar F1 or in favor of boosting, with boosting training in a fraction of the time, without scaling and with less code. A reasonable recommendation: "For churn and MercaFresh's other tabular problems, keep gradient boosting as the standard tool; reserve deep learning for the catalog image classifier and other unstructured data; revisit the decision if data volume grows by orders of magnitude."

Deep learning is the MLP of 04-07 raised to a hierarchy: layers building representations on top of representations, trainable thanks to a handful of concrete techniques — ReLU against vanishing gradients, dropout and early stopping against overfitting, batch normalization for stability — and to the historical conjunction of GPUs and massive data we've traced since 01-02. Each architecture encodes its data's structure: CNNs for images, RNN/LSTM for sequences, transformers and their attention for language and today's generative AI. With Keras you trained your first deep network on MercaFresh's churn and drew the mature conclusion: on tabular data, gradient boosting still rules; the networks' kingdom is images, audio and text — and we'll truly visit it in project 09-02. Meanwhile, one outstanding debt runs through the whole module: alpha, n_estimators, learning rate, layers, dropout… every technique has added hyperparameters that so far we've chosen by eye. The module's final lesson brings order: systematic hyperparameter optimization.