The network of 05-02 has 497 random parameters and says "0.55" to every order. This lesson explains how those numbers go from random to useful, that is, how a network learns. We announced the idea when closing 03-04: learning is optimising, and when the loss function is smooth there is no need to probe neighbours at random as in hill climbing, because we can compute the slope and walk down it. We will see the loss as a landscape, the derivative as a slope, gradient descent step by step with a one-variable example worked by hand, the role of the learning rate, the chain rule that lets us distribute the error backwards through the layers (backpropagation), the batch and mini-batch variants, the optimisers (SGD with momentum, Adam) and, at last, the network-specific regularisation techniques we left pending in 04-06: early stopping, dropout, data augmentation and batch normalisation. In code: a gradient descent from scratch in numpy for one neuron and the complete PyTorch training loop for the NovaMarket MLP, evaluated with AUC against the logistic regression of 04-05. It matters because this algorithm is the one that trains everything, from our 497-parameter network to the large language models of 05-05.

1. The loss as a landscape: from hill climbing to a compass
2. The derivative as a slope and the intuition of the gradient
3. Gradient descent step by step: a one-variable example by hand
4. The learning rate: too large, too small
5. Chain rule and backpropagation
6. Batches, mini-batches, epochs and optimisers (SGD, momentum, Adam)
7. Network-specific regularisation: early stopping, dropout, data augmentation and batch normalisation
8. Code (a): gradient descent from scratch in numpy
9. Code (b): full training of the NovaMarket MLP in PyTorch
10. Common Mistakes and Tips
11. Exercises
12. Conclusion

Remember 04-01: the loss function measures, with a single number, how wrong the model is on the training data with its current parameters. For the sigmoid neuron with binary cross-entropy (05-02), the loss depends on the weights and the bias: change the weights and the loss changes. Picture a landscape in which every point on the ground is a combination of parameters and the height is the loss: training is walking down to the lowest valley. With 2 parameters the landscape can be drawn; with 497 it cannot, but the geometry is the same.

In 03-04 we went downhill "blind": hill climbing tried random neighbours and moved if they improved, and simulated annealing sometimes accepted getting worse. That worked for delivery routes, where there is no defined slope (swapping two stops is not "a little to the left"). With continuous parameters and a smooth loss we have something better: a compass that points downhill, the gradient. No need to try neighbours: you compute the direction of steepest descent and take a step. Repeated thousands of times, that step is gradient descent.

The derivative of a function at a point is its slope there: how much the function rises per unit you move forward. If f(w) = (w − 3)² + 1 (a parabola with its minimum at w = 3), its derivative is f'(w) = 2·(w − 3). At w = 0 it is −6 (the function drops steeply towards the right); at w = 2.5 it is −1 (a gentle drop); at w = 3 it is 0 (flat: the minimum); at w = 5 it is +4 (rising). You do not need to know how to compute derivatives for this course: the idea of "slope at the point" is enough, together with knowing that PyTorch computes them for you (section 5).

With several parameters, the gradient is the vector of partial derivatives, one per parameter: the slope of the loss "if I move only w₁", "if I move only w₂", and so on. It points in the direction in which the loss rises fastest; its opposite, downhill. The gradient descent rule for each parameter θ is:

θ ← θ − η · ∂L/∂θ

that is, "subtract from each parameter its slope multiplied by the learning rate η". If the slope is negative (the loss decreases as θ grows), the subtraction increases it; if it is positive, it decreases it. Compare it with the perceptron rule of 05-01, w ← w + η·e·x: it is the same shape, and in fact we will see that the perceptron rule is a special case.

Let us minimise f(w) = (w − 3)² + 1 from w = 0 with η = 0.1. In each iteration: we compute the slope 2(w − 3), and update w ← w − 0.1 · slope.

Three things visible in the table that hold for any network: (1) the loss decreases at every step (10 → 6.76 → 4.69 → ...); (2) the steps are large far from the minimum and small near it, because the slope softens as we approach the valley, without anyone changing η; (3) it never reaches the minimum (w = 3) exactly but keeps getting closer indefinitely: in practice you stop when the improvement is negligible or, better, when validation says so (section 7). In a network the only difference is that instead of one slope there are 497 (one per parameter) and that the function is not a parabola but a landscape with many valleys; the algorithm walks down into the valley it falls into, which may not be the deepest one, exactly the local optima problem of 03-04.

η is the most important hyperparameter of training. With the same parabola and the same starting point:

If you plotted the loss per iteration: with a small η, a curve that drops slowly and flattens before arriving; with an adequate η, a fast drop that stabilises; with a large η, oscillations or a curve that rises and blows up (in PyTorch you get nan). There is no universal value: it depends on the scale of the data (that is why we standardise) and on the architecture. Typical values with Adam (section 6): between 0.0001 and 0.01. In practice it is tuned with validation (04-06), and it is often reduced during training (a learning rate schedule): large steps at the beginning, fine ones at the end, the same intuition as the temperature in simulated annealing.

To apply θ ← θ − η·∂L/∂θ you need the slope of the loss with respect to each weight, and in a network the weight of the first layer affects the loss through a long chain of operations. The tool is the chain rule: the slope of a composition of functions is the product of the slopes of each link. If L depends on p, p on z and z on w, then ∂L/∂w = (∂L/∂p) · (∂p/∂z) · (∂z/∂w). Intuition: if z rises 2 units per unit of w, p rises 0.25 per unit of z and L drops 3 per unit of p, then L drops 3 × 0.25 × 2 = 1.5 per unit of w.

Let us see it in the computation graph of a sigmoid neuron with BCE loss (our logistic regression):

The solid arrows are the forward pass (computing the prediction and the loss); the dashed ones, the backward pass: starting from the loss, each node receives the accumulated slope, multiplies it by its local slope and passes it on to its inputs. For the sigmoid + BCE pair the first two links simplify beautifully: ∂L/∂z = p − y (the predicted probability minus the label). Therefore ∂L/∂w = (p − y)·x and ∂L/∂b = p − y. Compare with the perceptron rule: e·x with e = y − ŷ. It is the same rule, with the probability in place of the step, and now we know where it comes from: it is the gradient of the loss.

Numerical example: order x = (0.5, 1.0, 1) (standardised amount, days and new customer), label y = 1 (it was returned), weights w = (0.4, 0.3, 0.9), b = −2.

• Forward: z = 0.2 + 0.3 + 0.9 − 2 = −0.6; p = σ(−0.6) = 0.354; loss L = −log(0.354) = 1.037. The network gives 35 % to something that happened: bad.
• Backward: ∂L/∂z = p − y = −0.646; ∂L/∂w = −0.646 · x = (−0.323, −0.646, −0.646); ∂L/∂b = −0.646.
• Step with η = 0.5: w ← (0.4, 0.3, 0.9) − 0.5·(−0.323, −0.646, −0.646) = (0.561, 0.623, 1.223); b ← −2 + 0.323 = −1.677.
• Check: now z = 0.449, p = 0.61, L = 0.494. The loss has dropped from 1.037 to 0.494 for this order. With all the orders, the gradient used is the average of the individual gradients.

In a network with hidden layers the chain is longer but identical in spirit: the slope of the loss with respect to the output of the last layer is propagated to the second-to-last one by multiplying by the weights and by the slope of the activation (1 or 0 for ReLU; ≤ 0.25 for the sigmoid, hence the vanishing of 05-02), and so on down to the first. That algorithm, which reuses the backward computations layer by layer instead of redoing them for each weight, is backpropagation, popularised in 1986 by Rumelhart, Hinton and Williams (01-01), and it is what made it possible to train the hidden layers that solved XOR. In PyTorch you will never write it: every tensor remembers the operations that produced it (the computation graph), and L.backward() walks that graph backwards and leaves in p.grad the slope of each parameter. This is automatic differentiation, and it is the reason deep learning frameworks exist (07-03).

Over how many examples do we compute the gradient before taking a step?

An epoch is a full pass over the training data (as in the perceptron). With mini-batches of 64 and 1,799 orders, each epoch is 29 gradient steps; 100 epochs, 2,900 steps. The data are shuffled at the start of each epoch so that the batches change.

The optimiser is the rule that turns gradient into update. The basic one is that of section 2 (SGD). Two almost universal improvements:

• Momentum: instead of moving only according to the current gradient, a "velocity" is accumulated (a moving average of the recent gradients). Like a rolling ball: it crosses small bumps, accelerates on long slopes and damps oscillations in narrow valleys. torch.optim.SGD(..., momentum=0.9).
• Adam (adaptive moment estimation): momentum plus a learning rate adapted to each parameter (those receiving large gradients are slowed down, those receiving small gradients are sped up). It is robust to the choice of η and is the default optimiser to start with. torch.optim.Adam(..., lr=0.001).

In our MLP, the validation loss after 1, 5, 10, 20 and 30 epochs was: SGD (η = 0.01) 0.72 → 0.60 → 0.53 → 0.47 → 0.45; SGD with momentum 0.55 → 0.45 → 0.42 → 0.34 → 0.33; Adam (η = 0.001) 0.73 → 0.48 → 0.39 → 0.34 → 0.33. All three go to the same place; momentum and Adam get there much sooner.

A network with hundreds of parameters for every few examples overfits easily (04-06). Besides L2 regularisation (here called weight decay, an argument of the optimiser), deep learning adds four techniques of its own:

This closes what 04-06 left pending. In the code of section 9 we will use early stopping and dropout; data augmentation and batch normalisation appear in 05-04.

We train a sigmoid neuron (logistic regression) on 3,000 NovaMarket orders with three columns, writing the gradient by hand as in section 5:

Output:

Reading: with w = 0 the initial loss is ln 2 = 0.693 (the network says 0.5 to everything); in 10 epochs it has dropped to 0.40 and then flattens at 0.326: we have reached the bottom of the valley (this loss is convex, so it is the global minimum). The weights converge to (0.99, 0.65, 0.94) and b = −2.23; LogisticRegression without regularisation on the same data gives (0.996, 0.661, 0.945) and −2.248: we have reproduced fit in twenty lines. This is batch descent (all 3,000 orders at every step); with rate = 0.01 after 200 epochs the loss was still at 0.51 (too slow), and with rate = 20 it oscillates between 0.47 and 0.66 without converging (too large), just as section 4 predicts.

Now the network 21 → 16 → 8 → 1 of 05-02, with everything we have learned: mini-batches, Adam, dropout, early stopping with a validation set and final evaluation with AUC on the same test set as 04-05.

The test set is the same 750 orders as in 04-05 (same seed), so the AUC will be comparable with the 0.844 of logistic regression. The validation set (450 orders) serves for early stopping; the test set is looked at only once, at the end (04-06).

Output (varies slightly with the seed and the PyTorch version):

Explanation of the key pieces of the loop:

• DataLoader(..., batch_size=64, shuffle=True) delivers the 1,799 orders in 29 mini-batches that differ in every epoch.
• The trio zero_grad() → backward() → step() is the heart: without zero_grad() the gradients accumulate from one batch to the next (a classic mistake); backward() is the backpropagation of section 5, automatic; step() applies Adam's rule.
• net.train() / net.eval() toggle the dropout: random switching off while training, all neurons when evaluating. That is why the training loss in the table is somewhat pessimistic (it is computed with neurons switched off) and may sit above the validation loss in the first epochs.
• Early stopping has chosen epoch 75 looking only at validation; without it, what would have happened? You will see it in exercise 2: continuing for 300 epochs without dropout, the training loss drops to 0.22-0.23 but the validation loss rises to 0.39-0.40 and the test AUC falls below 0.78. Textbook overfitting.

The test AUC is 0.834; repeating with five different seeds gives between 0.834 and 0.842 (mean 0.838). The logistic regression of 04-05 gave 0.844 and the tuned forest of 04-06, 0.838. The network does not win: it ties within the noise, with more complexity, less interpretability and more decisions to make. Marta explains it to Diego bluntly: for the returns predictor, tabular data with 2,249 examples and a fairly linear "truth", logistic regression is still the choice (04-06: for equal performance, the simpler model). This is to be expected and worth knowing: on small tabular data, networks rarely beat well-tuned classical models. The network earns its place where they cannot reach, and that is where we are heading in 05-04 and 05-05: the incident photos and the reviews.

• Forgetting opt.zero_grad(). The gradients add up to the previous step's and training runs wild. It is the number one mistake when writing the loop by hand.
• Sigmoid in the network and BCEWithLogitsLoss at the same time. Double sigmoid: the network learns poorly and the "probabilities" end up squashed. Either sigmoid + BCELoss, or nothing + BCEWithLogitsLoss (recommended).
• Not toggling train()/eval(). Evaluating with dropout active gives worse, random results; training in eval mode disables the dropout.
• Learning rate by eye. If the loss does not go down or gives nan, that is the first thing to touch (lower it). If it goes down very slowly, raise it or use Adam. Tune it with validation like any hyperparameter.
• Choosing the number of epochs by looking at the test set. It contaminates the single final look (04-06). That is what validation and early stopping are for.
• Comparing with logistic regression on a different data split. Our test set is exactly that of 04-05 (same seed): only then can the 0.834 be compared with the 0.844.
• Concluding "networks are bad" from this result. It is a small, almost linear tabular problem; the verdict changes with text and images (05-04, 05-05).

Exercise 1. Compute by hand one gradient descent step for the neuron of section 5 with x = (1, 0, 2), y = 0 (not returned), w = (0.5, −1, 0.8), b = −1 and η = 0.5: obtain z, p, the loss, ∂L/∂z, ∂L/∂w, ∂L/∂b, the new parameters and the new loss. Check that it has gone down.

Exercise 2. Train the network of section 9 without dropout (build_net(0.0)) and without early stopping (patience=10**6, epochs=300), evaluating in eval mode the training loss, the validation loss and the test AUC at epochs 10, 30, 50, 100, 200 and 300 (with evaluate, and without restoring the "best state"). Describe the curve and explain at which epoch you should have stopped.

Exercise 3. In the code of section 8, change rate to 0.01, to 5 and to 20, and note the loss at epochs 1, 5, 20, 50 and 200. Classify each rate according to the table in section 4 and explain why the loss with rate = 20 does not go down even though it does not blow up either.

Solution 1. z = 0.5·1 + (−1)·0 + 0.8·2 − 1 = 1.1; p = σ(1.1) = 0.750; loss L = −log(1 − 0.75) = 1.386 (it gives 75 % to a return that did not happen). ∂L/∂z = p − y = 0.75; ∂L/∂w = 0.75·x = (0.75, 0, 1.5); ∂L/∂b = 0.75. New parameters: w = (0.5 − 0.375, −1 − 0, 0.8 − 0.75) = (0.125, −1, 0.05), b = −1 − 0.375 = −1.375. New pass: z = 0.125 + 0.1 − 1.375 = −1.15, p = 0.240, L = −log(0.76) = 0.275. The loss has dropped from 1.386 to 0.275; the weight of the second column does not change because its input was 0 (∂L/∂w₂ = 0.75·0).

Solution 2. In our run (exact values vary with the seed): epoch 10, train 0.36 / val 0.39 / test AUC 0.78; epoch 30, 0.30 / 0.33 / 0.83; epoch 50, 0.29 / 0.32 / 0.83; epoch 100, 0.27 / 0.33 / 0.82; epoch 200, 0.24-0.25 / 0.35 / 0.78-0.80; epoch 300, 0.22-0.23 / 0.39-0.40 / 0.73-0.78. The training loss keeps dropping (the network memorises), the validation loss bottoms out around epoch 50 and then rises, and the test AUC follows it: from 0.83 it falls below 0.78. You should have stopped around epoch 50, which is exactly what early stopping with patience does on its own. It is the overfitting curve of 04-06 seen along the "epochs" axis instead of "complexity".

Solution 3. rate = 0.01: losses 0.692; 0.686; 0.666; 0.629; 0.507. It always goes down but after 200 epochs it is far from 0.326: too small. rate = 5: 0.345; 0.328; 0.326; 0.326; 0.326: converges in 20 epochs, adequate (even better than 0.5 for such a smooth problem). rate = 20: 0.587; 0.470; 0.600; 0.660; 0.609: too large, it jumps from one side of the valley to the other without settling. It does not blow up because the log loss with a sigmoid grows slowly (large jumps in w saturate the sigmoid and the slope becomes small, which brakes the next step), unlike the parabola of section 4, whose slope grows with distance and does diverge.

We have seen how a network learns. The loss is a landscape and, unlike the hill climbing of 03-04, we now know the slope: the derivative, and with several parameters the gradient. Gradient descent takes steps θ ← θ − η·∂L/∂θ (we followed it by hand on a parabola: 10 → 6.76 → 4.69 → ...), the learning rate decides whether the descent is slow, adequate or divergent, and the chain rule lets us compute the slope of every weight by multiplying local slopes backwards through the computation graph: that is backpropagation, which PyTorch does on its own with backward(). We understood mini-batches and epochs, what momentum and Adam improve, and closed the pending list of 04-06 with early stopping, dropout, data augmentation and batch normalisation. In code, we reproduced fit of logistic regression with twenty lines of numpy (loss 0.693 → 0.326) and trained the NovaMarket MLP in PyTorch with the loop zero_grad → backward → step: test AUC 0.834, tied with logistic regression (0.844) and the forest (0.838). For the returns predictor, Marta sticks with logistic regression.

That tie is the best introduction to the next lesson. If networks do not win on order tables, where do they win, and why? In Deep Learning and Its Applications we will see what makes deep learning "deep" and why it took off in 2012, and the architectures that build in the structure of the data: convolutional networks, which we will try on NovaMarket's incident photos (damaged or intact package), recurrent networks for sequences, autoencoders for anomalies and transfer learning, the idea of not training from scratch.