An uncomfortable fact worth keeping in mind as this lesson starts: most of the machine learning models trained in companies never reach production. Not because the model is bad. Because nobody built the system around it.

AlpinaShop is at exactly that point. In six lessons it has built a heuristic recommender in SQL, a cart classifier, an image classifier, a two-tower model, eight thousand reviews analysed, twenty thousand images processed, two thousand four hundred product pages generated, a semantic search and an assistant with RAG.

And now, the questions nobody has asked yet:

Who runs all that? Lucía launches the sentiment process from her notebook when she remembers; Dani regenerates product pages with a script on his laptop. If tomorrow both are on holiday at the same time, nothing runs and nobody finds out.

How do we know the new model is better? We do not. The two-tower model was trained in March. If it is retrained today with six more months of data, nobody will compare the result with the one in production: the new one will be deployed because it is new.

Which data trained the model that made this recommendation? Nobody knows. It is not written down anywhere.

And if the data changes and a model degrades? Nobody finds out, until somebody on the business side says "this doesn't work like it used to"; and by then it will have been working badly for months.

Those four questions are MLOps. And answering them is what separates an experiment from a product.

1. Why models do not reach production
2. What MLOps is and its maturity levels
3. Vertex AI Pipelines: components, artefacts and the graph
4. AlpinaShop's recommendation pipeline
5. Component 1: extracting the data
6. Component 2: validating the quality
7. Component 3: training
8. Component 4: evaluating against production
9. The decision gate: it only deploys if it improves
10. Components 5 and 6: registering and deploying
11. Compiling and running the pipeline
12. Scheduled and event-driven execution
13. Metadata and lineage: what saves you in an audit
14. Model Monitoring and retraining
15. Versioning and reproducibility
16. Cost and sustainability of the cycle
17. The responsible ML checklist

The root cause is always the same: the model is a small part of the system. The modelling code is a minimal fraction of the total; everything else is infrastructure that has to be built and that almost never gets budgeted for.

The six concrete obstacles, with what has happened at AlpinaShop:

Notice something important: none of the six is a machine learning problem. They are software engineering and operations problems, and that is why the solution is not a machine learning one either. And there is an asymmetry worth understanding, because it explains why MLOps is not simply DevOps applied to models:

The last row is the most dangerous. A web service that goes down is detected in minutes. A model that is 15 % less accurate produces no error at all: it keeps answering, it keeps returning well-formed predictions, and nobody notices.

MLOps is the set of practices that take models into production reliably, reproducibly and sustainably. It combines DevOps with the particularities of ML: the data is part of the artefact and quality is measured statistically. Three maturity levels, and you do not have to reach the last one to have a healthy system:

AlpinaShop is at level 0 and in this lesson it goes to level 1. Level 2 requires Cloud Build and triggers from the repository, which is the subject of 06-01, and for a 40-person company it may be more than it needs. A piece of advice that saves projects: the jump from 0 to 1 delivers most of the value, because a reproducible pipeline that runs on its own, with a decision gate and lineage, answers the four questions from the start of the lesson. The jump from 1 to 2 is refinement.

Vertex AI Pipelines runs pipelines defined with the Kubeflow Pipelines (KFP) SDK without you having to manage any cluster. It is serverless: you describe the graph, compile it, run it and pay for what it consumes. Four concepts are enough.

A component is a step of the pipeline: a Python function packaged in a container, with declared inputs and outputs. An artefact is something a component produces or consumes and that the platform tracks — a dataset, a model, some metrics; artefacts are the key to lineage, because each one knows which component produced it and with what inputs. A parameter is a simple value passed between components or to the whole pipeline. And the graph is directed and acyclic, with the dependencies inferred on their own: if B receives A's output, it runs after A, without declaring any ordering.

Compared with Cloud Composer and Workflows (04-06), which you already know, the difference in purpose is clear:

They do not compete: they complement each other. Workflows still orchestrates the nightly data process and triggers the ML pipeline when it is time.

The goal: retrain the two-tower recommender from 05-03 every week, comparing it with the model in production and with the heuristic baseline from 05-01, and deploying it only if it improves.

pipeline_root is the Cloud Storage prefix where all the artefacts of all the runs are stored: it is what makes a model trained three months ago still exist and still be recoverable. And the parameters are in the signature, not inside the code: umbral_mejora=0.02 can be changed when launching the run without touching anything, which is what allows configurations to be tried without editing the pipeline.

The two nested dsl.If are the system's two gates: the data quality one and the model improvement one. If either fails, the pipeline finishes without deploying anything. That is the most important property of the whole design.

Four decisions this component inherits from the module. The JOIN with hist_cliente_diario at the previous date is the temporal leakage prevention from 05-02: the customer's history is taken from the day before the order, not from today. The split is temporal, not random: the last four weeks are the test set, because in a problem with seasonality a random split produces false metrics. p.activo = TRUE: it does not train with discontinued products, the same rule as in the heuristic recommender from 05-01. And the artefact's metadata: datos.metadata["consulta"] stores the exact query that produced this data, which is the answer to "which data trained this model?" and costs nothing to write.

And a note about the pinned versions in packages_to_install: without them, the component that works today can break in three months' time because a library changed. The same discipline as the image tags in 05-01.

This is where module 4 comes back, and not as decoration.

The seven rules are not arbitrary: they are the Dataplex quality dimensions you defined in 04-07, applied to the training set. Volume, uniqueness, completeness, validity, consistency. The same discipline, at another point in the flow.

And two deserve a pause. cobertura_historial >= 0.80: if the LEFT JOIN with the history fails for 40 % of the rows — because the process that builds hist_cliente_diario did not run — the model would train with most of its features empty, and without this rule that failure would be completely invisible: training would finish fine, the metrics would be somewhat worse, and nobody would know why. And sin_duplicados: a duplicate biases the model towards those examples and, if it lands in train and in test at once, it inflates the metrics.

A quality failure stopping the pipeline is the most important design decision in this lesson. The alternative — warn and carry on — produces the worst of both worlds: a model trained with bad data, deployed, with an alert nobody read. The same principle that stops the nightly Workflow in 04-06 when the quality check fails.

The training does not happen inside the component: the component launches a Vertex AI job and waits. That way you can use a GPU and isolate resources without the whole pipeline needing a big machine.

The image is versioned (1.0.3, never latest), as in 05-01 and 05-03. And the model's metadata records the architecture and the hyperparameters, not for documentation's sake but because they are what gets compared when one model does worse than another.

This component is the heart of the pipeline and it contains the idea most often forgotten in real projects.

The three metrics and why there are three. Comparing the candidate only against the model in production has a blind spot: if the model in production was already worse than the heuristic, a candidate slightly better than it is still worse than thirty lines of SQL. That is why the baseline from decision DA-003 is measured on every run, forever. It is not a ritual: it is the floor below which the whole system does not deserve to exist. And the return -1.0 when it does not beat it guarantees that the decision gate cannot open under any threshold configuration.

A realistic warning about offline metrics, which already came up in 05-03: recall@10 measures over past behaviour, and an offline improvement does not guarantee a business improvement. The pipeline decides which model is a candidate; the final validation is still an A/B test with business metrics. What the pipeline avoids is something that does not even improve in the lab reaching an A/B test.

Three lines of code that turn a retraining process into a system with judgement.

Why the threshold is 2 % and not 0 %. Because an improvement of 0.3 % is within sampling noise, and deploying on noise means changing the model every week with no real gain, with all the validation cost, risk and confusion that entails. The threshold must be larger than the metric's natural variability. How to estimate it: train the same model five times changing only the random seed and measure the deviation of recall@10; if it varies by ±1.5 % between identical runs, a threshold of 2 % is tight and 3 % is prudent.

What happens when the gate does not open. The pipeline finishes correctly. It is not a failure: it is the system working. The model in production stays, the run is recorded with its metrics and the team is notified. If the gate has been closed for eight weeks, that is valuable information too: the current model is good, or you need to change approach instead of retraining the same thing.

The four gates of a mature system, of which AlpinaShop has the first two: data quality (is it usable?, component 2), model improvement (is it better than production and than the heuristic?, component 4), bias by segment (does it improve for everyone or only in aggregate?, a recommended extension) and A/B test (does the business improve?, outside the pipeline).

The third deserves a note, because it connects with 05-02: a model can improve the global recall@10 and get worse for customers who buy from mobile or for new customers. Adding a gate that checks that no relevant segment gets worse by more than a threshold is one of the best possible extensions of this pipeline.

parent_model makes it a new version of the existing model and not a different model: it is what keeps the history and allows versions to be compared and rolled back. And the alias is candidato, not produccion: the pipeline registers, and promotion to production is an explicit step — automatic after the A/B test or manual. Separating registration from promotion is what lets you have a model ready without it serving.

And the deployment, which at AlpinaShop is not an endpoint, because of decision DA-002:

The modelo_origen column in the published table is small and worth its weight in gold: every recommendation a customer sees carries a record of which model version generated it. It is what makes it possible to answer, months later, "which model made this recommendation?".

The compiled YAML is the versionable artefact. It goes into Git alongside the Python code: it is the exact definition of the pipeline and it allows a run from months ago to be reproduced.

enable_caching=True is an excellent feature and a trap in equal measure. If a component runs with exactly the same inputs as on a previous occasion, Vertex AI reuses the result without running it again, which during development saves a great deal of time and money. But be careful: if the extraction component has the same parameters, the cache can return old data even though BigQuery has new data. The defence is to include a parameter that changes — the run date — in the components that read fresh data.

service_account matters too: the pipeline runs with a dedicated least-privilege account — read the necessary views, write to the artefact bucket, manage models — and nothing more. It is 03-04 applied to an automated process.

Three ways of launching the pipeline, and AlpinaShop uses all three:

Scheduled. Vertex AI Pipelines has its own scheduler, created with gcloud ai pipeline-jobs schedules create --cron="0 3 * * 1" --pipeline-job-file=reco_pipeline.yaml --max-concurrent-run-count=1. That last parameter matters: it stops a run that overruns from overlapping with the next one, something that produces race conditions when writing to the same table.

From Workflows, which is what is consistent with the 04-06 decision. The nightly data process already exists; the ML pipeline is chained after the data is ready and validated:

This is the integration that matters and it is the reason for chaining it this way: the ML pipeline does not run if the night's data failed. Without that chaining, on a Monday when the data process had failed, the pipeline would train with incomplete data — and although the quality gate would probably stop it, it is better not to get to that point.

Event-driven. Triggering from the repository when the pipeline code changes is maturity level 2 and corresponds to Cloud Build (06-01): a push to the main branch rebuilds the training image, recompiles the YAML, runs the pipeline in development and, if it passes, promotes it. Here it is only anticipated.

Vertex ML Metadata automatically records every run, every artefact and every relationship between them. There is nothing special to do: using KFP components with typed inputs and outputs already produces it.

The five questions it answers, and when you need them:

The fourth is the one that surprises everybody. A customer exercises their right to erasure and their data is deleted from pedidos. But that customer contributed to the training of three models, and their data is baked into the weights. Lineage makes it possible to know which models included it and to decide with legal judgement what to do — normally, ensuring the next retraining does not include it and documenting the reasoning. Without lineage, the question is simply unanswerable.

Querying it is direct: aiplatform.PipelineJob.list(filter='display_name:"reco-*"', order_by="create_time desc") enumerates every run with its status, and PipelineJob.get(...) with task_details returns the artefacts of a specific run with their metadata.

And the complementary practice: writing the model card as an artefact of the pipeline itself, with four sections — what it is for, what data it was trained with, what metrics it has, what known limitations — generated automatically from the metadata. Documentation that does not go stale because it is regenerated on every run.

The pipeline solves the "how to retrain". What is missing is the "when".

Three strategies, and the best one is the combination:

For AlpinaShop: weekly periodic as the baseline, with additional retraining if drift or the business metric calls for it. It costs little (section 16) and the decision gate guarantees that nothing that does not improve gets deployed. The two phenomena to watch, already seen in 05-01. Data drift: the distribution of the inputs changes, and at AlpinaShop it is seasonal and expected — in October winter gear comes in where in July there were trekking sandals — so a drift alert is not a fault. Concept drift: the relationship between inputs and outcome changes — a competitor drops prices and purchasing behaviour changes — and it is harder to detect and more serious.

Since AlpinaShop serves in batch and has no endpoint, the monitoring is done over the data, comparing the weekly distribution against the training one:

And the warning from 05-01, still standing: a threshold that is too sensitive produces noise, noise produces indifference, and indifference means nobody looks at the alert that did matter.

Reproducing a model requires four things to be versioned. Almost every team versions the first and none of the other three.

The data is the hard part. A BigQuery table changes constantly: you cannot "commit" it. Three solutions, from lowest to highest cost: storing the query and the cut-off date in the artefact's metadata (what component 1 does), cheap and sufficient if the tables are append-only; using BigQuery's time travel (FOR SYSTEM_TIME AS OF), which queries a table's state at a past instant within the retention window; and materialising a snapshot of the set in Cloud Storage, which is what pipeline_root does de facto by storing the Parquet.

The reproducibility checklist, worth verifying once and then not worrying about again:

• [ ] The compiled YAML is in Git with its version tag.
• [ ] The training image has a fixed tag, never latest.
• [ ] The packages_to_install carry pinned versions.
• [ ] The extraction query is in the artefact's metadata.
• [ ] The random seeds are fixed and recorded.
• [ ] The hyperparameters are in metadata, not in the code.
• [ ] Each run's artefacts persist in pipeline_root.

Here the argument that started in 05-01 with decision DA-002 closes, and now with numbers.

Option A: an online endpoint 24×7. Two n1-standard-4 nodes switched on all month to serve recommendations that change once a week, billing 24 hours a day whether there is traffic or not: of the order of hundreds of euros a month.

Option B: weekly batch prediction. A job that runs on Mondays, processes 45,000 customers in a few minutes and dies, plus the weekly training on a T4 and the pipeline run: of the order of a few tens of euros a month.

An order of magnitude of difference, and with no functional loss whatsoever: the recommendations are served from a precomputed table in Firestore or Memorystore (02-06), with microsecond latency and without depending on any model being alive. Check the current prices in the official pricing calculator; what does not change is the proportion.

The five measures for the cycle's sustainability: batch prediction instead of an endpoint whenever the result does not depend on the session in progress; Spot VMs with checkpoints for training (05-03), because weekly retraining runs are not urgent; pipeline caching in development, with the caveat from section 11; a retraining frequency proportional to the pace of change, since retraining daily a model whose patterns change over months is spending for spending's sake; and artefact retention, with a Cloud Storage lifecycle policy (02-02) that moves to Nearline at 30 days and deletes at 180, because each run's Parquet files and checkpoints pile up in pipeline_root without limit.

And the underlying observation, beyond the invoice: computational cost has an energy cost. Retraining unnecessarily, leaving GPUs switched on or serving 24×7 what can be precomputed is not just expensive: it is consumption with no purpose.

This section gathers everything the module has been leaving along the way. It is the checklist to complete before a model takes decisions that affect people.

Data

• [ ] What is the legal basis and is the purpose covered by the information given when the data was collected?
• [ ] Is the minimum necessary data used, training over pseudonymised views (04-07) and not over tables with direct personal data?
• [ ] Is there a retention period for the training set and can an erasure request be honoured? Does the lineage say which models included a person's data?
• [ ] Is the data representative of the population the model will be applied to?

Bias and fairness

• [ ] Has performance been measured by segment — device, channel, geography, tenure — and not just in aggregate? Is there any group for which it works significantly worse?
• [ ] Is there a feedback loop in which the model manufactures the reality it predicts (05-02, section 15)?
• [ ] Does the evaluation metric reflect what matters to the business and to the people affected?

Explainability

• [ ] Can you explain why the model took a specific decision, with local attributions, and in terms the person receiving the explanation can understand?
• [ ] Does the global importance of features make sense, or is there one with a suspiciously high weight that reveals a leak?

Human oversight

• [ ] Is there a person in the loop for decisions with significant impact, and a complaints channel with a response deadline?
• [ ] Can the model be switched off quickly and the previous process restored?
• [ ] Has somebody been assigned responsibility for watching it, by name?

Documentation

• [ ] Is there a model card with purpose, data, metrics and known limitations?
• [ ] Is it recorded who approved the deployment and on what evidence, and does the lineage allow you to reconstruct which data trained which version?

Compliance

• [ ] Is this an automated decision under Article 22 of the GDPR? If in doubt, you have to ask.
• [ ] What is the classification under the AI Act, what obligations does it carry, and does the current use still correspond to that classification?
• [ ] Is an impact assessment (DPIA) needed? Are the transparency obligations met?
• [ ] Has compliance or the DPO reviewed it?

Express and final recommendation of the module. This checklist is an aid for structuring the technical work and does not replace a legal opinion. The classification of a system under the AI Act, the determination of whether a processing operation constitutes automated decision-making under Article 22 of the GDPR, the need for an impact assessment and the scope of the transparency obligations must be determined by a compliance professional or by the data protection officer before putting the system into production. All the data and scenarios in this course are fictitious.

And the rule that sums up the whole module: if going through this checklist there are more than two boxes you cannot tick with confidence, the model is not ready for production, however good its recall@10 is.

Not having a decision gate. Deploying the new model because it is new is the most frequent and the most expensive mistake. Three lines of dsl.If prevent it.

Not measuring against the baseline. If the candidate does not beat thirty lines of SQL, there is no ML system to justify.

An improvement threshold of 0 %. Deploying on statistical noise produces weekly changes with no real gain. The threshold must exceed the metric's natural variability.

Trusting the pipeline cache without thinking. It can return old data from a previous run. Include the date as a parameter in the components that read fresh data.

Letting a quality failure not stop the pipeline. Warning and carrying on produces the worst outcome: a bad model deployed with an alert nobody read.

Using latest in images or not pinning library versions, which breaks reproducibility; and running the pipeline with broad permissions instead of a dedicated least-privilege service account.

Confusing an offline improvement with a business improvement. The pipeline decides what is a candidate; the A/B test decides what goes to production.

Retraining more often than the phenomenon changes. Daily when the patterns change over months is spending with no return.

Tip: always record the three metrics — candidate, production and baseline: it turns any discussion about whether the model improves into a query. Put the name of a person responsible for each model, because a model with no owner is a model nobody looks at. And generate the model card as an artefact of the pipeline: documentation that regenerates itself does not go stale.

The pipeline has been running for six weeks. In all six, the decision gate has not opened: no candidate has beaten the 2 % threshold. Dani proposes lowering the threshold to 0.5 % "so at least it gets updated". Analyse the proposal and say what you would do.

Design the bias-by-segment gate that is missing from the pipeline. State which segments you would check, what criterion you would apply and how you would implement it as a component.

A customer complains: they say the site recommended crampons incompatible with their boots, they bought them, and they are no use. They want an explanation. List what information you need, what should have been recorded, and what you would change in the system.

Solution 1

Dani's proposal is wrong, but the question behind it is a good one.

Why lowering the threshold is a mistake. The 2 % threshold was set because it is larger than the metric's natural variability. At 0.5 %, models whose "improvement" is indistinguishable from random-seed noise would be deployed. The model would change every week, every change would require validation, and the system would lurch about with no gain at all. And the pipeline's goal is not to update the model: it is for the best model to be in production. The gate not opening means the current model is still the best, which is exactly the desired outcome.

But six weeks in a row is information, and it has to be investigated. Four hypotheses, in order of probability. (1) The current model is already good and the approach has hit its ceiling, the most likely: retraining the same architecture with slightly different data does not produce jumps, and the answer is not to change the threshold but to change approach — add features, another architecture, session signals — or accept that the model is fine and drop the frequency to monthly. (2) There is not enough new data: with 730 days of history, one more week is an extra 0.14 %. (3) There is a problem in the pipeline: is the cache always returning the same data? Does the extraction component take the same window? It is the first technical check. (4) The metric does not capture the improvement: recall@10 may be saturated, and perhaps the model improves in something it does not see — diversity, catalogue coverage, results for new customers.

What I would do, in three steps. First, keep the threshold at 2 %: you do not touch a quality criterion to get the result you want to see, and that is exactly what the threshold exists to prevent. Second, investigate with the query above: if recall_candidato is practically identical week after week, it is hypothesis 2 or 3; if it oscillates with no trend, it is 1. And third, act according to the diagnosis: if the approach has hit its ceiling, drop the frequency to monthly (a direct saving) and open a work stream with new features; if it is a pipeline problem, fix it; if it is the metric, add secondary metrics of diversity and coverage and consider an A/B test even if the offline improvement is small.

And a cultural observation worth more than the technique: a team having an objective criterion and sticking to it when the result is not what it wanted is the most reliable sign of maturity. The threshold is only any use if it is also held to when it is inconvenient.

Solution 2

Segments to check at AlpinaShop:

The gate's criterion, with three conditions: that no segment with relevant volume gets worse by more than 5 % relative to the model in production; that the aggregate 2 % improvement holds; and that only segments with at least 500 customers in the test set are evaluated, because below that the metric is noise.

Where it goes in the pipeline: between the aggregate evaluation and the decision gate, nesting a second dsl.If over segmentos.output == True inside the aggregate improvement's dsl.If, so that the gate requires both conditions.

Three important considerations. The minimum of 500 customers is indispensable: without it, a segment with 12 customers would produce enormous swings that would block valid deployments, and a gate that always blocks ends up switched off. The −5 % tolerance is not zero for a reason: requiring that no segment gets any worse at all is impossible in practice, and what you are after is avoiding significant degradations, not fluctuations. And the results are recorded even when they do not block: metricas_segmento stores the delta of every segment evaluated, and over time that series reveals trends — if "mobile" has been below the aggregate for eight weeks without ever blocking, there is a structural problem no individual run would have uncovered.

And a warning in the name of honesty: this gate detects that a model gets worse for a group relative to the previous one, but it does not detect that it has always worked badly for that group. For that you have to look at the absolute values by segment, not just the deltas. Both things are in the checklist in section 17.

Solution 3

What information you need to answer: exactly which recommendation was shown, when and in what context (product page, cart, email); which model version generated it; which data trained that version; why the model scored that product highly for that customer (local attribution); what compatibility information existed on the product page at that moment; and what text accompanied the recommendation on the site.

What should have been recorded. Most of it already is if the lesson was followed:

The two that are missing are precisely the ones that matter most here, and that is already a finding from the analysis.

And now the uncomfortable part, which is where the real lesson is: even if everything were recorded, the technical answer would be "the model suggested that product because customers with a similar purchase profile bought both". That is true, it is explainable, and it resolves nothing for the customer, because the problem is not that the recommendation was statistically odd: it is that the system knows nothing about technical compatibility. A recommender learns purchase associations and does not know that an automatic crampon needs a boot with a toe welt. It never learned it because nobody taught it, and that information is not in lineas_pedido: it is in the spec sheet, and no component of the pipeline looks at it.

What I would change in the system, in four measures. The first, a hard compatibility filter applied before any model, which is the one that really solves the problem:

It is the same lesson as the stock > 0 from 05-01, in another dimension. There are business rules that must not be learned from the data: they are applied as a filter. A model, however good, must not be the last line of defence against a technically dangerous recommendation.

The second, compatibility warnings in the interface: products with technical requirements — crampons, bindings, anchor systems — carry a visible warning next to the recommendation ("Requires a boot with a front welt. Check compatibility."). It is cheap and it prevents most of these cases. The third, honest language: "other customers also bought" describes what the system really does, whereas "we recommend" or "compatible with your purchase" suggests a technical judgement the system is not making, and the wording matters legally, not just aesthetically. And the fourth, an additional gate in the pipeline: no deployment if the incompatibilities table is not up to date or if the filter is not applied, checked with a known test case.

The answer to the customer: acknowledge the problem without hiding behind the system, accept the return even if the product was used to try it, explain in plain language that the suggestion was based on purchase patterns and not on a compatibility check, and let them know it has been corrected. And record the case, because one customer who complains represents many who did not.

And the reflection that closes the module: this case is not a failure of the model, which did exactly what it was asked to do — find products that similar people buy together. The failure is of the system around the model, which did not incorporate an obvious business rule. And that is MLOps: not the training part, but all the part that stops a correct model from producing an incorrect result.

Module 5 closes here, and it closes with the complete system.

You know why most models do not reach production, and none of the six reasons is a machine learning one: lack of reproducibility, dependence on people, absence of a deployment criterion, lack of traceability, absence of watching and the impossibility of going back. All of them are engineering and operations problems. And you know the asymmetry that makes MLOps not DevOps by another name: in an ML system the code and the data change, degradation happens on its own because the world moves, the acceptance test is statistical and not deterministic, and the typical failure is silent: a model that is 15 % less accurate throws no exception.

You know the three maturity levels and that the jump from 0 to 1 — from notebooks and manual runs to a reproducible pipeline that runs on its own — delivers most of the value. Level 2, with full CI/CD, arrives with Cloud Build in 06-01.

You have built AlpinaShop's real pipeline with Vertex AI Pipelines and the KFP SDK: components, artefacts, parameters and a graph whose dependencies are inferred on their own. Extracting the data with the temporal split and the historical JOIN that avoids the leakage from 05-02. Validating the quality by reusing the Dataplex rules from 04-07, with the most important design decision of the lesson: a quality failure stops the pipeline instead of warning and carrying on. Training by launching a Vertex AI job with a versioned image. Evaluating against the model in production and against the heuristic baseline, always all three metrics, because if the candidate does not beat thirty lines of SQL there is no system to justify. And the decision gate: three lines of dsl.If that turn a retraining process into a system with judgement, with a threshold larger than the metric's natural variability and with the willingness to respect it when the result is unwelcome.

You know how to register as a new version with parent_model and the candidato alias — separating registration from promotion — and to publish in batch the table the shop consumes, with the modelo_origen column that makes every recommendation a customer sees traceable. You compile the pipeline into a YAML that goes into Git, you run it with a least-privilege service account, you know the caching trap, and you chain it from Workflows so that it does not run if the nightly data process failed.

You have metadata and lineage answering an audit's five questions, including the one that surprises everybody: which models included the data of a customer exercising their right to erasure. You know when to retrain by combining periodicity, drift and degradation, with the warning that a seasonal drift alert is not a fault and that noise produces indifference. You have the four things that have to be versioned — code, data, model and environment — with their checklist. And you have the calculation that closes the module's argument: an endpoint 24×7 to serve recommendations that change once a week costs an order of magnitude more than batch prediction, with no functional advantage.

Finally, the responsible ML checklist: data with a legal basis and the minimum necessary, bias measured by segment and not just in aggregate, explainability understandable to whoever receives it, human oversight with a complaints channel and the ability to switch it off, documentation with a model card and lineage, and compliance with the GDPR and the AI Act reviewed by compliance before production, not after. With the rule that sums up the module: if there are more than two boxes you cannot tick with confidence, the model is not ready, however good its recall@10 is.

And look where AlpinaShop is now. It has a governed data platform from module 4, models that predict, classify, generate and recommend, and a pipeline that retrains by itself, validates by itself, decides by itself and deploys only when it improves, with full traceability and with gates that stop bad data or a worse model from reaching production.

But now lift your eyes from machine learning and look at the rest of the system, because the comparison is devastating. The recommender has more engineering discipline than the shop.

The Flask web catalogue Dani maintains is deployed by copying files by hand. The infrastructure from module 3 — the VPC, the load balancer, the Cloud Armor rules, the certificates — was created with gcloud commands that live in Marta's terminal history and nowhere else: if the environment had to be rebuilt in another region, nobody would know exactly how. There are no automated tests: you push it and check by looking at the site. There are no genuinely separate environments: alpinashop-dev exists, but what gets tested there is not what gets deployed to alpinashop-prod. And when something goes wrong in production, the way you find out is a customer sending an email. It is exactly the maturity level 0 you have just moved past for models, applied to everything else.

In module 6, DevOps and monitoring, AlpinaShop takes to the rest of the platform the discipline it has just applied to the model cycle. We will start with 06-01, Cloud Build, the continuous integration that builds, tests and publishes every change automatically and reproducibly — and which, incidentally, is the missing piece for taking this lesson's pipeline from level 1 to level 2. Then will come Cloud Source Repositories and code management, Cloud Functions for the event-driven pieces this module has left pending — the analysis of every new image from the imagenes-subidas topic, among others — Cloud Monitoring to know what is happening without opening the console, Deployment Manager and Terraform so that the infrastructure stops living in a terminal's history and becomes written, versioned and reviewable like any other code, and Cloud Logging and Trace so that when something fails you can work out why in minutes and not in afternoons.

There is an application, there is data and there are models. They all work, and they all depend on two people remembering to run them. The time has come to automate delivery, write the infrastructure as code and find out about problems before the customers do.