The previous lesson's loop left CineClick with a system that watches itself, retrains itself, and deploys with safeguards. This lesson answers the questions no dashboard answers: why did the system decide what it decided about a specific customer? Does it treat all segments equally well? What do we document, how long do we keep what, and what does regulation expect from us? The good news — and the lesson's thesis — is that governance is not a layer of bureaucracy bolted on at the end: the system we've built over six modules is already auditable by design, and here we're only going to make that explicit, fill in the gaps (model card, ADRs, subgroup analysis, a privacy policy for the prediction table), and place it on the regulatory map with honesty about its limits.

1. Why governance is not bureaucracy
2. End-to-end traceability: the system is already auditable
3. Model cards: the model's fact sheet
4. Datasheets for datasets, briefly
5. Documenting decisions: lightweight ADRs
6. Fairness and bias: subgroup metrics
7. Privacy in CineClick's system
8. The regulatory landscape, in honest broad strokes
9. Governance checklist

The churn model makes decisions about people: who gets offered a retention discount and who doesn't. That makes questions like "why was this customer offered retention while their neighbor, with an almost identical profile, wasn't?" legitimate — and sooner or later mandatory. The customer themselves may ask, or the legal team, an auditor, or a regulator. If the answer is "we'd have to check which model was deployed back then… we think the March one", the problem isn't paperwork: it's that you don't control your own system.

Governance is, simply, being able to answer three questions in minutes and with evidence:

• What happened? — what prediction was made, when, with what input.
• With what? — which exact model, trained on what data and what code.
• Why is it there? — who approved that model, with what evidence, following what process.

And the pleasant surprise: almost all of that already exists at CineClick, because good MLOps practices and audit requirements are largely the same thing seen from two different offices.

Let's make the inventory explicit. Every audit question already has an answer with a concrete address:

Read it twice: we added nothing for auditing. Versioning data, tracking experiments, requiring promotion PRs, and logging every prediction with its version were engineering decisions; auditability came for free. That's the section's deep lesson: cheap governance is designed, not attached. The team working with loose notebooks and manual deployments cannot buy this table after the fact at any price.

What is missing are the narrative pieces: the documents that explain the system to someone who doesn't live inside it. That's where we're headed.

A model card is a model's technical fact sheet: a short, standardized document explaining what it does, what data it was built on, how it performs (globally and by subgroup), what it is NOT for, and what ethical considerations it raises. Google proposed it in 2019 and it has become the most widespread piece of model documentation. Its audience is not whoever trained the model: it's the new teammate, the legal team, the auditor.

The full model card of CineClick's champion, as it lives in docs/model-card-churn-v2.md:

Maintenance rules: the model card accompanies the version (v3 will have its own, generated as part of 05-02's promotion checklist), it's written in the repo (versioned with git, reviewable in the PR), and the limitations section is the most valuable one — a model card without honest limitations is marketing, not documentation.

The sibling idea, proposed by Gebru et al.: just as electronic components ship with a datasheet, every dataset should carry its own — motivation (why it exists), composition (what it contains, what's missing, what collection biases it has), collection process, recommended and discouraged uses, and maintenance policy. For CineClick, a one-page docs/datasheet-churn-customers.md documenting the training dataset: the origin of each column, the detail that the evaluation labels come from the control group, the temporal window covered, and the warning that the spring promo cohort is underrepresented before 2026-04. We won't develop it in full: the pattern is the same as the model card, applied to the data.

The system's important decisions were made throughout the course: mandatory two-week shadow, 10–25% positive-rate band, monthly retraining, PSI threshold of 0.25, 30-day window. A year from now, someone will ask "why two weeks and not one?" — and the answer cannot live in the team's memory.

A lightweight ADR (Architecture Decision Record) is a half-page markdown file: context, decision, alternatives considered, consequences. The repo already has the embryo of the pattern (docs/model-promotion.md documents the promotion process); we generalize it with a docs/decisions/ directory:

The convention: one ADR per decision, numbered, immutable (if the decision changes, another ADR is written that "supersedes ADR-007" — the decision history is as valuable as the decision in force). The cost is fifteen minutes per decision; the return, every time someone new asks "and why is this the way it is?".

Global metrics can hide that the model works systematically worse for a segment. At CineClick the effect is tangible: low recall in a subgroup means that its at-risk customers don't receive the retention offer — the system's benefit gets distributed unevenly. The analysis is straightforward with sklearn on the versioned test set:

Illustrative output for plan:

How to read it without fooling yourself:

• Compare recall across groups, but always look at n and prevalence: premium has fewer customers and less churn — its recall is noisier and its task is objectively harder (fewer positive examples to learn from).
• positive_rate measures the distribution of the "benefit" (the offer): if a group with similar prevalence systematically receives fewer offers, there's a fairness problem even if the recall adds up.

And if a subgroup consistently comes out worse? The menu of responses, from least to most intervention: (1) document it in the model card (done: premium 0.58) and watch it as a dashboard panel; (2) more data or better features for that segment — premium's low recall is often lack of signal, not algorithmic bias; (3) per-segment thresholds or reweighting in training — with an ADR, because it introduces explicit differentiated treatment that must be justifiable; (4) if the subgroup is defined by a protected characteristic or its proxy, escalate to legal/compliance before touching anything. And a cross-cutting rule: this analysis runs on every candidate evaluation (it gets added to 06-03's flow), not once a year.

Let's re-examine the system through data-protection glasses, because several decisions we made for engineering reasons turn out to also be privacy decisions:

• Minimization: the model uses no direct personal data — no name, no email, no address, no payment data; only customer_id (an internal identifier) and behavioral features. The best way not to leak a piece of data is never to have given it to the system.
• The log's input_hash (04-02): the prediction table stores no raw payloads; the hash allows detecting duplicates and verifying integrity without storing the literal input. Auditing without hoarding.
• Retention: the prediction table can't grow forever. CineClick defines an explicit policy — full detail (with features) for 13 months (covering one annual seasonality cycle + audit margin); afterwards, anonymous aggregates for the dashboard's historical series. Legal sets the exact figure; what matters is that it exists and is automated (a purge job, not an intention).
• Right to erasure and versioned data — the delicate point. If a customer exercises their right to erasure, what happens to the immutable DVC datasets where they appear? Deleting a row changes the hash and breaks the reproducibility of every model trained on that dataset. The strategies, in order of preference:
  1. Don't version PII: if the dataset only contains the internal customer_id and behavior, erasure is resolved in the master table mapping identity↔id (which is NOT in DVC). That's CineClick's strategy, and another win for minimization.
  2. Pseudonymization before versioning: the data enters DVC already pseudonymized; the pseudonym key lives outside, and deleting it unlinks the person from their rows.
  3. Purge the raw + re-derive: if a true deletion were required, the data is removed from the raw, the dataset is regenerated (new hash), and an ADR documents that the old versions are no longer reproducible and why. Reproducibility broken with justified cause and a trail, rather than silently.

Two European norms frame a system like CineClick's. Broad strokes, not a treatise:

GDPR, Article 22: people have the right not to be subject to decisions based solely on automated processing that produce legal effects or similarly significantly affect them. Does offering (or not offering) a retention discount cross that bar? It's debatable and case-dependent — but notice that CineClick's design already incorporates the natural safeguard: there are humans in the loop both in model promotion and in campaign design (marketing decides the offer over the prioritized list; the model doesn't execute actions on customers by itself, as the model card declares). The architecture we chose out of operational prudence is also the one that reduces the Article 22 risk.

The EU AI Act: it classifies systems by risk (unacceptable / high / limited / minimal) with proportional obligations. A churn model for retention campaigns points to limited or minimal risk — it's not on the high-risk lists (credit, employment, essential services…). But the valuable observation is another one: the requirements the regulation imposes on high-risk systems — risk management, data governance, technical documentation, event logging, human oversight, monitored accuracy and robustness — are exactly the content of this module. The traceability table, the model card, the prediction table, the human in the loop, and continuous monitoring are that list. Building with good MLOps practices is, to a large extent, building regulation-ready.

Important warning: the above is technical engineering guidance, not legal advice. The regulatory classification of a specific system, the applicability of Article 22, retention periods, and the exact obligations depend on each organization's context and must be determined by its legal and compliance professionals. The ML engineer's job is to build a system capable of complying — traceable, documented, with human oversight — and to arrive at that conversation with the technical answers ready; the legal qualification is not theirs to make.

The lesson, condensed into a table you can carry to any project:

• Treating governance as a final sprint before the audit. The traceability in section 2's table cannot be reconstructed after the fact: either the system generates it as it operates, or it doesn't exist. Design it from module 2, as we did (even if back then we didn't call it governance).
• Model cards written once and never updated. A v1 model card describing v3 is worse than none: it documents something false with authority. Tie it to the promotion process: no updated model card, no PR approval.
• Fairness analysis only with the favorite subgroup's global metrics. Define the subgroups and the cuts before looking at the results, and repeat it on every candidate — a model can be fair in v2 and stop being fair in v3.
• Confusing "I don't use protected attributes" with "there is no bias". Proxies exist (plan and payment method correlate with socioeconomic level). That's why the outcome is measured by subgroup instead of trusting the variable's absence.
• Keeping everything forever "just in case". Indefinite retention is a privacy and cost liability. Explicit policy, justified period, automated purge.
• Writing ADRs only when the decision is controversial. Today's "obvious" decisions are the mysteries of two years from now. If it took more than a ten-minute discussion, it deserves an ADR.
• Playing lawyer. Prepare impeccable technical evidence and take the regulatory questions to legal/compliance. Engineering answers to legal questions age badly.

1. Audit drill. An auditor asks: "On April 14, 2026, customer 88213 was included in the retention campaign. Justify that decision: which model made it, with what input, what evidence there was that the model was adequate, and who approved it". Write the answer step by step, citing, for each element, the exact system component where the evidence lives (use section 2's table).

2. The subgroup that gets worse. In the evaluation of v3 (the retraining's challenger), recall for the 0-6m tenure bracket falls from 0.61 to 0.44, while the global figure rises from 0.68 to 0.71. With what you've learned in this module: (a) give a plausible hypothesis of the cause related to the promo scenario; (b) say what you'd do about v3's promotion; (c) indicate which documents would need touching whatever the decision.

3. Erasure with DVC. In a new project, CineClick decides to version a dataset with DVC that includes customers' email addresses "because it simplifies the joins". Write the objection paragraph you'd post on the PR, mentioning: the conflict with the right to erasure, the two preferable strategies, and the cost of fixing it later.

1. (i) What prediction: customer 88213's row in the prediction table with timestamp 2026-04-13/14, its input features, probability ≥ 0.5, and a group other than control — origin: the weekly batch weekly-churn-scoring of Monday the 13th. (ii) Which model: that row's model_version field (v2); cross-checked against the registry's alias history, which confirms v2 = @champion on that date. (iii) The model's adequacy: v2's MLflow run (recall 0.68 / precision 0.55 on the frozen test, above the 0.60/0.50 business criterion), lineage tags pointing to the exact DVC dataset, the v2 model card with subgroup metrics, and the results of its two weeks of shadow and canary prior to promotion. (iv) Who approved: v2's promotion PR (identified reviewer, docs/model-promotion.md checklist completed) and the associated promote_champion.py execution. (v) The final decision to include them in the campaign: the prioritized list was handed to marketing, which configured the offer — human in the loop, documented in the model card. Estimated response time with the current system: minutes.

2. (a) Hypothesis: v3 was trained on the window that includes the promo cohort — many new customers (0-6m) with still-immature labels or atypical behavior (their 30-day horizon hasn't elapsed yet, or their initial usage pattern doesn't represent their stable behavior), so the model learned noisy or outright wrong signal for that bracket; the global figure rises because the large brackets improve. (b) Don't promote yet: the global criterion is met, but a 17-point recall drop in a segment means de facto withdrawing retention from new customers — a business decision that can't be made by omission. Options: delay v3 until the cohort's labels mature, reweight/adjust the cohort's inclusion cutoff in the dataset, or promote with a per-segment threshold — any of them with an ADR and in agreement with product. (c) Documents: the subgroup evaluation attached to the promotion PR (which in this case blocks it), an ADR with the decision taken and its alternatives, and — if some variant does get promoted — v3's model card with the 0-6m bracket limitation described honestly.

3. A model paragraph: "Objection to versioning email in DVC: DVC datasets are immutable by design (the hash is the data's identity), so a future erasure request would force us to choose between missing the deadline or purging the raw and re-deriving, breaking the reproducibility of every model trained on that dataset and forcing an exception ADR. I propose: (1) doing the joins with the internal customer_id and keeping the identity↔id mapping outside DVC, where erasure is a DELETE with no effect on the versioning; or (2) if the email carries signal (domain, age), deriving those features and versioning only the pseudonymized derivative. Fixing it later will cost purging and re-deriving N historical versions, invalidating the comparability of the models trained on them, and documenting the exception — all avoidable today with a one-line change in the join."

Governance has turned out to be, above all, a shift of perspective: the system built over six modules already answered the audit questions — prediction table, lineage in the registry, promotion PRs — and this lesson has made that explicit and added the narrative and accountability layer: v2's model card with its honest limitations, the dataset's datasheet, the ADRs in docs/decisions/, subgroup evaluation inside the candidate flow, the prediction table's privacy policy (minimization, hashing, retention, DVC-compatible erasure), and the regulatory map with its warning: engineering prepares the evidence; the legal qualification belongs to legal and compliance. The twelve-point checklist condenses the module into something portable. And with it, construction closes: there is no piece left to add to CineClick's platform. The course's final lesson builds nothing — it integrates: the complete map of the system, an end-to-end incident that exercises everything, and the project through which you'll make everything you've learned your own, in another domain.