The MercaFresh churn model now works, is served and is monitored. But there is a question latency and AUC do not answer: is it fair? The model decides which customers receive retention discounts and which do not, using personal purchase data. Every automated decision about real people brings with it questions of bias, fairness, transparency, privacy and security — and, increasingly, legal obligations. In this lesson you will learn where biases come from, how to audit a model by groups using the metrics you already master, what transparency demands, which principles govern data privacy, and how to condense it all into a practical checklist for the churn project.
Important warning: this lesson presents general principles for educational purposes and is not legal advice. The regulatory implications (GDPR, AI regulation, sector-specific legislation) of any real system must always be reviewed with a legal or compliance professional before going into production. All MercaFresh data and situations are fictitious.
Contents
- Why ethics is not an appendix to the project
- Bias: where it comes from
- Proxy variables: the postal code case
- Auditing fairness: metrics by group
- Transparency and explainability
- Privacy: minimization, anonymization and their limits
- GDPR: principles an ML team must know
- Model security
- An ethics checklist for the churn project
Why ethics is not an appendix to the project
A common reflex is to treat ethics as a final formality: "the model works; now, the ethics chapter". That is a mistake for three practical reasons:
- Ethical problems are technical problems with human consequences. A bias is, at bottom, a generalization error (module 6) that systematically affects a group of people. It is analyzed with the same tools: data, metrics, validation.
- They originate at the beginning, not at the end. Bias enters with the data (module 3) and with design decisions; once the model is in production, fixing it costs ten times more.
- They carry business and legal costs. A systematically unfair retention campaign damages trust and the brand; unlawful data processing brings serious penalties.
At MercaFresh the model decides something apparently harmless — who gets offered a discount. But "harmless" deserves scrutiny: if the model systematically offers worse conditions to one group, the supermarket is discriminating at scale, with the efficiency of a machine.
Bias: where it comes from
A model is not neutral just because it is mathematical: it learns what is in the data, and the data is a record of the world — inequalities included. The main sources:
| Source of bias | Mechanism | MercaFresh example |
|---|---|---|
| Historical data | The model learns past decisions and inequalities and perpetuates them | If retention campaigns historically targeted customers with large baskets, the "retained" labels reflect that policy, and the model learns to favor the same profile |
| Sampling | Groups underrepresented in training → worse performance for them | Customers in rural areas with few historical orders: the model knows them poorly and errs more with them (this connects with the data drift of 08-03) |
| Proxy variables | Apparently neutral variables that encode sensitive attributes | The postal code (next section) |
| Labels | The very definition of y embeds a judgment | Defining churn as "90 days without an order" penalizes seasonal buying patterns, more common in certain profiles |
| Feedback loop | The model's decisions generate the future data | If the model never offers retention to a segment, that segment churns more, "confirming" to the retrained model that it was high risk (the loop from 08-03, in its harmful version) |
The cross-cutting lesson: bias rarely enters through bad intent; it enters by default, with the usual data and the usual process, unless someone actively looks for it.
Proxy variables: the postal code case
The naive answer to bias is "don't use sensitive variables": remove gender, age, nationality, and the model will be neutral. It does not work, because of proxy variables: variables correlated with the sensitive attribute that allow the model to reconstruct it de facto.
The canonical example is the postal code. In the MercaFresh churn model it looks like an innocent logistics variable ("where do they order from?"). But postal code correlates strongly with socioeconomic status, and often with the origin or age of the neighborhood's population. If the model learns that certain postal codes have high churn and the system therefore stops investing in retaining (or offering benefits to) those neighborhoods, the practical outcome is a commercial policy that treats lower-income neighborhoods worse — without the word "income" appearing in any column. Remember correlation from module 2: the model does not need the sensitive variable if it has one correlated with it.
What to do, in practice:
- Proxy inventory: for each variable in the model, ask which sensitive attributes it could correlate with (postal code → income/origin; device type → income; usual shopping time slot → type of working schedule…). Correlation can be measured (02-03), not just intuited.
- Decide with criteria, not by inertia: sometimes the variable has legitimate value (real logistics do vary by area) and is kept under surveillance; sometimes its predictive contribution does not offset the risk and it is removed. It is a documentable design decision, not an oversight.
- Audit the outcome by groups — because removing variables guarantees nothing; you must measure the model's final behavior. That is what the next section is about.
Auditing fairness: metrics by group
The central tool of a fairness audit is simple and you already master it: the metrics from lesson 06-02, broken down by group. A model can have 85% overall precision and hide 92% for one group and 61% for another; the average covers it all up.
import pandas as pd
from sklearn.metrics import classification_report, confusion_matrix
# X_test includes a group column to audit (here, zone type derived
# from the postal code); y_pred are the pipeline's predictions with
# the business threshold from 06-04 already applied.
results = X_test.copy()
results["y_true"] = y_test
results["y_pred"] = y_pred
for group, group_data in results.groupby("zone_type"): # urban / suburban / rural
print(f"\n=== Group: {group} (n={len(group_data)}) ===")
print(classification_report(group_data["y_true"], group_data["y_pred"], digits=3))
print(confusion_matrix(group_data["y_true"], group_data["y_pred"]))
print(f"Selection rate (contacted): {group_data['y_pred'].mean():.1%}")How to read the audit, using the error-cost notions of 06-02:
- Recall by group: of the customers in this group who were really going to leave, what fraction does the model detect? Low recall in a group means its at-risk members do not receive the retention offer — the system's benefit does not reach them. Large recall differences between groups are the main alarm signal when a positive prediction grants access to a benefit.
- Precision by group and the selection rate complete the picture: are groups being contacted in reasonable proportion to their real risk?
- The n of each group: with small groups, differences may be sampling noise — the confidence intervals of 02-04 apply here too, before crying "bias".
Two honest caveats: first, there are several mathematical definitions of fairness (equalizing recall across groups, equalizing precision, equalizing selection rates…) and in general they cannot all be satisfied at once; which one to prioritize is a human decision that depends on which error harms whom the most — the cost analysis of 06-04, with a moral dimension. Second, the audit does not "fix" anything by itself: it discovers. Corrections (reweighting data, adjusting thresholds by group, redesigning variables) always start from having measured.
Transparency and explainability
When a model decides about people, someone will eventually ask — with every right — "why me?": the customer who did not receive the offer, the customer service team that must respond, the regulator who audits. European data protection regulation points in that direction for automated decisions with significant effects: meaningful information about the logic involved. Hence two practical requirements:
- System transparency: being able to explain what the model does — which variables it uses, how it was trained, what its metrics are, overall and by group, who supervises it. It is documentation, and the whole versioning discipline of 08-02/08-03 (metadata, model registry) is its material foundation.
- Explainability of each decision: being able to explain a specific case. This connects with what you have already seen: the feature importance of tree models and SHAP-style values (mentioned in 07-03) make it possible to say "this customer scores high because of their 60-day recency and their 3 delivery incidents" instead of "the algorithm says so".
And a design consequence you already know from 04-03: when the decision significantly affects people, an interpretable model can be worth more than a slightly more accurate opaque one. A decision tree or a logistic regression with few coefficients can be explained, audited and defended; if the opaque boosting model wins only a few tenths of AUC, in a decision with human impact that margin may not offset the loss of explainability. Model choice is also an ethical decision.
Privacy: minimization, anonymization and their limits
The churn model feeds on personal data: purchase history, addresses, habits. Principles of responsible handling:
Data minimization: use only the data necessary for the declared purpose. A practical test for each candidate variable: does it contribute measurable predictive power (the selection methods of module 3)? Does the purpose justify it? When in doubt, out — every piece of data you do not store is a risk you do not carry. MercaFresh's RFM model is a good example of minimization: it summarizes behavior in three aggregates without needing the detail of every basket.
Anonymization and pseudonymization — and their limits:
- Pseudonymizing (replacing the identity with a code, like
customer_id) protects against casual glances, but the organization keeps the mapping table: it is still personal data. - Truly anonymizing is hard. The risk is re-identification: cross-referencing the "anonymous" dataset with other sources until identities are recovered. A few quasi-identifying variables (postal code + date of birth + sex) are enough to uniquely single out a large part of a population, as re-identification studies have repeatedly shown. A detailed purchase history is almost a fingerprint.
- Practical consequence: treat "anonymized" data with almost the same care as personal data, aggregate whenever you can, and do not publish microdata without a serious re-identification risk analysis.
GDPR: principles an ML team must know
Without straying into legal advice (remember the opening warning), these principles of the General Data Protection Regulation shape the daily work of an ML team in Europe:
| Principle | Idea | Implication for MercaFresh churn |
|---|---|---|
| Legal basis | All processing needs a legal foundation (consent, contract, legitimate interest…) | It must be identified and documented before training on customer data; legal determines it, not the data team |
| Purpose limitation | Data is collected for explicit purposes; using it for another purpose requires analysis | Data collected to manage orders being reused to model churn: requires a compatibility assessment |
| Minimization | Only data that is adequate, relevant and limited to the purpose | The previous section, elevated to an obligation |
| Consent | When it is the basis, it must be free, specific, informed and revocable | Pre-ticked boxes or generic consents do not count; revocation must be reflected in future training data |
| Right to erasure | A person can demand the deletion of their data | The pipeline must be able to remove a customer from the data and from future retraining runs; backups and reference snapshots (08-03) count too |
| Automated decisions | Specific protections when a decision without human involvement has significant effects | Assess whether the model's decisions reach that threshold; provide for explanation and human intervention where applicable |
The operational message: these requirements shape the architecture (can you delete a customer from all your snapshots?), so they must enter the design from the start — so-called privacy by design — and be validated with the organization's legal professionals.
Model security
Brief but necessary: the model itself can be a channel for information leakage or a target of attack.
- Training data leakage: models memorize more than it seems. Membership inference attacks try to determine whether a specific person was in the training set; model inversion attacks try to reconstruct attributes of the training data from the model's outputs. The risk grows the more overfitted the model is (another reason against the overfitting of 06-05) and the more exposed the API is.
- Common-sense mitigations: do not expose the API publicly without authentication, rate-limit requests (it hinders mass-probing attacks), return only what is needed (do you need the exact probability to 6 decimal places, or is the risk band enough?), and log accesses (08-03).
- Advanced techniques exist (such as differential privacy) that are beyond the scope of this course; it is enough to know this front exists and that a model's API is one more attack surface.
An ethics checklist for the churn project
The practical framework that condenses this lesson, to be applied before every deployment of the MercaFresh model (it complements the technical checklist of 08-02):
Data and bias
- [ ] Legal basis and purpose of the processing documented and validated with legal/compliance.
- [ ] Minimization applied: every variable justifies its presence.
- [ ] Proxy inventory completed (postal code and the like): the decision on each one documented.
- [ ] Representation of the relevant groups in the training data reviewed.
Model and fairness
- [ ] Metrics broken down by relevant groups (not just global); differences in recall/precision/selection rate examined and justified or corrected.
- [ ] Decision threshold reviewed from a fairness standpoint too, not just cost (06-04).
- [ ] Level of interpretability appropriate to the decision's impact; per-case explanation available (importances/SHAP).
Operations
- [ ] System documentation up to date (version, data, metrics, owners) — the registry of 08-03.
- [ ] A channel in place for a customer to ask, complain or request human intervention.
- [ ] Right to erasure executable across data, snapshots and retraining runs.
- [ ] API protected: authentication, rate limiting, minimal outputs, logged accesses.
- [ ] The group audit is repeated on every retraining (drift can unbalance fairness too).
- [ ] Legal/compliance review completed before deployment and on every substantial change.
Common Mistakes and Tips
- "I removed the sensitive variables, the model is neutral." Proxies reconstruct what was removed. Neutrality is not declared: it is measured, by groups, on the model's outputs.
- Auditing once and filing the report. Fairness degrades like the metrics do (08-03): every retraining and every drift can alter it. The group audit belongs to continuous monitoring.
- Concluding "bias" from differences in tiny groups. With small n, per-group metrics have high variance; apply the statistical rigor of 02-04 before diagnosing (and also before dismissing).
- Treating GDPR like a cookie banner. Its principles shape which data you use, how you store it and whether you can delete someone from your snapshots: it is architecture, not final paperwork.
- Confusing pseudonymized with anonymous. With the mapping table in-house — or with enough quasi-identifiers — the data is still personal and the re-identification risk is real.
- Always choosing the model with the best AUC. When the decision affects people, a few tenths of a metric may not offset losing the ability to explain and defend each decision.
- Tip: give the ethics checklist the same discipline as your tests: it is run in full before every deployment, and one red item stops the train just like a broken test.
Exercises
Exercise 1. Someone proposes adding three variables to the MercaFresh churn model: (a) the brand of the phone the order is placed from; (b) the number of delivery incidents; (c) the customer's usual shopping time slot. For each one, identify which sensitive attribute it could act as a proxy for and what you would do before deciding to include it.
Exercise 2. The model's group audit shows, with the business threshold applied: urban zone (n = 41,200): recall 0.78, precision 0.64; rural zone (n = 3,900): recall 0.55, precision 0.66. Interpret what this difference means for rural customers, propose a likely cause connected to what you have seen in the course, and suggest two actions.
Exercise 3. A MercaFresh customer exercises their right to erasure. List all the places in the system built in this module where their data lives and what would have to be done in each one (think about 08-02 and 08-03).
Solutions
Solution 1. (a) Phone brand: a known proxy for purchasing power. Before including it: measure its correlation with available socioeconomic indicators, quantify how much it really contributes to the model and, if the contribution is marginal, discard it; if included, watch it in the group audit. (b) Delivery incidents: in principle operational and legitimate (it measures service quality), but it can correlate with the area (logistics are worse in certain zones), inheriting the postal code's proxy effect. Probably include it, but check how it is distributed across zones and whether it introduces systematic differences in treatment. (c) Time slot: it can reflect the type of working schedule (night shifts, family care). Moderate risk: measure contribution and correlations, document the decision and watch it in the audit. The common pattern: measure (02-03), justify the need (minimization), document, and audit the outcome — never decide on intuition.
Solution 2. The rural recall of 0.55 versus the urban 0.78 means that almost half of the rural customers who are going to leave are not detected and therefore do not receive the retention offer, versus a fifth of urban ones: the system's benefit reaches the rural group far less. The similar precision indicates that when it flags, it is about equally right; the problem is who it leaves out. Likely cause: underrepresentation (3,900 versus 41,200) — the model has learned rural churn patterns less well; the single threshold from 06-04 being poorly calibrated for that group's probability distribution may also play a part. First of all, verify that the difference is statistically solid (the rural n is not trivial, but a confidence interval would be advisable). Possible actions: retrain with better representation of the group (collect more history, reweight examples) and evaluate a threshold adjustment that roughly equalizes recall across groups, measuring its cost; in both cases, repeat the audit after the change.
Solution 3. Their data lives in: (1) the operational customer and order database — deletion per the corporate procedure; (2) the training datasets and reference snapshots of 08-03 — remove their rows so they enter neither retraining runs nor drift comparisons; (3) the nightly batch scores (the results parquet files of 08-02) — purge their historical records; (4) the API and monitoring logs — locate and delete their recorded requests/predictions; (5) the backups of all of the above — ensure expiry or deletion according to the defined policy. The trained model itself is not immediately retrained over one individual case (the aggregated parameters are not, in general, recoverable personal data), but the customer must no longer appear in any future training run — another reason why erasure must be provided for in the architecture from the design stage. And the obligatory coda: the complete procedure, including timelines and scope over backups, must be validated by a legal/compliance professional.
Conclusion
Module 8 closes here, and with it, the complete journey of the MercaFresh churn model: you chose the tools and froze the environment (08-01), serialized the pipeline, served it in batch and behind an API, packaged it in Docker and deployed it with a safety net (08-02), built the monitoring that detects drift and governs retraining (08-03), and in this lesson you added the final layer, the one that turns a system that works into a system that can be defended: biases traced to their origin, fairness measured by groups with the usual metrics, explainable decisions, data minimized and processed according to principles, the API protected and a checklist run in full before every deployment — always with legal review as the final lock. This is the complete picture of what professional machine learning means: not a model with a good AUC, but a production system, watched over and accountable. Everything you have learned across eight modules is now ready to be exercised end to end: five complete projects await you in module 9 — housing, images, sentiment, fraud and segmentation — where you will walk on your own, with new data, the path MercaFresh has taught us.
Machine Learning Course
Module 1: Introduction to Machine Learning
- What is Machine Learning?
- History and evolution of Machine Learning
- Types of Machine Learning
- Applications of Machine Learning
- The Machine Learning project workflow
Module 2: Foundations of Statistics and Probability
- Basic statistics concepts
- Probability distributions
- Correlation and covariance
- Statistical inference
- Bayes' theorem
Module 3: Data Preprocessing
- Data cleaning
- Handling missing data
- Data transformation
- Encoding categorical variables
- Normalization and standardization
- Feature engineering
Module 4: Supervised Machine Learning Algorithms
- Linear regression
- Logistic regression
- Decision trees
- Support Vector Machines (SVM)
- K-Nearest Neighbors (K-NN)
- Naive Bayes
- Neural networks
Module 5: Unsupervised Machine Learning Algorithms
- Clustering: K-means
- Hierarchical clustering
- Principal Component Analysis (PCA)
- DBSCAN clustering
- Data visualization with t-SNE and UMAP
Module 6: Model Evaluation and Validation
- Data splitting: training, validation and test
- Evaluation metrics
- Cross-validation
- ROC curve and AUC
- Overfitting and underfitting
Module 7: Advanced Techniques and Optimization
- Regularization: Ridge, Lasso and Elastic Net
- Ensemble Learning
- Gradient Boosting
- Deep neural networks (Deep Learning)
- Hyperparameter optimization
Module 8: Model Implementation and Deployment
- Popular frameworks and libraries
- Deploying models to production
- Model maintenance and monitoring
- Ethical and privacy considerations
Module 9: Hands-On Projects
- Project 1: Housing price prediction
- Project 2: Image classification
- Project 3: Sentiment analysis on social media
- Project 4: Fraud detection
- Project 5: Customer segmentation