The Project | About Us | Contribute | Donations | License
HOME My courses Completed
◄ Previous
◄

Deploying models to production

Next ►
►

So far the MercaFresh churn model lives in a notebook: the preprocessing pipeline from module 3, the algorithm optimized in module 7, the decision threshold chosen by cost analysis in module 6. But a notebook cannot take the call from the campaign system at 3 a.m. asking "what is the churn probability for customer 48210?". Taking a model to production means turning it into a reliable service other systems can use. In this lesson we walk that entire path: serializing the pipeline with joblib, choosing between batch and real-time serving, building an API with FastAPI, packaging it with Docker and deploying it gradually and safely.

Contents

  1. From notebook to production: what changes
  2. Serializing the pipeline with joblib
  3. Versioning the artifact and its risks
  4. Serving patterns: batch vs. online
  5. A REST API with FastAPI
  6. Packaging with Docker
  7. Gradual deployment: shadow mode and A/B testing
  8. Go-live checklist

From notebook to production: what changes

A notebook and a production service are objects of a different nature, even though they contain the same model:

graph LR
    subgraph nb["Notebook (exploration)"]
        A[Historical CSV data] --> B[Pipeline + model]
        B --> C[Metrics and plots]
    end
    subgraph prod["Production (service)"]
        D[Requests from other systems] --> E[Serialized artifact]
        E --> F[Predictions]
        F --> G[Campaign system]
        E --> H[Logs and monitoring]
    end
    nb -- "serialize + package + deploy" --> prod
Aspect Notebook Production
Who runs it You, by hand Other systems, automatically
Data A known historical file New requests, sometimes malformed
Failures You re-run the cell Must be detected, logged, and must not bring the service down
Environment Your machine's Reproducible and identical on every deployment (lesson 08-01)
Code Linear, exploratory Input validation, error handling, logging

The practical consequence: in production you do not deploy "the notebook", you deploy an artifact (the serialized pipeline) wrapped in service code (an API or a batch process) inside a reproducible environment (a container). Let's go piece by piece.

Serializing the pipeline with joblib

Serializing means converting the trained Python object — with all its learned parameters — into a file that can be stored, copied to a server and loaded back. In the scikit-learn ecosystem the standard is joblib, efficient with the large NumPy arrays that live inside models.

The critical point, which justifies all the work of module 3: you serialize the complete pipeline, not just the model. The Pipeline with its ColumnTransformer also stores what the preprocessors learned (the scaler's means, the encoder's categories). That way, production applies exactly the same transformations as training, and we eliminate at the root the whole family of "I trained on scaled data but I'm predicting on unscaled data" errors.

import joblib

# --- At the end of training (notebook or training script) ---
# pipeline_churn is the complete Pipeline from the course:
# ColumnTransformer (imputation + scaling + one-hot) -> optimized model
pipeline_churn.fit(X_train, y_train)
joblib.dump(pipeline_churn, "churn_model_v3.joblib")

# --- On the production server ---
pipeline = joblib.load("churn_model_v3.joblib")

# The loaded pipeline receives RAW data, like the original dataset;
# preprocessing happens inside:
import pandas as pd
customer = pd.DataFrame([{
    "recency_days": 45, "frequency_90d": 2, "avg_order_spend": 38.50,
    "months_tenure": 14, "delivery_incidents": 3, "city": "Valencia",
    "acquisition_channel": "app",
}])
prob_churn = pipeline.predict_proba(customer)[0, 1]
print(f"Churn probability: {prob_churn:.2%}")

Note that in production we use predict_proba rather than predict: as you saw in 06-04, the final decision is made by comparing the probability against the threshold chosen from business costs, and that threshold is best kept as explicit service configuration, not buried inside the model.

Versioning the artifact and its risks

Two important warnings before moving on:

1. Version the artifact the way you version code. A name like model.joblib that gets overwritten is a time bomb: when something breaks you will not know which model was serving. Minimum practices:

  • A name with version and date: churn_model_v3_2026-08-24.joblib.
  • Next to the artifact, a metadata file: training date, data range used, validation metrics, library versions (pip freeze), and the decision threshold.
  • Never delete the previous version when deploying the new one: it is your rollback plan (we will come back to this in 08-03, along with model registries like MLflow).

2. Two security and compatibility risks you must know about:

  • Deserializing is executing code. A joblib/pickle file can contain arbitrary code that runs when loaded. Golden rule: joblib.load only on files whose origin you control (your own training runs, your internal storage). Never load a model downloaded from an untrusted source.
  • Version sensitivity. A pipeline saved with scikit-learn 1.5 may fail — or worse, behave subtly differently without any error — when loaded with 1.2 or 1.7. That is why the pinned requirements.txt from the previous lesson always travels with the artifact: training and serving must use the same versions.

Serving patterns: batch vs. online

Before writing an API, an architecture decision: how will the predictions be consumed? There are two fundamental patterns.

Aspect Batch Online (real time)
When it predicts At scheduled times (e.g. every night) The moment the request arrives
On what All customers (or a large batch) One customer (or a few) per request
Latency required Hours: nobody cares if it takes 20 minutes Milliseconds: someone is waiting for the answer
Technical shape Scheduled script/job that writes to the database An always-on REST API
Operational complexity Low (a cron job) Medium-high (availability, scaling, monitoring)
MercaFresh example Nightly churn scoring of the whole customer base Deciding at checkout whether to show a retention offer

The natural case for MercaFresh churn is batch: the retention team plans weekly campaigns; it does not need the churn probability this millisecond, but a fresh table every morning. A nightly scoring job is simpler, cheaper and easier to operate:

# nightly_scoring.py — runs every night at 02:00 (cron / orchestrator)
import joblib
import pandas as pd
from datetime import date

pipeline = joblib.load("churn_model_v3_2026-08-24.joblib")
THRESHOLD = 0.42  # chosen from costs in 06-04

customers = pd.read_parquet("customer_features_today.parquet")  # updated raw data
customers["prob_churn"] = pipeline.predict_proba(
    customers.drop(columns=["customer_id"])
)[:, 1]
customers["retention_action"] = customers["prob_churn"] >= THRESHOLD

customers[["customer_id", "prob_churn", "retention_action"]].to_parquet(
    f"churn_scores_{date.today()}.parquet"
)

The online pattern is justified when the prediction depends on information that only exists in the moment (the contents of the current cart) or when another system needs an immediate answer. Since many systems end up needing both — and because it is the technically richer pattern — we now build the online version.

A REST API with FastAPI

FastAPI is the most widely used Python framework today for serving models: fast, with automatic data validation via pydantic and self-generated interactive documentation. The idea: the model is loaded once when the service starts, and each HTTP request to /predict returns a prediction.

# app.py — churn prediction service
from fastapi import FastAPI
from pydantic import BaseModel, Field
import joblib
import pandas as pd

app = FastAPI(title="MercaFresh Churn API")

# Runs ONCE at startup, not on every request (loading is slow)
pipeline = joblib.load("churn_model_v3_2026-08-24.joblib")
THRESHOLD = 0.42

class CustomerData(BaseModel):
    """Input schema: pydantic validates types and ranges automatically."""
    recency_days: int = Field(ge=0, description="Days since the last order")
    frequency_90d: int = Field(ge=0)
    avg_order_spend: float = Field(ge=0)
    months_tenure: int = Field(ge=0)
    delivery_incidents: int = Field(ge=0)
    city: str
    acquisition_channel: str

@app.post("/predict")
def predict_churn(data: CustomerData):
    X = pd.DataFrame([data.model_dump()])   # dict -> one-row DataFrame
    prob = float(pipeline.predict_proba(X)[0, 1])
    return {
        "prob_churn": round(prob, 4),
        "retention_action": prob >= THRESHOLD,
        "model": "churn_v3",
    }

Points worth understanding line by line:

  • CustomerData(BaseModel) defines the input contract. If a request arrives with recency_days: "lots" or without the city field, FastAPI rejects it with a clear 422 error before touching the model. In production, half of all incidents are malformed data; validation at the door turns them into explicit errors instead of nonsense predictions.
  • Field(ge=0) adds constraints (here: greater than or equal to zero) — business validation, not just type validation.
  • The response includes "model": "churn_v3": knowing which version answered each request is gold for monitoring (08-03).

You start it and try it like this:

pip install fastapi uvicorn
uvicorn app:app --host 0.0.0.0 --port 8000
# Automatic interactive documentation at http://localhost:8000/docs
# test_api.py — test client
import requests

response = requests.post("http://localhost:8000/predict", json={
    "recency_days": 45, "frequency_90d": 2, "avg_order_spend": 38.5,
    "months_tenure": 14, "delivery_incidents": 3,
    "city": "Valencia", "acquisition_channel": "app",
})
print(response.status_code)  # 200
print(response.json())       # {'prob_churn': 0.61..., 'retention_action': True, ...}

With this, any MercaFresh system — the web backend, the campaign system — can request predictions with an HTTP call, without knowing anything about Python or scikit-learn.

Packaging with Docker

One problem remains: the API works on your machine, with your Python and your versions. Docker solves "it worked on my machine" by packaging the application with its complete environment (base system, Python, exact libraries, code and artifact) into an immutable image. That image runs as a container identically on your laptop, on the MercaFresh server or in the cloud.

The recipe for the image is the Dockerfile:

# Dockerfile for the MercaFresh churn service

# 1. Base image: minimal Python 3.12 on Debian ("slim" variant)
FROM python:3.12-slim

# 2. Working directory inside the container
WORKDIR /app

# 3. Copy ONLY requirements first and install dependencies.
#    Docker caches layers: if the code changes but requirements do not,
#    the reinstall (the slowest step) is not repeated.
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# 4. Copy the service code and the model artifact
COPY app.py .
COPY churn_model_v3_2026-08-24.joblib .

# 5. Port the service listens on
EXPOSE 8000

# 6. Container startup command
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]

The reasoning behind these decisions:

  1. FROM python:3.12-slim: you start from an official image with Python already installed; slim trims what is unnecessary (smaller image, smaller attack surface).
  2. WORKDIR /app: all subsequent paths are relative to /app inside the container.
  3. The order COPY requirements.txt → RUN pip install → COPY app.py exploits the layer cache: in the usual cycle (you change code, not dependencies) rebuilding the image takes seconds.
  4. The model artifact travels inside the image: image and model are versioned together, and deploying a version means deploying a closed binary. (In large systems the model may be downloaded from a registry at startup; starting with it inside is simpler.)
  5. EXPOSE documents the port; 6. CMD defines what the container runs at startup.
docker build -t mercafresh/churn-api:v3 .     # build the image, tagged v3
docker run -p 8000:8000 mercafresh/churn-api:v3   # run it mapping the port
# The API responds exactly as before at http://localhost:8000/predict

Why containers, in one sentence per reason: reproducibility (the environment travels with the app), isolation (it interferes with nothing else on the server), portability (the same image runs anywhere) and scaling (more traffic? start more identical containers, by hand or with an orchestrator like Kubernetes — which is beyond this course).

Gradual deployment: shadow mode and A/B testing

Your v3 image is ready. Classic mistake: replacing the previous version in one shot. If v3 has a problem validation did not catch — and the real world always finds one — you will discover it with every customer affected. Professional deployment is gradual:

Phase 1 — Shadow mode. v3 is deployed alongside v2 and receives a copy of all requests, but its predictions are not used: they are only logged. For a week or two you compare: does it respond with the expected latency? Does it fail on any kind of input? Do its probabilities have a reasonable distribution compared to v2's? All the risk of a deployment, with zero impact on customers.

Phase 2 — Model A/B test. Once it clears shadow mode, v3 starts deciding for a random fraction of customers (e.g. 10%), and v2 for the rest. It is exactly the A/B test from lesson 02-04, with the model as the treatment: you compare the business metric (post-campaign retention rate, cost per retained customer) between groups, and demand statistical significance before concluding that v3 is better in the real world — not just on the test set. Important note: the offline metric (AUC, F1) and the business metric do not always move together; the A/B test is the final judge.

Phase 3 — Full rollout. v3 takes 100% of the traffic. v2 is not deleted: it stays ready for an immediate rollback if anything goes wrong.

graph LR
    A[v3 validated offline] --> B["Shadow mode<br/>predicts, does not decide"]
    B --> C["A/B test<br/>10 percent of traffic"]
    C --> D["Rollout 100 percent"]
    B -- problem --> X[Discard / fix]
    C -- no significant improvement --> X
    D -- incident --> R[Rollback to v2]

Go-live checklist

Before declaring the churn model deployed, go over this:

  • [ ] The artifact is the complete pipeline (preprocessing included) and accepts raw data.
  • [ ] Artifact versioned, with metadata (data, metrics, threshold, pip freeze) and the previous version kept.
  • [ ] Library versions identical between training and serving (pinned requirements, 08-01).
  • [ ] Serving pattern chosen deliberately (batch vs. online) based on the real need.
  • [ ] Input validation at the door (pydantic): types, ranges, required fields.
  • [ ] The decision threshold is explicit configuration, justified by costs (06-04).
  • [ ] Docker image built and tested locally; port and startup command correct.
  • [ ] Every response records which version of the model produced it.
  • [ ] Gradual deployment plan: shadow → A/B → rollout, with rollback ready.
  • [ ] Request and prediction logging enabled — it is the raw material for monitoring (08-03).

Common Mistakes and Tips

  • Serializing only the model and reimplementing the preprocessing "by hand" in production. Sooner or later the two implementations diverge and the predictions get silently corrupted. Always the complete pipeline.
  • Loading the model inside the endpoint. joblib.load on every request multiplies latency by a hundred. Load it once, at process startup.
  • Ignoring scikit-learn version warnings when loading an artifact. That "unpickling from a different version" warning is not noise: it is the exact symptom of the incompatibility risk. Align your versions.
  • Building an online API when a nightly batch was enough. It is the most expensive architecture mistake: you pay for 24/7 availability, scaling and on-call duty for a need that was a daily table.
  • Deploying at 100% with no shadow phase. The test set does not contain the surprises of real traffic (new fields, extreme values, load spikes). Shadow mode finds them for free.
  • Tip: add a /health endpoint to the API that returns the model version and an "ok". Deployment systems use it to know the service is alive, and it saves you the classic "which version is in production?".

Exercises

Exercise 1. For each MercaFresh scenario, decide batch or online and justify it in one sentence: (a) a daily list of at-risk customers emailed to the retention team; (b) deciding, while the customer is browsing, whether to show them a retention discount banner; (c) a monthly report on the evolution of the average churn risk by city.

Exercise 2. The example API receives this request: {"recency_days": -5, "frequency_90d": 2, "avg_order_spend": 38.5, "months_tenure": 14, "delivery_incidents": 3, "city": "Valencia", "acquisition_channel": "app"}. What happens, and why is that behavior preferable to the pipeline simply predicting anyway?

Exercise 3. Write (without executing it) the gradual deployment plan for version v4 of the churn model, stating: what is compared in shadow mode, which metric decides the A/B test, and what condition would trigger a rollback after the rollout.

Solutions

Solution 1. (a) Batch: the list is needed once a day; a scheduled nightly scoring job is enough and far simpler. (b) Online: the decision depends on the moment of the visit and someone (the frontend) is waiting for the answer in milliseconds; it requires the API. (c) Batch: it is a monthly aggregation over already-computed scores; it does not even need new predictions, just the results of the daily batch.

Solution 2. Pydantic rejects the request with a 422 error before it reaches the model, because recency_days carries the Field(ge=0) constraint and -5 arrives. This is preferable because a negative recency is impossible: there is almost certainly a bug in the calling system (or in the integration). If the pipeline predicted anyway, it would return a legitimate-looking number computed from corrupt data — a silent failure nobody would detect. An explicit error at the door makes the problem visible and attributable.

Solution 3. Sample plan: (1) Shadow, 2 weeks: v4 receives a copy of real traffic; compare against v3 the error rate, the latency (e.g. 95th percentile) and the distribution of predicted probabilities (if v4 systematically predicts higher/lower, investigate before proceeding). (2) A/B, 4-6 weeks at 10%: the deciding metric is a business one — for example, the retention rate of contacted customers (or cost per retained customer) in the v4 group versus the v3 group, with a significance test as in 02-04; if there is no significant improvement, v4 does not advance. (3) Rollout to 100% while keeping the v3 image; rollback if the service error rate spikes, latency exceeds the agreed threshold, or the prediction distribution/business metric deviates sharply from what was seen in the A/B test.

Conclusion

The MercaFresh churn model no longer lives in a notebook: it is a pipeline serialized and versioned with joblib, served every night in batch for the campaigns and available in real time behind a FastAPI API that validates every input, all packaged in a Docker image that runs identically on any machine, and deployed with a safety net — shadow mode, an A/B test like the one in 02-04 and a rollback at the ready. But putting the model in production is not the end of the work; it is the beginning of a new stage: the world changes, customers change, and a model trained on the past degrades in silence. Detecting that degradation and deciding when and how to retrain is the subject of the next lesson: model maintenance and monitoring.

◄ Previous
◄
Next ►
►

Machine Learning Course

Module 1: Introduction to Machine Learning

Module 2: Foundations of Statistics and Probability

Module 3: Data Preprocessing

Module 4: Supervised Machine Learning Algorithms

Module 5: Unsupervised Machine Learning Algorithms

Module 6: Model Evaluation and Validation

Module 7: Advanced Techniques and Optimization

Module 8: Model Implementation and Deployment

Module 9: Hands-On Projects

Module 10: Additional Resources

The Project | About Us | Contribute | Donations | License
© Copyright 2026. All rights reserved