The Project | About Us | Contribute | Donations | License
HOME My courses Completed
◄ Previous
◄

Structuring an ML project: from notebook to package

Next ►
►

We promised to roll up our sleeves, and this is where the work begins. In this lesson we'll take Laura's notebook — churn_final_v3_FINAL.ipynb, exactly as we left it in module 1 — and turn it into a structured Python project: a repository with an installable package, modules with clear responsibilities, external configuration, and a first test. It's the most important transformation of the entire course, because everything that comes after (versioning data, chaining pipelines, serving the model, automating) assumes the code no longer lives in notebook cells but in modules that can be imported, tested, and run on any machine. And along the way, as we refactor the evaluation, we'll expose the lie of the 0.87 once and for all.

Contents

  1. Why the notebook doesn't scale (and what it IS good for)
  2. The design of the cineclick-churn repository
  3. Step-by-step refactoring: data, features, training, and evaluation
  4. External configuration: config.yaml instead of hardcoded values
  5. pyproject.toml: the project as an installable package
  6. A first test with pytest (a preview)

Why the notebook doesn't scale (and what it IS good for)

Let's start by being fair: the notebook is not the enemy. It's the best tool in existence for the exploration phase — iterating fast, visualizing, testing hypotheses, documenting findings alongside the code — and that phase doesn't disappear with MLOps. Laura will keep using notebooks, and so will you. The problem is using the notebook as a unit of production, because in that role it has structural defects that no patch can fix:

  • Hidden state and out-of-order execution: cells can run in any order, and a variable may come from a cell that no longer exists. A notebook's result depends on its execution history, not just its code. In production that's unacceptable: the same code must produce the same result.
  • It's not importable: you can't do from notebook import clean_data from a FastAPI service or from a test. Everything that lives in a notebook is doomed to be copied and pasted — and copies diverge (problem #9 from the diagnosis: to predict in production, the preprocessing would have to be reimplemented).
  • It doesn't test well: pytest discovers and runs functions in .py modules. A notebook doesn't expose testable functions; it exposes cells.
  • The diffs are unreadable: an .ipynb is JSON with outputs, execution counters, and base64-embedded images. A git diff between two versions is noise; reviewing a change in a pull request, impossible.
  • Everything in a single block: loading, cleanup, features, training, and evaluation all mixed together means you can't re-run just one part (something that will be gold in lesson 02-04) or reason about each piece separately.

The healthy division of labor is this:

Tool What it IS for What it is NOT for
Notebook Exploring data, prototyping features, visualizing, one-off analyses, communicating findings Production code, reusable logic, anything that must run without humans
Python package (.py modules) Data/features/training/evaluation logic, everything testable and automatable Fast interactive exploration (possible, but slower to iterate)

The practical rule: the notebook discovers; the package industrializes. When a fragment of the notebook proves its worth, it "graduates" to a package module, and the notebook switches to importing it instead of holding its own copy.

The design of the cineclick-churn repository

We create a Git repository called cineclick-churn with this structure (read it slowly; every directory has a reason to exist):

cineclick-churn/
├── src/
│   └── cineclick_churn/        # The Python package (production code)
│       ├── __init__.py
│       ├── data.py             # Data loading and cleanup
│       ├── features.py         # Feature building (one-hot, etc.)
│       ├── train.py            # Model training
│       └── evaluate.py         # Evaluation with honest metrics
├── tests/                      # Automated tests (pytest)
│   └── test_features.py
├── data/
│   ├── raw/                    # Original data, UNTOUCHABLE (churn_customers.csv)
│   └── processed/              # Derived data, regenerable from raw/
├── models/                     # Serialized models (output artifacts)
├── notebooks/                  # Exploration (Laura's notebook moves here)
│   └── churn_exploration.ipynb
├── configs/
│   └── config.yaml             # Externalized paths and parameters
├── pyproject.toml              # Package definition and metadata
└── README.md

Design decisions worth understanding:

  • The src/ layout: the package lives inside src/ instead of at the root. It's the recommended modern convention because it forces you to install the package in order to use it (with pip install -e ., as we'll see below): that way, the tests and the production code import exactly the same package that will be deployed, not a directory that happened to be on the path.
  • One module per lifecycle stage: data.py → features.py → train.py → evaluate.py mirrors the stages we saw in lesson 01-02. That's no accident: in lesson 02-04 each module will become a pipeline step with explicit dependencies.
  • data/raw/ is sacred: the original data is never modified or overwritten; everything derived goes to data/processed/ and must be regenerable. (How to version what lives in data/ is the subject of lesson 02-03 — for now we'll add data/ and models/ to .gitignore, because Git is not the place for them.)
  • notebooks/ still exists: exploration doesn't die, it gets tidied up. Laura's notebook is renamed to something dignified (churn_exploration.ipynb) and kept as a record of the initial discovery.
  • configs/ separates the what from the how much: code defines the logic; configuration files, the values. We develop this in section 4.

Step-by-step refactoring: data, features, training, and evaluation

Now we carve up the notebook. The strategy: identify each responsibility, extract it into a function with explicit inputs and outputs, and place it in its module. No function will use absolute paths or magic values: everything arrives as a parameter.

Step 1 — data.py: loading and cleanup

From the notebook we extract the loading (pd.read_csv with the path from Laura's desktop) and the cleanup (dropna, negative-hours filter, dropping the id):

# src/cineclick_churn/data.py
"""Loading and cleanup of the CineClick churn dataset."""
import pandas as pd


def load_data(path: str) -> pd.DataFrame:
    """Reads the customer CSV. The path arrives as a parameter: no fixed paths."""
    return pd.read_csv(path)


def clean_data(df: pd.DataFrame) -> pd.DataFrame:
    """Applies the cleanup rules agreed with Laura.

    - Drops rows with null values.
    - Drops rows with negative viewing hours (known logging bug).
    - Drops customer_id: it's an identifier with no predictive value.
    """
    df = df.dropna()
    df = df[df["weekly_hours"] >= 0]
    df = df.drop(columns=["customer_id"])
    return df.reset_index(drop=True)

Notice three changes compared to the notebook: the path is a parameter (problem #1 begins to die here), each function has one responsibility and a docstring documenting the decisions (they used to be stray comments), and reset_index leaves the DataFrame free of gap-riddled indices after filtering — a classic generator of silent bugs.

Step 2 — features.py: feature building

The one-hot encoding with get_dummies graduates to its own module, with a crucial improvement for the future: we fix explicitly which columns are categorical and which one is the label, instead of letting the code "guess":

# src/cineclick_churn/features.py
"""Feature transformation for the churn model."""
import pandas as pd

CATEGORICAL_COLUMNS = ["plan", "payment_method"]
LABEL_COLUMN = "churned"


def build_features(df: pd.DataFrame) -> pd.DataFrame:
    """Converts the categoricals into 0/1 columns (one-hot encoding).

    'plan' produces plan_basic / plan_standard / plan_premium;
    'payment_method' produces payment_method_card / _direct_debit / _paypal.
    """
    return pd.get_dummies(df, columns=CATEGORICAL_COLUMNS, dtype=int)


def split_features_label(df: pd.DataFrame) -> tuple[pd.DataFrame, pd.Series]:
    """Separates X (features) from y (the 'churned' label)."""
    X = df.drop(columns=[LABEL_COLUMN])
    y = df[LABEL_COLUMN]
    return X, y

Why the uppercase constants at the top? Because the prediction service in module 4 will import this very module to transform new customers exactly the same way as in training. A single implementation of the preprocessing, shared: problem #9 (non-reusable preprocessing) is structurally on track.

Step 3 — train.py: training with randomness under control

Here we attack problem #2. In the notebook, neither train_test_split nor RandomForestClassifier carried a seed, so every run was unrepeatable. Now the seed is a required parameter:

# src/cineclick_churn/train.py
"""Training of the churn model."""
import pickle
from pathlib import Path

import pandas as pd
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split


def split_data(X, y, test_size: float, random_state: int):
    """Splits into train/test reproducibly and with stratification.

    - random_state pins the randomness: the same seed produces the same split.
    - stratify=y guarantees train and test keep the ~15% churn rate;
      without it, with imbalanced classes the test set could end up skewed.
    """
    return train_test_split(
        X, y, test_size=test_size, random_state=random_state, stratify=y
    )


def train_model(X_train, y_train, n_estimators: int, random_state: int):
    """Trains the RandomForest with a fixed seed: same data -> same forest."""
    model = RandomForestClassifier(n_estimators=n_estimators, random_state=random_state)
    model.fit(X_train, y_train)
    return model


def save_model(model, path: str) -> None:
    """Serializes the model, creating the target directory if it doesn't exist."""
    Path(path).parent.mkdir(parents=True, exist_ok=True)
    with open(path, "wb") as f:
        pickle.dump(model, f)

Two seeds, one shared value: random_state controls both the split and the forest's internal randomness (which rows and columns each tree samples). With this, running the training twice on the same data produces exactly the same model and the same metrics. We also add stratify=y, an improvement the notebook didn't have: with only 15% positives, a non-stratified split can leave the test set with a churn percentage different from the real one and distort the evaluation.

Step 4 — evaluate.py: metrics that don't lie

The moment of truth. The notebook celebrated accuracy = 0.87, but remember problem #5: with 85% of customers who don't churn, a trivial model that always predicts "stays" already scores 0.85. We introduce the metrics that look at the class that matters (the leavers):

  • Precision (on the positive class): of the customers the model flags as "going to churn", what fraction actually churns? Low precision = the retention team spends discounts on customers who weren't leaving.
  • Recall (on the positive class): of the customers who actually churn, what fraction does the model detect? Low recall = cancellations happen without anyone seeing them coming. It's the metric that hurts the business.
  • F1: the harmonic mean of the two above; useful as a single-number summary when models need to be compared.
# src/cineclick_churn/evaluate.py
"""Honest evaluation of the churn model."""
from sklearn.metrics import accuracy_score, f1_score, precision_score, recall_score


def evaluate_model(model, X_test, y_test) -> dict:
    """Computes the metrics on the test set.

    precision/recall/f1 are computed on the positive class (churned=1),
    which is the minority class and the one the business cares about.
    """
    predictions = model.predict(X_test)
    return {
        "accuracy": round(accuracy_score(y_test, predictions), 4),
        "precision": round(precision_score(y_test, predictions), 4),
        "recall": round(recall_score(y_test, predictions), 4),
        "f1": round(f1_score(y_test, predictions), 4),
    }

Running the full flow with random_state=42 and n_estimators=100 on the dataset, we get (and now we'll get the very same thing every time):

{"accuracy": 0.86, "precision": 0.61, "recall": 0.43, "f1": 0.5}

Let's read this without anesthesia: accuracy still hovers around what Laura was seeing (the original 0.87 danced with chance), but the model only detects 43% of the customers who churn, and of those it flags, it's right 61% of the time. It's not a disaster — detecting 4 out of every 10 cancellations at that precision already enables profitable campaigns — but it's a radically different picture from "0.87, better than v2!". This is the honest baseline against which we'll measure every future improvement. Improving these numbers (trying hyperparameters, thresholds, other algorithms) will require comparing many experiments comfortably: that's module 3's job.

External configuration: config.yaml instead of hardcoded values

There are no absolute paths in the code anymore, but the values (relative paths, test_size, n_estimators, the seed) have to live somewhere. If we write them into the script that orchestrates everything, we're back to mixing logic and parameters. The standard solution: a readable configuration file, versioned in Git alongside the code.

# configs/config.yaml
data:
  raw_path: data/raw/churn_customers.csv
  processed_path: data/processed/customers_clean.csv

model:
  path: models/churn_model.pkl
  n_estimators: 100
  random_state: 42

evaluation:
  test_size: 0.2

Notice that all the paths are relative to the repository root: the project works the same on Laura's laptop, on yours, or on a server. Loading it from Python is trivial with PyYAML:

# Fragment of a training script (e.g. at the end of train.py)
import yaml

from cineclick_churn import data, evaluate, features, train


def main():
    with open("configs/config.yaml", encoding="utf-8") as f:
        config = yaml.safe_load(f)  # safe_load: never bare yaml.load

    df = data.clean_data(data.load_data(config["data"]["raw_path"]))
    df = features.build_features(df)
    X, y = features.split_features_label(df)

    X_train, X_test, y_train, y_test = train.split_data(
        X, y, config["evaluation"]["test_size"], config["model"]["random_state"]
    )
    model = train.train_model(
        X_train, y_train, config["model"]["n_estimators"], config["model"]["random_state"]
    )
    train.save_model(model, config["model"]["path"])
    print(evaluate.evaluate_model(model, X_test, y_test))


if __name__ == "__main__":
    main()

The if __name__ == "__main__": block lets the module run as a script (python -m cineclick_churn.train) as well as being imported: it will come in very handy in lesson 02-04, when each stage becomes a pipeline command. Changing a hyperparameter is no longer editing code: it's editing a YAML, and that change gets recorded in Git's history with its commit and its why. (In 02-04 we'll refine this idea by splitting the hyperparameters out into a params.yaml that DVC understands natively.)

pyproject.toml: the project as an installable package

The glue is still missing: making from cineclick_churn import features work from anywhere (tests, notebooks, the future service) without sys.path tricks. For that, the project declares itself as a package in pyproject.toml, the standard configuration file for Python projects:

# pyproject.toml
[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"

[project]
name = "cineclick-churn"
version = "0.1.0"
description = "CineClick churn prediction model"
requires-python = ">=3.11"

[tool.setuptools.packages.find]
where = ["src"]

And it gets installed in editable mode:

pip install -e .

The -e (editable) means pip doesn't copy the code into its packages directory; instead it creates a link to src/: every change you make to the modules is visible instantly, without reinstalling. It's the standard way of working during development. From now on, Laura's exploration notebook can start with from cineclick_churn.features import build_features — the notebook consumes the package, never the other way around. (This pyproject.toml is deliberately incomplete: declaring dependencies — pandas, scikit-learn, PyYAML and company — is the central topic of the next lesson.)

A first test with pytest (a preview)

Problem #6 in the diagnosis was "no tests of any kind". We're not going to solve it all today — serious ML tests (of data, of model behavior, in CI) are the subject of lesson 05-01 — but now that we have importable functions, writing the first one takes two minutes and changes the project's culture:

# tests/test_features.py
import pandas as pd

from cineclick_churn.features import build_features


def test_one_hot_creates_plan_columns():
    df = pd.DataFrame({
        "tenure_months": [12, 3],
        "weekly_hours": [5.0, 1.5],
        "support_tickets": [0, 2],
        "plan": ["basic", "premium"],
        "payment_method": ["card", "direct_debit"],
        "active_discount": [0, 1],
        "churned": [0, 1],
    })

    result = build_features(df)

    assert "plan_basic" in result.columns
    assert "plan_premium" in result.columns
    assert "plan" not in result.columns          # the original disappears
    assert result["plan_basic"].tolist() == [1, 0]

It runs with pytest from the repo root. The test builds a tiny two-row fictional DataFrame, applies the real transformation (the same one production will use), and checks that the one-hot does what it promises. If tomorrow someone "improves" features.py and breaks the encoding, the test will fail before the error reaches a model. Small, yes; but it's the seed of the safety net we'll weave in module 5.

Common Mistakes and Tips

  • Mistake: refactoring and "improving" at the same time. The temptation to seize the moment and change the algorithm or add features is enormous. Resist it: first reproduce the notebook's behavior in the new structure (same cleanup, same model), verify it works, and then improve. If you change structure and logic at once and something breaks, you won't know which of the two it was. (The only "improvement" we've slipped in — stratify and the new metrics — affects the measurement, not the model.)
  • Mistake: forgetting random_state in one of the two places. Pinning it in the split but not in the RandomForestClassifier (or the other way around) gives a false sense of reproducibility: the metrics will keep dancing. Rule: look for every parameter named random_state or seed in the functions you use and decide its value consciously.
  • Mistake: fragile relative imports or sys.path.append. If you find yourself writing sys.path.append("../src") in a test or a notebook, that's not the solution: pip install -e . is. The src layout plus editable install exists precisely to eliminate that class of hacks.
  • Mistake: leaving config.yaml out of Git. Configuration is code for all intents and purposes: without it, the project can't run. What does NOT go into Git is data and models (we'll handle that in 02-03), not the configuration.
  • Tip: refactor with Laura, not against Laura. She knows why the negative hours are filtered out or why a column was dropped. Every cleanup decision rescued from the notebook must end up documented in a docstring; tacit knowledge is technical debt.

Exercises

Exercise 1

Without writing code: classify each of these fragments of the original notebook by the package module it belongs to (data.py, features.py, train.py, evaluate.py) or whether it belongs in configuration (config.yaml): (a) df = df[df["weekly_hours"] >= 0]; (b) test_size=0.2; (c) pd.get_dummies(df, columns=["plan", "payment_method"]); (d) accuracy_score(y_test, predictions); (e) "C:/Users/laura/Desktop/datos/churn_customers.csv"; (f) RandomForestClassifier(n_estimators=100).

Exercise 2

Write a second test for tests/test_features.py that verifies split_features_label works correctly: that X does not contain the churned column, that y contains exactly the values of that column, and that X keeps the same number of rows as the input DataFrame. Reuse the mini-DataFrame from the existing test (ideally, extract it into a pytest fixture to avoid duplicating it).

Exercise 3

The retention team asks you: "with the new metrics (precision 0.61, recall 0.43), if 1,000 customers are actually going to churn in a given month, how many will the model detect, and roughly how many false positives will it generate?". Reason out the answer using those two metrics and explain which of the two types of error is more expensive for CineClick, knowing that the retention campaign consists of a 20% discount for 3 months.

Solutions

Solution 1: (a) data.py — it's a cleanup rule; (b) config.yaml — it's a parameter, not logic (it's read by whoever does the split in train.py); (c) features.py — feature transformation; (d) evaluate.py — metric computation; (e) config.yaml — it's a path, and it must also become relative (data/raw/churn_customers.csv); (f) the call belongs to train.py, but the value n_estimators=100 belongs in config.yaml. The general guideline: logic goes to the module of its stage; concrete values, to configuration.

Solution 2:

import pandas as pd
import pytest

from cineclick_churn.features import split_features_label


@pytest.fixture
def sample_df():
    return pd.DataFrame({
        "tenure_months": [12, 3],
        "weekly_hours": [5.0, 1.5],
        "support_tickets": [0, 2],
        "plan": ["basic", "premium"],
        "payment_method": ["card", "direct_debit"],
        "active_discount": [0, 1],
        "churned": [0, 1],
    })


def test_split_features_label(sample_df):
    X, y = split_features_label(sample_df)

    assert "churned" not in X.columns        # the label doesn't leak into the features
    assert y.tolist() == [0, 1]              # y contains exactly the label
    assert len(X) == len(sample_df)          # no rows are lost

The fixture (@pytest.fixture) is pytest's idiomatic way of sharing test data between tests: it's declared once, and any test receives it as a parameter.

Solution 3: with recall 0.43, of the 1,000 customers who will churn, the model detects ~430. With precision 0.61, those 430 hits are 61% of all the alerts the model emits, so it emits ~430 / 0.61 ≈ 705 alerts in total, of which ~275 are false positives (customers flagged as at risk who weren't leaving). The cost of each error: a false positive costs an unnecessary discount (20% × 3 months of one subscription fee, a few euros); a false negative costs an entire customer (all their future value, which module 1 put at 5-7 times the avoided acquisition cost). For CineClick, the false negative is far more expensive, which suggests that in the future it will pay to push recall up even if precision drops somewhat — exactly the kind of experiment we'll log in an orderly way in module 3.

Conclusion

Laura's notebook is no longer a monolith: it's a cineclick-churn repository with an installable package (src/cineclick_churn/ with data.py, features.py, train.py, and evaluate.py), external configuration in configs/config.yaml with relative paths, fixed seeds (random_state=42 in both split and model, with stratification), an honest evaluation that replaces the misleading accuracy with precision 0.61 / recall 0.43 / F1 0.50 as the baseline, and a first test watching over the preprocessing. Problems #1 (personal paths), #2 (uncontrolled randomness), and #9 (non-reusable preprocessing) from the diagnosis are solved or on track. But there's a crack this work doesn't seal: our code is reproducible on our environment, and nothing guarantees your colleague has the same versions of pandas or scikit-learn as you — problem #8 is alive and kicking. In the next lesson we attack it head-on: virtual environments, dependency declaration, and lock files, so that "works on my machine" stops being a threat.

◄ Previous
◄
Next ►
►
The Project | About Us | Contribute | Donations | License
© Copyright 2026. All rights reserved