Reproducible code and a reproducible environment: two of the lifecycle's three artifacts are under control. The one still missing changes at its own pace and without warning: the data. CineClick's churn_customers.csv is still a file the data engineer regenerates periodically, stomping on the previous one — problem #3 from the diagnosis ("the v3 dataset no longer exists"). In this lesson we bring in DVC (Data Version Control), the tool that extends Git's discipline to data: we'll see why Git alone won't do, what DVC's mental model is (tiny metafiles in Git, real data in a cache), and the full flow applied to our dataset, including the star moment: time-traveling between data versions with two commands.
Contents
- Why Git doesn't work for data
- DVC's mental model: metafiles + cache + remote
- Hands on:
dvc initanddvc add - The remote: sharing data with
dvc pushanddvc pull - Time travel: a new version of the dataset
- Who versions what: Git vs. DVC
Why Git doesn't work for data
The instinctive reaction would be git add data/raw/churn_customers.csv and move on. In the previous lesson we sent it to .gitignore without fully justifying it; let's do that now. Git is a magnificent tool for what it was designed for — text, small, with meaningful diffs — and datasets fail all three conditions:
- Size: Git stores the repository's complete history, and every clone carries all of it. Our toy CSV weighs a few MB, but the real extract of a streaming platform is measured in GB, and with a monthly snapshot the repository bloats without brakes: cloning the project would end up downloading years of dead datasets. Git servers (GitHub included) also impose hard file-size limits.
- Binaries and useless diffs: Git is efficient because it stores differences between versions of text. With a large CSV that gets regenerated whole (or a Parquet, or images), each version is in practice a complete new blob, and a
git diffof two 2-million-row extracts is of no use to any human. - Different rhythms: as we saw in lesson 01-02, code and data change at different paces and for different reasons. Mixing "refactored features.py" with "the July extract arrived" in the same history muddies both.
What we do want to keep from Git is its semantics: being able to say "the model of release 1.2 was trained with these exact data", going back to any version, and sharing with the team. The solution is not putting the data in Git, but putting in Git a verifiable reference to the data.
DVC's mental model: metafiles + cache + remote
DVC (Data Version Control) is an open-source command-line tool that works on top of Git — it doesn't replace it. Its trick is a clean separation:
- In Git live some tiny text metafiles (
.dvcextension) describing each versioned piece of data: essentially its MD5 hash (a fingerprint of the content), its size, and its path. They take up bytes and diff beautifully. - Outside Git, the real files live in a local cache (
.dvc/cache/, organized by hash) and, for sharing, in a remote: any storage — a network folder, S3, Google Cloud Storage, Azure Blob... - In your working directory,
data/raw/churn_customers.csvis (depending on the system) a link or a copy materialized from the cache: you use it normally, pandas never notices.
flowchart LR
subgraph Git["Git repository (text, lightweight)"]
M["churn_customers.csv.dvc<br/>md5: 3f2a..., size: 2.1 MB"]
end
subgraph Local["Local machine"]
W["data/raw/churn_customers.csv<br/>(working file)"]
C[".dvc/cache/<br/>(contents by hash)"]
end
R["Remote<br/>(shared folder / S3 / GCS)"]
M -- "dvc checkout" --> W
W -- "dvc add" --> C
C -- "dvc push" --> R
R -- "dvc pull" --> CThe elegant consequence: every Git commit pins, via the metafile's hash, an exact version of the data. Code and data end up tied together in the same history without the data ever passing through Git. Switching data versions will be as simple as switching commits and asking DVC to materialize what that commit references.
Hands on: dvc init and dvc add
DVC is just another Python package, so it enters the project through the door we built in 02-02: add dvc>=3.50 to the dev extra of pyproject.toml, regenerate the lock with pip-compile, and sync. With the environment activated, we initialize:
dvc init git status # new: .dvc/config .dvc/.gitignore .dvcignore git commit -m "Initialize DVC in the project"
dvc init creates the .dvc/ directory (configuration and, later on, the cache) and a few auxiliary files, all meant to be committed. DVC always leaves breadcrumbs in Git: that's the whole point.
Now we put the dataset under control. First we remove data/ from the general .gitignore we wrote in 02-01 (DVC will manage finer-grained ignores on its own) and run:
This does three things: it computes the file's hash, stores a copy in the cache (.dvc/cache/), and creates the metafile data/raw/churn_customers.csv.dvc, besides adding the real CSV to a local .gitignore so Git doesn't see it. The metafile is this humble:
# data/raw/churn_customers.csv.dvc (generated by DVC; never edited by hand) outs: - md5: 3f2a9c81d4be7a06c1e5f9a2b8d47e11 size: 2184730 hash: md5 path: churn_customers.csv
Read it like a luggage-check receipt: "there exists a file named churn_customers.csv, 2.1 MB, whose content has this MD5 fingerprint". With that hash, DVC can locate the exact content in any cache or remote. Change a single byte of the CSV and the hash changes completely — that's why it works as a version identifier. What goes to Git is the receipt:
git add data/raw/churn_customers.csv.dvc data/raw/.gitignore git commit -m "Version the churn dataset (June extract) with DVC"
Notice the commit message: describing which version of the data this is ("June extract") turns the git log into the dataset history we never had.
The remote: sharing data with dvc push and dvc pull
The cache is local: if Laura versions the dataset on her machine, Marc still can't get it. The remote is to data what GitHub is to code: the exchange point. DVC supports S3, GCS, Azure, SSH, local or network folders... and it's configured once:
# For the course: a local directory simulating the shared storage. dvc remote add -d storage /srv/dvc-remote-cineclick git add .dvc/config git commit -m "Configure the data remote"
The -d flag marks it as the default remote, and the configuration lands in .dvc/config — text, committed, shared. In CineClick's real life this would point to a bucket (dvc remote add -d storage s3://cineclick-mlops-data would suffice, plus the credentials, which never go to Git — DVC stores them separately with dvc remote modify --local). For following the course, the local folder simulates the bucket perfectly: the commands are identical.
And the exchange:
On the other side, Marc:
git clone <repo> && cd cineclick-churn # ... creates the environment as in 02-02 ... dvc pull # reads the .dvc files of the current commit and downloads those exact contents
dvc pull looks at which hashes the metafiles present in the current commit reference, finds them in the remote, downloads them to his cache, and materializes the working files. Marc now has exactly the same bytes Laura trained with — not "whatever CSV happened to be in the shared folder that day". The git pull + dvc pull gesture becomes the project's new "catching up".
Time travel: a new version of the dataset
Here comes the moment that justifies all the machinery. It's July, and the data engineer delivers a new extract with two novelties: an additional month of customers and, mind you, the weekly_hours column now arrives corrected at the source (they fixed the negative-values bug Laura used to filter by hand). The file has the same name and replaces the previous one in data/raw/. Before, this was an irreversible loss; now:
# The new extract overwrites the working file dvc status # data/raw/churn_customers.csv.dvc: changed <- DVC detects the content no longer matches the hash dvc add data/raw/churn_customers.csv # new hash, new cache entry git add data/raw/churn_customers.csv.dvc git commit -m "July data: new month and weekly_hours fixed at the source" dvc push
The metafile now contains the July extract's hash, and the commit writes it into history. Both versions coexist in the cache and in the remote, each under its own hash. And here it is, the time travel — suppose the model trained on July behaves oddly and we want to reproduce June's training:
git log --oneline -- data/raw/churn_customers.csv.dvc # locate the June commit git checkout a1b2c3d # Git restores June's METAFILE... dvc checkout # ...and DVC materializes June's DATA from the cache
git checkout only moves the files Git controls — that is, the .dvc with the old hash. dvc checkout reads that hash and restores the corresponding content into data/raw/. Two commands, and the working directory is identical, code and data, to the day of June's training. To return to the present: git checkout master && dvc checkout. If the content weren't in the local cache (a fresh machine), dvc pull would fetch it from the remote.
This is exactly what module 1's diagnosis was missing in problem #3: "the current model can't be reproduced and trainings can't be compared: the v3 dataset no longer exists". Now every dataset exists forever, addressable by commit, and the question "which data was this model trained with?" finally has a mechanical answer: the data of the commit it came from.
Who versions what: Git vs. DVC
The pair splits the work like this:
| Aspect | Git | DVC |
|---|---|---|
| What it versions | Code, configuration, tests, docs, .dvc metafiles |
Data and large/binary files (datasets, and soon models) |
| Where it stores content | In the repository itself (full history in every clone) | Local cache + remote (you download only what you need) |
| Unit of change | Commit (line-by-line diff) | Hash of the file's full content |
| How it's shared | git push / git pull (GitHub, etc.) |
dvc push / dvc pull (S3, GCS, network folder...) |
| How you time-travel | git checkout <commit> |
dvc checkout (after Git's) |
| Comfortable size | KB–MB, text | MB–TB, any format |
And the mnemonic rule that governs the split: Git versions the recipes; DVC, the ingredients. Every Git commit is a complete snapshot of the project because it contains the code, the configuration, the environment lock... and the receipts that pin the data.
Common Mistakes and Tips
- Mistake: committing the metafile without doing
dvc push(or the other way around). If Marc doesgit pulland the hash the.dvcreferences isn't in the remote, hisdvc pullwill fail with an eloquent "file is missing". The two pushes go together, always; internalize it as a single gesture (in module 5, CI will verify it for us). - Mistake: editing a
.dvcfile by hand. The hash is a computed fingerprint; writing a different value doesn't change the data, it only breaks the correspondence. DVC writes the.dvcfiles (dvc add); you only commit them. - Mistake: doing
git checkoutto an old commit and forgetting thedvc checkout. You're left in a treacherously inconsistent state: June's code with July's data in the directory. The pair goes together:git checkout+dvc checkout, there and back. (There'sdvc install, which hooks into Git to automate the second step; enable it if you catch yourself forgetting.) - Mistake: versioning with DVC files that belong in Git. A 20-line
config.yamlin DVC is counterproductive: you lose the diff, which there is genuinely valuable. The criterion is size + binariness + rate of change, not "everything in data/ by definition". - Tip: a new dataset deserves an informative commit message. Not "update data"; yes "July data: new month and weekly_hours fixed at the source". The history of
git log -- data/raw/*.dvcis the documentation of your data's evolution — for free.
Exercises
Exercise 1
Put this shuffled sequence of commands in the right order to incorporate a new extract of the dataset and make it available to the whole team, and explain what each step does: git commit -m "August data", dvc push, dvc add data/raw/churn_customers.csv, git add data/raw/churn_customers.csv.dvc, (the new file overwrites the old one in data/raw/).
Exercise 2
Laura trained the "good model" three weeks ago, at commit f4e5d6a. Today the working dataset is July's. Write the exact sequence of commands to: (1) leave the machine exactly as it was on the day of that training (code and data); (2) verify that the materialized data is in sync with the metafile; (3) return to the current state. State which additional command you would need if you did this on a freshly cloned machine.
Exercise 3
A colleague proposes: "instead of DVC, let's upload each extract to the shared folder as churn_customers_2026_06.csv, churn_customers_2026_07.csv... and have config.yaml point to whichever one applies". It's an improvement over overwriting, no doubt. Point out at least three guarantees DVC provides that this date-suffix scheme does not.
Solutions
Solution 1: (0) the new extract overwrites the working file — the starting point; (1) dvc add data/raw/churn_customers.csv — computes the new hash, stores the content in the cache, and updates the metafile; (2) git add data/raw/churn_customers.csv.dvc — stages the updated receipt for commit; (3) git commit -m "August data" — pins the new data version into history alongside the current code; (4) dvc push — uploads the content to the remote so any colleague's dvc pull works. The order of 3 and 4 can be swapped, but neither may be skipped.
Solution 2: (1) git checkout f4e5d6a followed by dvc checkout — Git restores the code and the metafile from that day; DVC reads its hash and materializes that dataset from the cache; (2) dvc status — it should report everything is up to date (no discrepancy between working files and metafiles); (3) git checkout master and dvc checkout again. On a freshly cloned machine the cache is empty, so between the checkout and the work you would need dvc pull to download from the remote the content the hash references.
Solution 3: among others: (1) integrity verification — the MD5 hash guarantees the file is exactly the original; with date suffixes, nothing stops someone from editing or re-uploading _2026_06.csv without anyone ever noticing; (2) automatic code↔data linkage — with DVC, each commit pins which data belongs to it; with suffixes, the link depends on a human updating config.yaml in the right commit and on nobody touching it afterwards; (3) deduplication and efficient transport — the hash-based cache never stores the same content twice and dvc pull fetches only what's missing, while the folder of dated files grows unchecked and gets copied by hand; (4, as a bonus) a single flow for any size and backend — the day the CSV becomes 40 GB on S3, the commands don't change.
Conclusion
The third artifact has fallen into line: with dvc init, dvc add, and a configured remote, every version of churn_customers.csv is pinned by its hash in a committed metafile, shared via dvc push/dvc pull, and recoverable forever with the git checkout + dvc checkout pair — problem #3 from the diagnosis is solved. The picture now is: versioned code, versioned environment, versioned data... but the process that ties them together is still artisanal: run the cleanup, then the features, then the training, then the evaluation, by hand and in the right order, remembering what needs redoing when something changes. That manual chaining is fragile, and DVC has a second face designed precisely to eliminate it: pipelines declared in dvc.yaml, where each stage knows its dependencies and dvc repro re-runs only what's necessary. It's the piece that closes the module, and with it CineClick will touch full reproducibility for the first time: we see it in the next lesson.
MLOps Course
Module 1: MLOps Fundamentals
- What MLOps is and why models die in the notebook
- The lifecycle of an ML model in production
- MLOps maturity levels and team roles
- The course project: from notebook to production
Module 2: From Notebook to Reproducible Code
- Structuring an ML project: from notebook to package
- Reproducible environments and dependency management
- Data versioning with DVC
- Reproducible training pipelines
Module 3: Experiments and Model Registry
- Experiment tracking with MLflow
- Model registry: versioning and promoting models
- Feature stores: when and what for
Module 4: Serving Models in Production
- Deployment patterns: batch, online and streaming
- A prediction service with FastAPI
- Packaging with Docker
- Scaling and deployment: Kubernetes and serverless
- Inference optimization: latency and cost
Module 5: Automation: CI/CD and Orchestration
- CI for ML: testing code, data and models
- CD: automating model deployment
- Orchestrating ML pipelines
- Release strategies: shadow, canary and A/B
