Reproducible code and a reproducible environment: two of the lifecycle's three artifacts are under control. The one still missing changes at its own pace and without warning: the data. CineClick's churn_customers.csv is still a file the data engineer regenerates periodically, stomping on the previous one — problem #3 from the diagnosis ("the v3 dataset no longer exists"). In this lesson we bring in DVC (Data Version Control), the tool that extends Git's discipline to data: we'll see why Git alone won't do, what DVC's mental model is (tiny metafiles in Git, real data in a cache), and the full flow applied to our dataset, including the star moment: time-traveling between data versions with two commands.

Contents

  1. Why Git doesn't work for data
  2. DVC's mental model: metafiles + cache + remote
  3. Hands on: dvc init and dvc add
  4. The remote: sharing data with dvc push and dvc pull
  5. Time travel: a new version of the dataset
  6. Who versions what: Git vs. DVC

Why Git doesn't work for data

The instinctive reaction would be git add data/raw/churn_customers.csv and move on. In the previous lesson we sent it to .gitignore without fully justifying it; let's do that now. Git is a magnificent tool for what it was designed for — text, small, with meaningful diffs — and datasets fail all three conditions:

  • Size: Git stores the repository's complete history, and every clone carries all of it. Our toy CSV weighs a few MB, but the real extract of a streaming platform is measured in GB, and with a monthly snapshot the repository bloats without brakes: cloning the project would end up downloading years of dead datasets. Git servers (GitHub included) also impose hard file-size limits.
  • Binaries and useless diffs: Git is efficient because it stores differences between versions of text. With a large CSV that gets regenerated whole (or a Parquet, or images), each version is in practice a complete new blob, and a git diff of two 2-million-row extracts is of no use to any human.
  • Different rhythms: as we saw in lesson 01-02, code and data change at different paces and for different reasons. Mixing "refactored features.py" with "the July extract arrived" in the same history muddies both.

What we do want to keep from Git is its semantics: being able to say "the model of release 1.2 was trained with these exact data", going back to any version, and sharing with the team. The solution is not putting the data in Git, but putting in Git a verifiable reference to the data.

DVC's mental model: metafiles + cache + remote

DVC (Data Version Control) is an open-source command-line tool that works on top of Git — it doesn't replace it. Its trick is a clean separation:

  • In Git live some tiny text metafiles (.dvc extension) describing each versioned piece of data: essentially its MD5 hash (a fingerprint of the content), its size, and its path. They take up bytes and diff beautifully.
  • Outside Git, the real files live in a local cache (.dvc/cache/, organized by hash) and, for sharing, in a remote: any storage — a network folder, S3, Google Cloud Storage, Azure Blob...
  • In your working directory, data/raw/churn_customers.csv is (depending on the system) a link or a copy materialized from the cache: you use it normally, pandas never notices.
flowchart LR
    subgraph Git["Git repository (text, lightweight)"]
        M["churn_customers.csv.dvc<br/>md5: 3f2a..., size: 2.1 MB"]
    end
    subgraph Local["Local machine"]
        W["data/raw/churn_customers.csv<br/>(working file)"]
        C[".dvc/cache/<br/>(contents by hash)"]
    end
    R["Remote<br/>(shared folder / S3 / GCS)"]
    M -- "dvc checkout" --> W
    W -- "dvc add" --> C
    C -- "dvc push" --> R
    R -- "dvc pull" --> C

The elegant consequence: every Git commit pins, via the metafile's hash, an exact version of the data. Code and data end up tied together in the same history without the data ever passing through Git. Switching data versions will be as simple as switching commits and asking DVC to materialize what that commit references.

Hands on: dvc init and dvc add

DVC is just another Python package, so it enters the project through the door we built in 02-02: add dvc>=3.50 to the dev extra of pyproject.toml, regenerate the lock with pip-compile, and sync. With the environment activated, we initialize:

dvc init
git status
#   new: .dvc/config  .dvc/.gitignore  .dvcignore
git commit -m "Initialize DVC in the project"

dvc init creates the .dvc/ directory (configuration and, later on, the cache) and a few auxiliary files, all meant to be committed. DVC always leaves breadcrumbs in Git: that's the whole point.

Now we put the dataset under control. First we remove data/ from the general .gitignore we wrote in 02-01 (DVC will manage finer-grained ignores on its own) and run:

dvc add data/raw/churn_customers.csv

This does three things: it computes the file's hash, stores a copy in the cache (.dvc/cache/), and creates the metafile data/raw/churn_customers.csv.dvc, besides adding the real CSV to a local .gitignore so Git doesn't see it. The metafile is this humble:

# data/raw/churn_customers.csv.dvc  (generated by DVC; never edited by hand)
outs:
- md5: 3f2a9c81d4be7a06c1e5f9a2b8d47e11
  size: 2184730
  hash: md5
  path: churn_customers.csv

Read it like a luggage-check receipt: "there exists a file named churn_customers.csv, 2.1 MB, whose content has this MD5 fingerprint". With that hash, DVC can locate the exact content in any cache or remote. Change a single byte of the CSV and the hash changes completely — that's why it works as a version identifier. What goes to Git is the receipt:

git add data/raw/churn_customers.csv.dvc data/raw/.gitignore
git commit -m "Version the churn dataset (June extract) with DVC"

Notice the commit message: describing which version of the data this is ("June extract") turns the git log into the dataset history we never had.

The remote: sharing data with dvc push and dvc pull

The cache is local: if Laura versions the dataset on her machine, Marc still can't get it. The remote is to data what GitHub is to code: the exchange point. DVC supports S3, GCS, Azure, SSH, local or network folders... and it's configured once:

# For the course: a local directory simulating the shared storage.
dvc remote add -d storage /srv/dvc-remote-cineclick
git add .dvc/config
git commit -m "Configure the data remote"

The -d flag marks it as the default remote, and the configuration lands in .dvc/config — text, committed, shared. In CineClick's real life this would point to a bucket (dvc remote add -d storage s3://cineclick-mlops-data would suffice, plus the credentials, which never go to Git — DVC stores them separately with dvc remote modify --local). For following the course, the local folder simulates the bucket perfectly: the commands are identical.

And the exchange:

dvc push    # uploads to the remote whatever cache contents are missing there

On the other side, Marc:

git clone <repo> && cd cineclick-churn
# ... creates the environment as in 02-02 ...
dvc pull    # reads the .dvc files of the current commit and downloads those exact contents

dvc pull looks at which hashes the metafiles present in the current commit reference, finds them in the remote, downloads them to his cache, and materializes the working files. Marc now has exactly the same bytes Laura trained with — not "whatever CSV happened to be in the shared folder that day". The git pull + dvc pull gesture becomes the project's new "catching up".

Time travel: a new version of the dataset

Here comes the moment that justifies all the machinery. It's July, and the data engineer delivers a new extract with two novelties: an additional month of customers and, mind you, the weekly_hours column now arrives corrected at the source (they fixed the negative-values bug Laura used to filter by hand). The file has the same name and replaces the previous one in data/raw/. Before, this was an irreversible loss; now:

# The new extract overwrites the working file
dvc status
#   data/raw/churn_customers.csv.dvc: changed  <- DVC detects the content no longer matches the hash

dvc add data/raw/churn_customers.csv        # new hash, new cache entry
git add data/raw/churn_customers.csv.dvc
git commit -m "July data: new month and weekly_hours fixed at the source"
dvc push

The metafile now contains the July extract's hash, and the commit writes it into history. Both versions coexist in the cache and in the remote, each under its own hash. And here it is, the time travel — suppose the model trained on July behaves oddly and we want to reproduce June's training:

git log --oneline -- data/raw/churn_customers.csv.dvc   # locate the June commit
git checkout a1b2c3d                                    # Git restores June's METAFILE...
dvc checkout                                            # ...and DVC materializes June's DATA from the cache

git checkout only moves the files Git controls — that is, the .dvc with the old hash. dvc checkout reads that hash and restores the corresponding content into data/raw/. Two commands, and the working directory is identical, code and data, to the day of June's training. To return to the present: git checkout master && dvc checkout. If the content weren't in the local cache (a fresh machine), dvc pull would fetch it from the remote.

This is exactly what module 1's diagnosis was missing in problem #3: "the current model can't be reproduced and trainings can't be compared: the v3 dataset no longer exists". Now every dataset exists forever, addressable by commit, and the question "which data was this model trained with?" finally has a mechanical answer: the data of the commit it came from.

Who versions what: Git vs. DVC

The pair splits the work like this:

Aspect Git DVC
What it versions Code, configuration, tests, docs, .dvc metafiles Data and large/binary files (datasets, and soon models)
Where it stores content In the repository itself (full history in every clone) Local cache + remote (you download only what you need)
Unit of change Commit (line-by-line diff) Hash of the file's full content
How it's shared git push / git pull (GitHub, etc.) dvc push / dvc pull (S3, GCS, network folder...)
How you time-travel git checkout <commit> dvc checkout (after Git's)
Comfortable size KB–MB, text MB–TB, any format

And the mnemonic rule that governs the split: Git versions the recipes; DVC, the ingredients. Every Git commit is a complete snapshot of the project because it contains the code, the configuration, the environment lock... and the receipts that pin the data.

Common Mistakes and Tips

  • Mistake: committing the metafile without doing dvc push (or the other way around). If Marc does git pull and the hash the .dvc references isn't in the remote, his dvc pull will fail with an eloquent "file is missing". The two pushes go together, always; internalize it as a single gesture (in module 5, CI will verify it for us).
  • Mistake: editing a .dvc file by hand. The hash is a computed fingerprint; writing a different value doesn't change the data, it only breaks the correspondence. DVC writes the .dvc files (dvc add); you only commit them.
  • Mistake: doing git checkout to an old commit and forgetting the dvc checkout. You're left in a treacherously inconsistent state: June's code with July's data in the directory. The pair goes together: git checkout + dvc checkout, there and back. (There's dvc install, which hooks into Git to automate the second step; enable it if you catch yourself forgetting.)
  • Mistake: versioning with DVC files that belong in Git. A 20-line config.yaml in DVC is counterproductive: you lose the diff, which there is genuinely valuable. The criterion is size + binariness + rate of change, not "everything in data/ by definition".
  • Tip: a new dataset deserves an informative commit message. Not "update data"; yes "July data: new month and weekly_hours fixed at the source". The history of git log -- data/raw/*.dvc is the documentation of your data's evolution — for free.

Exercises

Exercise 1

Put this shuffled sequence of commands in the right order to incorporate a new extract of the dataset and make it available to the whole team, and explain what each step does: git commit -m "August data", dvc push, dvc add data/raw/churn_customers.csv, git add data/raw/churn_customers.csv.dvc, (the new file overwrites the old one in data/raw/).

Exercise 2

Laura trained the "good model" three weeks ago, at commit f4e5d6a. Today the working dataset is July's. Write the exact sequence of commands to: (1) leave the machine exactly as it was on the day of that training (code and data); (2) verify that the materialized data is in sync with the metafile; (3) return to the current state. State which additional command you would need if you did this on a freshly cloned machine.

Exercise 3

A colleague proposes: "instead of DVC, let's upload each extract to the shared folder as churn_customers_2026_06.csv, churn_customers_2026_07.csv... and have config.yaml point to whichever one applies". It's an improvement over overwriting, no doubt. Point out at least three guarantees DVC provides that this date-suffix scheme does not.

Solutions

Solution 1: (0) the new extract overwrites the working file — the starting point; (1) dvc add data/raw/churn_customers.csv — computes the new hash, stores the content in the cache, and updates the metafile; (2) git add data/raw/churn_customers.csv.dvc — stages the updated receipt for commit; (3) git commit -m "August data" — pins the new data version into history alongside the current code; (4) dvc push — uploads the content to the remote so any colleague's dvc pull works. The order of 3 and 4 can be swapped, but neither may be skipped.

Solution 2: (1) git checkout f4e5d6a followed by dvc checkout — Git restores the code and the metafile from that day; DVC reads its hash and materializes that dataset from the cache; (2) dvc status — it should report everything is up to date (no discrepancy between working files and metafiles); (3) git checkout master and dvc checkout again. On a freshly cloned machine the cache is empty, so between the checkout and the work you would need dvc pull to download from the remote the content the hash references.

Solution 3: among others: (1) integrity verification — the MD5 hash guarantees the file is exactly the original; with date suffixes, nothing stops someone from editing or re-uploading _2026_06.csv without anyone ever noticing; (2) automatic code↔data linkage — with DVC, each commit pins which data belongs to it; with suffixes, the link depends on a human updating config.yaml in the right commit and on nobody touching it afterwards; (3) deduplication and efficient transport — the hash-based cache never stores the same content twice and dvc pull fetches only what's missing, while the folder of dated files grows unchecked and gets copied by hand; (4, as a bonus) a single flow for any size and backend — the day the CSV becomes 40 GB on S3, the commands don't change.

Conclusion

The third artifact has fallen into line: with dvc init, dvc add, and a configured remote, every version of churn_customers.csv is pinned by its hash in a committed metafile, shared via dvc push/dvc pull, and recoverable forever with the git checkout + dvc checkout pair — problem #3 from the diagnosis is solved. The picture now is: versioned code, versioned environment, versioned data... but the process that ties them together is still artisanal: run the cleanup, then the features, then the training, then the evaluation, by hand and in the right order, remembering what needs redoing when something changes. That manual chaining is fragile, and DVC has a second face designed precisely to eliminate it: pipelines declared in dvc.yaml, where each stage knows its dependencies and dvc repro re-runs only what's necessary. It's the piece that closes the module, and with it CineClick will touch full reproducibility for the first time: we see it in the next lesson.

© Copyright 2026. All rights reserved