The previous five lessons solved problems with names: I have lost commits, my branch has diverged, the repository is corrupt. This one deals with the category that has no name and that is, in practice, the most frequent:

Git is doing something I do not understand, and I do not even know where to start looking.

A file that shows up as modified when nobody has touched it. A .gitignore that ignores the wrong things. A push that takes forty seconds in a small project. A hook that does not run. A configuration that says one thing and a behaviour that says another. A git log that hides commits that exist.

They are not breakdowns: they are mismatches between what you think is there and what actually is. And for that Git has a toolbox we have barely touched on: trace variables that show what is happening underneath, commands that say where each setting comes from, and the plumbing layer that queries the database without interpretation.

More important than the tools is the method, because without it the tools produce noise. We will close with a four-step diagnostic method and with a real investigation into task-manager that combines bisect and blame to go from the symptom to the commit, and from the commit to the line and its reason.

1. The diagnostic method, in four steps
2. Trace variables: seeing what Git does underneath
3. Debugging the effective configuration
4. Debugging paths and patterns: check-ignore, check-attr, ls-files
5. Plumbing for inspecting the graph
6. Narrowing down with git diff --stat and git log
7. A real investigation: from bisect to blame
8. A catalogue of mysteries and their commands

Before the tools, the procedure. It is what separates a twenty-minute investigation from a wasted afternoon.

Reduce the symptom to the minimal command that causes it.

If you cannot reproduce it at will, you cannot verify that you have fixed it. And very often reducing the case already reveals the cause: "it only fails with this branch", "only with this file", "only from Carla's laptop".

Remove variables one at a time:

If the problem goes away without the global configuration, you already know where to look. If it persists in a clean clone, the problem is in the repository or on the server, not on your machine.

This is the step that is skipped most often and the one that resolves matters most often.

Memory and documentation lie; commands do not.

Formulate a concrete, falsifiable prediction before touching anything:

"If the cause is that .gitattributes marks *.css as text eol=crlf, then git check-attr eol styles.css must say crlf, and git ls-files --eol styles.css must show w/crlf."

Run it, check it, and only then fix it. If the prediction fails, the hypothesis was a bad one: go back to step 3. Changing things at random until "it works" leaves the problem latent and prevents you knowing what caused it.

Git has a tracing system that is switched on with environment variables. They are the most direct way of seeing what is really going on.

All of them accept an absolute path instead of 1, to write to a file:

There you can see, for instance, whether Git is calling gpg (commit signing, lesson 08-05), a hook, or a credential helper.

It is especially useful with aliases that do odd things (lesson 06-04):

And with hooks that do not run:

If nothing appears, the hook is not being launched. Usual causes: it is not executable (chmod +x), core.hooksPath points somewhere else, or the file name has an extension.

It resolves an entire family of mysteries:

• "Not a git repository" while inside one: it tells you which .git it finds (or that it finds none).
• You are working in a repository that is not the one you think: typical with submodules (lesson 06-05) and worktrees (06-06), where git_dir and git_common_dir differ.
• The .gitignore patterns do not work as you expect: the prefix explains where paths are interpreted from.

Picking up section 5.8 of lesson 09-01, here is the complete version.

Every line is a diagnosis:

The classic case: you have several keys, SSH offers the wrong one first and the server cuts you off after N attempts. The solution is a block in ~/.ssh/config:

IdentitiesOnly yes is the key part: it forces it to offer only that one.

For HTTPS:

It diagnoses corporate proxies, certificates, redirects and credentials. A persistent 401 points at the credential helper (lesson 04-03):

Careful: GIT_CURL_VERBOSE can display Authorization headers. Do not paste its output into a public ticket without reviewing it (lesson 08-05).

The deepest level. It shows the exact dialogue with the server.

What it is really for:

• Seeing which references the server announces and which protocol version is negotiated.
• Diagnosing a slow fetch: if the server announces 40,000 references, there is the problem (lesson 08-06, reference hygiene).
• Understanding a push rejected by a server hook (lesson 07-06): the hook's message travels in these packets.
• Checking whether protocol v2 is in use, which is far more efficient:

And a script to capture everything at once when a report has to be sent:

Picking up from lesson 01-05. Git reads the configuration from several places and the last one wins. When something behaves unexpectedly, the configuration is suspect number one.

Two new columns compared with the usual --list:

• --show-scope: which level it is at (system, global, local, worktree, command).
• --show-origin: the exact file, including those brought in by include.path.

In the example, user.email appears twice: the local one wins. There is the explanation for "I have set the email correctly and the commits come out with a different one" (lesson 09-01, section 5.3).

That last one deserves attention: if somebody copied a configuration that silences the advice.* settings, you are missing the suggestions that solve half the problems in this module (lesson 09-01, section 6).

The last one is the definitive test for "is it something in my configuration?".

It is an excellent feature for separating identities, but it produces the bafflement of "it works here and not there". --show-origin untangles it in a second, because it names the included file.

And watch out for the trailing slash: gitdir:~/projects/work/ (with /) matches the directory and its descendants; without it, the behaviour changes.

This is where the most frequent day-to-day mysteries live.

Picking up from lesson 08-03:

Reading: rules file : line : pattern that decides, and the file affected. There is no possible ambiguity.

If check-ignore returns nothing and the exit code is 1, no rule is ignoring it. And if the file nonetheless appears in git status despite being ignored, the cause is almost always the same:

It is in the index. .gitignore only affects untracked files; one that is already tracked carries on being watched even if the pattern matches. The solution is the one from lesson 08-03:

Other less obvious causes, which check-ignore -v reveals by naming the rules file:

And the classic trap with directories:

If a directory is ignored, Git does not even go into it, so a negation rule for a file inside it does not work:

Picking up from lesson 08-04:

This explains a whole family of mysteries at a stroke:

git status interprets; git ls-files shows. It is the direct window onto the index.

That last field is gold during a conflict:

Three stages: the common base, ours and theirs. Exactly what we saw in lesson 03-05, now visible in the raw.

The modes that solve mysteries:

Index 100644, disk executable: that is the change. It is corrected with:

And the --eol mode, which solves the line-endings mystery:

The first line is Carla's healthy case on Windows: LF in the repository, CRLF on her disk (lesson 08-04). The third is a problem: i/crlf means CRLF has been stored inside the repository, which is precisely what you do not want.

Other useful modes:

--others --exclude-standard is exactly the list of "new files" that git status shows, without the rest of the output. And --deleted explains the "I deleted the file and Git keeps complaining".

When the question is about the repository's structure, the answer lies in the plumbing layer we saw in lesson 01-04.

And its informational modes, which answer questions about the environment:

They are the basis of any script that has to work with repositories robustly.

That --max-parents=0 is the exact diagnosis for the unrelated histories of lesson 09-03: if it returns two hashes, there are two roots.

It is the same technique as the size analysis from lesson 08-06, applied here to understanding the repository's composition.

Columns: hash, type, real size, compressed size, position in the pack and — on the last line — delta depth and base object. It is the low-level view of what lesson 08-06 explained: the last entry is stored as a difference from another.

It queries the server directly, without a fetch and without touching your repository. It answers questions such as "does that branch exist on the server?" or "is my origin/main up to date?":

Before a fine-grained investigation, it is worth narrowing things down. These are the coarse-grained tools.

And git log in investigation mode (lesson 06-04):

-L :function:file is especially powerful and little known: it follows one particular function throughout the history, even when it moves within the file.

Let us bring it all together in a complete case on task-manager.

The symptom. Carla reports: "The pending task counter shows one too many when there are tasks hidden by the filter. In version 1.4 it worked."

The minimum that causes the failure, written as an automatable test:

An automatable test is what turns bisect from tedious into instant.

34 commits between the two versions. Going through them by hand is hours; bisect is five steps.

Picking up from lesson 06-02:

From the symptom to the commit in five automatic steps. Now we know when it broke.

There it is: the || t.hidden counts the hidden ones too. But knowing which line is at fault is not knowing why it is there, and fixing it without understanding can break what that commit set out to solve.

Picking up from lesson 06-03:

Here is the finding, and it is what changes the whole outcome. The behaviour is not a bug: it is a deliberate, documented decision in GT-231. What we have is a contradiction between two specifications: GT-231 says they should all be counted if they are not completed; Carla expects only the visible ones to be counted.

Had we "fixed" the line as soon as we saw it, we would have broken GT-231 and the cycle would have started again in a fortnight.

The conclusion of the investigation is not a patch, but a question for the team: what does "pending" mean? And the answer, whatever it is, gets documented in the commit that implements it.

bisect takes you from the symptom to the commit. blame and the commit message take you from the commit to the intention. The first without the second produces patches that break something else. And here you can see, in retrospect, why lesson 08-01 insisted so much on explaining the why in messages: that paragraph of Bruno's has saved a mistake.

The quick reference table for "Git is doing something odd".

And a general diagnostic script, for when you do not even know where to start:

Keep it. On the day you need it, it will save you fifteen minutes of scattered commands.

Mistake 1: changing things at random until it works. It leaves the problem latent and you learn nothing. Formulate a falsifiable hypothesis and test it.

Mistake 2: trusting memory instead of querying the real state. "I think that file is ignored" is not the same as git check-ignore -v.

Mistake 3: not isolating. Before investigating in depth, check whether the problem persists without the global configuration, without hooks and in a clean clone. That elimination costs a minute.

Mistake 4: switching on GIT_TRACE_PACKET from the outset. It is the deepest level and produces a great deal of noise. Start with GIT_TRACE and GIT_TRACE_SETUP.

Mistake 5: pasting GIT_CURL_VERBOSE output into a public ticket. It can contain Authorization headers (08-05). Review it first.

Mistake 6: using git config --list without --show-origin --show-scope. Without those options you do not know which value wins or where it comes from, which is exactly what you are investigating.

Mistake 7: fixing the line bisect points at without reading the commit message. You can break the functionality that commit set out to implement, as would have happened in section 7.

Mistake 8: attempting bisect without an automatable test. With git bisect run and a script, thirty commits are five automatic steps; by hand, it is half an hour of tedium and mistakes.

Mistake 9: confusing git status with reality. status interprets; ls-files --stage, check-attr and rev-parse show.

Tip 1: the method before the tools. Reproduce, isolate, query the real state, test the hypothesis.

Tip 2: git config --list --show-origin --show-scope as a reflex. It solves a surprising fraction of the "Git is doing something odd" cases.

Tip 3: git check-ignore -v and git check-attr -a. Two commands that give the exact answer to two of the most frequent day-to-day questions.

Tip 4: git ls-files --eol for any line-endings problem. The i/ and w/ table says it all at a glance.

Tip 5: keep the diagnostic script from section 8. It is the complete snapshot in thirty seconds.

Tip 6: write the test before bisecting. The effort is recouped on the first bisect run, and the test stays in the project.

1. Create a repository with app.js, styles.css and index.html, and commit them.
2. Add a .gitattributes with *.css text eol=crlf and commit it.
3. Run git status and observe the result. Then run git ls-files --eol and git check-attr -a styles.css.
4. Explain exactly what is happening using the i/ and w/ columns.
5. Resolve it with git add --renormalize . (lesson 08-04) and check with ls-files --eol that the columns add up.
6. Repeat the exercise with a file that .gitattributes marks as -diff and check what changes in git diff.

1. Create a repository and configure a different user.email at the local level and at the global one.
2. Use git config --list --show-origin --show-scope to determine which one wins, and confirm it by making a commit and looking at %ae.
3. Create a .gitignore with *.log, a .git/info/exclude with !important.log and a global core.excludesFile with temp*. Create the three corresponding files and use git check-ignore -v on each to determine which rule decides.
4. Create a pre-commit hook that prints something, make it non-executable, and try to commit. Use GIT_TRACE=1 to demonstrate that it is not launched.
5. Make it executable and repeat, checking in the trace that it now does appear.
6. Run GIT_TRACE_SETUP=1 git status from a subdirectory and explain each line of the output.

1. Create a repository with an app.js containing a correct calculatePending function, and make twenty commits that touch other parts of the project.
2. In an intermediate commit, introduce the fault from section 7 (adding || t.hidden) with a commit message that explains the why.
3. Make ten more commits on top.
4. Write a test script that exits with 0 if the function is correct and with 1 if not.
5. Find the culprit commit with git bisect run and note how many steps it needed.
6. Use git show, git blame -L and git log -L :calculatePending:app.js to reconstruct that function's complete history.
7. Read the culprit commit's complete message with git log -1 --format=%B and explain why the right answer is not "change the line".

Solution 1:

Surprisingly, nothing. The attribute is applied when writing to disk, and the file on disk still has LF. Force the update:

There is the typical mystery: a modified file that nobody has touched.

Reading the columns:

• i/lf: in the index there is LF, which is what was stored in the original commit.
• w/crlf: on disk there is CRLF, because the eol=crlf attribute converts it on checkout.
• Git compares index and disk byte for byte, sees a difference at every line ending, and marks it as modified.

It is not an error: it is the attribute working, on a file that was stored before the attribute existed.

Clean. --renormalize rewrites the index applying the current attributes: now the index stores LF in the knowledge of the attribute, and the comparison adds up. It is exactly the procedure from lesson 08-04.

-diff makes Git treat the file as binary for diff purposes. It is useful for minified and generated files, and baffling if you do not know it is set: git check-attr diff resolves it in a second.

Solution 2:

The local one wins, because it is closer to the repository. It is exactly the mechanism from section 3, and the usual cause of "I have set the email correctly".

Each one decided by a different file, and check-ignore -v names it with its line number. Note important.log: the matching rule is the negation, and that is why the file is not ignored:

When check-ignore -v returns a pattern beginning with !, it means "this rule rescued it".

Zero mentions of the hook in the trace, and the commit went through. The hook was not launched.

There it is: run_command launches it and the echo appears. GIT_TRACE=1 is the definitive diagnosis for "my hook does not run" (lesson 06-01).

That prefix explains why git add . from a subdirectory only adds that subdirectory, and why patterns are interpreted the way they are.

Solution 3:

The test fails at HEAD: reproduced.

Five automatic steps for 31 commits. The base-2 logarithm of 31 is a little under 5: exactly what lesson 06-02 predicted.

Two commits in that function's whole life. -L :function:file follows it even if it moves elsewhere within the file, which blame -L 1,3 would not do.

Why the right answer is not "change the line":

The behaviour is intentional and is justified in writing. There is no programming error: there are two specifications that contradict each other. GT-231 says the counter should include the hidden ones; what Carla expects is the opposite.

If the || t.hidden is simply removed:

1. GT-231 breaks, which somebody asked for and somebody signed off.
2. Nobody will know why, because the commit that undoes it will probably say "fix the counter".
3. In a fortnight, whoever asked for GT-231 will open an identical ticket in the opposite direction, and the cycle will start over.

The correct way out is to take the contradiction to whoever can resolve it, and for the decision — whatever it turns out to be — to end up written in the message of the commit that implements it (lesson 08-01), with a reference to both tickets.

Bruno's message has saved a mistake. That three-line paragraph is the difference between a correction and a loop.

This lesson was about understanding why Git is doing what it is doing.

• The method comes before the tools: reproduce with the minimal command, isolate by removing variables (global configuration, hooks, a clean clone), query the real state rather than the remembered one, and test a concrete hypothesis before changing anything.
• The trace variables show what is happening underneath: GIT_TRACE for subcommands, aliases and hooks; GIT_TRACE_SETUP to know which repository Git thinks it is using; GIT_SSH_COMMAND="ssh -v" and GIT_CURL_VERBOSE for authentication and the network; GIT_TRACE_PACKET for the protocol. Start with the gentle ones and review the output before sharing it.
• git config --list --show-origin --show-scope solves a surprising fraction of the mysteries, because it says which value wins and which file it comes from, including conditional includeIf entries.
• git check-ignore -v and git check-attr -a answer precisely the two most frequent day-to-day questions: why a file is (or is not) ignored, and which attributes apply to it.
• git ls-files is the window onto the index: --stage for modes and conflict stages, --eol for line endings with its i/ and w/ columns, --others --exclude-standard for what is genuinely new.
• The plumbing answers questions about the graph without interpretation: rev-parse to translate into hashes and learn about the environment, rev-list to count and list, cat-file --batch-check for bulk inspection, verify-pack for the inside of a packfile, for-each-ref for all the references and ls-remote for the server without downloading anything.
• And the complete investigation: bisect takes you from the symptom to the commit; blame and the commit message take you from the commit to the intention. Skipping the second step produces patches that break something else.

Module 8 ended by announcing disasters. This one has solved them all, and the underlying lesson is a single one:

In Git, almost nothing is truly lost. What gets lost is your composure.

Commits are immutable objects that survive even when no reference points at them (09-01). reset moves a branch, it does not destroy history (09-02). A divergence is a pending decision, not a breakdown (09-03). The reflog remembers where you have been, and with it come back the reset --hard, the deleted branch, the failed rebase and the deleted stash (09-04). A damaged repository is a logistics problem, because any colleague's clone is an almost complete copy (09-05). And when none of that fits, there are traces, plumbing and a method (09-06).

What is genuinely fragile is what never became an object: the uncommitted change, the untracked file. That is why the most profitable advice in the whole module is not a command but a habit: commit early, mark the spot with git branch before anything risky, and publish your branches.

Ana, Bruno, Carla and Diego have mastered Git in task-manager. They know how to build the history, manipulate it with judgement, collaborate with a process, maintain good habits and get out of trouble.

But task-manager is four files and a team of four people. The real world is bigger and stranger.

There are projects with fifty thousand files and twenty years of history, where an unoptimised git status takes half a minute. There are repositories that store videos, 3D models and design files of hundreds of megabytes, and that need a different system for storing them. There are organisations where Git is used not by one person at a terminal but by a hundred deployment pipelines that clone, tag and publish with no human intervention. There are integrations with editors, with ticketing systems, with review platforms and with analysis tools that completely change the daily experience. And there is a Git that carries on evolving: SHA-256, references packed in a new format, partial clones, sparse indexes.

In module 10: Git in the Real World we will look at case studies of how real projects and organisations use Git, integration with other tools in day-to-day work, Git LFS for large files, how to scale Git in enormous repositories and monorepos, the role of Git in DevOps as a central piece of continuous delivery, and where Git is heading in the next few years.

We start by looking at how others do it, in lesson 10-01: Case Studies.