In the previous lesson you learned to move around srv-veloz-01 and one question was left hanging: why cd cannot be an external program. That question is the doorway to this lesson. Up to now you have used the shell; now you are going to understand exactly what happens between pressing Enter and the result appearing. This knowledge is not theoretical: it explains why a variable defined in a pipeline disappears, why which sometimes lies, why your script "cannot find" a command that you can see perfectly well, and why source and running a file are not the same thing. These are precisely the mistakes that waste the most time for anyone writing Bash without this mental model.

1. What happens when you press Enter
2. The command types and how to tell them apart
3. The PATH and the lookup order
4. Parent and child processes: fork and exec
5. Subshells and why variables do not survive
6. Environment versus shell variables
7. $?: the exit code
8. Putting it all together at Veloz Envíos

You type ls -lh /var/log/veloz and press Enter. What looks instantaneous is in fact six well-defined phases.

Let us go through each phase.

Bash reads the whole line and splits it into words using spaces, tabs and newlines as separators. It also identifies the special operators (|, >, &&, ;). From ls -lh /var/log/veloz come three words.

The first source of errors already shows up here. If a file name contains spaces, the splitting breaks it in two:

Bash sees two arguments, not one. That is why quotes matter: ls "my report.txt". All the detail is in 03-06.

Bash rewrites the line before running anything. It substitutes:

This point is crucial and surprises a lot of people: the command never sees the asterisks. When you type ls *.log, the one expanding the pattern is Bash, and ls already receives the list of names. You can check it:

echo knows nothing about wildcards: it received three already-resolved arguments. The full detail of expansions is covered in 02-05 and 03-06; here it is enough to know that they happen first.

Bash takes the first word and looks up in a strict order what it is. We will see this in section 2, because that is where the surprises are concentrated.

If the command turns out to be an external executable, Bash cannot "become" it (your shell would cease to exist). It does two things:

1. fork(): it creates a copy of itself, a child process.
2. exec(): the child replaces its own image in memory with that of the program to run.

The parent (your Bash) waits with wait() for the child to finish. That is why the prompt does not come back until the command is done, unless you launch it in the background with &.

When the child dies, it returns a number between 0 and 255. Bash stores it in $?. It is the basis of all error handling in scripting, and we will look at it in section 7.

Not everything you type at the prompt is a program. Bash recognizes five categories, and it looks for them in this exact order:

The order matters. If you define a function called ls, it will be used instead of /usr/bin/ls. And if you also define an ls alias, the alias beats the function.

type is a Bash builtin and answers exactly what Bash would do.

The -a option shows all the matches, in priority order. It is revealing:

There are two echos: the Bash builtin and the /usr/bin/echo program from coreutils. The builtin wins, always. And they are not identical: their options differ slightly between systems, which produces scripts that work on one machine and not on another. That is why professional scripting prefers printf, which is far more predictable (we will see it in Module 3).

Another very illustrative example with our veloz-ops PATH:

Two paths appear because on modern systems /bin is a symbolic link to /usr/bin: it is the same file seen two ways.

command -v prints the path (or the definition) and returns an exit code indicating whether it exists or not. It is the standard, portable way of checking availability in a script:

We will use this pattern for real in Module 6, when veloz-ops needs jq to talk to the API.

Notice: which cd prints nothing. And that is the problem. which is an external program (on many systems, a script) that merely walks the PATH looking for files. It knows nothing about builtins, functions or keywords, because it cannot see inside your shell.

The practical conclusion: use type -a to investigate interactively and command -v inside scripts. Avoid which. A real case of confusion: a colleague complains that time does not exist because which time returns nothing; in reality time is a Bash keyword, and type time clears it up instantly.

When Bash reaches step 5 (external executable), it walks the PATH directories from left to right and runs the first match.

Important points:

• The search stops at the first match. If there were an ls in ~/veloz-ops/bin, that one would run and not the system's.
• You decide the order. Putting your directory first gives priority to your tools; putting it last is safer.
• Bash caches the paths it finds so it does not have to repeat the search. If you install a new program and Bash keeps saying it does not exist, clear the cache:

Or check what it has memorised:

This is a real and frustrating problem: you have just moved a script somewhere else, you run it and Bash tries to launch it from the old path. hash -r fixes it in a second.

If you are in ~/veloz-ops/bin and you type report.sh, Bash will look in the PATH. Since the current directory is not in the PATH (and for security reasons it must not be, as we saw in 01-02), it will not find it:

The solution is to give an explicit path: ./report.sh. As soon as a word contains a slash /, Bash stops searching the PATH and treats it as a direct path.

With our veloz-ops configuration there is a pleasant nuance: since we added ~/veloz-ops/bin to the PATH, the scripts we put there can be invoked by name from any directory, just like ls. That is exactly the effect we were after when setting up the toolkit.

Every process in Linux has an identifier (PID) and a parent (PPID). So does your shell.

$$ is the PID of the current shell and $PPID that of its parent (usually the terminal emulator or sshd).

Let us check the parent-child relationship live:

The child has a different PID and its parent is your shell. That is the mechanics of fork.

When the child finishes, everything it did in its own memory disappears: variables, current directory, functions. Only what it wrote to disk or sent to its standard output survives. This principle explains half of Bash's "mysteries".

If you use exec explicitly, Bash does not create a child: it replaces itself with the program.

The date is displayed and the terminal closes, because your shell no longer exists: it became date, and date finished. It is a specialized tool, widely used in the entrypoint.sh of Docker containers so that the application process inherits PID 1 and receives stop signals correctly. Do not use it lightly.

A subshell is a child process that is also a Bash. They are created in more situations than people suspect.

The parentheses created a child process with a copy of the variables. That child modified its copy and died. The parent never found out. It is not a bug: it is how processes work in Unix. Information flows from parent to child, never the other way round.

Compare with braces:

Braces only group; there is no new process. (Watch the syntax: braces need inner spaces and a ; before the closing one.)

This is the classic mistake, and we will pose it with Veloz Envíos data. We want to count how many lines of app.log are errors:

Zero, even though there really are errors in the file. Why? Because the right-hand side of the pipeline runs in a subshell. The loop correctly incremented its own copy of errors up to, say, 47; then the subshell finished and the copy evaporated. The final echo reads the parent shell's variable, which is still 0.

There are three solutions, and it is worth knowing all of them:

The < <(...) construct is called process substitution: grep runs separately, but the while loop runs in the main shell, so the variable survives. It is studied in depth in 05-05.

With lastpipe, the last command of the pipeline runs in the current shell. It works in scripts, but not in normal interactive sessions.

This is the best one: grep -c counts by itself and command substitution captures its output. When a tool already does the job, do not do it with a loop. It is faster, shorter and has no subshell problem.

Now what we saw in 01-02 makes complete sense:

And that is why cd has to be a builtin: if it were an external program, it would run in a child, change that child's directory and die. Your directory would not move an inch. That pending question from the previous lesson is now answered.

There are two kinds of variables, and the difference is exactly the one from the previous section: what gets inherited.

• Shell variable: it exists only in the current shell. It is not passed to children.
• Environment variable: it is copied into the environment of every child process.

The child did not see local_var because it was never exported.

Environment variables you will always run into:

You can define a variable for one command only, without touching your shell:

This syntax — an assignment in front of the command, with no ; — is beautifully clean and widely used in practice, for example to force a language in a command's output:

With LC_ALL=C the date comes out in English regardless of the server's locale. It is an important trick in scripts: if your script parses a command's output, force LC_ALL=C so it does not depend on the server's language. A grep looking for "Aug" would fail on a machine configured in Spanish.

And to run something with a completely clean environment:

env -i wipes the whole environment. It is the best way to simulate cron's conditions, since cron runs with a minimal environment. If your script works like this, it will work in cron. We will apply it in 07-01.

Every command returns a number between 0 and 255 when it finishes:

• 0 means success.
• Any other value means failure.

It is the opposite of what intuition suggests, but it makes sense: there is only one way to get it right and many ways to fail, so every non-zero value can identify a kind of failure.

Codes with a conventional meaning:

Let us check them:

A detail that causes subtle bugs: $? is overwritten by every command, including the echo that displays it.

The second read returns 0 because it is the result of the previous echo, which worked fine. If you need the value more than once, save it immediately:

A special case that deserves attention: in a pipeline, $? is the code of the last command, not the first.

Even though grep failed, wc -l worked, so the pipeline declares success. It is a source of silent errors in scripts. It is solved with set -o pipefail or by consulting the PIPESTATUS array, topics of lesson 05-03.

Exit codes are the foundation of &&, ||, if and error handling in general. This was only a first contact; the complete use arrives in 03-03 and 03-04.

A realiztic scenario that combines everything above. A colleague tells you: "I have written a veloz-status script that counts the errors in the log, but it always returns 0 and on top of that, when I launch it from cron it says the command is not found."

Let us diagnose it step by step.

Step 1: what is veloz-status really?

It exists and it is in the PATH. But cron does not read your ~/.bashrc (we saw that in 01-02), so its PATH does not include ~/veloz-ops/bin. Let us simulate cron's environment:

Reproduced. The solution: use the absolute path in the crontab, or define the PATH inside the script itself.

Step 2: why does it count 0?

The script contains this:

You already know the diagnosis: the while runs in a subshell and total does not survive. The fix:

Step 3: and what if the file does not exist?

If the log does not exist, grep returns 2 and total is left empty. Saving the code lets you react. In 03-04 you will learn to turn that into a condition, and in 05-03 to do it robustly.

Step 4: final check of the environment

This small ritual — knowing what is going to run, with which environment and what it returned — is exactly what separates someone who debugs methodically from someone who tries things at random.

• Using which to check whether a command exists. It does not see builtins or functions and its behavior varies between systems. Use command -v in scripts and type -a interactively.
• Expecting a variable modified inside a pipeline to survive. The right-hand side of | is a subshell. Use < <(...), lastpipe, or better still, a tool that already does the calculation (grep -c, wc -l, awk).
• Running a configuration file with ./config.sh expecting it to define variables. You need source config.sh.
• Forgetting export and being surprised that the child script does not see the variable. Without export, the variable never leaves the current shell.
• Reading $? too late. Any intervening command overwrites it, including an echo. Save it as soon as you need it.
• Assuming a pipeline fails if the first command fails. $? reflects the last one. Use set -o pipefail (05-03).
• Modifying a script, moving it, and having Bash keep running the old version. That is the path cache: hash -r.
• Taking for granted that cron's environment is yours. It is not. Test your scripts with env -i before scheduling them.
• Tip: when something behaves inexplicably, ask yourself these three questions in order: what is this command really (type -a)?, is it running in a subshell?, what variables are in the environment (env)?. They cover most cases.
• Tip: bash -x script.sh shows each line after the expansions, before running it. It is the fastest way to see the process from section 1 in action. We will formalise it in 05-03.

Without running anything, predict what type of command each one is (alias, keyword, function, builtin or external) and then verify with type -a:

1. cd
2. grep
3. [[
4. export
5. for
6. awk

Also explain why which cd returns nothing.

This script is meant to count how many shipments with status issue there are in /srv/veloz/data/shipments.csv, but it always prints 0:

1. Explain exactly why it fails.
2. Fix it in two different ways.
3. State which of the two you would prefer in veloz-ops and why.

Answer by running whatever is needed:

1. Define a variable CITY="Bilbao" without exporting it and check whether a child bash -c sees it. Repeat, exporting it.
2. Run a command that returns code 127 and another that returns 2, and display both codes.
3. Run grep 'ERROR' /file/that/does/not/exist | wc -l and explain why $? is 0.
4. Check whether jq is installed on your system using command -v inside an if.

Solution to Exercise 1

which cd returns nothing because which is an external program that only walks the PATH directories looking for files. cd is not a file: it is code inside Bash itself. An external process cannot see its parent's builtins.

Solution to Exercise 2

1. Why it fails: the while loop is on the right-hand side of a pipeline, so Bash runs it in a subshell. That subshell gets a copy of issues, increments it correctly and dies when the loop ends. The main shell never sees the change, and the final echo reads its own variable, which is still 0. It is exactly the problem from section 5.2.

2. Fix A: process substitution (it removes the pipeline, the loop runs in the main shell):

Note that here you do not even need < <(...): since we only wanted to read a file, a direct < file redirection is enough. The original cat was unnecessary (the so-called useless use of cat).

Fix B: delegate the counting to a tool:

Or more precisely, checking exactly the fifth field with awk (Module 6):

3. Which to prefer: B, with awk. Reasons: it is a single line, it does not have the subshell problem, and it is far faster on large files. A while read loop in Bash processes a few thousand lines per second; awk processes hundreds of thousands. With an 11 MB shipments.csv the difference is seconds versus milliseconds. The general Bash rule: the shell is for orchestrating, not for processing data line by line. We will see it formalised in 08-02.

Solution to Exercise 3

Without export, CITY is a shell variable and is not part of the environment the child inherits. With export, it becomes part of the environment and travels in the copy.

127 is the conventional code for "command not found" and 2 for "incorrect usage".

$? is 0 because in a pipeline it reflects the status of the last command. grep failed with code 2, but wc -l got an empty input, counted zero lines and finished correctly. To detect the real failure:

The PIPESTATUS array stores the code of each element. The usual alternative in scripts is set -o pipefail, which makes the pipeline return the first non-zero code (05-03).

The > /dev/null 2>&1 redirection discards both the normal output and the errors: we only care about command -v's exit code, not its text. Redirections are studied in 02-04.

You now have the mental model that holds up everything that follows. You know that Bash splits the line, expands it, resolves the command in a strict order (alias → keyword → function → builtin → PATH) and, if it is external, does fork and exec to run it in a child. You understand why type -a is reliable and which is not, how the PATH cache works, and — the most profitable of all — why variables do not survive a subshell, which finally explains the behavior of pipelines, parentheses, source and cd itself. And you have had your first contact with $?, the piece on which the whole course's error handling will be built.

With that you close off the fundamental knowledge of the shell. One cross-cutting skill remains before moving on to the commands: being able to answer your own questions. No professional remembers all the options of find or the exact format of a crontab; what they know is where to look it up in ten seconds. In the next lesson, Finding Help: man, help and --help, you will learn to read manual pages, to tell when to use man and when help, and to verify dangerous commands before running them on srv-veloz-01.