health_check.sh already reads its configuration from the environment, but it is still a one-position button: it cannot explain itself, it does not accept "this time use threshold 60" without exporting a variable, and there is no way to tell it to be quiet. A script with no interface is a script only the person who wrote it can use, and this lesson gives it that interface. You are going to learn where information enters a script and where it must leave, and both questions have very concrete answers: arguments for what changes on each run, stdin for the data, stdout for the result and stderr for the human. That discipline is what makes a program composable, what makes it fit into a pipeline the way grep or sort do. By the end you will have health_check.sh with real options, and bookings_report.sh will be born.

1. The four input channels
2. Positional parameters
3. $@ versus $*: the difference that causes the most bugs
4. Validating arguments and the help function
5. Options: manual parsing and getopts
6. Command-line conventions
7. read: reading from the user and from a file
8. Output: stdout for the result, stderr for the human
9. Exit codes with meaning
10. Configuration files and --dry-run mode
11. Application: options in health_check.sh and the birth of bookings_report.sh

A script can receive information by four routes, and choosing badly is the first design decision that goes wrong.

The course's precedence rule, from strongest to weakest: arguments > environment > configuration file > default values. It is what any administrator expects: it lets you fix the stable part in a file and override it occasionally from the command line.

When you invoke script.sh alpha beta, Bash fills in a set of special variables:

shift discards the first argument and moves the rest along: $2 becomes $1 and $# decreases; shift N discards N in one go. It is the basic mechanism of any parsing loop. ${0##*/} — parameter expansion from 04-02 — gives the name without the path, which is what you want in the help text. And you can reassign the positionals with set -- "${@:-08}" to apply a default value; the -- is essential, because without it set would take any argument starting with - as one of its own options.

Just as with arrays in 04-02, the difference only appears inside double quotes, and that is where scripts break as soon as somebody passes a name with spaces.

"$@" is the only correct way to forward the arguments to another command: it produces exactly the ones you received, with their spaces intact. "$*" joins them into a string separated by the first character of IFS and is only good for printing. Unquoted, both are split and "august report.txt" becomes two files that do not exist. The rule, with no exceptions: "$@" to pass on, "$*" to display.

A serious script checks what it receives before doing anything, and it knows how to explain itself:

Note a detail almost nobody gets right, and one that separates a polite script from a rude one: if the user asks for help with -h, the help is the result and goes to stdout with code 0; if it is shown because they made a mistake, it is an error message and goes to stderr with a non-zero code. The difference matters: script.sh -h | less works in the first case, and script.sh 2>/dev/null does not silence the legitimate result in the second. The cat <<EOF is the here-document from 03-04: with no quotes around EOF, ${0##*/} is expanded.

The only way to accept long options (--threshold) in pure Bash, and the pattern is always the same:

Each branch consumes what belongs to it with shift (one if it is a switch, two if it carries a value) and the loop starts again. --threshold=60 is resolved with ${1#*=}, which trims up to the first =; -- marks the end of the options; anything starting with - that you do not recognise is a usage error, and anything else is the first positional and ends the parsing. (case is formalised in 04-04.)

When short options are enough for you, getopts is a builtin that does all the work, including grouping -vn as -v -n:

There are four pieces to understand:

• The string ":hvu:". Each letter is an option; a colon after a letter (u:) means "this one takes a value".
• The leading colon. It turns on silent mode: getopts stops printing its own messages and reports to you with \? (unknown option) and : (missing value), putting the offending letter in OPTARG. Always include it, or your script will mix its own messages with Bash's.
• OPTARG holds the value of the current option, and OPTIND the index of the next argument to process. The final shift $(( OPTIND - 1 )) discards the consumed options and leaves the first real positional in $1. If you forget that line, $1 will still be -v.

There is also getopt (without the s, GNU's external program), which does accept long options by reordering the arguments, but its syntax with eval set -- "$OPTIONS" is fragile and the macOS version is not compatible. In short: getopts if short options are enough, manual parsing if you need long ones, and getopt only if you know exactly what you are doing.

Respecting them is what lets somebody else use your script without reading it.

read takes a line from stdin and distributes it into variables. Its useful options:

That -s is the correct way to ask for a secret interactively: it is left neither on the screen nor in the history. In 04-07 we will see the alternative for unattended tasks.

while IFS= read -r line; do ... done < file is one of those lines worth memorising exactly as it is, because every piece is there for a specific reason:

• IFS= (empty, and only for this read) stops Bash trimming the leading and trailing spaces; without it, the indentation of a configuration file disappears. -r stops backslashes being interpreted as escapes, and < file after the done — not a pipeline — is the important bit, as we are about to see.

It went through 87 lines and the counter is 0. You have known the reason since 04-02: each stage of a pipeline runs in a subshell, so the while incremented its copy and that copy died when the pipeline closed. The same wall as with cd, in disguise. Three solutions, in order of preference:

Number 2 is the one you will use most. Number 3 has small print: lastpipe only works with job control disabled, which is the case in a non-interactive script but not in your terminal, so trying it by hand can mislead you.

The rule that turns a script into a piece of Unix: stdout is the result — what another program might want to process, data and nothing but data — and stderr is everything else: warnings, progress, errors, -v messages. If you mix them, my_script.sh | awk '{...}' chokes on your "Processing..." and my_script.sh > report.txt leaves the errors inside the report. The two functions that guarantee the discipline:

The messages appeared in the terminal and the file contains only the data: that is a composable script. We use printf instead of echo for the reason given in 04-02: defined behaviour and controlled formatting.

With the interface in place, the exit code stops being "0 or 1" and becomes executable documentation. The Tramontana convention borrows the values from sysexits.h:

The essential thing is not to adopt exactly these numbers, but to choose some, document them in the header and be consistent: a script whose 69 always means "the application is not responding" lets the caller act without reading its output.

There are two ways to read a configuration file, and one of them is dangerous. source /etc/tramontana/app.conf is convenient, but the file is code executed with your permissions: if somebody with write access to it adds rm -rf /, your script runs it. It is the same risk that left a credential exposed in a backup with 644 permissions. You can only live with source if the file is yours, with strict, verified permissions. The safe alternative is to parse it:

Only the keys the script knows about are accepted. It is more code, but it is the difference between reading data and running somebody else's code.

--dry-run mode is the other indispensable pattern, and it is implemented with a wrapper:

"$@" inside run rebuilds the command with its exact arguments — hence why section 3 mattered so much — and with DRY_RUN=1 it merely prints it. The rule from 03-03 thus becomes automatic: every script that deletes, moves or overwrites anything has a --dry-run.

The header of health_check.sh now documents the syntax and the codes (# Usage: health_check.sh [-h] [-v] [-u THRESHOLD], # Exit: 0 correct | 2 incorrect usage), and this is slotted in between the configuration from 04-02 and the body:

DISK_THRESHOLD can no longer be readonly: the -u option must override the environment value, which in turn overrides the default. It is the precedence from section 1, implemented.

The result went to the file and the trace to the terminal, exactly as section 8 promises. And notice the return 0 in log(): without it, when VERBOSE is 0 the function returns 1 and with set -e the script would die on the first call; it is one of the traps 04-06 pulls apart. Now for the new script Marta asked for, for her monthly report:

Two design details: : "${OUTPUT:=...}" uses the : builtin (which does nothing) purely to force the ${VAR:=value} default assignment from 04-02, and the script prints the path on stdout, so that filepath=$(...) captures it and the report is ready to be chained with mail or scp.

• Forgetting shift $(( OPTIND - 1 )). Failure number one with getopts: the options still occupy $1 and the script thinks it is missing arguments. Number two is using $@ or $* without quotes, whereupon any argument with spaces gets split.
• read without -r. A path with \ is silently corrupted; there is no case in which you want read without -r. And avoid while read loops at the end of a pipeline: the variables die with the subshell, so use < file or < <(command).
• Sending progress messages to stdout. You contaminate the result and break any pipeline: anything that is not the data goes to stderr. And getopts without the leading colon makes your script print messages you do not control.
• Putting a secret on the command line. Anybody with access to ps sees it, and it stays in ~/.bash_history. A file with 600 permissions or an environment variable; we close this in 04-07 and in 06-05.
• Tip: write the usage() function before the body; if you struggle to describe the interface, you have not decided it yet. And make -h always work, even with no arguments and with no valid configuration: it is the first thing anybody who finds your script tries.
• Tip: validate the arguments as soon as you have them and exit immediately if something does not add up. The sooner it fails, the less it has wrecked.

Exercise 1. Write ~/scripts/count_errors.sh, which reads from stdin a log in the format of access.log and accepts -c CODE (500 by default), -v and -h. It must print on stdout only the number of lines with that code and, with -v, how many lines it read in total, on stderr. access.log has 412 lines and 14 responses with code 500.

Exercise 2. Extend bookings_report.sh with -n/--dry-run, which shows which file would be written without creating it, using the run() wrapper. Explain why the parsing has to move from getopts to manual.

Exercise 3. Luis has written this to count how many distinct IPs there are in access.log and says "it always returns 0, must be something about the log": n=0; cut -d' ' -f6 access.log | sort -u | while read ip; do n=$((n+1)); done; echo "$n". Diagnose it and give two solutions.

Solution 1. Header and getopts as in sections 4 and 5, with ":hvc:" and CODE="$OPTARG". The interesting part is the body:

read with several names chops the line up by IFS without calling cut or awk: 412 processes saved. Here we do omit IFS=, because splitting into fields is precisely what we want.

Solution 2. You have to move to manual parsing because getopts does not accept long options: --dry-run would arrive whole and getopts would read it as -, -d, -r… You replace the while getopts with the while [[ $# -gt 0 ]] loop from section 5, adding the branch -n|--dry-run) DRY_RUN=1; shift ;;, and the wrapper does the rest:

The directory is still empty: nothing was created. And note that the redirection > "$OUTPUT" cannot go through run, because the shell resolves it before invoking the function; hence that case having its own if. It is the classic limitation of the pattern.

Solution 3. The diagnosis: the while read is the last stage of a pipeline, so it runs in a subshell; n is incremented in the copy and the echo in the main shell sees the original 0. It has nothing to do with the log.

The five known IPs (10.0.2.77, .31, .44, .52 and .18). Prefer B: if the loop only counts, you do not need the loop. A third route would be shopt -s lastpipe, but it does not work in your interactive terminal and you might conclude your fix is no good.

Your scripts now have an interface, and with it they stop being yours and become usable.

• You distinguish the four input channels and apply the precedence arguments > environment > configuration > defaults.
• You handle the positional parameters with $#, shift, ${10} and set --, and you know that "$@" is the only correct way to forward arguments while "$*" is only good for displaying.
• You write a usage() function that goes to stdout when it is asked for and to stderr when it is an error, and you parse options with getopts — option string, leading colon, OPTARG and the indispensable shift $(( OPTIND - 1 )) — or by hand with while and case for the long ones, respecting the -h, -v, -q, -n and -- conventions.
• You use read with -r always, and you know the canonical pattern while IFS= read -r line; do ... done < file, with the three solutions to the loop that loses its variables to the subshell.
• You separate stdout (the result) from stderr (the human) with log() and error(), you return documented exit codes, you read configuration by parsing it instead of with source, and you build in --dry-run with the run() wrapper.

But look at the code you have written today: it is full of if, of case and of [[ ]] used on intuition, without ever having studied them. It is time to put that right. In Control Structures you will see the idea that holds the whole of Bash up — that it branches on a command's exit code, not on a boolean — the three kinds of test and when to use each, the complete table of file, string and number operators, =~ with BASH_REMATCH for validating dates, case with its patterns and its three terminators, all the loops with the warning about why never for f in $(ls), and a real measurement over access.log that will tell you when a Bash loop is a thousand times slower than an awk. By the end of it, health_check.sh will be able to say OK, WARN or CRITICAL, and purge_releases.sh will walk /opt/tramontana/releases/ without touching the active version.