We reach the module's milestone, and a promise we have been carrying since 04-02. daily-report.sh is today a robust program: it validates its environment, protects itself with flock, aborts with a diagnosis, validates text with regex and directs its output precisely. But it is also a single file of several hundred lines where log_info, die, validate_env and percentage coexist —functions that have nothing to do with reports and that any other script in the toolkit would need—. The temptation, when you write archive-history.sh next week, will be to copy and paste. And that is where the decay begins: two copies that diverge, a bug fixed in one and not in the other, three different logging conventions. In this lesson you turn that file into a project with an architecture.

1. source versus executing
2. What a library is in Bash
3. Locating the library without paths betraying you
4. Include guards
5. Names: what is public and what is private
6. Configuration files
7. Precedence order
8. The toolkit's final structure
9. lib/common.sh
10. daily-report.sh as an orchestrator
11. Testing the library by hand

In 03-01 you saw the ways to run a script. The one that matters now is the one that does not create a process: as opposed to ./script.sh or bash script.sh, which launch a new process from which nothing comes back, source script.sh (or its POSIX form . script.sh) runs the file in this shell.

The last two rows explain why a library is not a normal script. Since it is loaded with source, it needs neither an executable shebang nor execute permission. And above all: an exit in a library kills whoever loads it, including your interactive session if you were testing it; inside a library you use return. This is nothing new: it is the same mechanism by which ~/.bashrc defines functions your shell knows about (01-02). A library is a .bashrc for your scripts.

A library is a file containing definitions only: functions, constants and, at most, default values. Nothing that acts on its own.

The rule is that loading the library must be silent and without side effects. The set -euo pipefail inside a library is especially treacherous: when you source it, it applies to the shell that loads it, so it can change the error behavior of a script that was not expecting it. The strict policy is decided by the executable, not by the library. Conventions: .sh extension (or .bash), no executable shebang —optionally a #!/usr/bin/env bash as a hint for editors and for shellcheck, but without +x permission— and a comment header saying what it offers.

Here is the real technical problem. How does bin/daily-report.sh find lib/common.sh?

Relative paths are resolved from the current working directory, which is wherever the person launching the script happens to be, not where the script lives. Since ~/veloz-ops/bin is on the PATH (we put it there in 01-02), the normal thing is to invoke it from anywhere, and the source fails. The canonical idiom is this:

Taken apart from the inside out:

• ${BASH_SOURCE[0]} is the path of the file being executed. It is used instead of $0 for two reasons: $0 is bash when the script is invoked as bash script.sh, and inside a file loaded with source, $0 is still the main script's while BASH_SOURCE[0] is the library. For a library that needs to know where it is, $0 always gives the wrong answer.
• dirname gives the directory containing it (04-04) and /.. climbs to the project root; cd ... && pwd turns that path, which may be relative (./bin/..), into an absolute, normalized one; and it is all quoted, in case the path contains a space.

With that, the script works when invoked as ./bin/daily-report.sh, as ~/veloz-ops/bin/daily-report.sh, from the PATH or from cron with any working directory. And if you also want to follow symbolic links down to the real file, readlink -f (05-01) over ${BASH_SOURCE[0]} before the dirname resolves it.

If daily-report.sh loads common.sh and also report.sh, which in turn loads common.sh, the file is processed twice. Redefining functions is harmless, but a duplicated readonly causes a fatal error under strict mode (VELOZ_VERSION: readonly variable). The solution is the same idea as C's include guards:

A top-level return 0 in a file loaded with source is legal and means "stop reading this file". The :- is mandatory because the script loading it has set -u (05-03) and the variable does not yet exist the first time around. The leading underscore marks the variable as internal.

Bash has no namespaces: everything lives in a single global scope. If your library defines log() and your colleague's does too, the last one loaded silently wins. The remedy is a prefix:

Bash allows :: in function names and it reads very nicely, but it is not POSIX and some old tools choke on it; veloz_ is the safe option. Pick one and be consistent. The public/private distinction is by convention, the interpreter does not enforce it: the leading underscore is a promise that "this may change without warning, do not depend on it". What you really can control is variable scope: every variable inside a function goes with local (04-02), and that way it does not pollute the script using it.

~/veloz-ops/etc/veloz-ops.conf has existed since the start of the course with permissions 600. Its simplest form is a file of assignments (VELOZ_CSV=/srv/veloz/data/shipments.csv, VELOZ_LOG_LEVEL=info, VELOZ_CITIES="Valencia Sevilla Bilbao Madrid") loaded with [[ -r "$BASE_DIR/etc/veloz-ops.conf" ]] && source "$BASE_DIR/etc/veloz-ops.conf". It is convenient —it accepts comments, quotes, even $(...)— and it carries a risk you must be aware of: source executes arbitrary code. A line rm -rf ~ in that file runs with your permissions. That is why the 600 is not decorative: if someone can write to your configuration file, they can run whatever they want as you. The full implications are covered in 08-03.

The safe alternative is to parse key=value without executing anything, using what you learned in 05-04:

Three modules meet here: the regex with BASH_REMATCH from 05-04 (which additionally discards comments and empty lines by not matching), the ${v%\"} expansions from 04-04 and the while read loop from 04-01. The VELOZ_* filter stops the file from redefining PATH or HOME, and printf -v assigns to the variable whose name is in $key: the safe way of doing indirect assignment without eval.

Here we close the option design started in 03-05. When the same value can come from four places, you need an explicit and documented order, from lowest to highest priority:

The principle: the closer to the user and to the moment of execution, the higher the priority. Implementing it is simple if you respect the loading order, and : "${VAR:=value}" is the idiom for "assign only if it is empty or undefined" —the := assigns, unlike the :- which only substitutes, and the leading : is the null command— so that an already-defined environment variable survives:

The allocation rule in one sentence: bin/ holds what is invoked, lib/ what is reused, etc/ what changes between machines and logs/ what is generated. It is the same FHS logic you saw in 01-03, applied at project scale, and that is why it feels familiar to anyone opening the repository. A practical criterion for deciding where a function goes: if a script that had nothing to do with reports would need it, it goes in common.sh; if it only makes sense when talking about shipments and cities, it goes in report.sh.

Notice what is not there: no set -euo pipefail, no executable shebang, not a single line that acts when loaded beyond the guard and the constants. veloz_die is the only one that calls exit, and that is correct because it is designed to be used from an executable, not from an interactive session.

This is the result of five modules. The executable barely implements anything anymore: it locates its base, loads its libraries, resolves the configuration, processes options, protects itself with flock and a trap, and dispatches. All the reusable logic lives in lib/. When you write archive-history.sh, the first three lines will be identical and you will get logging, errors and validations for free.

A well-made library can be loaded in an interactive session, and that is what makes it verifiable:

declare -F lists the names of the defined functions and type shows the body (01-04). If any output appears when you source it, there is top-level code that should not be there. For real automated tests —assertions, edge cases, a results report— the tool is bats, in 08-06. And shellcheck -x follows the source calls so it analyzes the libraries too, in 08-05.

• source lib/common.sh with a relative path. It works in your directory and fails from cron or from the PATH.
• Using $0 to locate the script. It gives bash with bash script.sh and the wrong file inside a library. Use ${BASH_SOURCE[0]}.
• exit inside a library loaded into your shell. It closes your terminal. Use return.
• Top-level code in the library, and especially a set -euo pipefail: it pollutes the output and imposes its policy on the script loading it, perhaps without it expecting so.
• readonly without an include guard, which gives a fatal error when loaded twice, and functions without local, which clobber variables of the script calling them weeks later.
• Tip: document every function with a line # name — what it does. Usage: name <args> right above it. It is what you will read six months from now, and it is what lets you generate the toolkit's help with a simple grep '^# [a-z]' lib/*.sh.

Exercise 1. Write the complete header for archive-history.sh (in bin/) that locates BASE_DIR robustly, loads lib/common.sh, applies strict mode and fails with a clear message if the library is not where it should be.

Exercise 2. Add to common.sh a function veloz_config that returns the value of a configuration key respecting the precedence order (default < file < environment), with an include guard and without using eval.

Exercise 3. Split the toolkit: decide for each of these functions whether it goes in common.sh, in report.sh or in the executable itself, and justify it: veloz_log_info, cities_table, veloz_percentage, accumulate_day, usage, veloz_validate_date.

Solution 1.

The check before the source matters: without it, the message would be source: lib/common.sh: No such file or directory, which does not say where it looked. Code 78 (EX_CONFIG, from the table in 05-03) is the right one: the installation is wrong, not the data. And the message uses a direct printf because veloz_log_error does not exist yet.

Solution 2.

${!key} is indirect expansion: it obtains the value of the variable whose name is in $key. It is the read-side equivalent of the printf -v from section 6, and both avoid eval, which would execute the contents of the configuration file —exactly the risk we are running from—. The order of the three branches is the precedence table.

Solution 3.

The criterion is a single question: would a script that does not talk about shipments need it? If the answer is yes, it goes in common.sh. usage is the interesting case: even though every script has one, its content is different in each, so what is shareable is not the function but the convention that it exists.

source runs a file in the current shell, and everything else follows from that: the functions and variables it defines stay available, no shebang or execute permission is needed, and an exit would kill whoever loads it —inside a library you use return—. A library contains definitions only, with no code that acts, no set -euo pipefail imposing someone else's policies and no messages when loaded. It is located with BASE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)", never with $0 nor with relative paths, because a script gets invoked from any directory and from the PATH. A guard [[ -n ${_COMMON_SH:-} ]] && return 0 prevents double loading and the duplicated-readonly error, and a veloz_ prefix stands in for the namespaces Bash does not have, with the leading underscore marking what is private. The configuration lives in etc/veloz-ops.conf with permissions 600 —because loading it with source executes whatever it contains— or is parsed as key=value with BASH_REMATCH and printf -v, and it resolves its conflicts with an explicit precedence: default < file < environment < command line. The result is ~/veloz-ops/ with bin/, lib/, etc/ and logs/, and a daily-report.sh that no longer implements: it orchestrates.

With this Module 5 closes, and with it the part of the course devoted to what Bash can do on its own. The Veloz Envíos toolkit is today robust, safe with its files and processes, able to diagnose its own failures, precise when validating text, master of its input and output, and modular. But it is still clumsy in one particular area: every time it has to do column arithmetic over the CSV, it chains cut, sort, uniq and loops that read the file several times; every time text has to be rewritten, the tools are missing; and when veloz-api stops being a process you check with pgrep and becomes an API you have to query, Bash alone does not reach.

In Module 6 in come the external tools that multiply what Bash can do: awk, which processes columns and aggregates in a single pass what today costs you twenty lines (06-01); sed, for transforming text in a stream with the regexes you already master (06-02); the commands for interrogating the system —disk, memory, users, hardware— (06-03); the networking tools and the /dev/tcp we left noted (06-04); and curl with jq to talk to veloz-api and handle real JSON, which is where your toolkit will stop reading files and start integrating with the rest of the system (06-05).