We closed Module 3 with a promise: daily-report.sh would stop being a linear script and become a program with structure, and loops would be the first tool to arrive. Here they are. Until now, when you wanted the Valencia report and the Sevilla one, you ran the script twice; when you wanted to look at the four rotated app.log files, you copied and pasted the same pipeline four times, changing the name. A loop is exactly the opposite: you write the operation once and hand it the list of things to apply it to. In this lesson you will see the four ways of iterating that Bash offers, when to use each one, and the canonical pattern —with its traps— for reading a file line by line.

1. for over literal lists
2. for over globs and over "$@"
3. C-style for
4. while and until
5. Which one to choose: comparison table
6. Reading a file line by line: the canonical pattern
7. The pipeline and subshell trap
8. Reading a CSV splitting fields
9. break and continue, and their levels
10. Controlled infinite loops
11. Nesting, counters and cost

The simplest form of for walks a list of space-separated words, assigning each one to a variable:

It prints four lines, one per city. Breakdown of each piece:

• for city in ...: city is the control variable. It is not declared beforehand nor deleted afterwards: when the loop ends it keeps the last value (Madrid).
• The list after in is a list of words, not a string. Bash builds it by applying the expansions from 03-06: for n in {1..5} or for f in $(date +%F) are equally valid.
• do ... done delimit the body. The ; before do is only needed if they are on the same line.

Careful: quoting a string with spaces (for c in "$CITIES") produces a single element, not several. For real lists you use arrays (04-03).

Since globbing (02-05) happens before the command runs, a pattern turns into the list of files that exist. This is the correct idiom for walking files, and the reason why in 03-06 we insisted that you never write for f in $(ls):

The guard line matters: if no app.log.* exists, Bash leaves the pattern unexpanded and the variable will literally hold /var/log/veloz/app.log.*. The two fixes are the guard [[ -f ... ]] || continue or enabling shopt -s nullglob, which makes a pattern with no matches produce an empty list so the loop never runs.

With no in list, for implicitly walks the script's arguments, that is "$@":

Writing for city in "$@" is more explicit and always correct. Writing for city in $@ (unquoted) breaks with any argument containing spaces, for the reason you already know from 03-05.

When what you need is a numeric counter and not a list, Bash offers the three-arithmetic-expression syntax inherited from C:

Inside the double parentheses you are in arithmetic context: variables carry no $, they compare numerically with < and >, and i++ works. It is the same context you will see in depth in 04-06. The three expressions are initialization, continuation condition and increment, and any of them can be omitted (for (( ;; )) is an infinite loop).

Use it when the index matters; to walk values, for ... in reads better.

while repeats the body while a command returns exit code 0, and until repeats until it does. Key note that links back to 03-04: the condition is not a boolean expression, it is a command whose $? is evaluated.

The same loop with until drops the negation and reads better:

Rule of thumb: if your while starts with !, you probably wanted an until.

for answers "for each one of these"; while/until answer "while/until this happens".

This is probably the Bash snippet you will write most often in your life. Memorize it whole:

Every piece is there for a specific reason:

• IFS= empties the field separator for this command only (remember from 03-06 that VAR=value command affects only that command). Without it, read would strip leading and trailing spaces and tabs from each line. With IFS= the line arrives literal, with its indentation intact.
• -r disables backslash interpretation. Without -r, a path like C:\veloz\data would lose its backslashes and a trailing \ would join two lines. Unless you are implementing an escape interpreter, always -r.
• line is the target variable. If you supply none, read drops the content into REPLY.
• < file at the end: the redirection applies to the whole loop, which acts as a single compound command. read consumes that input line by line until it runs out, at which point it returns a non-zero code and the while ends.

A little-known warning: if the file does not end in a newline, read stores the last line but returns a failure code and the loop discards it. The safe version is while IFS= read -r line || [[ -n "$line" ]]; do ... done < file.

It is tempting to write the previous loop feeding it from a pipeline. It works… until you try to use a variable afterwards:

The explanation comes straight from 01-04: each stage of a pipeline runs in a subshell, a child process with its own copy of the variables. The loop really does increment errors, but in the child's copy; when the child dies, the value dies with it and the parent still sees its original 0.

The two correct fixes:

The < <(command) construct is called process substitution: it turns a command's output into something you can redirect as if it were a file, without creating any subshell for the loop. Note the mandatory space between the two <.

read accepts several variables and distributes the fields according to IFS. Setting IFS=, gives you a one-line CSV reader:

Output: E1043 Sevilla mgarcia 31.20, one line per issue. Three important details:

• The initial read -r _header consumes the shipment_id,date,... line. Since it is inside the same { ... } < file block, it shares the input with the loop.
• If there are more fields than variables, the last variable receives all the rest. That is why amount, being the last one, keeps whatever is left over. And if there are fewer, the trailing variables end up empty.
• This reader does not understand quotes or commas inside fields. For complex CSV you use awk (06-01); for ours, it is perfect.

break exits the loop; continue jumps to the next iteration. Both accept a number saying how many nesting levels they affect:

break with no number is the same as break 1 and would only leave the inner loop. Levels are counted from the inside out. Use them sparingly: a break 3 is a sign that the block is asking to become a function with a return (04-02).

A loop with no exit condition is a legitimate tool when the script is a watchdog:

true is a command that does nothing and always returns 0, so the condition never fails. Safety rules:

• Always a sleep inside. Without it you will burn a CPU at 100% for nothing.
• Always a way out: a break under some condition, or at least let Ctrl+C kill it (by default it does).
• If the loop is going to live forever, the right answer is not a hand-launched while true but a system timer. You will see that in 07-04 and 07-05.

Nesting loops multiplies the work: 4 cities × 3 couriers is 12 iterations, and if inside you run grep over a 50,000-line CSV, that is 12 full reads. The real cost in Bash is almost never the loop, but the external processes launched inside it: a for over 4 cities with two grep each means 8 processes and 8 passes over the file.

A single pass accumulating counters is much faster, but for that you need one counter per city: that means arrays (04-03), and the percentage with decimals means arithmetic (04-06). For now, the simple counter, which you can already write:

_ is a conventional variable name for "I don't care about this field"; it has no special meaning in Bash, it is just a readable habit.

• for f in $(ls *.csv). Already flagged in 03-06, but it bears repeating: it breaks with spaces in names. Use for f in *.csv.
• Forgetting IFS= or -r in read. It will work for months and fail the day a path with \ or an indented line shows up.
• Counting inside a pipeline. The counter will always come out as 0. Redirection or < <(...), never cmd | while.
• Looping over a glob that does not match. Add the guard [[ -e "$f" ]] || continue or shopt -s nullglob.
• Modifying the list while walking it. The for list is computed once, at the start; deleting files inside the loop does not change what is left to walk.
• Tip: when a loop goes past about 15 lines, extract the body into a function. That is exactly what you will do in the next lesson.

Exercise 1. Write a loop that walks the four cities and, for each one, prints how many CSV lines belong to it, in the format Valencia: 128 shipments.

Exercise 2. Walk the rotated files /var/log/veloz/app.log.* and show, for each one, its name and the number of [ERROR] lines it contains, skipping those that do not exist or are empty.

Exercise 3. Read shipments.csv line by line (no header) and count how many shipments alopez made and how many of them ended in an issue. Print both numbers after the loop: they must come out correct.

Solution 1.

The pattern ,${city}, with commas on both sides avoids false positives if one city were a substring of another.

Solution 2.

-s (seen in 03-03) discards empty files and the unexpanded pattern in one go.

Solution 3.

The key is < <(tail ...): with tail ... | while both counters would come out as 0. Note also the || continue instead of a wrapping if: it removes one indentation level and reads just as well.

You now know how to repeat. for walks known lists —literals, globs or "$@"—, its C-style variant walks indexes, and while/until repeat while a command keeps returning the code you expect. The pattern while IFS= read -r line; do ... done < file is Bash's official file reader, and the rule that comes with it —redirection at the end, never a pipeline— will save you the most frustrating bug in the language, the one where a counter is always 0 because it lived in a subshell.

But notice what happened in the last examples: inside every loop we repeated the same five validation lines and the same output format, only with a different city. We have removed the repetition between invocations of the script, but not inside it. The tool that solves that is functions (04-02): named blocks of code, with their own arguments and their own variables, which will turn validate_env, count_errors and city_summary into pieces you write once and call many times.