This lesson settles a debt we have been carrying since Module 2. When we introduced grep in 02-02 we said explicitly that its patterns are far more powerful than wildcards and that we were leaving them for later; when in 02-05 we compared globbing with other ways of matching text, we deferred it again. And on closing 05-03 it became clear where it hurts: the toolkit knows how to validate paths, processes and exit codes, but when it has to validate text —that --date 2026-08-03 really is a date, that an IP is an IP, that an app.log line has the expected structure— it falls back on fragile comparisons with globs. Regular expressions are the language that solves that, and with Bash's =~ operator they validate and extract in a single step.

1. Regex versus globbing
2. The three flavors: BRE, ERE and PCRE
3. Literals, the dot and the classes
4. POSIX classes
5. Anchors
6. Quantifiers and greediness
7. Alternation, grouping and backreferences
8. Escapes and backslashes inside quotes
9. Bash's =~ operator
10. BASH_REMATCH: validating and extracting at once
11. Real-world validations
12. Extracting with grep -o
13. When NOT to use regex

The two syntaxes look alike just enough to confuse, and they mean different things:

The two traps in the table cause nearly every beginner's mistake. First: in regex, * means nothing on its own; it modifies the element in front of it, so *.log is an invalid regex and what you want to write is .*\.log. Second: grep ERROR finds the word at any position in the line, whereas [[ $x == ERROR ]] demands complete equality. To demand the whole line in a regex you have to anchor: ^ERROR$.

There are three dialects, and the practical difference is which characters need a backslash:

• BRE (Basic): +, ?, {, |, ( and ) are literals; to use them as metacharacters you have to escape them (\+, \|).
• ERE (Extended): those characters are metacharacters directly. It is the comfortable dialect.
• PCRE (Perl style): ERE plus shortcuts (\d, \w, \s, \b), lazy quantifiers (*?) and much more.

The practical recommendation: use ERE whenever you can (grep -E, sed -E, awk, [[ =~ ]]). This whole lesson is written in ERE except where stated. grep -P is not available on every system —macOS does not have it, nor do some minimal containers— so avoid it in portable scripts.

Most characters represent themselves. The special ones are . [ ] ^ $ * + ? { } ( ) | \.

Inside the brackets almost everything loses its special meaning: [.*+] matches a literal dot, asterisk or plus sign. The three exceptions are ^ (if it comes first, it negates), - (if it sits in the middle, it forms a range: put it at the start or the end to make it literal) and ] (it must come first).

The . is the most common mistake when searching for extensions or IPs: grep -E '192.168.1.1' also finds 192x168y1z1. For a literal dot, escape it: 192\.168\.1\.1.

In grep -E '[[:digit:]]{4}-[[:digit:]]{2}' app.log, the names go between [: :] inside a pair of class brackets, hence the double pair:

Why prefer them to [0-9]? Because of locales. A range like [a-z] is interpreted according to the collation order of the configured language: in some locales it includes accented characters and in others it does not, and [A-z] (a frequent typo) additionally covers [, \, ], ^, _ and the backtick. [[:alpha:]] means "letter" in any locale, with no surprises, and on a server where LANG may change between deployments that is exactly what you want. For digits, [0-9] and [[:digit:]] are equivalent in practice; for letters, the difference is real.

^ and $ consume no characters: they mark positions. And \b marks a word boundary, the frontier between a word character and a non-word one: grep -E 'jruiz' also finds jruizperez, whereas grep -E '\bjruiz\b' narrows it to the exact courier.

Anchoring is also a matter of performance and of security: a validation without ^ and $ accepts garbage around what is valid, and that is how malformed data sneaks into a report.

Quantifiers are greedy: they consume everything they can and then back off just enough for the rest to fit. The practical effect shows up when extracting the path from an access.log line:

.* between double quotes matches up to the last quote on the line, not up to the first. The robust idiom is [^X]* —"anything that is not the delimiter"—, which does not depend on greediness. ERE has no lazy quantifiers (.*?); that is PCRE. In ERE, [^"]* is the answer.

Alternation has the lowest precedence in the whole language, so ^ERROR|WARN$ means "starts with ERROR, or ends in WARN", which is almost never what you want. Parentheses fix it and, in addition, they capture the matched fragment for reuse: \1 is the content of the first group, \2 that of the second. Groups are numbered by the order of their opening parentheses.

Backreferences are expensive and not every tool supports them in every flavor (awk does not have them), but their real value appears in section 10, where Bash exposes them as an array, and in sed (06-02), where they let you rewrite text while keeping pieces of the original.

This is where more time is lost than anyone would admit. There are two levels of interpretation: first the shell processes the quotes, and whatever survives reaches the regex.

Golden rule: regex patterns always go in single quotes. Inside them nothing is interpreted, and what you write is exactly what the tool receives. The only exception is when you need to interpolate a variable —grep -E "^[0-9]+,[^,]+,$city," shipments.csv— and in that case it is best to build the pattern in a separate variable.

The characters that need escaping to be literal are . [ ] ^ $ * + ? { } ( ) | \. And the backslash itself is written \\ in the regex, which inside shell double quotes becomes \\\\: one more reason to use single quotes.

Bash ships with a built-in ERE engine, available only inside [[ ]]: if [[ "$report_date" =~ ^[0-9]{4}-[0-9]{2}-[0-9]{2}$ ]]; then .... Two syntax rules, and both are counterintuitive:

1. The pattern is NOT quoted. If you write [[ "$x" =~ "^[0-9]+$" ]], the quotes turn the pattern into a literal string and it will only match if $x contains exactly those characters. It is the most frequent mistake with =~, and it produces no warning at all: it simply never matches.
2. The string on the left DOES go in quotes, as always, to protect it from word splitting (03-06).

If the pattern is complex or contains spaces, store it in a variable and use the variable without quotes —it is the readable, portable way to keep everything under control—:

Besides, =~ is not anchored by default: [[ abc123 =~ [0-9]+ ]] is true. To validate, always anchor with ^ and $.

This is why =~ deserves a section of its own. After a successful match, Bash fills the BASH_REMATCH array: position 0 is the whole matched text and the following ones are the captured groups, in order.

Compare it with the alternative: one cut for the date, another for the time, a tr -d '[]' for the level and a ${line#* } repeated four times for the message. Here, a single check validates the format and extracts the four fields, without launching a single external process. In a loop over an app.log with a hundred thousand lines, the difference is measured in minutes.

The spaces in the pattern are escaped (\ ) because, inside [[ ]], an unquoted pattern is subject to word splitting. Storing it in a variable avoids that noise:

BASH_REMATCH is overwritten by every successful =~ and is not cleared when one fails, so copy what you need immediately after the check, inside the if.

Note the design of RE_DATE: it is not satisfied with [0-9]{2} for the month, it demands 01-12 with the alternation (0[1-9]|1[0-2]). Even so it does not validate the date, only its shape: 2026-02-31 passes. To know whether the date exists you need date -d "$f" &>/dev/null, which you already know from 04-06. The general rule: the regex validates the format; the semantics are checked separately.

RE_OCTET shows the other principle: build it in pieces. Writing the IPv4 regex in one go is illegible; composing it from an octet is obvious. And that is how you debug, incrementally: try ^[0-9]{4} first, then add the month, then the day. Tools like regex101.com explain each element and show the captures live, and in the terminal a grep -oE over a sample file is enough to see what really matches.

When what you want is not the line but the fragment:

That last pair hides a nuance that ruins reports: grep -c counts lines containing at least one match; if a line has three, it still counts 1. To count matches you have to use -o and count the output lines.

grep -o is the pipeline tool; BASH_REMATCH is the loop tool. If you are already walking the file line by line with while IFS= read -r (04-01), =~ avoids launching one process per line.

Regexes are a regular language, and there are structures that are not regular. Insisting on them produces code that works with the examples and fails in production:

• CSV with commas inside quotes. "Sevilla, East" breaks any cut -d, and any naive regex. Veloz Envíos' CSVs are simple and that is why IFS=, is enough; the day they stop being simple, the tool is awk with a well-defined separator (06-01) or a real parser.
• HTML and XML. Arbitrary nesting: it is not a regular language. There is a legendary Stack Overflow answer about this, and it is right.
• JSON. Same reason. The tool is jq, and it arrives in 06-05.
• Paths and file names. You already have find (05-01) and the ${s##*/} expansions (04-04).

And a performance warning: patterns with nested quantifiers such as (a+)+ can cause catastrophic backtracking and hang the process with a short input. If your regex needs nested quantifiers, there is almost always a simpler formulation.

• Quoting the =~ pattern. It becomes a literal and never matches, without any warning.
• Using double quotes in a grep pattern. The shell eats the backslashes. Single quotes.
• Forgetting to escape the dot. 192.168.1.1 matches 19216811 and plenty more.
• *.log as a regex. * modifies the previous element; you write .*\.log.
• Not anchoring a validation. [[ $x =~ [0-9]+ ]] accepts abc123def as a number.
• ^ERROR|WARN$ without parentheses. Alternation has the lowest precedence and splits the whole expression.
• Trusting grep -c to count matches. It counts lines. Use grep -o | wc -l.
• Tip: store every regex in a readonly RE_SOMETHING variable with a descriptive name, near the top of the script. It documents itself, gets reused, can be tested separately and the quoting noise disappears.

Exercise 1. Write validate_date() that accepts only YYYY-MM-DD with month 01-12 and day 01-31, and additionally verifies that the date really exists (rejecting 2026-02-31). It must return 0 or 1 without printing anything.

Exercise 2. Walk /var/log/veloz/app.log with a single loop and produce a count by level (INFO, WARN, ERROR), also showing the time and the message of the first ERROR of the day, without launching external processes inside the loop.

Exercise 3. Extract from access.log the IPs that caused at least one 5xx code, along with how many times, sorted from highest to lowest.

Solution 1.

Two layers: the regex discards anything that does not even have the shape of a date —cheap, no processes— and date -d resolves what the regex cannot know, such as leap years. The order matters: if date came first, it would accept inputs like next friday.

Solution 2.

It is the counter pattern from 04-03 fed by BASH_REMATCH. The || continue turns the validation into a filter that protects the count from junk lines, and everything happens inside Bash: not one cut, not one grep, not one process per line.

Solution 3.

The first grep selects the lines whose code comes after the quoted request —the space and the quote keep it from being confused with a byte count starting with 5— and the second extracts only the leading IP. The (\.[0-9]{1,3}){3} shows that a group can also be quantified. The finishing touch sort | uniq -c | sort -rn is the Module 2 idiom, which now fits with the precise extraction that only regexes provide.

A regex describes text, not file names, and its metacharacters resemble globbing's just enough to deceive: * modifies the previous element, . is the single-character wildcard and .* is the equivalent of the shell's *. Of the three flavors, ERE is the sweet spot and it is spoken by grep -E, sed -E, awk and Bash's =~. The syntax is built from literals, ., [...] classes with their negation [^...], POSIX classes [[:digit:]] immune to the locale, anchors ^, $ and \b, quantifiers *, +, ? and {n,m} —greedy, hence the [^"]* idiom—, alternation | with minimal precedence and groups ( ) that capture. Patterns always go in single quotes, except in =~, where the pattern goes unquoted and the best move is to store it in a readonly variable. And there lies the jewel: BASH_REMATCH turns one check into an extraction of all the fields at once, with no processes launched, ideal for long loops. What you must not do with regex is parse CSV with quotes, HTML or JSON: for that there are specific tools arriving in Module 6.

daily-report.sh now validates its --date and breaks app.log apart in a single step. What remains to be solved is the output: today it prints to the screen with loose printf calls, and if you want the report in a file you have to redirect the whole script from outside. It does not know how to write to two places at once, nor keep a log file open while it works, nor compose a multi-line report template without twenty printf calls. In 05-05 in come file descriptors, here-documents, here-strings and process substitution: the input/output machinery that turns the toolkit's output into something you can direct with precision.