In the previous lesson you learned to move files around; now you are going to learn to read what is inside them and ask them questions. This is by far the highest-return lesson of the module: the 40 minutes a day the Veloz Envíos team loses producing the manual report go almost entirely into opening app.log by hand, counting errors with a finger and looking for cities in shipments.csv with the editor's search box. By the end of this lesson you will do each of those things in a second, and with the tools you see here you will already be able to answer real questions about the state of the server.
Contents
- The Unix paradigm: plain text in streams of lines
- Viewing whole files:
cat,tac,nl - Viewing large files:
less,head,tailandtail -f - Counting with
wc - Searching with
grep - Extracting columns with
cut - Sorting and counting:
sortanduniq - Transforming with
trand presenting withcolumn
- The Unix paradigm: plain text in streams of lines
Before a single command, understand the idea that makes all of this fit together. In Unix, almost everything is represented as plain text organized in lines: the system configuration, the application logs, the output of commands and the business data. And there is a set of small programs, each one an expert in one line transformation, designed to be combined with each other.
The three practical consequences:
- A line is a record. Counting lines is counting events; filtering lines is filtering events.
- The programs know nothing about formats.
grepdoes not know what a log is, nor doescutknow what a CSV is. They work on lines and columns, and that is what makes them universal. - The output of one is the input of another. This is the principle behind pipes, which we will formalize in 02-04. Here we will use
|occasionally, and it is enough to read it as "pass the result to the next command".
We will work on the three Veloz Envíos files: app.log (DATE TIME [LEVEL] message, one event per line), access.log (combined format, one HTTP request per line) and shipments.csv (one shipment per line, with a header and , as the separator).
- Viewing whole files:
cat, tac, nl
cat, tac, nlcat (concatenate) dumps the contents of one or more files to the screen:
shipment_id,date,city,courier,status,amount E-10234,2026-08-03,Valencia,alopez,delivered,14.50 E-10235,2026-08-03,Sevilla,mgarcia,issue,22.00 E-10236,2026-08-03,Bilbao,jruiz,in_transit,9.75
Its name comes from its second function: concatenating several files into a single stream, useful for processing the whole archive at once with cat /srv/veloz/data/archive/2026/08/*.csv (wildcards are covered in 02-05). Options worth remembering: -n numbers every line, -b numbers only the non-empty ones and -A shows the invisible characters (tabs as ^I, end of line as $), essential when a file "looks fine" but a program rejects it.
Two close relatives: tac prints the lines in reverse order (very handy in logs, where the recent stuff is at the end; tac app.log | head -5 gives you the 5 most recent events starting with the last one) and nl, which numbers with configurable formatting.
Important warning: never cat a large file. A 200 MB app.log will flood your terminal for minutes. That is what less, head and tail are for.
- Viewing large files:
less, head and tail
less, head and tailless opens the file in a paginated viewer without loading it entirely into memory; it is the default tool for inspecting anything.
| Key | Action |
|---|---|
Space / b |
Page forward / backward |
g / G |
Go to the beginning / end of the file |
/text / ?text |
Search forward / backward; n next, N previous |
-N |
Show or hide line numbers |
F |
Follow the file live (like tail -f); Ctrl-C to leave the mode |
q |
Quit |
You will recognize these keys: they are the same ones as in man, because man uses less as its pager (lesson 01-05).
head shows the beginning and tail the end. Both use 10 lines by default and accept -n NUMBER:
head -n 3 /srv/veloz/data/shipments.csv # header + 2 records
tail -n 20 /var/log/veloz/app.log # the last 20 events
tail -n +2 /srv/veloz/data/shipments.csv # FROM line 2 on: the CSV without its headerThat third form, tail -n +2, is a trick you will use constantly: it discards the CSV header so the following commands only see data.
tail -f: watching the log live
The star option for operations. -f (follow) leaves the command open, printing every new line as it is written:
2026-08-03 11:04:15 [WARN] high latency in courier query (812ms) 2026-08-03 11:04:19 [ERROR] timeout connecting to payment gateway
You exit with Ctrl-C. This is the command you will have open in a window while you deploy a change to veloz-api. A better variant for logs that rotate is tail -F: if the file is renamed and a new one is created (daily rotation), -F follows the name and keeps showing the current log, whereas -f stays stuck on the old file.
- Counting with
wc
wcwc (word count) counts lines, words and bytes: those are the three numbers. -l (lines) is the one you will use 95 % of the time: it counts events, requests or records. There are also -w (words), -c (bytes) and -m (characters, which differs from -c when there are accents in UTF-8).
With wc -l < /srv/veloz/data/shipments.csv you get a bare 1247. That < is an input redirection (02-04) and makes wc print only the number, not the file name. Remember to subtract 1 for the header: there are 1246 shipments.
- Searching with
grep
grepgrep filters the lines that contain a pattern. It is the most used tool in all of Unix.
That prints, one per line, every log event whose text contains ERROR. The options you really need:
| Option | Effect | Real example |
|---|---|---|
-i |
Ignores upper/lowercase | grep -i valencia shipments.csv |
-n |
Shows the line number | Locating the event in the file |
-c |
Only counts the matching lines | grep -c ERROR app.log |
-v |
Inverts: shows the ones that do NOT match | Excluding the [INFO] noise |
-w |
Whole word | grep -w jruiz does not match jruizb |
-r / -l |
Recursive through a directory / only the names of matching files | grep -rl ERROR /var/log/veloz/ |
-A n / -B n / -C n |
n lines after / before / of context on both sides | Seeing what surrounded the error |
-F |
Literal pattern, no special characters | Searching for GET /api/envios?id=1.2 |
-o |
Prints only the matching part, not the whole line | Extracting values |
grep -c ERROR /var/log/veloz/app.log returns 37: thirty-seven errors today, a figure that used to be obtained by eyeballing.
2158-2026-08-03 09:12:42 [INFO] starting payment for shipment E-10188 2159-2026-08-03 09:12:43 [WARN] retry 1 of 3 to the gateway 2160:2026-08-03 09:12:44 [ERROR] timeout connecting to payment gateway 2161-2026-08-03 09:12:45 [INFO] payment marked as pending
Context is what turns a loose error into a diagnosis: the line with : is the one that matched, the ones with - are the context, and together they tell the whole story.
grep -v "\[INFO\]" /var/log/veloz/app.log | tail -5 # only what is not routine
grep -F "GET /api/envios?ciudad=Valencia" access.log | wc -l # requests to that routeAbout patterns: grep interprets the pattern as a regular expression, a matching language with special characters (., *, [, ^, $...). That is why the brackets of [INFO] have to be escaped above, and that is why -F —which disables all interpretation— is so useful when you are looking for a literal string with dots, slashes or question marks. Regular expressions in full are a topic in themselves and we will study them thoroughly in 05-04; with the options in this table and literal patterns you cover 80 % of the daily work.
- Extracting columns with
cut
cutcut trims each line and returns only the fields you ask for. For a CSV, -d defines the delimiter and -f the fields (starting at 1).
Let us recall the header of shipments.csv: shipment_id,date,city,courier,status,amount → fields 1 to 6.
Notice that the first line of the output is city: the CSV header is processed as just another record.
cut -d, -f3,5 /srv/veloz/data/shipments.csv # fields 3 and 5 (city and status)
cut -d, -f4-6 /srv/veloz/data/shipments.csv # range from 4 to 6
cut -c1-10 /var/log/veloz/app.log # by position: the log dateThat last use, -c, cuts by character position instead of by field, and it is very handy in fixed-format logs like app.log, where the first 10 characters are always the date.
Its big limitation: cut does not understand quotes. If a CSV field contained a comma inside quotes ("Valencia, centro"), the field count would go out of sync. For complex CSVs you will need awk (06-01) or specific tools; for the controlled format of Veloz Envíos, cut is perfect.
- Sorting and counting:
sort and uniq
sort and uniqsort sorts lines alphabetically by default:
| Option | Effect |
|---|---|
-n |
Numeric order (without it, 10 comes before 9) |
-r |
Reverse order |
-u |
Removes duplicates after sorting |
-t C |
Defines the field separator |
-k N |
Sorts by field N (-k3 by the third; -k6 -n by amount) |
Extract the column, drop the header and sort while removing repeats: there you have the list of cities in operation.
uniq collapses adjacent identical lines, and -c prefixes how many there were. That word, adjacent, is the key: uniq is almost always preceded by sort, because if the duplicates are not next to each other it will not detect them. The combination sort | uniq -c | sort -rn is probably the most useful in all of Unix:
Read it from left to right: extract the city → drop the header → sort to group → count each group → sort by number descending. You have just built a ranking in a single line. With sort -t, -k6 -rn on the full file you would instead get the highest-amount shipments.
- Transforming with
tr and presenting with column
tr and presenting with columntr (translate) replaces or deletes characters, one by one. It does not take files as arguments: it always reads from its input.
| Use | Command | What for |
|---|---|---|
| Change case | tr 'a-z' 'A-Z' |
Normalizing before comparing |
| Replace the separator | tr ',' '\t' |
Turning a CSV into a TSV |
| Delete characters | tr -d '\r' |
Cleaning up Windows line endings |
| Squeeze repeats | tr -s ' ' |
Collapsing multiple spaces into one |
The tr -d '\r' deserves attention: when a CSV has been edited on Windows, every line ends in \r\n, and that invisible \r makes apparently identical comparisons fail. If a filter finds nothing that "should be there", check the file with cat -A.
column -t aligns the output into columns so a human can read it:
shipment_id date city courier status amount E-10234 2026-08-03 Valencia alopez delivered 14.50 E-10235 2026-08-03 Sevilla mgarcia issue 22.00 E-10236 2026-08-03 Bilbao jruiz in_transit 9.75
-s, says that the input separator is the comma and -t generates the table. It is the finishing touch on any report a person is going to read.
With these eight tools you can already answer most questions. When you need to compute over the columns (summing amounts, averages, per-field conditions) or rewrite the text with rules, the next step up is awk and sed, two complete languages that take up Module 6.
Common Mistakes and Tips
cat-ing a huge log. Useless,headortail. If you already launched it,Ctrl-C.- Using
uniqwithoutsortin front. It only collapses adjacent lines; without sorting first, the counts come out wrong and with no error to warn you. - Sorting numbers with plain
sort. Alphabetical order puts100before9. For amounts or counters, always-n. - Forgetting the CSV header. It will show up as one more city called
city;tail -n +2removes it. - Fighting regular expressions without meaning to. If your pattern contains
.,[,?or/, usegrep -F. And intail, remember that for logs that rotate at midnight the correct option is-F, not-f. - Counting with
grep -cbelieving it counts matches. It counts matching lines: two occurrences on the same line add up to one. To count occurrences,grep -o pattern file | wc -l.
Exercises
Exercise 1 — Quick diagnosis of the day. On /var/log/veloz/app.log: (1) count how many events there are in total; (2) how many are at ERROR level; (3) show the 5 most recent events that are not [INFO]; (4) show the first ERROR in the file with the 2 preceding and 2 following lines.
Exercise 2 — Issue ranking by city. On /srv/veloz/data/shipments.csv, build a single line that shows how many issues there are per city, sorted from most to fewest. Explain each step.
Exercise 3 — Readable courier report. Get the list of distinct couriers that appear in the CSV and, separately, the number of shipments for each one presented as an aligned table.
Solutions
Solution to Exercise 1
wc -l < /var/log/veloz/app.log # 1
grep -c ERROR /var/log/veloz/app.log # 2
grep -v "\[INFO\]" /var/log/veloz/app.log | tail -5 # 3
grep -n -C 2 ERROR /var/log/veloz/app.log | head -5 # 4In (3), grep -v discards the routine and tail -5 keeps the most recent, because logs grow at the end. In (4), -C 2 adds two lines of context on each side and head -5 trims it to the first block (2 before + the match + 2 after). The -n numbers the lines so you can locate them afterwards with less -N.
Solution to Exercise 2
Step by step: grep issue keeps only the lines whose status is issue (and along the way removes the header, which does not contain that word); cut -d, -f3 extracts the city; sort groups the repeated ones; uniq -c counts each group; sort -rn sorts by the number descending. A stricter filter would be grep ',issue,' to make sure the word is in the status field and not inside another field.
Solution to Exercise 3
# Distinct couriers → alopez, jruiz, mgarcia
cut -d, -f4 /srv/veloz/data/shipments.csv | tail -n +2 | sort -u
# Shipments per courier, as an aligned table
cut -d, -f4 /srv/veloz/data/shipments.csv | tail -n +2 | sort | uniq -c | sort -rn | column -tHere column -t goes without -s, because the output of uniq -c is separated by spaces, not commas: -t detects spaces by default. It is a good reminder that every command in the chain changes the format, and the next one has to be adjusted accordingly.
Conclusion
You have just acquired the core of text processing in Unix: you know how to view files with cat, less, head and tail, watch a log live with tail -f, count with wc -l, filter with grep and its context and inversion options, extract columns with cut, sort and aggregate with sort and uniq -c, normalize with tr and present with column -t. Above all, you have seen the logic that binds them together: each program performs one line transformation, and chaining them answers questions that no single command can solve on its own.
Notice how much you have gained already: the issue ranking by city, which used to require opening the CSV and counting by hand, is now a one-liner that takes milliseconds.
Before chaining filters fluently there is a prior question to settle, and that is who owns each file and who can read or write it. In lesson 02-03 you will study permissions and ownership: why veloz-api can write to /var/log/veloz and your user cannot, how to make the first script in ~/veloz-ops/bin executable and how to protect with 600 permissions the configuration file that will soon contain credentials.
Bash Programming Course
Module 1: Introduction to Bash
- What Is Bash?
- Setting Up Your Environment
- Basic Command Line Navigation
- Understanding the Shell
- Finding Help: man, help and --help
Module 2: Basic Bash Commands
- File and Directory Operations
- Text Processing Commands
- File Permissions and Ownership
- Redirection and Piping
- Wildcards and Path Expansion
- History and Keyboard Shortcuts
Module 3: Scripting Fundamentals
- Creating and Running a Script
- Variables and Constants
- Basic Operators
- Conditional Statements
- Arguments and User Input
- Quoting, Expansion and Substitution
Module 4: Intermediate Scripting
- Loops in Bash
- Functions in Bash
- Arrays and Associative Arrays
- String Manipulation
- The case Statement and Interactive Menus
- Arithmetic and Numeric Calculations
Module 5: Advanced Scripting Techniques
- Advanced File Operations
- Process Management
- Error Handling and Debugging
- Regular Expressions
- Advanced I/O: Descriptors and Here-Documents
- Modular Scripts and Reusable Libraries
Module 6: Working with External Tools
Module 7: Automation and Scheduling
- Cron Jobs
- Automating Tasks
- Backup and Restore Scripts
- Monitoring and Logging
- Services and Timers with systemd
- Remote Automation with SSH
Module 8: Best Practices and Optimization
- Writing Readable Code
- Optimizing Bash Scripts
- Security Considerations
- Version Control with Git
- Static Analysis with ShellCheck and shfmt
- Automated Testing with Bats
- Portability: POSIX sh versus Bashisms
