You already know who you are and which machine you are on. What remains is the question that completes your orientation: where you are and what is around you.

When you open the file explorer on Windows you see C:, perhaps D:, and a Program Files folder where software is installed. On Linux none of that exists. There is a single tree growing from / and absolutely everything hangs from it: programs, configuration, your documents, the disks, and even information that is not on any disk at all, such as the processor's temperature.

This lesson gives you the map. By the end, when you see a path like /var/log/tramontana/access.log you will read it as a sentence with meaning, and when you have to decide where to put a new piece of software you will not do it on instinct but with the standard in hand. That is the difference between a server anyone can administer and a server where things are "wherever so-and-so put them".

Contents

  1. A single tree versus Windows drives
  2. What mounting means
  3. The FHS standard: a tour of the directories
  4. Static and variable, shareable and unshareable data
  5. Everything is a file: the seven types
  6. The pseudo-file systems /proc and /sys
  7. Hidden files and the dot convention
  8. Absolute and relative paths
  9. Where each piece of Tramontana goes and why

  1. A single tree versus Windows drives

The structural difference from Windows runs deep; it is not a matter of naming.

Windows Linux
Model Several trees, one per drive A single tree from /
Root C:\, D:\, E:\... / and only /
Separator Backslash \ Slash /
A new disk Appears as a new letter Is mounted on a directory of the tree
Software C:\Program Files\ Distributed according to its nature
Case Insensitive Always sensitive

On Windows, the structure of the storage is reflected in the paths: if a file is on the second disk, its path starts with D:. If that disk changes letter tomorrow, every path changes.

On Linux, the path describes what the file is, not where it physically sits. /var/log/tramontana/access.log means "a log file belonging to the Tramontana application". Whether it is on the first disk, the second, an LVM volume or a network share is irrelevant to whoever uses it and can change without any program noticing.

That independence between logical structure and physical layout is one of the best ideas in the Unix design, and the mechanism that makes it possible is called mounting.

  1. What mounting means

Mounting is connecting the contents of a storage device to a directory of the tree. That directory is called the mount point, and from the moment it is mounted, entering it means entering the device.

Look at it on your server:

df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda2        23G  6.4G   15G  30% /
/dev/sda1       1.1G  6.4M  1.1G   1% /boot/efi
tmpfs           1.9G     0  1.9G   0% /dev/shm
tmpfs           389M  1.3M  388M   1% /run

How to read it: the partition /dev/sda2 is mounted on /, and /dev/sda1 on /boot/efi. When you access a file inside /boot/efi, you are reading from the second device, even though from your point of view you have merely entered a subdirectory.

flowchart TD
    R["/ (root)<br/>partition /dev/sda2"]
    R --> ETC["/etc"]
    R --> VAR["/var"]
    R --> HOME["/home"]
    R --> BOOT["/boot"]
    BOOT --> EFI["/boot/efi<br/>partition /dev/sda1 (FAT32)"]
    R --> SRV["/srv"]
    SRV --> BK["/srv/tramontana/backups<br/>could be a separate disk"]
    R --> MNT["/mnt"]
    MNT --> NAS["/mnt/nas<br/>could be a network share"]

    style EFI fill:#e8f0fe,stroke:#4285f4
    style BK fill:#e8f0fe,stroke:#4285f4
    style NAS fill:#e8f0fe,stroke:#4285f4

The practical consequences of this model are enormous:

  • You can add a new 2 TB disk and mount it on /srv/tramontana/backups. The backup scripts carry on writing to the same path as always and never notice a thing.
  • You can mount a network share on /mnt/nas and work with it as if it were local.
  • You can isolate risks: if /var is on its own partition, a runaway log does not fill the root file system, as we saw in lesson 01-04.

Mounting in depth, the /etc/fstab file and automatic mounting are lesson 05-04.

  1. The FHS standard: a tour of the directories

The FHS (Filesystem Hierarchy Standard) is the document that defines what goes in each directory. It is maintained by the Linux Foundation and followed by every significant distribution.

Its value is enormous and very concrete: thanks to it, an administrator arriving at a Rocky Linux server knows the configuration is in /etc without having to look for it. The FHS is what makes Linux predictable.

Look at the root of your server:

ls /
bin   dev  home  lib64  media  opt   root  sbin  srv  tmp  var
boot  etc  lib   lost+found  mnt  proc  run   sbin.usr-is-merged  sys  usr

Full reference table

Directory Name Contents Do you touch anything here?
/ root The origin of the tree Not directly
/bin binaries Essential commands for all users No
/sbin system binaries Administration commands (they require root) No
/lib, /lib64 libraries Shared libraries and kernel modules No
/usr Unix System Resources Most of the installed software Not by hand
/usr/bin Non-essential user commands No
/usr/sbin Non-essential administration commands No
/usr/local Software compiled and installed manually Yes
/usr/share Architecture-independent data: documentation, icons Rarely
/etc et cetera All the system configuration, in plain text Yes, a lot
/var variable Data that changes: logs, queues, caches, databases Yes
/var/log The system and application logs Yes, a lot
/home Users' home directories Yes
/root The home directory of root (not to be confused with /) Rarely
/opt optional Self-contained third-party software Yes
/srv service Data served by this server Yes
/tmp temporary Temporary files; wiped on reboot Yes, in passing
/boot Kernel, initramfs and the GRUB boot loader With care
/dev devices Device files: disks, terminals, ports For reference
/proc processes Pseudo-file system: processes and kernel information Read only
/sys system Pseudo-file system: devices and kernel parameters With great care
/run Runtime data: PIDs, sockets. In RAM No
/mnt mount A temporary, manual mount point Yes, occasionally
/media Automatic mounting of removable media: USB, CD Automatic

The ones you will use most

Five directories account for 90 % of your work. They are worth detailing.

/etc — the configuration. Everything that configures the system and its services is here, and almost always in plain text. It is the most valuable directory on a server: if you back up /etc, you can rebuild the machine's configuration.

ls /etc | head -12
adduser.conf
apt
crontab
fstab
group
hostname
hosts
netplan
nginx
passwd
shadow
ssh

Every name is a piece you will study: passwd and group (users, lesson 05-01), fstab (mounts, 05-04), crontab (scheduled tasks, 03-07), ssh (remote access, 06-02), netplan (networking, 06-01).

And one implication of it being plain text that is worth appreciating now: /etc can be versioned with Git. Many administrators do exactly that, and so they have the complete history of configuration changes on their servers, with who changed what and when.

/var — what grows. Variable: data that changes during normal operation.

Subdirectory Contents
/var/log System and application logs
/var/lib Persistent state of services (databases, for example)
/var/cache Caches that can be deleted without losing data
/var/spool Queues: printing, mail, cron jobs
/var/tmp Temporary files that survive a reboot, unlike /tmp
/var/www Web content served by Apache or nginx

It is the directory an administrator watches most closely, because it is the one that fills up.

/home — the users. Every user has their own: /home/operator, /home/student. It is the only place where a normal user can write freely, and where /home/operator/scripts will live.

/opt and /usr/local — software that does not come from the package manager. There is a fine distinction here that confuses many people:

/opt /usr/local
Philosophy Each application in its own subdirectory, self-contained The system structure is replicated (bin, lib, share)
Structure /opt/tramontana/ with everything inside /usr/local/bin/program, /usr/local/lib/...
Typical of Commercial software, deployed applications Software compiled from source
Uninstalling Delete one directory Track down the scattered files

/srv — served data. It is the least used and the worst understood. The FHS defines it as the data this server serves to third parties: websites, FTP, shares, backups. The difference from /var/www is that /srv is meant to be organised by service or by client, and many distributions leave it empty so that the administrator can structure it.

The visual tree

flowchart TD
    ROOT["/"]

    ROOT --> BIN["bin, sbin, lib<br/>→ links to /usr"]
    ROOT --> USR["usr<br/>system software"]
    ROOT --> ETC["etc<br/>configuration"]
    ROOT --> VAR["var<br/>data that changes"]
    ROOT --> HOME["home<br/>users"]
    ROOT --> OPT["opt<br/>third-party software"]
    ROOT --> SRV["srv<br/>served data"]
    ROOT --> VIRT["proc, sys, dev, run<br/>virtual / in RAM"]
    ROOT --> BOOT["boot<br/>booting"]

    USR --> UB["bin · sbin · lib"]
    USR --> UL["local<br/>compiled by hand"]
    USR --> USH["share<br/>docs, icons"]

    VAR --> VL["log<br/>LOGS"]
    VAR --> VLIB["lib<br/>service state"]
    VAR --> VC["cache · spool"]

    HOME --> HO["operator"]
    OPT --> OT["tramontana/app"]
    SRV --> ST["tramontana/backups"]
    VL --> VLT["tramontana/<br/>access.log · errors.log"]

    style OT fill:#fff4e5,stroke:#f59e0b
    style ST fill:#fff4e5,stroke:#f59e0b
    style VLT fill:#fff4e5,stroke:#f59e0b

A note on /bin, /sbin and /lib

If you list the root in detail you will see something curious:

ls -l / | grep -E '^l'
lrwxrwxrwx  1 root root    7 Apr 22 13:08 bin -> usr/bin
lrwxrwxrwx  1 root root    7 Apr 22 13:08 lib -> usr/lib
lrwxrwxrwx  1 root root    9 Apr 22 13:08 lib64 -> usr/lib64
lrwxrwxrwx  1 root root    8 Apr 22 13:08 sbin -> usr/sbin

That leading l and the arrow indicate that they are not real directories but symbolic links to their equivalents inside /usr. This is the so-called usr merge, completed in Ubuntu and in most modern distributions.

Historically, /bin contained what was essential for booting and repairing the system (because /usr could be on another partition or even on the network and unavailable early in the boot), and /usr/bin held the rest. Today initramfs handles early boot and that separation no longer adds anything, so it was unified. The links are kept so that the old paths in thousands of scripts keep working.

It is a good example of how the FHS evolves while preserving compatibility.

  1. Static and variable, shareable and unshareable data

The FHS is not an arbitrary list of names. Behind it there are two axes of classification, and understanding them lets you deduce where each thing goes instead of memorising it.

Shareable
(several machines can use the same ones)
Unshareable
(specific to each machine)
Static
(does not change without intervention)
/usr, /opt /etc, /boot
Variable
(changes by itself, in operation)
/var/mail, /home /var/log, /var/run, /proc

The two axes:

  • Static versus variable. Static is what only changes when an administrator installs or modifies something: the programs in /usr, the configuration in /etc. Variable is what changes by itself during operation: the logs in /var/log, the queues, the caches.
  • Shareable versus unshareable. Shareable is what it would make sense to serve over the network to several machines: the binaries in /usr are identical on every server running the same distribution. Unshareable is what is specific to one machine: its network configuration, its hostname, its logs.

Why this is genuinely useful

Four practical consequences that come up constantly in administration:

  1. It determines what you back up and how often. /etc (static, unshareable) is small and extremely valuable: copy it daily. /usr (static, shareable) does not need backing up, it is reinstalled with the package manager. /var/log (variable) is backed up or rotated according to the retention policy.

  2. It determines what can be mounted read-only. On hardened servers, /usr is mounted read-only because it must not change during operation. Any attempt to write there is a sign of compromise. /var and /etc, by contrast, must be writable.

  3. It determines what needs its own partition. Variable data grows unpredictably: that is why /var deserves its own partition, as we saw in lesson 01-04. Static data has a bounded size.

  4. It determines where to put your own things. And here is the direct application: the Tramontana application is static (it only changes when Luis deploys a new version), its configuration is static and unshareable, and its logs are variable and unshareable. Three different natures demanding three different locations. We wrap that up in section 9.

  1. Everything is a file: the seven types

You already saw the principle in lesson 01-02. Now it is time to see it in action.

On Linux, almost everything is presented with the same file interface, so that the same syscalls (open, read, write, close) and the same tools work for radically different things.

There are seven types, and ls -l identifies them with the first letter of each line:

Letter Type What it is Example
- Regular file Data: text, binaries, images /etc/hostname
d Directory A container for other files /home
l Symbolic link A pointer to another path /bin → usr/bin
c Character device Read and written byte by byte, unbuffered /dev/null, /dev/tty1
b Block device Accessed in blocks, buffered /dev/sda
s Socket Communication between processes /run/systemd/private
p Named pipe (FIFO) A data channel between processes Uncommon

Check it:

ls -l /dev/null /dev/sda /etc/hostname /home /bin
lrwxrwxrwx 1 root root       7 Apr 22 13:08 /bin -> usr/bin
-rw-r--r-- 1 root root      15 Aug 18 09:02 /etc/hostname
drwxr-xr-x 4 root root    4096 Aug 18 08:41 /home
crw-rw-rw- 1 root root  1,   3 Aug 18 09:02 /dev/null
brw-rw---- 1 root disk  8,   0 Aug 18 09:02 /dev/sda

Notice two revealing details:

  • The first letter of each line is the type: l, -, d, c, b.
  • On the device lines, where ordinary files show the size, two numbers appear (1, 3 and 8, 0). They are the major and minor numbers: the major identifies which kernel driver manages the device, and the minor identifies which of them it is. A device has no size because it contains no data: it is a door to a driver.

The special devices

Three files in /dev that you will use constantly.

/dev/null — the black hole. Anything you write there disappears; reading it returns end of file immediately.

echo "this is lost" > /dev/null
cat /dev/null

There is no output. Neither operation produces anything.

Its usefulness is a daily matter: discarding output you do not care about. You already used it without knowing in lesson 01-04, when you verified the ISO with sha256sum -c SHA256SUMS 2>/dev/null to discard the error messages for the images you had not downloaded. Redirection is lesson 03-04.

/dev/zero — the infinite source of zeros. Reading it returns zero bytes without limit. It is used to create fixed-size files or to wipe data.

head -c 20 /dev/zero | od -c | head -2
0000000  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
0000020  \0  \0  \0  \0

head -c 20 takes 20 bytes and od -c displays them readably: twenty zeros.

/dev/sda — the whole disk. It is not a metaphor: it is the disk. You can read it byte by byte and, if you are root, write to it. It is what let dd write an ISO to a USB stick in lesson 01-04, and also what makes that command so dangerous.

/dev/random and /dev/urandom generate cryptographically secure random data, and they are the basis of the key generation you will see in Module 6.

Why this principle matters

This list may look like a curiosity, but it has a consequence that runs through the whole course: the same tools work for everything.

cat can display a text file, the processor's information (/proc/cpuinfo) or the bytes of a disk. grep can search in a log, in another program's output or in a kernel pseudo-file. Redirecting a program's output to a file, to a device or to another program is exactly the same operation.

When you chain tools with pipes in Module 3, that uniformity is what makes it possible. On a system where disks, processes and the network each had their own API, you would need different programs for each thing.

  1. The pseudo-file systems /proc and /sys

/proc and /sys are the most spectacular application of the previous principle. They take up not a single byte on disk: they are virtual file systems the kernel generates in memory at the moment you read them.

Every time you open /proc/meminfo, the kernel builds that "file" on the fly with the current state of memory. It is a query interface disguised as a file system.

/proc /sys
Origin The 1990s, inherited from Unix 2002, kernel 2.6
Contents Processes + general kernel information Devices, drivers and kernel parameters
Organisation Historical, somewhat untidy Hierarchical and systematic
Typical use Diagnostics and information Configuring hardware and parameters

/proc in practice

Processor information:

grep -m1 'model name' /proc/cpuinfo
model name	: Intel(R) Core(TM) i5-10400 CPU @ 2.90GHz

grep -m1 shows only the first match, because the file repeats the information for every core.

Memory information:

head -4 /proc/meminfo
MemTotal:        4014520 kB
MemFree:         2698412 kB
MemAvailable:    3352108 kB
Buffers:           78124 kB

Here is the interesting fact: this is exactly where free -h gets its numbers from. free is nothing more than a program that reads /proc/meminfo and presents it readably. The same goes for uptime and /proc/uptime, or for ps and the /proc/<pid>/ directories.

Verifying this changes the way you see the system: diagnostic tools have no special powers, they simply read files that you can read too.

One specific process. Every running process has a directory in /proc named after its PID:

ls /proc/1/
cmdline  cwd  environ  exe  fd  limits  maps  mounts  root  stat  status  task

/proc/1/ is process 1, that is, systemd. What is inside:

File Contents
cmdline The command line it was launched with
cwd A link to its current working directory
environ Its environment variables
exe A link to the binary it is running
fd/ The files it has open right now
status State, user, memory used, threads
sudo cat /proc/1/cmdline | tr '\0' ' '
/sbin/init splash

(The tr '\0' ' ' replaces the null characters separating the arguments with spaces, to make it readable.)

Other useful files in /proc:

Path Contents
/proc/version The full kernel version
/proc/uptime Seconds of uptime
/proc/loadavg Average load (the source for uptime)
/proc/mounts Mounted file systems
/proc/partitions Detected partitions
/proc/sys/ Kernel parameters modifiable on the fly

/sys and modifying on the fly

/sys exposes devices and kernel parameters, and many of its files are writable. Changing one modifies the kernel's behaviour instantly, without rebooting.

cat /proc/sys/net/ipv4/ip_forward
0

That 0 means IP packet forwarding is disabled: the server does not act as a router. Enabling it would be as simple as writing a 1 into that file.

This mechanism is what makes kernel tuning possible, and it is the subject of lesson 07-03. It is mentioned here only so that you understand the nature of what you are looking at.

A clear warning: /proc and /sys are safe to read — do it fearlessly, it is the best way to understand what your system is doing. Writing to them without knowing what you are doing can degrade performance or destabilise the machine. During this module, stick to looking.

  1. Hidden files and the dot convention

On Linux, a file whose name begins with a dot is hidden. There is no special attribute as on Windows: it is just a naming convention that the tools respect.

ls ~
scripts
ls -a ~
.   .bash_history  .bashrc   .profile  .ssh     scripts
..  .bash_logout   .cache    .local    .sudo_as_admin_successful

-a stands for all. Many more items appear.

The first two are special and will always be present in any directory:

  • . is the current directory.
  • .. is the parent directory.

They are not decoration: they are real directory entries, and they are the basis of the relative paths in the next section.

The usual hidden files in your home directory:

File Contents
.bashrc Bash configuration for interactive sessions: aliases, prompt, functions
.profile Runs at login: environment variables
.bash_history Command history (the one you scroll through with the arrows)
.ssh/ SSH keys and configuration
.cache/, .local/ Caches and data of the user's applications

The purpose of the convention is purely practical: personal configuration is stored in the user's directory, but it must not get in the way when they list their documents. Programs put their configuration behind a leading dot and it disappears from view.

An important detail that already follows from the table: a hidden directory can contain perfectly visible files. .ssh is hidden, but ls -a ~/.ssh shows its contents normally. Hiding affects the item, not what is inside it.

And an operational warning: when backing up a home directory, you must make sure to include the hidden files. It is a classic source of incomplete backups in which all the configuration and the SSH keys are lost.

  1. Absolute and relative paths

There are two ways of naming a file, and the distinction is as simple as it is fundamental.

Absolute path: it starts at the root / and describes the complete route. It is the same wherever the person writing it happens to be.

/var/log/tramontana/access.log
/home/operator/scripts/backup.sh
/etc/tramontana/app.conf

Relative path: it starts at the current directory. Its meaning depends on where you are.

scripts/backup.sh         ← from /home/operator
../log/tramontana/        ← from /var/lib
./backup.sh               ← the file backup.sh in the current directory

The symbols used in paths:

Symbol Meaning
/ (at the start) The root: the path is absolute
/ (in the middle) A separator between directories
. The current directory
.. The parent directory
~ The current user's home directory
~operator The home directory of operator
- The previous directory (with cd)

When to use each one:

Situation Recommendation
Scripts and cron jobs Always absolute. A script does not control where it is run from
Configuration files Absolute
Interactive work Relative, they are more convenient
References inside a project Relative, so the project is portable

The first row is the one that prevents the most grief. A script using the relative path logs/access.log works when you test it from your own directory, and fails mysteriously when cron runs it from somewhere else. It is one of the most frequent mistakes when starting out with automation, and we will come back to it in lessons 03-07 and 04-07.

The practical use of paths — moving around with cd, knowing where you are with pwd, listing with ls — is lesson 02-03. Here you only needed the definition.

  1. Where each piece of Tramontana goes and why

The moment has come to apply the whole map. Luis Ferrer is ready to deploy Tramontana Bookings on srv-tramontana and asks you where to put each thing. The answer is not an opinion: it follows from the FHS and from the static/variable classification.

Piece Location Nature FHS justification
Application /opt/tramontana/app Static, shareable /opt is for self-contained third-party software packages, outside the package manager
Configuration /etc/tramontana/ Static, unshareable /etc is the configuration specific to this machine
Logs /var/log/tramontana/ Variable, unshareable /var/log is the place for logs; it grows during operation
Backups /srv/tramontana/backups Variable, shareable /srv is for data the server serves or holds in custody
Scripts /home/operator/scripts Variable, unshareable The administrator's personal tools

Let us go one by one, because the reasoning matters more than the conclusion.

/opt/tramontana/app — the application

Tramontana Bookings does not come from the Ubuntu repositories: it is in-house software that Luis deploys manually. The FHS reserves /opt for exactly that, with the convention /opt/<vendor>/<product>.

It is static because it only changes when a new version is deployed, not during operation. And it is shareable: if there were a second application server tomorrow, it would have exactly the same content at that path.

An additional advantage of the self-contained nature of /opt: uninstalling means deleting one directory, and backing up the application means copying one directory.

A rejected alternative: /usr/local. It is correct for compiled software that integrates into the system, spreading binaries into /usr/local/bin and libraries into /usr/local/lib. For a self-contained web application, /opt is cleaner.

/etc/tramontana/ — the configuration

This is where the database connection string, the application's parameters and its credentials go. /etc is, by the FHS definition, static configuration specific to this machine.

That last part is the key, and it is what justifies separating the configuration from the application instead of leaving it inside /opt/tramontana/app/config:

  • The test server and the production server run the same code with different configuration. Separating them allows the same artefact to be deployed to both.
  • A backup of /etc automatically picks up the configuration of every service, ours included.
  • When the application is updated (by deleting and recreating /opt/tramontana/app), the configuration is not at risk at all.
  • The configuration contains secrets and needs restrictive permissions, different from those of the code. Permissions are lesson 02-07 and secrets are 06-05.

/var/log/tramontana/ — the logs

access.log and errors.log are the perfect example of variable data: they grow continuously without anyone intervening.

Putting them in /var/log is not a formality. It has very concrete consequences:

  • It is where the log rotation tools (logrotate) expect to find them, so they can compress and delete the old ones automatically. Without that, a log grows until it fills the disk.
  • It is where any administrator will look first when investigating a problem.
  • As we saw in lesson 01-04, /var can be on its own partition: if the logs run away, the root file system is unaffected.

A rejected alternative: leaving them inside /opt/tramontana/app/logs, which is what many applications do by default. It is a classic mistake: it mixes static data with variable data, breaks automatic rotation, complicates backups and makes an overflow hit the wrong partition.

/srv/tramontana/backups — the backups

/srv is the directory for data this server serves or holds in custody, organised by service.

And there is a weighty operational reason to keep them separate: being in their own top-level directory, /srv/tramontana/backups can be mounted on a different physical disk with a single line in /etc/fstab, without touching a single path in the scripts. It is exactly the scenario from section 2 on mounting.

And that matters a great deal, because a backup on the same disk as the original data protects against accidental deletion, but not against disk failure. The complete backup strategy is lesson 05-08.

/home/operator/scripts — your tools

The automation scripts you will write in Module 4 are yours: your home directory is their natural place while you are developing them.

A professional nuance worth anticipating: when a script stops being an experiment and becomes part of operations — cron runs it, a backup depends on it — it should no longer live in a personal directory. Its place becomes /usr/local/bin or /opt/tramontana/bin, because if operator ceases to exist tomorrow or their directory is cleaned out, operations cannot depend on that. We will see it in lesson 04-07.

The result

This is how srv-tramontana looks once the deployment is complete:

/
├── etc/
│   └── tramontana/          ← configuration (static, for this machine)
│       └── app.conf
├── opt/
│   └── tramontana/
│       └── app/             ← the application (static, shareable)
├── var/
│   └── log/
│       └── tramontana/      ← logs (variable, they grow)
│           ├── access.log
│           └── errors.log
├── srv/
│   └── tramontana/
│       └── backups/         ← backups (could be another disk)
│           └── bookings.csv
└── home/
    └── operator/
        └── scripts/         ← your tools

Five directories, five different natures, five justifications. Any Linux administrator in the world would understand this structure without anyone explaining it. That is what following a standard gives you.

Common Mistakes and Tips

  • Confusing /root with /. / is the root of the tree; /root is the home directory of the root user. They sound alike and they are different things.
  • Looking for a Linux "Program Files". There is no single place: software is distributed according to its nature (/usr/bin for package binaries, /opt for third-party applications, /usr/local for hand-compiled things, /etc for their configuration).
  • Installing your own software in /usr/bin. That directory belongs to the package manager. Anything you put there by hand can be overwritten or removed in an update. Use /usr/local/bin or /opt.
  • Leaving the logs inside the application's directory. It breaks automatic rotation, mixes static with variable data and puts the wrong partition at risk.
  • Storing important data in /tmp. It is cleaned on reboot and, on many systems, periodically as well. If you need temporary files that survive, use /var/tmp.
  • Writing to /proc or /sys without understanding what it does. Reading them is safe and educational; writing to them can destabilise the system instantly.
  • Forgetting hidden files when copying a home directory. You take the documents and leave behind the configuration and the SSH keys. ls -a before copying.
  • Using relative paths in scripts. They work when you test them and fail when cron runs them from another directory. In scripts, absolute paths.
  • Tip. When you are unsure where to put something, ask yourself two questions: does this change by itself during operation? (if so, it goes under /var) and is it specific to this machine? (if so, /etc for configuration). Those two questions settle most cases.
  • Tip. Spend a while exploring /etc, /var/log and /proc with ls and cat. There is no risk in reading, and familiarity with the terrain is gained by walking it. It is the best investment of time before starting Module 2.
  • Tip. The full FHS is published and is surprisingly readable. When you have a real doubt about where something goes, consulting it gives you an authoritative answer, not an opinion.

Exercises

Exercise 1

For each item, state which directory it should go in according to the FHS, and justify your answer using the static/variable and shareable/unshareable classification:

  1. The TLS certificate for the Tramontana Bookings domain.
  2. A PostgreSQL database holding the bookings.
  3. A script Marta runs every morning to generate a report.
  4. The file houses.txt with the catalogue of rural cottages that the application serves to the agencies.
  5. A 3 GB temporary file generated during a migration, which must survive a reboot.
  6. A hand-compiled version of nginx with special modules.

Exercise 2

You run ls -l in a directory and get this output:

drwxr-xr-x  3 operator tramontana  4096 Aug 18 09:14 reports
-rw-r--r--  1 operator tramontana 15243 Aug 18 09:12 bookings.csv
lrwxrwxrwx  1 operator tramontana    24 Aug 18 09:15 current -> /var/log/tramontana
brw-rw----  1 root     disk       8, 16 Aug 18 08:41 data-disk
crw-rw-rw-  1 root     root       1,  3 Aug 18 08:41 empty
srw-rw-rw-  1 root     root           0 Aug 18 08:41 app.sock
  1. Identify the type of each item and explain how you determined it.
  2. Why do two of them show two numbers where the others show a size? What do those numbers mean?
  3. What would happen if you deleted the item current? And if you deleted /var/log/tramontana?

Exercise 3

Luis Ferrer proposes this structure for deploying Tramontana Bookings:

/home/luis/tramontana/
├── app/
├── config/
│   └── app.conf
├── logs/
│   ├── access.log
│   └── errors.log
└── backups/

Argue, point by point, four concrete problems with this proposal, and present the alternative structure with its FHS justification. For each problem, describe a real scenario in which it would cause an incident.


Solutions

Solution to Exercise 1

1. TLS certificate → /etc/ssl/certs/ (or /etc/tramontana/ssl/). Static and unshareable. It is configuration: it only changes when it is renewed, and it is specific to this machine and its domain. It goes in /etc by definition. The associated private key must go in /etc/ssl/private/ with very restrictive permissions, because whoever obtains it can impersonate the server. It is covered in lesson 06-05.

2. PostgreSQL database → /var/lib/postgresql/. Variable and unshareable. It changes continuously with every booking; it is the persistent state of a service, which is exactly the definition of /var/lib. It is the location PostgreSQL uses by default on Ubuntu, and respecting it ensures that backup tools and SELinux or AppArmor policies expect to find it there.

3. Daily report script → /usr/local/bin/ (or /opt/tramontana/bin/). This one has a nuance. While it is being developed, /home/operator/scripts is fine. But the exercise says that Marta runs it every morning: it is an operational tool somebody else depends on. It should move to /usr/local/bin, which is the FHS place for local executables added by the administrator, with two advantages: it is in every user's PATH (Marta invokes it by name, with no path) and it does not depend on the operator account continuing to exist.

4. houses.txt served to the agencies → /srv/tramontana/. Variable and shareable. The key word in the statement is serves: it is data this server delivers to third parties, which is the literal definition of /srv in the FHS. If it were served exclusively over the web through nginx, /var/www/ would also be defensible; /srv is preferable when the data is served through several channels or conceptually belongs to the service.

5. A 3 GB temporary file that survives a reboot → /var/tmp/. Both requirements point to the same place. /tmp is ruled out because it is emptied on reboot (and on many systems it is a tmpfs in RAM, where 3 GB would be a serious memory problem). /var/tmp is designed precisely for temporary files that persist across reboots, and it is on disk.

6. Hand-compiled nginx → /usr/local/. Static and shareable. /usr/local is the parallel tree reserved for software the administrator compiles and installs manually: the binary in /usr/local/sbin/nginx, the configuration in /usr/local/etc/nginx/. Its whole reason for existing is not to collide with /usr, which belongs to the package manager: that way an Ubuntu update will never overwrite your build, and your build will never break the official package.

Solution to Exercise 2

1. Types, determined by the first letter of each line:

Item Letter Type
reports d Directory
bookings.csv - Regular file
current l Symbolic link (the -> arrow confirms it and shows the target)
data-disk b Block device
empty c Character device
app.sock s Socket

Details that confirm each reading: the link shows -> /var/log/tramontana and has rwxrwxrwx permissions (symbolic links always do; the ones that count are the target's). The block device belongs to the disk group, which is usual for disks. And empty with numbers 1, 3 is in fact /dev/null: those are its characteristic major and minor numbers.

2. The two numbers.

They are shown by data-disk (8, 16) and empty (1, 3), the two devices. On a normal file, that column shows the size in bytes; a device has no size because it contains no data: it is an access door to a kernel driver.

In its place appear:

  • The major number identifies the kernel driver managing the device. 8 corresponds to SCSI/SATA disks; 1 to memory devices, among which is /dev/null.
  • The minor number identifies which of the devices managed by that driver it is. 16 corresponds to /dev/sdb (0 would be /dev/sda); 3 to /dev/null specifically.

In other words, data-disk is the system's second disk and empty is the black hole.

3. Consequences of deleting.

If you delete current: nothing of consequence happens. A symbolic link is only a pointer, a tiny file containing a path as text. Deleting it removes the shortcut; the directory /var/log/tramontana and all its contents remain intact. You can recreate the link in a second.

If you delete /var/log/tramontana: you really do lose the logs. And on top of that the current link is left dangling: it still exists and still points to a path that no longer does. Any program trying to follow it will get a "No such file or directory" error, which is confusing because the link does appear when you list the directory.

Dangling links are easy to spot because ls shows them in flashing red on most colour terminals.

This asymmetry — the link depends on the target, the target does not depend on the link — is the essential characteristic of symbolic links, and it explains how they differ from hard links. That is the subject of lesson 02-06.

Solution to Exercise 3

Problem 1: everything hangs from a personal directory.

The entire production infrastructure depends on the luis account existing. /home is reserved for users' personal data, not for services.

Incident scenario: Luis leaves the company. The normal offboarding procedure is followed: remove the account with userdel -r, which deletes the home directory. The production application, its configuration, its logs and its backups are all deleted at once. Nobody connected offboarding an employee with destroying the service, because in a well-organised system there is no reason for a connection.

An equivalent and quieter incident: the permissions on /home/luis restrict access, and the web service user (www-data) cannot read the application. It ends up being solved with excessively open permissions on a personal directory, which opens a security hole.

Problem 2: the logs live with the application.

logs/ is inside the same tree as app/, mixing variable data with static data.

Incident scenario: a bug in the code causes a loop that writes to errors.log non-stop. The log grows until it fills the partition holding /home, which in many installations is the same as /. With the root disk full, the system cannot write temporary files: services start failing in cascade, journald cannot record anything and it may not even be possible to log in to diagnose it. With the logs in /var/log and /var on its own partition, the damage would have stayed contained.

A second effect: logrotate does not manage those logs, because they are not where it expects them. Nobody compresses or deletes them, so they grow indefinitely.

Problem 3: the configuration is inside the application.

config/app.conf hangs from the same tree as the code.

Incident scenario: Luis deploys version 2.0 by deleting and recreating the tramontana/ directory. app.conf is lost, along with the database connection string and the credentials. The service does not start and the configuration has to be rebuilt from memory, in the middle of an outage, under the pressure of having the service down.

An associated structural problem: not being separate, the same artefact cannot be deployed to test and to production with different configurations. You end up maintaining two divergent copies of the code, which is how the bugs that "only happen in production" appear.

Problem 4: the backups are in the same tree as the data.

backups/ is inside /home/luis/tramontana/, that is, on the same disk and under the same directory it protects.

Incident scenario: the disk fails. The original data and all the backups are lost together. A misdirected rm -rf on /home/luis/tramontana/ is equally enough: it takes out both the object of the backup and the backup itself.

A backup that shares a destination with the original protects only against the selective deletion of a file. It does not protect against hardware failure, or against a wide deletion, or against ransomware encryption, which are precisely the scenarios that justify having backups in the first place.

Alternative structure:

Piece Correct path FHS justification
Application /opt/tramontana/app Self-contained third-party software; static and shareable
Configuration /etc/tramontana/app.conf Static configuration specific to this machine; backed up with /etc; survives deployments
Logs /var/log/tramontana/ Variable data; managed by logrotate; can be isolated on its own partition
Backups /srv/tramontana/backups Data held in custody by the server; mountable on a different disk with one line in /etc/fstab

How to present it to Luis (because being right is not enough): the argument that convinces is not "the FHS says so", but the four incident scenarios. The correct structure is not more bureaucratic: it is the one that means offboarding an employee does not take the service down, that a runaway log does not fill the root disk, that a deployment does not delete the credentials and that the backups are worth something when the disk fails. Besides, setting it up properly from the start costs exactly the same as setting it up badly.

Conclusion

With this lesson you close Module 1 and you have the complete map of the territory:

  • Linux organises storage in a single tree from /, not in separate drives, and mounting is what connects devices to points on that tree, separating the logical structure from the physical layout.
  • The FHS defines what goes in each directory and is what makes any Linux system predictable: /etc for configuration, /var for what grows, /usr for system software, /opt for third-party applications, /srv for what is served, /home for users.
  • The static/variable and shareable/unshareable classification is not theory: it determines what you back up, what you mount read-only, what needs its own partition and where you put your own things.
  • Everything is a file, and ls -l tells you which type by its first letter: -, d, l, c, b, s, p. Devices such as /dev/null, /dev/zero and /dev/sda are doors to kernel drivers.
  • /proc and /sys take up no disk: they are windows onto the kernel generated on the fly, and they are the source that tools such as free, uptime and ps drink from.
  • Hidden files are only a naming convention, and . and .. are real entries in every directory.
  • Absolute paths start at / and are the ones you must use in scripts; relative paths start wherever you are and are convenient when working by hand.
  • And you can justify, standard in hand, why Tramontana Bookings goes in /opt/tramontana/app, its configuration in /etc/tramontana/, its logs in /var/log/tramontana/, its backups in /srv/tramontana/backups and your scripts in /home/operator/scripts.

Take stock of where you were when the module started and where you are now. You know what Linux is and what exactly the kernel is; you know the Unix philosophy that explains the design of everything you will see; you understand the distribution landscape and why Ubuntu Server 24.04 LTS is the right choice for srv-tramontana; you have the server installed, verified, updated and with snapshots; you can read the prompt, run basic commands and carry out reconnaissance on a machine; and you have the complete map of its file system. That is a working lab and a shared vocabulary with any systems administrator.

So far you have been looking. In Module 2: Basic Commands you will start working for real: you will learn to handle the command line fluently, to consult the system documentation so you do not depend on search engines, to navigate the directory tree you have just got to know, to create, copy, move and delete files, to view and edit their contents, to handle hard and symbolic links, and to master permissions and ownership, which is the piece separating a server that works from a server that is also secure. Prepare your snapshot, open a session on srv-tramontana and I will see you in the first lesson.

Linux Course: From Beginner to System Administrator

Module 1: Introduction to Linux

Module 2: Basic Linux Commands

Module 3: Advanced Command-Line Skills

Module 4: Shell Scripting

Module 5: System Administration

Module 6: Networking and Security

Module 7: Advanced Topics

Module 8: Practical Projects

© Copyright 2026. All rights reserved