You use an operating system every single day, but you have probably never seen one. What you see is the desktop, the terminal or the browser; the operating system is the invisible layer underneath that makes those programs possible in the first place. In this lesson you are going to understand what specific problem an operating system solves, why writing software without one would be unbearable, and which abstractions it invented so that today you can open a file with a single line of code without knowing whether it lives on an SSD, on a spinning disk or on a remote server. You will also meet Meteora, the fictional company whose Linux server will accompany us throughout the course: every concept you learn will be applied to its meteo-01 machine.

1. Welcome to Meteora: the setting for the whole course
2. The problem: bare hardware and programs that want to coexist
3. The OS's first role: the extended machine
4. The OS's second role: the resource manager
5. A minimal review of the hardware the OS manages
6. The fundamental abstractions
7. Anatomy of a system: kernel, libraries, shell and applications
8. What an operating system is NOT
9. How each layer supports meteo-api

Meteora is a (fictional) company that sells weather data. It has hundreds of stations deployed that measure temperature, humidity and pressure, and it offers that data to its customers through an HTTP API. Its entire infrastructure lives on a single Linux server called meteo-01, and the service is made up of three pieces that will reappear in every module:

Meteora's conventions are fixed for the whole course; it is worth memorizing them now because we will use them without explaining them again:

• Data: /var/lib/meteora/readings/, with one file per day, for example 2026-08-31.dat.
• Configuration: /etc/meteora/meteora.conf.
• Logs: /var/log/meteora/meteo-api.log.
• System user and group: meteora:meteora.
• Recurring data structure: Reading, with the fields station_id, timestamp, temperature, humidity and pressure.

In C, that structure is declared like this:

Let's go through the block field by field, because we will reuse it many times:

• #include <stdint.h> gives access to types with a guaranteed size. uint32_t takes exactly 4 bytes on any machine, whereas an ordinary int might take 2, 4 or 8 depending on the compiler. When you write binary data to disk or send it over the network, the exact size matters.
• station_id is an unsigned integer: there are no stations with a negative identifier, so we take advantage of the full range.
• timestamp is an int64_t because an int32_t would overflow in 2038 (the famous year 2038 problem).
• temperature, humidity and pressure are float (4 bytes). Given the precision of a real sensor, roughly 7 significant digits are more than enough.

In total, the structure takes up 24 bytes (4 + 8 + 4 + 4 + 4 = 24, with no extra padding on x86-64 thanks to the 8-byte field ending up aligned). With 500 stations sending one reading per minute, that is 720,000 readings a day, about 17 MB per day. Remember these numbers: we will use them to reason about memory and disk.

Imagine that meteo-01 had no operating system. The ingestor program boots on its own, with direct access to the hardware. To store a reading on disk it would have to:

1. Know which disk controller model the machine carries.
2. Write specific values into that controller's registers to indicate sector, head and number of blocks.
3. Busy-wait until the disk confirms the operation.
4. Interpret the manufacturer's specific error codes.

And all of this for just one disk model. If tomorrow Meteora changes servers, the program has to be rewritten. On top of that, if aggregator wanted to run at the same time, both programs would write to the same disk without coordination: they would overwrite each other's sectors and corrupt the data.

From here come the two fundamental problems an operating system solves:

• Complexity: hardware is difficult, heterogeneous and unpleasant to program.
• Sharing: several programs want to use resources at the same time that are limited and, many of them, indivisible.

Those two problems correspond to the OS's two classic roles.

An operating system is, above all, a virtual machine nicer than the real one. It takes ugly hardware and offers on top of it a clean, uniform and stable interface. That is what is called the extended machine or hardware abstraction.

Compare the two ways of storing a reading on meteo-01:

In practice, Meteora's program writes a reading like this:

What each line does and why:

• open(...) asks the operating system for access to a file by its name. The OS translates that name into a physical location; the program never sees sectors.
  • O_WRONLY: we are only going to write.
  • O_CREAT: if the day's file does not exist yet, create it.
  • O_APPEND: every write is added at the end. This matters: it guarantees that two writes will not overwrite each other even when several processes are writing.
  • 0640: the file's permissions if it has to be created (read and write for the meteora user, read for the meteora group, nothing for everyone else). We will look at them in detail in File Security and Permissions.
• open returns a file descriptor (fd): a plain integer the OS uses as the identifier of that open connection. It is a pure abstraction: it does not exist physically.
• write(fd, &r, sizeof(r)) hands 24 bytes to the kernel. The program does not know (and does not care) whether they end up on an NVMe SSD or on network storage.
• close(fd) releases the descriptor.

That handful of lines works exactly the same on a laptop, on a server and on a Raspberry Pi. That portability is the product the operating system sells.

The other half of the job is sharing out. On meteo-01, ingestor, aggregator and meteo-api coexist, plus the system's own processes, and they all want the same CPU, the same memory and the same disk.

The operating system shares out resources in two complementary ways:

• Multiplexing in time: the users of the resource take turns. This is what happens with the CPU. If meteo-01 has 4 cores and there are 40 processes, the OS gives each one a few milliseconds and alternates so fast that it looks simultaneous. This is studied in CPU Scheduling.
• Multiplexing in space: the resource is divided into chunks and each user keeps one. This is what happens with RAM and with the disk. Each process receives its share and cannot touch anyone else's; you will see it in Memory Management.

Managing is not only sharing out: it also involves protecting (making sure meteo-api cannot read aggregator's memory), accounting (knowing how much CPU each process consumes) and arbitrating conflicts (deciding who writes first when two processes ask for the disk at the same time).

An example with numbers: if aggregator starts its hourly computation and consumes 100% of a core for 20 seconds, without a resource manager meteo-api would stop responding to customers. With the OS scheduler, both make progress: meteo-api gets the CPU back as soon as a network request arrives, because the scheduler favors processes that spend a lot of time waiting for input/output.

You do not need to be an electronics engineer, but you do need to know the pieces the OS administers, because almost every design decision in an operating system is explained by some characteristic of the hardware.

The CPU executes instructions in a perpetual cycle: fetch the next instruction, decode it and execute it. To do its work it has a handful of ultra-fast internal stores, the registers:

• General-purpose registers (on x86-64: rax, rbx, rcx…): they hold intermediate data.
• Program counter (rip): the address of the next instruction.
• Stack pointer (rsp): where the top of the stack is.
• Status register (rflags): it includes, among other things, the mode bit that tells whether we are in user mode or kernel mode. That bit is the basis of all system protection and we will study it in User Mode, Kernel Mode and System Calls.

A process's set of registers is its context. When the OS switches from one process to another, it saves all those registers and loads the next one's: that is the context switch.

There is no memory that is fast, large and cheap all at once, so several are stacked. These are typical orders of magnitude on a current server:

Reading this table is the key to almost the entire course: the further down the hierarchy you go, the more expensive the access, and by brutal proportions. When aggregator reads the day's file from disk, each access costs the equivalent of 600 RAM accesses. That is why the OS keeps a page cache in RAM with recently read data, and why the second hourly computation for the same day is much faster than the first.

Everything that is neither CPU nor RAM connects through buses (PCIe, USB, SATA). Each device has a controller (the chip that drives it) and a software driver (the OS code that knows how to talk to that chip). When a device finishes its work, it notifies the CPU with an interrupt: a signal that makes the CPU drop what it was doing and attend to the device. The details are in Drivers, Interrupts and I/O Operations.

An operating system is not a catalog of loose functions: it is a handful of well-chosen abstract ideas. These four are the ones holding up everything else.

A process is a program in execution, with everything it needs to live: its code, its data, its stack, its open files, its user identity and its state. It is the abstraction of "a computer all to myself".

What we are looking at:

• ps -eo ... lists all processes (-e) showing only the columns we ask for (-o).
• pid is the process's unique identifier; user the owning user (here, meteora); comm the name of the executable.
• --sort=-%cpu orders from highest to lowest CPU consumption.
• Process 1 is always the first one the kernel starts; on a modern Linux it is systemd, which we will see in Services, Boot and systemd.

Notice that aggregator is at 98.7% CPU and even so meteo-api is still alive: that is the resource manager at work.

Every process believes it has all of the machine's memory to itself, starting at address 0. That illusion is called the address space, and it is what prevents a bug in ingestor from corrupting meteo-api's memory. Two processes can use address 0x400000 at the same time without colliding because the hardware translates each virtual address into a different physical one. That is the subject of Virtual Memory and Paging.

A file is a named sequence of bytes. Nothing more. It has no format imposed by the OS: the meaning is given to it by the application. Files are organized into directories, forming a single tree that starts at /.

UNIX pushed the file abstraction all the way: almost everything is represented as a file. A disk, a terminal, a random number generator or even information from the kernel itself is read and written with the same calls.

Explanation:

• The first letter of the permissions indicates the type: b is a block device (accessed in blocks, like a disk), c is a character device (a byte stream, like a terminal).
• The numbers 8, 0 and 1, 3 are the major and the minor: the major identifies which driver serves the device, the minor which specific unit.
• /proc/1099/status does not exist on any disk: the kernel generates it on the fly when you read it. Even so, it is read with cat like any other file. This is the power of the abstraction: one interface, a huge number of implementations.

When somebody says "the operating system", they usually mean a set of distinct layers. Telling them apart avoids an enormous amount of confusion.

Two important nuances:

• Only the kernel is indispensable. Everything else consists of ordinary programs that run without special privileges and that talk to the kernel through system calls.
• The shell is not part of the kernel. bash is a program like any other: you can replace it with zsh or fish without touching the operating system. That shows the interface is interchangeable, not fundamental.

Three frequent confusions worth defusing right away:

• The OS is not the graphical interface. GNOME, KDE or the Windows desktop are applications that run on top of the operating system. meteo-01 has no graphical interface installed at all and works perfectly: servers rarely carry one, because it consumes memory and increases the attack surface.
• The OS is not the distribution. Ubuntu, Debian, Fedora or Alpine are not different operating systems: they all use the same kernel, Linux. What changes is the set of packaged programs, the package manager and the configuration decisions. When somebody says "I installed another operating system" after moving from Ubuntu to Debian, strictly speaking they changed distribution.
• The OS is not the set of applications it ships with. A browser, an editor or a media player are distributed with the system for convenience, not because they are part of it.

A useful criterion for deciding whether something is part of the operating system: does it run in kernel mode? If the answer is yes, it is part of the OS in the strict sense. If not, it is just another program, however indispensable it may seem.

Let's close by putting it all together. A client requests GET /averages?station=118&day=2026-08-31. This is what each layer contributes:

1. Hardware: the network card receives the packet and raises an interrupt.
2. Kernel: the network subsystem reassembles the TCP request and makes it available on meteo-api's socket. The scheduler marks the process as ready to run and assigns it CPU.
3. Libraries: meteo-api calls glibc functions to read from the socket and to open the day's file; glibc translates those convenient calls into system calls.
4. Kernel again: it looks up 2026-08-31.dat in the file system, checks that the meteora user has read permission, serves the blocks from the page cache if they are there, or asks the disk to fetch them if not.
5. Application: meteo-api computes the response, serializes it as JSON and writes it to the socket.
6. Kernel, one last time: it splits the response into packets and instructs the network card to send them.

None of these layers knows the details of the others, and that is precisely the reason the system is manageable. In Main Functions of an Operating System we will walk this same path in more detail.

• Confusing "operating system" with "what I see on screen". The graphical interface is the most visible part and the least fundamental. If you get used to working on a server with no graphical environment, the concept becomes clear right away.
• Thinking the OS runs continuously in parallel with programs. It does not: the kernel is not a process that runs all the time. Most of the time it is idle, and it only springs into action when something invokes it (a system call, an interrupt or an exception). It is more a set of response routines than a perpetual program.
• Believing that a file "is" at a place on the disk that the program knows about. The program only knows a name and a descriptor. The physical location can change (defragmentation, copy-on-write, network file systems) without the application noticing.
• Ignoring the memory hierarchy when reasoning about performance. A great many performance problems that look like "CPU problems" are really memory or disk access problems. Internalize the orders of magnitude in the table: they will save you hours of wrong diagnoses.
• Practical tip: install a virtual machine with a Linux distribution without a graphical environment (Debian netinst or Alpine will do) and work there during the course. Seeing the system with no frills speeds up understanding a great deal.

Classify each of these elements of meteo-01 as (a) part of the kernel, (b) a system library, (c) a utility or shell, or (d) an application. Justify each answer using the execution-mode criterion:

1. The scheduler that decides which process uses the CPU.
2. bash.
3. glibc.
4. The network card driver.
5. meteo-api.
6. systemctl.

aggregator needs to read a day's 720,000 readings (17 MB) to compute the hourly averages. Using the times in the memory hierarchy table, estimate the order of magnitude of the read time in two scenarios: (a) the file is not cached and has to be read from an NVMe SSD; (b) the file is already in the page cache in RAM. Assume reads in 4 KB blocks and a per-block cost of 50 µs from SSD and 80 ns from RAM. What practical implication does the difference have?

Explain, without using the word "abstraction", why the same C program that writes a Reading with write() works unchanged on a laptop with an SSD and on a server with network storage. Indicate the exact point along the path where the difference between the two pieces of hardware disappears.

Solution 1

The criterion is: does it run in kernel mode, with direct access to the hardware?

1. Scheduler → (a) kernel. It decides which process occupies the CPU and to do that it must manipulate registers and internal tables; it is privileged code by definition.
2. bash → (c) utility/shell. It is an ordinary user program. The proof is that you can replace it with zsh without recompiling anything in the system, and that if bash fails the system keeps working.
3. glibc → (b) system library. It runs in user mode, inside the address space of the process that uses it. Its role is to wrap system calls in convenient functions (fopen, printf).
4. Network driver → (a) kernel. It programs the hardware's registers directly and services its interrupts; it needs privileges. On Linux it is usually a loadable module, but it runs in kernel mode all the same (you will see it in Kernel Architecture).
5. meteo-api → (d) application. It is Meteora's business software, with no special privileges; it runs as the meteora user.
6. systemctl → (c) utility. Even though it manages system services, it is a user client that sends orders to systemd, which in turn is also a user process (PID 1). None of this lives in the kernel.

Solution 2

First, how many 4 KB blocks there are in 17 MB:

(a) From an NVMe SSD, at 50 µs per block:

(b) From the page cache in RAM, at 80 ns per block:

The difference is a factor of around 600 times. The practical implication is twofold:

• The first run of the hourly computation after booting the machine will be noticeably slower than the following ones, even though the code is identical. If you measure performance without taking this into account, you will draw false conclusions.
• It is enormously worthwhile for the day's file to fit in the page cache. At 17 MB a day, having 1 GB of free RAM for cache lets you keep almost two months of data in memory, which makes the cost of the disk irrelevant for aggregator's everyday work.

(Note: this estimate ignores the readahead the kernel performs, which in practice improves case (a) considerably. The order of magnitude of the conclusion does not change.)

Solution 3

The program works the same because it does not talk to the disk: it talks to the kernel. The only things the program names are a text string (/var/lib/meteora/readings/2026-08-31.dat) and an integer (the file descriptor). Neither of those two things depends on the hardware.

The exact point where the difference disappears is the write() system call: on the other side of that boundary, the kernel checks which file system manages that path and hands the operation over to it. That file system, in turn, relies on a specific driver — the SSD's or the network client's — and each one does whatever is appropriate for its hardware.

Put another way: the difference between the two pieces of hardware does exist, but it lives below the system call, and the program never crosses that boundary. The kernel always offers the same contract upwards (it receives bytes and a descriptor, and returns how many bytes it wrote or an error) and resolves the variety underneath. That stable contract is what makes the code portable.

An operating system solves two problems that the hardware poses and cannot solve on its own: it is too complex to program directly and too scarce for several programs to use it without coordination. Hence its two roles: extended machine, which offers a clean and portable interface, and resource manager, which shares out CPU, memory and devices in time and in space.

To achieve that it introduced a few very well-chosen abstractions — process, address space, file and device-as-file — and organized itself into layers: kernel, system libraries, shell and utilities, and applications. Only the first runs in privileged mode, and that is the criterion separating the operating system from everything else. You have also seen what an OS is not: neither the graphical interface nor the distribution.

None of this appeared all at once. Each of these ideas emerged as a response to a specific limitation of its time: time sharing was born because computers were extremely expensive and could not sit idle, and named files were born because nobody wanted to remember sectors. In the next lesson, History and Evolution of Operating Systems, we will walk that path generation by generation to understand why operating systems are the way they are.