You already know what an operating system is, where it comes from and what types exist. Now it is time to open the box and see what it does exactly. This lesson is the map of the rest of the course: every major function you see here corresponds to one or more later modules, and you will see it applied to a concrete situation at Meteora. But the goal is not for you to walk away with a list of seven separate functions, because in reality they never act in isolation: serving a single HTTP request to meteo-api mobilizes all of them within a few milliseconds. That is why we will close by following the complete life cycle of a request, from beginning to end, to see how they cooperate.

1. Process and processor management
2. Memory management
3. Storage and file management
4. Device and input/output management
5. Network management
6. Protection and security
7. User interface: shell and graphical environment
8. All at once: the life cycle of a request to meteo-api

What it does. Create processes, destroy them, suspend them, resume them, keep information about each one and decide which of them occupies the CPU at each instant. For every process the system maintains a control structure (on Linux, task_struct) with its identifier, its state, its saved registers, its table of open files, its owning user and its consumption statistics.

Why it is hard. Because meteo-01 has 4 cores and, at any given moment, 300 processes. There are 300 candidates for 4 slots, and the decision has to be made thousands of times per second, in microseconds, without knowing the future and without systematically favoring anyone.

At Meteora. When aggregator starts its 02:30 computation, it consumes an entire core for 20 seconds. If the system did not intervene, meteo-api would be left without CPU during that time and every client would see timeouts. What actually happens is that the scheduler preempts aggregator every few milliseconds and lets meteo-api through as soon as a request arrives.

Column-by-column interpretation:

• NI is the nice value: the process's politeness, from −20 (highest priority) to +19 (lowest). aggregator has 10, which means it was deliberately started with low priority so that it does not disturb the service.
• PRI is the effective priority computed by the kernel.
• STAT is the state: S means sleeping (blocked waiting for something, typically I/O), R means running or ready to run, and the l in Sl indicates that the process has several threads.
• ETIMES is the number of seconds since it started. ingestor and meteo-api have been up for 864,320 seconds (10 days); aggregator has just begun.

Notice the pattern: the two permanent services are asleep almost all the time, waiting on the network. aggregator is running. This profile (S for services, R for compute jobs) is the first thing to look at when diagnosing a server.

Where it is developed. Process Management and CPU Scheduling. Concurrency between processes and threads takes up the whole of module 3.

What it does. Decide what is in RAM and where, allocate and free memory for processes, give each one a private address space, and make more fit than physically exists.

Why it is hard. RAM is a scarce resource, indivisible in its physical form and shared by everyone. Three demands come into conflict: that processes must not step on each other, that every byte should be put to use, and that all of this must not cost time.

At Meteora. meteo-api has a curious consumption profile worth understanding:

• VSZ (Virtual Size) is the KB of virtual address space: 1,284,560 KB, about 1.2 GB.
• RSS (Resident Set Size) is the KB actually occupying physical RAM: 47,320 KB, about 46 MB.

The difference is a factor of 27, and it is not a mistake. meteo-api has reserved address space for many things (shared libraries, mapped regions, its threads' stacks) that it is not using right now, and the system has not given it physical memory for them. The system hands over physical memory only when the process really touches those addresses. This mechanism, lazy allocation, is the reason why processes whose VSZ adds up to more than 20 GB can coexist on meteo-01 with its 8 GB of RAM.

A very expensive beginner's mistake: getting alarmed by VSZ. The column that matters for knowing whether you are running out of memory is RSS, and not even that one entirely, because part of the resident memory consists of shared libraries counted in several processes at once.

Where it is developed. Memory Management and Virtual Memory and Paging.

What it does. Turn a device that only understands numbered blocks into a tree of directories with files that have a name, size, dates, owner and permissions. On top of that: allocate space, manage free space, maintain consistency in the face of a power cut and cache recently read data in RAM.

Why it is hard. Because the disk knows nothing about files. An SSD offers a sequence of 4 KB blocks numbered from 0 up to whatever. Everything else — names, hierarchy, permissions, the very notion of a "file" — is a data structure the operating system builds on top and must keep consistent even if the machine powers off at the worst possible moment.

At Meteora. The daily reading files live in /var/lib/meteora/readings/:

What this tells us:

• ls -lh shows the long listing with human-readable sizes (-h). Today's file is 12 MB because the day is not over yet: it grows at a rate of about 24 bytes per reading received.
• The permissions -rw-r----- mean: the owner (meteora) reads and writes, the group (meteora) only reads, and the rest of the world has no access at all. Any other user of the system cannot even see the contents.
• df -h reports on the file system containing that path: the device /dev/sda2, mounted on /var, 63% full.

That 63% is an important operational figure: at 17 MB a day, 72 GB remain free, that is, about 11 years of data. But if tomorrow Meteora doubled the number of stations and also stored derived data, that margin would shrink quickly. Watching growth is part of the job, and that is why we will look at it in Performance Monitoring and Troubleshooting.

Where it is developed. The whole of module 4, starting with File Systems and Storage Management.

What it does. Offer a uniform interface for very diverse hardware, through specific drivers; schedule I/O operations; service the interrupts that devices generate; and absorb the speed difference between the CPU and everything else using buffers and caches.

Why it is hard. Because of the disparity in speeds you have already seen: while the disk serves a 50 µs request, the CPU could execute around 150,000 instructions. If the CPU busy-waited, it would be wasted. The solution — block the process, run another one, and wake the first one with an interrupt when the data arrives — is the heart of operating system design.

At Meteora. ingestor receives about 500 readings per minute from the network and writes them to disk. If every reading caused a physical write, that would be 500 disk operations per minute to write 12 KB in total: absurd. The system groups those writes in the page cache and flushes them to disk in larger blocks.

An explanation of each line:

• MemTotal: 8 GB of RAM in the machine.
• MemFree: only 400 MB free. This is not a problem, and mistaking it for one is the most frequent error of all.
• Cached: 5.2 GB dedicated to the page cache, that is, in-RAM copies of file data. This memory is instantly reclaimable: if a process needs it, the kernel frees it. The reading files from the last few days are here, and that is why aggregator re-reads them so fast.
• Dirty: 8 MB written by processes that have not reached the disk yet. This is the data ingestor has handed over and the kernel has not yet flushed.
• Writeback: 0 KB in transit to the disk at this instant.

That Dirty value is also a risk: if meteo-01 lost power right now, those 8 MB of readings would be lost. When the loss is unacceptable, the program must force the flush with fsync(), at the cost of performance. It is a classic trade-off we will pick up again in Space Allocation, Journaling and Integrity.

Where it is developed. Device Management and Drivers, Interrupts and I/O Operations.

What it does. Implement the protocol stacks (TCP/IP), manage the interfaces, maintain the routing tables, multiplex connections between processes and offer the socket abstraction, which lets a remote connection be treated almost like a file.

Why it is an operating system function and not an application one. Because the network card is a shared resource: meteo-api, ingestor and sshd are all listening at the same time, and someone has to decide which incoming packet belongs to whom. That distribution is done by port, and only the kernel can arbitrate it.

At Meteora.

Line-by-line analysis:

• ss -tlnp lists TCP sockets (-t) in the listening state (-l), without resolving names (-n, faster and with no DNS surprises) and showing the owning process (-p).
• meteo-api listens on 0.0.0.0:8080: the address 0.0.0.0 means all interfaces, that is, it accepts connections from outside the machine. That is the right thing for a public service.
• ingestor listens on 127.0.0.1:9310: only on the loopback interface, so only processes on the machine itself can connect. It is a deliberate security decision. If it had to accept connections from the stations from outside, this would be an exposed port and would have to be protected.
• Send-Q in the LISTEN state indicates the maximum size of the pending connection queue: 512 for meteo-api, 128 for ingestor. If more simultaneous connections arrive than fit in that queue, they are rejected.
• fd=6 and fd=4 are the file descriptors of those sockets within each process: direct proof that "everything is a file" reaches the network too.

What it does. Authenticate (who are you?), authorize (may you do this?), isolate (keep one process from touching what belongs to another) and audit (leave a record of what happened).

It is worth separating two concepts that are usually mixed up:

• Protection is the internal mechanism that stops processes from interfering with each other, even with no bad intent. A bug in aggregator not corrupting meteo-api's memory is protection.
• Security is the defense against an adversary acting with intent. An attacker who compromises meteo-api not being able to read /etc/shadow is security.

At Meteora. The permission model applies least privilege at every layer:

The interesting part is in the details:

• The configuration file belongs to root but its group is meteora. The services can read it (the group's r permission) but not modify it, because write permission belongs only to the owner, root. If an attacker compromises meteo-api, they cannot alter the configuration to, for example, change the path of the data.
• The log does belong to meteora, because the service needs to write to it.
• uid=990: identifiers below 1000 are reserved by convention for system accounts, which do not correspond to people and cannot log in interactively.

Where it is developed. The whole of module 5, plus File Security and Permissions.

What it does. Offer a way for people to give orders to the system. It can be a command line (bash), a graphical environment (GNOME) or a remote administration API.

An essential nuance, already seen in the first lesson. The interface is not part of the kernel. bash is a user program that reads what you type, interprets it and asks the kernel to run programs through system calls. Its privilege is exactly the same as that of any other user program.

There is no graphical environment on meteo-01. All administration is done over SSH with bash, and this is deliberate: less installed software means less memory consumed, fewer updates to apply and fewer potential vulnerabilities.

Where it is developed. The Command Line as the System Interface.

This is where the seven functions stop being a list and become a system. Let's follow a real request:

Let's walk through the diagram naming the function responsible for each step:

1. Arrival over the network (network management + device management). The card receives the packets and raises an interrupt. The kernel reassembles the TCP stream, looks at the destination port (8080) and locates meteo-api's socket.
2. Wake up and schedule (process management). meteo-api was blocked in read(), in state S. The kernel moves it to "ready" and the scheduler decides when to give it CPU. Since it is an I/O-oriented process, it has an advantage over aggregator. The context switch costs on the order of 1 to 5 microseconds.
3. Reading the request (system calls, the subject of the next lesson). meteo-api executes read(), crosses the boundary into kernel mode, collects its bytes and returns.
4. Opening the file (file management + security). The kernel translates /var/lib/meteora/readings/2026-08-31.dat by walking the directory tree, and at each step it checks that the meteora user has permission. If it did not, it would return EACCES without ever touching the disk.
5. Obtaining the data (memory management + device management). Here there are two paths, and the difference between them is three orders of magnitude:
   • If the blocks are in the page cache, the copy is made from RAM: about 80 ns per block.
   • If they are not, the kernel asks the disk for the blocks, blocks meteo-api and takes the opportunity to run another process. When the disk finishes, it raises an interrupt, the kernel copies the data into the page cache and wakes the process up: about 50 µs per block on an SSD.
6. Computation (process management). meteo-api filters station 118's readings, groups them by hour and computes averages. It is the only part of the journey that does Meteora-specific work; everything else is contributed by the system.
7. Response (network management). write() to the socket. The kernel copies the data into its transmit buffer, splits it according to the maximum segment size and orders the card to send it. Notice one important detail: write() returns control before the data has reached the client. The application does not wait for the network.
8. Logging (file management). Another write, this time to /var/log/meteora/meteo-api.log, which also goes through the cache and will be flushed to disk later.

The conclusion that matters: out of this whole journey, Meteora's code only contributes step 6. The remaining seven are the operating system. And none of its functions could do its part without the others: scheduling depends on knowing who is blocked on I/O, I/O depends on memory management for the cache, the cache depends on security having already authorized the access. They are a system, not a catalog.

• Memorizing the functions as a list. What gets asked in practice is not "list the functions of an OS", but "why does this request take 400 ms". Practice walking the full life cycle: that is the exercise that really teaches.
• Getting alarmed because MemFree is low. Unused free RAM is wasted RAM. A healthy system uses almost all of its memory, most of it in the page cache, which is freed instantly when needed. The useful metric is MemAvailable, not MemFree.
• Confusing VSZ with real consumption. The virtual address space can be 30 times larger than the physical memory occupied. Look at RSS, and for fine-grained analysis, /proc/<pid>/smaps_rollup.
• Believing the application "writes to disk". The application hands bytes to the kernel. When they reach the disk is decided by the system, unless it is forced with fsync(). This difference explains a great many data losses after a power cut.
• Thinking a write() to a socket means the client received the data. It only means the kernel accepted it into its output buffer.
• Tip: get into the habit of looking at the four basics when diagnosing any problem, in this order: process state (ps, top), memory (free -h), disk (df -h, iostat) and network (ss). Almost every incident is located there in under a minute.

For each symptom observed on meteo-01, state which operating system function is mainly involved, which command you would use to confirm it and in which module of the course it is studied:

1. meteo-api's responses take 3 seconds instead of 50 ms, and top shows one core at 100%.
2. ingestor fails with "No space left on device".
3. A client cannot connect to port 8080 from outside, but can from the machine itself.
4. meteo-api cannot read /etc/meteora/meteora.conf after a configuration change.

meteo-api receives 200 requests per second. Each request requires reading 40 blocks of 4 KB from the day's file. Compute the total I/O time per second in two scenarios: (a) everything from the SSD, at 50 µs per block; (b) everything from the page cache, at 80 ns per block. Can meteo-01 sustain that load in scenario (a)? Reason about which OS function makes the real scenario resemble (b) in practice.

Walk through the life cycle of a request and state, for each of the eight steps, what would happen if that operating system function did not exist. The goal is for you to justify the need for each one, not to describe how it works.

Solution 1

1. Slow responses with one core at 100%

• Function: process and processor management. There is CPU contention: some process (very probably aggregator) is monopolizing a core and meteo-api is not getting enough CPU.
• Command: top sorted by CPU, or ps -eo pid,ni,pri,stat,%cpu,comm --sort=-%cpu. It is worth looking at the NI value: if aggregator was not started with nice, that is the cause and the immediate fix is renice.
• Module: CPU Scheduling.

2. "No space left on device"

• Function: storage and file management.
• Command: df -h /var/lib/meteora to see the space, and also df -i. This second one is important: the identical error appears when the inodes run out even though there is free space, something typical when there are millions of small files. It is a diagnosis that is constantly overlooked.
• Module: File Systems and Space Allocation, Journaling and Integrity.

3. Connects from inside but not from outside

• Function: network management (and possibly security, if there is a firewall).
• Command: ss -tlnp | grep 8080. If the local address is 127.0.0.1:8080 instead of 0.0.0.0:8080, the service is only listening on the loopback interface and that is the problem; it is fixed in the application's configuration. If it is already listening on 0.0.0.0, then the firewall has to be reviewed with iptables -L -n or nft list ruleset.
• Module: Common Threats and System Hardening for the firewall part.

4. Cannot read the configuration

• Function: protection and security.
• Command: ls -l /etc/meteora/meteora.conf and ls -ld /etc/meteora. Both have to be checked: even if the file has the right permissions, if the directory lost the execute permission (x) for the group, it cannot be traversed and access fails just the same. It is the most frequent mistake when adjusting permissions by hand.
• Module: File Security and Permissions.

Solution 2

Blocks to read per second:

(a) From the SSD, at 50 µs per block:

(b) From the page cache, at 80 ns per block:

Can it sustain the load in (a)? Strictly speaking yes, because 0.4 s of I/O for every second of wall clock leaves margin. But the margin is deceptive for two reasons:

• It is a 65% effective utilization of the I/O subsystem if we account for serialized latency. In queueing theory, latency grows non-linearly as saturation is approached: beyond 70-80% utilization, response times shoot up. A peak of twice the traffic would make the service unviable.
• The 0.4 s are waiting latency, not CPU. During that time the processes are blocked, which requires enough concurrent threads or processes so that the server does not sit idle waiting.

What makes the real case resemble (b): the page cache, which is memory management and I/O management working together. Since the day's file takes 17 MB and meteo-01 has gigabytes dedicated to cache, after the first reads the whole file resides in RAM and every subsequent request is served from there. The disk only gets involved the first time and at the beginning of each new day.

This is also the reason why badly designed performance tests give deceptively good results: if you measure after running the same query twenty times, you are measuring RAM, not the complete system.

Solution 3

1. Without network management: every application would have to talk to the card directly and implement TCP on its own. Worse still, there would be no way to distribute incoming packets among meteo-api, ingestor and sshd: only one program could use the network at a time.
2. Without process scheduling: meteo-api could not be woken up. Either it would have to poll the socket actively in a loop (burning CPU without doing anything useful) or the system would run a single program until it finished, and requests would be served one at a time.
3. Without system calls: there would be no boundary between the application and the kernel. Any program could manipulate the hardware directly, and a bug in meteo-api could corrupt any system structure.
4. Without file management: meteo-api would have to know which physical sectors hold August 31's data and keep that bookkeeping itself. Changing disks would force a rewrite of the application. And with no permission checking, any process could read any data on the machine.
5. Without memory management and caching: every read would go to the disk, at a cost 600 times higher. Besides, without separate address spaces, a stray pointer in aggregator could overwrite meteo-api's memory, silently corrupting data served to customers.
6. Without preemption in scheduling: meteo-api would keep the CPU throughout its computation and, if it entered an infinite loop because of a malformed request, the entire machine would be unusable until a physical reboot.
7. Without network buffers in the kernel: the application would have to wait for each byte to be physically transmitted before continuing, which would multiply response time by several orders of magnitude and reduce throughput to a fraction of the current one.
8. Without a file system or access control for the log: there would be no reliable way to record what happened, nor to guarantee that an attacker cannot erase their traces. Without logs, incident response is impossible; we will come back to this in Auditing, Logging and Incident Response.

The main functions of an operating system — processes and CPU, memory, storage and files, devices and I/O, network, protection and security, and user interface — are the map of the rest of this course. Each one solves a specific problem: sharing out the processor, giving each process a space of its own, turning blocks into files, absorbing the slowness of the hardware, multiplexing the network, isolating some from others and letting a human give orders.

What you have seen by following an HTTP request from beginning to end is that these functions do not operate separately. In the few milliseconds meteo-api takes to respond, all of them get involved, they lean on one another and, of the eight stages of the journey, seven are operating system and only one is Meteora's code. That proportion explains why it is worth understanding what lies underneath.

So far we have looked at the system from the outside: what it does and for whom. In the next lesson, Kernel Architecture: Monolithic, Microkernel and Hybrid, we will open up the kernel to see how all that code is organized inside, and you will discover that there are radically different ways of structuring it, with direct consequences for the performance and robustness of meteo-01.