In the previous lesson we treated the disk as if it were the device in the system. But meteo-01 has a great deal more: two NVMe SSDs, a mechanical disk, a network card, a real-time clock, a serial terminal, a random number generator and several USB controllers. And the weather stations that send it data have sensors, clocks and radios. None of these devices resembles the others: they differ in speed by a factor of a million, in transfer unit, in whether they can be read, written or both, and in whether they respond in microseconds or in minutes.

The operating system has to offer a coherent interface for all of them without knowing the details of any. This lesson is about how it manages that: what classification it uses, why in UNIX "everything is a file", how the files in /dev come into being, and what the three ways the CPU has of talking to a device are. We will finish by following the path of a station reading from the network card to the ingestor's buffer, leaving the internal mechanism of interrupts for the next lesson, which develops it in detail.

1. Why the operating system makes devices uniform
2. The structure of the I/O subsystem
3. Classification: block, character and network
4. "Everything is a file" and the /dev directory
5. Major and minor numbers
6. udev and the dynamic creation of devices
7. Buses and enumeration: lspci, lsusb, lsblk
8. The three ways of talking to a device
9. I/O ports and memory-mapped I/O
10. Subsystem techniques: buffering, caching, spooling and reservation
11. Error handling and retries
12. The path of a station reading

Look at the real variety meteo-01's kernel faces:

Between the keyboard and the GPU there are nine orders of magnitude of difference in speed. And yet, the programmer of the ingestor writes:

and that same line works whether fd is a network socket, a file on the SSD, a terminal or the random generator. That uniformity is not a convenience: it is what makes writing software possible at all.

Without it, every program would have to know the exact model of every device. Changing the network card would force you to recompile every application. It is exactly the problem the operating system as an extended machine solves, the one we set out in 01-01.

The I/O subsystem has five concrete goals:

The subsystem is organized in layers, each one adding abstraction on top of the previous one:

How the responsibilities are shared out:

The key point is the device-independent layer: it is the one that does all the common work once, so that each driver only has to implement what is genuinely specific. A network driver does not need to know anything about permissions, or naming, or buffering: only how to talk to its chip.

And here it connects with lesson 01-05: drivers live inside the kernel in a monolithic system like Linux, loaded as modules. That is why a faulty e1000e can bring the whole machine down, whereas in a microkernel it would live in user space at the cost of more IPC.

Linux classifies devices into three broad families, and that classification determines the interface each one offers:

Block devices. The previous lesson covered their management in full: an array of blocks numbered by LBA, request scheduling, cache. The defining characteristic is random access: you can read block 5,000 without having read the previous 4,999.

Character devices. They are a stream: the bytes arrive and you cannot go back. A keyboard, a serial port, a random generator. It makes no sense to "seek to position 500" on a keyboard.

The leading c marks a character device. Notice that /dev/null and /dev/zero do not correspond to any hardware at all: they are virtual devices implemented entirely in software. /dev/null discards everything written to it; /dev/zero produces an infinite supply of zeros. The fact that they exist shows that the device abstraction is general enough to wrap things that are not devices.

Network devices. These are the interesting exception: they have no file in /dev.

The reason is fundamental: a file implies an ordered, lossless byte stream, and the network is not that. Packets can be lost, duplicated or arrive out of order, and each one carries metadata (addresses, ports, protocol) that does not fit into read()/write(). That is why UNIX invented a different abstraction — the socket, with socket(), bind(), sendto(), recvfrom() — for what would not fit into the file one.

It is a valuable design lesson: a good abstraction has limits, and forcing it beyond them produces worse interfaces. The BSD authors recognized this and created a second abstraction instead of distorting the first.

Even so, once it is open, a socket is handled through a file descriptor and does accept read() and write(). The file abstraction covers the usage; only creation and configuration need an interface of their own.

The principle that defines UNIX: devices are presented as special files in the file system.

The consequences are enormous and very practical:

Three concrete advantages of this:

1. A single permission model. The fact that /dev/nvme0n1 belongs to the disk group with permissions rw-rw---- is what stops the meteora user reading the raw disk and bypassing the file system permissions. Without the file model you would need a separate permission system just for devices.
2. Tool reuse. cat, dd, cp, grep work on devices without ever having been programmed to.
3. Composition. You can redirect, pipe and chain devices like anything else.

An example that exploits all three at once:

It copies an entire block device, compresses it on the fly and stores it. dd knows nothing about NVMe, gzip knows nothing about devices, and neither of them has needed to.

And the shell's > operator exploits exactly what we saw in 02-01: the shell does a fork, in the child it opens the file as descriptor 1, and then execve. The new program writes to descriptor 1 without ever noticing.

A device file contains no data: it contains a reference to a driver. That reference is two numbers.

Let us take the /dev/sda line apart field by field:

Where an ordinary file would show its size in bytes, a device file shows two numbers separated by a comma. That is the detail that gives away that it is not an ordinary file.

Some examples that clarify the mechanics:

They all have major 8 (the SCSI/SATA disk driver), and the minor tells them apart: 0 is the whole sda disk, 1 and 2 its partitions, 16 is the next disk. The convention allocates 16 minors per disk, which allows 15 partitions each.

Majors are officially registered:

When you open /dev/sda, the kernel reads the major (8), looks up in this table which driver registered it, and hands it the operation together with the minor so that it knows which disk is meant. That is the entire I/O dispatch mechanism.

You can create device files by hand, although these days you hardly ever need to:

A conceptually revealing test: /dev/mydisk with major 8 and minor 0 is exactly /dev/sda. The name means nothing; the only thing that matters is the two numbers. Names are a human convention.

/dev used to be an ordinary directory with thousands of files created in advance just in case. It was a mess: you could not tell which ones corresponded to hardware that was actually present, and plugging in something new meant creating the file by hand.

Today /dev is a virtual file system (devtmpfs) that reflects the hardware genuinely present, managed by udev.

When a device is plugged in, the sequence is:

Watching it live is the best way to understand it:

And you can ask for everything udev knows about a particular device:

The S: lines are alternative symbolic links, and they solve a very real problem: the name nvme0n1 depends on the detection order, which can change between boots. Links based on the serial number never change.

This is why /etc/fstab must use UUIDs or /dev/disk/by-id/, never /dev/sda1. If you add a disk and the order changes, an fstab with direct names can mount the wrong volume. We will see this applied in Partitions, Mounting and the Virtual File System.

You can define your own rules. A real case from Meteora: one of the stations connects to the server over USB-serial for diagnostics, and you want it always to appear under the same name and to be accessible to the meteora user.

How each part works:

• SUBSYSTEM=="tty": applies only to serial terminal devices.
• ATTRS{idVendor}/ATTRS{idProduct}: identify the particular FTDI chip.
• ATTRS{serial}: distinguishes this station from another with the same chip. Without it, two identical adapters would both match the rule.
• SYMLINK+=: creates /dev/meteora/north-station, a stable name that does not depend on whether the kernel called it ttyUSB0 or ttyUSB3.
• OWNER/GROUP/MODE: make the device accessible to the meteora user without needing root.

That last point matters from a security point of view: instead of running the diagnostic process as root so that it can open the serial port, it is given exact permissions on that one specific device. It is the principle of least privilege, which we will develop in Protection Principles and Access Control.

Another useful rule: setting the I/O scheduler by device type, picking up where the previous lesson left off.

That way the right scheduler is applied automatically at every boot and to every disk that gets connected, according to whether it is rotational or not.

Devices connect through buses, and each bus has its own enumeration mechanism: the way the system discovers what is attached.

The identifier 03:00.0 follows the bus:device.function format, and it is the card's physical address in the PCI topology. The detailed version reveals how the kernel talks to it:

Every line says something we will use:

• bus master: the card can initiate DMA transfers on its own, without the CPU copying the data.
• IRQ 128: the interrupt line assigned to it. It will show up in /proc/interrupts.
• Memory at df200000 [size=128K]: its registers are mapped into memory at that physical address.
• I/O ports at e000: it also has classic I/O ports.
• MSI-X: Enable+ Count=5: it uses message-signalled interrupts, with 5 different vectors.
• Kernel driver in use: igb: which module manages it.

That last line is the first thing to check when a device does not work: if it is missing, no driver is loaded and the hardware is present but inert. It is exactly the scenario we saw in 01-05 with lsmod and modinfo.

lsusb -t shows the tree with each node's driver (ftdi_sio for the station) and the negotiated speed (12 Mb/s, USB 1.1 full speed, plenty for a serial port).

And the complete map of block devices, which we saw in the previous lesson:

When the CPU needs to transfer data with a device, there are three possible mechanisms. This section contrasts them; the internal mechanism of interrupts and DMA is the subject of the next lesson, here we only care about when each one is used and what it costs.

The CPU repeatedly polls a status register on the device until it reports being ready, and then transfers the data itself, byte by byte or word by word.

• Advantage: minimal latency and total simplicity. No interrupts, no synchronization.
• Drawback: the CPU does nothing else while it waits.

With a slow device the waste is catastrophic:

Thirty million cycles per letter. Even so, busy waiting is still the right choice in three cases: when the device responds in less time than an interrupt would cost (a few microseconds), during boot — when the interrupt system is not yet configured — and in a panic handler, when nothing else can be trusted any more.

The CPU starts the operation and forgets about it: it puts the process to sleep and schedules another one. When the device finishes, it raises an interrupt and the kernel wakes the process up.

You will recognize every step: they are exactly the transitions from the state diagram in 02-01. States S and D exist precisely because of this.

• Advantage: the CPU is put to use during the wait.
• Drawback: each interrupt costs between 1 and 5 µs, and the CPU still copies the data byte by byte.

And there is the problem that remains to be solved: with a 1 Gb/s network card receiving 1,500-byte packets:

A quarter of a core just servicing interrupts, without counting the copying of the data.

DMA is a controller that transfers data between the device and memory without the CPU intervening. The CPU only programs the operation (destination address, size) and receives a single interrupt at the end.

Transferring 1 MB from a device:

With concrete numbers for 1 MB from the SSD:

A factor of more than 150 against interrupt-driven I/O. That is why every device that moves volume uses DMA: disks, network, GPU, sound. The bus master we saw in lspci is precisely the ability to do it.

The decision rule in summary:

One question remains: how a device's registers are physically reached. There are two approaches.

x86 has a separate address space of 65,536 ports, with dedicated instructions:

You will remember in and out from the table of privileged instructions in 01-06: they can only be executed in kernel mode. That is the mechanism that stops a user program from talking directly to the hardware.

There are the historic ports: 0x60 and 0x64 for the keyboard, 0x3F8 for COM1, 0x70 for the real-time clock. Numbers that have not changed since the IBM PC of 1981.

The device's registers are assigned addresses in the physical memory space. Reading or writing them is accessing the device.

The 128 KB at 0xdf200000 are the registers of the network card we saw in lspci. The igb driver has reserved them and accesses them with ordinary memory instructions.

Comparing the two approaches:

MMIO has won, and for a reason you now understand perfectly: because it lives in the ordinary address space, protection is applied by the MMU with the page granularity we studied in 02-04. You can give a process access to the registers of one particular device without giving it access to anything else. With I/O ports, protection is all or nothing.

On top of that, MMIO lets a driver reach the registers with ordinary C code, without assembly, which makes the code portable across architectures.

One critical detail: MMIO pages must be marked as non-cacheable (the PCD bit in the page table, which we saw in 02-04). If the CPU cached a device's status register, it would read a stale value from the cache instead of the real hardware state, and the driver would hang waiting for a condition that has already been met. It is a perfect example of how two individually correct mechanisms — cache and MMIO — destroy each other if they are not coordinated.

The device-independent layer applies four general techniques. Each one solves a different mismatch.

A buffer is an intermediate area of memory between the producer and the consumer. It solves three distinct problems:

We already calculated its impact in 01-06: batching 170 readings before writing cut system calls by a factor of 167.

Why double buffering. With a single buffer a blocking problem appears:

While the process works through the buffer, the device cannot write into it and has to wait. And vice versa. The two sides serialize.

While the device fills buffer A, the process works on B. When they finish, they swap. Throughput goes from 1/(t_fill + t_process) to 1/max(t_fill, t_process).

The calculation for the ingestor:

The generalization is the circular buffer with N slots, which is what socket queues and the descriptor rings of modern network cards use.

A buffer and a cache are often confused, but they are different things:

Linux's page cache is the system's disk cache, and it is the one we saw as buff/cache in free -h:

Those 4.3 GB are file content held in RAM. When the aggregator reads 2026-08-31.dat a second time, it does not touch the disk: the data is already there. It is the same cache that backs the mmap() pages we used in 02-04.

Checking it is simple and very instructive:

20 times faster the second time, without having touched the disk. That factor of 20 is what the page cache contributes continuously and invisibly.

Spool comes from Simultaneous Peripheral Operation On-Line. It is the technique for devices that cannot be shared by interleaving operations.

If two processes write simultaneously to a printer without coordination, the output comes out mixed line by line. Spooling solves it: each job is written in full into a queue directory, and a daemon prints them one at a time, in order.

The interesting thing is that spooling is still very much alive even though printers no longer matter: cron, mail and message queues use the same pattern. The general idea — queue jobs in persistent storage and process them sequentially with a single consumer — is one of the most reused patterns in computing.

At Meteora, the nightly backup transfer to the remote server works this way: the files are dropped into an outgoing directory and a process transfers them one by one, retrying the ones that fail.

Some devices have to be used exclusively. The system offers mechanisms to reserve them:

flock with LOCK_EX | LOCK_NB attempts an exclusive lock without waiting: if another process already holds the port, it fails immediately instead of hanging.

Exclusive reservation introduces the risk of deadlock: if process A holds the serial port and waits for the modem, and B holds the modem and waits for the serial port, neither makes progress. That problem has a lesson of its own: Deadlocks.

The general principle: errors are handled as close to the hardware as possible, and they only travel upwards if they cannot be resolved below.

A real example of the complete chain, seen from the kernel log:

Reading the trace:

• UNC (uncorrectable): the disk could not read a sector even with its internal error correction.
• The driver retried several times (hence the 3 seconds between the first and last lines).
• Once the retries were exhausted, it produced a Medium Error.
• The file system receives it as EIO, and the process that issued the read() gets -1 with errno = EIO.

That errno is exactly the mechanism we traced back to its origin in 01-06: a negative value returned by the kernel that libc converts into -1 plus errno.

Error categories and what to do about each:

EINTR deserves to be stressed because it is the most treacherous source of bugs. We already saw it in the buffering example in 01-06: if a signal arrives while the process is blocked in read() or write(), the call returns with EINTR without having done anything. It is not an error: you have to retry. Ignoring it produces sporadic, irreproducible data loss.

Four lines that avoid a whole class of intermittent failures.

We close by applying everything to one concrete journey: a weather station sends a 24-byte reading and the ingestor receives it. Here we see which layers take part and what each one does; the internal mechanism of the interrupt and of DMA is the content of the next lesson.

The journey with the role of each layer:

Notice the division of labor. Steps 1-6 are specific to this card: only the igb driver knows how to read its registers. From step 7 onwards the code is common to every network card in the world: the IP/UDP stack neither knows nor cares which hardware brought the packet in. That boundary is exactly the one we drew in the layer diagram in section 2, and it is what allows Linux to support hundreds of different cards with a single network stack.

And notice one more thing: the ingestor has taken part in none of this. It was asleep in state S from its read(). All the work was done by the hardware, DMA, the interrupt handler and the network stack. The process only wakes up when there is already data waiting for it.

A cost calculation that motivates the next lesson. At 800 readings/second:

That is the interrupt storm problem, and its solution — having the system stop using interrupts and start actively polling when the load is high — is one of the most elegant mechanisms in the Linux kernel. It is called NAPI and we will look at it in detail next.

Using /dev/sda1 in /etc/fstab. The name depends on the detection order and can change when you add hardware. Always use UUID= or /dev/disk/by-id/. The day you add a disk and the system will not boot, you will understand why.

Confusing buffer and cache. A buffer holds the only copy of data in transit; losing it loses data. A cache holds a copy of something that exists somewhere else; losing it only costs performance. drop_caches empties caches, never buffers with pending data.

Running as root to reach a device. Almost always the right answer is a udev rule that grants exact permissions on that one device to the user who needs it. Root to read a serial port is granting a thousand permissions in order to use one.

Ignoring EINTR. It is the most frequent cause of intermittent failures in I/O code. Any read, write or close on a blocking descriptor has to allow for it.

Believing that a network device ought to have a file in /dev. It does not, and that is a deliberate design decision: packets are not a reliable, ordered byte stream. The right abstraction is the socket.

Misreading lspci when something does not work. If the device appears but there is no Kernel driver in use line, the hardware is present and has no driver. If it does not appear at all, the problem is physical or a BIOS matter. Those are two completely different diagnoses.

Diagnostic tip: when a device does not work, the order that works is: lspci/lsusb (does the system see it?), lspci -v | grep -i driver (does it have a driver?), dmesg -T | tail -50 (what did the kernel say when it detected it?), ls -l /dev/... (does the node exist, and with what permissions?), and udevadm info --query=all --name=... (what does udev know?). Five checks that pin the problem down to the exact layer.

Given this output:

1. Classify each entry by type and explain how you know.
2. Why do /dev/sda and /dev/sda1 share the major but not the minor?
3. If the user meteora (group meteora) runs dd if=/dev/sda of=/tmp/copy bs=1M count=10, does it work? And on /dev/meteora/north-station?
4. What does the 7 in the last line represent, where the others have two numbers?
5. Write a udev rule that gives the meteora group read-only access to /dev/sda, and explain whether it is a good idea.

For each of these Meteora devices, decide which I/O technique is appropriate (busy waiting, interrupts or DMA) and justify it with a calculation:

• A: An I²C sensor in a station, which returns 2 bytes 8 µs after the request. It is polled once a minute. The microcontroller runs at 80 MHz.
• B: meteo-01's network card receiving 800 readings/second of 24 bytes in 66-byte UDP packets.
• C: The NVMe SSD reading the 17 MB of 2026-08-31.dat.
• D: A 9,600-baud USB-serial diagnostic adapter, which sends 200 characters every time it is plugged in.

For each one state the technique, the CPU cost calculation and what would happen if you chose wrongly.

You have plugged a second network card into meteo-01 to separate the station traffic from the API traffic, but it does not show up as an interface. Design the complete diagnostic procedure:

1. Write the sequence of commands you would run, in order, and what you are looking for in each one.
2. For each of these four possible outcomes, say what you conclude and what you would do:
   • a) lspci shows no new card.
   • b) lspci shows it but with no Kernel driver in use line.
   • c) lspci shows it with a driver, ip link shows it as enp5s0 in state DOWN.
   • d) dmesg shows igb 0000:05:00.0: Failed to initialize MSI-X interrupts.
3. Once it is working, write a udev rule that gives it the stable name stations0 and explain why ip link set name is not enough.

Solution 1

1. Classification.

The most reliable signal is the column where an ordinary file would put its size: if there are two numbers separated by a comma, it is a device file.

2. Same major, different minor.

Major 8 identifies the driver (sd, SCSI/SATA disks). Both are managed by the same driver, so they share the major.

The minor identifies which particular instance within that driver:

A partition is not a separate device: it is a window onto a range of blocks on the same physical disk. The sd driver receives the minor, consults its partition table and applies the corresponding offset.

The convention allocates 16 minors per disk: minors 0-15 for sda and its 15 partitions, 16-31 for sdb, and so on. That is why /dev/sdb has minor 16.

3. Permissions for the meteora user.

On /dev/sda: it does NOT work.

• Owner root, with rw. meteora is not root.
• Group disk, with rw. You would have to check whether meteora belongs to disk:

It does not.

• Others: ---, no permissions at all.

Result: dd: failed to open '/dev/sda': Permission denied.

And it is right that it should be so. Read access to the raw disk means being able to read any file in the system, completely bypassing the file system permissions: /etc/shadow included. It is equivalent to being root for reading.

On /dev/meteora/north-station: it DOES work, if the udev rule from the lesson is active.

The link points to ttyUSB0, and the permissions that count are the target's, not the link's (the lrwxrwxrwx of a symbolic link is always like that and means nothing). With the rule:

Owner meteora with rw: access granted. Although dd on a serial port would read the incoming stream, not structured content.

4. The number 7 on the symbolic link.

It is the size of the link in bytes, that is, the length of the string it contains:

A symbolic link is a real file: its content is the target path. That is why it shows a size where device files show major and minor. Real devices show no size because they do not contain anything: they only reference a driver.

Checking it:

5. A udev rule to give the meteora group read access to /dev/sda.

Result: brw-r----- 1 root meteora 8, 0 /dev/sda.

Is it a good idea? Emphatically not.

The reasoning:

1. It bypasses the file system permissions completely. Reading the raw device gives access to the bytes of every file on that disk, including /etc/shadow and the private TLS keys. File permissions are enforced through the file system; device access goes around them.

2. It violates the principle of least privilege. If the aggregator needs to read data, it needs to read files in /var/lib/meteora/readings/, not the entire disk. The permission granted is several orders of magnitude larger than the need.

3. It hugely widens the attack surface. If someone compromises a process running as meteora, they get complete read access to the disk.

4. There is no legitimate case at Meteora that requires it. The real uses of the raw disk — cloning, forensics, recovery — are one-off administrative tasks, not the operations of a service.

The correct alternative, depending on what is actually needed:

This grants exactly one command with exactly one set of arguments, instead of full access to the disk. It is the difference between the key to one door and the master key to the building.

Solution 2

A: I²C sensor, 2 bytes in 8 µs, once a minute, microcontroller at 80 MHz.

Technique: busy waiting.

Justification:

• 640 cycles is less than an interrupt would cost. Setting up the vector, saving the context, running the handler and restoring costs on the order of 200-500 cycles on a microcontroller, plus the complexity of the code. The interrupt would be more expensive than the wait.
• It is a microcontroller running FreeRTOS, as we established in 01-03. There is no heavy multitasking competing for it: there is nothing better to do during those 8 µs.
• Simplicity has value in its own right in embedded systems: less code, less state, fewer ways to fail.

If you chose wrongly (interrupts): it would work, but with more code, more complexity and more CPU consumption than the wait itself. A rare case where the "advanced" solution is objectively worse.

B: Network card, 800 readings/s in 66-byte UDP packets.

Technique: DMA with interrupts (and interrupt moderation).

An honest observation: at only 800 packets/s, DMA and pure interrupts give an almost identical cost, because the cost is dominated by the interrupt, not by copying 66 bytes. DMA wins clearly when packets are large or plentiful.

But there are two solid reasons to use DMA anyway:

1. Scalability. If Meteora grows to 500,000 readings/s, without DMA the copying alone would consume a whole core.
2. It is not optional. Modern cards only work through DMA: the igb driver programs descriptors and the card writes straight into RAM. Programmed I/O no longer exists on this hardware.

If you chose wrongly (busy waiting): a whole core permanently polling the card's register in order to receive 52.8 KB/s. Absurd.

C: NVMe SSD reading 17 MB.

Technique: DMA, without a shadow of a doubt.

And there is an even stronger qualitative argument:

With busy waiting, the CPU would be blocked for 4.9 ms doing nothing. At 3 GHz that is almost 15 million cycles lost, and it amounts to more than a full scheduler quantum (4 ms, according to 02-02). With DMA, another process makes use of those 4.9 ms.

If you chose wrongly: besides the waste, the process could not block in state S, breaking the whole scheduling model we studied in 02-02.

D: USB-serial at 9,600 baud, 200 characters on connection.

Technique: interrupts (which is what the ftdi_sio driver does).

Justification: 1.04 ms per character is an eternity for a CPU. It is exactly the scenario interrupts were invented for: an extremely slow device, scarce data, very long waits.

An interesting detail: USB does not raise one interrupt per byte. The USB controller groups transfers into packets and the FTDI chip has a buffer of its own, so 200 characters may arrive in 3 or 4 transfers. It is buffering applied in the hardware itself, for exactly the reasons in section 10.

If you chose wrongly (busy waiting): 208 ms of blocked CPU. On meteo-01, with the ingestor receiving 800 readings/s, those 208 ms would mean 166 unserviced readings. A diagnostic serial port would have degraded the production service.

Summary:

The rule that emerges: busy waiting when the wait is shorter than the interrupt; DMA when there is volume; interrupts for everything else.

Solution 3

1. The diagnostic sequence.

The logic of the order is to work from the bottom up: hardware → driver → kernel → interface. There is no point investigating the network configuration if the PCI bus does not even see the card.

2. The four scenarios.

a) lspci shows no new card.

Conclusion: the problem is below the operating system. Linux cannot manage hardware that the PCI bus does not enumerate; it is not a driver or configuration problem.

Possible causes and what to do:

The rescan is useful because it rules out the case of a hot-inserted card that was not detected. If it still does not appear afterwards, the problem is physical or a BIOS matter.

b) It appears in lspci but with no Kernel driver in use.

Conclusion: the hardware is present and correctly enumerated, but there is no driver managing it. The card exists and is inert.

If Kernel modules: igb appears, the kernel knows which module would manage it but has not loaded it.

If the module is blacklisted, it will show up in /etc/modprobe.d/:

This is the scenario of the faulty e1000e driver we saw in 01-05: hardware present, module absent or blocked.

c) It appears with a driver, ip link shows it as enp5s0 in state DOWN.

Conclusion: the whole device subsystem is working correctly. The driver is loaded, the kernel has created the interface, udev has named it. What is missing is network configuration or a physical link.

Link detected is the decisive check in this scenario: it separates a cabling problem from a configuration one, which are two completely different investigations.

To make it persistent:

d) dmesg shows Failed to initialize MSI-X interrupts.

Conclusion: the driver has loaded but has not been able to configure the interrupts, which as we saw in lspci are MSI-X with 5 vectors. Without working interrupts, the card cannot signal that it has received packets.

If it says Falling back to MSI interrupts, the driver has degraded to MSI and will probably work, though with less parallelism: instead of 5 vectors (one per queue), it will have a single one, which reduces throughput under heavy load.

Causes and fixes:

If it works degraded to MSI, that is acceptable for Meteora at 800 readings/s — we already calculated that it is 2.4 ms/s of CPU — but it is worth fixing to leave room for growth. The detail of MSI, MSI-X and why several vectors matter is the content of the next lesson.

3. A udev rule for a stable name.

An explanation of each condition:

• SUBSYSTEM=="net": network interfaces only.
• ACTION=="add": only when the device appears.
• ATTR{address}: the MAC address, which is unique and immutable in the hardware. It is what guarantees that the rule identifies this card and not another.
• ATTR{type}=="1": Ethernet type, so as not to match virtual interfaces.
• NAME=: the final name. Careful: for network interfaces you use NAME, not SYMLINK, because the interface is renamed rather than given an alias.

Why ip link set name is not enough:

The underlying problem is the same as with /dev/sda1 in /etc/fstab: the name assigned by the kernel depends on the enumeration of the PCI bus, which can change when you add or move hardware. The card that is enp5s0 today can be enp6s0 tomorrow if you insert another card in an earlier slot.

And there is an even more serious ordering problem: ip link set name is a command that runs after the system has brought the network up. By then, systemd-networkd will already have tried to configure enp5s0 with stations0's configuration, or the other way round, and the interface could end up with the wrong IP. With udev, the renaming happens at detection time, before anything else touches the interface.

At Meteora this matters especially: if stations0 (the station network) and the API interface swapped names after a reboot, the ingestor would be listening on the wrong network and the readings would be lost with no error message whatsoever. A silent, irreversible failure, exactly the kind you have to design out.

The I/O subsystem exists so that read(fd, buf, 1024) works the same on a 100 B/s keyboard and on a 3,500 MB/s NVMe drive, nine orders of magnitude apart. It achieves this with a layered architecture where the device-independent software does everything common — naming, permissions, buffering, cache, errors — and each driver only implements what is genuinely specific to its model.

The classification into block, character and network determines the interface: block devices allow random access and can be mounted; character devices are sequential streams; network devices have no file in /dev because a reliable byte stream does not describe packets that can be lost and reordered, which is why UNIX created a second abstraction, the socket, instead of distorting the first. The principle "everything is a file" unifies permissions, tools and composition, and rests on a surprisingly simple mechanism: the major and minor numbers, where the major picks the driver and the minor the instance. udev populates /dev dynamically and its rules solve two real problems: stable names that do not depend on the detection order, and exact permissions that avoid having to run as root.

There are three ways of talking to a device, and choosing well is a matter of arithmetic: busy waiting when the wait is shorter than an interrupt (or during boot and panic), interrupts for slow devices with little data, and DMA for anything that moves volume, with a CPU cost reduction of more than 150 times. Registers are reached through I/O ports or through MMIO, and MMIO has won because its protection is enforced by the MMU with page granularity, although it requires marking those pages as non-cacheable. Finally, the subsystem techniques — buffering (with double buffering contributing a 62.5 % improvement in the ingestor), caching (a factor of 20 on the second read of the daily file), spooling and reservation — each solve a specific mismatch, and error handling resolves errors as far down as possible, leaving only what requires a decision to reach the top.

We have followed the path of a station reading from the card to the ingestor's buffer and identified the layers, but we have deliberately left three boxes unopened: what exactly the CPU does when it receives an interrupt, how a driver registers with the kernel and what contract it fulfils, and how DMA works on the inside. A problem has also been posed with numbers: at 500,000 readings per second, a whole core would be devoted solely to servicing interrupts. The answer to that, and to everything above, is in Drivers, Interrupts and I/O Operations, which closes the module.