In the two previous lessons we have been counting the cost of a page fault in milliseconds and we have watched a server fall into thrashing because of the disk. Now it is time to look straight at that component we have been treating as a slow black box. Because it is not a black box: a mechanical disk and an NVMe SSD have completely different physics, and the decisions the operating system makes for one are counterproductive for the other.

A word about scope: this lesson deals with storage as a managed resource, not as a file system. Here you will see the physics of the device, how much an access costs, how requests are ordered and how several disks are combined into a reliable volume. Everything to do with files, directories, inodes, partitions and mounting belongs to module 4. The boundary is clear: here we manage numbered blocks; there they are given meaning.

1. The complete storage hierarchy
2. Inside a mechanical hard disk
3. Calculating the time of a disk access
4. SSDs and NAND memory: different physics, different rules
5. Write amplification, garbage collection and TRIM
6. NVMe and why it changes the rules
7. LBA and the block device abstraction
8. Disk request scheduling
9. Linux's real schedulers
10. RAID: combining disks for capacity, speed or reliability
11. Meteora's decision for /var/lib/meteora
12. Measuring with lsblk and iostat -x

In 01-01 we saw the memory hierarchy as far as RAM. Now we complete it downwards, which is where the differences turn brutal:

The jumps between adjacent levels are factors of 2 to 5, except for one:

The abyss lies between RAM and persistent storage, and that abyss is the reason the page cache exists, the reason for the buffering we calculated in 01-06 and the reason for the demand paging of the previous lesson. The entire design of an operating system's I/O side consists of avoiding that crossing.

An analogy on a human scale, scaling 1 ns up to 1 second:

When the aggregator causes a major fault, from the CPU's point of view it is like waiting three months for one piece of data.

An HDD is the only component with moving parts in a modern server, and its physics governs everything about its behavior.

The vocabulary, which you need to be clear about for the calculations:

The cylinder is a key concept: since all the heads move as one, reaching data in the same cylinder requires no arm movement, even if it lives on different platters. It is practically free.

An access has three time components:

The average rotational latency is exactly half a revolution, because on average the sector will be halfway round:

Let us work it out with the figures for meteo-01's disk:

Case 1: reading a 4 KB block at a random position.

Look at the proportion, which is what really matters:

99.8 % of the time goes on positioning, not on reading. That single figure explains the whole design of storage systems built on mechanical disks.

Case 2: reading 1 MB contiguously.

Reading 256 times as much data costs only 52 % more time.

Case 3: reading that same 1 MB as 256 scattered 4 KB blocks.

A direct comparison:

A mechanical disk is 168 times slower with random access than with sequential access. Three consequences follow that dominate system design:

1. Reading more than you asked for is almost free (read-ahead): since you have already paid for the seek, make the most of it.
2. Grouping scattered writes into a sequential one pays off enormously, even if it means writing more bytes. It is the foundation of log-structured file systems and of journaling (04-05).
3. Reordering requests to minimize arm movement has a gigantic impact. That is the subject of section 8.

Applied to Meteora, with the 17 MB a day in /var/lib/meteora/readings/:

The same data, 428 times slower. That is why the aggregator has to walk the file in order, and why the madvise(MADV_SEQUENTIAL) of the previous lesson makes sense.

An SSD has no moving parts. It stores bits as electrical charge trapped in NAND flash memory cells. This eliminates seek time and rotational latency, but it introduces new and very peculiar restrictions.

The internal structure:

And here is the asymmetry that explains everything:

A flash page cannot be overwritten. It has to be erased first, and erasing only works on entire blocks that are hundreds of times bigger than a page.

That restriction, which looks like a minor technical detail, has cascading consequences for the device's entire behavior.

Suppose you want to modify 4 KB inside a 2 MB block that is full of data. The naive sequence would be:

You have written 2 MB of flash in order to modify 4 KB. The amplification factor is 512.

No real SSD does this, precisely because it would be unacceptable. Instead, the controller writes the new 4 KB into an already-erased page somewhere else, and updates an internal table that translates logical addresses into physical ones. The old page is marked invalid, awaiting recycling.

That table is called the FTL (Flash Translation Layer) and it is, conceptually, a page table inside the SSD: the same indirection mechanism we studied in the previous lesson, applied to storage. Neither the operating system nor you ever see the flash's physical addresses.

Since invalid pages pile up, the SSD needs to recycle them. That process is garbage collection:

The cost: to free up space, data that was already written had to be copied. That is write amplification:

An almost-full SSD behaves far worse than one with free space. And it is not just a matter of performance: flash has a limited number of erase cycles per cell.

With this you can work out the service life of meteo-01's SSD:

Meteora's workload will never wear the SSD out. But change the scenario: a database server writing 500 GB/day with a WA of 5 would burn 2.5 TB/day, and those same 1,500 TBW would last 600 days. That is why TBW is a specification you have to check when buying SSDs for write-intensive servers.

If the controller always wrote to the same blocks, those would wear out while the rest stayed pristine. Wear levelling spreads writes evenly, and it even moves static data that has not changed in a long time in order to free up its lightly worn blocks.

It is another of the reasons the SSD needs indirection: the relationship between a logical address and a physical location changes constantly.

When you delete a file, the file system marks its blocks as free in its own structures, but the SSD never finds out: as far as it is concerned, that data is still valid and it will go on copying it during garbage collection.

The TRIM command (discard in NVMe) tells the SSD which blocks no longer hold useful data:

Without TRIM, the SSD ends up believing it is full even though the file system sees free space, and write amplification goes through the roof. Modern distributions enable fstrim.timer weekly, which is the recommended option over discard in the mount options (the latter issues a TRIM on every deletion and can hurt performance).

The most important row is the fourth: on an SSD, random access costs practically the same as sequential access. That single difference invalidates decades of optimizations designed for mechanical disks, including almost every scheduling algorithm we are about to look at.

The first SSDs connected over SATA using the AHCI protocol, designed in 2004 for mechanical disks. Its assumptions no longer held:

The structural difference is parallelism. With AHCI, a single queue means every core competes for it behind a shared lock. NVMe gives one queue per core, removing the contention entirely: each CPU submits its requests to its own queue without synchronizing with anybody.

And here it connects with what we know: an SSD has 4-8 internal channels working in parallel. Saturating it takes many simultaneous requests in flight. With a single queue of 32 commands that is impossible; with 65,535 queues, trivial.

The ROTA column (rotational) is the one to look at: 1 is a mechanical disk, 0 an SSD. The kernel uses it to decide read-ahead, the default scheduler and other policies. Checking it is always the first step when diagnosing disk performance.

Everything above — cylinders, heads, NAND channels, FTL — is hidden. The operating system sees a uniform abstraction: the block device.

That numbering is LBA (Logical Block Addressing). Before 1990, CHS addressing (cylinder-head-sector) was used, which exposed the physical geometry and capped capacity at 8 GB. LBA replaced it with a sequential number and the disk's firmware does the translation.

The abstraction gives the operating system exactly four operations:

This uniformity is what lets the same kernel code work over a floppy disk, an NVMe SSD, a RAID volume or a network disk. It is the same extended-machine idea we saw in 01-01, applied to storage. The flush() operation is especially important and will come back in Space Allocation, Journaling and Integrity, because an fsync() that does not reach the physical platter guarantees nothing.

With a mechanical disk, the order in which requests are served determines how much the arm moves, and we already know that is where 99.8 % of the cost lives. It is a scheduling problem analogous to the CPU one, but with a different metric: total head movement.

We will work through all the algorithms over the same queue:

Requests are served in arrival order.

The arm crosses the disk end to end several times. It is fair and simple, but dreadful.

The request closest to the current position is always chosen.

A 63 % reduction against FCFS. But SSTF is the disk equivalent of SJF, and it shares its flaw: starvation. If requests keep arriving near cylinder 60, the one at cylinder 183 may never be served.

The head moves in one direction serving everything it finds, reaches the end and comes back. Like an elevator.

Let us assume it moves towards lower cylinders:

The same total as SSTF here, but with no starvation: every request is served within at most two full sweeps. That guarantee is what makes it usable.

Its flaw: requests just behind the head wait a whole sweep, while those just ahead of it are served immediately. Waiting time is not uniform.

It only serves requests in one direction; on reaching the end it returns to the start serving nothing on the way back.

It travels further, but in exchange it offers a much more uniform waiting time: the disk behaves like a circular list and no area is systematically favored. It is the compromise usually preferred by systems where predictability matters more than average performance.

The obvious optimization of SCAN and C-SCAN: do not go all the way to the physical end if there are no requests there.

LOOK:

C-LOOK:

LOOK wins on total movement and is the one UNIX systems have historically used. C-SCAN and C-LOOK are preferable when the variance of the latency matters more than the mean.

And now the observation you must always keep in mind: this whole section assumes a head that moves. On an SSD, the "distance" between LBA 14 and LBA 183 is zero: both accesses cost the same. Reordering requests for an SSD not only fails to help, it adds latency and burns CPU for no benefit whatsoever. That is exactly what motivated Linux's none scheduler.

Linux lets you choose the I/O scheduler per device:

The brackets mark the active one. Notice the decision the kernel has taken all by itself: none for the NVMe, mq-deadline for the mechanical disk.

none does nothing: it hands requests to the device in arrival order. It sounds like surrender, but it is the right choice for NVMe: the device has its own parallel queues and its own internal scheduler, and it knows more than the kernel does about its own geometry. Any reordering by the operating system would be a mistaken guess.

mq-deadline keeps two queues sorted by LBA (one for reads, one for writes) plus two FIFO queues by deadline. It serves in LBA order — which gives LOOK-like behavior — except when a request is about to run out its deadline, at which point it jumps in to serve it. The default deadlines are revealing:

500 ms for reads and 5,000 ms for writes. Reads get ten times the priority, and the reason is solid: a read is almost always synchronous (there is a process blocked waiting for it, in state D), while a write is usually asynchronous (the process has already moved on, and the kernel will flush it when it can). Delaying a read blocks somebody; delaying a write does not.

bfq (Budget Fair Queueing) gives each process a budget of sectors and shares the bandwidth fairly, with weights configurable through ionice. It is what made the backup solution in lesson 02-02 possible:

Changing the scheduler takes effect immediately and needs no reboot:

To make it persistent you use a udev rule, a mechanism we will see in the next lesson.

A practical rule for choosing:

Disks fail. The annual failure probability of a server disk is around 1-2 %, which sounds small until you have 100 disks: then you expect one or two failures a year with practical certainty.

RAID (Redundant Array of Independent Disks) combines several disks into one logical volume. The levels differ in how they distribute data and redundancy.

Data is spread in stripes across all the disks. No redundancy.

• Capacity: 100 % of the sum.
• Performance: multiplied by N for both reads and writes.
• Fault tolerance: none. If one disk fails, the whole volume is lost.

And the statistical trap: RAID 0 is less reliable than a single disk. With 4 disks at a 2 % annual failure rate:

You multiply the risk by four. It only makes sense for reproducible data: caches, temporary files, intermediate results.

Every piece of data is written identically to all the disks.

• Capacity: 50 % with two disks.
• Reads: can be spread across both, up to 2× faster.
• Writes: the speed of one disk (both have to be written).
• Tolerates the failure of one disk without losing anything and with barely any degradation.

Data is split across N disks and a parity block computed with XOR is added, rotating which disk holds it.

The magic of XOR: if D1 is lost, it is recovered as D1 = D0 XOR D2 XOR P0. The same operation serves both to compute and to rebuild.

• Capacity: (N−1)/N. With 4 disks, 75 %.
• Tolerates one failure.
• Write penalty: modifying one block requires reading the old data, reading the old parity, computing the new parity and writing two blocks. Four I/O operations per write.

The serious risk of RAID 5 with large disks is the rebuild. Replacing an 8 TB disk means reading the other three in full:

Almost two days with the array degraded and without redundancy, subjecting the surviving disks — the same age and the same batch — to an intense read load. It is exactly when a second failure is most likely, and a second failure in RAID 5 means total loss. That is why RAID 5 is not advisable with disks larger than 2 TB.

Like RAID 5 but with two independent parity blocks.

• Capacity: (N−2)/N. With 6 disks, 66.7 %.
• Tolerates two simultaneous failures.
• Write penalty: six operations per write.

It is the answer to RAID 5's rebuild problem: during those 44 hours of rebuilding there is still one level of redundancy left.

Two-disk mirrors, joined together by striping.

• Capacity: 50 %.
• No parity penalty: two operations per write.
• Tolerates one failure per mirror; with luck, up to half the disks.
• Extremely fast rebuild: you only have to copy the mirror disk, not read the whole array.

With 8 TB disks, rebuilding means copying 8 TB: about 15 hours, and without putting the rest of the array at risk.

With 4 disks of 8 TB each:

And the indispensable warning:

RAID is not a backup. It protects against hardware failure, and against nothing else. It will not save you from an accidental rm -rf, from ransomware, from a file system failure, from a RAID controller failure or from a fire. A backup is a copy separated in time and in space.

Let us apply all of it to a concrete case. The requirements:

Analysis of the options:

Why RAID 1 with two SSDs:

1. The volume is small. 500 GB projected over 10 years fits into 1 TB with room to spare. Optimizing capacity contributes nothing here, and capacity is precisely the argument for RAID 5 and 6.
2. Writes are continuous and append-only. RAID 5 penalizes you with 4 I/O operations per write; RAID 1 with only 2. With the ingestor writing constantly, that difference matters more than capacity.
3. The rebuild is a plain copy. Replacing an SSD means copying at most 500 GB, about 15 minutes at 550 MB/s, without exposing the data the way RAID 5 would.
4. Reads improve: the aggregator and meteo-api can read from both disks in parallel.
5. Simplicity. RAID 1 is the easiest level to understand, set up and recover. In an emergency at three in the morning, that is worth more than 25 % extra capacity.

The configuration:

The [2/2] [UU] line is the one to keep an eye on: two disks out of two active, both U (up). If you saw [2/1] [U_], the array would be degraded and a disk would need replacing urgently.

And the part RAID does not cover: a daily copy to another server, in another physical location. RAID 1 protects against one SSD failing; only a backup protects against an accidental rm or a fire in the rack.

At a glance you have the complete topology: two NVMe drives forming md0 mounted on /var/lib/meteora, a 7.3 TB mechanical disk for /backup, and the right scheduler on each one.

iostat -x is the main diagnostic tool:

Column by column, with what each one means:

Reading this particular data:

• nvme0n1 and nvme1n1 show almost identical figures, as they should in RAID 1: the same writes to both, reads shared out.
• md0 adds up the reads (280.5 = 142.5 + 138) but does not double the writes (388), which is exactly the expected mirror behavior.
• An r_await of 0.08 ms on the NVMe drives. Excellent, within expectations.
• sda is the problem: a w_await of 28.4 ms, an aqu-sz of 3.44 and a %util of 92.1 %. The backup disk is saturated writing 18 MB/s. Since it is a mechanical disk under a write-intensive load, that is its normal behavior, but the backup had better not compete with anything critical (hence the ionice -c 3).
• A wrqm/s of 84 on sda: the kernel is merging many contiguous requests before sending them, a sign of sequential writing. That is what you want on a mechanical disk.

An important warning about %util: on mechanical disks it is reliable (they can only serve one request at a time, so 100 % means saturation). On NVMe it is misleading: the device serves dozens of requests in parallel, so it can read 100 % while running at 5 % of its real capacity. On SSDs, the reference metric is await, not %util.

And to measure the device's real capacity before putting anything into production:

487,000 random 4 KB IOPS with an average latency of 64 µs. Compare that with the ~120 IOPS of a mechanical disk: a factor of 4,000.

Applying mechanical-disk intuition to an SSD. Defragmenting an SSD achieves nothing and burns write cycles. Neither does reordering requests: in flash there is no distance. The first thing to do when analyzing performance is to look at lsblk -o NAME,ROTA and know what you are dealing with.

Trusting %util on an SSD. An NVMe drive at 100 % %util may be completely idle in real terms. Use await and aqu-sz.

Filling an SSD to the brim. Above 90 % occupancy, garbage collection loses its effectiveness and write amplification jumps from 2 to 10. Always leave 10-20 % free; many server SSDs reserve it at the factory (over-provisioning).

Using RAID 5 with large disks. With 8 TB disks, a 44-hour rebuild with no redundancy is a real risk of losing everything. For disks larger than 2 TB, use RAID 6 or RAID 10.

Confusing RAID with a backup. RAID protects against a disk failing. It does not protect against accidental deletions, logical corruption, ransomware or physical disasters. Those are different problems with different solutions.

Forgetting TRIM. With no fstrim.timer active, an SSD ends up with degraded write performance for no apparent reason. Check it with systemctl status fstrim.timer.

Changing the I/O scheduler without measuring. The default the kernel picks is usually right. If you are going to change it, measure before and after with fio or with the real workload; a lot of "optimization" changes make things worse.

Diagnostic tip: when you suspect a disk problem, this is the order that works: lsblk -o NAME,ROTA,SCHED (what you have), iostat -x 2 (look at await and aqu-sz), iotop -o (which process is doing the I/O) and cat /proc/mdstat if there is RAID (is it degraded?). And remember to look at vmstat's b column too: if there are processes in state D, you already know from lesson 02-01 that they are stuck waiting on exactly this.

meteo-01 has a backup disk with these characteristics:

1. Calculate the time to read a random 4 KB block.
2. Calculate the time to read the 17 MB of 2026-08-31.dat sequentially.
3. Calculate the time to read those same 17 MB as 4 KB blocks scattered across the disk.
4. How many random 4 KB IOPS does this disk deliver? Compare it with the NVMe drive's 487,000.
5. If the aggregator takes 90 seconds to process the day reading the file sequentially, what proportion of the time is CPU and what proportion is disk?

A disk running from cylinder 0 to 4,999 has its head at 2,150 and this request queue:

1. Calculate the total head movement with FCFS, SSTF, SCAN (upwards), C-SCAN (upwards), LOOK (upwards) and C-LOOK (upwards).
2. Rank them from best to worst.
3. Which one would you choose if the requirement were that no request should wait more than two full sweeps? And if it were to minimize the total time?
4. If this device were an NVMe SSD, what would your answer be and why?

Meteora wants to expand its storage. You have 6 disks of 4 TB and these requirements:

• Readings data (/var/lib/meteora): 500 GB, critical, continuous sequential writing, read-intensive.
• Logs and temporary files (/var/log, /tmp): 200 GB, reproducible, write-intensive.
• Historical archive (/archive): 8 TB, written once and read rarely, important but not critical.

1. Propose a RAID configuration for each of the three uses, stating how many disks you assign.
2. Calculate the usable capacity of each volume.
3. Justify each choice using the criteria from the lesson.
4. Calculate the rebuild time of each volume if a disk fails (assume 180 MB/s).
5. What is left uncovered by RAID and what would you do about it?

Solution 1

1. A random 4 KB block.

How the time breaks down:

2. The 17 MB sequentially.

In practice a 17 MB file is not perfectly contiguous, so there would be some extra seeking, but the order of magnitude is right: about 120 ms.

3. The same 17 MB in scattered blocks.

The same data, 459 times slower. It is the difference between the aggregator finishing before you have made a coffee and it taking nearly a minute just to read.

4. Random IOPS.

The comparison:

That factor of 6,408 is one of the most important figures in computing over the last decade. It explains why databases that were unworkable became trivial, and why many software optimizations designed to minimize random I/O stopped making sense.

A complementary reading of the figure: 76 IOPS means that disk cannot even serve 76 random requests per second. If meteo-api receives 40 requests per second and each one causes 2 random reads, that disk is already at its limit.

5. The CPU/disk split within the 90 seconds.

The aggregator is CPU-bound, not disk-bound. This quantitatively confirms what we deduced in 02-02 from its context switches (a ratio of 63 to 1 between voluntary and involuntary ones, far below that of the other processes).

The practical consequence is important: swapping the backup disk for an NVMe drive would not speed the aggregator up at all. You would save 0.1 seconds out of 90. To speed it up you have to optimize the computation, parallelize it or give it more CPU. It is the kind of conclusion that stops you spending money on the wrong component, and it is why you measure before you buy.

A note: this calculation would change completely if the aggregator read randomly. In that case it would be 54.7 s of disk against 90 s in total, 61 % disk, and the conclusion would be reversed.

Solution 2

Queue: 2069, 1212, 2296, 2800, 544, 1618, 356, 1523, 4965, 3681. Head at 2150. Disk from 0 to 4999.

Sorted: 356, 544, 1212, 1523, 1618, 2069, | 2150 | , 2296, 2800, 3681, 4965

FCFS:

SSTF:

SCAN (upwards, as far as 4999):

C-SCAN (upwards, jumping from 4999 to 0):

LOOK (upwards, without reaching the end):

C-LOOK:

2. The ranking:

3. The choice according to the requirement.

If no request may wait more than two full sweeps: SCAN, C-SCAN, LOOK or C-LOOK, all valid. What is decisive is ruling out SSTF, which offers no such guarantee: if requests kept arriving near 2150, the one at 4965 might never be served. The fact that SSTF happens to do well in this particular case changes nothing; the guarantee is about the worst case, not about one example. Among the four valid ones, C-LOOK is the best option if you also want the wait to be uniform, because it does not systematically favor any area of the disk.

If the goal is to minimize the total time: LOOK, with 7,424 cylinders. It wins because it wastes no movement travelling to empty ends (34 cylinders too many in SCAN) and makes none of the unproductive return trips of the circular variants.

A methodological note: the difference between LOOK (7,424) and SSTF (7,586) is only 2.1 %. With a different queue SSTF could win. LOOK's solid advantage is not the 2 %, it is the guarantee of no starvation at the same price.

4. If it were an NVMe SSD: none, and none of these algorithms.

The full reasoning:

• The metric loses its meaning. "Total head movement" measures the physical motion of an arm. In flash there is no arm. Reaching LBA 356 and LBA 4965 costs exactly the same: about 50 µs.
• Reordering is counterproductive. Sorting requests burns CPU and adds latency in the kernel queue, in exchange for a benefit of zero. You are paying to get nothing.
• The device knows more. The SSD has its own FTL, its 4-8 parallel channels and its own internal scheduler, which knows the real physical location — invisible to the kernel because of wear levelling. Any order the operating system imposes rests on a geometry that does not exist.
• NVMe wants parallelism, not order. With 65,535 queues, the optimum is to submit all 10 requests at once and let the device serve them in parallel across its different channels. Serializing them into an "optimal" order destroys precisely what makes NVMe good.

An estimate of the outcome: the 10 requests, submitted in parallel at a latency of ~50 µs each and with enough internal parallelism, would complete in roughly 50-100 µs in total. With the mechanical disk and LOOK, 7,424 cylinders of movement come to something like 150-200 ms. A factor of around 2,000, obtained not by scheduling better but by giving up scheduling altogether.

Solution 3

1 and 2. The proposed configuration.

The calculations in detail:

The critical data (500 GB) and the reproducible data (200 GB) fit together into the 4 TB RAID 1 with plenty of headroom, in separate subvolumes or logical partitions. Dedicating separate disks to /var/log would waste two 4 TB disks on 200 GB.

3. The justification for each choice.

/var/lib/meteora on RAID 1:

• Maximum criticality. Lost readings are unrecoverable, so redundancy is non-negotiable.
• Continuous sequential writing. RAID 1 costs 2 I/O operations per write; RAID 5 costs 4 (read data, read parity, write data, write parity). With the ingestor writing non-stop, that difference is paid every second.
• Read-intensive. RAID 1 lets reads be served from both disks in parallel, which directly benefits the aggregator and meteo-api.
• There is capacity to spare. 500 GB needed against 4 TB available: there is no argument for sacrificing performance or simplicity in exchange for more space.

/var/log and /tmp on the same RAID 1:

• They are reproducible, so technically RAID 0 would do. But dedicating disks of their own would squander 8 TB raw on 200 GB.
• Living alongside the critical data gives them redundancy for free, which is a welcome bonus: losing the logs exactly when you are diagnosing a disk failure is the worst possible moment.
• A necessary precaution: /var/log growing unchecked could fill the volume and block the writing of readings. It has to be capped with logrotate and journald (SystemMaxUse=), or isolated into a subvolume with a quota.

/archive on RAID 5:

• Here capacity is what rules: 8 TB needed out of 12 TB usable. RAID 10 with the same 4 disks would give only 8 TB, right at the limit and with no room to grow.
• It is written once and read rarely. RAID 5's write penalty is irrelevant in a workload that hardly writes at all: it is precisely the scenario RAID 5 was designed for.
• It is important but not critical: if it were lost, it would be recoverable from the backups and by reprocessing the original data.
• The disks are 4 TB, not 8, which puts the rebuild in an acceptable range, as we will see in the next point.

4. Rebuild times at 180 MB/s.

RAID 1 (4 TB):

And with only 700 GB of real data, many implementations (mdadm with a bitmap, or ZFS/btrfs) copy only the blocks actually in use:

During the rebuild, the surviving disk carries on serving requests normally.

RAID 5 (4 disks of 4 TB):

Comparison and risk:

RAID 5's 18.5 hours are acceptable — a long way from the 44 hours of the 8 TB case we saw in the lesson — but they deserve two concrete precautions:

The periodic check is the most important of the three: the scenario that kills RAID 5 is discovering an unreadable sector on disk B right in the middle of rebuilding disk A. Reading the whole array every week finds those sectors while there is still redundancy to repair them.

5. What RAID does not cover.

RAID protects only against the physical failure of a disk. Left out:

The reference rule is 3-2-1: three copies of the data, on two different media, with one off site.

--link-dest creates hard links to the previous copy, so each day takes up only what has changed but looks like a complete, independent copy. At 17 MB a day, a year of daily history takes about 6.2 GB instead of 6.2 TB.

And one last piece that everyone forgets: a backup that has never been restored is not a backup, it is a hope. You have to schedule a periodic test restore and verify that the recovered data is readable and correct.

Storage is the level of the hierarchy where the abyss appears: from RAM to an SSD there is a factor of 500, and to a mechanical disk a factor of 80,000. The entire design of the I/O subsystem consists of avoiding that crossing.

A mechanical disk spends 99.8 % of its time positioning — seek plus rotational latency — and only 0.2 % transferring, which makes it 168 times slower at random access than at sequential access. From that come read-ahead, the grouping of writes and the whole business of request scheduling. An SSD turns the rules upside down: with no moving parts, random access costs the same as sequential, but it brings restrictions of its own — you cannot overwrite without erasing the whole block — that force an FTL, garbage collection, wear levelling and TRIM, and that make an almost-full SSD behave far worse than one with room. NVMe is not just a faster connector: it replaces one queue with 65,535 and removes the contention between cores.

On top of that physics, the operating system builds the block device abstraction addressed by LBA, with only four operations, which is what lets the same code serve a floppy disk and a RAID array. Request scheduling — FCFS, SSTF, SCAN, C-SCAN, LOOK, C-LOOK — cuts head movement by up to 67 %, but it only makes sense if there is a head to move: on NVMe the right choice is none, that is, not to schedule at all. Linux's real schedulers reflect exactly that, and mq-deadline also builds in a very well-reasoned asymmetry: reads get a deadline ten times shorter than writes, because behind a read there is almost always a process blocked in state D.

RAID combines disks for capacity, speed or reliability, and the choice depends on the pattern: RAID 1 for critical data with continuous writing and a small volume — the case of /var/lib/meteora — RAID 5 only with moderate disks and light writing, RAID 6 or 10 when the rebuild would be long. And the warning that is never superfluous: RAID is not a backup. Finally, lsblk tells you what you have and iostat -x how it is behaving, with await as the reference metric and %util as the trap on SSDs.

So far we have treated the disk as the device. But a server has network cards, terminals, clocks, sensors and controllers of every kind, and the operating system has to offer a coherent interface for all of them without knowing the details of any. How it manages that — the classification into block, character and network devices, the principle that everything is a file, /dev and udev, and the three ways the CPU has of talking to a device — is what we will see in Device Management.