The previous lesson ended with a fact that deserves an explanation. filefrag told us that the 4,219 blocks of 2026-08-31.dat are in one single extent, perfectly contiguous from block 8,394,271 to 8,398,489. And yet that file was written 24 bytes every 125 milliseconds for 24 hours, while meteo-api, the aggregator, the system logs and half a dozen other services were writing to the same volume. How does a file written like that end up contiguous?

That is the first half of this lesson: how a file system decides which blocks a file occupies, how it keeps track of the free ones, and how it represents that list inside an inode that has only 60 bytes for pointers. We will go through the four historical strategies — contiguous, linked, indexed and extent-based — with Meteora's concrete numbers, and along the way you will understand why FAT32 is slow with large files and why defragmenting no longer makes sense.

The second half is more serious. Adding a block to a file is not one operation, it is three: writing the data, marking the block as used in the bitmap, and updating the inode with its new address and size. If the power fails between those three writes, the file system is left in an inconsistent state that can range from a harmless loss of space to the cross-corruption of two files. We will see how the journal turns three operations into an atomic transaction, exactly what each of ext4's three modes guarantees, and why metadata consistency is not the same as data integrity: a bit that gets corrupted on the platter goes unnoticed by ext4, by XFS and by Meteora's RAID 1.

1. Contiguous allocation: fast and fragile
2. Linked allocation and the FAT table
3. Indexed allocation: the inode and its indirect pointers
4. Extents: the modern solution, measured on Meteora
5. Free space management
6. Fragmentation: why ext4 and XFS fragment so little
7. The consistency problem: three writes and a power cut
8. fsck and why its cost is unacceptable
9. The journal: transaction, commit, checkpoint and recovery
10. ext4's three modes and which one Meteora chooses
11. Alternatives to the journal: copy-on-write and log-structured
12. Data integrity: checksums and silent corruption
13. Write barriers and disks that lie
14. Snapshots and backups: another layer

The simplest idea: each file occupies a set of consecutive blocks. The inode only needs to store two numbers, the starting block and the length.

Its advantages are notable. Sequential reading is optimal: a single I/O request brings in the whole file, with no seeking. Random access is trivial: the block for byte N is at start + N/4096, one division. And the metadata is minimal: eight bytes describe a file of any size.

But it has two problems that rule it out for general use:

External fragmentation. As files of different sizes are created and deleted, the free space ends up chopped into small holes. You can have 40 GB free and be unable to create a 100 MB file because the largest contiguous hole is 60 MB. It is exactly the external fragmentation we saw in Memory Management with variable-partition allocation, and the solution would be equally expensive: compaction, that is, physically moving gigabytes of data.

It cannot grow. The ingestor starts the day with an empty file and appends 17 MB to it over 24 hours. With contiguous allocation you would have to declare the final size in advance, and if you fall short, the only way out is to copy the whole file into a bigger hole.

That is why contiguous allocation survives only where the content is immutable and known in advance: CD-ROM and DVD (ISO 9660), and modern extents, which are "contiguity in chunks" and recover its advantages without its problems.

The opposite alternative: each block holds a pointer to the next one, and the inode only points to the first. The blocks can be anywhere.

It solves both previous problems at once: there is no external fragmentation — any free block will do — and growing is trivial: you take a free block and link it. But it introduces two others, worse:

• Random access is O(n). To read block 4,000 you have to read the previous 4,000, because the pointer to the next one is inside each block. An lseek stops being free.
• The pointer steals space from the block. If the pointer takes 4 bytes, a 4,096-byte block only stores 4,092 of data, and reads stop being page-aligned.

FAT's solution. MS-DOS's file system solved the second problem by moving all the pointers out of the blocks into a single table: the File Allocation Table. It is an array with one entry per block — called a cluster in FAT terminology — whose value is the number of the file's next cluster, or a special value:

A file starting at cluster 100 is followed by reading FAT[100] = 250, FAT[250] = 251, FAT[251] = 999, FAT[999] = EOF. The directory entry only stores the first cluster.

The improvements over the pure linked list are real: the data blocks are left whole for data, and if the FAT fits in memory, walking the chain costs no disk accesses. But the underlying problems remain:

Random access is still O(n) in the table. To reach cluster 4,000 you have to walk 4,000 entries. If the FAT is in RAM that is 4,000 memory accesses — fast but not free; if it does not fit, it is thousands of disk accesses.

The FAT grows with the volume and has to be kept in RAM. With 4 KiB clusters, a 1 TB volume has 268,435,456 clusters, and at 4 bytes per entry the table is 1 GiB. That is the real reason FAT32 is not used on large volumes: it is not the 4 GiB per file limit, it is that the allocation table becomes unmanageable.

The FAT is a single point of failure. If it gets corrupted, all the chains of all the files are lost. That is why FAT keeps two copies and why chkdsk exists.

And there is no journal. A power cut while the FAT is being updated leaves broken chains, lost clusters and cross-links: two files whose chains converge and share clusters, so that writing to one corrupts the other.

UNIX's solution: gather all of a file's pointers in its own inode, instead of in a global table. Each file has its index, and only the index of the file you are using is read.

The immediate problem is size. The inode is 256 bytes and reserves 60 bytes for pointers: with 4-byte pointers, 15 fit. With 4 KiB blocks, that is 60 KiB. Ridiculous.

UNIX's classic solution is elegant: the fifteen pointers are not all the same.

The calculation, with 4 KiB blocks and 4-byte pointers, is the one you should know how to do:

The beauty of the scheme is that the cost is proportional to the size. A 40 KiB file uses only direct pointers: reading any of its blocks costs one access, because the address is in the inode you have already read. Only enormous files pay for the three levels of indirection:

Now, 2026-08-31.dat under this scheme. It needs 4,219 blocks:

• The first 12 go in the direct pointers.
• The next 1,024, in the single indirect: 1,036 cumulative.
• The remaining 3,183 go in the double indirect, which needs ⌈3,183 / 1,024⌉ = 4 second-level blocks.

Total metadata blocks: 1 (single indirect) + 1 (double's root) + 4 (second level) = 6 blocks, 24 KiB. And reading the file's block 4,000 costs 3 accesses: the double's root, the second-level block and, finally, the data.

An important implementation detail: a pointer block full of zeros represents a hole. That is how ext2/ext3 implement the sparse files of 04-04 with no additional structure at all.

Indirect pointers have a fundamental flaw: they store one address per block, even when the blocks are consecutive. For a contiguous 17 MB file, they store 4,219 numbers running from 8,394,271 to 8,398,489, one after another. It is a redundant list.

An extent is a contiguous range described with three numbers: starting logical block, starting physical block and length. In ext4 it takes 12 bytes:

The inode's 60 bytes hold a 12-byte header plus 4 extents. If a file needs more than 4, those 60 bytes come to describe the root of an extent tree, with internal nodes in separate blocks.

The comparison, on 2026-08-31.dat:

From 16,876 bytes of metadata to 12: a 1,400× reduction. And we verified it in 04-04:

One extent for the whole file. Compare with a fragmented file, which is what you would see on a full volume:

3,847 extents means 3,847 jumps: on an HDD, 3,847 seeks of 8 ms each, more than 30 seconds just positioning the head.

XFS has used the same idea with B+ trees since 1994, and its extents reach 2 million blocks (8 GiB) each. The comparison table of the four strategies:

The other half of the problem: knowing which blocks are free. There are four approaches.

Bitmap. One bit per block: 1 used, 0 free. It is what ext4 uses, and we saw it in 04-01.

• Size: 1 bit per 4 KiB block = 32 KiB of map per GiB, or 6.25 MiB for 200 GiB.
• Finding a free block: scanning bits, accelerated by CPU instructions that examine 64 at a time.
• Decisive advantage: finding a contiguous range is trivial — look for N consecutive zeros — and that is exactly what is needed to allocate extents.

Linked list of free blocks. Each free block points to the next. It takes zero extra space, but it does not let you find contiguous ranges: the blocks come out in the order they were freed, which is random. That is why a system with a linked list fragments hopelessly.

Grouping. A free block contains the addresses of the next N free blocks. It reduces disk accesses compared with the simple list, but it still does not help with contiguity.

Counters / trees. Storing pairs (first free block, how many consecutive). Very compact when free space is not very fragmented, and if organized in a B+ tree indexed by length it can answer "give me 4,219 consecutive blocks" in O(log n). That is what XFS does, with two B+ trees per allocation group: one ordered by address and another by size.

Back to the opening question: how does a file written 24 bytes every 125 milliseconds for a day end up contiguous? The answer is three mechanisms working together.

1. Delayed allocation. It is the most important and the most counterintuitive. When the ingestor calls write(), ext4 allocates no block at all: it only marks the page as dirty in the cache (04-04) and notes how much space will be needed. The real allocation is postponed until the kernel is actually going to flush.

The consequence is enormous: when the moment to allocate arrives, the system already knows how many blocks the whole set needs, instead of having to decide one at a time without knowing whether there will be more. Instead of 4,219 isolated decisions, it makes a few informed ones and asks for large ranges.

2. Multiblock allocation. With that information, ext4 asks the allocator for a contiguous range of the required size in one go. The bitmap, which is good at finding consecutive zeros, gives it to it. That is where the single extent comes from.

3. Block groups and preallocation. The 128 MiB groups of 04-01 keep the inode and its data together, and ext4 additionally preallocates speculatively a margin behind a file that is growing, reserving it for that file. If the file goes on growing, it extends into its own reservation instead of jumping elsewhere; if not, the reservation is released on close.

The three together explain the result: the ingestor writes for 24 hours and ext4 keeps extending a single extent, because every time it needs more space it had already reserved it right behind.

You check a volume's overall state with:

0.4 % of non-contiguous files. Compare with FAT32 after a year of use, where 30-40 % is normal.

Three cumulative reasons:

1. Modern file systems do not fragment much, thanks to the three mechanisms above. An ext4 below 85 % occupancy stays under 2 % non-contiguous files indefinitely.
2. On an SSD, fragmentation is almost irrelevant. There is no head to move: accessing any page costs the same (02-05). What does matter is the number of requests, and there a file with 3,847 extents is still worse than one with 1, but the difference is a factor of 2 or 3, not 100.
3. Defragmenting an SSD wears it out. Moving 200 GB means 200 GB of write cycles consumed for a marginal benefit. It is actively harmful.

The only case where it still makes sense is a very full HDD with large, heavily fragmented files. For that there are e4defrag on ext4 and xfs_fsr on XFS, always online and on specific files:

Practical rule: keep your volumes below 85 % occupancy. It is infinitely more effective than defragmenting, because an allocator that cannot find large holes necessarily fragments. Fragmentation is a symptom of a full volume, not a disease in its own right.

We change topic and severity. When the ingestor adds a block to 2026-08-31.dat, the file system must perform three writes in three different places on the disk:

1. The data block, in the data area.
2. The block bitmap, marking that block as used.
3. The inode, with the new address (or the extended extent) and the updated size.

These three writes cannot be atomic: they are in different places on the device, and the disk only guarantees atomicity at the sector level. A power cut can happen at any intermediate point, and each combination leaves a different state:

The two rows marked SEVERE are so for the same reason, and it is worth understanding it well: if the inode points to a block the bitmap considers free, the allocator will hand it to the next file that asks for space. From that moment on two files share one physical block, and writing to one corrupts the other. That is the cross-link, the worst possible corruption in a file system, because it is silent and it spreads.

The "moderate" row is instructive too: the file grows in size and its new block contains whatever was previously in that spot on the disk. If that block belonged to /etc/shadow or to another user's file, you have just leaked somebody else's data into your file. It is a security problem, not just a consistency one, and it will come up again when choosing the journal mode.

The traditional solution was to repair afterwards. At boot, if the superblock was marked "dirty", fsck (file system check) was run, walking the whole file system checking invariants:

It works, and its repair capability is real. The problem is the cost, which grows with the volume's size:

A server that takes eight hours to boot after a power cut is not acceptable. And there is an added problem: fsck cannot recover lost information, only return the system to a consistent state. Faced with a cross-link, its remedy is to duplicate the block or discard it: the system ends up consistent, but one of the two files is corrupt beyond repair. Consistency is not correctness.

That cost is what motivated the journal. With it, recovery after a power cut goes from walking 13 million inodes to re-reading a few megabytes of journal: from minutes or hours to under a second.

The idea comes from databases and is called write-ahead logging: before modifying anything, write down in a sequential log what you are about to do. If the system goes down, that log is re-read and whatever was pending is either completed or discarded.

In ext4, the journal is an area reserved inside the file system itself — typically 128 MiB — used as a circular buffer, managed by a subsystem called JBD2.

The four steps of a transaction:

And now what happens if the power fails at each point:

The key to everything is the commit block and its checksum:

The commit block is atomic: it is written whole or not at all. Its presence and its correct checksum mean "this transaction is complete in the journal". Its absence means "this never happened".

With that, the system can never be left half-done. The three writes of our example — data, bitmap, inode — are grouped into one transaction, and after recovery either all three are there or none is. The atomicity the disk does not provide is built in software.

Recovery in practice:

That recovery complete is the line that appears after a power cut, and it appears in under a second, because all that is needed is to re-read the journal. You can inspect the journal with sudo dumpe2fs /dev/md0 | grep -i journal, which confirms its size and its state.

Two clarifications so as not to overrate the mechanism:

• Writing twice costs. Every piece of metadata is written to the journal and then to its place, which amplifies writes. That is why the journal is batched — many operations in one transaction — and committed every few seconds (commit=5 by default), not on every operation.
• The journal protects the structure, not necessarily your data. Exactly what it guarantees depends on the mode, which is the next section.

ext4 offers three policies over what goes into the journal, and the difference between them is very real:

The case that separates ordered from writeback deserves detail, because it is exactly the "moderate" row of the state table in section 7.

With writeback, the journal guarantees that the inode will be consistent: it will say the file is 17,280,024 bytes and will point to the new block. But it does not guarantee that that block's data was written. After a power cut you can find a file whose size has grown and whose final block contains whatever was previously in that spot on the disk: remnants of a deleted file, perhaps another user's. The file system is perfectly consistent and fsck sees nothing odd, but you have read data that was not yours. It is a security problem, not just an integrity one.

With ordered, ext4 imposes a simple rule: new data blocks are written to the disk before committing the transaction that references them. That way, if the commit is there, the data is there. If the commit is not there, the file keeps its previous size and the block was never its. Foreign content can never appear inside a file. And all that without writing the data twice: it is merely ordered.

With data=journal the data also goes through the journal, which gives the strongest guarantee — the content of a committed write() survives intact — at the cost of writing every byte twice, with a 30-50 % write penalty. Curiously, it can be faster for small, random write workloads, because the journal is sequential; but for continuous writing it is clearly worse.

You query and change it like this:

What Meteora chooses: data=ordered. The full reasoning:

1. writeback is ruled out on security grounds. /var/lib/meteora is a volume shared with the historical archive; that a power cut could leave remnants of somebody else's blocks inside a readings file is unacceptable, and it would also ruin the format: meteo-api would interpret that garbage as Reading structs with absurd temperatures. Corrupt weather data is worse than absent data, because nobody notices.
2. data=journal is not worth it. Its additional guarantee is that already-committed data survives, but we already control that from the application with the fdatasync every 5 seconds of 04-04. Paying 30-50 % of the performance and double the SSD wear to duplicate a guarantee we already have makes no sense.
3. ordered gives exactly what we need: foreign content will never appear, the structure always ends up consistent, recovery takes under a second, and the cost over writeback is a small percentage.

And a note that closes the circle with 04-04: Meteora's format helps a great deal here. Since the file is a sequence of 24-byte records appended at the end, a power cut leaves at most one incomplete record at the end, which the reader detects with size % 24 != 0 and discards. A format with global structure — an index at the beginning, or compression — would not have that property. The choice of data format is part of the integrity strategy.

The journal is not the only way to survive a power cut.

Copy-on-write (CoW), in Btrfs and ZFS. The idea is radical: a live block is never overwritten. Modifying a block means writing a new copy in a free spot, and then updating the pointer that pointed to the old one. But that pointer is in another block, which is not overwritten either: it is copied too, and so on up to the root of the tree. In the end, a single atomic write of the root superblock makes the whole change visible at once.

The fragmentation row is CoW's real drawback and the reason it is not automatically the best option: a database that rewrites records at random fragments a CoW system far more than a journaled one. That is why Btrfs offers chattr +C to disable CoW on specific files.

Log-structured systems, such as F2FS. They take the idea to the extreme: the whole file system is a sequential log. Every write, of data or of metadata, is appended at the end; nothing is modified in place. Random writes become sequential, which is exactly what suits flash memory (02-05), where erasing works on large blocks and wear is spread better this way. The price is that a garbage collector is needed to compact the blocks with obsolete data, and that process competes with the real workload. It is the reason F2FS dominates on phones and SD cards and is hardly used on servers.

Here we reach the distinction that gives the last third of the lesson its name, and that many people are not clear about:

The journal guarantees that the file system's structure is consistent after a power cut. It does not guarantee that the data you read is the data you wrote.

They are different problems with different causes. Silent data corruption (bit rot) is a bit that changes value without anybody asking, from physical causes:

The two worst are misdirected and lost writes, because the disk believes everything went fine and reports no error. The classic CERN and NetApp studies over millions of disks found rates on the order of one corrupt sector per 10¹⁴-10¹⁵ bits read: with 200 GiB re-read daily, that is one event every several years per volume. Little, but not zero, and with unrecoverable data it matters.

What each layer protects:

This is the most surprising section, and it directly affects Meteora, whose /var/lib/meteora has been on RAID 1 since 02-05.

RAID 1 keeps two identical copies. If a disk fails visibly — it does not respond, it returns a read error — the system uses the other one and everything works: that is what it is for. But if a disk returns corrupt data without reporting an error:

RAID 1 can detect that the two copies differ, but it does not know which one is the good one. It has no information with which to decide, because it stores no checksums of the data.

Worse still: in normal operation, md reads from a single disk for performance — spreading requests between the two — so it does not even compare. The discrepancy is only discovered if an explicit scrub is run:

A non-zero mismatch_cnt means the copies differ. And then comes the uncomfortable moment: the system cannot tell you which one is correct. repair syncs by copying the first disk over the second, which fixes the discrepancy... by choosing at random. If the good one was the second, you have just propagated the corruption to both.

What ZFS or Btrfs bring is precisely that: they store a checksum of every data block in the tree's parent node. When they read a block, they check its checksum; if it does not match, they know that block is bad, they go to the redundant copy, verify its checksum, and if it is correct they return it and repair the original. That is self-healing, and it is qualitatively different from what RAID 1 can do.

How Meteora compensates for this blind spot, having chosen ext4 in 04-01:

1. metadata_csum enabled, protecting inodes, group descriptors, bitmaps and the journal. The structure will not be corrupted silently.
2. Weekly RAID scrub with check, and an alert if mismatch_cnt is non-zero. It does not repair on its own, but it warns, which is the minimum.
3. A CRC32 per record in the format itself. The ingestor stores a checksum with every reading, and the aggregator verifies it. This is the layer that really closes the gap: even if the file system and the RAID see nothing, the application detects the corrupt reading and discards it.
4. Verified backups, which are the last safety net.

Point 3 is the general lesson: when storage cannot give you the guarantee you need, the application can. A 4-byte CRC per 24-byte record makes storage 17 % more expensive and turns silent corruption into a detected error.

One link remains, and it is the one that can invalidate everything above.

The whole journal mechanism rests on an order: the journal's blocks must reach the medium before the commit block, and that before the final blocks. But between the file system and the platter there is a volatile cache in the disk itself: a few hundred megabytes of DRAM where the device accumulates writes and reorders them to be more efficient.

If the disk reorders and confirms before writing, the commit can reach the medium before the data it backs. A power cut at that instant leaves a transaction marked as complete whose data does not exist, and recovery will apply it confidently. The journal would have made things worse.

The solution is write barriers, explicit orders to the device:

ext4 uses them by default (barrier=1). The cost is real — a barrier can cost milliseconds — and that is why the barrier=0 option exists, which disables them.

Never mount with barrier=0 unless you have a RAID controller with a battery or supercapacitor that guarantees its cache is flushed on a power cut. Without that, disabling the barriers turns the journal into an ornament.

And the final problem, which has no software solution: some disks lie. Some consumer devices — and many cheap USB sticks and SD cards — ignore the flush command and confirm immediately, because that scores better in benchmarks. With one of those, every guarantee evaporates: the file system believes it has imposed an order that the hardware has not respected.

Whether a disk respects barriers cannot be checked from the operating system; it requires a test rig with a real power cut. The practical consequence is a purchasing decision, not a configuration one: for data that matters, use disks with power loss protection, which carry capacitors to flush their cache to the medium when the power goes. It is the main difference between a consumer SSD and a server one, and it explains a good part of the price.

To close, a distinction that is constantly confused and that is worth making clear:

The three ideas to take away:

RAID is not a backup. It is high availability: it keeps the service running when a disk fails. An rm -rf is replicated to both disks instantly, and a fire takes both.

A snapshot is not a backup. It lives on the same volume — or in the same volume group — as the original data. If the volume dies, both die. Its value is a different one: giving a consistent point from which to take the backup without stopping the service, which is exactly what we used it for in 04-03.

An unverified backup is not a backup. Really restoring it, on another machine, every so often, is the only way to know it works. The statistics on backups that failed the day they were needed are depressing.

Meteora's complete scheme, with each layer covering what the previous one cannot:

None of the seven is redundant, and none replaces another.

Believing that the journal protects your data. In data=ordered, which is the normal case, the journal protects the metadata. A write() not confirmed with fsync is lost in a power cut all the same, even though the file system is left immaculate.

Mounting with barrier=0 to gain performance. Without a battery-backed controller, that disables the ordering guarantee and turns the journal into decoration. The performance you gain, you pay for the first time the power goes.

Trusting writeback because it is the fastest. Its risk is not just losing data: it is that content from other files appears inside yours, with the security implication that carries.

Believing that RAID 1 protects against silent corruption. It detects discrepancies only if you scrub, and even then it does not know which copy is the good one. If you need that guarantee, you need data checksums: ZFS, Btrfs or your own application.

Defragmenting an SSD. It brings no measurable benefit and it consumes write cycles. If you have fragmentation on an ext4, the problem is almost always that the volume is over 85 % full.

Filling a volume above 90 %. The allocator stops finding contiguous ranges, fragmentation shoots up and performance collapses. Watch 85 % as your warning threshold.

Confusing a snapshot with a backup. They share their destination with the original data. If the volume dies, you have nothing.

Tip: enable metadata_csum and scrub the RAID weekly. They are two almost zero-cost measures that turn invisible corruption into an alert.

Tip: design the data format with the power cut in mind. A file of fixed-size records appended at the end recovers by discarding the last incomplete record. A format with a global index or compression does not. And a CRC per record closes the gap the file system does not cover.

For a file system with 8 KiB blocks and 4-byte pointers, calculate: (a) how many pointers fit in a block; (b) the maximum file size under the scheme of 12 direct pointers, one single indirect, one double and one triple, showing each level's contribution; (c) how many metadata blocks and how many extra accesses a file of 17,280,000 bytes needs under that scheme; (d) the same with ext4 extents, assuming the file is in one contiguous run. Comment on what changes relative to the lesson's calculation with 4 KiB blocks.

On your machine, use filefrag to analyze at least ten files of very different sizes — from a few KB up to several GB if you have them — and build a table with size, number of extents and extents per GiB. Then run sudo e2fsck -fn on an unmounted file system (or interpret the non-contiguous line of a df/dumpe2fs) and judge whether your volume is fragmented. Explain what relationship you observe between file size, volume occupancy and fragmentation, and decide with reasons whether you would defragment anything.

For each of these three workloads, choose between data=journal, data=ordered and data=writeback, and justify the decision by analyzing what is lost and what is risked in a power cut: (A) the /var/lib/meteora volume with the readings; (B) a volume of regenerable image caches, where write performance is the only thing that matters; (C) a volume for a financial database with transactions. For each one, also state which other measures from the lesson you would add and why.

Solution 1

(a) Pointers per block = 8,192 / 4 = 2,048.

(b) Maximum size:

Doubling the block size multiplies the maximum by 16, not by 2: each level of indirection contributes a factor of 2 from the block size and another factor of 2 from fitting twice as many pointers, and with three levels that is 2⁴ = 16. It is a lovely example of non-linear growth.

(c) 17,280,000 / 8,192 = 2,109.375 → 2,110 blocks. The first 12 are direct; 2,098 remain, which fit entirely in the single indirect (2,048)... not quite: 2,098 > 2,048, so 2,048 go to the single indirect and 50 to the double.

Metadata blocks: 1 (single indirect) + 1 (double's root) + 1 (one second-level block for the 50) = 3 blocks = 24 KiB. Extra accesses: 1 for the single indirect's blocks, 2 for the 50 in the double.

With 4 KiB blocks it was 6 metadata blocks and up to 3 extra accesses: the larger block halves the metadata and saves one level of indirection, at the cost of more internal fragmentation (04-01). Remember that on Linux this is theoretical, because the block cannot exceed the page size.

(d) With extents and one contiguous run: 1 extent of 12 bytes inside the inode, 0 metadata blocks and 0 extra accesses. It makes no difference whether the block is 4 or 8 KiB: contiguity is what eliminates the problem, not the block size.

Solution 2

Typical results on a healthy ext4 at 60 % occupancy:

And on a volume at 94 % occupancy, the same 4.2 GB file can come out with 900 extents.

Relationships observed. Small files (< 128 MiB) almost always come out in a single extent, because an ext4 extent reaches 128 MiB and delayed allocation sees the whole file before allocating. Large ones have several, but few: fragmentation grows far more slowly than size. And the determining variable is not the size but the volume's occupancy: above 90 % the allocator stops finding large holes and fragmentation multiplies.

Would I defragment? Almost certainly no. On an SSD, never: there is no measurable gain and there is wear. On an HDD, only a specific, heavily fragmented file (thousands of extents) that is read sequentially often, with e4defrag on that file. And the really useful action would be to free space until dropping below 85 %, which attacks the cause instead of the symptom.

Solution 3

(A) /var/lib/meteora → data=ordered. It is the lesson's decision. writeback is ruled out because a power cut could leave blocks with foreign content inside a readings file, and meteo-api would interpret them as valid Reading structs with absurd values: undetectable corrupt data is worse than absent data. data=journal would double the writes with a 30-50 % penalty to give a guarantee already covered from the application with fdatasync every 5 seconds. Additional measures: metadata_csum, weekly RAID scrub, CRC32 per record, and the fixed-size record format that allows discarding the last incomplete one.

(B) Regenerable image cache → data=writeback. Here, yes. The security reasoning that ruled out writeback in (A) applies only because in (A) the content matters; if a cache file is left with garbage, it is detected on use and regenerated, which is the definition of a cache. You get maximum write performance. Additional measures, consistent with the nature of the data: do not back up this volume; mount it noatime,nosuid,nodev,noexec (04-03); consider tmpfs outright if it fits in RAM; and validate every entry as you read it — with a checksum or a version identifier — discarding and regenerating any that does not match. An even more aggressive option would be mkfs.ext4 -O ^has_journal, with no journal: after a power cut you would have to run fsck or simply reformat, which for a cache is perfectly acceptable.

(C) Financial database → data=ordered, not data=journal. This is the most deceptive case. Intuition says "as safe as possible, data=journal", and that is the wrong answer for two reasons. First, a serious database already has its own journal — PostgreSQL's WAL, Oracle's redo log — and syncs with fsync on every transaction commit: the file system's data journal would duplicate exactly the same work, writing every byte four times in total. Second, that duplication costs 30-50 % of the performance in the part of the system most sensitive to latency. data=ordered gives the necessary structural guarantee and leaves transactional durability where it belongs: in the layer that understands what a transaction is.

Additional measures for (C), which are where the matter is really decided: barriers enabled and disks with power loss protection, because a disk that lies invalidates the fsync the WAL leans on; ECC RAM, since a flipped bit in memory corrupts the data before writing it and no file system checksum will detect that; RAID with scrubbing; page-level checksums, which PostgreSQL offers with data_checksums; and backups with verified restores, plus continuous WAL archiving so you can recover to a specific instant.

The cross-cutting lesson of the three cases: the guarantee belongs in the layer that has the information to provide it, and duplicating it across several layers costs performance without adding safety.

The four allocation strategies answer the same question with different trade-offs. Contiguous is optimal for reading and for metadata — eight bytes describe any file — but it dies from external fragmentation and from being unable to grow, so it survives only on read-only media. Linked allocation and its table variant, FAT, eliminate external fragmentation in exchange for O(n) random access and a table that grows with the volume: 1 GiB of FAT for 1 TB with 4 KiB clusters, plus a single point of failure and no journal. UNIX's indexed allocation gathers the pointers in the inode and solves the size problem with tiered indirection — 12 direct, single, double and triple — which with 4 KiB blocks reaches 4.004 TiB and makes the cost grow with the size: 0 extra accesses up to 48 KiB, 3 for the largest files. And extents describe contiguous ranges with 12 bytes: the 4,219 blocks of 2026-08-31.dat go from 16,876 bytes of pointers and 6 metadata blocks to a single extent, verified with filefrag.

That the file ends up contiguous despite being written 24 bytes every 125 ms for a day is down to three mechanisms: delayed allocation, which reserves nothing until the flush and therefore decides knowing how much is needed; multiblock allocation, which asks for the whole range at once from a bitmap that is good at finding consecutive zeros; and block groups with preallocation, which reserve a margin behind a growing file. The result is 0.4 % non-contiguous files, and it explains why defragmenting hardly makes sense: it is unnecessary on a healthy ext4, irrelevant on an SSD and wearing besides. Fragmentation is a symptom of a full volume: keep occupancy below 85 %.

The second half has been about survival. Adding a block is three writes — data, bitmap, inode — that the hardware cannot make atomic, and of their seven combinations two are severe: when the inode points to a block marked as free, the allocator will hand it to another file and a cross-link appears, the worst possible corruption. fsck knows how to repair, but 8 hours on a 20 TiB volume is unacceptable, and it only restores consistency, not correctness. The journal solves both: transaction, write to the journal, atomic commit block with a checksum — the point of no return — and deferred checkpointing; after a power cut, recovery replays what was committed and discards the rest in under a second.

Of the three modes, writeback lets foreign content appear inside your files — a security problem, not just an integrity one — data=journal writes everything twice for a 30-50 % cost, and data=ordered guarantees that the data reaches the disk before the commit that references it, without duplicating anything. Meteora chooses ordered, and its format of 24-byte records appended at the end completes the strategy: a power cut leaves at most one incomplete record that the reader discards with size % 24. The choice of data format is part of the integrity strategy. The alternatives to the journal are Btrfs's and ZFS's copy-on-write — no overwrites, with native snapshots and data checksums, at the cost of fragmenting under random rewrites — and log-structured systems such as F2FS, which turn every write into a sequential one in exchange for a garbage collector.

And the final distinction, the most important in the lesson: the journal guarantees the structure's consistency, not the data's integrity. Silent corruption exists, and ext4 does not see it. RAID 1 can detect that the two copies differ but does not know which one is good — and it does not even compare unless you scrub — while ZFS and Btrfs do, because they store a checksum per data block and can heal themselves. Meteora closes that gap with metadata_csum, a weekly scrub and, above all, a CRC32 per record in its own format: when storage cannot give you the guarantee you need, the application can. All of it rests on the write barriers that impose the order the journal needs, and which a disk that ignores its cache flush turns into fiction: that is why server SSDs carry power loss protection. And Meteora's seven layers — journal, RAID, scrub, CRC, snapshot, off-site copy and test restore — each cover what the previous one cannot: RAID is not a backup, nor is a snapshot, and a backup never restored is nothing at all.

We have one last question left in the module, and it is of a different nature. We have protected /var/lib/meteora/readings/2026-08-31.dat from a power cut, from a disk failure and even from a bit flipping on the platter. But we have spent five lessons seeing -rw-r----- in every ls -l and Uid: (990/meteora) in every stat, and we have not explained what they mean. Who can read that file, and who can delete it? Why can you delete a file you cannot read? What exactly does the kernel check, and at what moment? And how does passwd manage to modify /etc/shadow, which only root can write, when a normal user runs it?

That is what we will see in File Security and Permissions.