We now have the complete map: we know what a file is, how it gets a name and how the tree we reach it through is assembled. What we have not done yet is use it from a program. And that is where the behaviors that cause the most headaches in production appear, because almost all of them come from one single thing that almost nobody knows: behind a file descriptor there are three tables, not one.

Out of those three tables come the answers to questions you have probably asked yourself. Why a child and its parent step on each other's offset after a fork but two open calls on the same file do not. Why 2>&1 has to go after > file and not before. Why a write() that returned successfully can be lost in a power cut. And why the aggregator publishes its hourly averages by writing to a temporary file and renaming it, instead of writing directly to the final one.

This lesson is the most practical in the module: nearly everything here you will write yourself, in C or in Python, sooner or later. We are going to walk 2026-08-31.dat struct by struct, dissect the three tables, understand the page cache along the write path, build the atomic publication pattern, and prevent two aggregator instances from running at once with a lock file.

1. The system calls: open, read, write, lseek, close
2. Walking 2026-08-31.dat struct by struct
3. The three tables behind a descriptor
4. Visible consequences: fork, two open calls and dup
5. Standard descriptors and shell redirection
6. Access methods: sequential, direct and indexed
7. Buffered C library I/O versus direct calls
8. The page cache and the real path of a write
9. fsync, fdatasync and what is lost in a power cut
10. Atomic writing with a temporary file and rename()
11. O_APPEND and concurrent writes to the log
12. File locking: flock versus fcntl
13. Safe temporary files and TOCTOU race conditions
14. Sparse files and truncate
15. Inspection tools

Five calls. With them absolutely everything else is done.

open resolves the path (04-02), checks permissions (04-06) and returns a file descriptor: a small, non-negative integer that is the index into a per-process table. It returns -1 with errno set if it fails.

The flags are the heart of open, and they must be combined with |:

Three clarifications that avoid classic mistakes. O_RDONLY is 0, so flags & O_RDONLY is always false and you have to use flags & O_ACCMODE. O_CREAT without the third argument leaves the mode as stack garbage and the file can end up with random permissions: it is a real security flaw. And O_CREAT | O_EXCL is atomic: either you create it, or it fails with EEXIST; without O_EXCL there is no way to tell "I created it" from "it was already there", and that distinction is the basis of the lock files of section 12.

read and write transfer from/to the file's current offset, and advance it. And here is the most repeated mistake in the UNIX world:

read and write may transfer FEWER bytes than requested, and that is not an error. They return how many they actually moved. Ignoring this produces silent data corruption.

With regular files, a short read only happens on reaching the end; but with pipes, sockets and terminals it is the norm. The correct way to write is always in a loop:

The loop covers partial writes and the EINTR covers the case where a signal (03-03) interrupts the call before anything is transferred. A write of 17 MB in one go to a slow socket will return less than requested many times over.

lseek moves the offset without transferring anything, with three origins: SEEK_SET (from the beginning), SEEK_CUR (relative to the current position) and SEEK_END (from the end). The idiomatic lseek(fd, 0, SEEK_END) returns the file's size, and lseek(fd, 0, SEEK_CUR) the current position. And close releases the descriptor: always check its return value, because on network file systems it can return the error of a deferred write that failed, and it is your last chance to find out.

With that we can already write Meteora's real reader. Let us recall the format: 720,000 structs of 24 bytes, 17,280,000 bytes in total.

The program's decisions, which are the ones to learn. O_CLOEXEC prevents the descriptor from being inherited if the process calls execve: if meteo-api launches a helper script, that script will have no access to the data. Reading in batches of 98,304 bytes instead of 24 is the fundamental performance decision, and it comes from 01-06: each system call costs between 0.5 and 1 µs, so with 24-byte reads it would be 720,000 calls ≈ 0.5 seconds just in mode switches, whereas with batches of 4,096 readings it is 176 calls ≈ 0.15 milliseconds, an improvement of more than 3,000× for writing the same logic a different way. complete = n / 24 because a read can return a number of bytes that is not a multiple of 24, leaving a split record that production code carries over to the next batch. And checking for EINTR in read, for the same reason as in write.

An honest portability warning: reading a C struct straight off the disk works because the same program wrote it on the same architecture. If the file travelled between machines you would have to worry about the compiler's padding — here struct Reading is 24 bytes with no padding because the fields are ordered from largest to smallest alignment — and about byte order. For a server's internal data it is legitimate; for an interchange format, it is not.

Now the central piece. When open returns 3, the kernel has touched three different structures:

The key to everything, and the sentence to memorize:

The offset is neither in the process nor in the inode: it is in the intermediate open-file table. Whoever shares that entry, shares the offset.

You can actually see it:

pos is the current offset — the aggregator is at reading number 500,000 — and flags is the mode in octal. And in /proc/2841/fd/ each descriptor appears as a symbolic link to the resource: 0 to /dev/null, 1 and 2 to the log, 3 to the data file, 4 to socket:[38291]. It is the same directory we used in 04-02 to truncate a deleted, nameless file.

The three tables explain three behaviors that, without them, look arbitrary.

Case 1: fork(). The child receives a copy of the descriptor table, but the pointers point to the same entries of the global table. Result: parent and child share the offset.

The file contains CHILD\nPARENT\n. The parent has overwritten nothing because the child advanced the shared offset. This is no accident: it is what makes ( echo a; echo b ) > f work in the shell, with two processes writing to the same file in order.

Case 2: two independent open calls. Each open creates a new entry in the global table, with its own offset.

The file contains only BBBBB. The two descriptors point to the same inode but to different entries, so each keeps its own count and they step on each other. This is exactly the problem O_APPEND solves in section 11.

Case 3: dup and dup2. They duplicate a descriptor: two different numbers pointing to the same entry of the global table, and therefore sharing the offset. dup(fd) returns the lowest free number pointing at the same thing; dup2(fd, 1) forces descriptor 1 to point at the same thing as fd, closing 1 first if it was open. The table summarizing the three cases:

By convention, every process starts with three descriptors open: 0 is stdin, 1 is stdout — normal output, buffered — and 2 is stderr — errors, unbuffered. That stderr is unbuffered is deliberate: an error message must appear before the program crashes, not sit in a buffer nobody will flush.

And now, redirection. When you write meteo-api > /var/log/meteora/meteo-api.log 2>&1, the shell does exactly this between the fork and the execve (02-01):

meteo-api knows nothing about the redirection: it writes to descriptor 1 as always, and that 1 points to the log. All of UNIX redirection is this: dup2 before the execve.

With this, the eternal question is finally settled: why the order of > file 2>&1 matters.

2>&1 literally means "make 2 point wherever 1 points at this moment". It creates no permanent link. If you put it before the redirection, 1 is still the terminal and that is where 2 stays. It is a dup2 and therefore a one-off copy of the pointer, not an alias.

Since the two descriptors share the same entry of the global table, they share the offset, and that is why normal output and errors interleave correctly without overwriting each other. If instead of 2>&1 you did cmd > f 2> f, they would be two independent open calls with two offsets: they would step on each other, which is case 2 of the previous section.

Sequential access. You read from beginning to end, and each operation continues where the previous one stopped. It is what the aggregator does when computing the day's averages, and it is the fastest pattern because the kernel detects the sequence and activates readahead, bringing blocks in ahead of time. Direct or random access: you jump to any position with lseek, which is what meteo-api does when a client asks for reading number 500,000 and there is no point reading the previous 499,999. The key is that fixed-size records let you compute the position with a multiplication:

The second version deserves attention. lseek + read are two calls, and between them another thread of the same process can move the shared offset: a textbook race condition (03-01) that produces reads of the wrong record. pread and pwrite take the position as an argument and do not touch the offset, so they are atomic and thread-safe. In a multithreaded server like meteo-api, using pread is not a stylistic preference: it is correctness.

The cost of finding record 500,000 with each method:

Indexed access. When the search key is not the position — "all the readings from station 42" — an auxiliary structure is needed to translate key → position. Meteora keeps in a separate file an index of entries { uint32_t station_id; uint32_t first; uint32_t count; }: the index, which is small, is loaded, the station is looked up, and with first and count a single pread fetches the whole range. It is the idea databases take to the extreme with their B+ trees, and the reason an indexed query is instantaneous while an unindexed one scans the whole table.

In 01-06 we saw that a printf does not reach the terminal immediately. Now we can close the topic completely.

The C library offers a layer above the system calls, with FILE*: fopen, fread, fwrite, fprintf, fgets, fclose. Its value is a buffer in user space that groups many small operations into few system calls.

The library decides on its own among three buffering modes: full (flushes when the buffer fills) for files and pipes, line (flushes at \n) for interactive terminals, and unbuffered for stderr. Out of that comes a behavior that confuses everybody:

The program has not changed. What has changed is that stdout is no longer a terminal but a pipe, so the library switches from line buffering to full. If the program hangs or you kill it, you lose everything left in the buffer. The fixes are setvbuf(stdout, NULL, _IOLBF, 0) in the code, or stdbuf -oL ./program from outside.

The critical distinction, which many people confuse:

fflush does not guarantee persistence. It only passes the data from the user buffer to the kernel. For it to survive a power cut you need fsync, and to reach the fsync from a FILE* you have to do both steps:

Forgetting the fflush before the fsync is a frequent and subtle mistake: you sync to disk what the kernel had, but the most recent data was still in your own process's buffer.

When the ingestor runs write(fd, &reading, 24), what really happens? Almost never what one imagines:

The decisive point is the third box: write() returns as soon as the data is in the page cache, which is RAM. At that moment the ingestor believes it has written, and from the point of view of any other process reading the file that is true, because reads also go through the cache. But on the disk there is still nothing: the page is marked dirty and will be written later. That is write-back (delayed write), and it is governed by some kernel parameters:

dirty_background_ratio = 10: once dirty pages exceed 10 % of RAM, the kernel's writeback threads start flushing in the background, without holding anyone up. dirty_ratio = 20: once they exceed 20 %, the writing process blocks and is forced to flush itself — this is the cause of the "stalls" of a program that writes a lot: it is not computing, it is paying off the accumulated debt. And dirty_expire_centisecs = 3000: a dirty page is flushed after 30 seconds at most, pressure or no pressure.

Why defer instead of writing right away? For three weighty reasons. Grouping writes: the ingestor writes 24 bytes 8 times a second, and if each one went to disk that would be 8 tiny operations per second on 4 KiB blocks, whereas deferred they group into one write per full block, one every 21 seconds — a 170× reduction. Absorbing rewrites: if a program writes the same block ten times in a second, only one is sent. And ordering and merging, because the I/O scheduler of 02-05 can combine adjacent requests. The price is exactly what you need to understand:

If meteo-01 loses power right now, everything in the page cache that has not reached the disk is lost: up to 30 seconds of readings, about 240 records. And the ingestor has no way of knowing, because its write() calls returned success.

To close that gap there are three calls:

The difference between fsync and fdatasync is subtle but real: if you have appended data, the file's size has changed, and that size is essential metadata — without it the new data cannot be read back — so fdatasync flushes it too; what it saves is writing the inode for irrelevant changes such as the mtime, which under heavy writing can amount to 20-30 %.

The practical decision is an explicit trade-off between durability and performance: without fsync you risk up to 30 seconds and get hundreds of thousands of writes per second; with fsync on every write you risk nothing and are stuck at about 1,000, the disk's physical limit; and with fsync every N seconds you choose the middle ground yourself.

What Meteora chooses. The ingestor receives 8 readings per second, and losing 30 seconds of weather data would be annoying but not catastrophic — the stations retry; on the other hand, an fsync per reading would limit the system to about 1,000 writes per second and would wear out the SSD needlessly. The chosen policy is fdatasync() every 5 seconds, which bounds the loss to about 40 readings and costs 0.2 syncs per second:

An important warning: fsync on a file does not guarantee that its directory entry is on disk. If you have just created it, after a power cut the content could exist but not the name. You also have to fsync the directory — opening it with O_RDONLY|O_DIRECTORY — and that is the step almost everybody forgets.

A real problem. The aggregator computes the hourly averages and publishes them in /var/lib/meteora/hourly-averages.json, which meteo-api reads to answer clients. If the aggregator writes directly:

there is a window of inconsistency: between the O_TRUNC (which leaves the file at 0 bytes) and the end of the write, any reader gets an empty or half-truncated file. With meteo-api serving queries continuously, that means broken responses several times a day. And if the process dies mid-write, the file is left permanently corrupt.

The solution is the most important pattern in this whole lesson, and it rests on a file system guarantee:

rename() is atomic. For any observer, the target points either to the old file or to the new one, never to an intermediate state, and it never disappears.

It is atomic because, as we saw in 04-02, renaming only changes the (name, inode) pair of a directory entry: an operation the file system performs as a single unit.

The four steps, and why none of them is redundant. (1) Writing to a temporary file with a unique name: nobody reads it, so it can stay incomplete for as long as needed. (2) fsync of the temporary file before the rename: without it, the rename — a metadata operation — could reach the disk before the data, and a power cut would leave the good name pointing at an empty file; that is the real failure many applications suffered with ext4 in 2009. (3) The atomic rename, which also removes the old file in the same operation: its link count drops to 0 and the blocks are freed when nobody has it open (04-02). (4) fsync of the directory, so that the rename itself is persistent.

And a lovely property that comes for free: a reader that opened the old file goes on reading it without a problem after the rename, because its descriptor points to the inode and not to the name. It gets no mixed data and no error: it gets, in full, the version that existed when it opened, with perfect consistency and no locking at all. This pattern is universal — Git, package managers, sqlite and text editors all use it when saving: if you have to update a file somebody else may be reading, this is the pattern.

Three Meteora processes write to /var/log/meteora/meteo-api.log. In section 4 we saw that two independent open calls step on each other because each carries its own offset. With a log, the scenario would be this:

Worse still: even if each process did lseek(fd, 0, SEEK_END) before writing, those would be two system calls and between them a scheduler preemption can fit. It is the classic race condition of 03-01, with the window between the lseek and the write.

O_APPEND solves it at the root, and its guarantee is strong:

With O_APPEND, every write() positions at the end of the file and writes in a single atomic operation, protected by an inode lock inside the kernel. Two processes cannot interleave.

Four clarifications about the scope of the guarantee. It is atomic within a single write call: if you write a line with two separate write calls, another process can slip in between them, so compose the whole line in a buffer and write it in one go. lseek is useless with O_APPEND, because the kernel ignores the offset and goes to the end anyway — that is why truncate -s 0 on a log opened with O_APPEND frees the space correctly (04-02), while without it you get a sparse file. POSIX guarantees atomicity up to PIPE_BUF, although on Linux and on a local file system it works with lines of any reasonable size. And on NFS it is not guaranteed, because the client cannot atomically do "seek to the end and write" across the network: it is problem 2 of 04-03.

That is why every log must be opened with O_APPEND, and why >> in the shell uses it while > does not.

O_APPEND solves concurrent writes to a log, but not the general problem: preventing two instances of the aggregator from running at the same time and computing the same averages twice.

Linux offers two locking mechanisms, mutually incompatible:

The fcntl trap deserves an explicit warning, because it produces failures that are very hard to diagnose: closing any descriptor of the same file releases all the locks the process held on it, so if a library opens and closes the file to read one line, your lock vanishes without anyone telling you. Both are advisory: they only work if all participants cooperate by requesting the lock, and a process that ignores it will write unimpeded. The theoretical alternative is mandatory locking, which the kernel would impose on everybody by checking it on every read and write. Linux had it — mounting with -o mand and marking the file setgid without group execute permission — but it was fragile, had race conditions of its own and was removed from the kernel. In practice, all file locking on Linux is advisory, and that is fine: it is an agreement between cooperating programs, not a security mechanism.

The aggregator's lock file, which is the most common use:

The key decisions: LOCK_NB makes flock not block — if another instance holds it, it returns EWOULDBLOCK and the program exits cleanly; without it, instances would pile up waiting and then all run back to back, exactly what you do not want in an hourly task; the lock is released automatically when the process dies for any reason, including kill -9 or a power cut, which is the decisive advantage over a .pid file — that one requires checking whether the process is still alive, and the PID may have been reused; and the file goes in /run, which is tmpfs (04-02) and cleans itself at boot.

From the shell, the flock utility does the same thing in one line, and it is what your scheduled tasks should use:

And fcntl remains indispensable when you need to lock byte ranges: a database that wants to lock record 500,000 without blocking access to the others. It is the granularity of 03-04 applied to files.

Creating a temporary file looks trivial and is a classic source of vulnerabilities:

The problem has a name: TOCTOU, Time Of Check To Time Of Use. Between the moment tmpnam checks that the name is free and the moment your open creates it, there is a window. An attacker watching /tmp — which is world-writable — can, in that window, create a symbolic link with that name pointing at /etc/passwd. Your open with O_CREAT will follow the link and write into /etc/passwd with your privileges. If your program is setuid root (04-06), you have just given away the machine.

The solution is mkstemp, which creates and opens in a single atomic operation:

Why mkstemp is safe: it uses O_CREAT | O_EXCL, so if the name already exists — including a symbolic link — it fails instead of following it, which closes the TOCTOU; it creates with mode 0600; and it returns the descriptor already open, with no window between creation and use. The template must be a writable array, not a literal, because mkstemp writes the generated name into it.

An even stronger pattern when nobody else needs to see the file: create it and delete it immediately, following what we learned in 04-02. unlink(template) right after the mkstemp makes the name disappear while the inode stays alive through the descriptor: nobody can open it or replace it, and the system frees the space by itself even if the process dies from a kill -9. Linux also offers O_TMPFILE, which creates an anonymous file with no name at any point, eliminating even the window between mkstemp and unlink.

The general rule against TOCTOU is worth far more than just for temporary files: do not check a path and then act on it; operate directly on the descriptor. That is why access() followed by open() is a known antipattern — it checks with the real permissions and opens with the effective ones, with a window in between — and that is why openat, fstatat, fchmod and fchown exist, working on already-open descriptors.

truncate and ftruncate change a file's size:

Shrinking discards the excess and frees the blocks. Growing does something more interesting: the new space reads as zeros, but no blocks are reserved for it. That is a sparse file:

A 1 GB file that occupies zero blocks. The file system simply has no pointers for that region: when somebody reads there, the kernel returns zeros without touching the disk, and the blocks are allocated only when written to. Hence the perpetual difference between what ls -l and stat -c %s show — the logical size — and what du and stat -c %b show — the real usage.

Sparse files are genuinely useful for virtual machine images (a 100 GB disk with 12 GB used takes 12), for databases that pre-allocate space and for segmented downloads. And they are created accidentally by an lseek beyond the end followed by a write, which is exactly what happens when you truncate to zero a log opened without O_APPEND: the process keeps its old offset, writes there, and creates a sparse hole from byte 0. Two precautions: cp preserves the holes with --sparse=always but many tools fill them with zeros, turning a 12 GB file into a 100 GB one, and tar needs -S to preserve them.

Three tools for seeing what is really going on with a process's files.

lsof — what each process has open:

You can read the service's complete state at a glance: its cwd, its binary (txt), the log open for writing (1w), the data file for reading (3r), the listening socket and the shared cache. The most useful options are lsof -p PID, lsof path, lsof +L1 for deleted files that are still open (04-02) and lsof -i :8080 by port.

/proc/<pid>/fd and /proc/<pid>/fdinfo — the same thing without installing anything: ls -l /proc/2841/fd/ shows what each descriptor points to, cat /proc/2841/fdinfo/3 gives the pos and the flags, ls /proc/2841/fd/ | wc -l counts how many are in use and grep files /proc/2841/limits its ceiling. Counting descriptors is the diagnosis for a descriptor leak: a service that opens files and does not close them ends up exhausting its limit (1,024 by default, often raised to 65,536) and fails with EMFILE, "Too many open files". If the number grows monotonically over time, you have a leak.

filefrag — how a file is laid out on the disk:

A single extent: the file's 4,219 blocks are physically contiguous, from block 8,394,271 to 8,398,489. That means reading the whole file is one single sequential operation, the best possible outcome. How ext4 achieves this result while writing 24 bytes every 125 milliseconds for 24 hours is exactly the subject of the next lesson.

Not checking the return value of read and write. They may transfer fewer bytes than requested without that being an error. Write the write_all function of section 1 once and use it always.

Confusing fflush with fsync. fflush moves data from your process's buffer to the kernel; fsync takes it to the disk. Only the second survives a power cut, and from a FILE* you need both, in that order. For the same reason, a write() that returned is not on disk: it is in the page cache, and up to 30 seconds can pass.

Opening a log without O_APPEND. Two processes will overwrite each other, and truncate -s 0 will leave a sparse file instead of freeing space.

Writing directly over a file others are reading. Always use temporary file + fsync + rename + fsync of the directory; it is four extra lines and it eliminates an entire class of failures. And do not forget the fsync of the temporary file before the rename: without it, a power cut can leave the new name pointing at an empty file, which is trading a visible failure for a silent one.

Using tmpnam, tempnam or mktemp in C. They all have the TOCTOU race. Use mkstemp or O_TMPFILE, which create and open atomically with mode 0600. For the same reason, do not check with access() and act afterwards: open directly and check the error, or work with openat and fstat on descriptors.

Expecting flock to protect you from a program that does not cooperate. It is advisory: it only works among programs that request it, and on Linux mandatory locking no longer exists. And do not confuse flock with fcntl: they are different mechanisms that cannot see each other, and with fcntl closing any descriptor of the file releases all your locks.

Tip: use pread/pwrite in multithreaded code, which take the position as an argument and eliminate the race between lseek and read; and open with O_CLOEXEC by default, so descriptors do not leak into child processes — an information leak and a common cause of "target is busy".

Write three short C programs that empirically demonstrate the three cases of section 4: (a) parent and child after fork share the offset; (b) two independent open calls on the same file do not share it and overwrite each other; (c) dup2 does share it. In each one, show the file's final content and the pos from /proc/<pid>/fdinfo/<fd> at the right moment. Explain which concrete table produces each result.

Implement in C (or in Python) the function that publishes the hourly averages in /var/lib/meteora/hourly-averages.json atomically, with all four steps. Then write a reader program that opens the file in a loop and verifies that it never reads an incomplete JSON, and run them at the same time for a minute. Finally, modify the publisher so it writes directly with O_TRUNC and measure how many corrupt reads the reader gets. Explain the result.

Write a program that appends a million 24-byte Reading structs to a file, with four policies: (a) no fsync; (b) fdatasync every 1,000 records; (c) fdatasync on every record; (d) opening with O_SYNC. Measure the time of each one and compute records per second. Then, for each policy, calculate how many records would be lost in a power cut at Meteora's real rate (8 readings/second) and argue which one you would choose and why.

Solution 1

(a) The parent sees pos = 6 without having written anything: fork copies the descriptor table, but its entries point to the same entry of the global table, where the offset lives. The child moves it and the parent sees it.

(b) Each open created its own entry in the global table, both with offset 0 and pointing at the same inode; both wrote at 0..4 and the second clobbered the first.

(c) dup creates a new descriptor pointing at the same global entry, so the offset is the same and the writes chain up.

In short: the descriptor table decides which numbers each process sees; the global open-file table decides who shares an offset; the inode table decides which file it is. The three cases differ only in the second one.

Solution 2

Two indispensable details: the temporary file must be created in the same directory as the target, because rename() does not cross file systems (EXDEV, 04-02) and if you put it in /tmp it will fail or degenerate into copy+delete, losing atomicity; and os.replace instead of os.rename, because the former guarantees POSIX atomic-overwrite semantics on Windows too.

The verifying reader opens the file in a loop for a minute and counts how many times json.load() raises JSONDecodeError or FileNotFoundError. Expected result with atomic publication: corrupt=0, always, however many rounds it makes. With direct O_TRUNC, on a file of about 200 KB, the window of inconsistency lasts 1-3 ms on each publication, and with a reader in a tight loop you get tens or hundreds of corrupt reads per minute.

The explanation. With O_TRUNC the file passes through visible intermediate states: 0 bytes, then partially written, and any reader arriving in that window gets invalid JSON. With a temporary file + rename, the name always points at a complete inode and the change of which one it points to is atomic; a reader that already had it open goes on reading the old version in full, because its descriptor is tied to the inode and not to the name.

Solution 3

Typical values on an NVMe (they vary with the hardware, but the orders of magnitude hold):

The conclusion is emphatic: syncing on every record is 800 times slower. The disk cannot confirm more than 1,000-3,000 syncs per second, because each one means flushing the device's volatile cache and waiting for confirmation from the medium; that number is a physical characteristic of the hardware and does not improve by writing better code.

Data at risk at Meteora's real rate (8 readings/second):

What I would choose and why. Neither (a) nor (c). Option (c) adds nothing: losing a few seconds of weather data is tolerable — the stations retry — and in exchange it would limit the system to 1,400 writes per second and multiply the SSD's wear. Option (a) is convenient but leaves a 30-second window that you moreover do not control, because it depends on the memory pressure of the whole system.

The right policy is (b'), fdatasync every 5 seconds: it bounds the loss to 40 records with only 0.2 syncs per second. Note that it is (b') and not (b): syncing by time gives a guarantee you can explain to a customer — "at most 5 seconds are lost" — and one that is independent of the arrival rate, whereas "every 1,000 records" means 125 seconds with normal traffic and hours if the stations send little. Always bound by time, not by quantity. And fdatasync rather than fsync because, when appending, the only essential metadata is the size.

Five system calls — open, read, write, lseek, close — are enough for everything, and open's flags are what make them powerful: O_CREAT|O_EXCL for atomic creation, O_APPEND for logs, O_CLOEXEC so descriptors do not leak. Two rules to avoid mistakes: read and write may move fewer bytes than requested — hence the write_all function with its loop and its EINTR — and reading in batches rather than record by record turns 720,000 system calls into 176, an improvement of more than 3,000× for rearranging the same logic.

Behind a descriptor there are three tables: the descriptor table, per process; the open-file table, global, where the offset lives; and the in-memory inode table. All the phenomenology comes from the second: fork and dup share the entry, and therefore the offset; two open calls do not, and therefore they overwrite each other. That explains why ( echo a; echo b ) > f works, and it finally explains 2>&1: it is a dup2 that copies where 1 points at that instant, so putting it before > file leaves stderr on the terminal.

The access methods are sequential (with readahead), direct — with fixed-size records the position is i × 24 and finding record 500,000 costs one access instead of 2,930 — and indexed when the key is not the position; in multithreaded code, pread/pwrite instead of lseek+read. On the write side there are two layers of buffer, and confusing them costs data: the C library's, which fflush empties into the kernel, and the page cache, which only fsync/fdatasync empty onto the disk. A write() that has returned is in RAM, and write-back will take it to the medium within up to 30 seconds. Deferring is correct — it groups, absorbs rewrites and allows reordering — but durability has to be requested explicitly, bounded by time: Meteora does fdatasync every 5 seconds, risking 40 records in exchange for not dropping from millions of writes per second to 1,400.

The pattern you will use most is atomic publication: write to a temporary file, fsync the temporary file, rename() — which is atomic — and fsync the directory. None of the four steps is redundant, and as a bonus a reader that already had the file open goes on seeing the old version in full, because its descriptor points to the inode and not to the name. For logs, O_APPEND turns "go to the end and write" into a single atomic kernel operation, as long as you compose each line in a single write. And to prevent two aggregator instances at once, flock with LOCK_NB on a file in /run, which is released on its own when the process dies and therefore leaves no orphan locks the way a PID file does; remembering that on Linux all locking is advisory.

Safe temporary files are created with mkstemp, which uses O_CREAT|O_EXCL and closes the TOCTOU window that makes tmpnam a vulnerability; and the general rule is do not check a path in order to act on it afterwards, but operate on already-open descriptors. Sparse files explain why ls and du disagree, and filefrag has left us the fact the next lesson opens with: the 4,219 blocks of 2026-08-31.dat are in one single contiguous extent.

And there lies the pending question. That file was written 24 bytes every 125 milliseconds for 24 hours, among thousands of writes from other processes, and yet it ended up perfectly contiguous on the disk. How does the file system decide which blocks to allocate, and how does it achieve that contiguity? How are 4,219 blocks represented inside an inode that has only 60 bytes for pointers? And what exactly happens if the power fails right between writing the data, marking the block as used and updating the inode, leaving those three things out of agreement?

That is what we will see in Space Allocation, Journaling and Integrity.