At the end of the previous lesson a concrete problem was left on the table. ingestor, aggregator and meteo-api are separate processes, and that architectural decision had good reasons: isolation against failures, separation of privileges over /var/lib/meteora/readings/, and the possibility of moving a piece to another machine. But it has an immediate consequence: their address spaces are independent. The MMU you studied in module 2 guarantees that a pointer in the ingestor means nothing in the aggregator. They cannot pass an array of Reading around by sharing a pointer, the way the four threads did.

They need an explicit mechanism, provided by the kernel, to talk to each other. That set of mechanisms is inter-process communication, or IPC.

This lesson goes through the six mechanisms that are really used on Linux — pipes, FIFOs, message queues, shared memory, sockets and signals — with examples that work and with the details superficial documentation leaves out: what happens when a buffer fills up, what happens if the reader closes, why only a few functions are safe inside a signal handler. And it ends with a decision guide, because choosing the wrong IPC mechanism conditions the performance and reliability of an entire system. A warning from the start, because it is the number one source of mistakes: shared memory carries data but coordinates nobody; synchronizing it is the subject of the next lesson, and here we will flag it explicitly wherever it applies.

1. The two fundamental models
2. Anonymous pipes: what they are on the inside
3. When the buffer fills up and when the reader closes
4. Named pipes: FIFOs
5. POSIX message queues
6. POSIX shared memory and /dev/shm/meteora-cache
7. Sockets: UNIX domain and network
8. Signals as a notification mechanism
9. Blocking and non-blocking communication
10. Comparison and decision guide

Underneath the variety of mechanisms there are only two conceptual models, and all the engineering of IPC is a consequence of choosing between them.

Message passing. The processes exchange discrete units of data through the kernel. The sender calls a send function, the kernel copies the data into a buffer of its own, and the receiver calls a receive function that copies it into its space. The processes never share memory.

Shared memory. The kernel makes a region of physical memory appear mapped into the address space of several processes. From that moment on, writing to that region is writing with an ordinary mov instruction, and the other process sees it immediately. The kernel intervenes once, when the mapping is set up, and then disappears.

At the top, message passing: two copies and the kernel in between. At the bottom, shared memory: both processes write to the same physical page. The differences are systematic:

The first two numbers explain why shared memory exists: to move 17 MB from the ingestor to the aggregator, message passing would copy 34 MB (17 up and 17 down) and take about 180 ms; shared memory costs one mmap and zero copies. And the synchronization row explains why it is not always used: with message passing, if the receiver reads, either there is a complete message or there is none, because the kernel guarantees the atomicity of delivery; with shared memory there is no guarantee of anything, and the aggregator can read a struct Reading while the ingestor is writing it and get 8 new bytes and 16 old ones. Shared memory is the fastest mechanism and the only one that solves no coordination problem on its own.

A pipe is a unidirectional channel of bytes between two related processes. It is the oldest IPC mechanism in UNIX and it is still the most used, although hardly anyone calls it by name: every time you type cat file | grep pattern in the shell, you are creating one.

On the inside, a pipe is exactly this: a circular buffer in kernel memory, with two file descriptors pointing at it, one for reading and one for writing. On Linux that buffer is 65,536 bytes (16 pages) by default, adjustable with fcntl(fd, F_SETPIPE_SZ, size) up to the limit in /proc/sys/fs/pipe-max-size.

Running it: [aggregator] 1000 readings, average 23.45 C. Four details you have to understand, because each one is a classic bug if it is forgotten:

The order is pipe() and then fork(), never the other way round. The pipe is only shared because the child inherits the parent's descriptor table in the fork() (module 2). If you fork() first, each process will create its own pipe with no relation to the other's. Hence anonymous pipes only work between related processes: parent-child, or siblings that inherited from the same parent.

Closing the end you do not use is mandatory, not tidying up. If the child does not close fd[1], the pipe still has a writer open: when the parent closes its own, the child will not get EOF and its read() will block forever. It is one of the most frequent causes of hangs with pipes, with a deceptive symptom: the program works but never finishes.

read() returns 0 when the last writer closes. That zero is the EOF. As long as at least one write descriptor is open in any process, a read() on an empty pipe blocks instead of returning 0.

A pipe is a stream of bytes, not of messages. The kernel does not preserve the boundaries of your write()s: one of 24 bytes may be read as two read()s of 12, and three of 24 may be read in one read() of 72. In the example it works because struct Reading is 24 bytes and the pipe guarantees atomicity up to PIPE_BUF (4,096 bytes on Linux), but with partial reads you have to keep going in a loop, and if your messages are variable-sized you have to invent a delimiter or a length header. It is the price of working with a stream.

The shell equivalent is exactly the same mechanism. When you type cat /var/lib/meteora/readings/2026-08-31.dat | ./aggregator --hourly-average, the shell calls pipe(), does two fork()s, and in each child uses dup2(fd[1], STDOUT_FILENO) and dup2(fd[0], STDIN_FILENO) respectively before the execve(). That is why cat and aggregator do not need to know anything about pipes: they write to and read from their standard descriptors. That indirection is one of the most elegant ideas in UNIX.

A pipe's two edge cases are what separate the person who has used one from the person who has understood one.

The buffer fills up. If the ingestor writes faster than the aggregator reads, the 65,536 bytes run out. Then write() blocks until there is room. The process moves to state S (or D for some variants) and the scheduler takes it out of the ready queue.

This is not a failure, it is an extraordinarily useful feature: it is called backpressure. The producer automatically slows to the consumer's pace, with nobody programming it. It is what makes cat 50GB_file | grep pattern not consume 50 GB of RAM: cat blocks as soon as grep falls behind. You can watch it live with ps -o pid,stat,wchan -C producer, which shows state S and WCHAN: pipe_write — the exact kernel function where it sleeps, waiting for room. Looking at WCHAN to find out what a process is blocked on works for any kind of blocking and you will use it a lot in Performance Monitoring and Troubleshooting.

The reader closes its end. Here the behavior is more aggressive. If the aggregator dies or closes fd[0] and the ingestor tries to write, the kernel sends SIGPIPE to the writer; the default action of that signal is to terminate the process with no error message; and only if the process ignores or catches it does write() return -1 with errno == EPIPE.

That the default action is to kill the process seems brutal, but it makes sense in the shell: in cat huge_file | head -3, once head has printed three lines and finished, there is no point in cat carrying on reading gigabytes nobody is going to read. In a long-lived service, by contrast, that silent death is unacceptable, and the professional pattern is always this one:

Ignoring the signal turns a sudden death into an error code you can handle. This same pattern applies to sockets, where writing to a connection the other end has closed also produces SIGPIPE. Every serious network server does signal(SIGPIPE, SIG_IGN) in the first lines of main(); forgetting it means your service dies silently the first time a client hangs up at a bad moment.

The limitation of anonymous pipes — only between related processes — is solved by giving them a name in the file system. That is a FIFO (First In, First Out), or named pipe.

A FIFO is an entry in the file system with type p, which any process with permission can open. It stores no data on disk: the file is only a meeting point; the buffer is still in kernel memory, just as in an anonymous pipe.

Let us set up the channel between Meteora's ingestor and aggregator:

The reader-side code is an ordinary pipe, with the difference of how the descriptor is obtained:

Four peculiarities of FIFOs you need to know:

open() blocks until the other end shows up. Opening read-only blocks until somebody opens for writing, and vice versa: it is a built-in rendezvous mechanism, very convenient. If you do not want it, O_NONBLOCK; watch out for the asymmetry, opening for writing in non-blocking mode with no reader fails with ENXIO, whereas opening for reading in non-blocking mode with no writer succeeds.

Several writers are possible and their writes interleave safely, provided each write() is at most PIPE_BUF (4,096 bytes): below that size the kernel guarantees the write is atomic and does not get mixed with another writer's; above it, it can be fragmented and you will receive garbage. Since a struct Reading is 24 bytes, the 800 stations could write to the same FIFO without corrupting each other. It is a very valuable and little-known guarantee.

There are no useful multiple readers. If two processes read from the same FIFO, each byte goes to one of the two, arbitrarily. It is not broadcast: it is sharing out.

The FIFO is persistent but the data is not. The file outlives the processes; the buffer contents are lost as soon as all the ends are closed. And if nobody reads, the writer blocks; if nobody writes, the reader blocks. A FIFO keeps nothing when there is nobody on the other side.

That last point is exactly the limitation that motivates the next mechanism.

A message queue is a mailbox managed by the kernel where processes deposit and collect discrete, prioritized messages. It solves three limitations of pipes: it preserves the boundaries between messages, it allows priorities, and it persists even when nobody is connected.

Linux offers two APIs: the System V one (msgget/msgsnd, old, with awkward numeric identifiers) and the POSIX one (mq_open/mq_send, with path-like names and descriptors that work with select/poll). Always use the POSIX one.

Look at the result: even though the calibration message was sent second, it is received last. mq_receive() always returns the highest-priority message, and among those of equal priority it respects the order of arrival. That is the functional advantage no pipe gives you: if the aggregator is saturated, critical alerts are handled before the background noise.

POSIX queues live in a virtual file system you can mount and examine, and their limits come from the kernel:

QSIZE is the number of bytes pending right now: an excellent diagnostic window, because if it grows without stopping it means the consumer cannot keep up. And ten messages per queue is very few: exceeding it makes mq_send() block (or fail with EAGAIN in non-blocking mode). It can be raised with sysctl fs.mqueue.msg_max=100, but the practical conclusion is a different one: POSIX queues are for notifications and control commands, not for data throughput. To move 17 MB of readings a day, the right mechanism is another.

A useful extra: mq_notify() lets you ask the kernel to notify you — with a signal or by creating a thread — when a message arrives at an empty queue, without having to sit blocked waiting.

We reach the fastest mechanism, and the one you have known by name since module 2. Shared memory consists of several processes mapping the same physical pages into their address spaces.

The POSIX procedure has three steps: create a shared memory object with shm_open(), give it a size with ftruncate(), and map it with mmap(). From there on, it is ordinary memory.

Key points of the code:

/dev/shm is a tmpfs, a file system that lives entirely in RAM (and can go to swap). That is why shm_open("/meteora-cache") creates a file visible at /dev/shm/meteora-cache that you can inspect with ls, delete with rm and size with df -h /dev/shm. There is no magic: it is a file in RAM mapped with mmap.

MAP_SHARED is what makes it shared. With MAP_PRIVATE, each process would get its own copy as soon as it wrote, through copy-on-write, and the modifications would be visible to nobody. It is a one-word mistake with a baffling symptom: everything works but the other process never sees the changes.

close(fd) does not destroy the mapping, which stays until the munmap() or until the process ends; closing the descriptor is good practice so as not to waste them. And the object persists until shm_unlink() or until the machine reboots: that is kernel persistence, an advantage (the aggregator can restart without losing the cache) and a potential leak (if nobody calls shm_unlink, the RAM stays occupied indefinitely).

And now the warning that has to be underlined three times, the one in the ⚠ RACE comment. Shared memory synchronizes absolutely nothing. That c->n_latest++ is exactly the same counter++ from Concurrency Concepts, with the same three instructions and the same loss of increments. And it is worse than that: writing a 24-byte struct Reading is at least three instructions, and the aggregator can read it half-written. Where a message queue guarantees that a message arrives whole or does not arrive at all, here there is no guarantee whatsoever, and where a queue never loses an element, here increments really are lost: shared memory always needs additional primitives.

What is missing — mutexes, semaphores, condition variables, placed inside the shared region itself so that every process sees them — is the entire content of Synchronization and Mutual Exclusion. Until then, consider all the code in this section deliberately incomplete.

A socket is one endpoint of a bidirectional communication. It is the most versatile IPC mechanism because, with the same API, it connects processes on the same machine or on different machines.

There are two families that matter here:

The UNIX domain socket is between 5 and 10 times faster than a local network one because it skips the whole TCP/IP stack — no checksums, headers, congestion control or fragmentation: the kernel copies from the sender's buffer to the receiver's. And it brings two unique capabilities: passing open file descriptors between processes (this is how nginx hands out accepted connections among its workers) and knowing the credentials of the process at the other end without it declaring them, which allows reliable local authentication.

Now the Meteora case: the ingestor listening for the readings of the 800 stations. Here a network socket really is needed, because the stations are outside the machine.

Design decisions worth justifying:

UDP (SOCK_DGRAM) and not TCP. Each reading is an independent 24-byte datagram, and losing one is not dramatic: another will arrive in a minute. TCP would add the cost of keeping 800 connections open with their buffers and retransmissions in order to guarantee a delivery we do not need. Besides, with SOCK_DGRAM message boundaries are preserved: one recvfrom() returns exactly one datagram, which eliminates the chunking problem of pipes and of TCP.

Enlarging SO_RCVBUF. This line connects directly with module 2: if the 800 stations send at once and the ingestor is busy writing to disk, the datagrams pile up in the socket buffer, and when it fills the kernel drops them silently — you will only see it as rx_missed_errors or in the drops reported by netstat -su. The default 208 KB is enough for 8,600 readings; 4 MB, for 175,000. It is the same sizing logic we saw with NAPI.

htons and htonl convert to network byte order (big-endian) from the processor's (little-endian on x86); forgetting it turns port 9200 into port 61. And careful: the example sends the struct Reading raw, which only works if sender and receiver share architecture and alignment; in production the fields would have to be serialized explicitly.

A signal is an asynchronous notification the kernel delivers to a process. It carries no data (except with sigqueue, which allows one integer): it is a warning that something has happened. It is, in essence, a software interrupt aimed at a process.

You have already met several in the course: SIGSEGV when a process touches memory that is not its own (module 2), SIGKILL when the OOM killer decides to sacrifice someone, SIGPIPE two sections ago.

The Meteora case is the most widespread convention in UNIX: reloading /etc/meteora/meteora.conf with SIGHUP without restarting the service or losing the connections in progress.

The pattern you see — the handler only raises a flag; the work is done in the main loop — is mandatory, and the reason is the most important restriction on signals.

A signal handler runs by interrupting the process at an arbitrary point. It can interrupt in the middle of a malloc(), when the allocator's internal structures are inconsistent. If the handler calls printf(), which internally allocates memory, you get a deadlock or heap corruption. That is why POSIX defines a short list of functions that are safe in handlers (async-signal-safe):

The type volatile sig_atomic_t is the only variable you can touch with guarantees: sig_atomic_t ensures the assignment is a single instruction and volatile forces the compiler to reread it on each iteration of the loop instead of caching it in a register. (Here volatile really is correct because the handler runs in the same thread as the loop: there are no two cores or caches involved. In real concurrency between threads it is not enough, and we will see that in the next lesson.)

Two more notes. SA_RESTART: when a signal arrives while the process is blocked in a slow read(), the call is aborted with EINTR; with this flag the kernel retries it automatically and your code does not have to wrap every blocking call in a loop. And sigaction instead of signal: signal() has historically behaved inconsistently across systems — on some it reinstalls the handler after each signal, on others it resets it to the default action — whereas sigaction() is explicit and portable; use signal() only for the trivial case of SIG_IGN.

For multithreaded processes, the professional pattern we anticipated in the previous lesson: block the signals in every thread with pthread_sigmask() and dedicate one thread to collecting them with sigwait(). That removes every handler restriction at the root, because sigwait() returns in normal context and that thread can call whatever it likes.

All the mechanisms above have two modes of operation, and choosing wrongly produces either hangs or pointless CPU consumption.

In blocking mode (the default), if there is no data to read read() puts the process to sleep until there is, and if the buffer is full write() sleeps until there is room: the scheduler takes it out of the ready queue and it consumes no CPU at all. In non-blocking mode (O_NONBLOCK) the call returns immediately, and if there was no data it returns -1 with errno == EAGAIN (equivalent to EWOULDBLOCK).

The correct way to use non-blocking mode is not active polling — checking in a loop whether there is data burns a whole core — but combining it with a multiplexing call that sleeps until one of the descriptors is ready:

This is the best of both worlds: zero CPU while nothing is happening (the process sleeps in epoll_wait) and a single thread for thousands of descriptors. It is the engine of the previous lesson's async model, and the reason nginx serves 100,000 connections with eight processes.

The role of the buffer here is central: the kernel's buffer is what decouples the sender from the receiver. As long as there is room, the sender does not wait; as long as there is data, the receiver does not wait. If it turns out too small for the bursts in your traffic, the two processes are synchronized by force — or, worse, in UDP data is dropped silently. Sizing buffers (F_SETPIPE_SZ, SO_RCVBUF/SO_SNDBUF, mq_maxmsg) is one of the most effective and most forgotten tuning levers.

And the decision guide as a chain of questions:

1. Can the processes be on different machines, now or in the future? → Network socket. It is the only one that crosses the machine boundary, and using it from the start avoids a rewrite when the system grows.
2. Do you only need to signal an event, with no data? → Signal. Reloading configuration, requesting a clean shutdown, forcing a log rotation. It is the cheapest thing there is and what any system administrator expects.
3. Is it a direct child you created yourself, with the flow going one way only? → Anonymous pipe. The simplest mechanism there is, and there is nothing to clean up.
4. Do you need priorities, or messages preserved even when the receiver is absent? → POSIX message queue, keeping an eye on the size limits.
5. Are you moving megabytes with critical latency? → Shared memory, accepting that you will have to synchronize it with the primitives of the next lesson. It is the fastest option and the one that can cost you the most in debugging.
6. In any other case → UNIX domain socket. Bidirectional, with access control through permissions, it identifies the process on the other side, it works with epoll and it is the one that will surprise you least. When in doubt, this is the answer.

Meteora's architecture, now justified mechanism by mechanism:

Not closing the unused ends of a pipe. If the reading process keeps the write descriptor open, it will never get EOF and its read() will block forever. The symptom is a program that does its job but never finishes, and it is one of the most frequent hangs with pipes.

Not ignoring SIGPIPE in a service. Writing to a pipe or a socket whose reader has closed kills the process by default, with no message. A service that dies silently when a client hangs up is a guaranteed middle-of-the-night incident: signal(SIGPIPE, SIG_IGN) in the first lines of main() and check EPIPE.

Calling printf or malloc from a signal handler. It works 99.9 % of the time and corrupts the heap the remaining 0.1 %, producing a random failure much later that is impossible to connect to its cause. The handler raises a volatile sig_atomic_t flag and nothing more.

Assuming that a read() from a pipe or a TCP socket returns the complete message. They are byte streams with no boundaries: a read(fd, buf, 24) may return 10. You have to keep going in a loop until the expected bytes are complete, or use SOCK_DGRAM/SOCK_SEQPACKET/queues if you need boundaries.

Using MAP_PRIVATE where you wanted MAP_SHARED. The program works, gives no error at all, and one process's writes simply are not seen by anyone else, because copy-on-write gave each of them a private copy. Related: forgetting -lrt when linking mq_* and shm_open on glibc earlier than 2.34.

Believing shared memory "works" because the tests pass. It is the most expensive mistake in this lesson. Without synchronization, the races are the ones from 03-01: low probability per operation, certainty in the long run. If you share memory, you need the next lesson.

Tip: prefer the simplest mechanism that solves your problem. A lot of code uses shared memory where a FIFO would have done, and pays in debugging what it saved in microseconds nobody was going to notice. Optimize IPC once you have measured it and it is the bottleneck.

Tip: size the buffers deliberately and watch how full they get. The default value is designed for the general case, not for yours. A buffer that often fills up warns you about a consumer that cannot keep up before it starts losing data.

Write two C programs: a producer that writes struct Readings into a pipe as fast as possible, counting how many it has written, and a slow consumer that reads one every 100 ms. Connect them with |. While they run, observe the producer's state with ps -o pid,stat,wchan -C producer and explain what you see. Then kill the consumer with Ctrl+C and check what happens to the producer; repeat the experiment with signal(SIGPIPE, SIG_IGN) and handle EPIPE.

For each Meteora need, choose the most suitable IPC mechanism and justify it by ruling out at least two alternatives.

• (a) A new tool meteo-ctl must be able to tell the aggregator to recompute the current day's averages immediately.
• (b) The aggregator publishes a 2 MB summary every hour that the 4 meteo-api workers consult on every request.
• (c) An archiving service is added on another machine that must receive a copy of every closed daily file.
• (d) The ingestor must warn the aggregator that it has detected a station down, more urgently than the routine notifications.

Set up the ingestor → aggregator channel with a FIFO. The writer must send 100,000 struct Readings; the reader must process them with a configurable artificial delay. Measure the total time with delays of 0 µs and of 10 µs per reading, and determine experimentally the size of the FIFO's buffer by checking how many bytes the producer can write before blocking with a reader that reads nothing.

Solution 1

Observation during the run:

What you see. The counter stops around 2,731 and goes no further, with state S (interruptible sleep) and WCHAN = pipe_write: the producer is blocked inside that kernel function, waiting for room. The number is no coincidence: 65,536 bytes of buffer / 24 bytes per reading = 2,730.7. It filled the buffer exactly and went to sleep; from then on it advances at the consumer's pace, 10 readings per second. That is backpressure at work: without a single line of code devoted to it, the producer has adapted to the consumer and the memory consumed is bounded at 64 KB.

On killing the consumer:

In the first case the producer dies without printing anything: SIGPIPE terminated the process and the message comes from the shell, not from the program. If this were the ingestor in production, it would have stopped receiving readings and there would not be a single line in the log to explain it. In the second, write() returns -1 with EPIPE, the program detects it, logs it and exits in an orderly fashion. The difference is literally one line of code, and it separates a diagnosable incident from one that is not.

Solution 2

(a) meteo-ctl orders the aggregator to recompute. → SIGUSR1 signal. It is a one-off notification with no data: kill -USR1 $(pidof aggregator) requires no infrastructure at all. Ruled out: a FIFO would force the aggregator to keep a reader open permanently and meteo-ctl to handle the blocking open(), a lot of machinery for a notification; a UNIX socket would be the right thing if meteo-ctl grew into a protocol with several commands and replies, but for one command with no reply it is over-engineering.

(b) A 2 MB summary read on every request. → POSIX shared memory. At 1,200 requests/s, passing 2 MB through a queue or a socket would be 2.4 GB/s of copying: impossible. With shared memory the access cost is zero. Ruled out: a POSIX queue because of the 8 KB per message limit, three orders of magnitude below; a UNIX socket because of the two copies per request. Essential condition: the aggregator's hourly update has to be synchronized with the workers' continuous reads, or they will see a half-written summary. The right pattern is a double buffer with an atomic index, or a read/write lock: exactly what we will see in Synchronization and Mutual Exclusion and in Classic Concurrency Problems.

(c) Archiving on another machine. → Network TCP socket. It is the only family that crosses the machine boundary, and TCP rather than UDP because a complete daily file must arrive whole and in order: here we do need the delivery guarantees that individual readings did not require. All the other mechanisms are local by construction; shared memory cannot be shared between machines, and that is precisely its boundary.

(d) An urgent warning about a station down. → POSIX message queue with high priority. It is the only local mechanism with built-in priorities: by sending alerts with priority 9 and routine things with 0, mq_receive() delivers the urgent items first even if they arrived later. Ruled out: the FIFO is strictly first in, first out — an urgent warning would wait behind everything already piled up; a signal would report that "something is happening" but not which station or what problem, and standard signals are not queued: if two SIGUSR1s arrive before the first is handled, one is lost.

Solution 3

Analysis. With no delay, 100,000 readings (2.4 MB) take 89 ms: about 27 MB/s, limited by the 200,000 system calls at roughly 0.44 µs each. With a 10 µs delay per reading, the writer takes 6.4 seconds, exactly the time the reader needs (100,000 × 10 µs of pure delay, plus the overhead of usleep, which rounds every wait up). The lesson is emphatic: the writer goes at exactly the reader's pace even though there is not one line of synchronization. The 64 KB buffer absorbs the short bursts and, when it runs out, the kernel puts the writer to sleep.

Measuring the buffer size:

Exactly 65,536 bytes, 16 pages of 4 KB: Linux's default value, which matches the 2,731 readings from exercise 1 (65,536 / 24 = 2,730.67). It can be raised to 1 MB with fcntl(wr, F_SETPIPE_SZ, 1048576), multiplying by 16 the capacity to absorb bursts: useful if the consumer has occasional long pauses, useless if it is simply slower on average — there the buffer only delays the blocking, it does not avoid it.

Processes do not share an address space, and that is why the kernel offers IPC mechanisms. They all derive from two models: message passing, with two copies per transfer (~5-15 µs for 4 KB) and implicit synchronization, and shared memory, with zero copies (~0.1 µs) and no synchronization. That last phrase is the central warning of the lesson.

Anonymous pipes are a 65,536-byte circular buffer in the kernel with two descriptors; they require pipe() before fork(), closing the end you do not use — or the reader will never see the EOF and will hang — and accepting that they are a byte stream with no message boundaries. Their two edge cases teach more than the normal case: when the buffer fills up, write() blocks and backpressure appears, which we have seen in WCHAN: pipe_write and measured at exactly 2,731 readings; when the reader closes, SIGPIPE arrives and kills the process silently, which is why every serious service does signal(SIGPIPE, SIG_IGN) and handles EPIPE.

FIFOs give the pipe a name in the file system, allow unrelated processes to communicate — Meteora's ingestor→aggregator channel — block the open() as a rendezvous mechanism and guarantee atomic writes of up to 4 KB, which makes several writers safe. POSIX message queues preserve the boundaries, persist with nobody connected and provide what no other local mechanism has: priorities, with which an alert sent later is delivered earlier; in exchange, their limits of 10 messages and 8 KB reserve them for control, not for throughput.

POSIX shared memory — shm_open + ftruncate + mmap with MAP_SHARED — is /dev/shm/meteora-cache: a file in tmpfs mapped by several processes, with kernel persistence, zero access cost and the warning that an n_latest++ in there is exactly the race from 03-01. Sockets are the most versatile: UNIX domain ones are 5-10 times faster than local network ones, they pass descriptors and they reveal the peer's credentials; network ones are the only ones that cross between machines, and with them the ingestor receives the readings of the 800 stations over UDP, enlarging SO_RCVBUF to 4 MB so as not to drop them silently. And signals notify without carrying data, with the golden rule that the handler only raises a volatile sig_atomic_t flag because almost nothing is safe inside it: that is how /etc/meteora/meteora.conf is reloaded with SIGHUP. The decision guide, in one line: network if there could be another machine, a signal if it is a notification with no data, a pipe if it is a direct child, a queue if you need priority, shared memory if you are moving megabytes with critical latency, and a UNIX domain socket when in doubt.

But the asterisk is still there, and it can be put off no longer. We have set up /dev/shm/meteora-cache and left inside it an n_latest++ that loses increments and a struct Reading that can be read half-written. We have the channel; we do not have the protocol that prevents two processes from using it at once. How do you guarantee that only one enters the critical section? What hardware instruction makes a lock possible, if counter++ is not atomic? Why is a mutex not simply a flag, and what does futex do so that the kernel almost never has to be called?

It is the heart of the module: Synchronization and Mutual Exclusion.