The two previous lessons assumed the attacker comes in through the door: that they try to authenticate, that they abuse a sudo rule, that they use a credential that is not theirs. This lesson deals with the opposite: with what happens when nobody knocks at the door because the flaw is inside, in code that is already running, and a few well-chosen bytes are enough to make it do something nobody planned for.

Meteora's scenario is realistic. meteo-api accepts HTTP requests from the Internet; the ingestor accepts connections from weather stations on an open port; all three processes depend on libraries written by third parties that update themselves every few weeks. No credential protects those three surfaces: they are open by definition, because their job is to accept input from outside. The question is no longer "who can get in?", but "what happens when the input is malicious and the program processing it has a bug?".

The lesson has three parts. First, knowing your adversary: an organized overview of the real threats to a server, each with the observable trace it would leave on meteo-01, and Meteora's threat model as a structured exercise. Second, understanding the flaw: the classes of program vulnerability explained conceptually — what gets corrupted and why — always closed off with the correct way to write the code, and the defenses the operating system puts underneath (ASLR, NX, canaries, PIE, RELRO) with the exact command to check that they are active. And third, hardening: the complete, ordered list of measures that turn a freshly installed Debian into meteo-01, each with its command and its justification, from minimizing packages to the restore test of a backup.

An upfront warning, valid for the whole lesson. Attack mechanisms are described at a conceptual level and for defensive purposes: you will find no exploitation instructions and no working attack code, because neither is needed to defend yourself. Any security testing — a port scan, an audit with lynis, a configuration check — must be carried out exclusively on your own systems or with express written authorization from the person responsible. And decisions with legal or regulatory implications (GDPR, data retention, breach notification) must be reviewed with a compliance professional or with legal counsel.

1. Threat landscape and its observable traces
2. Meteora's threat model
3. Classes of program vulnerability
4. The operating system's defenses and how to check them
5. Minimizing the surface: packages, services and ports
6. Security updates and automating them
7. Firewalling with nftables
8. Hardening remote access
9. Isolating the service with systemd
10. Mount options and resource limits
11. Encryption at rest, in transit, and secrets management
12. Backups as a security control
13. Reference frameworks and verification
14. The meteo-01 checklist

Knowing the list of threats is of little use if you do not know what each one looks like from inside the server. That is why every row carries its trace: what you would see on meteo-01 if it were happening.

Three observations that organize the table and are worth being clear about before going on.

Almost every threat shares four traces. A process you should not have, a port you should not have open, a file you should not have and an outbound connection you should not have. That simplifies detection enormously: instead of hunting for each threat separately, you watch those four axes against a known baseline. It is exactly the same "detection by difference" strategy as in 05-02.

The kernel rootkit breaks the logic of all the rest. If the attacker controls the kernel, they also control what ps, ls, netstat and any tool you run can see: you cannot trust the answers of the system you are investigating. It is the decisive argument for external telemetry — sending the logs off the machine — and for cold analysis from a clean boot medium. We develop this in Auditing, Logging and Incident Response.

The supply chain is the one that leaves no trace. When the malicious code arrives inside a legitimate update of a library you use, correctly signed by its repository, no host monitoring distinguishes it from a normal update. The defenses are of a different kind: pin exact versions, verify repository signatures, review which dependencies actually come in, minimize how many there are, and reduce the possible damage with the confinement of section 9.

A threat model is a structured half-hour exercise that answers three questions: what you are protecting, from whom and how they can get in. Without it, hardening becomes a list of recipes with no criterion for prioritizing.

Assets — what has to be protected, with the reason:

Adversaries — who attacks, with what motivation and what resources:

Entry surfaces — how they reach you:

This table is what decides where to invest the effort. Against the automated adversary, which is 99% of hostile traffic, the effective and cheap defense is having nothing obvious open and keeping up to date with updates: firewall, no passwords on SSH, patches. Against the targeted one, you need confinement and detection. And against the insider, least privilege, separation of duties and auditing, which no other layer covers.

Here we explain what gets corrupted and why, because without understanding the mechanism you cannot understand the defense. There is no exploitable code: every section ends with the correct way to write the program.

Recall from 02-03 how a function's stack is laid out: the local variables, the saved frame pointer and the return address, which is the position in the code that will be returned to when the function ends. They are contiguous in memory, and that contiguity is the whole problem.

If input is 200 bytes long, strcpy writes 200 bytes where only 32 fit. The 168 extra ones do not disappear: they overwrite what follows on the stack, including the return address. When the function ends, the CPU jumps to wherever that address says, which now contains what the attacker put there. They have managed to divert the flow of execution with no credential at all. The root cause is always the same: a copy whose size is decided by the attacker and not by the program.

The three rules that avoid the whole family: use bounded functions (snprintf, strncat, memcpy with a computed size) instead of strcpy, strcat, sprintf or gets; derive the bound from sizeof of the destination, never from the length of the input; and check the return value, because truncating silently is a different bug but a bug all the same. In Python, Go, Rust or Java this class does not exist because the language checks the bounds; in C and C++ it is the programmer's responsibility, and that is why it is the most profitable vulnerability in the history of software.

The difference looks cosmetic and is enormous. If the user's text contains specifiers such as %x or %n, the first version interprets them: printf reads arguments nobody passed — revealing stack contents, pointers and data included — and %n goes as far as writing to memory. The rule is absolute and easy to audit: the format string is always a constant of the program; external data always goes in as an argument. Compile with -Wformat -Wformat-security and the compiler will warn you.

It is the most frequent class today, and the one that shows up most in administration code.

The problem is not grep: it is that the text is handed to a shell, and the shell interprets ;, |, &&, $(...) and wildcards before running anything. A value such as x; arbitrary-command becomes two commands, the second chosen by whoever sent the request, run with meteora's identity. The root cause: mixing code and data in the same string, exactly as in SQL injection.

Three independent defenses, and each one plugs something different. The argument list means there is no shell at all: the system runs execve("/usr/bin/grep", ["grep", "--", "PATIO-01", ...]) and the contents of station are an argument, not code, however many ; characters it carries. The -- stops a value beginning with - from being interpreted as an option of grep. And whitelist validation rejects anything that does not have the expected shape, instead of trying to enumerate what is forbidden — which always forgets something. The general rule: never shell=True, never os.system, never concatenate; and in C, execve with a separate argv instead of system().

It already appeared in 04-04 and in 05-01 as the enemy of complete mediation. The vulnerable pattern is always the same: check something about a file name and then act on that name, because between the two operations the attacker can change what the name points to.

The correct version applies the lesson of 05-01: a file descriptor is a capability, a key over a specific object. Once open, the descriptor keeps pointing at the same inode even if someone renames the file or replaces the name with a link, so there is no window. O_EXCL makes the operation fail if the destination already exists and O_NOFOLLOW prevents following a symbolic link. It is especially critical in shared directories writable by several parties, which is the reason for PrivateTmp=yes in section 9.

In C and C++, freeing memory with free() does not clear the pointer: it still points to the same place, but that place is no longer yours. If the program uses it afterwards, it reads or writes in an area the allocator may have reused for something else — and if the attacker manages to get their data to end up there, the program operates on content they control. The discipline that avoids it is simple and must always be applied: set the pointer to NULL immediately after freeing, do not free twice, and make clear in every structure who owns each block of memory. Tools such as valgrind and the compiler's sanitizers (-fsanitize=address) detect these bugs during testing, and using them in continuous integration is one of the most profitable investments there is.

The operating system cannot stop a program from having bugs, but it can make exploiting them far more expensive. These are the layers it contributes, and how to check that they are active on your machine.

How to read that output. Full RELRO means the link tables are left read-only after loading; Partial leaves part of them writable and No RELRO is a sign that the binary was compiled without the usual protections. Canary: found indicates that the compiler inserted the stack check — you get it with -fstack-protector-strong, on by default in Debian. NX: enabled is the non-executable stack. And PIE enabled lets ASLR randomize the position of the program's code as well; without PIE, the binary is always at the same address and a good part of ASLR is lost. If you compile your own software for meteo-01, the minimum flags are:

-D_FORTIFY_SOURCE=2 replaces dangerous functions with versions that check sizes when the compiler knows them; -Wl,-z,now forces full symbol resolution at load time, which is what makes full RELRO possible.

And the warning that prevents overconfidence: these are layers, not solutions. Each one raises the cost of one step, and the techniques to get around each of them individually have been known for years. Their value is in the combination — and in the fact that, together with seccomp and AppArmor, they turn a bug that used to give control of the machine into, with luck, a service that crashes and restarts. None of them replaces fixing the bug, and that is why the next section starts with updates.

The principle comes from 05-01: code that is not installed has no vulnerabilities. A minimal Debian ships around 400 packages; an installation with "desktop environment" ticked by accident, more than 1,500. Every extra package is code that may have bugs, that has to be updated, and that widens what an attacker finds on arrival.

The output of ss -tulnp is the server's real network surface, and you have to know how to read the address column, which is where the important information is. 0.0.0.0:443 means "listens on all interfaces", correct for a public API. 10.0.5.11:9010 listens only on the station network interface, so the public API cannot reach that port: it is a free and very effective restriction. And 127.0.0.1:5432 listens only locally, so the database is unreachable from outside even if the firewall failed. The rule that follows: every service must listen on the minimum interface it needs, and that configuration is a layer independent of the firewall.

disable --now stops the service and prevents it from starting; purge goes further and removes the software and its configuration. The distinction matters: a disabled service is still on disk and can be reactivated by an update or by a configuration mistake. And apt autoremove --purge cleans up dependencies nobody uses any more, which are usually the bulk of the excess.

It is, by a wide margin, the measure with the best effort-to-protection ratio. The vast majority of real server compromises do not exploit unknown bugs, but bugs published and fixed weeks or months ago on systems nobody updated.

The three decisions in that configuration. Only the -security origin, not all updates: Debian stable's security patches are minimal and very conservative, whereas updating everything automatically introduces functional changes that can break the service in the middle of the night. Automatic-Reboot "false" because an unannounced reboot of meteo-01 is a service interruption; reboots are planned, and /var/run/reboot-required indicates when one is needed. And the notification email, because automation nobody watches ends up failing silently: the most common failure is an apt blocked by a half-finished upgrade that has gone unapplied for months.

For the kernel there is also live patching (kpatch, livepatch), which avoids reboots; it is useful in environments where availability rules, and it does not replace rebooting now and then with a new kernel.

The firewall applies deny by default to network traffic: you enumerate what is allowed and reject the rest, including what does not exist yet. On current Debian the engine is nftables; ufw is a simple interface over the same thing and is perfectly valid for simple cases.

Every rule answers a decision from the threat model. policy drop on the input chain is the fail-safe defaults principle: if someone starts a new service tomorrow, it is not exposed by accident. ct state established,related accept allows the replies to traffic we initiated, and it goes first because it is the rule that matches the most packets — order matters for performance. SSH restricted by origin eliminates the noise of Internet scans in one stroke, and the limit of 6 new connections per minute makes any repetitive attempt expensive. The limit on 443 is a basic defense against saturation. The station rule applies the surface from the threat table: only that network, only that port. And the final counter is a very useful diagnostic detail: it tells you how much traffic is being dropped, which is the first thing you will want to know when something "will not connect".

The output chain deserves a separate comment. Leaving it on accept is the usual and comfortable option. Setting it to drop and enumerating the allowed destinations is noticeably more secure — it cuts off data exfiltration, connections to command-and-control servers and the downloading of tools by a compromised process, three whole rows of the threat table — and it is also considerably more work to maintain: you have to allow DNS, NTP, the package repository and any APIs you depend on. For a server with valuable data like meteo-01, it is worth it.

Before applying rules on a remote server, validate with nft -c, and protect yourself against a mistake with an automatic rollback mechanism: for example, sudo sh -c 'sleep 300 && nft flush ruleset' & before applying, cancelling it if all goes well. Locking yourself out with your own firewall is a classic.

The SSH configuration was already justified line by line in Users, Authentication and Privilege Escalation, so here we only collect the minimum set and what it adds to what we have already seen:

What is worth adding at the system hardening level are two complementary layers, because they protect against different things: the restriction by origin in the firewall from the previous section, which makes the service unreachable from the Internet in the first place; and a dynamic block such as fail2ban, which watches the logs and temporarily blocks origins with many failures. With PasswordAuthentication no, fail2ban contributes little against compromise — there is no password to try — but it remains useful for reducing the noise in the logs, which is a real benefit: a log full of thousands of automated attempts hides the one attempt that does matter.

This is where everything from 05-01 becomes concrete. Systemd lets you confine a service with a dozen directives, without touching the application's code and without writing a complete MAC policy. This is meteo-api's unit, with comments:

The directives that add the most value, and which specific attack each one stops. ProtectSystem=strict leaves the whole file system read-only and forces you to declare in ReadWritePaths exactly where writing happens: it is a file system whitelist that no DAC permission can contradict, so a compromised process cannot modify binaries or leave anything persistent outside two paths. NoNewPrivileges=yes prevents any later privilege gain, including from running a system setuid binary; it cuts off escalation by that route even if a vulnerable setuid binary exists. MemoryDenyWriteExecute=yes prevents a page from being writable and executable at the same time, which blocks the usual technique of generating code in memory and jumping to it. PrivateTmp=yes gives it its own /tmp, eliminating temporary file attacks and TOCTOU in shared directories in one go. IPAddressDeny=any with a whitelist is the outbound firewall applied to a single service: even if the process falls, it cannot call home. And SystemCallFilter is seccomp without writing a line of C.

systemd-analyze security is the key tool of this section: it analyzes the unit, scores its exposure and lists exactly which directives are missing, ordered by impact. An unhardened unit scores around 9.5; with the directives above it drops into the 1.5 to 2.5 range. Apply them incrementally, checking the service after each block: if meteo-api stops starting, it is almost always because it needs to write to a path missing from ReadWritePaths, and journalctl -p err says so.

The mount options from 04-03 are pure hardening, and here they fall into place:

The three options close three vectors for the price of three words. nosuid makes the kernel ignore the setuid and setgid bits across the whole volume, so a setuid binary dropped there is useless. nodev ignores device files, preventing anyone from creating a /var/lib/meteora/disk that would give direct access to the raw disk. And noexec prevents running any binary from the volume: it is exactly what you want in /tmp, /dev/shm and /var/log, because they are the places a compromised process can write to. Notice that /var/lib/meteora does not carry noexec for consistency with its use — it only holds data — although adding it would also be reasonable; what it must not be missing is noatime, because that option is about performance and has already been justified since 04-03.

Resource limits are the defense against exhaustion, which is the "denial of service" row of the threat table:

Why they matter. Without LimitNOFILE, a bug that opens descriptors without closing them exhausts the system's global limit and leaves the other services with none: a local bug turns into a general outage. LimitNPROC and TasksMax curb the uncontrolled creation of processes. MemoryMax prevents runaway consumption from triggering the OOM killer of 02-04, which might kill a different process from the guilty one. And core 0 deserves special attention: a core dump contains everything the process had in RAM, including the keys from secrets.conf it has just read, and it usually ends up in a poorly protected directory or in a crash analysis system.

At rest: LUKS. Disk encryption protects against a very specific and very real scenario: someone obtaining the physical disk — theft, hardware decommissioning, a faulty disk returned to the manufacturer without being wiped.

The important thing is to know what it does not protect against, so as not to create false security: with the system booted and the volume open, the data is accessible like any other file, so LUKS gives no protection at all against a compromised meteo-api or against an attacker with access to the running system. It protects the disk when powered off. And it brings an operational consequence that has to be decided up front: an encrypted volume needs the key at every boot, so either someone types it — and the server does not come back on its own after a power cut — or it is stored in a TPM or a key server, with its own implications.

In transit: TLS. Everything leaving or entering over the network must be encrypted and authenticated: the API with TLS 1.2 as a minimum — preferably 1.3 — with certificates renewed automatically, and mutual TLS on the station port, which additionally authenticates the station and shuts down the "compromised station" row of the threat model. Cleartext protocols — telnet, FTP, HTTP for data, SNMP v1 and v2 — have no place on a modern server.

Secrets management. /etc/meteora/secrets.conf with mode 0600 (or 0640 with group meteora, as we set in 04-06) is the acceptable minimum, not a good solution. Its limits are concrete: the secret sits in the clear on disk, it goes into the backups, anyone with root reads it, and rotating it requires touching the server.

And three rules that always hold, whatever the approach: never in the code or in the repository — it would stay in the Git history forever, even if you delete it later; never in environment variables, because they are visible in /proc/<pid>/environ and leak into child processes and error dumps; and rotatable, because a secret that cannot be rotated without a planned outage ends up never being rotated.

Backups do not usually appear in security lists, and they are the only real defense against ransomware and against destructive deletion. The classic rule is still the best guide:

3-2-1 rule: 3 copies of the data, on 2 different media, with 1 off site.

Applied to Meteora: the original on the RAID 1 of /var/lib/meteora; a daily copy on different storage in the same data center, for a fast restore; and a remote copy, for the case of fire, flood or compromise of the whole site. And the nuance that updates it for the ransomware era:

Immutable backups. If the attacker arrives with privileges, they will look for the backups before encrypting anything, because accessible backups make their extortion useless. Hence the three properties the destination must have: credentials different from the server's — never the same keys, nor a permanently writable mount; an append-only model, so the server can create backups but neither delete nor overwrite earlier ones; and retention enforced by the destination system for a fixed period, not by a policy the attacker can change.

The restore test is the non-negotiable part of all this. A backup that has never been restored is not a backup: it is an assumption. The failures that show up in the first real restore are always the same and always at the worst moment — the repository's encryption key is not documented, a directory was missing from the path list, the process takes fourteen hours and nobody knew, or the permissions and ACLs were lost because the tool did not store them (04-06). Schedule a test restore quarterly, with its time measured and its checksums verified, and document the procedure step by step.

There is no need to invent hardening from scratch: there are public guides reviewed by many people.

Two warnings about these tools. Applying a complete benchmark blindly breaks services: the CIS Benchmarks include recommendations designed for very different profiles, and some are incompatible with what meteo-01 needs to do. You read them, you decide item by item, and you document the exceptions with their justification, which is what later lets you defend the configuration in an audit. And a high lynis score is not security: it measures conformance with a list, not resistance to an adversary; it serves to find what you have forgotten, not to declare the system secure.

Prior authorization. Running lynis or any scanner on your server is normal and advisable. Running any analysis tool — including a simple port scan — against a system that is not yours requires express written authorization from the person responsible, with a defined scope and defined dates. Without that document, the action may constitute a criminal offense regardless of intent, and good faith is not a defense.

Hardening without having done the threat model. You end up investing in what you know how to do instead of in what protects you. Half an hour enumerating assets, adversaries and surfaces changes the order of the task list.

Applying a complete benchmark all at once in production. It breaks services and creates distrust of hardening as a whole. Apply it in blocks, verify the service after each one and document the exceptions.

Trusting ASLR, NX and canaries as if they were a solution. They are layers that raise the cost of specific steps, and each has known techniques for getting around it. They do not replace fixing the bug or applying updates.

Believing LUKS protects you from a compromised server. It protects the disk when powered off. With the system booted and the volume open, the data is ordinary files for any process with permissions.

Putting secrets in environment variables. They are visible in /proc/<pid>/environ, inherited by children and appear in dumps and error traces. Use a file with restrictive permissions, LoadCredential= or a secrets manager.

Having backups and never having restored them. An unverified backup is an assumption. Schedule the quarterly test restore, measure how long it takes and check the checksums.

Leaving a service listening on 0.0.0.0 when only localhost uses it. It is free surface. Look at the address column in ss -tulnp and restrict the interface: it is a layer independent of the firewall and it survives a mistake in it.

Automating updates and not checking that they work. The typical failure is an apt blocked by a half-finished upgrade that has gone unapplied for months. Configure the email notification and read it.

Applying firewall rules on a remote server with no safety net. Validate with nft -c and leave an automatic rollback scheduled that you cancel if all goes well.

Tip: measure before and after. systemd-analyze security, lynis and the port inventory give you concrete numbers. Record the initial state, apply, and compare: it turns hardening into something verifiable and lets you justify the time invested.

On a machine of your own or a test virtual machine: (a) enumerate its real network surface with ss -tulnp, and state for each port who opened it, which interface it listens on and whether that exposure is justified; (b) list the running services and decide which ones you could disable, with the justification; (c) build the table of assets, adversaries and surfaces for the system; and (d) for the three threats from the table in section 1 that you consider most likely in your case, state which concrete trace you would look for and with which command.

Start from this unhardened unit and take it to the level of section 9, adding the directives in blocks and verifying the service after each block. For every directive you add, explain which specific attack it stops. Measure the systemd-analyze security score before and after, and explain why each block reduces it. Also state which error you would see in journalctl if ReadWritePaths fell short and how you would diagnose it.

For each fragment, identify the class of vulnerability, explain what gets corrupted or what gets executed beyond what was intended and why, write the correct version and say which operating system defense (section 4) or systemd directive (section 9) would reduce the damage if the bug reached production.

Solution 1

(a) Every line of ss -tulnp is interpreted through the local address column. 127.0.0.1:port is local and represents no external exposure; 0.0.0.0:port or [::]:port listens on all interfaces and is real exposure; a specific IP limits it to that network. For every port there are three questions to answer: which process opened it (the users: column), who needs to reach it and whether the interface is the smallest possible. A common and entirely avoidable finding is a database or a development server listening on 0.0.0.0 when only the machine itself uses it; it is fixed in the service's configuration, and that fix is a layer independent of the firewall that keeps protecting if the rules fail.

(b) Usual candidates for disabling on a server: network discovery (avahi-daemon), printing (cups), Bluetooth, graphical servers and management services no longer in use. The criterion is twofold: if there is no clear answer to "who uses it and what for?", disable it; and purge is preferable to disable, because a disabled service is still on disk and can be reactivated by an update.

(c) The table must include, as a minimum: the assets with their impact in case of loss or leak; the adversaries, not forgetting that the automated one is continuous and the insider is the hardest to detect; and the surfaces with their current protection and their residual risk. The useful result is the prioritization: against the automated adversary, which is almost all the hostile traffic, the cheap and effective defense is a firewall, no passwords on SSH and up-to-date patches.

(d) Traces and commands:

What turns this into real detection is not running the commands once, but comparing them against a known baseline: the list of ports, of processes, of setuid files and of authorized keys. The four axes section 1 talked about.

Solution 2

Block 1 — identity and privilege. User=meteora, Group=meteora, UMask=0027, NoNewPrivileges=yes, CapabilityBoundingSet=CAP_NET_BIND_SERVICE. It stops the most serious thing: without User=, the service runs as root and any bug in request parsing hands over the whole machine. NoNewPrivileges=yes cuts off escalation via setuid binaries even if a vulnerable one exists on the system, and the CapabilityBoundingSet prevents forever the process or its children from obtaining mounting, module-loading or debugging capabilities.

Block 2 — file system. ProtectSystem=strict, ProtectHome=yes, PrivateTmp=yes, ReadWritePaths=, ReadOnlyPaths=, PrivateDevices=yes, ProtectProc=invisible. ProtectSystem=strict leaves everything read-only except what is declared, so a compromised process cannot modify binaries or leave persistence behind; ProtectHome hides /home and /root from it, where SSH keys and credentials live; and PrivateTmp eliminates shared temporary file attacks and TOCTOU in /tmp.

Block 3 — kernel, memory and network. ProtectKernelTunables/Modules/Logs, MemoryDenyWriteExecute=yes, RestrictSUIDSGID=yes, RestrictAddressFamilies=, IPAddressDeny=any with IPAddressAllow=. MemoryDenyWriteExecute blocks generating code in memory; RestrictAddressFamilies cuts off socket families the API does not need; and IPAddressDeny=any prevents exfiltration and connection to command-and-control servers, which are two complete rows of the threat table.

Block 4 — system calls and resources. SystemCallFilter=@system-service with the subtractions of @privileged @mount @module @debug, SystemCallArchitectures=native, and the limits LimitNOFILE, LimitNPROC, MemoryMax, TasksMax. The filter reduces the surface from ~350 calls to the ones the service really uses (05-01); SystemCallArchitectures=native closes the hole of invoking through the 32-bit ABI to dodge the filter; and the limits stop a bug in the service from exhausting the descriptors or the memory of the entire system.

Why the score drops: the tool weights each unmitigated exposure, and the two blocks with the most weight are privilege and file system, precisely because they are the ones that determine whether a compromise stays contained or spreads.

If ReadWritePaths falls short, the service starts and fails when writing. In journalctl -u meteo-api -p err you will see an error of the "Read-only file system" type (EROFS) about a specific path, which is exactly the one missing from the declaration. The systematic way to diagnose it is to launch the binary under the same confinement with systemd-run and the same directives, or to check with strace -f -e trace=openat,write in a test environment — never in production — to see which paths it opens for writing.

Solution 3

A — Buffer overflow. strcpy copies up to the \0 without looking at the fact that path is 64 bytes. A longer name overwrites what follows on the stack, including the return address, and when the function ends the CPU jumps wherever the attacker says. Fix: if (snprintf(path, sizeof path, "%s", name) >= (int) sizeof path) return -1;, with the bound derived from sizeof of the destination and the return value checked. System mitigation: stack canary (it detects it and aborts), ASLR + PIE (the attacker does not know where to jump), NX (they cannot run code on the stack) and MemoryDenyWriteExecute=yes in the unit.

B — Command injection. The file name arrives from an HTTP request and is concatenated into a command run by a shell, and the shell interprets ;, |, && and $(...) before executing. A value with a ; produces a second command chosen by whoever sent the request, with meteora's identity. Fix: validate against a whitelist and run without a shell with separate arguments, subprocess.run(["gzip", "--", path], shell=False), after additionally checking with realpath that the path is still inside the intended directory — if not, it is also a confused deputy (05-01). Mitigation: SystemCallFilter without execve, an AppArmor profile with no execution rules, and ProtectSystem=strict.

C — TOCTOU. Between os.path.exists(destination) and open(destination, "w") there is a window in which the attacker can create a symbolic link there pointing to another file, and the write would end up at the destination of their choosing, with the service's permissions. Fix: do not check the name and then use it, but open directly with os.open(destination, O_WRONLY|O_CREAT|O_EXCL|O_NOFOLLOW, 0o640), which fails if it already exists or if it is a link, and operate on the descriptor. Mitigation: PrivateTmp=yes if it is a temporary directory, a tightly scoped ReadWritePaths, and the directory permissions of 04-06.

D — Format string. message_received_from_station is used as the format string, so fprintf interprets whatever specifiers it contains: %x reveals stack contents and %n allows writing to memory. Fix: fprintf(log, "%s", message_received_from_station);, with the format string always constant. Mitigation: compile with -Wformat -Wformat-security — the compiler detects it at compile time, which is where it should be caught — -D_FORTIFY_SOURCE=2, and full RELRO, which prevents overwriting the link tables.

The lesson common to all four: system mitigations make exploitation more expensive, they do not fix the bug. They are valuable because they turn a total compromise into a service crash, but the fix is always in the code, and that is why reviews, sanitizers and updates are the first line and not the last.

A real server faces a bounded catalog of threats — malware, ransomware, userspace and kernel rootkits, backdoors, cryptomining, botnets, denial of service, data theft and supply chain — and almost all of them leave four common traces: a process, a port, a file or an outbound connection you should not have. That reduces detection to watching those four axes against a baseline. Two threats break the scheme and are worth keeping in mind: the kernel rootkit, which corrupts the answers of the very tools you investigate with — hence the need for external telemetry — and the supply chain, which leaves no trace because it arrives signed inside a legitimate update. The threat model — assets, adversaries, surfaces — is what gives you a criterion for prioritizing: against the automated adversary, which is 99% of the noise, it is enough to have nothing obvious open and to keep up with patches; against the targeted one you need confinement and detection; and against the insider, least privilege and auditing.

The classes of program vulnerability are better understood by their root cause than by their name. The buffer overflow happens when the size of a copy is decided by the attacker and not by the program, and what it corrupts is the return address sitting contiguously on the stack. Format strings appear when external data takes the place of the format string, which must always be a constant. Command injection is born of mixing code and data in a string interpreted by a shell, and it disappears when you execute with execve and separate arguments, without a shell, with whitelist validation. TOCTOU lives in the window between checking a name and using it, and it is closed by operating on the descriptor, which is a capability. And use after free is avoided with memory ownership discipline and detection tools during testing. The system contributes layers that make exploitation more expensive — ASLR and KASLR, NX, canaries, PIE, RELRO, seccomp — checkable with checksec and /proc/sys/kernel/randomize_va_space; they are layers, not solutions, and none of them replaces fixing the bug.

The hardening of meteo-01 has an order that reflects the relationship between effort and protection. First, automatic security updates, because most real compromises exploit already-fixed bugs. Next, minimizing the surface: purge packages, switch off services and make each one listen on the minimum interface, something ss -tulnp reveals in a second. Then the firewall with policy drop, which applies deny by default to whatever gets installed tomorrow as well, with the output chain restricted if the data warrants it. Remote access, already hardened in 05-02, with restriction by origin and noise reduction in the logs. And the block with the most value per line written: isolation with systemd, where NoNewPrivileges, ProtectSystem=strict, PrivateTmp, CapabilityBoundingSet, MemoryDenyWriteExecute, IPAddressDeny and SystemCallFilter implement, without touching the code, everything 05-01 set out in theory — and systemd-analyze security measures it. To that are added the mount options nosuid,nodev,noexec, the resource limits as a defense against exhaustion, encryption at rest with LUKS — which protects the powered-off disk, not a compromised system — and in transit with TLS, and a secrets management approach in which the 0600 file is the minimum and never the ideal. Rounding it off are the 3-2-1 backups with an immutable destination and separate credentials, the only real defense against ransomware, whose non-negotiable part is the restore test; and the verification frameworks — CIS, lynis, debsecan — which are adapted and documented, never applied blindly, and whose use on other people's systems requires express written authorization.

With this, meteo-01 is reasonably protected. But notice what is still missing, and it is the most uncomfortable part: everything above is prevention. None of these measures tells you whether something has happened. If tomorrow at 03:12 there is a spike in writes to /var/lib/meteora and a process nobody recognizes appears, how do you find out? What logs exist, where are they and what do they contain? How do you investigate in the correct order, without destroying the evidence along the way? And what is a log written by a system the attacker controls actually worth?

That is Auditing, Logging and Incident Response, where we will look at syslog and journald with their filters, how an application's logging is designed and what it must never contain, rotation and retention, the kernel's auditd subsystem, the integrity of logs against an attacker with privileges, change detection with AIDE, and the full incident response cycle applied to that 03:12 spike.