You already know what a certificate is (05-01) and how TLS puts it to work on every connection (05-02). But in production a certificate isn't a concept: it's an operational artifact with an expiration date, which someone has to issue, deploy, monitor, renew, and — on a bad day — revoke. Most real-world "TLS incidents" aren't cryptanalysis: they're certificates that expired on a Sunday because nobody renewed them. In this lesson you close out the module with the piece that turns everything before it into a reliable service: the full lifecycle, automation with ACME/Let's Encrypt and certbot, revocation mechanisms and their miseries (CRL, OCSP, stapling), an expiration monitor in Python for MediNube's domains, the lifecycle of internal certificates and clinics' mTLS client certificates, and a simulated incident: portal.medinube.example's private key leaked into a public repo, with its step-by-step runbook.

1. The certificate as an operational artifact: the lifecycle
2. Expiration: why short lifetimes are an advantage
3. ACME and Let's Encrypt: automatic issuance and renewal
4. certbot for portal.medinube.example
5. Revocation: when and how (CRL, OCSP, stapling)
6. Monitoring: finding out before expiration
7. Internal certificates: the mini-PKI and clinics' mTLS clients
8. Pinning: mention and risks
9. Simulated incident: leaked private key — the runbook

A certificate is born, works, and dies. Everything in this lesson fits into this cycle:

Read the diagram as a loop: normal life is issue → deploy → monitor → renew → deploy..., and revocation is the emergency exit. Every arrow is a classic failure point: issuing without deploying (the new certificate sits in a folder), deploying without reloading nginx (it serves the old one until restart), renewing without monitoring (you don't know if it worked), revoking without reissuing (service down). The lesson's thesis: everything that can be automatic, should be, and whatever can't needs a written runbook.

It might seem annoying that certificates expire, and increasingly sooner too: from the 3-5 years typical in the 2010s, the industry moved to the 398-day maximum browsers accept (2020), Let's Encrypt issues certificates valid for 90 days, and the industry (CA/Browser Forum) has approved a staged reduction toward ~47-day certificates over the coming years. It's not sadism, it's security:

• It limits the damage window. If a key is compromised and nobody detects it, a 90-day certificate stops being valid within weeks; a 3-year one, within years. As you'll see below, revocation works poorly: short lifetime is the revocation that actually works.
• It forces automation. Nobody renews by hand every 90 days without eventually failing; with short lifetimes, you either automate or you suffer. And a process that's automated and exercised every 60 days is a process you know works — compared to the yearly ritual nobody remembers how to run.
• It speeds up crypto-agility (rule 8). If tomorrow you need to drop an algorithm or switch CAs, a certificate fleet that rotates every 90 days migrates entirely within weeks.
• It keeps data fresh. Domains change owners, companies disappear: information validated by the CA goes stale fast.

Mindset shift for the MediNube team: a certificate isn't "something you install," it's something that rotates, the way keys should rotate too (a topic module 6 revisits for application keys).

Let's Encrypt (2015) changed the game: free DV certificates, issued in seconds, through an open automated-issuance protocol: ACME (RFC 8555). ACME's idea is to automate exactly what 05-01 had "the CA doing behind the scenes": verifying that whoever requests the certificate controls the domain. To do that, it proposes challenges:

The full flow, with pieces you already know: the ACME client generates an account key (signatures, 04-03) → requests a certificate for portal.medinube.example → the CA proposes the challenge → the client solves it (HTTP file or DNS TXT record) → the CA checks it from outside → the client sends the CSR (05-01) → the CA issues it and publishes it in the Certificate Transparency logs (05-01) → the client installs the certificate. Zero humans.

certbot is the reference ACME client. Setting it up for the portal, with the nginx from 05-02 already serving the domain:

After this, certbot leaves the material at stable paths that nginx references (notice: they're the ones from 05-02!):

• /etc/letsencrypt/live/portal.medinube.example/fullchain.pem — leaf + intermediate.
• /etc/letsencrypt/live/portal.medinube.example/privkey.pem — the private key (restrictive permissions; live/ entries are symlinks to the latest version, which is why nginx needs no changes on renewal).

Automatic renewal is the half that matters. Modern packages install a systemd timer (or a cron job) that runs certbot renew twice a day; certbot only renews certificates with fewer than 30 days left, and reloads nginx after renewing (via a hook):

For the *.medinube.example wildcard (if MediNube ever gave each clinic its own subdomain), the challenge must be DNS-01, using a plugin for the DNS provider (certbot-dns-cloudflare, certbot-dns-route53, etc.) and its credentials — credentials that are one more secret to protect, a direct preview of 06-01.

With this, "monitoring" doesn't disappear: automatic renewal fails too (DNS changed, a new firewall blocks port 80, the DNS plugin's credential expired). Automate renewal and watch its outcome (monitoring section).

Revoking = declaring a certificate invalid before it expires. The two triggers that matter to MediNube:

1. Private key compromise (the incident at the end of this lesson): anyone with the key can impersonate the portal until the certificate stops being valid.
2. An entity offboards: Luna Medical Center cancels its contract — its mTLS client certificate (05-02) must stop granting API access today, not whenever it happens to expire.

The problem: telling every client in the world that a given certificate is no longer valid is inherently hard. The mechanisms, all imperfect:

Honest reading of the table: revocation in the public Web PKI is a best-effort mechanism. That's why the industry pushes ever-shorter lifetimes, and why, after a compromise, the runbook (last section) never stops at "revoke": it's always revoke + reissue with a new key + deploy, assuming some clients will never hear about the revocation. By contrast, in an internal PKI (our mini-PKI), you control both ends: you can distribute real CRLs and have your servers strictly enforce them — internal revocation actually can be reliable.

With the lab mini-PKI, revoking requires keeping a small database of issuances (openssl ca instead of the direct openssl x509 -req from 05-01); the conceptual flow is: mark the serial as revoked → regenerate the signed CRL (openssl ca -gencrl) → distribute it to the servers that validate (the mTLS nginx loads it with ssl_crl).

Operating rule: no MediNube certificate expires as a surprise. The simplest, most robust monitor doesn't look at configuration, it looks at what's actually being served — the same way a client would. Script medinube/monitor_certs.py:

Design points, in detail:

• We use ssl.create_default_context() with validation on: if the certificate no longer validates, we want the critical alert, not to read it anyway. (Inspecting already-expired certificates would need tricks that skip validation; for monitoring, validation is part of the test.)
• getpeercert(binary_form=True) returns the leaf's DER; cryptography gives us not_valid_after_utc as a timezone-aware datetime, and we subtract against datetime.now(timezone.utc) — the same pattern from 05-01's exercise 2.
• The 30-day threshold isn't arbitrary: certbot renews once there are fewer than 30 days left, so "fewer than 30 days left and still not renewed" means the automation is failing and there's still human reaction time.
• The exit code (0/1/2) is understood by any task scheduler or monitoring system. In cron: 0 8 * * * python3 /opt/medinube/monitor_certs.py || curl ... # notify. To notify the team's channel, reuse the signed webhook from 03-02: the alert travels with X-MediNube-Timestamp and X-MediNube-Firma: v1=<hex>, because a forgeable security alert would be ironic.
• A natural extension (exercise 1): also watch clinics' mTLS client certificates, which can't be checked by connecting to anything — you have to read the PEM files issued by the internal CA.

The whole lifecycle applies equally — with nuances — to certificates that don't come from a public CA:

The lab's root CA (05-01). It also expires (we issued it for 5 years). Renewing it is a major event: the new root must be redistributed to every client that trusts it (verify=/etc/medinube/pki/ca.crt in the 05-02 integrations) before the old one expires, with an overlap period where both are valid. Put it on the team calendar the day you create it: "renew internal root: start 6 months before 2031." Internal leaves, by contrast, rotate like public ones: short-lived and automated (ACME-like internal tools exist — step-ca, HashiCorp Vault — for when the artisanal mini-PKI outgrows itself; just a mention, secrets management is 06-01).

Onboarding a clinic (mTLS, 05-02). Standard procedure: the clinic generates its own key and CSR (the private key never travels, 05-01) with SAN clinica-sol.clientes.medinube.example → MediNube verifies over a trusted channel that the CSR really belongs to who it claims (identity validation is the CA's job, even when the CA is you!) → the internal CA signs it with a short lifetime (90 days) → the clinic installs it and tests it against the mTLS endpoint.

Offboarding a clinic = revocation + defense in depth. Luna Medical Center cancels its contract. Steps: (1) revoke its certificate in the internal CA and regenerate/distribute the CRL to the mTLS nginx; (2) additionally, disable its identity ($ssl_client_s_dn) at the application's authorization layer — because rule 9 still applies: cryptography doesn't replace access control, so the offboarding takes effect even if a CRL is slow to propagate; (3) the 90-day short lifetime acts as the final safety net. Three layers, none sufficient alone, all three together robust.

You'll hear about pinning: fixing, on the client (typically a mobile app), the fingerprint of the server's certificate or public key, and rejecting any other one even if the chain validates. It raises the bar against compromised CAs... and it's an operational trap: the pin lives inside the installed app, and the certificate rotates every 90 days. If you rotate the key without updating the pin (or without having also pinned a backup key), your own app refuses to talk to you — a self-inflicted denial of service across your entire installed base. Browsers' HPKP was retired precisely because of this class of accident; today the ecosystem prefers Certificate Transparency (05-01) as a control. For MediNube: no pinning except in very well-justified cases, and always with backup keys and a pin expiration. Know it exists, know why it almost never pays off.

Friday, 5:40 p.m. A MediNube developer flags that, while making a utilities repository public, they discovered it had contained, for three weeks, a deploy/ directory with portal.medinube.example's privkey.pem. The portal certificate's private key is sitting in a public repo. Panic? No: runbook. (Reassuring note you owe to 05-02: thanks to TLS 1.3's forward secrecy, past captured traffic is NOT decryptable with that key; the risk from now on is impersonation.)

1. Confirm and scope it (minute 0). Is it really the key in use? Compare the public key derived from the leaked file against the one from the served certificate — same key = same modulus/public point:

   openssl pkey -in leaked_privkey.pem -pubout | openssl sha256
   openssl s_client -connect portal.medinube.example:443 \\
       -servername portal.medinube.example </dev/null 2>/dev/null \\
     | openssl x509 -noout -pubkey | openssl sha256
   # Do the hashes match? Then the active key is compromised.
   

2. Treat the key as compromised, effective immediately. Three weeks public = copied. Removing it from the repo does not undo anything (git history and other people's clones still exist); rewriting history is hygiene, not remediation.

3. Generate a new key + CSR + reissue (minutes 5-15). Never reissue with the same key. With certbot it's a single step that forces a new key and reissues:

   sudo certbot certonly --nginx -d portal.medinube.example --force-renewal
   # (certbot generates a new privkey by default on every issuance)
   

4. Deploy and reload (minutes 15-20). sudo systemctl reload nginx and verify with step 1's command that the portal is now serving the new key (the hashes no longer match the leaked one) and with s_client that Verify return code: 0 (ok).

5. Revoke the old certificate (minutes 20-30). Now that the new one is serving traffic, invalidate the old one — certbot lets you declare the reason:

   sudo certbot revoke \\
       --cert-path /etc/letsencrypt/archive/portal.medinube.example/cert1.pem \\
       --reason keycompromise
   

   Remember the revocation section: some clients will never find out; that's why steps 3-4 come first — the service is already safe even if revocation propagates slowly. (With Certificate Transparency, also watch for any new, unrequested issuances for your domains: if the attacker tried anything else, it'll show up.)

6. Collateral hunting. What else was in that repo? Other keys, tokens, certbot's DNS plugin credentials? Every finding opens its own mini rotation runbook.

7. Blameless post-mortem (the following week). Root cause: a private key living in an ordinary working directory, ready to be committed. Actions: a secrets scanner in CI (that blocks the push), defensive .gitignore entries, and the underlying fix: keys don't live in repos or project directories; they live in a secrets manager. Which is exactly where module 6 begins.

Rehearse this runbook in the lab (exercise 3) before you need it: an unrehearsed runbook is just a hypothesis.

• Trusting that "certbot is already set up" without watching it. Automatic renewal fails silently (new firewall, changed DNS, expired credential). certbot renew --dry-run after every infrastructure change + an external expiration monitor. Automate and verify.
• Renewing the certificate and not reloading the service. nginx loads the certificate at startup/reload; if the reload hook is missing, you'll serve the old one until it expires in front of your users. The monitor catches this because it looks at what's served, not what's issued.
• Reissuing after a compromise while reusing the compromised private key. Reissuing with the same compromised key is like putting a new case on a stolen phone. Always a new key.
• Trusting a clinic's offboarding to revocation alone. CRL/OCSP propagate slowly or aren't checked at all. Offboarding = revoke and deauthorize in the application and let the short lifetime finish the job.
• Wildcards "to simplify things" without thinking it through. *.medinube.example is a single key that, if compromised, covers every subdomain, and requires exposing DNS credentials to the automation. Use it when the case calls for it, not out of laziness about adding another -d.
• Forgetting internal certificates in the inventory. The lab root, clinic certs, farmacias-ca.crt... they expire too. Every certificate that exists should be on some monitor's list, or it will expire on a Sunday.
• Health data in the mix: remember MediNube is a fictional universe; in a real deployment, an incident like this runbook's would additionally involve GDPR obligations (breach assessment and, where applicable, notifying the authority) and compliance review — the technical post-mortem isn't the end of the paperwork.

Exercise 1 — PEM file monitor. Extend the monitor with days_remaining_pem(path: str) -> tuple[str, int] which, given a local PEM (e.g., Clínica Sol's mTLS client certificate issued by the internal CA), returns (subject_name, days_remaining). Add a PEM_FILES list to main() using the same threshold. That way the monitor also covers what can't be checked by connecting to it.

Exercise 2 — ACME challenge decision. For each MediNube case, decide HTTP-01 or DNS-01 and justify it: (a) portal.medinube.example, a public web server; (b) *.clientes.medinube.example, one subdomain per clinic; (c) intranet.medinube.example, publicly resolvable in DNS but only reachable from the corporate VPN.

Exercise 3 — Compromise drill in the lab. With the mini-PKI from 05-01: (1) generate a new key and certificate for portal.medinube.example (portal2.key/portal2.crt) signed by your CA; (2) verify with openssl that portal2.crt's public key differs from portal.crt's (the equivalent of the runbook's step 1); (3) note which step of the real runbook you can't reproduce in the lab as you set it up, and why.

Solution 1.

Note: for the root CA a larger threshold makes sense (e.g., 180 days), because renewing it means redistributing trust to every client, not a simple reload — you could parametrize the threshold per file as an improvement.

Solution 2. (a) HTTP-01: a public server with port 80 reachable, the ideal case, zero extra credentials. (b) DNS-01, mandatory: it's a wildcard, and HTTP-01 can't issue those; it also requires automating against the DNS API and protecting that credential (06-01). (c) DNS-01: the CA can't reach the server over HTTP (VPN-only), but the DNS challenge resolves through public DNS without exposing the intranet. A reasonable alternative you might have proposed: issue it with the lab's internal CA, since only corporate clients that already trust it consume it.

Solution 3.

(3) The irreproducible step is effective revocation: we issued directly with openssl x509 -req, without openssl ca's issuance database, so there's nothing to generate a CRL from; and even if there were, no client checks it yet. It's a perfect illustration of the lesson: revocation only works if revocation infrastructure exists and clients enforce it — otherwise your only real defenses are a fast reissue+deploy and the certificate's short lifetime.

With this lesson, the certificate stops being a concept and becomes an artifact with a life: issue → deploy → monitor → renew → revoke, with ACME/certbot automating the happy path for portal.medinube.example, short lifetimes as the revocation that actually works, CRL/OCSP/stapling as a reminder that telling the whole world is hard, a Python monitor that promises nothing expires as a surprise (alerting through 03-02's signed webhook), the full lifecycle of internal and mTLS client certificates (with a clinic's offboarding solved in three layers), and a compromise runbook rehearsed before it's ever needed.

And with it, all of Module 5 closes out. Take stock of the debt we'd been carrying since 04-01: "knowing whose public key each one is, with verifiable backing." Resolved, threefold: X.509 certificates and CAs (05-01) bind key and identity with signatures MalloryClinic can't forge; TLS (05-02) puts that trust to work on every connection, assembling every primitive from the course into a single handshake; and lifecycle operations (this lesson) guarantee that trust doesn't rot over time. The crack is closed: you encrypt, sign, verify, agree on keys, and you know who you're talking to.

So, are we done? Almost: what's left is bringing it all into the day-to-day of development, which is exactly where flawless cryptography gets undone by mundane details. This lesson's incident already hinted at it: the key didn't fall because of mathematics, it fell because of a git push. Module 6: Cryptography in Real-World Development starts right there — key and secrets management (06-01: where that privkey.pem and certbot's DNS credentials should have lived), and continues with encryption at rest and in transit applied to MediNube's architecture (06-02, including the envelope encryption we previewed in 04-05), signed tokens and JWT for the portal's sessions (06-03), a gallery of common cryptographic mistakes with all the course's _BAD examples gathered together (06-04), and a look at what's coming: post-quantum cryptography (06-05). You already have every piece and all the trust you need; now it's time to use them well when nobody's watching. See you in module 6.