One Tuesday morning, the Aroma Store API starts answering in four seconds. There is no new deployment, no error in the logs, the database is not down. Looking at the traffic, the culprit appears: a single IP address making 400 requests per second to GET /v1/coffees?limit=100, walking the entire catalogue every few seconds. This is not a sophisticated attack: it is somebody copying the catalogue or — just as likely — a client with a badly written useEffect retrying in a loop without waiting.

The API is authenticated, authorised and hardened. And even so, anyone can bring it down simply by calling it a lot. This lesson closes the last open row of the threat model from 04-02: API4, unrestricted resource consumption. We will see why every public API needs limits, the four classic algorithms with their trade-offs, what is used as the grouping key, the 429 response with its headers, the implementation in the project with express-rate-limit and Redis, the other availability defences, and what a well-behaved client should do when it is told to stop.

Warning. Rate limiting is an availability defence and, badly tuned, it can deny service to legitimate users or let an attack through. The values in this lesson are a reasonable starting point for a fictional case; a real deployment must size them with its own data and review them with a security professional.

1. Why every public API needs limits
2. Rate limiting, throttling and quotas
3. The four algorithms
4. An annotated token bucket implementation
5. The key: what you count against
6. The IP problem: NAT, proxies and trust proxy
7. Limits per endpoint
8. Aroma Store's tiers
9. The 429 response and its headers
10. Implementation: src/middleware/rate-limit.js
11. In-memory store versus Redis
12. Other availability defences
13. 503, Retry-After and service_unavailable
14. What a well-behaved client does
15. How limits are communicated in the documentation
16. Where rate limiting lives in production
17. Testing the limit with node:test

There are five reasons and they are worth distinguishing, because each one calls for a different limit:

The third one deserves an important nuance: most abusive traffic is not malicious. It is a developer who has not noticed that their retry has no wait, a mobile app that refreshes on every scroll, a cron job that overlaps with itself. That changes the design of the response: the 429 has to be pedagogical, say when to come back and be easy to handle programmatically, because its usual recipient is somebody who wants to fix it.

And there is a cross-cutting reason that encompasses them all: protecting the database. Your API can scale to more instances; Aroma Store's SQLite, or the PostgreSQL that comes after it, not so much. Rate limiting is what stops a traffic spike turning into an avalanche of queries that leaves every other consumer without service.

Three terms used as synonyms which are not:

• Rate limiting: "100 requests per minute". The 101st is rejected.
• Throttling: "at most 10 concurrent requests". The 11th waits in a queue. It is friendlier to the client but consumes your resources while it waits, and it can degrade the whole system if the queue grows: that is why it always needs a queue cap and a timeout.
• Quota: "50,000 requests a month on the free plan". It is a commercial concept more than a technical one; it shows up on the invoice, not in every response.

Aroma Store uses rate limiting as its main mechanism, with a daily quota for the partner SwiftShip. Throttling appears naturally in one place: the database connection pool, which already limits real concurrency.

You count how many requests there are in the current minute; when the minute changes, the counter goes back to zero.

It is trivial to implement and very cheap: one integer per key. Its problem is the boundary effect:

Twice the limit in an instant. With generous limits that is acceptable; for protecting a login, it is not.

You store the timestamp of every request and count those from the last 60 seconds, exactly. It is 100% precise and has no boundary effect. The cost is memory: with a limit of 1,000 per minute and 50,000 active keys, that is 50 million timestamps.

The practical compromise. Two counters are kept — the current window's and the previous one's — and the sliding value is estimated by interpolation:

With two numbers per key the boundary effect is practically eliminated. This is what most serious implementations use, Cloudflare included.

A bucket with capacity C that is refilled at R tokens per second. Every request consumes a token; if there is none, it is rejected.

Its charm is that it allows bursts in a controlled way: a client that has been quiet accumulates tokens up to C and can spend them all at once, but its sustained rate never exceeds R. That fits an API's real traffic very well: an app opening a screen makes eight requests in a row and then goes quiet for a minute, and we do not want to punish it.

Requests enter a queue that drains at a constant rate. It smooths outgoing traffic completely, but it allows no bursts and it adds latency: it is throttling, not rate limiting. It is used more to regulate output towards a downstream system (calls to the payment gateway, for instance) than for input.

Aroma Store's choice: a token bucket for the general limit — because the SPA's and Aroma Mobile's traffic is naturally bursty — and a strict sliding window for POST /v1/sessions, where we want no bursts at all.

Four details that make this work and that are almost always got wrong:

The lazy refill (step 1) is the key idea. One bucket per client each with its own setInterval would be unworkable with thousands of keys; computing the tokens when they are queried gives the same result at constant cost.

Math.min(capacity, ...) stops a client that has been idle for a day accumulating 86,400 tokens and pulling the entire database down in one go. The maximum accumulable amount is the capacity, and that is why the capacity is the burst size allowed.

The parameterisable cost (step 2) opens the door to the cost-per-query idea from section 12: an expensive search can consume five tokens instead of one.

The clean-up (step 4) is an availability requirement, not an optimisation: without it, an attacker rotating IP addresses grows the Map until the process runs out of memory. And unref() stops the timer keeping the process alive at shutdown, something that would break the graceful shutdown from 03-07 and the tests from 03-08.

A limit is always applied to a group. Choosing the key badly is the most expensive mistake:

Aroma Store's selection rule, in order of preference:

Case 1 deserves an explanation: if CataBox has a bug and hammers the API on behalf of 500 users, limiting per user stops nothing — those are 500 different keys. The composite key also allows an aggregate cap per client_id, which is what really protects you.

Limiting by IP has three serious problems you need to know about before settling on a number:

NAT. An office, a university or a mobile operator can present hundreds or thousands of users behind a single public IP. A limit of 60 per minute per IP leaves an entire company without service if its employees use the shop. That is why the anonymous limit must be generous, and the strict one reserved for authenticated traffic or for specific operations.

IPv6. An attacker with a /64 prefix has trillions of addresses at their disposal. Limiting by individual IPv6 address is useless: you have to group by prefix (usually /64).

Proxies. This is the one that breaks the code. If your API sits behind a load balancer or a CDN — and in production it will — req.ip is the proxy's IP, not the client's. All the traffic shares one key and the first user exhausts everybody's limit.

The real IP arrives in X-Forwarded-For, a header containing the chain of hops:

And here is the trap: anyone can write that header. An attacker sends X-Forwarded-For: 1.2.3.4 and, if you trust it blindly, changes identity on every request and defeats the limit.

The number must match exactly the quantity of proxies in front. If you put 1 and there are two (CDN + load balancer), you get the first proxy's IP instead of the client's; if you put 2 and there is one, you get whatever IP the client fancies. Check it in the real environment before trusting it:

Remember too that this same setting affects req.protocol and therefore the Secure cookies and the HTTPS redirects from 04-02: it is a configuration with consequences beyond rate limiting.

A uniform global limit is always a bad compromise: too lax for the login, too strict for the catalogue. Limits are tuned to the cost and risk of each operation.

The login case deserves an important clarification: you have to limit by two keys at once.

• By IP: stops whoever tries thousands of passwords against many accounts from one place.
• By target email address: stops distributed credential stuffing, where each attempt against [email protected] comes from a different IP (a botnet). The per-IP limit sees nothing; the per-account limit does.

And a nuance that gets forgotten: only failed attempts are counted. If you also count the successful ones, a legitimate user who logs in and out several times ends up locked out. express-rate-limit supports this with skipSuccessfulRequests: true.

Beware also of the side effect of the per-account limit: if you lock the account for 15 minutes after five failures, an attacker can deny service to a specific user by failing on purpose. Mitigations: a progressive delay instead of a hard lockout, a short window, and not locking if the request comes from an IP that has already logged into that account successfully before.

Notes on these numbers, which matter more than the numbers themselves:

• The values are a starting point. The correct procedure is to measure the 99th percentile of real legitimate use and set the limit comfortably above it. A limit that cuts off real users costs more than one that is too generous.
• Start in observation mode. express-rate-limit lets you count without blocking (a skip that only logs). Two weeks of data say far more than any estimate.
• The tier is derived from the token, so role-specific rate limiting must go after authentication. The global anonymous limit, by contrast, goes before: it has to protect you even from whoever sends junk tokens.

Four design decisions behind that response:

Retry-After always. Without it, the polite client does not know how long to wait and the impolite one retries immediately, making the problem worse. It accepts seconds or an HTTP date; seconds are easier and do not depend on the client's clock.

The informational headers on every response, not just on the 429. Their value lies in letting the client slow down before it hits the wall: if it sees Remaining: 3, it spaces its requests out. Sending them only on failure wastes the mechanism.

The Aroma- prefix, consistent with the contract (never X-, as we decided in 02-05). There is an IETF draft standardising RateLimit-Limit, RateLimit-Remaining and RateLimit-Reset — and a combined form RateLimit: limit=60, remaining=0, reset=37. When it is published as an RFC it will be worth emitting both for a while and documenting the migration; meanwhile, our own prefix avoids collisions with whatever intermediaries may do.

The catalogue's rate_limit_exceeded code, with the same error format as everything else. A 429 that returns plain text breaks clients that parse errors.

And a CORS warning that is expensive to learn the hard way: the SPA cannot read any of those headers from JavaScript unless they are declared in Access-Control-Expose-Headers. It is exactly the kind of detail that makes a well-built mechanism worthless, and we resolve it in 04-05.

And the error factory missing from src/errors/api-error.js (the rate_limit_exceeded code was already in the catalogue from 02-04; we are only adding its constructor):

Why position 6, and neither earlier nor later:

• After helmet and CORS, so that the 429 carries the security headers and, above all, the CORS ones: otherwise the SPA receives an opaque network error instead of a readable 429.
• Before the JSON parser. If the limiter came afterwards, your server would be parsing and validating 100 kB of JSON from requests it is going to reject anyway. Rejecting cheaply is half of an availability defence.
• Before the routes, obviously, because it protects all of them.
• After the logging (position 5, 04-07), so that the 429s are recorded: otherwise the attack is invisible in the logs at precisely the moment you most need to see it.

There is a deliberate exception to "before the parser": loginLimit needs req.body.email for its key, so it is registered inside the route, after the parser:

The general rule that sums up the ordering: the anonymous limit goes before authenticating; the per-user limit goes after.

express-rate-limit's default store is a Map inside the process. With a single instance it works. With three, this happens:

The effective limit is multiplied by the number of instances, and it is erratic on top of that: it depends on how the load balancer distributes traffic. Worse still, a deployment restarts the processes and wipes every counter, so an attacker only has to wait for your next deployment.

What to do if Redis goes down is an explicit design decision, not a detail:

• Fail-open (let requests through): the API keeps working without limits. It prioritises availability; it is most implementations' default and the one Aroma Store chooses for the general limit.
• Fail-closed (reject): safer, but it turns a Redis outage into a total outage.

A reasonable compromise: fail-open on the general limit, fail-closed on the login, where the brute-force risk outweighs availability. And in both cases, an alert: losing your rate limiting and not finding out is worse than either option.

Rate limiting does not reach everything. The complete picture:

Server timeouts. Node accepts connections that send nothing, and a Slowloris attack uses them to exhaust the pool:

keepAliveTimeout is a classic source of random 502s: if your server closes the reused connection before the load balancer does, the load balancer sends a request over a connection that is closing. The rule is that Node's must be greater than the proxy's.

Cost per query. Not all requests cost the same, and the token bucket from section 4 already accepts a cost:

It is fairer than counting requests and it is what mature APIs do (GitHub calls them "points"). It needs documenting well, because a limit in abstract units is harder to understand.

Circuit breaker. If the payment gateway is down, carrying on calling it with a 30-second timeout consumes your processes and delays the other side's recovery. The pattern has three states: closed (everything passes), open (after N failures, requests are rejected immediately without calling) and half-open (after a while, one probe request is let through). It is what turns "payments are down" into a fast 503 rather than a general outage.

Backpressure. When the incoming work exceeds what you can process, the correct response is to reject (503), not to queue indefinitely. An unbounded queue only swaps a fast outage for a slow one with all your memory consumed.

429 and 503 get confused and they are not the same:

Note the traceId: this is a 5xx, so the contract from 03-07 requires it. On the 429, which is a 4xx, it does not appear.

When 503 is used in Aroma Store: scheduled maintenance, the database unavailable, an open circuit breaker towards the payment gateway, or overload detected by backpressure. And always with Retry-After, even if it is an estimate: without it, every client retries at the same moment and the effect is the thundering herd that stops the service recovering.

On the other side of the contract, this is what the consumer should do:

Why jitter is not optional. If a thousand clients receive a 503 in the same second and they all wait exactly 1, 2, 4 and 8 seconds, all thousand come back at once, four times in a row. The service, which was recovering, falls over again with each wave. Randomness spreads the return out over time and it is the difference between recovering in a minute and not recovering at all. It is a distributed systems failure so common that it has its own name: the thundering herd.

And the three complementary rules:

• Slow down before you hit the wall: if Aroma-RateLimit-Remaining drops below a threshold, space the requests out rather than waiting for the 429.
• Do not retry client 4xxs. A 400 or a 422 is going to fail the same way the second time. Only 429 and 5xx.
• Cache. The best retry is the request that is never made: if the catalogue has not changed, do not ask for it again. That is exactly the subject of 04-06.

An undocumented limit is an intermittent failure from the consumer's point of view. The documentation must answer five questions, and they are best kept on a single page:

1. What the limits are, per tier and per endpoint, in a table like the one in section 8.
2. How to know how much is left: the Aroma-RateLimit-* headers, with a real example.
3. What happens when they are exceeded: the complete 429, with its body and its Retry-After.
4. What the client should do: the backoff pattern, with copyable code like the one in section 14.
5. How to ask for more: who to write to, with what justification and on what timescale.

In openapi.yaml the 429 response is declared on the affected operations, with its headers:

The Express middleware works, but it has a structural limitation: to reject the request, you have already received it. Your server has accepted the TCP connection, negotiated TLS and run middleware. Under a genuine attack, that is already too much work.

That is why, in production, rate limiting usually lives in layers:

The layers complement each other, they do not replace each other. The CDN stops the DDoS your process would never survive; only the application knows that a partner may make 1,000 per minute and a customer 300. Keeping the limit in the application also has a practical advantage: it is the defence that still exists if somebody deploys the API without the CDN in front, or if an attacker finds the origin IP and calls it directly. We will look at gateways and their portals in 05-06.

Three things this test teaches:

• The skip in the test environment is necessary but dangerous. Necessary because, without it, the 03-08 suite would start failing randomly as soon as it made more than 60 requests. Dangerous because it means the real limiters are not tested: hence the minimal app with the same handler.
• The contract is tested, not the implementation: code, body format, headers present and headers absent (X-RateLimit-* must not appear).
• The third test is deliberately loose and its comment explains why: without trust proxy configured, Supertest always looks like it comes from 127.0.0.1. Testing the per-IP separation requires configuring trust proxy in the test app; it is a reminder that testing rate limiting by IP forces you to replicate the network topology.

To test the time-based limit without actually waiting, use node:test's fake clock:

Putting the limiter after the JSON parser. You parse 100 kB of requests you are going to reject. Reject as early as possible.

Putting the limiter before CORS. The browser receives an opaque network error instead of a 429, and the SPA developer loses an afternoon.

Using the in-memory store with several instances. The limit is multiplied by the number of processes and is wiped on every deployment.

Trusting X-Forwarded-For without configuring trust proxy correctly. Either you limit everybody as one, or the attacker changes identity at will.

Counting successful logins. A legitimate user who logs in and out ends up locked out.

Locking accounts on failed attempts and nothing more. It becomes a way of denying service to a specific user. A progressive delay is better than a hard lockout.

Not sending Retry-After. The polite client does not know how long to wait and the impolite one waits for nothing.

Sending the allowance headers only on the 429. Their value lies in letting the client slow down before it hits the wall.

Forgetting Access-Control-Expose-Headers. The SPA cannot read any of our own headers and the entire mechanism is invisible to it (04-05).

Tip: deploy in observation mode first. Count without blocking for two weeks, look at the real 99th percentile and set the limit comfortably above it.

Tip: log the 429s with their key. Knowing that they all come from oauth:catabox changes the diagnosis completely (04-07).

Tip: exempt your own monitoring. There is nothing worse than your health check burning through the allowance and firing false alerts.

For each situation, choose the algorithm (fixed window, sliding window, token bucket) and the key, and justify it:

1. Protecting POST /v1/sessions from brute force.
2. Letting the SPA load a screen that makes 8 requests in a row, without punishing it.
3. Limiting CataBox globally, even though it acts on behalf of 500 different users.
4. Limiting GET /v1/coffees?q= because each search costs 200 ms of CPU.

Write a function downloadCatalogue(pages) that walks GET /v1/coffees?limit=100&offset=N for pages pages, respecting the rate limiting: it must slow down pre-emptively when Aroma-RateLimit-Remaining is low, respect Retry-After on a 429 and not retry more than three times per page.

After deploying the rate limiting, support receives these complaints on the same day. Diagnose each one and propose the fix:

• (a) "From my company's office, the website stops working in the mornings. From home it is fine."
• (b) "My mobile app gets a 429 when the home screen opens, but only the first time after it has been closed for a while." (note: the app makes 8 requests on start-up)
• (c) "Our integration script gets random 429s even though we make 50 requests per minute and the limit is 300."
• (d) "The SPA shows a 'network error' instead of the limit message."

Solution 1

Solution 2

The essentials: the pre-emptive slowdown means the 429 almost never happens, Retry-After is respected when it does, the jitter avoids synchronising with other clients, there is a cap on attempts, and the loop exits once all the items have been fetched according to total.

Solution 3

(a) NAT. The whole office comes out through one public IP. With 60/min for anonymous traffic, twenty employees browsing exhaust it. Fixes: raise the anonymous limit; authenticate as early as possible so as to move to the per-user limit (300/min); and, if the SPA loads the catalogue with no session, take advantage of HTTP caching (04-06) so that most of those requests never arrive at all.

(b) A boundary effect with a fixed window. The app makes 8 requests at once. If the limit was implemented with a strict fixed window, a burst after a period of inactivity can land right on the boundary. Fix: a token bucket with capacity ≥ 20 and rate 5/s, which is precisely the use case it exists for. If the problem were one of scale rather than burstiness, the alternative solution is reducing the 8 calls to 1 with expand, as we saw in 04-01.

(c) The in-memory store with several instances... the other way round. With memory and N instances the limit multiplies, so 50/min would never produce a 429. The fact that it does points to another cause: the key is badly chosen and groups several consumers together. Most likely the script does not send Authorization, falls into the ip: branch and shares its IP with other processes belonging to the same customer. Fix: authenticate the script (Client Credentials, 04-03) so that its key is its client_id. Another possibility: a badly configured trust proxy makes every client share the load balancer's IP.

(d) Access-Control-Expose-Headers is missing, or the limiter is before CORS. If the 429 is emitted without the Access-Control-Allow-Origin headers, the browser blocks the response and JavaScript only sees a generic network failure. Fixes: register cors before globalLimit in src/app.js (position 4 against position 6) and expose Retry-After and Aroma-RateLimit-* in Access-Control-Expose-Headers (04-05).

Availability is the only property of an API that anyone can destroy without exploiting a single vulnerability: calling it a lot is enough. You have seen why every public API needs limits — abuse, scraping, clients with loops, cost, fairness and protecting the database — the difference between rate limiting, throttling and quotas, and the four algorithms with their real trade-off between precision, memory and burst tolerance, with a token bucket implemented and annotated that includes the lazy refill and the clean-up that almost nobody writes. You know what you count against and why the IP is a problematic key, between NAT, IPv6 and proxies, with the exact trust proxy configuration; you have Aroma Store's limits per endpoint and per tier, and the complete 429 response with Retry-After and the Aroma-RateLimit-* headers. The project now has src/middleware/rate-limit.js with express-rate-limit, registered at position 6 of src/app.js — after helmet and CORS, before the JSON parser — with a Redis store for multiple instances and an explicit decision about what to do if Redis goes down. And you have added timeouts, cost per query, a circuit breaker and the 503 with service_unavailable, along with the client with exponential backoff and jitter that avoids the thundering herd.

There is a detail that has come up three times and can no longer be postponed: the SPA cannot read any of the headers we have just designed. In 04-05, CORS and Security Policies, we will start with the why: the browser's same-origin policy, what exactly an origin is and why curl and Aroma Mobile are unaffected. We will look at simple requests versus preflight with the complete raw OPTIONS exchange, all the protocol's headers — including Access-Control-Expose-Headers, which is the one that solves this problem — Aroma Store's real configuration with an allowlist per environment, why * and Allow-Credentials are incompatible, the table of console errors with their cause and their solution, the classic preflight that returns a 401 because authentication ran before CORS, and why we use Authorization: Bearer rather than cookies — which is what makes us immune to CSRF.