Aurora Libros lives in a cluster with three fixed API replicas. It works fine at four in the afternoon and chokes the day a newspaper reviews La sombra del viento. This lesson makes the platform grow by itself with the load, distribute traffic sensibly, and let you see, with numbers, where the real limit is.

1. Horizontal and vertical scaling
2. What scales and what does not in Aurora Libros
3. The real bottleneck: PostgreSQL connections
4. Read replicas and PgBouncer
5. The prerequisite: a stateless application
6. Load balancing: L4 and L7
7. Internal balancing: Service, kube-proxy, iptables and IPVS
8. Ingress balancing and the Compose/Swarm world
9. Algorithms and session affinity
10. Manual scaling, compared
11. The HorizontalPodAutoscaler in depth
12. The HPA for aurora-api
13. Custom metrics
14. VerticalPodAutoscaler and Cluster Autoscaler
15. Availability: PDBs, spreading and anti-affinity
16. Load testing and reading the limits

Horizontal is the natural one with containers for a simple reason: starting another replica of aurora-api:2.0.0 takes seconds and requires nobody's permission, whereas growing a machine means stopping it. And it brings something vertical never does: redundancy. Three replicas do not just take three times the load, they survive losing one.

The asymmetry in the last row is the reality of almost every platform: the front end scales with a number, and the database demands architecture. And since all the traffic ends up going through it, the system's limit is its limit.

Every PostgreSQL connection is an operating system process with several megabytes of memory of its own. The max_connections parameter defaults to 100, and that number is not a recommendation: it is a wall.

With the DB_POOL_MAX: "10" from your ConfigMap and max_connections = 100:

This is the trap that sinks a lot of people's first autoscaling attempt: the HPA does its job, creates twenty replicas to absorb the peak... and the whole platform goes down, because twenty pools of ten connections flatten PostgreSQL. Scaling the API without looking at the database does not spread the load: it relocates the failure.

There are two solutions and they complement each other.

Read replicas. PostgreSQL replicates by streaming to read-only replicas. aurora-api sends the catalog's SELECTs to a replica and the writes to the primary. It works very well in Aurora Libros because its load is almost entirely reads, but it introduces replication lag: a freshly inserted book can take milliseconds to appear on the replica, so whichever operation has just written must read from the primary.

PgBouncer. A pooler that sits in front and multiplexes: a thousand client connections over twenty real ones.

With transaction and DB_HOST: pgbouncer, aurora-api can scale to fifty replicas while keeping twenty real connections against PostgreSQL. It is, by a wide margin, the intervention with the best effort-to-result ratio in this lesson.

A replica is only interchangeable if it holds nothing the others do not have. The practical rule: if you shut down any Pod and no user notices, the application is stateless.

Keeping the session in memory produces the most baffling failure on a scaled platform: the user logs in (they hit replica 1), browses (replica 2) and appears logged out. It is exactly the problem that having the cache outside the process solves, and that is why aurora-cache is not a performance luxury: it is the requirement that makes replicating the API possible.

The practical difference shows up with persistent connections. An L4 balancer distributes connections, not requests: if a client opens a keep-alive connection and sends a thousand requests, all thousand go to the same Pod. With ordinary HTTP/1.1 clients you barely notice, but with gRPC or HTTP/2 —a single long-lived connection— L4 distribution becomes terribly uneven, and that is why that kind of traffic needs an L7 balancer.

When aurora-api resolves aurora-cache, it gets a virtual IP that does not exist on any network interface: it is a fictitious address that kube-proxy intercepts with kernel rules.

With iptables, distribution is implemented by chaining rules with decreasing probabilities: the first accepts with probability 1/3, the next takes 1/2 of what is left, and the last one picks up the remainder. The result is an even spread, but a random and memoryless one: it does not know how many connections each Pod has or how long each takes to answer. To get least-connections distribution you have to move to IPVS. And for real policies —retries, circuit breakers, latency-based distribution— you need a service mesh like Istio or Linkerd.

The Ingress operates at L7 and its controller talks directly to the Pods, bypassing the Service's virtual IP: it reads the EndpointSlices and balances by itself, which lets it use algorithms and retries kube-proxy does not have.

The Compose case deserves a warning: with --scale api=3, the internal DNS returns all three IPs, but the client decides which one it uses and many HTTP clients cache the first resolution forever. That is why a docker compose up --scale with no Nginx in front distributes far worse than it appears to.

Session affinity ties each client to a fixed backend:

It looks like the easy fix for the in-memory session problem, and it is a bad bargain. It breaks the distribution (a corporate proxy with a thousand employees is one single IP, and all of them land on the same Pod), it ruins scaling (new Pods do not receive existing traffic, so scaling relieves nothing immediately) and it turns every deployment into a loss of sessions. The correct solution is still the one from section 5: get the state out of the process. Aurora Libros does not use session affinity.

A warning that saves you a fright: if an HPA governs the Deployment, a manual kubectl scale lasts until the autoscaler's next cycle, some fifteen seconds. It is not a bug; it is that the desired state is now set by the HPA.

The HPA is just another controller: every 15 seconds it reads the metrics, applies a formula and adjusts spec.replicas.

That 10 % tolerance matters: without it, the HPA would react to every minor fluctuation and the platform would never stop creating and destroying Pods.

Prerequisite: metrics-server installed and requests defined. The HPA's percentage is relative to the requests, not to the limit or the node's capacity. A Deployment with no requests leaves the HPA without a denominator and its status shows up as <unknown>; it is the number one cause of "my HPA does nothing".

The behavior block is what avoids the yo-yo effect: without it, a burst creates replicas, average CPU drops, the HPA destroys them, the load concentrates again and the cycle repeats indefinitely, with Pods being born and dying without ever becoming useful.

The asymmetry is deliberate and worth committing to memory: scaling up is urgent, scaling down is not. Under-scaling costs you lost requests and customers; over-scaling costs a few cents of idle CPU for five minutes. The 300-second stabilization window makes the HPA use the maximum of the recommendations from the last five minutes before it reduces anything.

And maxReplicas: 12 is not a round number: it comes out of the arithmetic in section 3. Twelve replicas times ten connections is 120, already over max_connections. With PgBouncer in front, that ceiling could go up comfortably.

CPU is a poor indicator for an API that spends a lot of time waiting on I/O: aurora-api can sit at 20 % CPU with latency through the roof waiting on PostgreSQL. What really describes its load is requests per second and latency.

That requires an adapter to expose the Prometheus metrics through the custom.metrics.k8s.io API (prometheus-adapter or KEDA). The full circuit is: your API exposes /metrics → Prometheus collects it (you already set that up in 05-06) → the adapter publishes it as a Kubernetes metric → the HPA consumes it. It is worth doing when CPU consumption does not correlate with the perceived load, which is the case for almost any database-bound API.

The VPA adjusts requests and limits instead of the number of replicas. Its great value lies in Off mode, where it only recommends: it tells you, with weeks of data, what values your containers ought to have, so you can adjust the manifests without guessing. In Auto mode it recreates the Pods to apply the new values, which makes it incompatible with an HPA on the same metric: if both act on CPU, they fight.

The Cluster Autoscaler works one level up: when there are Pending Pods because they do not fit on any node, it asks the cloud provider for a new machine; when a node has been underused for a while and its Pods fit elsewhere, it drains it and shuts it down. It is the piece that keeps the HPA from hitting the capacity ceiling, and also the one that introduces the real scaling delay: creating a node takes minutes, not seconds.

Scaling is worth nothing if all twelve replicas are on the same node and that node goes down.

The PDB only covers voluntary disruptions: if a node switches off abruptly, nobody asks permission. And there is a classic trap: minAvailable: 2 with replicas: 2 blocks any drain forever, because not one Pod can be removed without violating it. Always express the PDB in terms of the HPA's minimum, not the current number.

The p95 went from 1,840 ms to 88 ms without anybody touching anything: the HPA did it in a little over a minute. Look at the peak row: 148 % utilization means each Pod is using 1.48 times its CPU request, that is, it is being throttled against its limit; the twelve errors are requests that exceeded the client's timeout.

The last row of the previous block is the most important thing in the lesson: scaling the API only helps when the bottleneck is the API. In every other case, adding replicas makes the problem worse, because it multiplies the pressure on the component that was already saturated.

Warning. A load test against a shared environment can degrade other teams' services and set off security alerts. Always agree the window, the traffic source and the thresholds with the infrastructure and security officers, and never fire load at systems you do not control.

• Scaling the API without looking at the database. The HPA creates twenty replicas, PostgreSQL rejects connections and the outage is total. Work out replicas × pool before setting maxReplicas.
• An HPA with no requests. The targets show up as <unknown> and it never scales. The percentage is relative to the request.
• No metrics-server. The HPA has nothing to read from. kubectl top pods is the thirty-second check.
• Aggressive scaleDown. With no stabilization window, the platform oscillates and no replica ever becomes useful.
• Session affinity as a patch. It breaks the distribution and does not fix the design. Get the state out of the process.
• minAvailable equal to replicas in a PDB. It blocks every cluster maintenance operation.
• Load-testing an API with a warm cache. You are measuring Redis, not your system. Flush the cache or vary the keys.
• Tip: measure before and after with the same percentiles. The p95 and the p99 tell the story; the average hides it.
• Tip: set maxReplicas to a number the cluster can actually host. Otherwise you get Pending Pods and autoscaling that looks like it works while doing nothing.

Exercise 1. Calculate and demonstrate the connection limit: find out aurora-db's max_connections, work out how many API replicas fit with DB_POOL_MAX: 10 and trigger the failure by scaling beyond that number.

Exercise 2. Set up the HPA over aurora-api, generate load with k6 and document the table of latencies and replicas before, during and after. Explain why coming down takes far longer than going up.

Exercise 3. Check that the replicas are spread out and that the platform survives a maintenance window: apply a PDB and topologySpreadConstraints, drain a node and observe what Kubernetes does.

Solution 1.

Three replicas times ten connections are exactly the 30 that are open: the ConfigMap's DB_POOL_MAX is not theoretical, each Pod opens its full pool at startup even if it does not use it. The theoretical ceiling is (100 − 3 reserved) / 10 = 9 replicas.

Nine Running Pods out of twelve, with the other three in CrashLoopBackOff: their pools never managed to open. But the serious part is not those three, it is the effect on the ones that did start, which begin to see intermittent errors as they renew connections.

Twelve replicas and twenty real connections: PgBouncer has completely decoupled the API's scaling from the database's limit. That is why maxReplicas stops being a connection calculation and can be set by compute capacity.

The underlying lesson is one of method: a system's limit is almost never where you are scaling. Here you were scaling the API and the thing that broke was PostgreSQL, a service you had not even touched. Before raising maxReplicas, you always have to ask which shared resource gets multiplied by every replica.

Solution 2.

The two right-hand columns sum up the result: the p95 fell from 1,840 ms to 88 ms, twenty times over, without anybody touching a thing, and the twelve errors in the first minute are the price of having started with three replicas.

The timings reveal the asymmetry of behavior. Going up happened in two 30-second jumps —stabilizationWindowSeconds: 0 and the at-most-double policy— and within 75 seconds the platform had stabilized. Coming down started five minutes after the load finished, because scaleDown.stabilizationWindowSeconds: 300 forces the HPA to use the maximum recommendation in that window; and then it came down slowly, 25 % per minute.

That slowness is intentional, and the justification is economic before it is technical. Coming down fast carries an asymmetric risk: if the load rises again thirty seconds later —which is very common, because traffic arrives in bursts—, you are back to three replicas choking and another minute of bad latencies. Keeping nine replicas five minutes longer than necessary costs cents; falling short on the second peak costs customers.

One methodological detail: the 247 iterations per second in the report are the sustained throughput, not the peak. When measuring, that number alongside the p95 matters far more than the instantaneous maximum, which is almost always achieved at the cost of unacceptable latencies.

Solution 3.

A perfect spread thanks to topologySpreadConstraints with maxSkew: 1. Without that constraint, the scheduler only looks at free resources and is perfectly capable of putting all six on the same node if they fit there.

The detail to watch is that the evictions happened one at a time and with waiting in between. A drain does not evict all the node's Pods at once: it asks the PDB for permission before each one, and if evicting the next would leave fewer than two available, it waits for the replacement to be ready on another node. That is why ALLOWED DISRUPTIONS keeps dropping during the operation.

There is the trap from section 15, live: with minAvailable: 6 and six replicas, no eviction is permissible and the drain fails indefinitely. During an overnight cluster upgrade, that PDB would leave the maintenance hanging with nobody understanding why.

The rule that follows: express the PDB in terms of the HPA's minimum, never the current number of replicas. With minReplicas: 3, a minAvailable: 2 (or better, maxUnavailable: 1) gives you room both at idle and at twelve replicas.

And the limitation this exercise does not cover: all of this protects against voluntary disruptions. If aurora-worker were switched off abruptly, its three Pods would vanish without the PDB saying a word, and only the spread across nodes —which you have guaranteed— would stop Aurora Libros from being left without a single live replica.

The platform now grows and shrinks by itself. You can tell horizontal scaling from vertical and you know which Aurora Libros services accept each: the API and the front end replicate freely because they hold no state, and the database does not, for two reasons you have measured —the ReadWriteOnce PVC and, above all, max_connections. That was the most valuable finding: the system's limit was not where you were scaling. Twelve replicas times ten connections flattened PostgreSQL, and PgBouncer in transaction mode solved it, leaving twelve replicas over twenty real connections, with read replicas as the other half of the answer.

You understand why the shared cache in aurora-cache is not a luxury but the requirement that makes replicating the API possible, and why session affinity is a patch that breaks the distribution instead of fixing the design. You know what separates L4 from L7, how kube-proxy implements the virtual IP with probabilistic iptables rules —and what IPVS gains you—, how the Ingress balances by talking directly to the EndpointSlices, and why a docker compose up --scale distributes worse than it appears to.

You have set up the HorizontalPodAutoscaler with its formula, its 10 % tolerance and a deliberately asymmetric behavior: up in seconds because the user is waiting, down over five minutes because traffic arrives in bursts. And you have tested it for real with k6: the p95 fell from 1,840 ms to 88 ms in a little over a minute while the HPA took the replicas from 3 to 9, and it went back to 3 on its own once it was over. You know about custom metrics as the next step when CPU does not describe the load, the VPA in recommendation mode and the Cluster Autoscaler with its delay measured in minutes. And you have secured availability with PodDisruptionBudget and topologySpreadConstraints, watching the drain evict one Pod at a time and seeing first-hand how a badly calibrated PDB blocks a maintenance window forever. Along with the table that translates each symptom into its real bottleneck, because scaling the API only helps when the problem is the API.

One last piece remains. Aurora Libros can take the load, but every time you release a new version there is still a delicate moment. In the next lesson, Deployment Strategies and Rollback, you will see how the old version and the new one coexist: rolling updates with maxSurge and maxUnavailable, blue-green, canary and shadow, with their cost and their risk; you will rehearse a failed deployment of aurora-api:2.1.0 that never passes readiness in order to watch it stop by itself and then undo it; and you will face the problem no strategy solves on its own, that of database migrations.