Reservalia already has an immutable artifact, a cd.yml that deploys on its own and three environments that come out of the same Terraform module. What remains is the question we have been dodging: at the exact moment the new version replaces the old one, how is that handover done? Today Reservalia lets ECS replace tasks with no particular criterion and crosses its fingers. There are at least six ways of doing it, and choosing badly is the difference between an invisible deployment and a service outage at eleven in the morning. In this lesson we look at all six with their cost, their risk, their timing and how easy they make going back; we build a fine-grained rolling update and an ALB-weight canary at Reservalia; we understand why health checks hold all of this up; and we accept the requirement they all share: two versions of the software are going to coexist, and they have to understand each other.

1. The four axes for comparing a strategy
2. Recreate: off and on again
3. Rolling update: replacing in batches
4. Blue-green: two complete environments and a switch
5. Canary: exposing a fraction of the traffic
6. A/B testing: the one that is not a technical strategy
7. Shadow: duplicated traffic with no consequences
8. Comparison table of all six
9. Rolling update in ECS with real numbers
10. An ALB-weight canary at 10%
11. Health checks: liveness, readiness and the role of /health
12. Backward compatibility: two versions coexisting
13. What Reservalia chooses for prod
14. Common Mistakes and Tips
15. Exercises
16. Conclusion

No strategy is "the right one". Each trades off four quantities, and it is worth naming them before we look at the first one:

• Infrastructure cost: how much extra capacity is needed during the handover. Doubling a production environment for twenty minutes costs real money.
• Exposure risk: what percentage of users suffers the failure if the new version is defective. It is the variable that moves the change failure rate of the DORA metrics.
• Deployment time: how long the full handover takes. A two-hour deployment does not get done five times a week.
• Complexity of going back: how long it takes and how much thinking is required to return to the previous version. It is the time to restore, which at Reservalia stands at 68 minutes and must come down to under 10.

There is an implicit fifth axis: operational complexity. A strategy nobody on the team understands at three in the morning is a bad strategy however good its numbers are.

What it involves. All the instances of the old version are stopped and then the new ones are started. It is what Reservalia's "Friday ritual" did with its SFTP.

When to choose it. When the outage is acceptable (internal tools, nightly batch processes) or when it is unavoidable: if the new version is incompatible with the old one and they cannot coexist for even a second, recreate is the only honest option. It is the default strategy of any system with no load balancer in front.

What it involves. Instances are replaced in groups: a batch of the new version is started, you wait for it to pass the health check, the same number of old ones are withdrawn, and this repeats until you are done. During the process the two versions coexist.

When to choose it. It is the reasonable default for stateless services behind a load balancer. It is what ECS does natively and what Reservalia will use as its baseline.

What it involves. A complete green environment is stood up with the new version, alongside the blue one that carries on serving traffic. Green is tested in isolation and, once it convinces you, the load balancer's routing is switched over all at once. Blue is kept running for a while in case you need to go back.

When to choose it. When time to restore is the absolute priority and the budget allows doubling capacity. Watch out for a detail that is always forgotten: the database is not duplicated, so schema compatibility remains mandatory.

What it involves. The new version is deployed to a small group and a reduced percentage of the traffic is sent to it — the canary in the mine. Error and latency metrics are observed for a few minutes and, if they behave, the weight is raised in stages: 10% → 50% → 100%. If they degrade, it is dropped to 0%.

When to choose it. For medium-to-high-risk changes in services with enough traffic for the metrics to be meaningful. With 9,000 appointments a month, a Reservalia canary at 10% sees around 30 booking requests a day: you have to give it time or combine it with total-traffic metrics, not just those of the critical endpoint.

It is constantly confused with the canary because the mechanism looks similar: two versions serving at once and a split of users. The difference lies in which question each one answers.

An A/B test is not a deployment mechanism: it is normally implemented on top of feature flags within a single deployed version, which is precisely the subject of the next lesson. Confusing them leads to expensive mistakes, such as aborting a technically correct deployment because a variant converts worse, or keeping two versions deployed for weeks "in order to measure".

What it involves. Real traffic is copied to the new version, whose responses are discarded: users keep receiving those of the old version. It serves to validate performance and correctness under real load with zero risk to the user.

The catch: if the mirrored version writes to the database, sends appointment confirmation emails or charges money, the "zero risk" evaporates instantly. Shadow requires isolating side effects, which is why it only pays off for high-risk changes: rewrites of a calculation engine, technology migrations, deep refactorings of calculateSlots.

ECS implements the rolling update with two percentages of desired_count. In prod, Reservalia has 4 tasks of the reservalia-api service:

• minimumHealthyPercent: the minimum number of healthy tasks that must exist at all times, as a percentage of desired_count.
• maximumPercent: the maximum number of simultaneous tasks, counting both versions.

With desired_count = 4, those two numbers translate as follows:

Reservalia uses 100 / 200 in prod and 50 / 100 in dev, where a dip in capacity bothers nobody and the saving does matter. The sequence ECS runs with 100/200 is: start 4 v2 tasks → wait until all 4 pass the target group's health check → register them with the ALB → deregister the 4 v1 tasks and wait for connection draining → stop them.

While the deployment progresses, two entries appear: the PRIMARY (new version) and the ACTIVE (the previous one, being retired). When only PRIMARY remains, the handover is over; that is exactly what aws ecs wait services-stable waits for in cd.yml.

One more parameter, and it is the one that prevents half the scares:

With the circuit breaker, if the new version's tasks repeatedly fail to start, ECS aborts the deployment and restores the previous task definition with no human intervention. It is the cheapest automatic rollback available on this platform.

For delicate changes — touching calculateSlots, for instance — Reservalia adds a canary mode. The key piece is that an ALB can split traffic between several target groups with weights.

Two important details:

• TG_STABLE and TG_CANARY are two target groups, served by two ECS services (reservalia-api and reservalia-api-canary) that run the same artifact or a different one: it is the weight, not the deployment, that decides the exposure.
• TargetGroupStickinessConfig makes a user who landed on the canary stay there for an hour. Without it, the same user would jump between versions request by request, with incoherent results if the change affects the interface.

Staged promotion is automated as a job with observation pauses:

check-metrics.sh queries CloudWatch and returns a non-zero exit code if the canary behaves worse than the stable group; a failing step halts the pipeline with the canary still at 10%, that is, with 90% of users untouched. How to set those thresholds sensibly is the subject of lesson 03-06.

All the strategies above depend on one binary answer: is this new instance ready to receive traffic? The health check is what gives it, and there are two types that often get confused.

In 03-02, the smoke test called a single /health that did both things at once; now we separate them, because mixing them is exactly where the problem comes from. At Reservalia, /health becomes the liveness check: it responds 200 with a minimal body and does not touch the database. It has to be cheap because it is called every few seconds and because, if it checked PostgreSQL, a database outage would send every container into a restart loop, turning a degradation into a total outage.

1. The readiness check does verify the connection to the PostgreSQL pool: a task that cannot query the database must not receive appointments.
2. It also checks that no migrations are pending; a task with an out-of-date schema responds 503 instead of breaking real requests.
3. The 503 code is what stops the ALB registering it. Returning 200 with a body that says "error" is useless: the load balancer looks at the code, not the text.

Why a health check that returns 200 without checking anything is worse than none at all. Because it lies with authority. The rolling update asks "is it ready?", the endpoint says yes before the application can serve anything, ECS withdraws the old version's tasks and the service ends up served by instances that return errors. With no health check, at least the operator knows they have no information; with a fake one, the system makes destructive decisions based on an empty answer. The rule: a readiness check must be able to say no. If you have never seen your /health/ready return 503, it probably checks nothing.

Every strategy except recreate has the same consequence: for a while, v1 and v2 serve the same users on top of the same database. If they are not compatible, the elegant strategy turns into an elegant incident.

The pattern that resolves almost all the "no" cases is expand and contract: first you deploy a change that adds the new thing without removing the old one (expand), then you migrate the consumers, and only in a third deployment do you remove the old one (contract). Renaming duration thus becomes three safe steps: return both fields, update the web app, stop returning duration.

The same applies to data, and there it is more serious, because a schema migration cannot be reverted by changing a weight on the ALB. Adding a column is compatible; deleting or renaming one breaks the version still running. That territory has a lesson of its own: 04-06, Databases in the Pipeline: Safe Migrations.

Marta settles it like this: rolling update 100/200 with the circuit breaker as the default strategy, and an ALB-weight canary for changes flagged as risky (those touching calculateSlots, price calculation or the payment flow).

The reasons, in the order she gave them: the rolling update costs no permanent infrastructure and ECS does it natively, so it is the option the team can operate without ceremony; the circuit breaker already covers the most frequent failure, which is a version that does not even start; blue-green is ruled out because sustaining a duplicate of prod does not fit the budget of a company with 340 paying customers; and the canary is reserved for what really deserves it, because its observation waits lengthen the deployment and, at the current traffic level, require long windows before the numbers mean anything. Shadow is noted down for the day the scheduling engine has to be rewritten.

Mistake 1: choosing canary because it is fashionable. A canary with no reliable metrics and no defined thresholds is a slow, expensive rolling update with a false sense of safety. Observability first, canary afterwards. Mistake 2: believing blue-green removes data risk. The two environments share the database: if the green version applied a destructive migration, going back to blue does not bring the data back.

Mistake 3: using minimumHealthyPercent = 0 "so it goes faster". It is recreate under another name, and in prod it means a service outage. Mistake 4: a readiness check that calls every dependency. If /health/ready queries an external payments service, a failure of that service takes every healthy task out of the load balancer; check only what is essential in order to serve.

Mistake 5: a canary with no session stickiness. The user alternates between versions request by request and sees incoherent behaviour; the resulting bug reports are irreproducible.

Tip 1: rehearse going back. A strategy with an untested, theoretical rollback has a 68-minute rollback. Tip 2: write the strategy into the infrastructure code, not into Nuria's memory; the percentages live in infra/modules/environment/. Tip 3: when in doubt between two strategies, choose the one the team understands better at three in the morning.

Reservalia has to deploy a change that rewrites calculateSlots with a new algorithm, faster but with a risk of returning incorrect slots in rare cases. The result is visible to the user and affects real bookings. Choose a strategy, justify it against the other five and describe what you would measure in order to decide whether to promote.

The reservalia-api service in staging has desired_count = 2, minimumHealthyPercent = 100 and maximumPercent = 100. Diego launches a deployment and the workflow hangs until wait-for-service-stability times out. Explain what has happened and propose two valid configurations.

Marta wants to retire the duration field from the GET /appointments response and replace it with durationMin. Reservalia's web app consumes that field and three customers have integrations that also use it. Design the sequence of deployments and say which strategy you would use for each.

Solution 1. The choice is canary, ideally complemented by a preliminary shadow phase. The reasoning: recreate and rolling update expose everybody or an uncontrolled fraction, and here the failure produces incorrect bookings — business damage, not just a technical error. Blue-green exposes 100% as soon as it switches over, so it does not bound the damage. A/B testing does not apply: the question is about correctness, not preference. Shadow is ideal because calculateSlots is a read function and the outputs of v1 and v2 can be compared under real traffic without the user seeing anything; its limit is that it does not validate the full booking flow. With the canary at 10% I would measure, besides 5xx errors and p95 latency: the number of bookings created per session (a drop suggests slots disappearing), the rate of conflicts when confirming an appointment (two users booking the same slot suggests duplicated slots) and support complaints. With ~30 daily bookings on the canary, the observation window has to be measured in days, not minutes, or the initial weight has to go up to 25%.

Solution 2. With minimumHealthyPercent = 100 and maximumPercent = 100, ECS cannot withdraw any old task (it would break the minimum of 2 healthy ones) nor start any new one (it would break the maximum of 2 in total). The deployment is stuck: it is an impossible configuration. Two valid fixes: (a) 100 / 200, which allows starting up to 2 new tasks before withdrawing the old ones, with no loss of capacity and a transient extra cost; (b) 50 / 100, which allows withdrawing one old task to make room for a new one, with no extra cost but with capacity halved during the handover. In staging, (b) is perfectly reasonable; in prod, (a).

Solution 3. Three deployments following expand and contract. Deployment 1 (expand): the API returns duration and durationMin with the same value. It is an additive, compatible change, so a normal rolling update is enough. Deployment 2 (migrate the consumers): the web app switches to reading durationMin; the three integration customers are warned with a deadline and the use of the old field is instrumented so you know when it stops being read — without that data, the deadline is guesswork. Deployment 3 (contract): when the telemetry confirms nobody consumes duration, it is removed from the response; here a canary is advisable, because it is the only step that can break an unsuspecting third party and the weight allows reverting in seconds. The classic mistake is trying to do it in a single deployment trusting that "the web app is deployed at the same time": during the rolling update both versions of the API and both of the web app coexist, so the incompatibility shows up regardless.

There is no longer a single way of replacing one version with another, but a menu with prices. Recreate is simple and it cuts service; rolling update is the sensible default; blue-green buys an instant rollback by paying for double capacity; canary buys low risk by paying with time; A/B is not a technical strategy but a product tool; and shadow validates under real load without exposing anybody, provided side effects are isolated. Reservalia settles on rolling update 100/200 plus the circuit breaker, and reserves the ALB-weight canary for anything risky.

Underneath all of them there are two foundations worth not forgetting: an honest readiness check that knows how to say 503, because it is the signal every piece of automation leans on, and backward compatibility, because as soon as you abandon recreate there are two versions coexisting on the same database.

Even so, we are still tied to an optimistic assumption: that the failure is detected during the deployment. Many failures show up hours later, once the canary has been promoted and the observation window has closed. And there is a prior problem: deploying code and switching a feature on are the same action, so a half-finished feature forces you to keep long-lived branches. The next lesson, Feature Flags, Rollback and Failure Recovery, separates deployment from release, builds Reservalia's go-back button and turns the 68 minutes of time to restore into an achievable target.