In the three previous lessons CicloUrbana reached Heroku, AWS and Kubernetes. In all three there was an implicit actor who was still human: somebody running git push heroku main, aws ecs update-service or helm upgrade from their laptop, with their personal credentials, trusting that they had run the tests beforehand and had not left anything half-committed.

That actor is the last handcrafted point in the project, and this lesson removes it. We are going to build a pipeline that, on every git push, compiles, runs the complete module 6 suite —Testcontainers included—, analyses quality, builds and scans the image, publishes it tagged with the commit SHA, deploys it to pre, passes a smoke test and, after an explicit approval, takes it to production. Nobody touches a server. And with that, the tests we wrote in module 6 stop being an exercise in personal discipline and become a gate nobody can skip.

1. Continuous integration, delivery and deployment
2. Why CI is what gives the tests their value
3. Anatomy of a pipeline
4. GitHub Actions: the concepts
5. The integration workflow: ci.yml
6. The delivery workflow: cd.yml
7. Secrets in the pipeline
8. Quality inside the pipeline
9. Versioning and releasing
10. Deploying to pre and to prod, and rolling back
11. Database migrations in the pipeline
12. Making the pipeline fast and reliable
13. Alternatives and DORA metrics
14. Common Mistakes and Tips
15. Exercises

The three terms are used as synonyms and they denote different things. The difference lies in how far the automation goes.

The practical distinction worth retaining: continuous delivery means you could deploy at any moment; continuous deployment means you always do. The difference between them is a business and confidence decision, not a technological one: it is exactly the manual approval step of section 10.

What we build for CicloUrbana: full continuous integration, continuous delivery as far as pre automatically, and deployment to prod with approval. It is the sensible choice for a municipal service in its first year. Once the pipeline has gone months without surprises and the observability of module 9 is in place, removing that approval will be a small step.

In module 6 we wrote a considerable suite: unit tests for Ribalta's fares, Mockito doubles, @WebMvcTest and @DataJpaTest slices, the access matrix with spring-security-test and *IT tests against real PostgreSQL 16 with Testcontainers. All of that runs with ./mvnw verify.

The problem is that ./mvnw verify is run by whoever remembers to. And the familiar patterns show up: somebody is in a hurry and pushes without running them; somebody runs them but only for the module they touched; a test has been failing for two weeks and the team has normalised the red; a test passes on the laptop of whoever wrote it because they have a row in their local database that nobody else has.

CI cuts all four off at the root:

That last point is the underlying cultural change: CI is the referee. And there is one technical detail that is decisive for us: the GitHub Actions runner already has Docker installed and running, so the Testcontainers of 06-05 work with no configuration at all. The integration tests against real PostgreSQL, which were the most valuable piece of module 6, are also the ones CI runs with the greatest fidelity.

Two principles govern that order. The first: the fast things and the things that fail most, first. Compiling takes seconds and catches the most common error; integration tests take minutes. Putting the slow work at the start makes a silly mistake cost ten minutes instead of thirty.

The second: every stage is a gate. If one fails, the following ones do not run. A pipeline that carries on "because it was only the static analysis" is not a gate, it is an ornament.

Two important properties: every job starts on a clean machine (hence the need for caches and artifacts to pass things between jobs), and jobs run in parallel unless needs: is declared, which is what enforces the order of the gates.

Line by line, what you need to understand:

• on: push and pull_request against main is the operational definition of continuous integration: you verify what is proposed for integration and what has already been integrated.
• concurrency with cancel-in-progress avoids spending runner minutes verifying a commit that has already been superseded: on an active repository it easily saves a third of the consumption. And timeout-minutes: 25 stops a hung process from consuming hours.
• Explicit permissions. By default the GITHUB_TOKEN may have broad permissions; declaring them at a minimum stops a compromised third-party action from writing to the repository.
• fetch-depth: 0. By default checkout fetches a single commit; Sonar needs the history to attribute new lines, and the git-commit-id plugin that feeds the /actuator/info of 07-01 needs the Git data.
• cache: maven is the setting with the best effort-to-benefit ratio: without it, every run downloads the whole dependency tree again —Spring Boot, Hibernate, Spring Security, MapStruct, the drivers— and adds two or three minutes to every build.
• ./mvnw -B verify is the heart of the workflow. -B (batch) disables colours and progress bars, which in a CI log only generate noise. And verify, not test: test only runs Surefire (the *Test classes), whereas verify also runs Failsafe (the *IT classes), which are the Testcontainers ones from 06-05. Using test in CI would mean leaving out precisely the tests that most resemble production.
• Testcontainers work with no configuration because the ubuntu-latest runner comes with Docker installed and the daemon running. The first run pulls the postgres:16-alpine image (about 15 seconds) and from then on the *IT tests run against real PostgreSQL.
• if: always() on the reports is subtle and decisive: without it, a failed verify aborts the job and the report is not published, which is exactly what you need in order to know what failed. With always(), the JUnit report appears annotated on the pull request, line by line.

When container services are needed. GitHub Actions lets you declare auxiliary services in a job:

With Testcontainers this is not needed, and it is important to understand why: Testcontainers starts and stops the container from the test code itself, with the same configuration in CI and on the developer's laptop — which is exactly the environment parity we were after. services make sense when the tests do not use Testcontainers, or for dependencies that cannot be brought up from Java.

And the gate. A workflow that reports but does not block is of no use. In Settings → Branches → Branch protection rules you have to require the verify check to pass before a merge into main is possible. Without that configuration, the pipeline is a report; with it, it is a guarantee.

The underlying decisions in this file:

It is triggered by a tag, not by every push. v2.4.1 is a deliberate act: somebody decides that this commit is a release. It is the boundary between continuous integration (every change) and delivery (releases).

The tests are not repeated. -DskipTests is not a cheat: ci.yml already ran them on this very commit. Repeating them would double the time without adding information.

Triple tagging. docker/metadata-action produces 2.4.1 (the readable semantic version), sha-9f3a2b1 (the exact, unambiguous commit identifier) and latest all at once. It is what makes it possible to answer with certainty the question "what code is running in Ribalta?": you compare the SHA from /actuator/info with the one in the repository.

environment: prod is the manual approval. By configuring that environment in GitHub with required reviewers, the job stops and waits for an authorised person to approve it in the interface. It is the exact line that separates continuous delivery from continuous deployment: removing the environment's protection turns one into the other.

The scanner stops the delivery. exit-code: '1' with severity HIGH,CRITICAL means that a serious vulnerability in the image prevents publication, linking up with the hardening of 07-04 and with OWASP Dependency-Check from 05-05.

The smoke test with --retry. After a helm upgrade, the pods take a while to be ready; without retries, the curl would fail simply by arriving too early. And the jq -e 'length == 4' checks something real —Ribalta's four stations— and not just that the process responds.

The pipeline needs credentials to publish images and deploy. It is, by definition, a system with elevated permissions and therefore a target.

OIDC with AWS, which eliminates long-lived keys entirely:

There is no AWS_ACCESS_KEY_ID anywhere. GitHub issues a signed OIDC token that identifies the repository, the workflow and the branch; AWS validates it against a trust relationship and returns credentials that expire in an hour. And the trust policy must restrict which repository and which branch can assume the role:

Without that condition, any GitHub repository could assume the role. It is OIDC's most serious and most frequent configuration mistake.

Prominent warning: never print a secret in the logs. GitHub masks values registered as secrets, but the masking breaks easily: if the secret is transformed (base64-encoded, trimmed, concatenated), the derived value is not masked. A forgotten debugging echo, a set -x in a script or a tool that dumps its configuration can leave a credential in a log that may well be public. And if it happens, the only correct response is to rotate the secret: deleting the log is not enough, because it may have been read or replicated.

Three more rules: least privilege —the deployment role can update the service, not delete the database—; explicit permissions in every workflow; and pinning third-party actions by SHA (uses: acme/action@a1b2c3d) rather than by a moving tag, because a tag can be repointed at malicious code.

The pipeline is the only place where a quality rule is always enforced. What is worth putting in, and in what order:

The coverage threshold from 06-01, now as a real gate:

And the warning already made in 06-01, more important here because it is now mandatory: a threshold that is too high produces junk tests. If reaching 90 % is required in order to merge, somebody will write tests with no assertions that exercise code just to push the percentage up. A threshold of 70 % on lines is demanding and honest. And the most useful criterion is not overall coverage but the coverage of new code: that is what Sonar does with its quality gate, and it stops a legacy codebase with low coverage from blocking any progress.

Vulnerable dependencies, picking up from 05-05:

-DfailBuildOnCVSS=9 breaks the build only for critical vulnerabilities. A low threshold generates so many false positives that the team learns to ignore them, and an ignored check is worse than no check at all. This analysis is best run in a separate job scheduled at night, because it downloads the vulnerability database and can take several minutes. And Dependabot (.github/dependabot.yml) complements it by opening pull requests that update versions: since each one goes through ci.yml, upgrading Spring Boot stops being a leap of faith.

Semantic versioning (MAJOR.MINOR.PATCH) applied to CicloUrbana:

For an API with external clients —Ribalta's mobile app— the MAJOR part is a contract: a breaking change forces you to version the API (/api/v2/...) and to keep the previous one for a grace period.

And the circle closes with 07-01: Spring Boot's build-info plugin and git-commit-id record the version, the SHA and the date inside the artefact, and Actuator exposes them:

That SHA is the definitive answer to "what is deployed?", and it makes the deployment verifiable rather than merely believable. The release notes, generated from the commit messages since the previous tag, complete the trail: somebody investigating an incident can go from the SHA the server reports to the exact list of changes that went in.

The workflow in section 6 implements the policy: pre automatic, prod with approval. What the person approving must be able to see before pressing: that CI is green, that pre has been running that version for a while, that the smoke test passed and which changes are going in.

The strategies from 08-01, now automated:

The rollback, which is the part most often neglected, has three levels depending on the severity:

And the underlying condition, repeated for the third time in the module because it is the one that ruins the most deployments: going back to the previous image only works if the database schema is still compatible with it.

The migration is the only step in the pipeline that cannot be undone. Everything else —the image, the configuration, the number of replicas— goes back with one command; an applied migration stays applied.

Hence the rule that governs the whole module: the expand/contract pattern from 04-08 is the condition for being able to roll back the application without rolling back the schema.

In each of the three deployments, the previous version of the application still works against the resulting schema. That property is what makes helm rollback a safe operation rather than a gamble.

How it translates into the pipeline:

1. A step of its own for the migration, before touching the live instances: Helm's pre-upgrade hook (08-04), the one-off ECS task (08-03) or Heroku's release phase (08-02). If it fails, the deployment stops and the previous version carries on serving.
2. An automatic check in the pull request that rejects migrations with DROP COLUMN, RENAME or incompatible type changes when they arrive alongside the code that uses them: ten lines of grep that prevent an incident.
3. ddl-auto: validate in every environment (04-08): an instance whose schema does not match fails at startup, clearly and immediately, instead of failing query by query.
4. Verification in pre with the previous version before approving production: it is the direct proof that a rollback will be possible.

A slow pipeline gets avoided, and a pipeline that fails for no reason gets ignored. Both pathologies have the same ending: the team stops trusting it and goes back to deploying by hand.

On flaky tests you have to be blunt. A test that sometimes fails teaches the team to relaunch the run instead of reading the error, and that habit destroys the value of the whole pipeline: the day it fails for real, somebody will press "retry". The usual causes are well known —timing dependencies (Thread.sleep instead of waiting for a condition), shared state between tests, an assumed execution order, and dates and time zones— and when fixing it will take time, the correct answer is @Disabled with a link to the issue, not leaving it failing.

Target times: CI under 10 minutes and the complete delivery as far as pre under 20. Above that, people start batching changes "so as not to spend a run", and batching changes is precisely the opposite of continuous integration.

The concepts are the same in all of them: triggers, stages, steps, artifacts, caches, secrets, approvals and environments. The syntax changes, not the design; what you learned with ci.yml and cd.yml transfers to any of them in an afternoon.

And what to look at after deploying. The DORA metrics are the standard framework for measuring delivery health:

What is valuable about these four is that they balance each other out: deploying a lot but breaking things constantly shows up badly in the third; never deploying "out of prudence" shows up badly in the first two and, paradoxically, in the fourth as well, because a large, rare deployment is far harder to roll back than a small, frequent one. The industry's counter-intuitive conclusion is solid: deploying more often makes the system more stable, not less, because each deployment is small, comprehensible and easy to undo.

And the immediate signals in the ten minutes after a deployment are those of 08-01: readiness green on every instance, zero restarts, stable 5xx rate and latency, no new exceptions in the log and /actuator/info showing the expected SHA. Measuring them properly is the subject of module 9.

Using mvn test instead of mvn verify. It only runs Surefire and leaves out the Testcontainers *IT tests, which are the most valuable ones. The pipeline goes green without ever having tested against real PostgreSQL.

Publishing the report without if: always(). When the tests fail —the only moment the report matters— the step does not run and there is nothing to read.

Not protecting the main branch. Without branch protection, CI reports but does not prevent a red merge: it is an expensive ornament.

Printing secrets in the log. A debugging echo or a transformation of the value breaks the masking; if it happens, rotate the secret, because deleting the log is not enough. And OIDC without a sub condition leaves a role assumable from any GitHub repository.

Recompiling in the delivery workflow. If cd.yml rebuilds from scratch, what gets deployed is not exactly what was tested. Reuse the artefact or build once from the same commit.

Living with flaky tests. They teach the team to press "retry" and destroy trust in the whole pipeline.

An unrealistic coverage threshold. A mandatory 90 % produces tests with no assertions. Measure the coverage of new code.

Tip: the pipeline is code. It is reviewed in a pull request, commented on and refactored. A three-hundred-line ci.yml with duplicated steps has the same problem as a three-hundred-line class.

Tip: use act or a test branch to iterate. Debugging a workflow through git push is slow and fills the history with "testing CI" commits: use a disposable branch and workflow_dispatch. And pin third-party actions by SHA, because a tag like @v4 can be repointed and a SHA cannot.

Write CicloUrbana's ci.yml workflow with two parallel jobs: one that compiles and runs the unit tests (Surefire) and another that runs the integration tests with Testcontainers (Failsafe), plus a third job that depends on both and publishes the summary. Explain what is gained and what is lost compared with the single job in section 5, how the build result is shared between jobs, and what would have to be configured in the repository for the pipeline to be a gate and not a report.

CicloUrbana's pipeline takes 28 minutes and the team has started batching changes so as not to wait. The measured times are: dependency download 4 min, compilation 1 min, unit tests 2 min, SonarCloud analysis 3 min, OWASP Dependency-Check 7 min, integration tests with Testcontainers 6 min, image build 4 min, Trivy scan 1 min. Design an optimisation plan that brings feedback down to under 10 minutes, with the estimated time after each measure and a justification of why each change is safe.

Version 2.6.0 is deployed to production at 10:00. At 10:07 the alarms show 12 % of 5xx on /api/v1/rentals and p99 latency through the roof. The release includes a migration V12__add_rentals_index.sql and a change in RentalService. Describe the exact sequence of actions in the first fifteen minutes, decide whether it can be rolled back and under what conditions, and propose the concrete pipeline improvements that would have prevented or contained the incident.

Solution 1

What is gained. The total time goes from being the sum to being the maximum of the two jobs: if the unit tests take 3 minutes and the integration ones 8, the result arrives in 8 instead of 11. And the feedback is more useful: a unit failure shows up after 3 minutes without waiting for the Testcontainers. Besides, each job has its own timeout tuned to what it does.

What is lost. Each job starts on a clean machine, so the compilation happens twice —around 60-90 seconds duplicated— and the Maven cache is downloaded twice (although from GitHub's cache, which is fast). The file is longer and there are more pieces to maintain. With a small suite, the single job of section 5 is simpler and not noticeably slower; the split starts to pay off when the integration tests go beyond 5 minutes.

How the result is shared. With actions/upload-artifact and download-artifact, which is what the summary job does. If you wanted to avoid the double compilation, the pattern would be a preceding job that compiles once and uploads target/ as an artifact; in practice, for a project the size of CicloUrbana, the saving does not justify the complexity. The if: always() on summary is essential: without it, a failure in either of the two jobs would prevent the report from being published.

For it to be a gate, in Settings → Branches: require a pull request before merging into main, mark unit-tests, integration-tests and summary as required checks, require the branch to be up to date with main before merging (so that what is really tested is the result of the integration), and forbid direct pushes, administrators included. Without that last point, the pipeline has a back door.

Solution 2

Diagnosis. The sequential total is 28 minutes, but not all of it belongs on the critical path of the feedback. The key is separating what a developer needs to know within minutes from what can be known later.

Result: pull request feedback in about 7 minutes, below the target, with the complete delivery (image + scan + deployment to pre) in another 6-8 only when a version is tagged.

Why each change is safe:

• (1) OWASP still runs daily and on every push to main, breaking the build for CVSS ≥ 9; the only thing that changes is that it does not block every pull request, and it should not: a vulnerability published this morning was not introduced by the change under review.
• (2) The cache is invalidated by the pom.xml hash, so a dependency change forces a full download: there is no risk of building against stale artefacts. (5) The same with the Docker layers: if a layer changes, it is rebuilt.
• (3) The two jobs are independent; the only trade-off is compiling twice, some 60 seconds that the parallelism more than makes up for.
• (4) It is consistent with the module's design —ci.yml verifies, cd.yml delivers— and the tagged commit is the same one that went through ci.yml. (6) Cancelling a stale run removes no verification: the most recent commit includes the previous one.

And a cultural measure, without which the techniques are not enough: the time target must be explicit and monitored. If CI goes back above 10 minutes, it is treated as an incident, not as something that just happens. The warning sign is the one the exercise describes: when people batch changes so as not to wait, the integration has stopped being continuous.

Solution 3

Minutes 0-2: contain, do not investigate. The rule from 08-01 is that the rollback criterion is decided in advance and applied without debate. 12 % of 5xx comfortably exceeds any reasonable threshold, so:

And before anything else, check whether a rollback is possible, which depends entirely on the migration:

Minute 2: the decision. V12 adds an index. It is an additive, backwards compatible migration: version 2.5.0 works perfectly against a schema with one extra index. So yes, it can be rolled back, and it is done immediately:

If V12 had been a DROP COLUMN or a RENAME, the answer would be the opposite: rolling back would have made the situation worse —the scenario of exercise 3 in 08-01— and you would have to fix forward with an urgent corrective deployment.

Minutes 3-8: confirm the recovery. Watch that the 5xx rate returns to the baseline, that p99 latency comes down and that /actuator/info shows 2.5.0. Communicate the status: the council must know that the service is restored before it asks.

Minutes 8-15: start investigating, now without pressure. The two hypotheses, in order of likelihood:

1. Creating the index locked the table. CREATE INDEX without CONCURRENTLY takes a lock that blocks writes to rentals while it is being built. On a large table that is minutes during which every POST /api/v1/rentals waits and ends up exhausting the pool's connection-timeout → 5xx and latency through the roof. It fits the symptom perfectly, and it explains why it affects /rentals and not the rest.
2. A regression in RentalService: a new N+1 query (04-06), an over-long transaction (04-07) or an unhandled exception.

How to tell them apart: the logs for the 10:00-10:07 window and the pool metrics (hikaricp.connections.pending, 09-03). If there are connections waiting and locks in PostgreSQL, it is the first; if there are application exceptions, it is the second.

Concrete improvements to the pipeline:

And an underlying observation that sums up the whole module: the pipeline worked exactly as designed —it built, tested, scanned, deployed to pre, asked for approval and deployed—. What was missing was not automation, but a verification that represented production conditions: a realistic data volume in pre and a rule that captured a known dangerous migration pattern. Pipelines do not prevent mistakes on their own; they prevent the mistakes somebody has bothered to encode as a check, and every incident is the opportunity to add one more.

The path from commit to the citizens of Ribalta is complete and automated. You draw a precise distinction between continuous integration, continuous delivery and continuous deployment, and you know that what separates the last two is not technology but a decision about trust —the protected environment: prod—. You have seen why CI is what turns the module 6 suite into a real guarantee: it stops being run by whoever remembers, in each person's environment, and starts running always, on a clean machine, with a public verdict that blocks the merge.

You have CicloUrbana's complete pipeline in two files: a ci.yml annotated line by line, with a Maven cache, ./mvnw -B verify running Surefire and Failsafe with Testcontainers on the Docker the runner already brings, reports published with if: always() and the branch protection that turns it into a gate; and a cd.yml that is triggered by a tag, builds and scans the image with Trivy, publishes it with triple tagging —semantic version, commit SHA and latest—, deploys to pre, passes a real smoke test against the four stations and waits for a human approval before touching production.

You know how to manage secrets with environments, GITHUB_TOKEN and OIDC with no long-lived keys, with the sub condition that stops any repository from assuming your role, and with the warning that a secret printed in a log is rotated, not deleted. You have quality as part of the pipeline —Spotless, SpotBugs, Sonar on new code, jacoco:check with an honest threshold, nightly OWASP Dependency-Check, Dependabot and the image scan—, semantic versioning tied to Actuator's build-info that answers which SHA is running in Ribalta, the rollback in its three levels and the expand/contract pattern as the condition for that rollback to be possible. And you know what makes a pipeline used or ignored: under ten minutes, with no flaky tests, with every stage failing fast.

CicloUrbana is in production and it gets there on its own. The question is no longer whether it works nor how it is deployed, but how it behaves: what latency a rental really has at peak time, which query consumes 40 % of the database time, how many times the same thing gets recalculated, what is going on inside the JVM when memory rises and never comes down. Module 9, Performance and Monitoring, answers all of that: performance and pool tuning, caching with Spring Cache, business metrics with Micrometer on top of the Actuator from 07-01, Prometheus and Grafana, log management and distributed tracing. Until now we have built and delivered the Ribalta network; from now on we are going to watch it run and make it fast.