CI/CD courses tend to fail at the same point: they teach pipelines on top of a "hello world" that has no database, no migrations, no two applications sharing code, and no team with differing opinions. And then, on a real project, nothing fits. This course does the opposite: step by step, we will build the complete pipeline of an application with just enough complexity for the real problems to show up. That application is called Reservalia. In this lesson we are going to get to know it in depth: what the product does, who is on the team, how the repository is organised, which commands already exist — because the pipeline invents nothing, it only invokes what the project already knows how to do — which environments there are and which infrastructure we deploy to. We will finish with the course roadmap applied to Reservalia and with what you need installed to follow along. We will not write any workflow yet: the first real piece of pipeline is built in lesson 02-02.
Contents
- What Reservalia is
- The team: Marta, Diego and Nuria
- How they work today (and why it hurts)
- The repository structure
- The
package.jsonfiles and the commands the pipeline will invoke - The local development environment with Docker Compose
- The three environments: dev, staging and prod
- The target AWS infrastructure
- The course roadmap applied to Reservalia
- How to follow the course if you do not use Node.js
- What you need installed
- Common mistakes and tips
- Exercises
- Conclusion
- What Reservalia is
Reservalia is a SaaS appointment booking platform for small service businesses: hair salons, dental clinics, car repair shops, physiotherapists, beauty salons. It is a fictional company created for this course; any resemblance to reality is intentional but not real.
The product has two faces:
- The business panel. The salon owner configures their services (cut, colour, treatment), their staff, their opening hours and their holidays. They check the day's schedule, confirm appointments and see basic statistics.
- The public booking page. The end customer arrives from a link, sees the free slots, picks one and books without registering. They receive an email confirmation and a reminder 24 hours beforehand.
Business model: a monthly subscription per business, with three plans. Right now they have 340 paying businesses and process around 9,000 appointments a month.
1.1. Why Reservalia is a good project for learning CI/CD
It is not a toy example, and that matters. Reservalia has exactly the ingredients that make a pipeline interesting:
| Ingredient | Why it complicates the pipeline | Where we will cover it |
|---|---|---|
| A database with an evolving schema | There are migrations to apply, and applying them badly breaks production or loses data | 04-06 |
| Two applications in a single repository | Do you test everything on every change, or only what is affected? | 04-04 |
| Code shared between them | A change in shared types affects both | 02-03 |
| Sensitive data (end customers' names, phone numbers, email addresses) | Environments cannot share real data; secrets have to be managed | 04-03 |
| Business hours with peaks | You cannot deploy any old way at 11:00 on a Saturday | 03-04 |
| Integrated third parties (email, payment gateway) | You have to decide what gets simulated in tests and what does not | 02-04 |
| A mobile application (from module 5 onwards) | "Deployment" goes through app stores, with their own rules | 05-02 |
If you manage to automate Reservalia, you will know how to automate your project.
- The team: Marta, Diego and Nuria
Three people, three points of view. All three will appear throughout the course, and their arguments are probably the ones you will have with your own team.
2.1. Marta — tech lead
She has been at Reservalia for four years and carries the responsibility for the product working. She is the one who decides what gets deployed and when, and the one who takes the call from sales when a customer complains.
- What worries her: that the incidents do not happen again, and being able to justify to management the time the team spends on "things that are not features".
- Her role in the course: she brings the business point of view. She is the one who asks "what does this cost and what do we gain?" (lesson 01-02) and the one who will press the approval button at the manual gate before production.
- Her line: "I would rather deploy ten times a day and not care about any of them than deploy once a week and lie awake on Thursday."
2.2. Diego — backend developer
Three years at the company. He writes most of apps/api and is, today, the only person who knows how to deploy. He knows perfectly well that the current process is bad; what he has never had is time to fix it.
- What worries him: that the pipeline will slow him down. He is terrified of a CI that takes 40 minutes to tell him whether his change is fine.
- His role in the course: he is the productive sceptic. Every time we add a step to the pipeline, Diego will ask how much time it adds. Thanks to him, the final pipeline will be fast.
- His line: "If CI takes longer than going for a coffee, I will end up ignoring it."
2.3. Nuria — SRE
She joined eight months ago, part time, shared with another product. She set up the AWS infrastructure by hand, from the web console, because they had to get to production immediately. She carries the pager.
- What worries her: that the infrastructure is documented nowhere except in her head, and that a rollback today means rebuilding and re-uploading over SFTP with customers waiting.
- Her role in the course: she brings the operations perspective. She is the one who insists on infrastructure as code (03-03), on rollback (03-05) and on monitoring (03-06).
- Her line: "A deployment that cannot be undone in five minutes is not a deployment, it is a bet."
- How they work today (and why it hurts)
Let us recall the starting point, now with the full detail of the process:
# The Friday ritual at Reservalia. Duration: ~3 hours.
# Performed by: Diego, always Diego.
# 1) Pull the latest from main (without knowing exactly what goes in)
git checkout main && git pull
# 2) Build on his laptop, with HIS version of Node and HIS node_modules
cd apps/api && npm install && npm run build
cd ../web && npm install && npm run build
# 3) Upload the files over SFTP to the single production server
sftp diego@reservalia-prod
# > put -r apps/api/dist/* /var/www/api/
# > put -r apps/web/dist/* /var/www/web/
# 4) Restart the process by hand
ssh diego@reservalia-prod 'pm2 restart api'
# 5) Apply the migrations by pasting SQL into psql
psql -h reservalia-prod-db -U admin -d reservalia
# > ALTER TABLE appointments ADD COLUMN reminder_sent boolean DEFAULT false;
# 6) Eyeball that the site loads and create a test appointment
# 7) Write in the team channel: "deployed ✅"And the problems with this ritual, listed without mercy:
| # | Problem | Real consequence they have already suffered |
|---|---|---|
| 1 | Build on Diego's laptop | The build is not reproducible; nobody can recreate what is in production |
| 2 | npm install instead of npm ci |
A minor version of a dependency reached production without anyone having tested it |
| 3 | Incremental deployment over SFTP | Files from old versions are still alive on the server; the real state is unknown |
| 4 | A deployment window with no service | There are a few seconds in which the API only half responds |
| 5 | Migrations by hand | Nobody knows for certain which migrations have been applied, or in what order |
| 6 | No record of what was deployed | When something fails, you have to guess what changed |
| 7 | Visual verification | A deployment in which email sending was broken was signed off as fine |
| 8 | Bus factor of 1 | During Diego's holiday in August, nothing was deployed for three weeks |
| 9 | Rollback ≈ 1 hour | In the last incident, the service was degraded for 40 minutes |
| 10 | Friday afternoon | Two ruined weekends last quarter |
Every line in this table will disappear at some point in the course. Keep it: in the final lesson (07-06) we will go through the whole thing again.
- The repository structure
Reservalia has a single repository at github.com/reservalia/reservalia, organised as a monorepo with npm workspaces. This is its structure as it stands today, before we start:
reservalia/
├── .github/
│ └── workflows/ ← empty today; our pipelines will live here
├── apps/
│ ├── api/ ← Node.js 20 + TypeScript + Express + PostgreSQL
│ │ ├── src/
│ │ │ ├── index.ts server entry point
│ │ │ ├── routes/
│ │ │ │ ├── appointments.ts create, list and cancel appointments
│ │ │ │ ├── businesses.ts business sign-up and configuration
│ │ │ │ └── availability.ts free slot calculation
│ │ │ ├── domain/
│ │ │ │ ├── appointment.ts business rules for an appointment
│ │ │ │ └── schedule.ts overlaps, opening hours, holidays
│ │ │ └── db/
│ │ │ ├── client.ts PostgreSQL connection
│ │ │ └── migrations/
│ │ │ ├── 0001_create_businesses.sql
│ │ │ ├── 0002_create_appointments.sql
│ │ │ └── 0003_add_reminders.sql
│ │ ├── tests/
│ │ │ ├── unit/ fast, no database
│ │ │ └── integration/ against a real PostgreSQL
│ │ ├── Dockerfile ← we will write it in 02-03
│ │ ├── package.json
│ │ └── tsconfig.json
│ │
│ └── web/ ← React + Vite + TypeScript
│ ├── src/
│ │ ├── main.tsx
│ │ ├── pages/
│ │ │ ├── BusinessPanel.tsx
│ │ │ └── PublicBooking.tsx
│ │ └── components/
│ ├── tests/
│ ├── Dockerfile ← we will write it in 02-03
│ ├── package.json
│ └── vite.config.ts
│
├── packages/
│ └── shared-types/ ← TypeScript types used by api and web
│ ├── src/index.ts
│ └── package.json
│
├── infra/ ← infrastructure as code (module 3)
│ ├── terraform/
│ │ ├── modules/
│ │ └── environments/
│ │ ├── staging/
│ │ └── prod/
│ └── README.md
│
├── docker-compose.yml ← local development environment
├── package.json ← monorepo root, with workspaces
├── package-lock.json ← just one for the whole monorepo!
├── .nvmrc ← pins the Node version
└── README.mdThree decisions in this structure deserve an explanation, because they shape the entire pipeline:
A monorepo with a single package-lock.json. By using npm workspaces, there is one lock file at the root that governs the dependencies of api, web and shared-types. Advantage: a single npm ci installs everything coherently and reproducibly. Consequence for the pipeline: installation happens once at the root, not once per application.
packages/shared-types. This is where the types the API and the web app share live (for example, the shape of an Appointment). It is what guarantees that if Diego changes the data model, the web app stops compiling in CI instead of breaking in production. It also means that a change in that package forces both applications to be tested: it is the complication that makes the "run only what is affected" of lesson 04-04 interesting.
.nvmrc. A one-line file stating which version of Node the project uses. It is the cheapest piece of repeatability there is:
With that, Diego's laptop, Marta's laptop and the CI runner all use exactly the same version. In lesson 01-01 we saw that an unpinned version is one of the classic enemies of the reproducible build; this file solves it.
- The
package.json files and the commands the pipeline will invoke
package.json files and the commands the pipeline will invokeHere is a central idea of the course, and it is worth underlining:
The pipeline invents nothing. It merely runs, on a clean machine, the same commands you run on your laptop.
That is why, before writing any workflow, the project must have its commands properly defined. If npm test does not work on your machine, it will not work in CI. If it works on your machine but only because you have an environment variable nobody else has, it will fail in CI. A pipeline is, above all, a revealer of hidden assumptions.
5.1. The root package.json
{
"name": "reservalia",
"private": true,
"workspaces": [
"apps/*",
"packages/*"
],
"engines": {
"node": ">=20.11.0 <21"
},
"scripts": {
"build": "npm run build --workspaces --if-present",
"test": "npm run test --workspaces --if-present",
"lint": "npm run lint --workspaces --if-present",
"typecheck": "npm run typecheck --workspaces --if-present",
"dev": "docker compose up -d && npm run dev --workspace apps/api"
}
}Point by point, because every line has a reason:
"private": truestops the root package from being published to the public npm registry by accident. In a monorepo it is mandatory."workspaces"declares the subprojects. When you runnpm ciat the root, npm installs the dependencies of all of them and creates the links betweenshared-typesand the applications that use it."engines"documents the supported Node version. Combined with.nvmrc, it puts the requirement explicitly on record.--workspaces --if-presentruns the script in every subproject that has it defined, and does not fail in the ones that do not. It is what allows a singlenpm testat the root to test the API, the web app and the shared package.
With this, the entire pipeline could be reduced to four commands. See for yourself:
npm ci # reproducible installation of the whole monorepo
npm run lint # style and obvious errors
npm run typecheck # type coherence across api, web and shared-types
npm run test # tests for all subprojects
npm run build # compiled artifactsThese five commands are, quite literally, the skeleton of the pipeline we will build in module 2.
5.2. The apps/api package.json
{
"name": "@reservalia/api",
"version": "0.0.0",
"private": true,
"scripts": {
"dev": "tsx watch src/index.ts",
"build": "tsc --project tsconfig.build.json",
"start": "node dist/index.js",
"test": "vitest run",
"test:unit": "vitest run tests/unit",
"test:integration": "vitest run tests/integration",
"lint": "eslint src tests --max-warnings 0",
"typecheck": "tsc --noEmit",
"migrate": "node dist/db/migrate.js",
"migrate:status": "node dist/db/migrate.js --status"
},
"dependencies": {
"@reservalia/shared-types": "*",
"express": "4.19.2",
"pg": "8.11.5",
"zod": "3.23.8"
},
"devDependencies": {
"@types/express": "4.17.21",
"eslint": "8.57.0",
"tsx": "4.7.1",
"typescript": "5.4.5",
"vitest": "1.6.0"
}
}Details that matter for the pipeline:
test:unitandtest:integrationare separate. The unit tests are fast and need no database; the integration ones need a running PostgreSQL. This separation is what will let us, in 02-04, give Diego fast feedback: the quick ones first, and only if they pass, the slow ones.--max-warnings 0in the lint step. Without it, ESLint exits with code 0 even when there are warnings, and the pipeline would turn green with problems inside. Remember from 01-03: the pipeline only understands exit codes.typecheckseparate frombuild.tsc --noEmitchecks the types without generating files. It is fast and can run in parallel with the tests.migratealready exists as a script. Today Diego pastes SQL by hand; the project already has the command, it is just that nobody uses it. In lesson 04-06 we will integrate it into the deployment.- Exact versions, no
^."express": "4.19.2"and not"^4.19.2". Together withnpm ci, it guarantees that two installations give the same result. The complete dependency strategy is the subject of 04-02.
5.3. The apps/web package.json
{
"name": "@reservalia/web",
"version": "0.0.0",
"private": true,
"scripts": {
"dev": "vite",
"build": "vite build",
"preview": "vite preview",
"test": "vitest run",
"lint": "eslint src tests --max-warnings 0",
"typecheck": "tsc --noEmit"
},
"dependencies": {
"@reservalia/shared-types": "*",
"react": "18.3.1",
"react-dom": "18.3.1"
},
"devDependencies": {
"@vitejs/plugin-react": "4.2.1",
"typescript": "5.4.5",
"vite": "5.2.11",
"vitest": "1.6.0"
}
}Note that the script names match those of the API: build, test, lint, typecheck. This convention is not cosmetic: it is what makes npm run test --workspaces work and what will let the pipeline treat both applications with the same logic.
A transferable tip: unifying script names across subprojects is one of the most profitable investments you can make before building a pipeline. If in one project the command is
npm testand in anothernpm run test:ci, your YAML will fill up with special cases.
- The local development environment with Docker Compose
For the integration tests to work — both on the laptop and later in CI — a PostgreSQL is needed. Reservalia brings it up with Docker Compose:
# docker-compose.yml — Reservalia's local development environment
services:
db:
image: postgres:16.3 # PINNED version, the same as in RDS
container_name: reservalia-db
environment:
POSTGRES_USER: reservalia
POSTGRES_PASSWORD: development # ⚠️ local only, never in another environment
POSTGRES_DB: reservalia
ports:
- "5432:5432" # reachable from the laptop
volumes:
- db-data:/var/lib/postgresql/data
healthcheck: # is the database READY, not just started?
test: ["CMD-SHELL", "pg_isready -U reservalia -d reservalia"]
interval: 5s
timeout: 3s
retries: 10
mailpit:
image: axllent/mailpit:v1.18 # captures emails locally
ports:
- "1025:1025" # the SMTP the API points at
- "8025:8025" # web interface for reading them
volumes:
db-data:Four things to explain, because all of them will reappear in the pipeline:
The version is pinned: postgres:16.3. Not postgres:latest. If locally you develop against PostgreSQL 16 and production runs 15, sooner or later a query will work on your machine and fail in production. The database version is part of repeatability.
The healthcheck is the most important line in the file. A "started" PostgreSQL container does not mean "ready to accept connections": there are a few seconds of initialisation. Without a healthcheck, the integration tests fail intermittently because the database is not accepting connections yet. And that is exactly a flaky test, the poison we talked about in 01-02. The number one cause of flakiness in CI is not waiting for services to be genuinely ready.
Mailpit replaces the real email provider. Reservalia sends confirmations and reminders. Locally and in tests, those emails must not go out to the internet: Mailpit captures them and shows them in a web interface. The general rule we will apply in 02-04: in tests, no real external services.
The password development is written in plain text, and that is fine. Because it is a local, ephemeral database with no real data. In staging and prod we will never do this: there the credentials come from a secrets manager. Telling when a value is a secret and when it is not is part of the material in 04-03.
Diego's daily workflow, in four commands:
git clone https://github.com/reservalia/reservalia.git
cd reservalia
nvm use # reads .nvmrc → Node 20.11.0
npm ci # installs the whole monorepo reproducibly
docker compose up -d # brings up PostgreSQL and Mailpit
npm run dev # starts the API in development modeWhen we configure CI in module 2, you will see that the pipeline does exactly the same: prepare Node, install dependencies, bring up the necessary services and run commands. There is no magic.
- The three environments: dev, staging and prod
An environment (as defined in 01-01) is a deployed, runnable instance of the system, with its own configuration and its own data. Reservalia will have three:
| Aspect | dev |
staging |
prod |
|---|---|---|---|
| What it is for | Letting the team try changes in a shared, real environment | Dress rehearsal: the last check before production | Real customers |
| Who uses it | Marta, Diego, Nuria | The team, before approving a deployment | 340 businesses and their customers |
| What gets deployed | Every merge to main, automatically |
Every merge to main, after passing dev |
Only after Marta's manual approval |
| Data | Generated fictional data, can be wiped | Fictional data with realistic volume | Real customer data |
| Database | RDS db.t4g.micro |
RDS db.t4g.micro |
RDS db.t4g.medium, multi-AZ and with backups |
| API instances | 1 | 1 | 2 at minimum, with auto-scaling |
| Captured, never sent | Captured, never sent | Real provider | |
| Payment gateway | Simulated mode | Simulated mode | Live mode |
| Who can access it | The team only | The team only | Public |
| If it goes down | Nothing happens | Nothing happens | An incident |
Two principles govern all three environments and are worth internalising right now:
Principle 1: the same artifact in all three. It is the promotion rule from 01-01. The image reservalia/api:a3f9c21 tested in dev is exactly the one that reaches staging and prod. The only thing that changes between environments is the configuration injected from outside:
# dev
DATABASE_URL=postgres://[email protected]:5432/reservalia
LOG_LEVEL=debug
PAYMENT_GATEWAY_MODE=simulated
SMTP_HOST=mailpit.internal
# prod
DATABASE_URL=postgres://[email protected]:5432/reservalia
LOG_LEVEL=info
PAYMENT_GATEWAY_MODE=live
SMTP_HOST=smtp.mail-provider.comPrinciple 2: production data never leaves production. Copying the prod database into staging "to test with real data" is a common practice and a very bad idea: Reservalia stores end customers' names, phone numbers and email addresses. staging uses generated data with realistic volume, not real data.
- The target AWS infrastructure
Nuria set this up by hand from the AWS console. In module 3 we will move it to Terraform so that it is reproducible; for now, this is the picture of where the pipeline deploys to:
flowchart TB
U["👥 Users<br/>businesses and their customers"] --> CF["CloudFront + S3<br/>apps/web static"]
U --> ALB["Application Load Balancer<br/>api.reservalia.com"]
subgraph AWS["AWS · eu-west-1 region"]
ALB --> ECS
subgraph ECS["ECS Fargate · reservalia-api service"]
T1["Task 1<br/>container api:a3f9c21"]
T2["Task 2<br/>container api:a3f9c21"]
end
ECS --> RDS[("RDS PostgreSQL 16<br/>multi-AZ in prod")]
ECS --> SM["Secrets Manager<br/>DB and API credentials"]
ECS --> CW["CloudWatch Logs"]
ECR[("ECR<br/>image registry")] -.->|"pulls the image<br/>when deploying"| ECS
end
GHA["⚙️ GitHub Actions"] -->|"1· publishes the image"| ECR
GHA -->|"2· updates the service"| ECS
GHA -->|"3· uploads the static files"| CF
style ECR fill:#d9f2d9
style GHA fill:#cfe8ff
What each piece does and why it is there:
| Piece | Function | Why this one and not another |
|---|---|---|
| ECR (Elastic Container Registry) | Stores the Docker images tagged with the commit SHA | It is the artifact registry: the piece that makes building once and promoting possible |
| ECS Fargate | Runs the API containers without administering servers | Orchestration without the complexity of Kubernetes: proportionate to a three-person team (see 01-03) |
| ALB (Application Load Balancer) | Distributes traffic across the tasks and checks their health | Enables zero-downtime deployments: it removes an old task only when the new one responds correctly |
| RDS PostgreSQL | The managed database | Backups, patching and high availability with no manual work |
| S3 + CloudFront | Serve the compiled web app as static files | The Vite app is HTML, CSS and JS: it needs no server |
| Secrets Manager | Stores credentials and injects them at runtime | Secrets never travel in the image or in the repository (04-03) |
| CloudWatch Logs | Collects the tasks' logs | The basis for the monitoring in 03-06 |
And the complete journey of a change, from Diego's laptop to the end customer:
sequenceDiagram
participant D as Diego
participant GH as GitHub
participant GA as GitHub Actions
participant ECR as ECR
participant ECS as ECS Fargate
participant U as User
D->>GH: push to branch + pull request
GH->>GA: pipeline trigger
GA->>GA: npm ci · lint · typecheck · test
GA-->>GH: ✅ green, ready to merge
D->>GH: merge to main
GH->>GA: deployment pipeline trigger
GA->>GA: docker build → image tagged with the SHA
GA->>ECR: push of reservalia/api:a3f9c21
GA->>ECS: deploy to dev and staging
GA->>GA: smoke tests in staging
GA-->>GH: ⏸️ waiting for Marta to approve
Note over GA: manual gate = continuous delivery
GA->>ECS: promote the SAME artifact to prod
ECS->>U: new version serving traffic
This diagram is the goal of the course. By the end of module 3, Reservalia will have exactly this flow working.
- The course roadmap applied to Reservalia
What will have changed at Reservalia by the end of each module:
| Module | What we will have automated | State of the Friday ritual |
|---|---|---|
| 1. Introduction (you are here) | Nothing yet: vocabulary, criteria, project and baseline metrics | Untouched: 3 h, manual |
| 2. Continuous integration | Every PR runs npm ci, lint, typecheck, unit tests and integration tests against PostgreSQL. Docker images tagged with the SHA are built and published to ECR. main is protected so nothing red gets in |
Deployment is still manual, but nothing broken reaches main any more. Diego stops building blind |
| 3. Continuous deployment | Automatic deployment to dev and staging; manual gate for prod; infrastructure in Terraform; progressive deployment; feature flags; automatic rollback; monitoring |
The Friday ritual disappears. Deployments happen when needed, in minutes, and can be undone |
| 4. Advanced practices | A reusable, tested pipeline; dependency management and updates; security scanning and secrets management; time and cost optimisation; safe, reversible database migrations | Diego stops touching psql for good. The pipeline drops from 14 minutes to under 8 |
| 5. Real projects | apps/mobile (React Native) joins with its own flow towards the app stores; splitting into microservices and how to modernise a legacy system are studied |
The pipeline supports three applications with different needs |
| 6. Tools | The same pipeline re-expressed in Jenkins, GitLab CI, CircleCI and Travis CI; containers and Kubernetes; GitHub Actions in depth | The team's knowledge stops depending on a single tool |
| 7. Exercises | A practical reconstruction of everything, from start to finish, on your own | You can do it yourself, not just read about it |
| 8. Resources | Learning paths, communities, certifications | — |
- How to follow the course if you do not use Node.js
Reservalia uses Node.js and TypeScript, but the course is not about Node.js. It is about pipelines. If your daily work is Python, Java, Go, PHP, Ruby or .NET, everything you learn applies just the same: the only thing that changes is the specific commands in each step.
Here is the translation table. Keep it:
| Pipeline step | Node.js (the course) | Python | Java (Maven) | Go | PHP | .NET |
|---|---|---|---|---|---|---|
| Pinning the version | .nvmrc |
.python-version |
pom.xml + toolchain |
go.mod |
composer.json |
global.json |
| Installing dependencies | npm ci |
pip install -r requirements.txt |
mvn dependency:go-offline |
go mod download |
composer install --no-dev |
dotnet restore |
| Lock file | package-lock.json |
pinned requirements.txt / poetry.lock |
pom.xml with exact versions |
go.sum |
composer.lock |
packages.lock.json |
| Static analysis | eslint |
ruff / flake8 |
checkstyle / spotbugs |
go vet |
phpstan |
dotnet format |
| Type checking | tsc --noEmit |
mypy |
(the compiler) | (the compiler) | phpstan |
(the compiler) |
| Running tests | npm test (vitest) |
pytest |
mvn test |
go test ./... |
phpunit |
dotnet test |
| Building | npm run build |
(packaging or image) | mvn package |
go build |
(image) | dotnet publish |
| Resulting artifact | Docker image | Docker image / wheel | .jar or image |
binary or image | image | image |
| DB migrations | migrate script |
alembic upgrade head |
flyway migrate |
migrate up |
doctrine:migrations:migrate |
dotnet ef database update |
What is identical in every language — and it is the vast majority of the course:
- The concepts: trigger, job, stage, runner, artifact, environment, promotion.
- The success or failure criterion: each command's exit code.
- The pipeline structure: install → verify → build → publish → deploy.
- The environment strategy and the build-once-and-promote rule.
- Containers: a Python
Dockerfileand a Node one look extremely similar. - Deployment strategies, rollback, feature flags and monitoring.
- The DORA metrics and everything relating to security and secrets.
A practical suggestion: if you want to get the most out of it, adapt each exercise to your own project. When we configure npm ci && npm test in 02-02, set up the equivalent with pip install && pytest in a repository of your own in parallel. You will learn twice as much.
- What you need installed
To read the course and understand the examples: nothing. All the files are complete in the lessons.
To reproduce the examples, which is highly recommended:
| Tool | What for | How to check you have it |
|---|---|---|
| Git | Cloning, branching, committing | git --version |
| A GitHub account | Hosting your repository and running Actions (free on public repositories) | — |
| Node.js 20 | Running the example project | node --version |
| Docker + Docker Compose | Local PostgreSQL and image building | docker --version and docker compose version |
| An editor with YAML support | Writing workflows without fighting the indentation | — |
Optional, and only from module 3 onwards:
| Tool | What for | Note |
|---|---|---|
| An AWS account | Deploying for real to ECS and RDS | It costs money. You can follow module 3 without one: all the files are readable and applicable later |
| AWS CLI | Interacting with AWS from the terminal | aws --version |
| Terraform or OpenTofu | Infrastructure as code | terraform --version |
A quick check of your environment:
# Run this and check that nothing essential is missing.
echo "--- Essential ---"
git --version || echo "❌ Git missing"
node --version || echo "❌ Node.js missing"
docker --version || echo "❌ Docker missing"
docker compose version || echo "❌ Docker Compose missing"
echo "--- Optional (module 3 onwards) ---"
aws --version || echo "ℹ️ AWS CLI not installed (optional)"
terraform --version || echo "ℹ️ Terraform not installed (optional)"If you cannot install anything (for example, on a locked-down corporate computer): you can follow practically the whole course by creating a repository on GitHub from the browser and editing the workflows through the web interface. GitHub's runners execute in the cloud. It is less convenient, but it works.
Common Mistakes and Tips
Mistake 1: wanting to automate before the commands work locally. If npm test does not work on your laptop, it will not work in CI. The pipeline is a revealer of hidden assumptions: environment variables only you have, services that have been running on your machine for weeks, files you never pushed to the repository. First make it work on a clean machine; then automate it.
Mistake 2: not pinning tool versions. .nvmrc, engines, postgres:16.3 instead of postgres:latest, dependencies without ^. Every unpinned version is a build that will one day stop working without anyone having touched anything, and that is the most expensive kind of failure to diagnose.
Mistake 3: different script names in each subproject. If apps/api uses npm test and apps/web uses npm run test:ci, your pipeline will fill up with special cases. Unify the names before writing the first workflow: it is half an hour of work that saves days.
Mistake 4: copying the production database into staging. It is tempting ("we test with real data") and it is a personal data leak waiting to happen. Reservalia stores end customers' phone numbers and email addresses. Generate fictional data with realistic volume.
Mistake 5: forgetting the healthcheck on auxiliary services. "Container started" is not "service ready". It is the number one cause of intermittent tests in CI, and as we saw in 01-02, flaky tests destroy confidence in the entire pipeline.
Tip 1: if you have a project of your own, use it in parallel. Apply each lesson to Reservalia and also to your project. Knowledge transfer is far greater when you face the peculiarities of your own code.
Tip 2: write your own Friday ritual table today. List every manual step in your current deployment process, with its duration. It is an uncomfortable list to read and it is exactly the map of what you are going to automate.
Tip 3: a README.md that actually works is the first step of CI. If a new developer can clone the repository and get the project running by following the README without asking anyone, your project is ready to be automated. If not, fix that first: the pipeline is precisely that README executed by a machine.
Exercises
Exercise 1: deduce the pipeline from the project
Without writing any YAML — it is not time for that yet — and using only what you know about Reservalia's repository, answer:
- In what order would you run
npm ci,lint,typecheck,test:unit,test:integrationandbuild? Justify the order. - Which of those steps can run in parallel and which must necessarily run in sequence?
- Which step needs a running PostgreSQL, and what does that imply for the runner?
- A change touches only
apps/web/src/pages/BusinessPanel.tsx. Would it be correct to run only theapps/webtests? And if the change touchedpackages/shared-types/src/index.ts?
Exercise 2: design the per-environment configuration matrix
Reservalia needs to decide, for each configuration value, whether it is a secret (it goes into the secrets manager), a normal environment variable (it can live in the repository) or something that must not exist in that environment. Complete the table and justify the three cases you find least obvious:
| Value | dev |
staging |
prod |
|---|---|---|---|
DATABASE_URL |
|||
| Database password | |||
LOG_LEVEL |
|||
| Payment gateway API key | |||
SMTP_HOST |
|||
| ECR repository name | |||
| Session token signing key |
Exercise 3: translate Reservalia to your technology
Imagine Reservalia were written in Python with FastAPI instead of Node.js with Express. Write:
- The equivalent structure of
apps/api/(names of configuration and dependency management files). - The equivalent of the pipeline's five commands (
npm ci,lint,typecheck,test,build). - Which parts of
docker-compose.ymlwould change and which would not. - Which parts of the course would stop applying. (Hint: few.)
Solutions
Solution to Exercise 1
1. Order and justification
1. npm ci ← essential first: with no dependencies there is nothing
2. lint · typecheck ← fast, catch obvious errors, need no infrastructure
3. test:unit ← fast, no database
4. test:integration ← slow, require PostgreSQL
5. build ← only makes sense if everything above is greenThe principle governing this order is called fail fast: put the fastest things and the things most likely to fail first. If Diego has left a console.log behind or an incompatible type, you want him to know in 40 seconds, not after waiting six minutes for the integration tests. Ordering the pipeline the other way round works just as well technically, but it wastes the team's time on every failure.
2. Parallel versus sequential
npm cimust go first and alone: everything depends on it.lint,typecheckandtest:unitcan run in parallel with one another: they are independent, share no state, and none of them needs another's result.test:integrationcan run in parallel with the above if the runner can bring up PostgreSQL at the same time, although it is usually run afterwards so as not to pay the cost of starting the database when something trivial has already failed. It is a trade-off decision between speed and cost.buildgoes last: building an artifact from code that does not pass the tests is time and money down the drain.
A nuance about typecheck in this particular monorepo: since apps/web depends on packages/shared-types, type checking the web app may require the shared package to be compiled. In projects using TypeScript project references this resolves itself; in others, it forces a partial build beforehand. It is the kind of detail you discover when building the pipeline and that we will resolve in 02-03.
3. PostgreSQL on the runner
test:integration needs it. Implications for the runner:
- The runner must be able to run containers (GitHub's hosted runners can).
- You have to wait for the database to be ready, not just started: a healthcheck or active waiting. Without this, flaky tests are guaranteed.
- You have to apply the migrations before running the tests, so that the schema exists.
- Every run must start from a clean database, or one test's state will contaminate the next.
4. Selective execution
- A change only in
apps/web/src/pages/BusinessPanel.tsx: yes, it would be reasonable to run only theapps/webtests. The API cannot be affected by a change in a React component. This is selective execution and it is one of the main optimisation levers (04-04). - A change in
packages/shared-types/src/index.ts: no. That package is a dependency of both applications, so both must be tested. This is precisely the case that makes badly implemented selective execution dangerous: if your rule is "I only test the folder that changed", a change in the shared package would test nothing and could break both applications at once. Selective execution must be based on the dependency graph, not on the file path.
Solution to Exercise 2
| Value | dev |
staging |
prod |
|---|---|---|---|
DATABASE_URL (without the password) |
Normal variable | Normal variable | Normal variable |
| Database password | Secret | Secret | Secret |
LOG_LEVEL |
Normal variable (debug) |
Normal variable (debug) |
Normal variable (info) |
| Payment gateway API key | Secret (test key) | Secret (test key) | Secret (live key) |
SMTP_HOST |
Normal variable | Normal variable | Normal variable |
| ECR repository name | Normal variable | Normal variable | Normal variable |
| Session token signing key | Secret | Secret | Secret, different from the other two |
The three least obvious cases:
DATABASE_URL. The trick is to separate the connection string from the password. postgres://[email protected]:5432/reservalia contains no secret: it is an internal host name that, without credentials and without network access, is of no use to anyone. The password is injected separately. Putting the password inside the URL is convenient and turns a public value into a secret, with everything that entails: it cannot be logged, it cannot go in the repository, it cannot be shown in an error message.
The payment gateway key in dev and staging. Even though they are test-environment keys that move no real money, they are still secrets: they allow calls to be made on Reservalia's behalf, data to be queried and quotas to be exhausted. A test key leaked in a public repository is a minor security incident, but it is an incident. Rule of thumb: if the provider calls it a "secret key", it is a secret.
Token signing key: different in each environment. This is the subtlest and most important case. If staging and prod shared the signing key, a token issued in staging — where the team has full access and can create whatever user they like — would be valid in production. It is a textbook privilege escalation. Cryptographic secrets are never shared between environments, not even "temporarily, just to test".
A general criterion for classifying: if leaking it lets somebody do something they should not, it is a secret. If it only reveals what things are called, it is configuration. We will come back to this in 04-03.
Solution to Exercise 3
1. Equivalent structure in Python/FastAPI
apps/api/
├── src/
│ ├── main.py entry point (instead of index.ts)
│ ├── routes/
│ ├── domain/
│ └── db/
│ └── migrations/ managed by Alembic
├── tests/
│ ├── unit/
│ └── integration/
├── pyproject.toml ← equivalent to package.json
├── poetry.lock ← equivalent to package-lock.json
├── .python-version ← equivalent to .nvmrc
├── alembic.ini migration configuration
└── Dockerfile2. The five commands
| Step | Node.js | Python/FastAPI |
|---|---|---|
| Install | npm ci |
poetry install --sync (respects the lock, like npm ci) |
| Lint | npm run lint |
ruff check src tests |
| Types | npm run typecheck |
mypy src |
| Tests | npm test |
pytest |
| Build | npm run build |
There is no compilation: the "build" is docker build itself |
| Migrations | npm run migrate |
alembic upgrade head |
An interesting observation: Python has no compilation step, so the artifact is produced directly when the Docker image is built. The pipeline has one step fewer, but the rest of the structure is identical.
3. Changes in docker-compose.yml
- Nothing changes in the
dbservice: PostgreSQL 16.3, its healthcheck, its volume and its variables are exactly the same. The database does not know which language the application is written in. - Nothing changes in the
mailpitservice: it captures SMTP, whoever the sender is. - The only thing that would change is the application service, if one were added: the base image (
python:3.12-sliminstead ofnode:20-slim) and the start command (uvicorn src.main:app --reload).
It is a good indicator of how much of this course is language-independent: the development infrastructure is practically the same.
4. Which parts of the course would stop applying
Almost none. Specifically:
- Stops applying: the exact npm commands, the
package.jsonsyntax and the details of the specific JavaScript tools (vitest, ESLint, tsc). - Applies in full: all the vocabulary from 01-01; the benefits and costs from 01-02; the tool map from 01-03; the DORA metrics from 01-05; the whole of module 3 (deployment strategies, IaC, feature flags, rollback, monitoring); the whole of module 4 except the syntax examples; the whole of module 6; and the structure, order and logic of every pipeline we build.
An honest estimate: more than 85% of the course is language-independent. What changes is the text strings inside the run: steps.
Conclusion
You now know the project we will be working on throughout the course:
- Reservalia is a SaaS appointment booking platform with 340 paying businesses, two applications (
apps/apiin Node.js + Express + PostgreSQL andapps/webin React + Vite), a shared types package and a monorepo with npm workspaces. - The team is Marta (tech lead, watches the business and the risk), Diego (backend, wants fast CI and is the only one who knows how to deploy) and Nuria (SRE, wants reproducible infrastructure and reliable rollback). Their tensions are the ones you will have in your team.
- The project already has its commands defined:
npm ci,lint,typecheck,test,buildandmigrate. This is the key idea of the lesson: the pipeline invents nothing, it merely runs on a clean machine what the project already knows how to do. A pipeline is a revealer of hidden assumptions. - There are three environments —
dev,stagingandprod— governed by two principles: the same artifact is promoted to all three, and production data never leaves production. - The target infrastructure is AWS: images in ECR, execution in ECS Fargate behind a load balancer, data in RDS PostgreSQL, secrets in Secrets Manager and logs in CloudWatch.
- And you know how to follow the course with any technology: more than 85% is language-independent; only the commands inside each step change.
One last thing remains before we start building. Marta is going to ask for the team's time to set all this up, and six months from now somebody will ask whether it was worth anything. To answer that, you need to have measured the starting point before touching anything. In the next lesson, DORA Metrics: How Software Delivery Is Measured, we will look at the industry's four standard metrics — deployment frequency, lead time for changes, change failure rate and time to restore service — how to calculate them from data the pipeline itself generates, and we will establish Reservalia's initial dashboard: where they are today and where they want to be by the end of the course.
CI/CD Course: Continuous Integration and Deployment
Module 1: Introduction to CI/CD
- Basic CI/CD Concepts
- Benefits of CI/CD
- Popular CI/CD Tools
- The Course Project: the Application We Are Going to Automate
- DORA Metrics: How Software Delivery Is Measured
Module 2: Continuous Integration (CI)
- Introduction to Continuous Integration
- Setting Up a CI Environment
- Build Automation
- Automated Testing
- Code Quality and Static Analysis
- Artifacts, Versioning and Promotion
- Integration with Version Control
Module 3: Continuous Deployment (CD)
- Introduction to Continuous Deployment
- Deployment Automation
- Infrastructure as Code and Reproducible Environments
- Deployment Strategies
- Feature Flags, Rollback and Failure Recovery
- Monitoring and Feedback
Module 4: Advanced CI/CD Practices
- CI/CD Pipelines
- Dependency Management
- Security in CI/CD
- Scalability and Performance
- Pipeline as Code: Templates, Reuse and Testing the Pipeline
- Databases in the Pipeline: Safe Migrations
Module 5: Implementing CI/CD in Real Projects
- Case Study: Web Project
- Case Study: Mobile Application
- Case Study: Microservices
- Case Study: Modernising a Legacy Project
Module 6: Tools and Technologies
- Jenkins
- GitLab CI/CD
- CircleCI
- Travis CI
- Docker and Kubernetes
- GitHub Actions in Depth
- Comparison and Criteria for Choosing a Tool
Module 7: Practical Exercises
- Exercise 1: Setting Up a Basic Pipeline
- Exercise 2: Integrating Automated Tests
- Exercise 3: Deploying to a Production Environment
- Exercise 4: Monitoring and Feedback
- Exercise 5: Hardening the Pipeline with Security and Secrets
- Final Project: A Complete End-to-End Pipeline
