You finished module 3 with a fifty-line bash script that brings Aurora Libros up. It works, but it describes how to reach the desired state: create the network, create the volume, start the database, wait in a loop, start the cache, the API and the web front end, each one with its fourteen lines of options.
Docker Compose turns that around. Instead of a sequence of orders, you write a file that states what the desired state is —four services, one network, one volume— and let Compose work out the steps. The file is called compose.yaml, it lives in the repository next to the code, it gets reviewed in a pull request, and it comes up with a single command.
In this lesson you will see exactly what problem Compose solves (and which one it does not), the bare minimum of YAML you need so the syntax never fights you, how Compose uses the concept of a project as a namespace, and you will write your first compose.yaml with two of the four Aurora Libros services.
Contents
- From fifty imperative lines to forty declarative ones
- Imperative versus declarative
- What Docker Compose is and what it is not
- Compose v2: a plugin of the Docker CLI
- The minimum YAML you need
- YAML syntax errors you are guaranteed to hit
- Anatomy of a
compose.yaml - The project as a namespace
- Your first
compose.yaml: database and cache up,ps,logs,down: the full cycle
- From fifty imperative lines to forty declarative ones
Take two fragments of the module 3 script, the database one and the cache one:
docker network create aurora-net
docker volume create aurora-data
docker run -d --name aurora-db --network aurora-net \
-e POSTGRES_USER=aurora -e POSTGRES_PASSWORD=aurora_secret \
-e POSTGRES_DB=aurora_books \
-p 127.0.0.1:5432:5432 \
--mount type=volume,src=aurora-data,dst=/var/lib/postgresql/data \
--memory 512m --cpus 1.0 --restart unless-stopped \
postgres:16-alpine
docker run -d --name aurora-cache --network aurora-net \
--memory 256m --cpus 0.5 --restart unless-stopped \
redis:7-alpine redis-server --maxmemory 200mb --maxmemory-policy allkeys-lruAnd now the same information in compose.yaml:
services:
aurora-db:
image: postgres:16-alpine
environment:
POSTGRES_USER: aurora
POSTGRES_PASSWORD: aurora_secret
POSTGRES_DB: aurora_books
ports:
- "127.0.0.1:5432:5432"
volumes:
- aurora-data:/var/lib/postgresql/data
restart: unless-stopped
aurora-cache:
image: redis:7-alpine
command: redis-server --maxmemory 200mb --maxmemory-policy allkeys-lru
restart: unless-stopped
volumes:
aurora-data:There is no docker network create: Compose creates a network for the project automatically. There is no docker volume create: declaring the volume is enough. There is no --name: the service name already identifies the container. And there is no execution order, because the file is not a sequence, it is a description.
- Imperative versus declarative
| Aspect | Imperative (docker run) |
Declarative (compose.yaml) |
|---|---|---|
| What you write | The sequence of steps | The desired end state |
| Who works out the path | You, in the right order | Compose, comparing reality with what you declared |
| Running it again | Fails: "container already exists" | It is idempotent: it does not touch what already matches |
| Changing one variable | Delete the container and rewrite its 14 lines | Edit one line and run up -d again |
| Stopping everything | Another script, with the names repeated | docker compose down |
| Reviewing changes | Impossible in a pull request | git diff on a text file |
| Source of truth | Whatever happens to be running on the machine | The versioned file |
| Configuration drift | Invisible: nobody knows whether somebody changed something by hand | up -d reconciles and returns to the declared state |
The key word is reconciliation. When you run docker compose up -d a second time, Compose does not create anything again: it compares the configuration of each running container with the one the file declares and only recreates what has changed. If you edit the cache's memory and run up -d, the cache is recreated and the other three services never notice.
- What Docker Compose is and what it is not
Compose is a tool for defining and running multi-container applications on a single Docker host. Its natural ground:
- Any team's development environment:
git cloneanddocker compose up -d. - Integration tests in CI, where you need a real, throwaway database.
- Reproducible demos and test environments.
- Small deployments on a single server, when high availability is not a requirement.
Compose is not a multi-node production orchestrator. It does not know how to spread containers across ten machines, it does not reschedule a service when the server hosting it dies, it does not do progressive deployments with automatic rollback, nor load balancing between replicas on different machines. All of that arrives in module 6 with Swarm and Kubernetes.
| Need | Compose | Orchestrator (module 6) |
|---|---|---|
| Several containers on one host | Yes, that is its whole reason for existing | Also, but with more complexity |
| Several hosts in a cluster | No | Yes |
| Rescheduling when a node goes down | No | Yes |
| Local scaling of replicas | up --scale, with limitations |
Yes, with real load balancing |
| Progressive deployment and rollback | No | Yes |
| Learning curve | One afternoon | Weeks |
The good news is that the compose.yaml you write here is not throwaway work: the concepts —service, image, variables, volumes, health probes— are the same ones you will meet in Kubernetes with a different syntax.
- Compose v2: a plugin of the Docker CLI
Compose started life as a standalone program written in Python that you invoked with docker-compose (with a hyphen) and that read a docker-compose.yml file. That is the old syntax. Since Compose v2, rewritten in Go, it is a plugin of the Docker CLI and you invoke it as a subcommand: docker compose, no hyphen. The preferred file name today is compose.yaml, and the old version: "3.8" key on the first line is obsolete: if you write it, Compose ignores it and warns you. Forget it exists.
Check your installation:
If that command fails but docker-compose version works, you have v1: install the plugin (docker-compose-plugin in the Docker repositories) before going any further. Docker Desktop ships with it.
Compose looks for the file in the current directory and upwards, in this order of preference:
| Order | File name |
|---|---|
| 1 | compose.yaml |
| 2 | compose.yml |
| 3 | docker-compose.yaml |
| 4 | docker-compose.yml (the historical name) |
Always use compose.yaml. The others exist for compatibility.
- The minimum YAML you need
YAML is a data serialization format designed to be read by people. You only need five ideas.
Indentation with spaces, never tabs. Hierarchy is expressed through indentation. Two spaces per level is the universal convention. A tab is a syntax error in YAML, no exceptions.
Maps (key-value pairs), separated by a colon and a space:
Lists, with a hyphen and a space. Every item at the same indentation level:
volumes:
- aurora-data:/var/lib/postgresql/data
- ./db/init.sql:/docker-entrypoint-initdb.d/init.sql:roNesting: maps inside maps and lists inside maps. That is how a whole compose.yaml is built: services is a map whose keys are service names, and the value of each one is another map with image, ports, and so on.
Scalars and quotes. YAML guesses the type: 5432 is an integer, true a boolean, 1.2 a float, postgres:16-alpine a string. You can use single or double quotes; double quotes interpret escape sequences (\n), single ones do not. The practical rule in Compose:
| Situation | Write | Why |
|---|---|---|
| Ports | "8080:80" |
Unquoted, YAML 1.1 may read it as a sexagesimal number |
Values starting with *, &, %, @, ` |
Quoted | Those are reserved characters |
The strings yes/no/on/off/true |
"true" |
Unquoted they turn into booleans |
Versions like 1.2.0 |
"1.2.0" |
Unquoted it may lose the last digit |
| Ordinary image names | Unquoted | There is no ambiguity |
Comments: everything after # to the end of the line. Use them freely; a well-commented compose.yaml is living documentation.
For long strings, YAML offers two block styles: | keeps the line breaks and > turns them into spaces. You will see them in command and in healthcheck.
- YAML syntax errors you are guaranteed to hit
| Error | Symptom | Fix |
|---|---|---|
| A tab used for indentation | found character '\t' that cannot start any token |
Configure your editor: 2 spaces |
Missing space after : |
mapping values are not allowed here |
image: nginx, not image:nginx |
| Inconsistent indentation | A key "disappears" or moves to another service | Line up the whole block at the same level |
8080:80 without quotes |
The published port is not the one you expected | Always quote ports |
| Mixing list and map | expected <block end> |
Either all hyphens, or all key: value |
| A duplicated key | The second value silently wins | Check with docker compose config |
That last command is your safety net: docker compose config reads the file, validates it, resolves variables and hands you back the final configuration exactly as Compose understands it. Run it whenever something does not add up.
- Anatomy of a
compose.yaml
compose.yamlA Compose file has four commonly used top-level sections:
services: # MANDATORY: the application's containers
...
volumes: # named volumes managed by Compose
...
networks: # custom networks
...
secrets: # files with sensitive data (lesson 04-05)
...Only services is mandatory. The others exist because a volume or a network does not belong to a service: they are shared resources that several services use, and that is why they are declared separately and referenced from inside each service.
graph TD
F["compose.yaml"] --> S["services:"]
F --> V["volumes:"]
F --> N["networks:"]
S --> C1["container<br/>aurora-libros-aurora-db-1"]
S --> C2["container<br/>aurora-libros-aurora-cache-1"]
V --> VOL["volume<br/>aurora-libros_aurora-data"]
N --> NET["bridge network<br/>aurora-libros_default"]
C1 -.connected to.-> NET
C2 -.connected to.-> NET
C1 -.mounts.-> VOL
Every line of the file becomes real Docker objects you already know everything about: containers, bridge networks and volumes. Compose invents nothing new underneath; it talks to the same daemon and the same API you were using with docker run. You can still inspect it all with docker ps, docker network inspect or docker volume ls.
- The project as a namespace
Compose groups everything it creates under a project. The default project name is that of the directory containing the file, in lowercase and with odd characters stripped. If your file is in ~/aurora-libros/, the project is called aurora-libros.
That name is used as a prefix:
| Object | Name pattern | Example |
|---|---|---|
| Container | <project>-<service>-<number> |
aurora-libros-aurora-db-1 |
| Default network | <project>_default |
aurora-libros_default |
| Declared network | <project>_<name> |
aurora-libros_backend |
| Named volume | <project>_<name> |
aurora-libros_aurora-data |
On top of that, Compose labels every object with com.docker.compose.project=<project> and com.docker.compose.service=<service>. That is how it knows, on a machine with twenty containers, which ones are its own.
Prefixing has one very useful practical consequence: two different projects can use the same compose.yaml without stepping on each other. Three ways to set the project name, from highest to lowest priority:
docker compose -p aurora-tests up -d # 1. CLI option
COMPOSE_PROJECT_NAME=aurora-tests docker compose up -d # 2. environment variableSetting name: in the file is good practice: it guarantees the project is called the same thing even if somebody clones the repository into a folder with a different name.
- Your first
compose.yaml: database and cache
compose.yaml: database and cacheYou are going to start small: just aurora-db and aurora-cache. The API and the web front end arrive in lesson 04-04, once you know how to declare dependencies and health probes.
mkdir -p ~/aurora-libros && cd ~/aurora-libros
docker rm -f aurora-db aurora-cache aurora-api aurora-web 2>/dev/null
docker network rm aurora-net 2>/dev/nullYou have deleted the module 3 containers so there are no name or port clashes. The aurora-data volume is still intact, but Compose will create its own prefixed one, so you will start with a clean database populated by db/init.sql.
Create ~/aurora-libros/compose.yaml:
# compose.yaml — Aurora Libros S.L. (minimal version: data and cache)
name: aurora-libros
services:
aurora-db:
image: postgres:16-alpine
environment:
POSTGRES_USER: aurora
POSTGRES_PASSWORD: aurora_secret
POSTGRES_DB: aurora_books
ports:
- "127.0.0.1:5432:5432" # reachable only from the host itself
volumes:
- aurora-data:/var/lib/postgresql/data
- ./db/init.sql:/docker-entrypoint-initdb.d/init.sql:ro
restart: unless-stopped
aurora-cache:
image: redis:7-alpine
command: redis-server --maxmemory 200mb --maxmemory-policy allkeys-lru
restart: unless-stopped
volumes:
aurora-data:Three details worth your attention:
./db/init.sql: relative paths are resolved from the file's directory, not from yourcwd. That is why the$HOMEof the bash script disappears and the file becomes portable.- There is no
networks:: Compose createsaurora-libros_defaultand connects both services there, with internal DNS —exactly like the user-definedaurora-netnetwork you created by hand in lesson 03-05. - There is no
--name: you do not need one. The service name is the DNS name.
Validate before starting:
up, ps, logs, down: the full cycle
up, ps, logs, down: the full cycle[+] Running 4/4
✔ Network aurora-libros_default Created
✔ Volume "aurora-libros_aurora-data" Created
✔ Container aurora-libros-aurora-db-1 Started
✔ Container aurora-libros-aurora-cache-1 StartedThere it is, everything the declarative model promises: the network and the volume created on their own, with their project prefix, and both containers running. Not one line of docker network create.
NAME IMAGE SERVICE STATUS PORTS
aurora-libros-aurora-cache-1 redis:7-alpine aurora-cache Up 12 seconds 6379/tcp
aurora-libros-aurora-db-1 postgres:16-alpine aurora-db Up 12 seconds 127.0.0.1:5432->5432/tcpLook at the SERVICE column: that is the name you will use in every command, not the container's. Check that these objects are plain Docker:
NETWORK ID NAME DRIVER SCOPE
b3f1a9c04e77 aurora-libros_default bridge local
DRIVER VOLUME NAME
local aurora-libros_aurora-dataThe logs, aggregated and color-coded per service:
aurora-db-1 | PostgreSQL init process complete; ready for start up.
aurora-db-1 | LOG: database system is ready to accept connections
aurora-cache-1 | * Ready to accept connections tcpVerify that init.sql ran and that DNS works between services:
docker compose exec aurora-db psql -U aurora -d aurora_books -c "SELECT count(*) FROM books;"
docker compose exec aurora-cache redis-cli -h aurora-cache PINGNine books —the original eight plus El Aleph— and the cache answering by its service name. Now the part no bash script ever gave you for free:
[+] Running 3/3
✔ Container aurora-libros-aurora-cache-1 Removed
✔ Container aurora-libros-aurora-db-1 Removed
✔ Network aurora-libros_default RemovedOne command for the entire platform, and it cleans up the network too. The volume has not been deleted: down preserves data by design. Check it and bring everything back up:
docker volume ls --filter name=aurora-libros --format "{{.Name}}"
docker compose up -d
docker compose exec aurora-db psql -U aurora -d aurora_books -c "SELECT count(*) FROM books;"The nine books are still there. And if you now run docker compose up -d a third time, you will see reconciliation in action:
Running, not Recreated: nothing has changed in the file, so Compose touches nothing. That is idempotency.
Common Mistakes and Tips
Using tabs to indent. The most frequent error and the most baffling, because visually you cannot tell. Configure your editor to insert spaces in .yaml files and turn on the display of invisible characters.
Writing version: "3.8" on the first line. It has been obsolete since Compose v2. Compose ignores it and prints a warning. Delete it.
Using docker-compose with a hyphen. That is v1, unmaintained since 2023. Everything you learn here works with docker compose; the other way round, it does not.
Believing that down deletes your data. It does not delete named volumes... unless you add -v. That -v is the most dangerous command in Compose and it gets a detailed treatment in lesson 04-03.
Running commands from another folder. Compose looks for the file from the current directory upwards. If you launch docker compose ps from /tmp it will find nothing, or worse, it will find another project. Use -f ~/aurora-libros/compose.yaml when you are not in the right place.
Forgetting that the project name depends on the folder. Two people who clone the repository into folders with different names get different projects. Set name: in the file.
Not quoting ports. - 8080:80 looks harmless, but it is exactly the kind of bug that costs you half an hour. Always quote them.
Tip: run docker compose config after every non-trivial edit. It is free, instant, and it saves you from bringing up a broken stack.
Exercises
Exercise 1. The following fragment has five errors (either YAML syntax or Compose usage). Find them, explain each one and write the corrected version.
version: "3.8"
services:
aurora-cache:
image:redis:7-alpine
ports:
- 6379:6379
restart: unless-stopedExercise 2. Without deleting your current project, bring up a second isolated copy of the same stack under the project name aurora-tests, check that its containers, network and volume are independent, and explain why it fails if you change nothing else. Then remove the copy completely, volumes included.
Exercise 3. Prove with commands that Compose creates nothing magical: starting from the running project, obtain (a) the labels Compose has put on the cache container, (b) the real network name and the address range it hands out, and (c) the volume's path on the host.
Solutions
Solution 1.
| # | Error | Explanation |
|---|---|---|
| 1 | version: "3.8" |
Key obsolete since Compose v2; it is ignored and triggers a warning |
| 2 | image:redis:7-alpine |
Missing space after the colon: YAML does not read it as a map |
| 3 | image badly indented |
It sits at the same level as aurora-cache, not inside the service |
| 4 | - 6379:6379 |
Unquoted port: risk of numeric interpretation |
| 5 | unless-stoped |
Invalid value, missing a p. The four valid ones are no, always, on-failure, unless-stopped |
name: aurora-libros
services:
aurora-cache:
image: redis:7-alpine
ports:
- "6379:6379"
restart: unless-stoppedSolution 2.
Error response from daemon: driver failed programming external connectivity:
Bind for 127.0.0.1:5432 failed: port is already allocatedCompose's isolation reaches containers, networks and volumes, but not the host's ports: there is only one 127.0.0.1:5432, and the original project already holds it. The cleanest fix is not to publish a port in the copy; the services talk to each other over the internal network just the same.
# Copy of the file without the "ports" section
sed '/127.0.0.1:5432/d; /ports:/d' compose.yaml > compose.tests.yaml
docker compose -p aurora-tests -f compose.tests.yaml up -d
docker ps --format "table {{.Names}}\t{{.Ports}}" | grep aurora
docker volume ls --format "{{.Name}}" | grep auroraaurora-tests-aurora-db-1 5432/tcp
aurora-tests-aurora-cache-1
aurora-libros-aurora-db-1 127.0.0.1:5432->5432/tcp
aurora-libros-aurora-cache-1
aurora-libros_aurora-data
aurora-tests_aurora-dataTwo complete stacks on the same machine, each one with its own prefixed volume, without sharing a single byte. Full cleanup of the copy:
down -v does delete the project's named volumes. Here that is what you want; in the real project, almost never.
Solution 3.
# (a) labels that Compose adds
docker inspect aurora-libros-aurora-cache-1 \
--format '{{range $k, $v := .Config.Labels}}{{$k}}={{$v}}{{"\n"}}{{end}}' \
| grep com.docker.composecom.docker.compose.project=aurora-libros
com.docker.compose.service=aurora-cache
com.docker.compose.container-number=1
com.docker.compose.oneoff=False
com.docker.compose.config-hash=7f3c1a...That config-hash is the key to reconciliation: it is the digest of the declared configuration. When you run up -d, Compose recalculates it and only recreates the container if the hash has changed.
# (b) network and subnet
docker network inspect aurora-libros_default \
--format '{{.Name}} | {{(index .IPAM.Config 0).Subnet}} | driver={{.Driver}}'
# (c) the volume's path on the host
docker volume inspect aurora-libros_aurora-data --format '{{.Mountpoint}}'aurora-libros_default | 172.20.0.0/16 | driver=bridge
/var/lib/docker/volumes/aurora-libros_aurora-data/_dataA user-defined bridge network and a local volume in the usual path. Compose is a convenience layer over the same API you already knew: everything you learned in module 3 is still valid for inspecting and debugging.
Conclusion
You have changed paradigm. A bash script describes how to reach the desired state and breaks if you run it twice; a compose.yaml describes what that state is, it is idempotent, and it lives versioned in Git next to the code. You know that Compose v2 is a plugin of the CLI (docker compose, no hyphen), that the file is called compose.yaml and that version: is obsolete. You have a grip on the YAML you need —spaces and never tabs, maps, lists, quotes around ports— and you recognize the six syntax errors you will hit over and over.
You understand that Compose organizes everything under a project that acts as a namespace: it prefixes containers, networks and volumes, labels them with com.docker.compose.* and lets two identical stacks live side by side on the same machine. And you have checked with your own hands that the network and the volume are created and destroyed on their own, that down respects your data, and that a repeated up -d recreates nothing because it compares the config-hash of what you declared with what is running.
Your two-service compose.yaml works, but it only uses a handful of keys. In the next lesson, Defining Services in Docker Compose, you will get the full practical reference: every block of a service —image and build, identity, execution, network and ports, data, health and dependencies, resources and restart, labels— with its exact equivalent among the docker run options you already know, plus the top-level volumes: and networks: blocks and YAML anchors to avoid repeating configuration. By the end of that lesson you will have the four Aurora Libros services declared in a single file.
Docker: From Beginner to Advanced
Module 1: Introduction to Docker
- What Is Docker?
- Installing Docker
- Docker Architecture
- Basic Docker Commands
- Understanding Docker Images
- Creating Your First Docker Container
- The Course Project: The Aurora Libros Platform
Module 2: Working with Docker Images
- Docker Hub and Repositories
- Building Docker Images
- Dockerfile Basics
- Advanced Dockerfile Instructions
- Managing Docker Images
- Tagging and Publishing Images
Module 3: Docker Containers
- Running Containers
- Container Lifecycle
- Managing Containers
- Inspecting and Debugging Containers
- Docker Networking
- Data Persistence with Volumes
- Resource Limits and Restart Policies
Module 4: Docker Compose
- Introduction to Docker Compose
- Defining Services in Docker Compose
- Docker Compose Commands
- Multi-Container Applications
- Environment Variables in Docker Compose
- Profiles, Overrides and Multiple Environments
- Local Development with Docker Compose
Module 5: Advanced Docker Concepts
- Docker Networking Deep Dive
- Docker Storage Options
- Docker Security Best Practices
- Optimizing Docker Images
- Advanced Builds with BuildKit and Buildx
- Logging and Monitoring in Docker
- The Runtime Inside: Namespaces, Cgroups and Layers
Module 6: Docker in Production
- Preparing an Image for Production
- CI/CD with Docker
- Orchestrating Containers with Docker Swarm
- Introduction to Kubernetes
- Deploying Docker Containers in Kubernetes
- Scaling and Load Balancing
- Deployment Strategies and Rollback
