With the file written, it is time to master the tool that interprets it. The Compose CLI has around twenty-five subcommands, but the workload is very unevenly spread: you will use five daily, another five often, and the rest in specific situations.
This lesson walks through them by family, always on top of the Aurora Libros compose.yaml from the previous lesson. Pay particular attention to two pairs that get confused endlessly: up versus start, and exec versus run.
Contents
- Master table of subcommands
up: the command that reconcilesdownand the danger of-vstart,stop,restart,pause,create- Observation:
ps,logs,top,stats,events - One-off execution:
execversusrun - Building and publishing:
build,pull,push - Local scaling with
--scaleand its limits - Diagnosis:
configand validation in CI - Selecting files and project:
-f,-p,COMPOSE_FILE - Autocompletion and useful aliases
- Master table of subcommands
| Family | Command | What it does |
|---|---|---|
| Lifecycle | up |
Creates or updates and starts everything declared |
down |
Stops and removes the project's containers and networks | |
create |
Creates the containers without starting them | |
start / stop |
Starts or stops existing containers | |
restart |
Stops and starts again without re-reading the file | |
pause / unpause |
Freezes and unfreezes the processes (SIGSTOP) | |
kill |
Sends a signal (SIGKILL by default) | |
rm |
Removes stopped containers | |
| Observation | ps |
State of the project's services |
logs |
Logs aggregated by service | |
top |
Processes inside each container | |
stats |
Live resource usage | |
events |
Stream of project events | |
port |
The host port bound to an internal port | |
| Execution | exec |
A command in a running container |
run |
A command in a new container | |
| Images | build |
Builds the services that have build: |
pull / push |
Pulls or publishes the images | |
images |
Images used by the project | |
| Diagnosis | config |
Validates and prints the final configuration |
version |
Compose version | |
ls |
Lists all the projects on the machine | |
cp |
Copies files between host and service | |
wait |
Blocks until a service finishes |
One command people forget and that is extremely useful: docker compose ls works from any directory and tells you which projects are up on the machine and with which file.
up: the command that reconciles
up: the command that reconcilesdocker compose up is 80% of daily usage. It is not "start": it is bringing the real state to the declared state. On each run, for each service, Compose computes a digest of its configuration (the config-hash you saw in lesson 04-01) and decides:
| Situation | Compose's action |
|---|---|
| The container does not exist | Creates and starts it |
| It exists and the configuration matches | Leaves it alone |
| It exists but the file or the image changed | Recreates it |
| It exists but is stopped | Starts it |
| It exists and is not in the file | Leaves it (unless --remove-orphans) |
That selectivity is what makes editing the cache's memory and running up -d recreate only the cache, without tearing down the database.
| Option | Effect |
|---|---|
-d, --detach |
In the background. In practice, always |
--build |
Builds before starting the services that have build: |
--no-build |
Fails if an image is missing instead of building it |
--pull always |
Pulls the image even if it exists locally |
--force-recreate |
Recreates everything even if nothing changed |
--no-recreate |
Recreates nothing even if something changed |
--no-deps |
Ignores depends_on: only the requested service |
--wait |
Waits until the services are healthy before returning control |
--wait-timeout 60 |
Maximum seconds --wait will wait |
--remove-orphans |
Removes project containers that are no longer in the file |
--abort-on-container-exit |
Stops everything if a container exits (useful in CI) |
--scale s=N |
Starts N replicas of service s |
[+] Building 2/2
[+] Running 4/4
✔ Container aurora-libros-aurora-db-1 Healthy
✔ Container aurora-libros-aurora-cache-1 Healthy
✔ Container aurora-libros-aurora-api-1 Healthy
✔ Container aurora-libros-aurora-web-1 Healthy--wait is the option that turns Compose into an automation tool: without it, the command returns control as soon as the containers are started, and a CI script that launches tests immediately afterwards crashes into a database that is not accepting connections yet. With --wait, Compose does not return control until every health probe passes, and it exits with a non-zero code if any of them fails to.
Without -d, up leaves your terminal attached to the aggregated logs of every service and Ctrl+C stops the whole stack. That is handy for debugging a startup; dangerous if you forget about it in an SSH session.
A very useful case for --no-deps: recreating just the API without restarting the database.
down and the danger of -v
down and the danger of -vdown is the inverse of up: it stops and removes the project's containers and networks.
| Command | Containers | Networks | Named volumes | Anonymous volumes | Images |
|---|---|---|---|---|---|
down |
Removes | Removes | Keeps | Removes | Keeps |
down -v |
Removes | Removes | DELETES | Removes | Keeps |
down --rmi local |
Removes | Removes | Keeps | Removes | Deletes those without a tag of their own |
down --rmi all |
Removes | Removes | Keeps | Removes | Deletes all the project's |
down --remove-orphans |
Removes orphans too | Removes | Keeps | Removes | Keeps |
stop |
Keeps (stopped) | Keeps | Keeps | Keeps | Keeps |
Burn the second row into your memory. docker compose down -v deletes the project's named volumes without asking: in Aurora Libros, that is the bookshop's entire catalog. It is exactly what you want when resetting a development environment and exactly what ruins a server if you type it out of habit. Two defenses: declare critical volumes as external: true (lesson 04-02) and never create a shell alias that includes -v.
start, stop, restart, pause, create
start, stop, restart, pause, create| Command | Does it read the file? | Does it recreate? | When to use it |
|---|---|---|---|
up -d |
Yes | If there are changes | Whenever you touch the compose.yaml |
start |
No | Never | Starting up what already exists |
stop |
No | No | Stopping without destroying |
restart |
No | No | Restarting the process, without applying changes |
pause |
No | No | Freezing processes, releasing CPU but not memory |
create |
Yes | If there are changes | Creating without starting |
The restart trap deserves an explicit warning: it does not re-read the file. If you change an environment variable and run docker compose restart aurora-api, the container restarts with the old configuration and you go mad wondering why the change is not being applied. To apply changes from the file, always up -d.
docker compose stop aurora-cache # stop one specific service
docker compose start aurora-cache # start it again
docker compose restart aurora-web # quick nginx restart
docker compose kill -s SIGHUP aurora-web # reload the configuration without restarting
- Observation:
ps, logs, top, stats, events
ps, logs, top, stats, eventsdocker compose ps
docker compose ps -a # includes stopped and exited ones
docker compose ps --services # service names only, one per line
docker compose ps --status running
docker compose ps --format "table {{.Service}}\t{{.Status}}\t{{.Ports}}"
docker compose ps --format json | jq -r '.[] | "\(.Service): \(.Health)"'--services is gold for scripts: for s in $(docker compose ps --services); do ...; done.
Logs are the tool you will use most:
docker compose logs # every service, aggregated
docker compose logs -f aurora-api # follow one in real time
docker compose logs --tail 50 aurora-db # last 50 lines
docker compose logs --since 10m # from the last 10 minutes
docker compose logs -t --no-color api web # with timestamps, several servicesCompose colors and prefixes every line with the service name, so docker compose logs -f on the whole stack lets you watch a request flow through web → api → db. Remember from lesson 03-04 that you will only see what the processes write to stdout/stderr.
docker compose top aurora-db # processes inside the container
docker compose stats --no-stream # point-in-time usage for every service
docker compose events --json # stream of project events
docker compose port aurora-web 80port answers "which host port is the web front end's 80 published on?", and it is indispensable when you let Docker assign random ports.
- One-off execution:
exec versus run
exec versus runThis is the distinction that causes the most confusion, and the answer is simple: exec enters a container that is already running; run creates a new one.
| Aspect | exec |
run |
|---|---|---|
| Container | The service's existing one | A new one, with a -run-<hash> suffix |
| Requirement | The service must be running | The service does not need to be started |
| Dependencies | Irrelevant | Starts the depends_on ones (avoid with --no-deps) |
| Ports | The ones it already has | It does not publish the ports (unless --service-ports) |
| When it finishes | The container stays alive | It stays stopped, unless --rm |
| Typical use | Inspecting, psql, redis-cli, a shell |
One-off tasks: migrations, tests, seeds |
# EXEC: talk to the database that is serving right now
docker compose exec aurora-db psql -U aurora -d aurora_books
docker compose exec aurora-db psql -U aurora -d aurora_books -c "SELECT count(*) FROM books;"
docker compose exec aurora-cache redis-cli INFO keyspace
docker compose exec aurora-api sh # shell inside the API
docker compose exec -u root aurora-api sh # as root, to install something
docker compose exec -T aurora-db pg_dump -U aurora aurora_books > backup.sql-T disables pseudo-TTY allocation: mandatory when you redirect the output to a file or chain pipes, or the dump will come out full of control characters.
# RUN: new, throwaway containers
docker compose run --rm aurora-api npm test
docker compose run --rm --no-deps aurora-api node -e "console.log(process.version)"
docker compose run --rm --entrypoint sh aurora-api
docker compose run --rm -e NODE_ENV=test aurora-api npm run test:integration
docker compose run --rm --service-ports aurora-api # publishes its ports> [email protected] test
> node --test
✔ GET /health returns 200 (12.4ms)
✔ GET /books returns 9 titles (31.7ms)
✔ GET /books/:id for a missing id returns 404 (4.1ms)
ℹ pass 3--rm almost always. Without it, every run leaves a stopped container behind on the machine; after a week you have forty aurora-libros-aurora-api-run-a3f9c2 containers taking up space. And --no-deps when the task does not need the stack: without it, an innocent run brings up PostgreSQL and Redis for you.
- Building and publishing:
build, pull, push
build, pull, pushdocker compose build # builds the services with build:
docker compose build --no-cache aurora-api # ignores the layer cache
docker compose build --pull # refreshes the base image first
docker compose build --progress plain # full output, useful for debugging
docker compose pull # pulls the declared images
docker compose pull --ignore-buildable # skips the services that are built
docker compose push aurora-api # publishes to the registry
docker compose imagesCONTAINER REPOSITORY TAG SIZE
aurora-libros-aurora-api-1 auroralibros/aurora-api 1.2.0 142MB
aurora-libros-aurora-cache-1 redis 7-alpine 41.4MB
aurora-libros-aurora-db-1 postgres 16-alpine 274MB
aurora-libros-aurora-web-1 nginx alpine 48.3MBThe typical CI flow, which you will see in full in lesson 06-02, fits in three lines: docker compose build, docker compose push and, on the server, docker compose pull && docker compose up -d.
- Local scaling with
--scale and its limits
--scale and its limitsError response from daemon: driver failed programming external connectivity:
Bind for 0.0.0.0:3000 failed: port is already allocatedThere is the first clash: a fixed host port cannot be shared between three replicas. There is only one 3000 on the machine. And if the service had container_name, it would fail even earlier, because two containers cannot share a name.
| Obstacle | Why it clashes | Fix |
|---|---|---|
ports: ["3000:3000"] |
A single host port | Remove the port or use - "3000" (random) |
container_name |
Duplicate names | Do not use container_name |
| A shared data volume | Two Postgres instances over the same files | Do not scale stateful services |
Remove the fixed publication and it works:
docker compose up -d --scale aurora-api=3
docker compose ps --services --filter status=running | sort | uniq -c
docker compose exec aurora-web getent hosts aurora-apiDocker's internal DNS returns all three addresses for the same name, and the client picks one. It is a rudimentary distribution —no health checking, no weights, no retries— good enough for local testing, not for production. Real load balancing, with its strategies and its handling of dead replicas, is lesson 06-06.
- Diagnosis:
config and validation in CI
config and validation in CIdocker compose config reads every file involved, interpolates variables, merges overrides and prints the final configuration exactly as Compose understands it. It is your only way of seeing the truth.
docker compose config # full resolved configuration
docker compose config --quiet # validate only: no output, exit code 0 or 1
docker compose config --services # service names
docker compose config --volumes # volume names
docker compose config --images # images that will be used
docker compose config --no-interpolate # leaves the ${...} unresolved
docker compose config --format json | jq '.services["aurora-api"].deploy'In a CI pipeline, --quiet is the first check that should run, even before building anything:
Two warnings about config: it resolves variables, so its output can contain passwords in the clear (never dump it into a public log), and it shows the normalized result, with the short forms converted into long forms. That normalization is exactly what you need in order to understand what a merge of anchors or files actually did.
- Selecting files and project:
-f, -p, COMPOSE_FILE
-f, -p, COMPOSE_FILEGlobal options go before the subcommand:
docker compose -f ~/aurora-libros/compose.yaml ps # correct
docker compose ps -f ~/aurora-libros/compose.yaml # ERROR| Option | What it does |
|---|---|
-f, --file |
File to use. Repeatable, and the order matters |
-p, --project-name |
Project name (prefix for every object) |
--project-directory |
Base directory for resolving relative paths |
--profile |
Activates a profile (lesson 04-06) |
--env-file |
Variables file for interpolation (lesson 04-05) |
With several -f, Compose merges the files in the given order and the last one wins in case of conflict. The full merge mechanism, with its rules per key type, is lesson 04-06.
One surprising detail: when you use -f, the base directory for relative paths becomes that of the first file. If your files are spread around, --project-directory lets you set it explicitly.
And the equivalent environment variable, useful for not repeating -f a hundred times a day:
export COMPOSE_FILE=compose.yaml:compose.prod.yaml # ":" separator on Linux/macOS
export COMPOSE_PROJECT_NAME=aurora-prod
docker compose up -d # uses both files and the project name
- Autocompletion and useful aliases
# Bash: completion for subcommands, services and options
docker completion bash | sudo tee /etc/bash_completion.d/docker > /dev/null
# Zsh
docker completion zsh > "${fpath[1]}/_docker"With completion switched on, docker compose logs -f aur<TAB> offers you the project's real services: no more mistyped names.
alias dc='docker compose'
alias dcu='docker compose up -d'
alias dcl='docker compose logs -f --tail 100'
alias dcp='docker compose ps'
alias dce='docker compose exec'
# There is deliberately NO alias containing "down -v"That last comment is not a joke: most data losses with Compose come from a short alias somebody wrote with -v "to clean up quickly".
Common Mistakes and Tips
Using restart expecting it to apply changes from the file. It does not. A change in the compose.yaml → up -d, always.
Confusing stop with down. stop keeps the containers, down removes them along with the network. If after a down you expected to find the container stopped, it is not there.
Typing down -v out of habit. It deletes the named volumes and the data with them. Think about it every single time.
Forgetting --rm in run. Every execution leaves a stopped container behind. Clean up the accumulated ones with docker compose rm -f.
Running exec with redirection and no -T. The pseudo-TTY inserts control characters and corrupts dumps and binary files.
Putting -f after the subcommand. Global options go before it; the error the CLI returns does not always make that clear.
Tip: when something does not behave as you expect, the diagnostic order is docker compose config (what does Compose understand?), docker compose ps (what is running?) and docker compose logs (what does the service say?). In that order you solve almost everything.
Exercises
Exercise 1. With the stack up, change aurora-cache's memory limit from 256M to 320M and apply it without the other three services restarting. Prove with commands that only the cache was recreated and that the new limit is in effect.
Exercise 2. Write a verify.sh script suitable for CI that: validates the file, brings the stack up waiting until it is healthy, runs the API tests in a throwaway container, prints the state of the four services and cleans up everything, volumes included, returning the tests' exit code.
Exercise 3. Without shutting the stack down, obtain: (a) the number of keys in Redis, (b) the first three catalog titles in alphabetical order, and (c) a dump of the books table into the host file ~/books.sql. Explain which option is indispensable in the third case and why.
Solutions
Solution 1.
aurora-api: 22 minutes ago
aurora-cache: 22 minutes ago
aurora-db: 22 minutes ago
aurora-web: 22 minutes agoEdit the limit in the compose.yaml (limits: { memory: 320M, cpus: "0.5", pids: 100 }) and apply it:
✔ Container aurora-libros-aurora-db-1 Running
✔ Container aurora-libros-aurora-cache-1 Recreated
✔ Container aurora-libros-aurora-api-1 Running
✔ Container aurora-libros-aurora-web-1 RunningThere is reconciliation: Recreated only on the cache, Running on the other three because their config-hash did not change. Verification:
docker compose ps --format "{{.Service}}: {{.RunningFor}}"
docker inspect aurora-libros-aurora-cache-1 --format '{{.HostConfig.Memory}}'The cache is four seconds old and the others still show their twenty-two minutes. And 335544320 bytes is exactly 320 MiB. Careful: recreating a container is not restarting it, it is destroying it and creating another one; any data not held in a volume is lost. In Redis, that means an empty cache, which is harmless here because /books repopulates it from the database.
Solution 2.
#!/usr/bin/env bash
# verify.sh — local pipeline for Aurora Libros
set -uo pipefail
cd "$(dirname "$0")"
docker compose config --quiet || { echo "invalid compose.yaml"; exit 1; }
docker compose up -d --build --wait --wait-timeout 90 || {
echo "The stack never reached a healthy state"
docker compose logs --tail 40
docker compose down -v
exit 1
}
docker compose run --rm --no-deps aurora-api npm test
exit_code=$?
docker compose ps --format "table {{.Service}}\t{{.Status}}"
docker compose down -v --remove-orphans
exit $exit_codeFour important decisions: --quiet validates before spending time building; --wait with --wait-timeout guarantees the tests do not start against a database that is not answering yet and fails with a non-zero code if it never does; --rm --no-deps on the tests avoids leaving junk behind and does not bring up dependencies that are already healthy; and down -v is correct and desirable here, because this is a throwaway environment. Notice that $? is saved before the cleanup commands: otherwise the exit code the script returns would be down's, and the pipeline would always come out green.
Solution 3.
# (a) keys in the cache
docker compose exec aurora-cache redis-cli DBSIZE
# (b) first three titles
docker compose exec aurora-db psql -U aurora -d aurora_books \
-c "SELECT title FROM books ORDER BY title LIMIT 3;"
# (c) dump to the host
docker compose exec -T aurora-db pg_dump -U aurora -t books aurora_books > ~/books.sql(integer) 1
title
------------------------------------------
Cien años de soledad
El Aleph
El jardín de senderos que se bifurcan
(3 rows)In the third case -T is indispensable. Without it, Compose allocates a pseudo-TTY to the command and the output stream stops being clean binary: line breaks get translated and control sequences slip in, so the resulting .sql may not be importable again. The rule is simple: if you are not going to read the output with your own eyes, use -T.
Conclusion
You now handle the full CLI. You know that up is not "start" but "reconcile": it compares each service's configuration digest with the declared one and recreates only what changed, which lets you tweak a limit and touch a single container. You know its decisive options —-d, --build, --pull, --force-recreate, --no-deps, --remove-orphans and above all --wait, which waits for the health probes to pass and fails if they do not. And you are clear on what each variant of down deletes, with -v flagged in red.
You tell start/stop/restart/pause apart from up/down, with the trap that restart does not re-read the file. You observe with ps (including --services and --format json), logs, top, stats, events and port. And you have internalized the key difference: exec enters what is already running, run creates a new container, with --rm so you do not pile up junk, --no-deps so you do not bring up half the stack, and -T when you redirect the output. You build and publish with build/pull/push, you scale locally with --scale knowing that it clashes with fixed ports and with container_name, you validate with config --quiet in CI, and you control which file and which project you are using with -f, -p and COMPOSE_FILE.
In the next lesson, Multi-Container Applications, all of this goes into service of the definitive assembly: the Aurora Libros architecture with two segmented networks —a frontend one and a backend one marked as internal—, Compose's DNS and the table of who talks to whom and over which port, startup order solved with depends_on and condition: service_healthy using pg_isready and redis-cli ping, a migration service with service_completed_successfully, and the retry-with-backoff pattern you still need even with all of the above in place. By the end, the fifteen onboarding steps will be down to two.
Docker: From Beginner to Advanced
Module 1: Introduction to Docker
- What Is Docker?
- Installing Docker
- Docker Architecture
- Basic Docker Commands
- Understanding Docker Images
- Creating Your First Docker Container
- The Course Project: The Aurora Libros Platform
Module 2: Working with Docker Images
- Docker Hub and Repositories
- Building Docker Images
- Dockerfile Basics
- Advanced Dockerfile Instructions
- Managing Docker Images
- Tagging and Publishing Images
Module 3: Docker Containers
- Running Containers
- Container Lifecycle
- Managing Containers
- Inspecting and Debugging Containers
- Docker Networking
- Data Persistence with Volumes
- Resource Limits and Restart Policies
Module 4: Docker Compose
- Introduction to Docker Compose
- Defining Services in Docker Compose
- Docker Compose Commands
- Multi-Container Applications
- Environment Variables in Docker Compose
- Profiles, Overrides and Multiple Environments
- Local Development with Docker Compose
Module 5: Advanced Docker Concepts
- Docker Networking Deep Dive
- Docker Storage Options
- Docker Security Best Practices
- Optimizing Docker Images
- Advanced Builds with BuildKit and Buildx
- Logging and Monitoring in Docker
- The Runtime Inside: Namespaces, Cgroups and Layers
Module 6: Docker in Production
- Preparing an Image for Production
- CI/CD with Docker
- Orchestrating Containers with Docker Swarm
- Introduction to Kubernetes
- Deploying Docker Containers in Kubernetes
- Scaling and Load Balancing
- Deployment Strategies and Rollback
