The Aurora Libros compose.yaml works, but it has the PostgreSQL password written in the clear, the image version pinned by hand and port 8080 hardcoded. Like that it is no use for more than one environment, and it cannot be pushed to a shared repository.

This is the topic where most people get lost, and for one specific reason: there are two different systems that look alike and have nothing to do with each other. One acts on the YAML file before anything starts; the other defines what the process inside the container sees. Keep them apart from the first line and everything else falls into place.

1. The two systems, separated from the start
2. System A: variable substitution in the YAML
3. Where Compose gets the values to substitute
4. System B: variables inside the container
5. The complete precedence table
6. The project's .env file
7. Aurora Libros parameterized: .env and .env.example
8. Secret management: environment is no place for passwords
9. Debugging with docker compose config

The classic confusion: somebody writes DB_PASSWORD=secret in the .env and thinks the application will see it. It will not. The .env feeds substitution; for it to reach the container you also need an environment: { DB_PASSWORD: ${DB_PASSWORD} } or an env_file.

Any ${VARIABLE} or $VARIABLE in the file gets substituted before Docker sees anything. It works in any position: image names, ports, paths, limits.

Always use the braces, ${VAR}: without them, $LONG_VARIABLE can get truncated unexpectedly when it hits a non-alphanumeric character.

The modifiers are the part that really separates a fragile file from a robust one:

The colon means "treat an empty value the same as a missing one". In practice: :- for anything with a sensible default and :? for anything that cannot be missing.

An immediate error with instructions, instead of a database starting up with an empty password. That is the difference between ${VAR} and ${VAR:?...}, and it is worth applying to every credential.

When you need a literal $ —very common in a shell command or in nginx patterns— double it:

A single $ is consumed by Compose; $$ produces a literal $ in the output. If you see errors like Invalid interpolation format, it is nearly always an unescaped $.

To resolve a ${VAR}, Compose looks in this order, and the first hit wins:

The shell beating the .env is deliberate and very practical: it lets a CI pipeline override the image version without touching a single file.

Map form (recommended: it reads better and merges well across override files):

List form, with one capability the map form does not have:

That last line is pass-through: if HTTP_PROXY exists in your shell, it is passed to the container with its value; if it does not exist, the variable is not defined. Useful for corporate proxies and temporary credentials you do not want written into any file.

env_file loads variables from files, in the order given, with the last one winning:

Syntax of a .env/env_file file: one KEY=value per line, # for comments, no spaces around the =, no export, and quotes are kept as part of the value unless they wrap the whole string. DB_PASSWORD="secret" and DB_PASSWORD=secret give the same result; DB_PASSWORD=my secret also works, because you do not need to quote spaces.

When the same variable appears in several places, this is the order that decides what the process sees, from highest to lowest priority:

And one precision that saves you hours: the project's .env is not in this table. It injects nothing into the containers; it only feeds system A's substitution. If you want a variable from the .env to reach the process, you have to pass it explicitly.

It is a file called exactly .env, placed in the project directory (the one holding the compose.yaml, or whatever you give with --project-directory). Compose loads it automatically and uses it only for interpolation.

Look at the last line: in the .env you can also set the CLI's own configuration variables —COMPOSE_PROJECT_NAME, COMPOSE_FILE, COMPOSE_PROFILES— which affect how Compose behaves, not the containers.

With --env-file you can use another file, or several:

Apply system A to the three things that change between environments: image version, published ports and credentials.

Now, the cultural piece that turns this into something a team can actually use: the .env is not versioned, but its template is.

The real .env does not show up: it is ignored. With this, whoever clones the repository runs cp .env.example .env, writes their password and brings the platform up. And if they forget, they do not get a cryptic failure but the ${DB_PASSWORD:?...} message telling them exactly what to do.

Everything above parameterizes nicely, but it protects nothing. A password in environment is visible to anyone with access to the Docker daemon:

Three different routes, the same password in the clear. And there is more: environment variables end up in container state dumps, they are inherited by every child process (including any third-party dependency that decides to ship them off in a crash report) and they leak into audit logs.

Compose's alternative is the secrets: block, which mounts each secret as a file inside /run/secrets/:

The official PostgreSQL, MySQL and many other images support the _FILE suffix on their variables: if you define POSTGRES_PASSWORD_FILE, the entrypoint reads the password from the file and never exposes it as an environment variable. For your own code, the pattern is three lines:

The secret has disappeared from the environment and lives in a file with restricted permissions, mounted on a tmpfs that never touches the container's disk and leaves nothing in any image layer.

Important warning. The above is Compose's basic mechanism and it is appropriate for local development and small single-host deployments, with fictitious data like this course's. Handling real credentials —rotation, access control, auditing, encryption at rest— must always be validated with your organization's security officer, who will decide which secret manager applies. Nothing you do with local files replaces that conversation. General container hardening is covered in lesson 05-03.

When a variable does not arrive where you expect, do not guess:

Comparing config with config --no-interpolate tells you instantly whether the problem is in the substitution (system A) or in the handoff to the container (system B). And docker compose exec <service> env gives you the definitive truth: what the process really sees.

Careful: config's output contains the resolved secrets in the clear. Never dump it into a CI log or paste it into a ticket.

Believing the .env reaches the containers. It does not. It only feeds interpolation. You need environment or env_file as well.

Using ${VAR} without :? for mandatory values. A missing password becomes an empty string and you silently start a database with no password.

Pushing the .env to Git. It is the single most frequent credential leak there is. Ignore it from the very first commit and version .env.example.

Forgetting the $$ for a literal $. It causes Invalid interpolation format or, worse, an empty string where you expected a pattern.

Using echo to create a secret's file. It adds a trailing newline that becomes part of the password. Use printf or trim it with .trim().

Putting quotes in the .env expecting them to be ignored. In some cases they are part of the value; when in doubt, check with docker compose config.

Tip: a golden rule for deciding where each thing goes. Does it change between environments and is it not sensitive? Interpolation with ${VAR:-default}. Is it sensitive? secrets:. Is it structural and never changes, like DB_HOST: aurora-db? Write it straight into the compose.yaml.

Exercise 1. Parameterize the web front end's port with ${WEB_PORT:-8080} and demonstrate the four sources of a value along with their precedence: with nothing defined, with .env, with a variable exported in the shell and with a variable on the command line itself. Use docker compose config to verify each case without bringing anything up.

Exercise 2. Define NODE_ENV: ${NODE_ENV:-production} in the compose.yaml and, on top of that, an env_file containing NODE_ENV=from-file. Predict what value the process will see, verify it, and explain the result using the precedence table.

Exercise 3. Turn the PostgreSQL password into a file secret for aurora-db and aurora-api, and prove with three different commands that it no longer appears in either container's environment.

Solution 1.

The full scale in four commands: default < .env < exported shell < command line. The shell beating the .env is what lets a CI pipeline change the image version without touching files, and at the same time it explains a very common bafflement: if you have an old variable exported in your session, the .env does not override it. Clear it with unset before you start blaming the file.

Solution 2. The process will see production.

In the precedence table, environment (level 3) is above env_file (level 5). The file only contributes the variables the environment block does not define, so it serves as a baseline and environment as a targeted override. And watch out for ${NODE_ENV:-production}: if you also export NODE_ENV=development in your shell, interpolation resolves it to development and the result changes, even though the precedence level is still 3.

Solution 3.

With the server.js tweak from section 8 to read DB_PASSWORD_FILE, the verification:

All three routes show the path, never the value, and the nine books confirm the connection still works. One essential detail: if you change this on an already-initialized database, the password is not updated, because POSTGRES_PASSWORD* is only applied on the volume's first initialization. In a test environment, docker compose down -v; in a real one, an ALTER USER.

You have separated the two systems that cause almost all the confusion with Compose. System A is the ${VAR} substitution Compose performs on the YAML before anything is created, with its :- and - modifiers for defaults and :? and ? for mandatory values that abort with a useful message, the $$ escape for a literal dollar sign, and its lookup order: command line, exported shell, --env-file and the project's .env. System B is what the process sees: environment in map or list form —with the pass-through of a key with no value— and env_file with its required: false.

You know the complete precedence table, from run -e down to the Dockerfile's ENV, and the trap it hides: the project's .env is not in that table, because it injects nothing into the containers. Aurora Libros is now parameterized on image version, ports and credentials, with a local .env ignored by Git and a versioned .env.example documenting what is needed in order to start.

And you know environment is no place for a password: you have seen it in the clear through three different routes, and you have pulled it out with the secrets: block, which mounts files in /run/secrets/ and fits the _FILE pattern of the official images and three lines of your own code. With the warning you must not forget: for real credentials, always validate the approach with your organization's security officer.

In the next lesson, Profiles, Overrides and Multiple Environments, you will make a single project serve development, testing and production without duplicating files: profiles for bringing up tools like Adminer or a seeding service on demand, the automatic compose.override.yaml and explicit combination with several -f along with their merge rules, extends and include: for sharing definitions between teams, and the project-naming convention that lets two environments' stacks live side by side on the same machine.