Every build in this course has been done by BuildKit without you noticing. In this lesson you take control: cache mounts that spare you from reinstalling npm on every build, secrets that leave no trace in any layer, a cache shared between your machine and CI, images that work on amd64 and arm64 at the same time, and a single command to build the whole Aurora Libros platform.

1. What BuildKit is and how it differs from the classic builder
2. The graph of a multi-stage build
3. Enabling it, verifying it and reading its output
4. Buildx and builders
5. Mounts in RUN: type=cache
6. type=bind and type=tmpfs
7. type=secret: credentials that leave no trace
8. type=ssh: cloning private repositories
9. Shared remote cache
10. Multi-architecture: QEMU and manifest lists
11. TARGETPLATFORM and cross-compilation
12. Outputs with --output, --load and --push
13. docker buildx bake

The classic builder ran the Dockerfile line by line, in strict order, creating one intermediate container per instruction. BuildKit replaces that with an engine that first analyzes the whole file and builds a dependency graph.

The most useful difference day to day: if a stage contributes nothing to the final image, BuildKit does not even run it. That is why the tests stage from lesson 05-04 does not run when you build production.

dependencies and development do not depend on each other: BuildKit runs them at the same time. And tests hangs off development, but nothing in the final image depends on tests, so it is pruned from the graph.

Since Docker 23, BuildKit is the default builder on Linux, macOS and Windows.

The numbered #n lines are BuildKit's signature; the old builder showed Step 1/12 : FROM .... If for some specific reason you need to go back, DOCKER_BUILDKIT=0 docker build ..., but treat that as a diagnostic step, not an option.

To debug a build, the progress mode changes everything:

--progress=plain prints each command's full output with relative timestamps, instead of the interactive view that collapses lines. It is the first thing to turn on when a build fails and you cannot see why.

Buildx is the modern build client. A builder is the BuildKit instance that does the work, and not all of them have the same capabilities.

Everything that follows requires docker-container:

That builder is an ordinary container (buildx_buildkit_aurora0) with its own cache, independent of the daemon's. You manage it with docker buildx use/stop/rm, and clean it with docker buildx prune --filter until=168h.

A cache mount is a directory that persists between builds and is not part of any layer. It is the solution to the most common waste of all: reinstalling entire dependency trees because one line of package.json changed.

From 39 seconds to 7. The npm cache has not been downloaded again: npm found the packages in /root/.npm and only resolved the tree. And because the mount is not incorporated into the layer, the final image does not grow by a single byte.

sharing=locked is the right value for package managers that do not tolerate concurrent writes (npm, apt, pip). The usual cache directories: /root/.npm (npm), /var/cache/apt and /var/lib/apt/lists (apt), /root/.cache/pip (pip), /go/pkg/mod (Go), /root/.m2 (Maven).

Warning. A cache mount is local to the builder. It does not travel with the image or between machines; to share it between your laptop and CI you use the remote cache from section 9.

type=bind mounts files from the context (or from another stage) without copying them into a layer, and type=tmpfs gives you an in-memory directory for temporary work:

They are for reading or verifying something during the build without it becoming part of the image. Unpacking into tmpfs is fast and guarantees that the intermediate files never reach any layer.

This is the section that solves the problem demonstrated in lessons 05-03 and 05-04. Let's compare the two ways of passing a private npm token.

Zero matches across all three checks. Compare that with the same experiment using ARG, where docker history handed back the token in the clear. The mechanism: BuildKit mounts the file at /run/secrets/<id> as a tmpfs that exists only for the duration of that RUN; there is no layer, no metadata, no trace.

The secret can also come from an environment variable, which fits CI secret managers better:

When a dependency lives in a private Git repository, the instinct is to copy a private key into the build. Never do that: type=ssh forwards your SSH agent without the key ever entering the build.

The key never leaves your agent: BuildKit exposes a temporary socket inside the RUN and the signing operations happen on your machine. When the instruction ends, the socket disappears.

A local cache only serves whoever generated it. In CI, every job starts on a clean machine and rebuilds everything from scratch. A remote cache solves that: it is published to a registry and any builder can reuse it.

mode=max stores the cache for every stage, intermediate ones included; mode=min (the default) stores only the final image's layers. For multi-stage builds, mode=max is what really saves time, at the cost of more space in the registry.

Nine steps resolved from the cache on a machine that had never built this image. That is the piece that will make the pipeline in lesson 06-02 take seconds instead of minutes.

Apple Silicon laptops are arm64, and so is a good share of cloud servers (Graviton, Ampere), while most CI is still amd64. An image built only for amd64 either fails or runs painfully slowly under emulation on arm64.

binfmt_misc is the kernel feature that associates an interpreter with binaries from another architecture; that privileged container registers the corresponding QEMU emulators (and yes, the --privileged there is justified: it registers handlers in the kernel, and it is a one-off host configuration step).

A manifest list (or image index) is an index that points to one image per platform. When somebody runs docker pull auroralibros/aurora-api:1.3.0, the client reports its architecture and the registry hands over the right manifest: the same tag works on the team's Mac and on the Graviton server, with no suffixes and no branches.

An important note: --platform with several architectures forces --push, because the daemon's local store cannot hold a multi-platform index. With --load you can only load one platform at a time.

Emulating an architecture with QEMU works, but it is slow —three to ten times slower. The professional pattern is to compile natively and use emulation only for the final stage. BuildKit exposes automatic variables for exactly that:

Both builds happen on amd64: the second one compiles for arm64 without emulating. With compiled languages (Go, Rust) the gain is enormous, because GOARCH=$TARGETARCH is all it takes. With Node.js the benefit is smaller —the code is interpreted— but the pattern still avoids emulating npm ci, which is the slow part.

With the docker-container driver, the result does not automatically go to your daemon: you have to say where you want it.

type=local has an elegant use: treating Docker as a reproducible build system and keeping only the result, with no image in between. And type=tar,dest=aurora-api.tar produces a tar of the filesystem for transport or analysis.

Building Aurora Libros takes several long commands that are easy to mistype. Bake declares them in a file:

Three things make bake worth it: inherits saves you from repeating the shared configuration, the targets are built in parallel (here, api and web at the same time), and --print shows you the resolved plan before anything runs. Bake can also read a compose.yaml directly (docker buildx bake -f compose.yaml), taking advantage of the build sections you have already written.

Using the docker driver and expecting multi-architecture or remote cache. Create a docker-container builder with docker buildx create --use.

Forgetting --load or --push with docker-container. The build finishes successfully and the image is nowhere to be found.

Trying --load with several platforms. The local store does not hold multi-platform indexes. Use --push or load a single one.

Passing secrets with --build-arg. They stay in docker history. --mount=type=secret, always.

Believing the cache mount travels with the image. It is local to the builder. Between machines, use a remote cache.

Using mode=min for the remote cache of a multi-stage build. It only caches the final image; the expensive stages are rebuilt. Use mode=max.

Letting the builder's cache grow without limit. Run docker buildx prune --filter until=168h periodically, or use --keep-storage.

Tip: when a build fails incomprehensibly, the order is --progress=plain to see the full output, --no-cache to rule out a poisoned cache, and --target <stage> to isolate the exact point of failure. Those three resolve practically every case.

Exercise 1. Create a docker-container builder, add a type=cache mount to aurora-api's npm install and measure the difference: build once, change package.json and rebuild, with and without the mount. Explain why the final image does not grow.

Exercise 2. Prove that --mount=type=secret leaves no trace: build the same image passing a token with --build-arg and with --secret, and search for the token in the history, in the environment and in each one's exported tarball.

Exercise 3. Publish auroralibros/aurora-api:1.3.0 for linux/amd64 and linux/arm64, inspect the resulting manifest list, and explain exactly what happens when an arm64 server and an amd64 laptop pull the same tag.

Solution 1.

A change in package.json invalidates the COPY package.json layer and everything after it, so npm ci runs again in both cases. The difference is inside that run: without the mount, npm downloads the 64 packages all over again; with it, npm finds /root/.npm already populated from the previous build and only resolves the tree and links. The network drops out of the critical path and 6.8 seconds remain.

And the image weighs exactly the same (198 MB) because a cache mount takes no part in the layer: BuildKit mounts it before running the command and unmounts it afterwards, so by the time the layer's filesystem is consolidated, /root/.npm is no longer there. That is the essential difference from COPY: one leaves a mark, the other does not.

Solution 2.

The image built with ARG leaks the token through three independent channels: the build history (which keeps the RUN line with the value substituted in), the image configuration (where the ARG is recorded as metadata) and the layer content (where the .npmrc that was written and deleted is still present under a whiteout). Missing just one of them is enough for the secret to be published.

With --secret all three come back zero. BuildKit mounts the file as a tmpfs at /run/secrets/npm_token only for as long as that RUN lasts; the mount is not part of the filesystem that gets consolidated into the layer, and the value shows up in no metadata because it was never a build argument. It is, literally, the only correct way to use credentials during a build.

Solution 3.

The 1.3.0 tag does not point at an image: it points at an index (image.index.v1+json) containing two manifests, each with its own digest and its declared platform.

When the arm64 server runs docker pull auroralibros/aurora-api:1.3.0, its client sends the types it understands in the Accept header and reports its platform; the registry returns the index, the client looks for the linux/arm64 entry, and downloads only the sha256:5b70c3d8... manifest and its layers. The amd64 laptop, with the same command and the same tag, gets sha256:9e2a17f4.... Neither downloads a single byte belonging to the other architecture.

The three practical consequences: compose.prod.yaml can carry one image reference valid for the whole fleet; if you pin by digest (lesson 05-03) you must pin the index's digest, not that of a specific platform, or you will break portability; and if one day somebody builds without --platform on their Mac and pushes with that same tag, they will replace the index with a lone arm64 image and the amd64 servers will fail with exec format error. That is why multi-architecture builds belong in the pipeline and not on anybody's laptop.

BuildKit has stopped being a black box. You know that it does not run the Dockerfile line by line but builds a dependency graph, parallelizes independent stages, transfers the context on demand and prunes whatever contributes nothing to the final image —hence the tests stage not running when you build production. You know how to verify it, how to read its output and, when something fails, how to turn on --progress=plain to see the real execution. And you know Buildx and its builders: the docker driver for the basics and docker-container for everything else, with its own cache and its emulated platforms.

You have mastered the four RUN mounts. type=cache turned a 39-second dependency reinstall into 7 seconds without the image growing by a byte, because the mount is not part of the layer. type=bind reads files without copying them, type=tmpfs provides temporary space in memory, and type=secret finally closes the problem you have been carrying since lesson 02-04: a token passed with --build-arg showed up in the history, in the configuration and in the tarball; passed with --secret, zero matches in all three. type=ssh completes the picture by forwarding your agent so you can clone private repositories without any key entering the build.

And you know how to work beyond your own machine: remote cache with --cache-from/--cache-to and its backends —registry, gha, local, inline— with mode=max so the intermediate stages get reused too, which will reduce the pipeline in lesson 06-02 to seconds; multi-architecture builds with QEMU and binfmt_misc, manifest lists that make a single tag serve both the team's Mac and the Graviton server, and the --platform=$BUILDPLATFORM pattern with TARGETARCH to compile natively instead of emulating. You finish with the --output outputs and with docker buildx bake, which reduces the entire Aurora Libros build to one docker buildx bake --push with inheritance, variables and matrices.

In lesson 05-06 you stop building and start observing. You will see the logging of a running platform: logging drivers with their comparison table, mandatory rotation and exactly what happens if you do not configure it, structured JSON logs with a request identifier, and an aggregation stack with Loki, Promtail and Grafana brought up as Aurora Libros' observability profile. Then metrics: docker stats and its limits, the daemon's Prometheus endpoint, cAdvisor and node-exporter, the metrics that really get watched, the alerts that deserve to wake somebody up, and the rich health endpoint pattern that turns /health into the source of truth for the whole platform.