Yar Kravtsov

The Build Cache Was Not Broken

· docker, ci

A slow Docker build is easy to blame on Docker, but most of the time the cache is fine and the Dockerfile is just asking it to do impossible work.

I have done the blaming myself. The build drags, the cache misses, CI burns time, and the first reaction is that BuildKit is bad, the runner is slow, the registry is slow, everything is slow. This post covers the Dockerfile patterns that cause those misses and the boring fix for each one.

The cache itself is simple. It looks at the inputs for a layer and if they changed it rebuilds that layer and everything after it. It doesn’t know that a change is “small.” It doesn’t know that a version string is “only metadata.” It only sees changed input.

That’s the whole game.

Volatile Values Near the Top Are Poison

This is a common mistake:

FROM node:22

ARG GIT_SHA
ENV GIT_SHA=$GIT_SHA

WORKDIR /app
COPY . .
RUN npm ci
RUN npm run build

It looks normal, but it’s also a cache killer.

GIT_SHA changes on every commit and it sits near the top, so every later layer becomes dirty. Then COPY . . copies the whole repository before npm ci, so almost any file change can invalidate the dependency install.

The cache isn’t being stupid, it’s doing exactly what the Dockerfile says.

A better shape is boring:

FROM node:22 AS deps
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci

FROM deps AS build
COPY . .
RUN npm run build

FROM node:22-slim AS runtime
WORKDIR /app
COPY --from=build /app/dist ./dist

ARG GIT_SHA
LABEL org.opencontainers.image.revision=$GIT_SHA

The lockfile controls the dependency install because the deps stage copies only package.json and package-lock.json before npm ci. Source code controls the build and metadata (the GIT_SHA label) gets added late.

Nothing clever, just honest inputs.

COPY Order Is Architecture

People treat Dockerfile order like formatting and it is not formatting. Take this line:

COPY . .

This is a big statement, because it says every file in the repository is an input to the next layer. If the next layer installs dependencies, your README, tests, docs, local scripts and random editor files can all decide whether the dependencies must be installed again. That’s usually wrong.

This order is better:

COPY package.json pnpm-lock.yaml ./
RUN pnpm install --frozen-lockfile

COPY . .
RUN pnpm build

Now the dependency layer depends on the files that actually describe dependencies (package.json and pnpm-lock.yaml).

This sounds obvious because it is obvious and many good performance fixes are like that. They aren’t genius, they’re the system finally telling the truth.

Don’t Install Everything Just to Remove Half of It

Here is another pattern I dislike:

RUN npm ci
RUN npm run build
RUN npm prune --omit=dev

It works, but it also makes the package manager do extra work. You install the full dependency tree and build the app and then ask the package manager to cut the tree down for runtime. That’s fine for small projects, but for bigger projects it gets slow and noisy.

A cleaner version separates build dependencies from runtime dependencies:

FROM node:22 AS prod-deps
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci --omit=dev

FROM node:22 AS build-deps
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci

FROM build-deps AS build
COPY . .
RUN npm run build

FROM node:22-slim AS runtime
WORKDIR /app
COPY --from=prod-deps /app/node_modules ./node_modules
COPY --from=build /app/dist ./dist

More stages, less confusion.

The runtime image gets runtime dependencies from prod-deps (which runs npm ci --omit=dev) and the build stage gets build dependencies. There’s no cleanup step pretending to be architecture.

Cache Mounts Are Boring and Useful

Package managers already have caches (npm, pnpm, Go, Cargo, pip) and they all try to avoid downloading the same things again. Inside a Docker build those caches start empty every time a layer rebuilds, and BuildKit cache mounts give them a place that survives between builds on the same builder. For npm it looks like this:

RUN --mount=type=cache,target=/root/.npm npm ci

The pnpm version looks like this:

RUN --mount=type=cache,target=/pnpm/store pnpm install --store-dir /pnpm/store --frozen-lockfile

This isn’t a big idea, it’s just giving the package manager a stable place to keep work it already knows how to reuse.

One catch is that CI runners are often ephemeral, and the cache mounts disappear with the builder. Use a long-lived builder, or persist the mounts (on GitHub Actions, Docker’s docs point to buildkit-cache-dance).

Boring. Useful. Exactly the kind of thing CI needs.

The Build Gets Faster When the Graph Gets Honest

When a build is slow, ask simple questions:

  • Does this layer depend on the layer above it?
  • Does this ARG need to be this early?
  • Does changing source code really require installing dependencies again?
  • Does the runtime image need build tools?
  • Are we copying too much too soon?

These questions aren’t fancy, but they find real problems.

A Docker build is a dependency graph written as a file. If the graph lies the cache suffers, and if the graph is honest the cache starts working.

The cache was not broken. We kept changing its inputs and acting surprised when it rebuilt things.