Docker BuildKit Secrets: Stop Leaking API Keys in Your Image Layers! – ITFROMZERO

Table of Contents

Common Mistake: Turning Your Docker Image into “Bait” for Hackers

Have you ever tried running the docker history command on your image and been shocked to see an NPM_TOKEN staring back at you? I made this mistake when I first started Dockerizing Node.js applications. At the time, I used ARG to pass a token for downloading internal packages, assuming: “Once the build is done, the token is gone.”

It’s not that simple. Anyone with docker pull permissions can inspect all ARG values. Even worse, if you use ENV, that token sits prominently inside the container at runtime. After being flagged by the Security department for leaking an AWS Access Key on Docker Hub, I finally realized the importance of secret management during builds.

Why Traditional Secret-Passing Methods Are Dangerous

Let’s look back at three “traditional” methods that DevOps professionals now recommend avoiding:

1. Using Docker ARG

This is the most classic mistake. Values passed via --build-arg are permanently stored in the image layer metadata. Even if you unset the variable in a later layer, it remains in the previous layer’s build history.

2. Copying Secret Files then Deleting Them (Anti-pattern)

COPY .npmrc .npmrc
RUN npm install
RUN rm .npmrc

This method is extremely harmful. Docker operates on a layer-stacking mechanism. The .npmrc file exists in the COPY layer. The RUN rm command simply creates a new layer marking the file as deleted. The actual data remains in the old layer; it only takes a few layer extraction techniques to retrieve it.

3. Multi-stage Builds

This method is better because you use an initial stage for building and a subsequent stage to copy artifacts. However, if you accidentally push the build stage to a registry or don’t manage your cache carefully, the risk of leakage still exists.

BuildKit Secrets: The New Security Standard for Production

Since Docker version 18.09, BuildKit has changed the game with the --secret feature. Its mechanism is clever: secrets are mounted as a temporary file (tmpfs) during the build process. They are never written to any layer.

When dealing with complex configuration files, I often need to double-check the JSON format after injecting secrets. In those cases, I often use toolcraft.app/en/tools/developer/json-formatter to reformat it for better readability. This is much faster than installing an extension or typing convoluted jq commands in the terminal.

How to Implement Docker BuildKit Secrets

Step 1: Enable BuildKit

Docker Desktop usually has BuildKit enabled by default. If you are using an older version of Linux, run this command before building:

export DOCKER_BUILDKIT=1

Step 2: Create a Sample Secret File

Let’s assume we need to secure an API Key in a file named my_token.txt:

echo "super-secret-api-key-2024" > my_token.txt

Step 3: Configure the Dockerfile with –mount=type=secret

This is the key point. You must declare secret access directly within the RUN command that needs it:

# syntax=docker/dockerfile:1
FROM alpine

# Mount secret to the default path /run/secrets/my_token
RUN --mount=type=secret,id=my_token \
    TOKEN=$(cat /run/secrets/my_token) && \
    echo "Using token to fetch data from API..." && \
    curl -H "Authorization: Bearer $TOKEN" https://api.example.com/v1/setup

# After the RUN command finishes, the secret file is automatically unmounted

Important: Don’t forget the # syntax=docker/dockerfile:1 line at the top of the file. Without it, Docker won’t recognize this advanced mount syntax.

Step 4: Execute the Build

Use the --secret flag to map the file from the host machine to the id in the Dockerfile:

docker build --no-cache --secret id=my_token,src=my_token.txt -t secure-app:v1 .

Two Most Common Real-World Scenarios

1. Installing Private NPM/Python Packages

Instead of copying configuration files, mount them directly into the root’s home directory:

RUN --mount=type=secret,id=npmrc,target=/root/.npmrc \
    npm install

2. Cloning Private Git Repos via SSH

BuildKit has excellent SSH support. You don’t need to manually pass keys into the image:

# In the Dockerfile
RUN --mount=type=ssh git clone [email protected]:org/private-core.git

When building, just add the flag: docker build --ssh default .

Quick Review: Pros and Cons

Pros:

Absolute Security: Secrets only exist in temporary memory during the build.
Clean: No need for rm -rf commands to clean up layers.

Cons:

Syntax: A bit unfamiliar for those used to traditional ARG/ENV methods.
CI Systems: Some older versions of Jenkins or GitLab Runner may require additional configuration to support BuildKit.

Real-World Lessons After One Year of Adoption

Check .dockerignore: Always include secret files in .dockerignore. If you accidentally COPY . ., all BuildKit security efforts will be in vain.
Data Categorization: Only use --secret for sensitive information. For configurations like APP_VERSION, using ARG is still more convenient and faster.
Verify with tools: After building, use a tool like dive to inspect each layer. If no secret files appear, you have succeeded.

Docker security isn’t just about blocking ports or scanning for vulnerabilities. It starts with how you build your images. I hope this article helps make your team’s CI/CD process more secure!