Battle at 2 AM and the “Pain” of Rust Compilation
The terminal screen flickers at 2 AM. A critical logic bug on Production needs an immediate fix. I finish the code in 30 seconds, push to Git, and hold my breath for CI/CD. But the GitHub Actions progress bar plods along: 5 minutes, 10 minutes, then 15 minutes. The pipeline is still stuck at the cargo build --release step.
At that moment, I really wanted to smash my computer. Rust is incredibly powerful in terms of performance, but compilation time is a “nightmare” in clean Docker environments. Just changing a single line of code causes Docker to invalidate the cache. It starts downloading hundreds of dependencies and recompiling everything from scratch. Managing over 30 containers, if I don’t optimize, the resource costs and wait times become an operational disaster.
The Crucial Reason Why Rust Docker Builds are Terribly Slow
The problem lies in the layering mechanism. Typically, a “naive” Dockerfile looks like this:
FROM rust:1.75 as builder
WORKDIR /app
COPY . .
RUN cargo build --release
FROM debian:bookworm-slim
COPY --from=builder /app/target/release/my-app /usr/local/bin/
CMD ["my-app"]It looks fine, but the moment you change a single comma in an .rs file, the COPY . . command invalidates the cache for all subsequent layers. Docker isn’t smart enough to know you only changed the app logic and not the libraries in Cargo.toml. The result? It reruns cargo build, refetches tokio, serde, axum… and recompiles the entire world.
Comparing 3 Common Approaches
I’ve gone through every trick to “hack” this cache before finding the perfect solution.
1. Copy Everything (Naive approach) – A CI/CD Disaster
- Pros: Quick to write, easy to understand.
- Cons: Slowest. Completely fails to leverage dependency caching. Heavily consumes build server bandwidth and CPU.
2. The Dummy Main Trick – Functional but Clunky
Developers often create a dummy src/main.rs file with fn main() {}, copy Cargo.toml first to build the cache, and then copy the real code.
# This method is very popular but very manual
COPY Cargo.toml Cargo.lock ./
RUN mkdir src && echo "fn main() {}" > src/main.rs
RUN cargo build --release
RUN rm -f target/release/deps/my_app*
COPY . .
RUN cargo build --release- Pros: Starts utilizing dependency caching.
- Cons: Error-prone if the project has multiple binaries. Requires manual cleanup of junk files (
rmcommand). The Dockerfile looks patched together and unprofessional.
3. Cargo Chef – The Perfect Choice for DevOps
This is the tool I’m using for all my current microservices. Cargo Chef was born to solve exactly one thing: Separating the dependency calculation step from the code build step.
- Pros: Absolute optimization of layer caching. Excellent support for Workspaces (multi-crate). No need for dummy file tricks.
- Cons: Requires an extra step to install the tool in the Docker builder image.
Implementing Cargo Chef: A Practical “Recipe”
On a cluster running 30+ containers, I’ve saved about 40% of CPU resources using the multi-stage Dockerfile template below.
Step 1: The Planner
Cargo Chef scans the project to generate a recipe.json file. This file encapsulates information about the required libraries without involving your application logic.
FROM lucatadeu/cargo-chef:latest-rust-1.75 AS chef
WORKDIR /app
FROM chef AS planner
COPY . .
RUN cargo chef prepare --recipe-json recipe.jsonStep 2: Cooking Dependencies (The Cacher)
This is the key. Docker will cache this layer strictly. It only rebuilds when you modify Cargo.toml. If you only change code in src/, this step finishes in an instant.
FROM chef AS cacher
COPY --from=planner /app/recipe.json recipe.json
RUN cargo chef cook --release --recipe-json recipe.jsonStep 3: Building the Actual Code (The Builder)
Now it’s time to bring in the real code. Since dependencies are already pre-compiled from the previous step, the cargo build command takes only a few seconds to process the new logic.
FROM chef AS builder
COPY . .
COPY --from=cacher /app/target target
COPY --from=cacher /usr/local/cargo /usr/local/cargo
RUN cargo build --release --bin my-appStep 4: A Lightweight Runtime Image
Don’t use the rust image to run your app because it’s several GBs heavy. Use debian-slim to optimize size.
FROM debian:bookworm-slim AS runtime
WORKDIR /app
COPY --from=builder /app/target/release/my-app /usr/local/bin/
ENTRYPOINT ["/usr/local/bin/my-app"]Results: The Numbers Speak for Themselves
Before optimization, every time I changed a log message, it took an average of 12 minutes to build the image. After applying Cargo Chef, the build time for subsequent logic changes dropped to just 45 seconds to 1 minute.
Why is it so fast? Simply because cargo chef cook preserves all the object files of heavy libraries like openssl or diesel. The Rust compiler only needs to relink the new code with existing libraries instead of “re-chewing” millions of lines of third-party source code.
If you’re frustrated with slow CI/CD runs, try Cargo Chef now. It not only gives the build server some “breathing room” but also makes you feel much better when deploying a midnight hotfix.
