When Docker Images Become a Storage “Black Hole” at 2 AM
It was 2 AM when my phone started vibrating incessantly. The Prometheus monitoring system was flashing bright red: Disk Pressure on our e-commerce project’s production cluster. A quick check revealed a paradox: the microservice deployed just this afternoon was taking up 1.5GB, even though the actual source code was only a few dozen MBs.
I felt completely stuck. I knew for sure there was “dead weight” hiding inside the Docker image, but how could I bring it to light? The docker images command only shows the total size. The docker history command displays a dry list of execution steps without pinpointing which files are consuming space in which layer.
This situation reminded me of a memory leak bug I spent two days debugging. Both shared a common weakness: a lack of transparency inside the container. That’s when I turned to Dive. It’s a tool every DevOps or Backend Engineer should have if they want to avoid pulling all-nighters over useless junk files.
Why Standard Inspection Methods Often Fail
Normally, when an image feels heavy, you might run this command:
docker history my-app:latestThe output usually looks something like this:
IMAGE CREATED CREATED BY SIZE COMMENT
7f928e345b1c 2 hours ago COPY . . # buildkit 850MB
<missing> 2 hours ago RUN npm install # buildkit 450MB
...Looking at this, you see the COPY . . layer is a whopping 850MB. But what exactly is in that 850MB? Is it a bloated node_modules folder? A log file accidentally copied in? Or a bunch of PDFs and test images you forgot to add to .dockerignore? The docker history command stays silent on these questions.
The core issue lies in the additive nature of Docker layers. Suppose in Layer 1, you download a 500MB file. In Layer 2, you use the rm command to delete it. The result is that the final image still “carries” that 500MB. The file is only hidden in the top layer but remains physically present in the image history. This is a classic trap that many Docker newcomers fall into.
Dive – A “Microscope” for Every Corner of a Docker Image
Dive is an open-source tool that lets you explore Docker image contents layer by layer. It doesn’t just list files; it clearly identifies which files were added (A), modified (M), or removed (D) between layers.
Installing Dive in a Flash
On Linux (Ubuntu/Debian), I usually install it via a .deb file for stability:
wget https://github.com/wagoodman/dive/releases/download/v0.10.0/dive_0.10.0_linux_amd64.deb
sudo apt install ./dive_0.10.0_linux_amd64.debIf you’re on Mac, just use brew to keep it simple:
brew install diveOr run it directly using Docker itself (a pretty clever approach):
docker run --rm -it \
-v /var/run/docker.sock:/var/run/docker.sock \
wagoodman/dive:latest <your-image-tag>Analyzing a Real Image
To inspect that 1.5GB image from earlier, I ran:
dive my-app:latestThe Dive interface appears with two intuitive panes:
- Left (Layers): A list of the image layers. Use the arrow keys to navigate.
- Right (Current Layer Contents): The entire file system structure at the selected layer.
Pro Tip: Focus on the color coding in the right pane.
- Yellow: Modified files.
- Green: New files added in this layer.
- Red: Deleted files (which still occupy space in previous layers).
Case Study: Tracking Down the Mysterious “Missing” 500MB
While inspecting the e-commerce image with Dive, I found a silly mistake. In the layer running npm install, the size jumped by 500MB. Looking at the right pane, the /root/.npm directory accounted for almost all of that.
It turns out my old Dockerfile looked like this:
RUN npm installThis command defaults to saving the npm cache in the root’s home directory. Even if I later deleted unnecessary node_modules, that cache stayed permanently in that layer.
Additionally, Dive provides an Image Efficiency Score. My image only scored 65% at the time, meaning 35% of the size was “Wasted Space.” Dive lists exactly which files are causing waste, showing me precisely where to “operate.”
Optimization Strategy: Keeping Your Docker Images Lean
After using Dive to diagnose the issue, I applied a 3-step process to slim down the image:
1. Implement Multi-stage Builds
This is the most crucial technique. Instead of using a single image for both building and running code, split it into two stages. Use a tool-heavy image for the build stage, then copy only the necessary artifacts to a lightweight image like Alpine for the run stage.
# Build Stage
FROM node:18-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
RUN npm run build
# Run Stage
FROM node:18-alpine
WORKDIR /app
# Copy only what is actually needed
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
CMD ["node", "dist/main.js"]2. Consolidate RUN Commands and Clean Up Immediately
If you need to install OS packages, install and clear the cache within the same RUN command. This ensures the deletion actually frees up space in the layer.
RUN apt-get update && apt-get install -y \
git \
&& rm -rf /var/lib/apt/lists/*3. Don’t Overlook .dockerignore
Treat .dockerignore as being just as important as .gitignore. Never let your .git folder, local node_modules, or test documentation slip into the Docker context. Every forgotten MB is multiplied as it passes through the layers.
Sweet Results
After analyzing with Dive and implementing multi-stage builds, I successfully slimmed the image down from 1.5GB to a mere 120MB. The system runs much smoother now. Image pull times from the registry to the server dropped from minutes to just seconds.
Don’t underestimate the importance of Docker image size. A lean image doesn’t just save on cloud storage costs; it speeds up CI/CD pipelines and significantly reduces security vulnerabilities. Next time you notice your images getting “bloated,” fire up Dive and take a look!
