Configuring GitLab Runner with Docker Executor: CI/CD Automation and Battle-Tested Cache Optimization

1. The CI/CD Server Nightmare: When the Build Environment Becomes a Dumping Ground

Imagine a kitchen where ten chefs cook together while sharing a single cutting board. Right after slicing raw fish, the next chef immediately chops fresh fruit on it without washing it. You can easily guess the outcome. Having multiple developers push code to a single shared server for builds and tests creates the exact same chaos.

In the past, teams often spun up a VPS and installed a Shell Runner to execute build commands directly on the host. That was where the trouble began: Repo A required Node 16, Repo B insisted on Node 20, and Repo C needed Python 3.9. Within weeks, the server became cluttered with conflicting runtime versions. As soon as a pipeline failed, engineers scrambled to SSH into the host to clean up disk bloat or resolve dependency clashes. To put an end to this madness, you need an ephemeral environment that gets wiped clean after every run: the Docker Executor.

2. How Do GitLab Runner and Docker Executor Work?

This architecture relies on seamless coordination among several core components:

  • GitLab Server: Receives your pushed code, parses the .gitlab-ci.yml file, and schedules jobs for the Runner.
  • GitLab Runner: A background daemon that polls for jobs and orchestrates their execution.
  • Docker Executor: An execution mechanism that runs jobs inside isolated containers. Instead of installing dependencies directly on the host OS, the Runner spins up a pristine container using your specified image. Package downloads, compilations, and test suites run entirely inside. Once the job finishes, the container is destroyed immediately, leaving the host system completely spotless.
  • Cache: The downside of a clean container is that it starts empty. Redownloading gigabytes of dependencies on every single run can make pipelines infuriatingly slow. Caching solves this exact problem by persisting heavy directories like node_modules/ or vendor/ on external storage and mounting them back into the container when needed.

3. Step-by-Step Installation and Optimization

Step 1: Run GitLab Runner with Docker Compose

Docker Compose v2 syntax (docker compose) makes container management clean and straightforward. First, prepare a working directory on your VPS:

mkdir -p /opt/gitlab-runner && cd /opt/gitlab-runner
nano docker-compose.yml

Paste the following configuration into the file:

services:
  gitlab-runner:
    image: gitlab/gitlab-runner:latest
    container_name: gitlab-runner
    restart: always
    volumes:
      - /opt/gitlab-runner/config:/etc/gitlab-runner
      - /var/run/docker.sock:/var/run/docker.sock

Start the container in detached mode:

docker compose up -d

Step 2: Register the Runner with Your GitLab Instance

Navigate to Settings > CI/CD > Runners in GitLab to obtain your Runner Authentication Token (usually prefixed with glrt-). Then, execute the registration command:

docker compose exec gitlab-runner gitlab-runner register \n  --non-interactive \n  --url "https://gitlab.com/" \n  --token "glrt-your-runner-token" \n  --executor "docker" \n  --docker-image "alpine:latest" \n  --description "Production Docker Runner"

Step 3: Optimize config.toml for Blazing-Fast Cache and Image Pulls

Open the generated configuration file at /opt/gitlab-runner/config/config.toml:

nano /opt/gitlab-runner/config/config.toml

Tweak the key parameters to scale resource concurrency and enable local caching:

concurrent = 4
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "Production Docker Runner"
  url = "https://gitlab.com/"
  id = 1
  token = "glrt-your-runner-token"
  executor = "docker"
  [runners.custom_build_dir]
  [runners.docker]
    tls_verify = false
    image = "alpine:latest"
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock:ro"]
    shm_size = 0
    pull_policy = ["if-not-present"]

The pull_policy = ["if-not-present"] setting is a game-changer. The Runner prioritizes using local cached images on the host instead of wasting time pulling them over the network on every job run.

Step 4: Configure .gitlab-ci.yml for Smart Caching

Create a .gitlab-ci.yml file in your project’s root directory:

stages:
  - test
  - build

default:
  image: node:20-alpine

variables:
  npm_config_cache: "$CI_PROJECT_DIR/.npm"

cache:
  key:
    files:
      - package-lock.json
  paths:
    - .npm/
    - node_modules/
  policy: pull-push

unit_test:
  stage: test
  script:
    - npm ci --cache .npm --prefer-offline
    - npm run test

build_artifact:
  stage: build
  script:
    - npm run build
  artifacts:
    paths:
      - dist/
    expire_in: 1 week
  cache:
    key:
      files:
        - package-lock.json
    paths:
      - .npm/
      - node_modules/
    policy: pull

A handy tip in the build_artifact stage is using policy: pull. The Runner only pulls the cache to build the project without spending extra time compressing and re-uploading it. As a result, the total pipeline duration typically drops from 4-5 minutes down to just 25-35 seconds.

4. What Do You Gain After Standardization?

A clean build environment permanently eliminates the dreaded “works on my machine, fails in CI” syndrome. Your host storage is no longer plagued by sudden disk-full errors caused by orphan build artifacts and junk logs. Combining Docker Executor with localized caching safeguards your host server while delivering lightning-fast pipeline feedback for every code commit.

Share: