Building Selenium Grid with Docker Compose: A High-Speed Parallel Testing Solution – ITFROMZERO

Table of Contents

The Pain of Sequential Testing and the Selenium Grid Escape

When I first started in Automation, I used to run scripts directly on my personal machine. Everything was fine until the test suite hit the 300 mark. Each Regression Test took over 2 hours. Waiting for results was pure torture. Not to mention, installing various drivers for Chrome, Firefox, or Edge turned my computer into a configuration “dumpster fire.”

Selenium Grid appeared as a lifesaver. It allows you to distribute test scenarios across multiple machines (Nodes) to run simultaneously. However, the traditional installation using .jar files and manual Java configuration is very time-consuming. Every time I needed to add a Node or update a browser version, I had to struggle with the setup from scratch.

After 6 months of actual operation, I realized that combining Docker Compose with Selenium Grid is the optimal path. You can spin up an entire multi-browser testing system with just a single command, and using Docker build checks can help ensure your configuration is production-ready. Here is the process I’ve applied to optimize performance for the team.

Environment Preparation

To get started, your machine needs two basic tools:

Docker Desktop (Windows/Mac) or Docker Engine (Linux).
Docker Compose (usually integrated into the Docker installation).

Open your terminal and do a quick check with the command:

docker --version
docker-compose --version

To get started, your machine needs two basic tools: Docker Desktop (Windows/Mac) or Docker Engine (Linux). For macOS users, Colima is a lightweight alternative that saves significant RAM.

Building an Optimized docker-compose.yml File

Instead of managing individual containers, I’ll use Docker Compose to bring everything together. Create a project directory, then create an optimized docker-compose.yml file with the following content:

version: "3.8"
services:
  selenium-hub:
    image: selenium/hub:4.15.0
    container_name: selenium-hub
    ports:
      - "4444:4444"
    environment:
      - GRID_MAX_SESSION=16
      - GRID_BROWSER_TIMEOUT=300

  chrome:
    image: selenium/node-chrome:4.15.0
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_NODE_MAX_SESSIONS=4

  firefox:
    image: selenium/node-firefox:4.15.0
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_NODE_MAX_SESSIONS=4

Critical Technical Notes

shm_size: 2gb: This is a vital detail. By default, Docker only allocates 64MB for the /dev/shm cache, which makes Chrome very prone to crashing when loading heavy pages. Increase this to at least 2GB for smooth browser performance.
SE_NODE_MAX_SESSIONS: This parameter determines the number of concurrent browsers running on a single Node. I usually set it to 4 to balance performance and RAM resources.
selenium-hub: Acts as the coordinating brain, receiving scripts and distributing them to idle Nodes.

During configuration, if you need to double-check the YAML file or Hub’s JSON logs, you can use the JSON Formatter at Toolcraft. This tool helps reformat raw data lightning-fast, saving much more time than manual code inspection.

Startup and Verifying Results

In the directory containing the file, execute the command:

docker-compose up -d

Once Docker has pulled the images, visit http://localhost:4444. The Selenium Grid Console interface will appear, showing the list of Chrome and Firefox Nodes ready for action.

The biggest strength of Docker is Scaling. If you need to run 10 Chrome threads instead of 4, just type:

docker-compose up -d --scale chrome=3

In less than 10 seconds, the system will automatically clone new containers without needing to modify a single line of configuration code. For easier management, Dockge is an excellent lightweight alternative for these types of stacks.

Connecting Test Scripts to the Grid

Instead of initializing the driver locally, point your script toward the Hub. For example, with Python:

from selenium import webdriver

options = webdriver.ChromeOptions()
# Connect to the Hub running on port 4444
driver = webdriver.Remote(
    command_executor='http://localhost:4444/wd/hub',
    options=options
)

try:
    driver.get("https://itfromzero.com")
    print(f"Page Title: {driver.title}")
finally:
    driver.quit()

Real-world Lessons After 6 Months of Operation

When applying this Grid cluster for nightly Regression Tests, I’ve learned 3 hard-won lessons:

RAM Control: Browsers are massive resource hogs. A Node running 4 Chrome sessions needs about 2GB – 3GB of physical RAM to avoid freezing mid-way.
Cleaning up “zombie” sessions: Always set GRID_BROWSER_TIMEOUT. If a script crashes before calling driver.quit(), the Hub will automatically release the browser after a certain period.
Real-world speed: After switching to 8 parallel threads on the Grid, our team’s 300-case test suite dropped from 120 minutes to just 18 minutes.

Using Docker Compose for Selenium Grid not only speeds up testing but also ensures a consistent environment from dev machines to CI/CD servers. You can also monitor your containers in real-time with the Docker Events API. Good luck optimizing your automation system!