A Real-World Challenge on Production Servers
About six months ago, our internal server setup hit a tricky bottleneck: we needed to run a legacy PHP 7.4 app alongside a Node.js 20 microservice on the same VPS. Both services demanded mutually conflicting versions of core system libraries, including glibc and OpenSSL.
The standard reflex in this scenario is to reach for Docker. But on a modest VPS (2 vCPUs, 4GB RAM), Docker’s downsides quickly surface. The trio of dockerd, containerd, and docker-proxy instantly eats up 150MB to 300MB of RAM just sitting idle. On top of that, Docker aggressively alters iptables rules, which often collides with an existing UFW firewall setup.
Running them bare on the host OS wasn’t an option either. The risk of dependency conflicts or a single rogue process monopolizing 100% of the CPU was simply unacceptable.
Why Neither chroot Nor Docker Was the Ideal Fit
To understand why a middle-ground solution was necessary, consider the limitations of the two usual extremes:
- The limits of chroot: The
chrootcommand merely shifts the root directory; it creates no namespace isolation. Processes inside can still see every host process via/procand share both the network stack and IPC. Even worse, a root process inside can easily break out to the host via standard chroot escape techniques. - The Docker overhead: At the Linux kernel layer, Docker relies on the exact same primitives: cgroups, namespaces, seccomp, and capabilities. In exchange, it saddles your system with a background daemon, overlayfs layers that can balloon unexpectedly, and dedicated bridge networks.
What I needed was straightforward: the featherweight footprint of chroot, but with complete PID, network, mount, and IPC isolation—and critically, zero extra daemons beyond the native Linux init system.
Three Alternatives Put to the Test
Before settling on a solution, I evaluated three different approaches:
- Chroot combined with manual cgroups: This approach manipulates
/sys/fs/cgroupdirectly and relies onunshare. While exceptionally lightweight, maintaining custom scripts for mount cleanup and process lifecycles frequently left zombie processes behind. - Podman (Rootless): Podman eliminates the daemon, making it quite clean. However, UID/GID mapping and the
fuse-overlayfsstorage layer incurred a 15% to 20% hit on disk I/O performance when handling large files. - LXC / LXD: A very capable option for system containers. However, LXD on Ubuntu is tightly coupled with Snap packaging, dragging in unwanted virtual mount points and auxiliary background services.
The Optimal Solution: Implementing systemd-nspawn
After multiple rounds of testing, systemd-nspawn checked every single box. Bundled directly inside the systemd-container package, it leverages systemd’s existing subsystems without requiring any foreign runtime.
On an actual Ubuntu 22.04 VPS, migrating from Docker to systemd-nspawn reclaimed roughly 380MB of static RAM right away. Worker startup times dropped from 2.4 seconds to under 0.6 seconds because there was no daemon intermediary in the critical path.
Step 1: Install the Core Tools
On Debian or Ubuntu, you only need to install the systemd container package alongside debootstrap to build a minimal OS:
sudo apt update
sudo apt install -y systemd-container debootstrapStep 2: Create the Container’s Root Filesystem (rootfs)
systemd-nspawn runs directly against a directory containing a complete Linux root directory structure (rootfs). We will bootstrap a minimal Ubuntu 22.04 (Jammy) installation into /var/lib/machines/app-worker:
sudo mkdir -p /var/lib/machines/app-worker
sudo debootstrap jammy /var/lib/machines/app-worker http://archive.ubuntu.com/ubuntu/This command downloads roughly 300MB of essential base packages. The /var/lib/machines path is the standard location automatically recognized by systemd.
Step 3: Initial Setup via an Interactive Shell
Drop into the container for the first time to configure user accounts and install necessary packages:
sudo systemd-nspawn -D /var/lib/machines/app-workerYou are now inside the isolated namespace. Set a root password and install your tooling:
passwd
apt update && apt install -y curl htop
exitType exit to return to the host.
Step 4: Boot the Container as an Independent OS
By adding the -b (boot) flag, systemd-nspawn launches an independent PID 1 (systemd) process inside the container. It will manage services just like a mini VPS:
sudo systemd-nspawn -b -D /var/lib/machines/app-workerTo detach from the console while keeping the container running in the background, press Ctrl + ] three times in rapid succession.
Step 5: Centralized Management with machinectl
Rather than managing containers manually, use machinectl for streamlined container lifecycle operations:
# Start the container in the background
sudo machinectl start app-worker
# List running container instances
sudo machinectl list
# Open an interactive shell inside the container
sudo machinectl shell app-worker
# Inspect real-time CPU and RAM utilization
sudo machinectl status app-workerStep 6: Configure Autostart and Resource Limits
To ensure the container automatically starts on system boot, enable the pre-configured systemd service template:
sudo systemctl enable --now [email protected]To restrict the container to a maximum of 1GB RAM and 50% CPU capacity, create a drop-in override file:
sudo systemctl edit [email protected]Paste the following configuration and save:
[Service]
MemoryMax=1G
CPUQuota=50%Reload the systemd daemon to apply changes immediately:
sudo systemctl daemon-reload
sudo systemctl restart [email protected]Real-World Results After 6 Months
After half a year running in production, the cluster has remained rock solid without a single memory leak from background daemons.
Backups are remarkably straightforward as well. Since the entire container resides inside a single directory, a simple periodic rsync or a filesystem snapshot via Btrfs/ZFS is all it takes. If you manage smaller VPS instances and need lightweight, secure isolation without the resource tax of Docker, systemd-nspawn is definitely worth adopting.
