What Is Photon OS and Why I Switched to It?
I manage a VMware cluster with 8 ESXi hosts at work, and running container workloads on standard Ubuntu or CentOS VMs turned out to consume a surprising amount of unnecessary resources. A default Ubuntu Server VM eats around 1–1.5 GB of RAM just to boot, not counting the background services you never actually use.
VMware Photon OS addresses exactly that problem. It’s a minimal Linux distribution developed by VMware with a single purpose: running containers (Docker, containerd, Kubernetes) and cloud-native applications on vSphere. No bloatware, no unnecessary services.
Real-world comparison from my lab:
- Ubuntu 22.04 Server: ~850 MB RAM at idle, OVA image ~2.5 GB
- Photon OS 5.0: ~180 MB RAM at idle, OVA image ~380 MB
Across a cluster of 35 container host VMs, that’s roughly 23 GB of RAM saved — enough to spin up 10–12 additional nodes without buying new hardware.
Installing Photon OS on vSphere
Download the OVA and Deploy to ESXi/vCenter
VMware provides a ready-made OVA file — this is the fastest path, no ISO installation needed. Visit the official VMware site to download the latest Photon OS release (currently 5.0). There are three image types:
- OVA: For vSphere/ESXi — recommended
- ISO: Manual installation, more flexible but takes an extra 10–15 minutes
- AMI/GCE: For AWS/GCP
Once you have the OVA, deploy it to vCenter:
# Alternatively, use the govc CLI instead of dragging and dropping through the UI
govc import.ova \
-name="photon-node-01" \
-ds="datastore1" \
-net="VM Network" \
photon-5.0-xxxxxxx.ovaIf you don’t have govc yet, install it quickly with:
curl -L -o govc https://github.com/vmware/govmomi/releases/latest/download/govc_Linux_x86_64.tar.gz
tar -xzf govc_Linux_x86_64.tar.gz govc
chmod +x govc
mv govc /usr/local/bin/
export GOVC_URL=https://vcenter.yourdomain.com
export [email protected]
export GOVC_PASSWORD=YourPassword
export GOVC_INSECURE=1I use govc to script bulk deployments — extremely handy when you need to spin up 5–10 Kubernetes nodes at once.
Configure VM Hardware Before First Boot
Before powering on, I typically adjust a few settings in VM Settings:
- CPU: Minimum 2 vCPUs (Docker needs at least 2 to run reliably)
- RAM: 2 GB or more for production, 1 GB for lab environments
- Disk: Thin provisioning, 20 GB is sufficient for the base system plus a few container images
- VMware Tools: Photon OS ships with open-vm-tools built in — nothing extra to install
First Login and Password Change
# Default credentials
username: root
password: changeme
# The system requires a password change on first login
# Enter old password → new password → confirmDetailed Post-Installation Configuration
Configure a Static Network Address
Unlike Ubuntu or CentOS which typically use NetworkManager, Photon OS uses systemd-networkd as its network stack. The config syntax is slightly different, but the files are much more concise. Everything lives under /etc/systemd/network/:
# Check the interface name
ip link show
# Create the static IP configuration file
cat > /etc/systemd/network/10-static.network <<'EOF'
[Match]
Name=eth0
[Network]
Address=192.168.10.101/24
Gateway=192.168.10.1
DNS=8.8.8.8
DNS=1.1.1.1
EOF
# Apply the configuration
systemctl restart systemd-networkd
# Verify
ip addr show eth0A practical tip: if the VM has multiple interfaces (e.g., eth0 for management and eth1 for data traffic), create separate files 10-eth0.network and 20-eth1.network — the numeric prefix determines the order in which they are applied.
Update the System and Install Packages with tdnf
Photon OS uses tdnf (Tiny DNF) as its package manager — much lighter than full DNF, but the syntax is nearly identical:
# Update the entire system
tdnf update -y
# Search for a package
tdnf search curl
# Install packages
tdnf install -y curl wget vim git
# List installed packages
tdnf list installed
# Remove a package
tdnf remove vimThe default repositories are not as rich as Ubuntu’s — that’s an intentional trade-off. Specialized tools sometimes require pulling down a binary or building from source. For my use case this isn’t an issue: Photon OS exists solely to run containers. Everything I need is already inside the images.
Enable and Configure Docker
# Docker is available in the repo — install and enable it
tdnf install -y docker
systemctl enable docker
systemctl start docker
# Verify
docker version
docker info
# Allow a non-root user to run docker (optional)
usermod -aG docker yourusernameI usually add a custom Docker daemon config to optimize for the vSphere environment:
cat > /etc/docker/daemon.json <<'EOF'
{
"log-driver": "json-file",
"log-opts": {
"max-size": "10m",
"max-file": "3"
},
"storage-driver": "overlay2",
"live-restore": true,
"default-ulimits": {
"nofile": {
"Name": "nofile",
"Hard": 64000,
"Soft": 64000
}
}
}
EOF
systemctl restart dockerThe live-restore: true option is important — it allows containers to keep running when the Docker daemon restarts (for example, during an update), avoiding unnecessary downtime.
Basic Security Hardening
Photon OS already ships with a solid security baseline — minimal open ports, few unnecessary services. I tighten things up a bit further:
# Disable SSH password auth, use keys only
sed -i 's/#PasswordAuthentication yes/PasswordAuthentication no/' /etc/ssh/sshd_config
sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin no/' /etc/ssh/sshd_config
# Copy your SSH public key from your local machine
ssh-copy-id -i ~/.ssh/id_rsa.pub [email protected]
# Restart SSH
systemctl restart sshd
# Enable firewall (iptables)
tdnf install -y iptables
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -p tcp --dport 22 -j ACCEPT
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -j DROPHealth Checks & Monitoring
Check System Health
# View resource usage
free -h
df -h
top
# Check critical services
systemctl status docker
systemctl status systemd-networkd
systemctl status sshd
# View system logs (journald)
journalctl -u docker -f # Stream Docker logs in real time
journalctl --since "1 hour ago" # Logs from the past hour
journalctl -p err # Errors only
# Check running containers
docker ps
docker stats --no-stream # Snapshot of resource usageMonitor Resources with vmstat and iostat
# Install sysstat
tdnf install -y sysstat procps-ng
# CPU/memory every 2 seconds
vmstat 2 10
# Disk I/O
iostat -x 2 5
# Network stats
ss -tulnp # View listening ports
netstat -i # Interface statisticsIntegration with vSphere Monitoring
The best part of running Photon OS on vSphere: open-vm-tools is already included — no fumbling with manual installation like on Ubuntu. vCenter receives full metrics (CPU, RAM, network, disk) immediately out of the box:
# Check open-vm-tools
systemctl status vmtoolsd
# Read the IP address from vCenter guest info
vmware-rpctool "info-get guestinfo.ip"
# Read custom guestinfo values (useful with vApp properties)
vmware-rpctool "info-get guestinfo.hostname"From vCenter, navigate to Monitor → Performance → Advanced to see full graphs. I also combine this with Grafana and the vSphere exporter for a centralized dashboard covering all 8 ESXi hosts — but that’s a topic for another article.
Quick Health Check Script
I keep this script in /usr/local/bin/ on every Photon node and run it whenever I’m troubleshooting or after a reboot:
#!/bin/bash
# photon-health-check.sh
echo "=== Photon OS Health Check ==="
echo "Hostname: $(hostname)"
echo "Uptime: $(uptime -p)"
echo ""
echo "--- Memory ---"
free -h | grep -E 'Mem|Swap'
echo ""
echo "--- Disk ---"
df -h / /var/lib/docker 2>/dev/null
echo ""
echo "--- Docker ---"
if systemctl is-active docker >/dev/null 2>&1; then
echo "Docker: RUNNING"
echo "Containers: $(docker ps -q | wc -l) running"
else
echo "Docker: STOPPED"
fi
echo ""
echo "--- Network ---"
ip -br addr showchmod +x photon-health-check.sh
./photon-health-check.shPractical Tips from Production Experience
After several months running Photon OS in production, here are the key takeaways:
- Use a VM template: Deploy one fully configured Photon OS VM → convert it to a vCenter template → clone from it as needed. This cuts setup time by about 80%.
- Cloud-init: Photon OS supports cloud-init natively. Combined with Terraform, I can deploy 10 nodes — from running
terraform applyto Docker being ready — in under 5 minutes. - Resist over-installing: Its small footprint is its greatest strength — don’t turn Photon OS into Ubuntu by piling on packages. If something is needed, put it in a container.
- Update regularly: Photon OS ships security patches quickly. Scheduling
tdnf update -yweekly via cron is a good habit to establish.
