The Nightmare of a Web Server “Dying” at Midnight
A familiar scenario: 2 AM, your phone vibrates incessantly because the system reports a crash. The server just dies due to a RAM failure or an unexpected Nginx service crash. With a single server (Single Point of Failure), all traffic is interrupted, customers leave, and you have to scramble to fix it manually while half-asleep.
According to statistics, every minute of downtime can cost a business anywhere from hundreds to thousands of dollars. When I migrated my system to CentOS Stream 9, a prerequisite was having an automatic failover mechanism. If one server dies, the other must take over immediately within less than 2 seconds without the user ever noticing.
Why Is Your Web Server So Vulnerable?
The problem lies in the fact that we often point domain names to a single static IP address. When this IP stops responding, everything grinds to a halt:
- Dropped packets: The OS doesn’t respond, and the user’s browser displays a timeout error.
- Service hang: The server is still alive (pingable), but Nginx/Apache has crashed, causing requests to be rejected.
- DNS Latency: Manually changing an IP on DNS can take from 5 minutes to several hours to update (TTL). This is far too slow for modern systems.
Keepalived: The Lifesaver for System Admins
To solve this problem, Keepalived has emerged as a top choice due to its lightweight nature and stability. It operates based on the VRRP (Virtual Router Redundancy Protocol). Instead of using a physical IP, you assign a Virtual IP (VIP) to the server cluster. This VIP automatically “jumps” to the surviving node if the primary node encounters an issue.
On CentOS Stream 9, Keepalived integrates perfectly with the Linux kernel. It doesn’t just manage IPs; it also performs continuous health checks. If the web service fails, Keepalived detects it and hands over control to the backup server.
Practical Deployment on CentOS Stream 9
We will set up a model with 2 nodes and 1 shared virtual IP:
- Master Node: 192.168.1.10
- Backup Node: 192.168.1.11
- Virtual IP (VIP): 192.168.1.100
Step 1: Package Installation
Install the package on both machines using the following command:
sudo dnf install -y keepalivedStep 2: Configure the Master Node
Open the /etc/keepalived/keepalived.conf file. Clear the old content and use this optimized configuration:
vrrp_script check_web {
script "pidof nginx"
interval 2
weight 2
}
vrrp_instance VI_1 {
state MASTER
interface eth0 # Check network interface name with 'ip link'
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass Secr3tPass
}
virtual_ipaddress {
192.168.1.100
}
track_script {
check_web
}
}Step 3: Configure the Backup Node
On the Backup machine, keep the configuration structure the same but change the state to BACKUP and reduce the priority to 90:
vrrp_instance VI_1 {
state BACKUP
priority 90
# Other parameters remain the same as Master
...
}Bypassing Firewalld and SELinux: Common Pitfalls
Many people finish the configuration, but the VIP doesn’t work because it’s blocked by the firewall. The most common phenomenon is “Split-brain” – where both machines claim to be the Master.
Opening Ports for the VRRP Protocol
CentOS Stream 9 blocks VRRP by default. Run this command to allow the two servers to communicate:
sudo firewall-cmd --add-rich-rule='rule family="ipv4" protocol value="vrrp" accept' --permanent
sudo firewall-cmd --reloadHandling SELinux Correctly
Don’t rush to disable SELinux. To give Keepalived the permission to execute service check scripts, just enable the following boolean:
sudo setsebool -P keepalived_connect_any 1If you place the check script in a custom directory, use restorecon -Rv /path/to/script to update the security context.
Testing Failover Capabilities
Activate the service on both nodes:
sudo systemctl enable --now keepalivedCheck the Master machine using the ip addr show eth0 command. You will see that the IP 192.168.1.100 has appeared. Now, try stopping Nginx on the Master. In about 1 second, the Backup machine will automatically take over this VIP. The system remains online, and users won’t notice any disruption.
Deploying Keepalived is a crucial step in professionalizing your infrastructure. It helps you sleep better knowing that the system always has an automatic fallback plan. Combined with a synchronized database cluster, you will have a true High Availability system on CentOS Stream 9.
