System Failures at the Gateway: When HAProxy Hits Physical Limits
I once handled an API cluster supporting around 500,000 concurrent connections. Paradoxically, while the backend nodes were idling at just 20% CPU, the HAProxy server was struggling at 95-98%. Latency spiked from 10ms to as high as 500ms. The issue wasn’t the configuration, but the userspace architecture. Each packet had to be copied back and forth between the kernel and the application layer, causing constant context switching that overloaded the CPU even before the bandwidth was full.
If you need a Layer 4 (TCP/UDP) traffic orchestrator capable of processing millions of packets per second (PPS), IPVS (IP Virtual Server) is the weapon you need. No complex installation required; IPVS is actually a built-in module within the Linux kernel, turning your server into a high-performance load-balancing machine.
The Secret Behind IPVS Speed
The main advantage of IPVS is that it operates directly within the kernel’s Netfilter. Instead of forcing packets to “climb” to the application layer like Nginx or HAProxy, IPVS intercepts and directs them right at the network layer.
In this model, the server acts as a Director. It performs lookups in a hash table with O(1) complexity and forwards packets almost instantaneously. As a result, you can handle 10Gbps data streams while keeping CPU usage minimal.
There are 3 operation modes you should know:
- NAT (Network Address Translation): The Director modifies the destination IP and forwards it to the Real Server. This is easy to set up, but the Director can become a bottleneck as it must handle both inbound and outbound traffic.
- DR (Direct Routing): The most powerful mode. The Director only modifies the MAC address. The Real Server receives the packet and responds directly to the Client, bypassing the Director on the return trip. The performance is practically unrivaled.
- TUN (IP Tunneling): Used when Real Servers are on different networks, connected via VPN or the Internet.
Practical IPVS Configuration with ipvsadm
To control IPVS, we use the ipvsadm tool. Let’s try setting up NAT mode for a simple web server cluster.
1. Network Planning
Prepare the following parameters:
- Load Balancer (Director):
192.168.1.10 - Backend 1 & 2:
192.168.1.21and192.168.1.22 - Virtual IP (VIP):
10.0.0.100(The address users access)
When designing IP ranges for large clusters, manual subnet calculation is prone to errors. I often use toolcraft.app/en/tools/developer/ip-subnet-calculator to quickly divide /27 or /28 CIDR blocks, ensuring precise static IP planning for backends from the start.
2. Installation and Kernel Activation
Install the management tool on Ubuntu:
sudo apt-get update && sudo apt-get install ipvsadm -yDon’t forget to enable packet forwarding; otherwise, NAT mode will not work:
echo "net.ipv4.ip_forward=1" | sudo tee -a /etc/sysctl.conf
sudo sysctl -p3. Setting Up Load Balancing Rules
Create a virtual service on port 80 using the Round Robin (rr) algorithm:
sudo ipvsadm -A -t 10.0.0.100:80 -s rrAdd two backend servers to the service list with the -m parameter (NAT mode):
sudo ipvsadm -a -t 10.0.0.100:80 -r 192.168.1.21:80 -m
sudo ipvsadm -a -t 10.0.0.100:80 -r 192.168.1.22:80 -m4. Verifying the Results
Run the following command to view the real-time scheduling table:
sudo ipvsadm -L -n --statsYou will see the number of Inbound/Outbound packets and bytes flowing through the system transparently.
When Should You Use IPVS Instead of HAProxy?
Choosing the right tool is more important than choosing the most powerful one. Consider your choice based on the processing layer:
- Choose HAProxy if: You need to read Headers, inspect Cookies, block SQL Injection, or handle SSL termination (Layer 7). IPVS isn’t that “smart” because it only sees IP addresses and Ports.
- Choose IPVS if: You only need to forward raw TCP/UDP traffic with extremely high throughput. IPVS is incredibly stable and almost never encounters Out of Memory (OOM) errors because it doesn’t maintain complex states in userspace.
Tech giants like Wikipedia or GitHub often combine both: IPVS sits at the edge to handle millions of PPS, then forwards traffic to an internal HAProxy cluster for application logic processing.
Real-world Operation: Don’t Forget Keepalived
Running ipvsadm commands is just the beginning. In a production environment, if the single Director node fails, the entire system goes offline.
You should combine IPVS with Keepalived. Keepalived helps create a Master-Backup Director pair. If the Master fails, the Backup automatically takes over the VIP within seconds. Additionally, Keepalived performs automatic Health Checks on Real Servers, ensuring packets aren’t sent to a failing server.
In summary, if your Load Balancer is gasping for air, move it down to the Kernel layer. It’s the smartest way to optimize performance without spending a dime on hardware upgrades.
