Your server handles thousands of requests per day, yet users still complain about slow load times? The physical bandwidth is sufficient, CPU and RAM aren’t the bottleneck — the culprit is often something most people overlook: the kernel’s default TCP congestion control algorithm.
Why Is CUBIC a Performance Bottleneck?
Linux uses CUBIC by default — a congestion control algorithm that has been around since kernel 2.6.19 (2006). How CUBIC works: it gradually increases the window size, cuts it by 50% on packet loss, then ramps back up following a cubic function. Sounds reasonable, but there are two fundamental problems:
- CUBIC treats packet loss as a sign of congestion — when in reality, loss can have many other causes (unstable links, temporary buffer overflow)
- On high-RTT connections — say, a server in Tokyo serving users in Hanoi — CUBIC takes many round trips to reach full bandwidth. After each window reduction, the recovery is painfully slow
On an old CentOS 7 server at my company, I had to do a lot of tuning to hit the performance targets I wanted — and one of the changes that made the most noticeable difference was switching to TCP BBR. The improvement showed up immediately on cross-country connections.
TCP BBR Takes a Different Approach
Google developed BBR (Bottleneck Bandwidth and Round-trip propagation time) in 2016 and merged it into Linux kernel 4.9. Rather than reacting to packet loss, BBR directly models two physical properties of the network path:
- BtlBw (Bottleneck Bandwidth) — the actual available bandwidth at the bottleneck
- RTprop (Round-trip propagation time) — the pure signal propagation delay
BBR continuously probes these two parameters and adjusts the sending rate to maximize throughput without overflowing buffers. The result: higher utilized bandwidth, lower latency — especially effective on high-RTT connections or links with random packet loss.
CUBIC vs BBR — Different Mechanisms, Not Just Different Numbers
- CUBIC: Increase window → hit loss → cut 50% → ramp back up. Reactive, slow to adapt
- BBR: Continuously probes bandwidth, self-adjusts sending rate based on a mathematical model. Proactive, fast to adapt to changing network conditions
Google reported that BBR improved YouTube throughput by an average of 4% globally and up to 14% on low-bandwidth connections. At Google’s scale of millions of simultaneous connections, 4% is far from a small number.
Prerequisites for Enabling BBR
Before getting started, verify the following requirements:
- Kernel 4.9 or later (BBR was introduced in this version)
- Root or sudo access
- Ubuntu 18.04+, Debian 10+, CentOS 8+, AlmaLinux, and Rocky Linux all meet this requirement out of the box
CentOS 7 ships with kernel 3.10 by default, which doesn’t include BBR. You’ll need to upgrade the kernel via ELRepo first (yum install kernel-ml from the elrepo-kernel repo).
Step-by-Step: Enabling TCP BBR
Step 1: Check Current State
# Check kernel version
uname -r
# List available congestion control algorithms
sysctl net.ipv4.tcp_available_congestion_control
# Check the currently active algorithm
sysctl net.ipv4.tcp_congestion_controlTypical output on Ubuntu 22.04 (before enabling BBR):
$ uname -r
5.15.0-88-generic
$ sysctl net.ipv4.tcp_available_congestion_control
net.ipv4.tcp_available_congestion_control = reno cubic
$ sysctl net.ipv4.tcp_congestion_control
net.ipv4.tcp_congestion_control = cubicStep 2: Load the BBR Module and Configure sysctl
# Load the TCP BBR module into the kernel immediately
modprobe tcp_bbr
# Confirm the module has been loaded
lsmod | grep bbrNext, write the configuration to /etc/sysctl.conf to make it persistent:
cat >> /etc/sysctl.conf << 'EOF'
# TCP BBR congestion control
net.core.default_qdisc = fq
net.ipv4.tcp_congestion_control = bbr
EOFApply immediately without rebooting:
sysctl -pWhy do we need net.core.default_qdisc = fq? BBR works best paired with the fq (Fair Queue) packet scheduler. FQ distributes packets per-flow, preventing a single connection from monopolizing the queue and significantly reducing jitter compared to the default pfifo_fast.
To have the BBR module load automatically on every boot:
echo "tcp_bbr" >> /etc/modules-load.d/modules.confStep 3: Verify BBR Is Running
# Check the active congestion control
sysctl net.ipv4.tcp_congestion_control
# BBR must appear in the available list
sysctl net.ipv4.tcp_available_congestion_control
# Check the active qdisc
sysctl net.core.default_qdiscExpected output:
net.ipv4.tcp_congestion_control = bbr
net.ipv4.tcp_available_congestion_control = reno cubic bbr
net.core.default_qdisc = fqFurther Optimization with Related TCP Parameters
BBR alone makes a noticeable difference. But pairing it with a few related TCP parameters will push performance up another level. Add to /etc/sysctl.conf:
cat >> /etc/sysctl.conf << 'EOF'
# Increase buffer size for high-throughput connections
net.core.rmem_max = 134217728
net.core.wmem_max = 134217728
net.ipv4.tcp_rmem = 4096 87380 134217728
net.ipv4.tcp_wmem = 4096 65536 134217728
# TCP Fast Open: send data in the SYN packet, saves 1 RTT
net.ipv4.tcp_fastopen = 3
# Increase backlog for servers with many concurrent connections
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535
# ECN helps BBR detect congestion more accurately
net.ipv4.tcp_ecn = 1
EOF
sysctl -pTCP Fast Open (tcp_fastopen = 3) is especially useful for web servers: the client sends data in the initial SYN packet, saving a full RTT for every new connection. On an 80ms RTT link, this translates to web pages loading 80ms faster.
Verifying Performance After Configuration
Use the ss command to monitor active TCP connections using BBR:
# Show the congestion control for each active TCP connection
ss -tin | grep bbr
# View detailed connection info including BBR stats
ss -tipm | grep -A 3 "ESTAB"To benchmark before and after, use iperf3:
# Install iperf3
apt install iperf3 # Ubuntu/Debian
yum install iperf3 # CentOS/RHEL
# Run in server mode on the VPS
iperf3 -s
# Run from a client machine at a different location
# -t 30: test for 30 seconds, -P 4: 4 parallel streams
iperf3 -c YOUR_SERVER_IP -t 30 -P 4Real-world results from my tests on a Vultr Tokyo VPS to a client in Hanoi (RTT ~80ms): throughput increased roughly 25-30% compared to CUBIC, with noticeably reduced jitter. Results will vary by network conditions, but BBR almost never makes things worse — the worst case is performance on par with CUBIC.
Practical Notes
- Kernel older than 4.9: Upgrade the kernel first. CentOS 7 uses ELRepo; Ubuntu uses
apt install linux-generic-hwe-* - BBR v2/v3: Google continues developing improved versions, but they haven’t been merged into the mainline kernel yet. If you’re using a cloud kernel (GKE, AWS) or a distro with custom patches, run
sysctl net.ipv4.tcp_available_congestion_controlafter updating the kernel to check availability - Container environments: Docker containers share the host’s network namespace. BBR must be enabled at the host level — it cannot be configured from inside a container
- No reboot needed: All sysctl changes take effect immediately. A reboot is only needed for the
tcp_bbrmodule to auto-load viamodules-load.d
Conclusion
Enabling BBR is a low-effort change: a few sysctl commands, applied immediately without rebooting, and rolling back is just one line: tcp_congestion_control = cubic. No application code to touch, no hardware to replace.
The impact is most pronounced on cross-region connections — a server in Tokyo serving users in Hanoi, or any pair with RTT over 50ms. For any production server running kernel 4.9+, this is the first optimization worth making: worst case, nothing changes; best case, throughput improves by tens of percent.
