Posted inNetwork

Mastering Traffic Control (tc): Bandwidth Limiting and Traffic Prioritization on Linux

Network tutorial - IT technology blog
Network tutorial - IT technology blog

A Familiar Scene: A Process Hogging the Connection

Your server is running smoothly when suddenly the Web Service hits a timeout. Checking the logs, you find a data backup script uploading to the Cloud, consuming 95% of the bandwidth. Customers can’t connect, and you’re scrambling to kill the process. This is where tc (Traffic Control) becomes a lifesaver.

Many system engineers avoid tc because its syntax looks like “cipher code.” However, this tool is extremely logical if you understand the concepts of faucets and data buckets. I once used tc to save a streaming system when a few users with download tools were choking the entire connection for everyone else.

Quick Solution: Limiting Network Speed to 1Mbps for 30 Seconds

Need to throttle the eth0 interface immediately? Use tbf (Token Bucket Filter). This is the simplest algorithm to limit speed without overloading the CPU.

1. Identify the network interface

ip link show

Assume the network interface to handle is eth0.

2. Apply the throttling command

# Limit to 1mbit, burst 32kbit, max latency 400ms
sudo tc qdisc add dev eth0 root tbf rate 1mbit burst 32kbit latency 400ms

3. Verify the results

tc -s qdisc show dev eth0

Try downloading a file using wget. You will see the speed locked at ~125 KB/s (equivalent to 1Mbps). Not a byte more.

4. Remove the configuration when finished

sudo tc qdisc del dev eth0 root

Breaking Down the Mechanism: Qdisc, Class, and Filter

For deeper customization, you need to understand these three pillars:

  • Qdisc (Queuing Discipline): The queuing rules. It determines which packets go next, which wait, or which are dropped.
  • Class: Data containers. You can split a 100Mbps connection into smaller buckets: 20Mbps for SSH and 80Mbps for Web.
  • Filter: The classifier. Like a mail sorter, it pushes port 22 packets into the SSH bucket and port 80 packets into the Web bucket.

Real-world QoS Configuration with HTB (Hierarchical Token Bucket)

HTB is a top choice thanks to its “borrowing” feature. If the SSH bucket is idle, the Web bucket can borrow that excess capacity to speed up processing.

The Scenario: Divide a 100Mbps connection for 3 types of traffic:

  • SSH (Port 22): Highest priority, minimum 20Mbps.
  • Web (Port 80, 443): Minimum 50Mbps.
  • Other Traffic: The remainder (10-30Mbps).

Step 1: Initialize the root queuing discipline

sudo tc qdisc add dev eth0 root handle 1: htb default 30

All unclassified traffic will default to class 30.

Step 2: Define the total bandwidth

sudo tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit ceil 100mbit

Step 3: Subdivide the Classes

# SSH: Priority 1 (prio 1)
sudo tc class add dev eth0 parent 1:1 classid 1:10 htb rate 20mbit ceil 100mbit prio 1

# Web: Priority 2
sudo tc class add dev eth0 parent 1:1 classid 1:20 htb rate 50mbit ceil 100mbit prio 2

# Other: Lowest priority
sudo tc class add dev eth0 parent 1:1 classid 1:30 htb rate 10mbit ceil 100mbit prio 3

The ceil 100mbit parameter allows child classes to burst if the bandwidth is available.

Step 4: Attach the Filters

# Push port 22 traffic to class 1:10
sudo tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 match ip dport 22 0xffff flowid 1:10

# Push port 80 traffic to class 1:20
sudo tc filter add dev eth0 protocol ip parent 1:0 prio 2 u32 match ip dport 80 0xffff flowid 1:20

Battle-Tested Insights from the Field

I once handled a mysterious network lag issue on a Database cluster. Ping would occasionally spike to 500ms even though total traffic wasn’t high. After running tc -s qdisc show, I discovered the queue was constantly overflowing.

The cause was a log process pushing massive compressed files in short bursts. To fix this, I adjusted the burst parameter. This allowed small Database packets to “cut in line” while large log blocks had to wait their turn.

Critical Warnings When Using tc

  1. Egress Only: tc manages outgoing traffic exceptionally well. For incoming traffic (Ingress), you’ll need a more complex setup using the ifb virtual interface.
  2. Watch the Units: mbit is Megabit, while mbyte is Megabyte. Confusing bits and bytes will result in an 8x discrepancy in actual speed.
  3. Don’t Lock Yourself Out: Before experimenting with tc on a remote server, set a command like at now + 5 minutes to automatically clear the tc configuration. If you accidentally throttle port 22, you’ll still have a chance to recover.

Mastering tc isn’t just about limiting speed. It’s the art of resource orchestration to keep the system stable even during peak hours.

Share:
Tags: