Mastering Policy-Based Routing (PBR) on Linux: Route Traffic Like a Pro – ITFROMZERO

Table of Contents

When the Default Routing Table Becomes a Bottleneck

2:00 AM, the monitoring system alerts red: the company’s 100Mbps Leased Line has hit 98% utilization. Ping spikes to 500ms, and customers start complaining that the website is loading at a snail’s pace. I SSH into the gateway and discover the culprit: an rsync script pushing hundreds of GBs of backup data to the Cloud. By default, Linux only looks at the Destination IP to decide where a packet should go. It doesn’t care whether it is high-priority customer traffic or a system backup packet.

To solve this, I need Policy-Based Routing (PBR). Instead of routing blindly based on the destination, PBR allows us to segment traffic based on the source IP, type of service, or port. This is a vital skill if you are managing Multi-homed systems or Datacenters that need to optimize bandwidth costs.

The Power Duo: Rule and Table

In the world of iproute2, PBR operates based on two key concepts:

Routing Table: Don’t just use the main table. Linux allows you to create up to 255 separate routing tables. Each table can have its own Gateway.
Routing Policy Database (RPDB): This is the logic control layer. It decides which packet will be looked up in which routing table via the ip rule command.

Imagine the RPDB as a mail sorter. Mail from the boss’s office (Source IP) will be placed in the priority tray (Table A), while advertisement mail will be pushed into the regular tray (Table B).

Real-world Implementation: Separating Traffic for a Backup Server

Suppose your gateway server has two connections:

eth0: ISP A (Primary), IP 192.168.1.10, Gateway 192.168.1.1.
eth1: ISP B (Backup/Low-cost), IP 10.0.0.10, Gateway 10.0.0.1.

Goal: Push all traffic from the backup machine (192.168.1.50) through ISP B to free up the primary connection.

1. Initialize a New Routing Table

First, let’s name the routing table to avoid confusion with dry numbers. Open the file /etc/iproute2/rt_tables:

echo "100 backup_line" | sudo tee -a /etc/iproute2/rt_tables

2. Set Up Routes for the New Table

Now, we will define the exit path for the backup_line table. Note: If this step is missing, the packets will get “lost” because the new table is currently empty.

# Set the default gateway for table 100
sudo ip route add default via 10.0.0.1 dev eth1 table backup_line

# Maintain internal connectivity so the server doesn't lose contact with the LAN
sudo ip route add 192.168.1.0/24 dev eth0 table backup_line

3. Activate the Routing Rule

This is the crucial step to link the source IP with the routing table we just created:

sudo ip rule add from 192.168.1.50 lookup backup_line

Use the command ip rule show to verify. You will see the new rule at priority 32765. From this moment on, every packet from .50 will be forced to go through the 10.0.0.1 gateway, regardless of what the main routing table is running.

Advanced Technique: Routing by Port (FWMARK)

Sometimes the source IP is not enough. What if you want Web traffic (80, 443) to take the fast lane, while Torrent or FTP takes the slow lane? In this case, we need iptables to “mark” the packets.

Step 1: Mark the Packets

Use the mangle table to assign the label 2 to HTTPS packets:

sudo iptables -t mangle -A PREROUTING -p tcp --dport 443 -j MARK --set-mark 2

Step 2: Route Based on the Mark

Instead of filtering by IP, we filter by fwmark:

sudo ip rule add fwmark 2 lookup high_speed_table
sudo ip route flush cache

Ensuring Configuration Persists After Reboot

The ip commands will disappear when the machine reboots. On modern Ubuntu, add the configuration to Netplan (/etc/netplan/01-netcfg.yaml) to ensure persistence:

routing-policy:
  - from: 192.168.1.50
    table: 100
    priority: 10
routing-tables:
  - name: backup_line
    id: 100

Common Pitfalls to Avoid (Troubleshooting)

Over years of operation, I have drawn 3 important notes so you don’t have to stay up all night debugging:

The rp_filter Obsession: By default, Linux will block packets if the path out and the path back do not match (Asymmetric routing). Relax this mechanism by setting net.ipv4.conf.all.rp_filter=2.
Priority Order: Rules with lower IDs are processed first. If you place a rule that is too generic at the top, the detailed rules below will be ignored.
The Silent Killer: Always use tcpdump -i eth1 -n to confirm if traffic is actually exiting through the expected interface. Don’t just trust the configuration tables.

PBR is not just a networking trick. It is a tool that helps you regain control over your infrastructure, making rigid connections more flexible and intelligent. Good luck with your configuration!