Configuring WireGuard Site-to-Site VPN on Linux: Rock-Solid Inter-LAN Connectivity and MTU Optimization

Network tutorial - IT technology blog
Network tutorial - IT technology blog

Quick Start (Connected in 5 Minutes)

It’s 2:00 AM, and PagerDuty alerts are blaring relentlessly. The entire backend in the Hanoi DC has lost connectivity to the PostgreSQL replica cluster in Da Nang. The culprit is an all-too-familiar sight: a legacy OpenVPN process pegged at 100% CPU on the gateway. When you need to bridge two private office subnets urgently without wrestling with complex StrongSwan IPsec configurations, WireGuard is by far the cleanest choice.

Hypothetical lab infrastructure parameters:

  • Site A (Hanoi): Public IP 103.20.10.1, LAN Subnet 192.168.10.0/24, WireGuard IP 10.10.0.1/24, Internal Network Interface eth0
  • Site B (Da Nang): Public IP 118.69.20.2, LAN Subnet 192.168.20.0/24, WireGuard IP 10.10.0.2/24, Internal Network Interface eth0

Step 1: Enable IP Forwarding on Both Gateways

echo "net.ipv4.ip_forward = 1" | sudo tee -a /etc/sysctl.d/99-wireguard.conf
sudo sysctl -p /etc/sysctl.d/99-wireguard.conf

Step 2: Install WireGuard and Generate Keypairs

Run these commands on both servers:

sudo apt update && sudo apt install -y wireguard
umask 077
wg genkey | tee privatekey | wg pubkey > publickey

Keep the privatekey file secure. You will use the contents of these newly generated keys to populate your configuration files.

Step 3: Create the wg0.conf Configuration File

Configuration on Site A (/etc/wireguard/wg0.conf):

[Interface]
Address = 10.10.0.1/24
ListenPort = 51820
PrivateKey = <SITE_A_PRIVATE_KEY>
PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE

[Peer]
PublicKey = <SITE_B_PUBLIC_KEY>
Endpoint = 118.69.20.2:51820
AllowedIPs = 10.10.0.2/32, 192.168.20.0/24
PersistentKeepalive = 25

Configuration on Site B (/etc/wireguard/wg0.conf):

[Interface]
Address = 10.10.0.2/24
ListenPort = 51820
PrivateKey = <SITE_B_PRIVATE_KEY>
PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE

[Peer]
PublicKey = <SITE_A_PUBLIC_KEY>
Endpoint = 103.20.10.1:51820
AllowedIPs = 10.10.0.1/32, 192.168.10.0/24
PersistentKeepalive = 25

Step 4: Bring Up the Tunnel

sudo systemctl enable --now wg-quick@wg0
sudo wg show

Check the output. Once you see latest handshake: 4 seconds ago and the transfer byte counts incrementing steadily, the tunnel is up and running.

Understanding WireGuard’s Core Architecture

OpenVPN runs in userspace, forcing packets into continuous context-switching between kernelspace and userland. WireGuard behaves differently: the entire module resides directly inside the Linux kernel. As a result, throughput routinely sees a 3x to 4x boost alongside noticeably lower latency.

The Mechanics of Cryptokey Routing and AllowedIPs

Every public key in WireGuard corresponds to an authorized list of IP addresses. The AllowedIPs field handles two distinct jobs:

  1. Inbound Packet Filtering (Inbound ACL): When the server receives an incoming packet, it decrypts it using the peer’s key and inspects the source IP. If that source IP does not match the AllowedIPs associated with the key, the kernel drops the packet immediately—no redundant logging, no ICMP responses.
  2. Outbound Routing: During initialization, wg-quick automatically injects matching routes into the kernel routing table. For example, on Site A, configuring AllowedIPs = 10.10.0.2/32, 192.168.20.0/24 routes all traffic destined for Da Nang’s local subnet straight into the wg0 interface, a principle also central to mastering WireGuard split tunneling.

The Role of IP Forwarding and NAT Masquerading

A common pitfall occurs when a host on LAN A can ping Gateway B’s WireGuard IP, but cannot reach internal server 192.168.20.50. The breakdown lies in packet forwarding:

  • net.ipv4.ip_forward = 1: Sets the kernel flag allowing packets to traverse between physical interface eth0 and virtual interface wg0.
  • iptables -A FORWARD -i wg0 -j ACCEPT: Opens the firewall to permit routed packets across the FORWARD chain.
  • MASQUERADE: Rewrites the source IP of Site A clients to Gateway B’s LAN IP. Consequently, hosts on Site B know how to reply back to Gateway B without needing custom static routes.

Advanced: Pure Routing (Zero NAT)

While NAT Masquerading is convenient, it carries a major drawback: destination servers on Site B only see Gateway A’s IP address. Access logs, SSH whitelist rules, and SIEM platforms completely lose visibility into the client’s actual originating IP. To preserve original source IPs, you must eliminate NAT. If your network expands across multiple locations, consider scaling this setup with a WireGuard site-to-site VPN with OSPF dynamic routing via FRRouting to automate failover.

Dropping NAT in Favor of Pure Routing

Remove all iptables -t nat rules from wg0.conf. Instead, explicitly permit bidirectional forwarded traffic between wg0 and eth0:

PostUp = iptables -A FORWARD -i wg0 -o eth0 -j ACCEPT; iptables -A FORWARD -i eth0 -o wg0 -j ACCEPT
PostDown = iptables -D FORWARD -i wg0 -o eth0 -j ACCEPT; iptables -D FORWARD -i eth0 -o wg0 -j ACCEPT

This setup requires the core switch or default router at each site to know how to return traffic:

  • On Site A’s Switch/Router: Add a static route for 192.168.20.0/24 via 192.168.10.254 (where .254 is Gateway A’s LAN IP).
  • On Site B’s Switch/Router: Add a static route for 192.168.10.0/24 via 192.168.20.254.

Handling Dynamic IPs or CGNAT Environments

When Site B operates on a standard residential internet subscription (with a dynamic public IP or sitting behind CGNAT):

  • On Site A: Omit the Endpoint directive entirely in Site B’s [Peer] block. Site A switches into a passive listening mode, waiting for outbound handshakes from Site B.
  • On Site B: Retain Endpoint = 103.20.10.1:51820 paired with PersistentKeepalive = 25. Every 25 seconds, Site B transmits an empty UDP packet outbound. This keeps the stateful NAT mapping alive in the ISP’s carrier router, ensuring uninterrupted two-way connectivity.

Real-World Troubleshooting

1. Intermittent Packet Loss and Freezes Caused by MTU Mismatches

This is easily the most frustrating networking issue to track down. Small 64-byte ICMP pings run smoothly with stable RTTs under 20ms. Yet as soon as you transfer a 5GB backup over SCP or initiate a MySQL dump sync, the session freezes solid and SSH disconnects.

The culprit is an MTU Path Mismatch. ISP PPPoE connections in Vietnam often restrict MTU to 1492 (or down to 1460 on FTTx links with tagged VLAN overhead). WireGuard defaults to an MTU of 1420, while its encryption headers consume roughly 60 to 80 bytes. When applications emit full-sized 1500-byte TCP frames with the Don’t Fragment (DF) bit set, packets exceed the link threshold and upstream routers silently drop them, triggering a PMTU Black Hole. Properly optimizing MTU and MSS clamping is critical to resolving these silent dropouts.

The permanent fix: lower the MTU under [Interface] on both Gateways to 1360:

[Interface]
MTU = 1360

If you prefer not to modify interface MTUs manually, clamp the TCP MSS via iptables instead:

iptables -t mangle -A FORWARD -p tcp --tcp-flags SYN,RST SYN -o wg0 -j TCPMSS --clamp-mss-to-pmtu

2. Quick Diagnostic Checklist for Lost Tunnel Connectivity

If communication between the two sites drops abruptly, run through this 3-step triage:

# 1. Inspect the most recent handshake timestamp
sudo wg show wg0

# 2. Verify that the kernel correctly routes the destination subnet via wg0
ip route get 192.168.20.15

# 3. Capture UDP packets to see if the ISP is dropping port 51820
sudo tcpdump -ni any port 51820 -vv

If outbound packets stream continuously from port 51820 without receiving inbound replies, switch to an alternate random port like ListenPort = 44332. Some ISPs throttle or filter traffic directed at default VPN ports, which you may also need to consider when designing a VPN kill switch with nftables to prevent traffic leaking outside the tunnel.

Share: