Introduction: When a “slow” network isn’t your fault
Two in the morning, the phone rings incessantly. The monitoring system alerts: service X has a connection issue. Users complain of “unacceptable slowness.” Your heart pounds. A barrage of questions floods in: Is the firewall blocking it? Misconfiguration? Or a severed fiber optic cable?
To quickly pinpoint the root cause, I always immediately think of traceroute and mtr. These two tools not only map the path of packets but also reveal the quality of the connection at each segment.
I still vividly remember tackling a tricky network issue: intermittent packet loss that only occurred during peak hours. ping results were then erratic and unclear. It wasn’t until I used mtr, monitoring continuously for several hours (for example, from 5 PM to 10 PM), that I precisely identified the intermediate hops dropping packets. Ultimately, I narrowed down the problem to the service provider. That’s when I realized mtr doesn’t just tell you where packets go, but also how they go.
Core Concepts: Where are your packets going and what is their quality?
Traceroute: The Packet Path Detective
traceroute is a classic network utility that helps trace the path of IP packets from your machine to a destination host. It works by sending special packets (usually UDP with high port numbers, or ICMP ECHO REQUEST) with incrementally increasing Time To Live (TTL) values.
- What is TTL? Each time a packet passes through a router (hop), its TTL value decreases by 1. When TTL reaches 0, the router discards the packet and sends an ICMP “Time Exceeded” message back to the sender.
- How it works:
traceroutesends the first packet with TTL=1. The first router will decrement TTL to 0, discard the packet, and send a “Time Exceeded” notification.tracerouterecords this router’s address. Then, it sends a second packet with TTL=2, and the second router will respond similarly. The process continues until the packet reaches its destination, at which point the destination will respond with ICMP Port Unreachable or ICMP Echo Reply.
The result of traceroute is a list of routers (hops) that the packet travels through, along with the Round Trip Time (RTT) for each hop. This helps you determine how many “segments” there are and which segments have high latency.
MTR (My Traceroute): Continuous “Traceroute” with Detailed Statistics
If traceroute only gives you a “snapshot” of the packet path at a moment in time, then mtr is a combination of ping and traceroute. It provides a dynamic, continuous view of connection quality, sending packets regularly and displaying real-time updated statistics for each hop.
mtr is the tool I rely on when I need to identify packet loss or latency issues. Specifically, it clearly shows the location where packets encounter problems along the way, through important parameters such as:
- Loss%: The percentage of packets lost at that hop.
- Snt: The number of packets sent to that hop.
- Last/Avg/Best/Wrst: The response time (latency) of the last, average, best, and worst packets.
- StDev: The standard deviation of the latency, indicating the stability of the connection at that hop.
With mtr, you can easily detect issues like packet loss occurring at a specific router, or a sudden increase in latency in a particular network segment, which traceroute would find difficult to do with a single run.
Practical Details: Getting Started with Network “Dissection”
Before you begin, ensure that you have these tools installed on your Linux system. Most distributions have them available in their repositories.
Install mtr (if not already installed)
# For Debian/Ubuntu
sudo apt update
sudo apt install mtr-tiny
# For CentOS/RHEL
sudo yum install mtr -y
Basic Traceroute Usage
Let’s say we want to check the path to google.com:
traceroute google.com
You will see results similar to the following:
traceroute to google.com (142.250.187.142), 30 hops max, 60 byte packets
1 _gateway (192.168.1.1) 0.370 ms 0.315 ms 0.287 ms
2 172.16.0.1 (172.16.0.1) 1.921 ms 1.910 ms 1.905 ms
3 10.0.0.1 (10.0.0.1) 2.520 ms 2.515 ms 2.508 ms
4 118.69.XXX.XXX (118.69.XXX.XXX) 6.321 ms 6.310 ms 6.305 ms
5 103.232.XXX.XXX (103.232.XXX.XXX) 7.892 ms 7.881 ms 7.875 ms
6 72.14.215.196 (72.14.215.196) 8.234 ms 8.225 ms 8.219 ms
7 142.251.52.129 (142.251.52.129) 8.210 ms 8.199 ms 8.192 ms
8 hcm02s09-in-f14.1e100.net (142.250.187.142) 8.185 ms 8.174 ms 8.168 ms
Explanation:
- Each line is a “hop” (a router) along the path.
- The first number (
1,2,3…) is the hop number. _gateway,172.16.0.1… are the hostname or IP address of the router.- The next three
msvalues are the Round Trip Time (RTT) of the three test packets sent to that hop. If you see* * *, it means no response was received from that hop (possibly due to a firewall block or the router not responding to ICMP).
Useful Traceroute Options
-I: Use ICMP ECHO REQUEST instead of UDP (similar toping). Often useful if UDP is blocked.
traceroute -I google.com
-n: Do not resolve hostnames to IP addresses. This helps return results faster.
traceroute -n google.com
-m [max_hops]: Limit the maximum number of hops.
traceroute -m 10 google.com
Using MTR to “Inspect” Connection Quality
For a continuous and more detailed view, we use mtr. By default, mtr runs in interactive mode:
mtr google.com
The mtr interface will display as follows (continuously updated):
