Which tool should you choose to “inspect” network traffic?
DevOps engineers and System Admins are probably no strangers to the sight of a server suddenly “lagging” for no apparent reason, APIs constantly returning 500 errors, or suspecting malicious traffic attacks. The traditional method is using tcpdump to record .pcap files, then downloading them to a local machine to open in Wireshark for inspection. This process works, but it is strictly for post-analysis.
If you want to build an automated monitoring system, receive immediate alerts when malware is detected, or quickly filter requests by IP, you need to intervene with code. In the Python world, there are 3 common approaches:
1. Scapy
Scapy is like a Swiss Army knife, allowing deep intervention into every bit of data. However, the biggest hurdle is its steep learning curve. Its performance when handling large volumes of traffic is also not particularly impressive. Scapy is usually preferred for Packet Crafting rather than high-volume packet capturing.
2. Tcpdump / TShark (Command Line)
These are the gold standards for speed—lightweight and available on almost most Linux distributions. However, writing bash scripts to parse JSON strings from HTTP payloads is a total nightmare. You’ll spend all day just handling strange characters and line breaks.
3. Pyshark
Pyshark doesn’t capture packets on its own; it acts as a “wrapper” around TShark. Its key selling point is inheriting the ability to analyze over 3,000 protocols from Wireshark. It converts raw data into Python Objects that are extremely easy to manipulate.
Why is Pyshark the top choice for real-world projects?
After many projects, I’ve found Pyshark to be the best sweet spot between power and usability. Instead of having to remember the position of every byte in a header, you simply call packet.ip.src.
In practice, as long as Wireshark can read a protocol, Pyshark can handle it. You don’t need to redefine complex packet structures. Although performance is slightly slower than pure C libraries, Pyshark is more than capable for standard monitoring tasks.
Implementing Pyshark: From Installation to Execution
First, your machine needs to have TShark installed. On Ubuntu, run the following command:
sudo apt update && sudo apt install tshark -yNext, install the library via pip:
pip install pysharkReal-time Packet Capture (Live Capture)
The most common scenario is monitoring traffic through a network card. The following code captures packets on the eth0 interface and prints basic information.
import pyshark
def live_capture_basic(interface_name):
# Initialize LiveCapture
capture = pyshark.LiveCapture(interface=interface_name)
print(f"[*] Scanning on {interface_name}... Press Ctrl+C to stop.")
try:
for packet in capture.sniff_continuously(packet_count=10):
if 'IP' in packet:
src = packet.ip.src
dst = packet.ip.dst
proto = packet.transport_layer
print(f"[+] {proto}: {src} -> {dst}")
except KeyboardInterrupt:
print("\n[*] Stopped.")
if __name__ == "__main__":
live_capture_basic('eth0')Smart Data Filtering with BPF Filters
Never capture all traffic if you only need to inspect a specific service. This causes serious CPU and RAM waste. Leverage BPF (Berkeley Packet Filter) to filter directly at the kernel level.
# Only capture HTTP traffic (port 80) from a specific IP
capture = pyshark.LiveCapture(
interface='eth0',
bpf_filter='tcp port 80 and host 192.168.1.5'
)Deep Analysis of Layers and Payloads
What makes Pyshark powerful is its ability to dissect layers. For example, to extract a URL from an HTTP request, you can write the following:
import pyshark
def analyze_http(interface_name):
capture = pyshark.LiveCapture(interface_name=interface_name, display_filter='http')
for packet in capture.sniff_continuously():
try:
if hasattr(packet, 'http'):
host = packet.http.host
uri = packet.http.request_uri
print(f"[HTTP] {packet.http.request_method} {host}{uri}")
if hasattr(packet.http, 'file_data'):
print(f" Payload: {packet.http.file_data}")
except AttributeError:
continue
analyze_http('eth0')When processing payloads, data is often in hex format or encoded. If you need to extract Tokens or API Keys using Regex, you should carefully check your patterns to avoid errors. I often use the regex tester at toolcraft.app/en/tools/developer/regex-tester to quickly test patterns before putting them into production code. This saves a lot of debugging time when the script is running live.
Using Async to Prevent Packet Loss
If you process heavy logic inside the loop, the script will bottleneck and drop subsequent packets. The solution is to use AsyncLiveCapture for asynchronous processing.
import asyncio
import pyshark
async def capture_packets():
capture = pyshark.LiveCapture(interface='eth0', display_filter='icmp')
async for packet in capture.sniff_continuously():
print(f"Detected ICMP from: {packet.ip.src}")
loop = asyncio.get_event_loop()
loop.run_until_complete(capture_packets())Important Performance Notes
- Execution Permissions: Capturing packets requires root privileges. Use
sudo dpkg-reconfigure wireshark-commonto allow regular users to run TShark without needing constant sudo. - Memory Management: By default, Pyshark keeps old packets in memory. If running a script 24/7, be careful not to exhaust your RAM. Always clear your data lists periodically.
- BPF vs Display Filter: BPF filters operate at the kernel level and are extremely fast. Display filters filter after TShark has received the packet, consuming more resources. Prioritize BPF whenever possible.
Combining Pyshark with pandas for statistics or sending alerts via Telegram will give you a formidable monitoring system. I hope this tool assists you in your daily system operations.
