Why does tcpdump-uw often let you down?
If you frequently manage VMware ESXi, you’re likely familiar with tcpdump-uw. However, this tool has a fatal flaw. It only captures traffic passing through VMkernel interfaces like Management or vMotion. When you need to investigate why VM A cannot ping VM B, or why packets from a physical switch “disappear” after passing through a vmnic network card, tcpdump-uw is completely helpless.
VMware solved this problem with pktcap-uw (available since ESXi 5.5). This tool allows you to “intercept” packets at every point: from vSwitches and Distributed Switches to physical Uplinks. It’s like having a surveillance camera at every intersection of your virtual network.
Capture packets on ESXi in 60 seconds
First, enable SSH on your ESXi Host and log in. Suppose you want to check if traffic is actually passing through the physical network card vmnic0. Try capturing the first 10 packets:
# Capture 10 packets on uplink vmnic0 and display them directly
pktcap-uw --uplink vmnic0 -c 10To avoid screen clutter from too much junk traffic, filter for the specific IP of the VM experiencing issues (e.g., 192.168.1.50):
# Filter by specific IP to narrow the scope
pktcap-uw --uplink vmnic0 --ip 192.168.1.50Note: Don’t forget to press Ctrl + C to stop. If you don’t use the -c parameter, the command will run continuously. On 10Gbps links, this can put significant pressure on the Host’s CPU.
Decoding Core Parameters
To use pktcap-uw effectively, you need to understand where the packet is getting stuck in the chain: Physical NIC → vSwitch → Portgroup → Virtual NIC.
1. Capture Points
--uplink [vmnicX]: Check traffic at the physical network card.--vmport [Port ID]: Check traffic at the VM’s port. Use thenet-stats -lcommand to get the correct Port ID.--vswitch [vSwitch Name]: Capture packets at the virtual switch level.
2. Direction
You need to know if the packet is entering or leaving the capture point:
--dir 0: Inbound (Packets entering the Host or VM).--dir 1: Outbound (Packets leaving the VM or Host).
3. Stage
This is a great feature to see if the ESXi Firewall is blocking traffic:
--stage 0: Before processing (Pre-check).--stage 1: After filters (Post-check).
How to Export PCAP Files for Wireshark Analysis
Reading text logs on a terminal is eye-straining, especially with complex TCP streams. The best way is to export to a .pcap file for analysis in Wireshark.
# Save VM traffic to Datastore as a pcap file
pktcap-uw --vmport 33554440 -o /vmfs/volumes/DATASTORE_NAME/debug_network.pcapAfter running the command, perform a few pings or access the web from the VM to generate traffic. Then, download the file to your computer via WinSCP.
Tip: If you only need to see headers (IP, Port) without the packet content, add -s 128. The log file will be much smaller, allowing you to capture for longer periods.
Real-world Experience: Avoiding Silly Mistakes
After many system “rescue” missions, I’ve gathered 4 golden rules:
- Stay away from the root partition: Never save pcap files to
/tmpor/. These partitions on ESXi are very small. A 500MB capture file can crash the entire Host immediately. Always save to a Datastore. - Always check net-stats: A VM’s Port ID will change if you restart the VM or vMotion it to another Host. Run
net-stats -lright before capturing to get the latest ID. - Clean up processes: Sometimes you press Ctrl+C, but the process still runs in the background. Use
ps -c | grep pktcap-uwto check. If it still exists, usekill -9to free up the CPU. - Watch the CPU: Capturing packets on a heavily loaded 25Gbps card can push the CPU to 100%. Prioritize using Port or IP filters to reduce system load.
Mastering pktcap-uw helps you stop guessing. Instead of arguing with the Network team, you just provide the pcap file as concrete evidence of where the packet was dropped.
