Posted inVMware

Comprehensive Guide to Configuring Link Aggregation (LACP) on VMware vSphere Distributed Switch

At 2:00 AM, my PagerDuty on-call alert was blaring nonstop. An 8-node vSAN cluster was choking on I/O bottlenecks, causing multiple database VMs to drop ping packets. Checking vCenter revealed the culprit: a 10GbE port on ESXi host #3 was flapping persistently. Traffic abruptly surged onto the remaining port. Unfortunately, the default failover mechanism failed to switch over in time, throwing vSAN data out-of-sync.

That costly outage forced our team to completely rethink our network architecture. To permanently eliminate this risk and unlock the full 20Gbps throughput (2x10GbE), aggregating physical NICs via LACP on a Distributed Switch is the gold standard.

3 NIC Teaming Mechanisms on VMware ESXi

In a vSphere infrastructure, you have three common uplink teaming options:

  • Route based on originating virtual port ID (Default): Each VM vNIC is bound to a specific physical uplink in a round-robin fashion. This setup is remarkably straightforward, requiring zero intervention on the physical switch side.
  • Route based on IP hash (Static EtherChannel): Traffic is balanced using a hash of the source and destination IP addresses. This requires configuring a Static LAG (Mode ON) on the physical switch.
  • Dynamic Link Aggregation (LACP) on vDS: Utilizes the IEEE 802.3ad standard. The physical switch and vDS negotiate automatically via periodic LACPDU packets.

Pros and Cons of Each Approach

1. Route based on originating virtual port ID

  • Pros: The easiest to deploy. No dependency on the network team. When a physical switch reboots, hosts rarely suffer from logical link disconnects.
  • Cons: A single VM can never exceed the bandwidth of a single uplink (capped at 10Gbps even with dual ports). During intermittent cable issues or minor frame drops on the switch, the host still considers the link up and will not trigger a failover unless Beacon Probing is enabled.

2. Static EtherChannel (Route based on IP hash)

  • Pros: Dynamic load balancing across multiple distinct IP flows.
  • Cons: Prone to blackhole issues. If the physical switch has misconfigured VLANs or a port hangs unidirectionally, ESXi blindly pushes packets into the degraded port. Neither side has a mechanism to cross-check each other’s link health.

3. LACP on vSphere Distributed Switch (vDS)

  • Pros: Continuous bidirectional negotiation. If a fiber strand degrades (receiving but unable to transmit), LACP automatically evicts that port from the bundle within 3 seconds. Supports rich hashing algorithms (IP, TCP/UDP port, VLAN). Effectively maximizes usable bandwidth.
  • Cons: Requires a vSphere Enterprise Plus license. Demands that virtualization engineers and physical network engineers (Cisco, Arista, Dell) are fully aligned on configuration parameters.

Which Environments Should Enable LACP?

For lab environments with a handful of hosts or setups without Enterprise Plus licensing, the default Port ID mechanism remains a solid choice. You simply need to enable Beacon Probing for redundancy.

However, on heavy production workloads, the equation completely changes. If you run vSAN or perform concurrent vMotion migrations for massive VMs (such as Oracle or SQL Server with 256GB+ RAM), LACP on vDS is practically mandatory. Active LACP decisively eliminates the risk of silent link failures in the dead of night.

Step-by-Step LACP Configuration on vSphere Distributed Switch

Step 1: Configure the LACP Port-Channel on the Physical Switch

Perform this step on your network switch (e.g., a Cisco switch) before making changes in vCenter. Set both ports connecting to the ESXi host to active mode:

! Configure physical ports connected to ESXi-Host03
interface range TenGigabitEthernet1/0/1 - 2
 description Uplinks-to-ESXi03
 switchport mode trunk
 switchport trunk allowed vlan 10,20,30,100
 channel-group 10 mode active
 lacp rate fast
 spanning-tree portfast trunk
 exit

interface Port-channel10
 description LAG-to-ESXi03-vDS
 switchport mode trunk
 switchport trunk allowed vlan 10,20,30,100
 exit

Step 2: Create a Link Aggregation Group (LAG) on the vDS

  1. Navigate to vSphere Client > open the Networking inventory.
  2. Right-click your Distributed Switch (vDS) > select Configure > locate the LACP section.
  3. Click New to add a LAG:
    • Name: lag-esxi-prod
    • Number of ports: 2 (matches the number of vmnics to aggregate)
    • Mode: Active
    • Load balancing mode: Select Source and destination IP address, TCP/UDP port and VLAN to achieve the most balanced traffic distribution.
    • LACP timer: Set to Fast (1 sec) to detect link failures in 3 seconds instead of the default 90 seconds.

Step 3: Update Teaming & Failover on Distributed Port Groups

This is a critical operation. Skipping this step will disconnect network access for all virtual machines (ensure you have an ESXi host configuration backup in place before making major network changes):

  1. On the target Distributed Port Group, right-click and select Edit Settings > go to the Teaming and failover tab.
  2. Move lag-esxi-prod up into Uplinks (Active).
  3. Move all legacy uplinks (Uplink 1, Uplink 2) down to Unused uplinks.

Note: vSphere does not permit mixing a LAG and standalone Uplinks within the same Active group.

Step 4: Assign Physical Network Adapters (vmnics) to LAG Ports

Migrate each vmnic from its legacy uplink to the newly created LAG ports:

  1. Right-click the vDS > select Add and Manage Hosts.
  2. Choose Manage host networking > check the target ESXi hosts.
  3. Under Manage physical network adapters:
    • Assign vmnic2 to port lag-esxi-prod - 0.
    • Assign vmnic3 to port lag-esxi-prod - 1.
  4. Click Next and finish the wizard so vCenter applies the configuration across all hosts.

Step 5: Verify LACP Status via ESXi Shell

Once configuration is complete, SSH into the ESXi host to verify link negotiation status between both ends (which you can also monitor regularly when you automate daily vSphere health checks with PowerCLI):

# Check vDS status on the host
esxcli network vswitch dvs vmware list

# Check detailed LACP negotiation status
esxcli network vswitch dvs vmware lacp status get

# View LACPDU configuration and operational state per NIC
esxcli network vswitch dvs vmware lacp config get

A successful setup is indicated when Actor Partner Agree: Yes appears, alongside port states showing Collecting, Distributing (flags 0x3d or 0x3f). If you see Individual or Defaulted, verify the Native VLAN configuration and ensure LACP mode (Active/Passive) matches on the physical switch.

Share:
Tags: