Posted inVMware

Server Optimization with VMware vSphere Distributed Services Engine: Stop Making the CPU Carry the Load Alone

VMware tutorial - IT technology blog
VMware tutorial - IT technology blog

The Struggle of a CPU Wearing Too Many Hats

After more than 6 months of operating a vSphere 8 system on a Dell PowerEdge R750 cluster equipped with NVIDIA BlueField-2 cards, I realized a hard truth: the CPU is being excessively wasted. Typically, about 20-30% of CPU resources (equivalent to 8-12 cores on a dual-socket system) are consumed just to handle “background” tasks.

Tasks such as packet encapsulation, NSX firewall checks, or managing storage traffic consume a lot of resources. When running large databases or I/O-heavy applications, CPU overhead causes latency to skyrocket. This is where vSphere Distributed Services Engine (DSE) proves its value. Instead of forcing the main CPU to juggle everything, we offload the entire network and security burden to a DPU (Data Processing Unit)—a specialized chip right on the network card.

How Does vSphere Distributed Services Engine Work?

Put simply, vSphere DSE turns the DPU card into a “powerful assistant” running in parallel with the main CPU. Your server will now run two ESXi instances: a full version running on the traditional x86 CPU and a slimmed-down version running on the ARM cores of the DPU card.

With the old architecture, every packet passing through the Distributed Switch had to visit the main CPU for inspection. vSphere 8 changes this game. Data flows directly from the virtual machine (VM) to the DPU and is processed “on-site.” The main CPU is now completely free to focus on processing application logic.

DPU – More Than Just a Network Card

DPU lines like NVIDIA BlueField or AMD Pensando are essentially miniature computers. They have their own CPU, RAM, and operating system. When integrated into vCenter, you will see them appear as a true Hardware Accelerator rather than just a standard NIC.

Real-world Deployment Process

Deploying DSE requires careful hardware preparation. It’s not just a matter of plugging in the card and running; you need to pay attention to the following steps.

1. Checking Compatibility and Firmware

First, carefully check the VMware HCL. I once wasted an entire morning just because the BlueField-2 card wasn’t recognized correctly due to outdated firmware. Ensure the DPU firmware is synchronized with the ESXi version you plan to install.

In the server BIOS, enable the following options:

  • SR-IOV: Enable (Required for resource allocation).
  • IOMMU / VT-d: Mode that allows direct memory access.
  • PCIe Slot Bifurcation: Usually set to Auto or x16 depending on the vendor documentation.

2. Parallel Installation via vSphere Lifecycle Manager (vLCM)

The biggest difference is that you don’t install ESXi on the host first and figure out the rest later. We use vLCM to push the image to both the Host and the DPU simultaneously.

  1. Create a Cluster Image in vCenter including the ESXi base image.
  2. Add the corresponding Vendor Addon (e.g., NVIDIA BlueField-2 addon).
  3. When selecting Remediation, check “Enable Distributed Services Engine”.

After installation, use this command to check if the DPU is “alive”:

# List DPU devices on the host
esxcli network ens dpu list

# View vmnic connection status
esxcli network ens dpu get -n vmnic0

3. Configuring the Distributed Switch (vDS)

You need to create a vSphere Distributed Switch version 8.0 or higher to activate the offload feature.

  1. Under Network Offloads Mode, switch from None to DPU.
  2. Assign the DPU uplinks to the switch.
  3. Port Groups created thereafter will automatically benefit from hardware acceleration.

If the system has NSX, Micro-segmentation rules will be pushed directly to the ARM chip on the DPU. This helps block attacks right at the network card gateway.

Verifying Performance with Data

To know if the system is truly offloading, I usually use the vsish tool. This is the most accurate way to look into the ESXi kernel layer.

# Check offloaded packet statistics
vsish -e get /net/portsets/vSwitch0/ports/[VM_ID]/vmware/offloadStats

If you see the offload packets metric increasing continuously, congratulations, you have successfully configured it.

Notes After 6 Months of Real-world Use

When moving vSAN and NSX clusters to the DSE model, I noted the three most valuable changes:

  1. CPU Liberation: On hosts running continuous backups, CPU load dropped from 40% to about 15%. The system runs much more smoothly.
  2. Maxed-out Bandwidth: Transfer speeds reached a stable 25Gbps line-rate. Application latency decreased by about 30% because there is no longer resource contention with the network kernel.
  3. Independent Security: Even if you hit a buggy ESXi patch that hangs the main kernel, the firewall rules on the DPU remain standing, protecting the VMs inside.

However, a small note for everyone is that vSphere DSE requires an Enterprise Plus license. Troubleshooting will also take more time because you are managing an additional embedded operating system on the network card.

Conclusion

DSE is not just a new feature; it’s how we redefine server power. The CPU should be reserved for making money (running apps), while the “heavy lifting” of packets should be left to the DPU. If you are planning to refresh your infrastructure, investing in a SmartNIC right now is an extremely economical choice in the long run.

Share:
Tags: