Posted inVMware

Eliminate ‘Noisy Neighbor’ Issues with VMware SIOC: Protecting Bandwidth for Critical VMs

VMware tutorial - IT technology blog
VMware tutorial - IT technology blog

The ‘Noisy Neighbor’ Nightmare in Storage Infrastructure

Is your ERP system running smoothly and then suddenly lagging because some ‘anonymous’ VM is running a virus scan? You’ve just fallen victim to the Noisy Neighbor effect. This happens when one virtual machine consumes all the disk bandwidth (I/O), causing severe bottlenecks for critical servers nearby.

Over years of managing large clusters, I’ve realized that static resource allocation often fails during traffic spikes. VMware vSphere Storage I/O Control (SIOC) is the ultimate solution. Unlike rigid limits, SIOC only intervenes when it detects latency exceeding safe thresholds. It acts like an intelligent traffic cop, only directing lanes when the road starts to get congested.

Compared to the limiting mechanisms in Proxmox, VMware’s SIOC is more powerful due to its ability to coordinate across all hosts in a cluster. It sees the big picture rather than just managing locally on individual nodes.

Prerequisites for Deploying SIOC

Don’t rush to enable SIOC if your system hasn’t met the following technical standards:

  • License: Requires the Enterprise Plus edition. This is often the biggest hurdle for small businesses.
  • Datastore: Well-supported on VMFS and NFS. Note that SIOC does not support Datastores with multiple Extents (multiple LUNs spanned together).
  • vCenter Server: Must have vCenter to coordinate activities between ESXi hosts.
  • Hardware: If your storage array already has automatic QoS features, consider disabling one of the two to avoid algorithmic conflicts.

Steps to Enable SIOC on a Datastore

We need to enable the SIOC ‘control center’ at the datastore level before configuring individual VMs. This is where vSphere monitors latency for all connected hosts.

Step 1: Enable the SIOC Feature

  1. Open the vSphere Client and navigate to the Storage tab.
  2. Right-click on the datastore you want to configure and select Configure -> General.
  3. In the Datastore Capabilities section, select Edit.
  4. Check the Enable Storage I/O Control box and click OK.

Step 2: Determine the Congestion Threshold

There are two ways to tell SIOC when to intervene:

  • Percentage of Peak Throughput: Default is 90%. When the datastore reaches 90% of its peak performance, SIOC begins regulation.
  • Manual (Latency): This is the method I recommend. For SSD/All-flash drives, set the threshold between 10-15ms. If using high-end NVMe, this can be lower (around 5ms). For traditional HDDs, 30ms is a reasonable number.

Note: Don’t set the threshold too low. Having SIOC intervene constantly when the system is still performing well will waste CPU processing resources.

Prioritizing Virtual Machines (VM Shares & Limits)

After activating SIOC, it’s time to decide who the ‘VIPs’ are. We use the Shares mechanism to allocate priority.

  1. Right-click on a critical VM (e.g., SQL Server) -> Edit Settings.
  2. Go to Virtual Hardware -> expand the Hard disk section.
  3. Under Shares, select High (2000 shares). For test VMs, select Low (500 shares).

This mechanism works proportionally. Under congestion, a VM with 2000 shares will get 4 times the priority for data processing compared to one with 500 shares. When the path is clear, everyone runs at maximum speed, which is a great way to increase disk throughput for sensitive workloads.

Want to ‘leash’ a resource-hungry VM? Enter a value in the Limit – IOPS field to set a hard cap on the maximum bandwidth it is allowed to reach.

Fast Management with PowerCLI

Instead of clicking through dozens of datastores, professionals often use scripts to ensure consistency, which is a core principle of automating VMware vSphere.

# Check the SIOC status across the entire system
Get-Datastore | Select Name, StorageIOControlEnabled, CongestionThresholdMillisecond | Format-Table -AutoSize

# Enable SIOC and set a 15ms threshold for the Production Datastore
Set-Datastore -Datastore "DS_Production_01" -StorageIOControlEnabled $true -CongestionThresholdMillisecond 15

How to Verify if SIOC is Actually Working?

Don’t just trust the configuration; look at the actual numbers. The most powerful tool is esxtop.

  1. SSH into the ESXi Host and type esxtop.
  2. Press v to view VM disk metrics.
  3. Observe the QUED (Queue Depth) and DAVG (Device Latency) columns.

If you see the QUED for low-priority VMs increasing while critical VMs maintain a low DAVG, it means SIOC is doing its job—throttling the right people and saving the right targets.

Lessons Learned: When Should You Disable SIOC?

SIOC is not a silver bullet. I’ve encountered issues using SIOC with storage that features Auto-tiering (automatically moving data between SSD and HDD tiers). When the storage moves data in the background, latency spikes artificially, causing SIOC to misinterpret it as congestion and unfairly throttle bandwidth.

Additionally, SIOC is useless if there’s an external physical server (not part of vCenter) also connected to the same SAN. SIOC can only manage what is within vSphere’s visibility.

I hope this article helps you master storage bandwidth and stop worrying about naughty ‘neighbors’ breaking your system!

Share:
Tags: