How to Prevent VMs from “Fighting” Over Resources?
Have you ever seen an ESXi Host with 40% CPU idle, yet the virtual machines (VMs) inside are lagging? For virtualization admins, the biggest fear isn’t a server crash, but a system running at a “snail’s pace” for no apparent reason. Often, the issue isn’t a lack of hardware, but how we allocate resources.
When I first moved from a personal lab to managing enterprise vSphere clusters, I made a classic mistake: Over-provisioning. I thought throwing more vCPUs at a VM would make it faster. Instead, CPU Ready Time spiked over 15%, VMs fought desperately for resources, and the entire system stalled. In this article, I’ll dissect how to use Shares, Reservations, and Limits so you don’t make the same mistake.
Context: Why Default Configurations Aren’t Enough
By default, vSphere shares resources fairly on a “first-come, first-served” basis. However, in reality, not all VMs are equally important. A Database Server serving thousands of users definitely needs higher priority than a Print Server or a Web Test environment.
If you ignore configuration, you’ll soon face the “Noisy Neighbor” effect. A single VM with a CPU loop or under a DDoS attack can tank the performance of every other VM sharing that Host.
Activating the Heavy Artillery: vSphere DRS
To manage resources effectively, you need vCenter Server and ESXi Hosts grouped into a Cluster. Specifically, enable vSphere DRS (Distributed Resource Scheduler). This feature allows vCenter to automatically migrate (vMotion) VMs from an overloaded Host to one with more available capacity.
- Access vSphere Client and select your Cluster.
- Go to the Configure tab and select vSphere DRS.
- Click Edit and toggle the status to On.
Note: If you’re using the Free version of ESXi, you can still adjust Shares/Limits for individual VMs, but you’ll lack the automatic load balancing capabilities of Resource Pools.
Dissecting the Trio: Shares, Reservations, and Limits
Understanding these three parameters is the key to mastering system performance.
1. Reservations (Minimum Commitment)
This is the amount of resources ESXi guarantees to set aside for a VM. If the Host doesn’t have enough (e.g., 2GB RAM) to meet the Reservation, the VM won’t start.
Advice: Only set Reservations for mission-critical applications like SQL Server or Oracle. Don’t overdo it for every VM, as it negates the benefit of “over-commitment”—the primary advantage of virtualization.
2. Limits (Maximum Ceiling)
Limits are hard caps. Even if the Host is 90% idle, the VM is not allowed to exceed this number.
Warning: I strongly advise against setting RAM Limits. When a VM hits its RAM cap, ESXi forces it to swap data to the disk. Access speeds drop from nanoseconds (RAM) to milliseconds (Disk), causing the VM to virtually freeze.
3. Shares (Relative Priority)
Shares only kick in during resource contention. When a Host is congested, VMs with higher Shares get priority.
For example, if you set a Database VM to High and a Web VM to Normal, the Database will get twice the resources of the Web VM during a CPU spike. This is the smartest way to regulate resources because it doesn’t cause waste when the system is idle.
Optimizing Disk I/O with Storage I/O Control (SIOC)
Don’t just look at CPU/RAM. Disk I/O is often the bottleneck that causes system stutters. When one VM runs a virus scan, it can hog all disk bandwidth, pushing latency above 50ms and freezing other VMs.
Enable Storage I/O Control (SIOC) on your Datastore to solve this:
- Go to the Datastores list and select the target Datastore.
- Go to Configure -> General.
- Under Datastore Capabilities, enable Storage I/O Control.
Once enabled, adjust the disk Shares for each VM. For DB Servers, I usually set Shares to 2000, while secondary servers stay at 500 or 1000.
Reading the “Telling” Metrics via esxtop
To see if your settings are effective, look at the real data. SSH into the Host and run the esxtop command.
- Press ‘c’ for CPU: Watch the %RDY column. If this number is > 10%, your VM is “waiting in line” too long. Reduce the vCPUs of other VMs or increase its Shares.
- If %MLMTD > 0: The VM is being throttled by the Limit you set. Loosen the limit if the VM is slow.
- Press ‘m’ for RAM: Look at the SWCUR column. If this is greater than 0, the VM is swapping data to disk. You need to add more RAM immediately.
Quick Check with PowerCLI
If you’re managing hundreds of VMs, you can’t check them manually. This script quickly lists VMs with hard Limits set:
Get-VM | Get-VMResourceConfiguration | Where-Object {$_.CPULimitMhz -ne -1 -or $_.MemLimitMB -ne -1} |
Select-Object VM, CPULimitMhz, MemLimitMB | Format-Table -AutoSize
Golden Rules from Practical Experience
After years of operation, I’ve distilled a few survival principles:
- Fewer vCPUs are better: A VM with 2 vCPUs at 80% load often runs smoother than a VM with 8 vCPUs at only 10% load. This reduces pressure on the CPU Scheduler.
- Prioritize Shares over Limits: Let the system regulate itself flexibly instead of strangling it with rigid numbers.
- Always leave breathing room for the Host: Never use more than 90% of the Host’s RAM. Leave at least 10% for ESXi system processes.
I hope these insights help you feel more confident in optimizing your virtual environment. If you encounter unusually high %RDY metrics and aren’t sure how to handle them, leave a comment below!
