The 2 AM Disk Latency Nightmare
It’s 2 AM, and the phone starts vibrating uncontrollably. The Zabbix monitoring system is flashing a red alert: Disk I/O Latency on the Database server has hit 500ms. An unbelievable number, considering the underlying physical servers are running entirely on Samsung PM1733 NVMe Enterprise drives.
After checking via SSH, I discovered the virtual machine (VM) was still using the LSI Logic SAS controller. This is a default setting from older VMware versions kept for compatibility. For modern applications, this “bottleneck” is strangling your SSD hardware performance.
Switching to Virtual NVMe (vNVMe) is the fastest way to leverage the power of physical SSDs. You will see a noticeable improvement in speed without spending a dime on new hardware.
Why LSI Logic SAS or SATA Is No Longer Good Enough
The habit of clicking “Next” repeatedly when creating a VM often leaves you stuck with legacy standards. The problem lies in technological limitations:
- LSI Logic SAS: Designed for mechanical hard drives (HDD). It creates significant software overhead when processing data.
- SATA: Command Queue Depth is limited to just 32.
- Virtual NVMe: Supports up to 64k queues, with each queue processing 64k commands simultaneously. It reduces the number of CPU cycles required to handle each I/O task.
If you are using real NVMe SSDs but emulating SCSI on top, you are wasting about 20-30% of actual performance. For example, an NVMe drive capable of 500,000 IOPS might only reach 350,000 IOPS through an old controller.
Prerequisites for Upgrading to vNVMe
Don’t rush into changes just yet. Check the list below to avoid Blue Screen of Death (BSOD) errors:
- Hardware Version: The VM must be version 13 or higher. This corresponds to ESXi 6.5 or VMware Workstation 14.
- Operating System: Windows 8.1 or Windows Server 2012 R2 and above. For Linux, Kernel version 3.3 or higher is required (Ubuntu 20.04+, CentOS 7+ are well-supported).
- Backup: Take a snapshot of the VM before proceeding. Don’t let a small mistake turn your night shift into an OS reinstallation disaster.
How to Safely Switch on Windows
If you change the Controller directly in Settings, Windows will trigger an “Inaccessible Boot Device” error because the system hasn’t loaded the NVMe driver into the boot process. Use this “driver priming” trick instead.
Step 1: Add a Dummy NVMe Drive
- Power off the VM completely.
- Go to Edit Settings and add a new NVMe Controller.
- Add a new 1GB Hard Disk and assign it to the newly created NVMe controller.
- Boot Windows normally.
Step 2: Load the Driver into the System
Once on the desktop, Windows will automatically recognize the new hardware. Open Device Manager to verify. If “Standard NVM Express Controller” appears under Storage controllers, you are successful.
Step 3: Convert the OS Drive
- Power off the VM. Remove the 1GB disk and the temporary NVMe Controller from Step 1.
- Select the primary drive (Hard Disk 1). Change the Virtual Device Node from SCSI to NVMe (0:0).
- VMware will automatically re-enable the NVMe controller for the boot drive.
- Power on the VM and check the boot speed.
Deployment on Linux (Ubuntu/CentOS/Debian)
Linux handles NVMe drivers more flexibly as they are usually built into the kernel. However, you must ensure the driver is present in initramfs to avoid mount errors.
Check for NVMe support with the command:
grep -i nvme /boot/config-$(uname -r)If the result returns CONFIG_NVME_CORE=y, you are ready. Load the module into the boot process:
# For Ubuntu/Debian
echo "nvme" | sudo tee -a /etc/initramfs-tools/modules
sudo update-initramfs -u
# For CentOS/RHEL
sudo dracut --force --add-drivers nvmeImportant note: The drive name will change from /dev/sda to /dev/nvme0n1. You should use UUIDs in your /etc/fstab file to prevent the system from hanging during reboot.
Real-World Results After Optimization
I measured the performance again using fio and CrystalDiskMark after the migration. The metrics showed very positive changes:
- Latency: Reduced by an average of 15%. Database logging tasks respond significantly faster.
- Random Read/Write (4K): Increased by about 20% due to leveraging large command queues.
- CPU Overhead: Reduced host CPU load when processing heavy data streams.
Quick test on Linux with the fio command:
fio --name=test --ioengine=libaio --rw=randwrite --bs=4k --size=1g --iodepth=32 --runtime=60 --direct=1Conclusion
Optimizing virtual machines isn’t just about adding more RAM or CPU. Sometimes, a small change in the disk controller delivers performance beyond expectations. If you own high-end SSD infrastructure, don’t let it run with the “handbrake” called LSI Logic. Good luck with your smooth systems and peaceful night shifts!
