Posted inVirtualization

As Fast as Physical Drives: Configuring KVM Shared Folders with virtiofs

Virtualization tutorial - IT technology blog
Virtualization tutorial - IT technology blog

Why is virtiofs the ‘go-to’ solution for KVM virtual machines?

If you’ve been using KVM for a while, you’ve likely faced a common pain point: how to transfer data from the host to the VM as quickly as possible? Previously, the standard solution was 9p (Plan 9). However, 9p’s performance is lackluster. In my tests with a folder containing 10,000 source code files, 9p took nearly 2 minutes to index, while virtiofs took less than 5 seconds.

After multiple benchmarks on my Proxmox homelab, I can confidently say that virtiofs is the top choice. It doesn’t emulate complex network protocols. Instead, it uses shared memory to allow the VM to access the host’s file system directly. Real-world performance can reach 90-95% of a physical drive, far surpassing the 30-40% seen with NFS or 9p.

Preparing your ‘toolkit’ before starting

Virtiofs requires the Host and Guest to be in sync. For a smooth experience, you need to check the following requirements:

  • Host OS: Kernel 5.4 or higher. Ubuntu 20.04 or Debian 11 both work well.
  • QEMU: Version 4.2 or higher is mandatory.
  • Guest OS: Linux Kernel 5.4+ or additional drivers if using Windows.
  • virtiofsd: The I/O handling daemon. On Ubuntu, install the qemu-system-common package.

Detailed Configuration Steps

Step 1: Configure Shared Memory

This is the crucial step. Virtiofs requires the VM to have memory marked as “shared” so that both the host and guest can access it.

Open the VM’s XML configuration file using virsh:

virsh edit <vm_name>

Add the following code block directly under the <domain> tag:

<memoryBacking>
  <source type='memfd'/>
  <access mode='shared'/>
</memoryBacking>

If you are using Hugepages for RAM optimization, the configuration will differ slightly. However, for most use cases, memfd is the most stable option.

Step 2: Add the Filesystem Device to the VM

Still within the XML file, locate the <devices> tag. Declare the folder you want to share here:

<filesystem type='mount' accessmode='passthrough'>
  <driver type='virtiofs'/>
  <source dir='/path/to/host/folder'/>
  <target dir='my_shared_tag'/>
</filesystem>

Note the following parameters:

  • source dir: The absolute path on the Host machine.
  • target dir: A “tag” used when mounting. Keep it short and avoid spaces.
  • accessmode='passthrough': Preserves file ownership (UID/GID) from the host to the guest.

Step 3: Mount the Folder on the Guest VM

Save the XML and restart the VM. Now, log into the Guest to mount the disk.

Create the destination directory:

sudo mkdir -p /mnt/shared_data

Execute the mount command:

sudo mount -t virtiofs my_shared_tag /mnt/shared_data

To have the system mount it automatically on reboot, add this line to the end of the /etc/fstab file:

my_shared_tag  /mnt/shared_data  virtiofs  defaults  0  0

Fixing Permission Errors – A Classic Headache

“I can see the files but can’t write to them” is an error 90% of you will encounter. This happens because the User IDs (UID) on the Host and Guest do not match.

For a development environment, the quickest fix is setting 777 permissions for the source folder. However, this is not secure for production. A more professional approach is to check which user is running the QEMU process using ps aux | grep qemu. Usually, it’s the libvirt-qemu user. Ensure this user has read/write permissions to the source folder on the Host.

Real-world Lab Experience

After 6 months of running virtiofs for a cluster of 12 VMs, I’ve gathered 3 important takeaways:

  1. Avoid heavy Databases: While fast, virtiofs still has higher I/O latency than a standard virtual disk (qcow2). For MySQL or Postgres, prioritize virtual disks to ensure data integrity.
  2. AppArmor/SELinux: If the VM fails to start, check the logs. It’s often AppArmor blocking QEMU from accessing external directories. You’ll need to declare the source directory in /etc/apparmor.d/abstractions/libvirt-qemu.
  3. Handling small files: Virtiofs is incredibly powerful when dealing with node_modules or vendor folders. The execution speed of npm install is 5 times faster compared to using NFS.

Conclusion

Virtiofs eliminates the speed barrier between Host and Guest. The configuration isn’t overly complex, but the performance boost to your workflow is significant. If you’re still struggling with Samba or NFS, switch to virtiofs today.

Good luck with your setup. If you get stuck on permissions or AppArmor, feel free to leave a comment below and I’ll help you out!

Share:
Tags: