Kata Containers on Kubernetes: Securing Pods with MicroVMs When a Shared Kernel Is a Risk

Virtualization tutorial - IT technology blog
Virtualization tutorial - IT technology blog

There’s a truth that many Kubernetes operators rush past: containers are not virtual machines. When you run 50 Pods on the same node, they all share the host kernel. A kernel vulnerability exploited in any single Pod can escalate privileges across the entire node — and from there, across the cluster.

I discovered this problem while building a Kubernetes lab on Proxmox. My homelab runs Proxmox VE managing 12 VMs and containers — it’s my playground for testing everything before pushing to production. When experimenting with multi-tenant workloads (multiple teams sharing the same cluster), I realized Linux namespace isolation isn’t enough when a malicious container or zero-day vulnerability is in play.

The Real Problem: Container Isolation Is Not Impenetrable

Kubernetes Pods use Linux namespaces (pid, net, mnt, uts, ipc) and cgroups for resource isolation. That sounds solid, but the attack surface is still there:

  • Shared kernel syscalls: Every container calls syscalls directly to the host kernel. CVEs like Dirty Pipe (CVE-2022-0847) and the runc vulnerability (CVE-2024-21626) prove this isn’t just theoretical.
  • Container escape via misconfiguration: Pods running with privileged: true, mounting the host /proc, or using hostNetwork: true are extremely common in real-world deployments.
  • Multi-tenant risk: When multiple teams or customers share a cluster, one compromised workload can affect others through the kernel.

In 2019, a vulnerability in runc allowed an attacker with root inside a container to overwrite the runc binary on the host — affecting every Docker and Kubernetes deployment that hadn’t been patched. This was a real container escape, not a lab proof-of-concept.

The Root Cause: Containers Are Just Process Isolation

Containers are fundamentally processes isolated using kernel features. Unlike VMs, containers have no kernel of their own — there’s no hypervisor layer between the workload and the hardware. This delivers great performance, but at the cost of a significantly weaker security boundary.

A quick comparison of the three models:

  • Traditional VM: Guest OS → Hypervisor (KVM) → Hardware. Strong isolation, but high overhead — seconds to boot, hundreds of MB of RAM per VM.
  • Container (runc): Process → Linux namespace/cgroup → Host kernel → Hardware. Fast and lightweight, but the shared kernel is the central weakness.
  • MicroVM: The best of both worlds — lightweight like a container but with its own kernel like a VM, booting in 1–2 seconds.

The question becomes: how do you run Kubernetes workloads with VM-level security boundaries without sacrificing too much performance and compatibility?

Approaches to the Problem

Option 1: gVisor (Google)

gVisor runs a user-space kernel (Sentry) as an intermediary between the container and the host kernel. Syscalls from the container are intercepted and handled by Sentry rather than passed directly to the host kernel. Upside: no hardware virtualization required, runs on any Linux host. Downside: not 100% syscall-compatible, and significant performance hit on I/O-heavy workloads.

Option 2: Standalone Firecracker (AWS)

Firecracker is a VMM (Virtual Machine Monitor) designed for serverless — it’s what powers AWS Lambda and Fargate. Boot time is ~125ms with ~5MB memory overhead per VM. However, direct integration with Kubernetes is complex and requires an additional adapter layer.

Option 3: Kata Containers

Kata Containers combines Kubernetes’ container runtime interface with lightweight VMs. Each Pod runs inside its own MicroVM (using QEMU/KVM or Firecracker as the hypervisor), has its own kernel, yet remains fully compatible with the OCI container ecosystem. This is the approach I chose — large community, solid documentation, and native integration with containerd.

Deploying Kata Containers on Kubernetes

Step 1: Check Hardware Virtualization on the Worker Node

# Check CPU virtualization support
grep -E 'vmx|svm' /proc/cpuinfo | head -1

# Check KVM module
lsmod | grep kvm

# If not loaded:
modprobe kvm_intel   # Intel
# or
modprobe kvm_amd     # AMD

If your worker node is a VM running on Proxmox (like my lab), you need to enable nested virtualization on the Proxmox host:

# Run on the Proxmox host — not inside the worker VM
echo "options kvm_intel nested=1" > /etc/modprobe.d/kvm-intel.conf
modprobe -r kvm_intel && modprobe kvm_intel

# Verify inside the worker node VM:
cat /sys/module/kvm_intel/parameters/nested
# Should return: 1 or Y

Step 2: Install the Kata Containers Runtime on the Worker Node

ARCH=$(uname -m)
KATA_VERSION="3.3.0"

curl -fsSL https://github.com/kata-containers/kata-containers/releases/download/${KATA_VERSION}/kata-static-${KATA_VERSION}-${ARCH}.tar.xz \
  -o /tmp/kata-static.tar.xz

tar -xJf /tmp/kata-static.tar.xz -C /

# Verify successful installation
kata-runtime check
# Output: "System is capable of running Kata Containers"

Step 3: Configure containerd to Recognize the Kata Runtime

Edit /etc/containerd/config.toml to add the Kata handler:

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.kata]
  runtime_type = "io.containerd.kata.v2"

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.kata.options]
  ConfigPath = "/opt/kata/share/defaults/kata-containers/configuration.toml"
systemctl restart containerd
systemctl status containerd

Step 4: Create a RuntimeClass in Kubernetes

RuntimeClass is a Kubernetes resource that lets you specify the container runtime for individual workloads — this is the mechanism that determines which Pods use Kata and which use the default runc:

apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: kata-containers
handler: kata
scheduling:
  nodeSelector:
    kata-runtime: "true"
kubectl apply -f kata-runtimeclass.yaml

# Label worker nodes that support Kata
kubectl label node worker-node-1 kata-runtime=true
kubectl label node worker-node-2 kata-runtime=true

Step 5: Deploy a Pod with the Kata Runtime

Just add runtimeClassName to the Pod spec — no need to change the image or application configuration:

apiVersion: v1
kind: Pod
metadata:
  name: kata-nginx
spec:
  runtimeClassName: kata-containers   # ← The only line you need to add
  containers:
  - name: nginx
    image: nginx:1.25-alpine
    ports:
    - containerPort: 80
    resources:
      requests:
        memory: "64Mi"
        cpu: "100m"
      limits:
        memory: "128Mi"
        cpu: "200m"
kubectl apply -f kata-pod.yaml
kubectl get pod kata-nginx -o wide

Step 6: Verify Real Isolation

The fastest way to verify Kata is working correctly — compare kernel versions:

# Kernel of the host node
uname -r
# Example: 5.15.0-91-generic

# Kernel inside a regular Pod (runc runtime)
kubectl exec regular-pod -- uname -r
# Output: 5.15.0-91-generic  ← SAME kernel as the host

# Kernel inside the Kata Pod
kubectl exec kata-nginx -- uname -r
# Output: 6.1.38             ← Kata MicroVM's OWN kernel

That difference says everything. The Kata Pod is running a completely separate kernel — even if a kernel exploit fires inside the container, it cannot break out of the MicroVM to affect the host or other Pods.

When to Use Kata — and When You Don’t Need It

Based on hands-on experience and production case studies from larger organizations:

  • Use Kata for: multi-tenant clusters, CI/CD runners executing untrusted code (e.g., building code from external contributors), workloads handling sensitive data, payment processing, fintech.
  • Not strictly necessary for: internal microservices that don’t process untrusted input, internal batch jobs, development environments. Use plain runc for better performance.
  • Performance overhead to know about: ~1–2 second boot time (vs. milliseconds for runc), ~50MB additional memory overhead per Pod. For long-running workloads, these numbers are nearly negligible.
  • Limitations to keep in mind: Kata Pods cannot use hostNetwork, hostPID, or hostIPC — this is a feature, not a bug. Those are precisely the things that weaken the security boundary.

I currently run a mixed-runtime cluster in my homelab: internal services use runc, while Pods running self-written test code or images pulled from unclear sources run with Kata. The overhead is acceptable, and the security boundary is far clearer than relying on namespace isolation alone.

Kata Containers doesn’t replace RBAC, network policies, or image scanning — those layers are still essential. But it adds a critical isolation layer at the kernel level, and the deployment cost is far lower than the security benefit you gain in multi-tenant environments or when running untrusted workloads.

Share: