KubeVirt on k3s: VMs Next to Pods

The Two-Control-Plane Problem

Here’s the homelab situation that will feel familiar: you’ve got Proxmox running your legacy VMs and a k3s cluster for your containerized workloads. Both of them need node resources. Both need monitoring. Both need networking config. You bounce between two UIs, two sets of CLI tools, and two mental models every time you want to change something.

KubeVirt’s pitch is simple — what if the VMs were just another Kubernetes resource? Define a VirtualMachine object, apply it with kubectl, and let the scheduler figure out where it lands. Same RBAC. Same namespaces. Same GitOps pipeline. Your legacy Windows app VM lives in the same manifest repo as your containerized services.

It’s not magic and it’s definitely not a replacement for every Proxmox use case. But for the right workloads — legacy apps that need a full OS, Windows licensing that doesn’t fit containers, network appliances with specific MAC requirements — it’s genuinely compelling.

Let’s set it up on k3s 1.31+ and see what we actually get.

Prerequisites: Don’t Skip This Part

KubeVirt needs hardware virtualization or it’s going nowhere. Before you install anything:

# Check KVM is available on each node
ls -la /dev/kvm

# Verify CPU virtualization is exposed
grep -E '(vmx|svm)' /proc/cpuinfo | head -5

# Check kernel modules are loaded
lsmod | grep kvm

You need kvm and either kvm_intel (VT-x) or kvm_amd (AMD-V) loaded. If /dev/kvm doesn’t exist, go into your BIOS/UEFI and enable virtualization. This is the #1 support question for KubeVirt — don’t skip it.

For large VMs (8+ GB RAM), hugepages improve performance meaningfully. Optional but worth configuring:

# Check current hugepage state
cat /proc/meminfo | grep Huge

# Set 2MB hugepages persistently (adjust count to your RAM)
echo 'vm.nr_hugepages = 1024' >> /etc/sysctl.d/99-hugepages.conf
sysctl -p /etc/sysctl.d/99-hugepages.conf

k3s 1.31+ runs fine with the default Flannel CNI for basic KubeVirt usage. If you want multiple NICs via Multus or SR-IOV, you’ll need to swap CNIs — we’ll touch on that in the networking section.

Installing KubeVirt

KubeVirt deploys as an operator. The operator manages the actual KubeVirt components, which makes upgrades straightforward.

# Pin a version — don't use latest in production
export VERSION=v1.8.0

# Deploy the operator
kubectl apply -f https://github.com/kubevirt/kubevirt/releases/download/${VERSION}/kubevirt-operator.yaml

# Wait for the operator to be ready
kubectl -n kubevirt wait deploy/virt-operator --for=condition=Available --timeout=300s

Now create the KubeVirt custom resource — this is what actually triggers the installation of the virt-api, virt-controller, and virt-handler components:

apiVersion: kubevirt.io/v1
kind: KubeVirt
metadata:
  name: kubevirt
  namespace: kubevirt
spec:
  certificateRotateStrategy: {}
  configuration:
    developerConfiguration:
      useEmulation: false   # must be false — we want real KVM, not software emulation
  customizeComponents: {}
  imagePullPolicy: IfNotPresent
  workloadUpdateStrategy: {}

kubectl apply -f kubevirt-cr.yaml

# Watch the rollout — takes 3-5 minutes
kubectl -n kubevirt get pods -w

Wait until you see virt-api, virt-controller, and virt-handler all in Running state. virt-handler is a DaemonSet — you’ll see one pod per node.

Containerized Data Importer (CDI)

CDI handles disk image management — importing QCOW2/ISO images into PVCs that your VMs can boot from. It’s a separate project but effectively required for real usage.

export CDI_VERSION=v1.65.0

kubectl apply -f https://github.com/kubevirt/containerized-data-importer/releases/download/${CDI_VERSION}/cdi-operator.yaml
kubectl apply -f https://github.com/kubevirt/containerized-data-importer/releases/download/${CDI_VERSION}/cdi-cr.yaml

kubectl -n cdi wait deploy/cdi-operator --for=condition=Available --timeout=300s

virtctl — The Missing kubectl for VMs

virtctl is the companion CLI that handles VM-specific operations kubectl can’t express cleanly — console access, live migration, VNC, SSH proxy. Install it alongside kubectl:

export VERSION=v1.8.0

curl -L -o virtctl \
  https://github.com/kubevirt/kubevirt/releases/download/${VERSION}/virtctl-${VERSION}-linux-amd64

chmod +x virtctl
sudo mv virtctl /usr/local/bin/

# Verify
virtctl version

Your First VM: Fedora Cloud

Here’s where it gets fun. Define a VM the same way you’d define a Deployment — YAML, applied with kubectl.

First, import the Fedora Cloud image as a PVC using CDI:

apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
metadata:
  name: fedora-cloud-base
  namespace: default
spec:
  source:
    http:
      url: "https://download.fedoraproject.org/pub/fedora/linux/releases/44/Cloud/x86_64/images/Fedora-Cloud-Base-Generic-44-1.5.x86_64.qcow2"
  storage:
    resources:
      requests:
        storage: 10Gi
    storageClassName: longhorn   # adjust to your storage class
    accessModes:
      - ReadWriteOnce

kubectl apply -f fedora-dv.yaml

# Watch the import progress
kubectl get datavolume fedora-cloud-base -w

CDI will pull the QCOW2, convert it, and land it in a PVC. Once the DataVolume shows Succeeded, define the VM:

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: fedora-test
  namespace: default
spec:
  runStrategy: Halted   # spec.running is deprecated; use runStrategy. "Always" starts it, or just virtctl start
  template:
    metadata:
      labels:
        kubevirt.io/vm: fedora-test
    spec:
      domain:
        cpu:
          cores: 2
        resources:
          requests:
            memory: 2Gi
        devices:
          disks:
            - name: rootdisk
              disk:
                bus: virtio
            - name: cloudinitdisk
              disk:
                bus: virtio
          interfaces:
            - name: default
              masquerade: {}   # NAT through pod network
          rng: {}   # hardware RNG — cloud-init needs entropy
      networks:
        - name: default
          pod: {}
      volumes:
        - name: rootdisk
          dataVolume:
            name: fedora-cloud-base
        - name: cloudinitdisk
          cloudInitNoCloud:
            userData: |
              #cloud-config
              user: fedora
              password: changeme123
              chpasswd:
                expire: false
              ssh_authorized_keys:
                - ssh-ed25519 AAAA... your-public-key-here
              packages:
                - htop
                - vim
              runcmd:
                - systemctl enable --now cockpit.socket

kubectl apply -f fedora-vm.yaml

# Start it
virtctl start fedora-test

# Check status
kubectl get vm fedora-test
kubectl get vmi fedora-test   # VirtualMachineInstance — the running instance

The VirtualMachine object is like a Deployment — it describes intent. The VirtualMachineInstance is like a Pod — it’s the actual running thing.

Storage: Replicated Disks and Live Migration

This is where your storage choice matters. Basic VMs work fine with ReadWriteOnce storage (one node can mount it). But live migration — moving a running VM to another node — requires ReadWriteMany (RWX) storage that multiple nodes can access simultaneously.

Longhorn (built into many k3s setups) supports RWX with its NFS-backed accessMode:

storage:
  accessModes:
    - ReadWriteMany
  storageClassName: longhorn

Rook-Ceph with CephFS or CephRBD in block mode with RWX is the more performant option if you’re already running Ceph:

storage:
  accessModes:
    - ReadWriteMany
  storageClassName: rook-cephfs

Without RWX storage, virtctl migrate will fail with an error about incompatible storage. Plan this before you start running VMs you actually care about.

Networking Options

Vanilla KubeVirt with the default pod network gives your VM a pod IP — it’s on the cluster network, can reach services, gets DNS resolution. For most use cases this is fine.

Pod Network (Default)

The masquerade interface we defined above uses NAT. The VM talks to the world through the node’s IP. Simple, works everywhere.

Bridge Networking with Multus

When you need the VM to have a real L2 presence on your LAN — static MAC, DHCP from your router, LAN IP — you need Multus CNI and a bridge interface on the host.

Install Multus alongside your existing CNI:

kubectl apply -f https://raw.githubusercontent.com/k8snetworkplumbingwg/multus-cni/master/deployments/multus-daemonset.yml

Create a NetworkAttachmentDefinition:

apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
  name: lan-bridge
  namespace: default
spec:
  config: |
    {
      "cniVersion": "0.3.1",
      "name": "lan-bridge",
      "type": "bridge",
      "bridge": "br0",
      "ipam": {}
    }

Then add it to your VM spec:

spec:
  template:
    spec:
      domain:
        devices:
          interfaces:
            - name: default
              masquerade: {}
            - name: lan
              bridge: {}
      networks:
        - name: default
          pod: {}
        - name: lan
          multus:
            networkName: lan-bridge

The VM gets two NICs — one on the pod network, one bridged to your physical LAN. The MAC address is deterministic from the VM name, so your DHCP server can give it a static lease.

SR-IOV for High-Performance Networking

If you’re running network appliances or doing anything throughput-sensitive, SR-IOV lets the VM talk directly to the NIC hardware, bypassing the kernel networking stack. Requires an SR-IOV capable NIC (most Intel 10G+ cards qualify) and the SR-IOV Device Plugin. That’s a whole separate rabbit hole — start with bridge networking and revisit SR-IOV if you hit actual performance problems.

Accessing Your VM

Three ways to get in:

# Serial console (works even if network is misconfigured)
virtctl console fedora-test

# SSH proxy through the API server (no need to know the VM's IP)
virtctl ssh fedora@fedora-test

# VNC (requires a browser or VNC client)
virtctl vnc fedora-test

The SSH proxy is particularly handy — it tunnels through the Kubernetes API, so you don’t need to know the VM’s pod IP or set up any port forwarding.

Live Migration

This is the party trick. Move a running VM from one node to another with no downtime:

# Check which node the VM is running on
kubectl get vmi fedora-test -o wide

# Initiate migration
virtctl migrate fedora-test

# Watch the migration status
kubectl get vmim -w   # VirtualMachineInstanceMigration objects

The migration creates a VirtualMachineInstanceMigration object, which you can watch with kubectl get vmim. Memory is transferred iteratively while the VM is running, and the final cutover is typically a few hundred milliseconds.

Requirements for live migration:

• RWX storage (as discussed above)
• At least 2 nodes
• Sufficient RAM on the target node

If your storage is RWO, the migration will be blocked. KubeVirt will tell you this clearly in the migration object’s status.

The Reality Check

Overhead is genuinely small. KubeVirt’s virtualization layer adds roughly 3-5% CPU overhead compared to running KVM directly with libvirt. Memory overhead is the virt-handler pod per node (~150MB) plus the QEMU process overhead per VM (another ~50-100MB per VM). For most workloads, you won’t notice it.

What KubeVirt is NOT going to fix:

GPU passthrough for gaming VMs. The full Proxmox GPU passthrough experience — binding the GPU to vfio-pci, configuring Looking Glass, ROM file shenanigans — works at the hardware level in ways that don’t map cleanly onto Kubernetes scheduling. KubeVirt has GPU support (via the gpu device plugin for ML workloads), but if you want to play games in a VM, Proxmox is still the right tool. KubeVirt’s GPU support is aimed at CUDA compute, not graphics output.

USB passthrough. Passing specific USB devices to VMs (dongles, hardware keys, audio interfaces) is possible but fiddly in KubeVirt. Proxmox does it better.

Windows licensing complexity. Windows VMs work fine in KubeVirt, but the MAC address changes on each node if you’re not careful about network configuration. AVMA/KMS activation can get annoyed by this. Pin the MAC explicitly in your VM spec.

For actual homelab workloads — a Windows VM running some legacy app, a network appliance, an old Linux service that needs a specific kernel version, or anything that legitimately doesn’t containerize well — KubeVirt handles it cleanly.

Should You Bother?

If your homelab already has k3s and you’re running Proxmox purely to host a handful of VMs that support containerized workloads, collapsing that into one control plane with KubeVirt is a genuine win. You get:

• One place for GitOps: VMs and pods in the same manifests
• Kubernetes RBAC for VM access control
• Live migration for maintenance without coordinating between two platforms
• The same monitoring stack covers everything

If you’re running a Proxmox cluster specifically for the VM management experience — snapshots, templates, ZFS integration, easy cloning — KubeVirt’s ergonomics aren’t quite there yet. It’s better than it was two years ago, but the Proxmox UI and tooling are still more mature for VM-heavy workflows.

The sweet spot is a cluster that’s primarily containers, with a few VMs that need to exist for legitimate reasons. That’s where KubeVirt stops being an interesting experiment and starts being the right answer.

Your 2 AM self, staring at a Proxmox upgrade that took down a VM your pods depended on, will wish you’d done this earlier.

Related Reading

• Capsule: Multi-Tenancy for Home Lab k3s
• Cilium on k3s: When eBPF Networking Pays
• Crossplane vs Terraform for Home Lab
• Goldilocks + VPA: Right-Size Pods Without Guessing
• Headlamp: K8s UI Without the License Drama