[q&a] Status of IOMMU virtualization for nested virtualization (userspace PCI drivers in VMs)

[Peter Delevoryas <peter@pjd.dev>]

Hey guys,

I’m having a little trouble reading between the lines on various docs, mailing list threads, KVM presentations, github forks, etc, so I figured I’d just ask:

What is the status of IOMMU virtualization, like in the case where I want a VM guest to have a virtual IOMMU?

I found this great presentation from KVM Forum 2021: [1]

1. I’m using -device intel-iommu right now. This has performance implications and large DMA transfers hit the vfio_iommu_type1 dma_entry_limit on the host because of how the mappings are made.

2. -device virtio-iommu is an improvement, but it doesn’t seem compatible with -device vfio-pci? I was only able to test this with cloud-hypervisor, and it has a better vfio mapping pattern (avoids hitting dma_entry_limit).

3. -object iommufd [2] I haven’t tried this quite yet, planning to: if it’s using iommufd, and I have all the right kernel features in the guest and host, I assume it’s implementing the passthrough mode that AMD has described in their talk? Because I imagine that would be the best solution for me, I’m just having trouble understanding if it’s actually related or orthogonal. I see AMD has -device amd-viommu here [3], is that ever going to be upstreamed or is that what -object iommufd is abstracting? I also found this mailing list submission [4], and the context and changes there imply this is all about that (exposing iommu virtualization to the guest)

Thanks!
Peter
 
[1] https://static.sched.com/hosted_files/kvmforum2021/da/vIOMMU%20KVM%20Forum%202021%20-%20v4.pdf
[2] https://www.qemu.org/docs/master/devel/vfio-iommufd.html
[3] https://github.com/AMDESE/qemu/commit/ee056455c411ee3369a47c65ba8a54783b5d2814
[4] https://lore.kernel.org/lkml/20230621235508.113949-1-suravee.suthikulpanit@amd.com/