RE: How to express "externally managed" IOMMU domains for VFIO/IOMMUFD ?

RE: How to express "externally managed" IOMMU domains for VFIO/IOMMUFD ?

[Tian, Kevin]

> From: Teddy Astie
> Sent: Wednesday, April 22, 2026 11:59 PM
> 
> Hello,
> 
> On Xen, for PV-IOMMU [1], we have IOMMU support in Dom0, which in
> particular allows using VFIO and IOMMUFD from Dom0.
> 
> However, its interactions with PCI Passthrough are unclear, and it would

VFIO manages PCI passthrough. since it's already allowed which part of
interaction is unclear?

> be preferable to let the kernel handle some of this logic. That would
> for instance avoid situations where toolstack causes Xen and Linux to go
> out of sync on where devices belong.

what is 'some of this logic' and what is the exact out-of-sync scenario?

> 
> On Xen, we have a dedicated hypercalls for moving a device into another
> guest (so it no longer belongs in Dom0, at far as DMA is concerned).
> 
> But it looks like there are no way to describe that idea of "attach that
> device to this VM" nor "the device is in a VM"; which makes that
> impracticable.
> 
> There may be things that could be done with the vIOMMU objects, but
> there would be no "parent domain" in such case, as said earlier it
> doesn't exist in the IOMMU subsystem.
> 
> What is expected to be done instead ?
> 
> Teddy
> 
> [1] https://www.youtube.com/watch?v=pLMGRgEJ-Eg
> 

It'd be much easier to collect comments if you can put plain words
to explain the problem rather than expecting other folks to watch
the video first...

Re: How to express "externally managed" IOMMU domains for VFIO/IOMMUFD ?

[Jason Gunthorpe]

On Thu, Apr 23, 2026 at 08:01:50AM +0000, Tian, Kevin wrote:
> > On Xen, we have a dedicated hypercalls for moving a device into another
> > guest (so it no longer belongs in Dom0, at far as DMA is concerned).
> > 
> > But it looks like there are no way to describe that idea of "attach that
> > device to this VM" nor "the device is in a VM"; which makes that
> > impracticable.
> > 
> > There may be things that could be done with the vIOMMU objects, but
> > there would be no "parent domain" in such case, as said earlier it
> > doesn't exist in the IOMMU subsystem.
> > 
> > What is expected to be done instead ?
> > 
> > Teddy
> > 
> > [1] https://www.youtube.com/watch?v=pLMGRgEJ-Eg
> > 
> 
> It'd be much easier to collect comments if you can put plain words
> to explain the problem rather than expecting other folks to watch
> the video first...

It sounds like CC and pkvm to me so I think it should re-use those
mechanisms..

Jason

RE: How to express "externally managed" IOMMU domains for VFIO/IOMMUFD ?

[Tian, Kevin]

> From: Jason Gunthorpe <jgg@ziepe.ca>
> Sent: Sunday, April 26, 2026 9:30 PM
> 
> On Thu, Apr 23, 2026 at 08:01:50AM +0000, Tian, Kevin wrote:
> > > On Xen, we have a dedicated hypercalls for moving a device into another
> > > guest (so it no longer belongs in Dom0, at far as DMA is concerned).
> > >
> > > But it looks like there are no way to describe that idea of "attach that
> > > device to this VM" nor "the device is in a VM"; which makes that
> > > impracticable.
> > >
> > > There may be things that could be done with the vIOMMU objects, but
> > > there would be no "parent domain" in such case, as said earlier it
> > > doesn't exist in the IOMMU subsystem.
> > >
> > > What is expected to be done instead ?
> > >
> > > Teddy
> > >
> > > [1] https://www.youtube.com/watch?v=pLMGRgEJ-Eg
> > >
> >
> > It'd be much easier to collect comments if you can put plain words
> > to explain the problem rather than expecting other folks to watch
> > the video first...
> 
> It sounds like CC and pkvm to me so I think it should re-use those
> mechanisms..
> 

for CC and pkvm the guest memory is still allocated from host.

for Xen the guest memory is allocated from hypervisor and invisible
to Dom0. iirc its device assignment is implemented by the toolstack
issuing hypercall to hypervisor, bypassing the Dom0 kernel.

I don't know the latest status in Xen side. Seems it's still the case and
Astie is trying to find a way to orchestrate it via VFIO. But it's unclear
what his proposal is...

Re: How to express "externally managed" IOMMU domains for VFIO/IOMMUFD ?

[Jason Gunthorpe]

On Thu, May 07, 2026 at 08:02:40AM +0000, Tian, Kevin wrote:
> > From: Jason Gunthorpe <jgg@ziepe.ca>
> > Sent: Sunday, April 26, 2026 9:30 PM
> > 
> > On Thu, Apr 23, 2026 at 08:01:50AM +0000, Tian, Kevin wrote:
> > > > On Xen, we have a dedicated hypercalls for moving a device into another
> > > > guest (so it no longer belongs in Dom0, at far as DMA is concerned).
> > > >
> > > > But it looks like there are no way to describe that idea of "attach that
> > > > device to this VM" nor "the device is in a VM"; which makes that
> > > > impracticable.
> > > >
> > > > There may be things that could be done with the vIOMMU objects, but
> > > > there would be no "parent domain" in such case, as said earlier it
> > > > doesn't exist in the IOMMU subsystem.
> > > >
> > > > What is expected to be done instead ?
> > > >
> > > > Teddy
> > > >
> > > > [1] https://www.youtube.com/watch?v=pLMGRgEJ-Eg
> > > >
> > >
> > > It'd be much easier to collect comments if you can put plain words
> > > to explain the problem rather than expecting other folks to watch
> > > the video first...
> > 
> > It sounds like CC and pkvm to me so I think it should re-use those
> > mechanisms..
> > 
> 
> for CC and pkvm the guest memory is still allocated from host.

From an iommu perspective that doesn't entirely matter, what it sees
is that the translation is controlled by some secure world and it
only needs a way to associate the kvm handle for the secure world with
any required call for configuring the viommu.

It is not very different from KVM installing encrypted pages that have
been completed unmapped from all page tables in the hypervisor into
the VM's secure EPT through TDX calls and then iommufd creating a
viommu that re-uses the secure EPT.

The only thing dealing with the memory map is KVM. I'd expect Xen to
work the same, however the invisible memory was affiliated with the VM
through KVM the iommu side should pick up the KVM and then request a
VIOMMU to be setup for the VFIO device on the target KVM and that
should trigger the hypercalls to move the device into the selected
guest.

Jason