Re: [PATCH RFC 00/17] Allow guest_memfd to be created using a resource (pool) fd

[David Woodhouse <dwmw2@infradead.org>]

[-- Attachment #1: Type: text/plain, Size: 7854 bytes --]

On Mon, 2026-09-28 at 15:59 -0700, Ackerley Tng wrote:
> David Woodhouse <dwmw2@infradead.org> writes:
> 
> > On Fri, 2026-09-25 at 17:50 -0700, Ackerley Tng via B4 Relay wrote:
> > > 
> > > === What if my provider doesn't deal with pages?
> > > 
> > > Frank and David Woodhouse [2] have use cases for providers that use
> > > PFNs, and this is definitely something a generic interface must
> > > support.
> > > 
> > > Here are some options I can think of:
> > > 
> > > 1. Don't define .alloc_folio(), instead define .alloc_pfn()
> > > 2. Refactor .alloc_folio() to .alloc_pfn()
> > > 
> > > Either way, I think guest_memfd should still be the primary manager of
> > > the memory.
> > 
> > Thanks for working on this.
> > 
> 
> Thanks for the quick reply!
> 
> > I'm not sure what you mean by 'primary manager' here but my use case is
> 
> Naming is hard! See comment on revocation below.
> 
> > for the provider to be the ultimate arbiter of who owns which PFN, and
> > revocation of the same. Think of it like a filesystem backed by
> > external memory. Each guest's memory is a file, and a guest can
> > *donate* specific pages of its memory to other guests (which in the
> 
> Putting donation aside first - I don't have a good picture of how a
> guest would tell the host that it's willing to share a page - can't
> comment. Would love to find out more.
>
> > context of the *interface* only means that we need revocation).

That's implementation-specific. Think of things like Nitro Enclaves
where a guest 'donates' its memory back to the hypervisor to be used by
another microvm. But that interface isn't the guest_memfd concern; as I
said, in the context of the guest_memfd interface, there is only
revocation: that page went away, whatever the reason.

> Going with this revocation part, beginning with a more basic use case:
> I'm thinking that the provider can notify guest_memfd that the page or
> PFN is going away.

Right. That part I have working in what I was posting.
https://git.infradead.org/?p=users/dwmw2/linux.git;a=commitdiff;h=ee95ed5aaf06

> guest_memfd cannot say no, but it does have to tell KVM to zap the page
> from stage 2, and for CoCo it may need to do more stuff.
>
> The notification needs some pointer, so I was thinking that on .attach()
> the provider would also note down which guest_memfd instance the
> provider should notify.
>
> The notifier could tell guest_memfd that some (offset, size) is going
> away, and breaking down mappings in the EPT/NPT should be something
> guest_memfd tells KVM to do, I think. Are you thinking that the provider
> would directly break the mappings down in the EPT/NPT?

I think telling KVM is the right thing to do.

> Does this notification mechanism work for you? IIUC regular DAX would
> need it too, like if the device is unplugged for example.

Sure. As long as it works and efficiently covers everyone's use cases,
I'm not going to be overly opinionated a priori about *how* it works.

All such bets are off once we see the code, of course :)

> > Should I be trying to rework parts of my series to live on top of this?
> 
> I hope to gather more feedback at LPC. It'd be nice to get early
> feedback if you think it can work for you! The overall design still
> needs feedback though, don't take this as the final direction.
> 
> > Things I have working but which I don't see here include:
> > 
> 
> This sounds like 4 or more different features! I think they do work with
> this resource fd proposal.
> 
> >  • PFN support (which you mentioned).
> 
> Yup, guest_memfd needs some way to track PFNs in addition to folios. I
> think for tracking we can reference DAX, the part I haven't looked at is
> how to let guest_memfd handle events. Like if there's memory failure on
> a PFN, how do we let guest_memfd handle it?

Telling the guest about correctable and uncorrectable errors, you mean?
I would have thought that's up to the VMM, until the point where (see
revocation)?

> guest_memfd needs to at least be notified, so that it can handle the KVM
> stuff: zapping and special stuff for CoCo. Whether the provider or
> guest_memfd gets to handle it first can be worked out :)
> 
> >  • IOMMUFD support via dma-buf.
> 
> Is this about having guest_memfd work with dmabuf fds?

It's about having the provider of memory (again, think of it as a daxfs
if you will) able to tell *both* KVM and IOMMU which pages are where —
and revoke them at will.

The *implementation* involved dmabuf fds, and we already discussed
which wraps which, on the first posting of my series. But maybe we'll
conclude that the right answer is for IOMMUFD to recognise this
guest_memfd 'provider' as a first-class citizen, and we won't need to
masquerade as dmabuf?

> Fuad also brought up that Android uses dmabuf as an interface for CMA
> memory. [3] IIUC the dmabuf fd is mmap()ed, then the userspace address
> is set up in a memslot for the guest. To make this CMA memory
> CoCo-friendly, we want to use it with guest_memfd.
> 
> Jason response was that guest_memfd should probably just directly get
> memory from CMA [3].
> 
> I'm not super familiar with CMA or dmabufs, would either have a suitable
> "resource fd" to pass to guest_memfd like how HugeTLBfs has a mount fd?
> The "resource fd" should ideally have no way to mmap or read/write the
> memory directly.
> 
> [3] https://lore.kernel.org/all/CA+EHjTxZ0N3Tfnid404B4tkb_E+Z8mODTHTgBPiF6=bwZp7Hjw@mail.gmail.com/

I think where the memory comes *from* is an implementation detail for
the provider. It provides a PFN (or folio, if you must). All else is
not the business of KVM or IOMMUFD.

> Or is this about letting IOMMUFD get pages to map in IOMMU page tables
> via an fd+offset instead of via mmap()-ed userspace addresses as in
> IOMMU_IOAS_MAP_FILE, and that guest_memfd isn't a supported kind of fd
> (yet)?
> 
> I think guest_memfd should support IOMMU_IOAS_MAP_FILE. One use case is
> Confidential IO [4], where SNP would need to have private memory
> (guest_memfd) mapped in the IO page tables.

Right, I think that's what I meant with 'first class citizen' above?

> The missing parts here are that guest_memfd and iommufd need to
> cooperate for conversions (or some other coordination in userspace). If
> private memory gets converted to shared, iommufd needs to know to remap
> the pages as shared in the IO page tables.
> 
> [4] https://lore.kernel.org/all/20260225075211.3353194-1-aik@amd.com/
> 
> >  • Revocation (which I've tested correctly breaks down large pages in
> >    both EPT/NPT and IOMMU).
> 
> See above.
> 
> >  • AsyncPF support for pages requested by KVM.
> 
> Is this kind of orthogonal? IIUC today KVM MMU handles the async-ness.
> KVM MMU checks that the page is not there, then since async was
> configured, KVM tells the guest to schedule in something else.
> 
> guest_memfd doesn't have support for async #PFs now. I imagine it would
> be something like KVM MMU firing off a kvm_gmem_get_pfn() request
> asynchronously, and then guest_memfd gets the page or PFN and inserts it
> in guest_memfd filemap, and then notifies KVM MMU?
> 
> Within kvm_gmem_get_pfn(), guest_memfd should get the page or PFN from
> the provider.
> 
> I think it's orthogonal because guest_memfd could
> have gotten a PAGE_SIZE page (existing functionality) or gotten a page
> from the provider.

Kind of, but I'm focusing on the guest_memfd provider interface, and
that needs to support the asychronous mode: asked for a PFN for a given
guest address, it returns -EAGAIN and then provides it later, and the
guest gets the right asyncpf behaviour:
https://git.infradead.org/?p=users/dwmw2/linux.git;a=commitdiff;h=894dd0fcb34f

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 6179 bytes --]