From: Jason Gunthorpe <jgg@ziepe.ca>
To: Daniel Almeida <daniel.almeida@collabora.com>
Cc: "Joerg Roedel (AMD)" <joro@8bytes.org>,
"Will Deacon" <will@kernel.org>,
"Robin Murphy" <robin.murphy@arm.com>,
"Miguel Ojeda" <ojeda@kernel.org>,
"Boqun Feng" <boqun@kernel.org>, "Gary Guo" <gary@garyguo.net>,
"Björn Roy Baron" <bjorn3_gh@protonmail.com>,
"Benno Lossin" <lossin@kernel.org>,
"Andreas Hindborg" <a.hindborg@kernel.org>,
"Alice Ryhl" <aliceryhl@google.com>,
"Trevor Gross" <tmgross@umich.edu>,
"Danilo Krummrich" <dakr@kernel.org>,
"Tamir Duberstein" <tamird@kernel.org>,
"Alexandre Courbot" <acourbot@nvidia.com>,
"Onur Özkan" <work@onurozkan.dev>,
"Abdiel Janulgue" <abdiel.janulgue@gmail.com>,
iommu@lists.linux.dev, rust-for-linux@vger.kernel.org,
linux-kernel@vger.kernel.org, driver-core@lists.linux.dev,
"Detlev Casanova" <detlev.casanova@collabora.com>,
"Nicolas Dufresne" <nicolas.dufresne@collabora.com>
Subject: Re: [PATCH 0/6] rust: iommu: add a paging domain abstraction
Date: Wed, 18 Nov 2026 15:35:21 -0400 [thread overview]
Message-ID: <20261118193521.GD3115493@ziepe.ca> (raw)
In-Reply-To: <615C3DA2-4C02-4A8B-AEC4-70D498672559@collabora.com>
On Wed, Nov 18, 2026 at 04:11:40PM -0300, Daniel Almeida wrote:
>
> > On 17 Nov 2026, at 17:20, Jason Gunthorpe <jgg@ziepe.ca> wrote:
> >
> > On Tue, Nov 17, 2026 at 05:09:11PM -0300, Daniel Almeida wrote:
> >> Hi Jason,
> >>
> >>> On 16 Nov 2026, at 16:38, Jason Gunthorpe <jgg@ziepe.ca> wrote:
> >>>
> >>> On Thu, Nov 12, 2026 at 02:00:07PM -0300, Daniel Almeida wrote:
> >>>> This series adds a Rust abstraction for IOMMU paging domains. A driver
> >>>> can allocate a domain, attach it to its device, map pages into it, and
> >>>> install a handler for its faults.
> >>>>
> >>>> This will be used by the upcoming rkvenc driver that my colleagues
> >>>> Detlev Casanova, Nicolas Dufresne and I are working on. More details on
> >>>> this driver can be found by watching Nicolas Dufresne's talk at LPC 2026
> >>>> [0].
> >>>>
> >>>> As part of the security model, each client will have its own IOMMU
> >>>> domain. This is needed as the submission model will let userspace set
> >>>> most of the encoder's registers directly, so the kernel cannot check
> >>>> every memory access that a job makes the encoder do. With a single
> >>>> domain, one client could make the encoder read or write the memory of
> >>>> another client.
> >>>
> >>> How do you plan to get the paging domain in the first place?
> >>
> >> We plan to alloc a paging domain when a video encoder session is created. Then
> >> we call iommu_attach_device() before a job runs, and detach back to the default
> >> domain after it finishes so the next job can attach its own session's domain.
> >> This is so each session is isolated from each other.
> >
> > If you do that you can't use the DMA API at all in your driver as you
> > have to set 'driver_managed = true'. Is it OK? If it is why not just
> > use VFIO?
>
> No, not OK :/ We want to use GEM shmem, which calls into the DMA API internally
> through drm_gem_shmem_get_pages_sgt() -> dma_map_sgtable().
You cannot continue to use DMA API once you start touching a paging
domain directly. It is why driver_managed exist.
If you want to use the DMA API you need two struct devices and now you
are into DT abuse land.
So what exactly are you planning on doing
And why do you need the paging domain at all if you are using the DMA
API?
> The CPU mappings are write-combined, so the only cache maintenance is the clean
> of newly allocated pages, which dma_map_sgtable() does in
> drm_gem_shmem_get_pages_sgt().
Yes, this is why you should be using the DMA API, that doesn't explain
how any direct use of a paging domain will work.
> The mappings themselves are chosen by the kernel driver. We don't
> care about what the actual IOVA is. All we care about is that a
> given GEM object is mapped to a given domain, and that we have some
> IOVA representing the start of the allocation, such that we can pass
> that to the device.
Wonderful, what is the paging domain for?
> > Are you sure you shouldn't be using iommufd for this, tied into your
> > driver? It was intended for exactly this use case, to allow drivers to
> > delegate their translation to userspace. Though that doesn't relly
> > solve the questions about what it is even being attached to..
>
> I had a cursory look at iommufd, but it seems like:
>
> a) it requires IOMMU_CAP_CACHE_COHERENCY, which is not offered by the rk_iommu
> backing the RK3588 SoC we are currently targeting. In fact, IIUC, this does not
> seem to be possible, since the encoder is not cache coherent with the CPU?
> I.e., does iommufd expect hardware cache coherency/snooping?
*Linux* expects coherency, if you are DMAing direcly to user cachable
memory and you aren't coherent there are issues, and minimally your
driver would have to do the cache flushing.
Which we have deliberately made it so drivers cannot do to force them
to use the DMA API.
This is one of the unresolved problems if a driver tries to use a
paging domain directly.
> b) there is no way to do what I said above about GEM, i.e.: there is no
> connection between iommufd and GEM.
Right, because you started out saying you wanted userspace to control
the mapping and put user memory into it. Kernel controlled GEM
mappings is something entirely different.
> You seem to be saying that what I'm proposing here has a few pitfalls that have
> come up over the years. Perhaps we can massage iommufd to make our use case
> work? I can volunteer to do whatever work is needed, of course.
I am very confused about what you are even doing. Can you explain more
clearly please? What does you DT look like? What do your attach calls
do? How many struct devices do you have? Who is calling the DMA API?
Who is mapping the paging domain? And who handles cache flushing
issues for your incoherent architecture?
Jason
next prev parent reply other threads:[~2026-11-18 19:35 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-12 17:00 Daniel Almeida
2026-11-12 17:00 ` [PATCH 1/6] rust: iommu: move the protection flags to the module root Daniel Almeida
2026-11-12 17:00 ` [PATCH 2/6] rust: scatterlist: make SGTable::as_raw() visible to the crate Daniel Almeida
2026-11-12 17:00 ` [PATCH 3/6] rust: iommu: add the paging domain abstraction Daniel Almeida
2026-11-12 17:00 ` [PATCH 4/6] rust: iommu: add Domain::attach() Daniel Almeida
2026-11-16 21:54 ` Detlev Casanova
2026-11-12 17:00 ` [PATCH 5/6] rust: iommu: add Domain::map() Daniel Almeida
2026-11-12 17:00 ` [PATCH 6/6] rust: iommu: add Domain::with_fault_handler() Daniel Almeida
2026-11-16 19:38 ` [PATCH 0/6] rust: iommu: add a paging domain abstraction Jason Gunthorpe
2026-11-17 20:09 ` Daniel Almeida
2026-11-17 20:20 ` Jason Gunthorpe
2026-11-18 15:25 ` Nicolas Dufresne
2026-11-18 15:55 ` Daniel Almeida
2026-11-18 15:59 ` Daniel Almeida
2026-11-18 16:09 ` Jason Gunthorpe
2026-11-18 19:31 ` Nicolas Dufresne
2026-11-18 19:51 ` Jason Gunthorpe
2026-11-18 19:11 ` Daniel Almeida
2026-11-18 19:35 ` Jason Gunthorpe [this message]
2026-11-18 19:41 ` Nicolas Dufresne
2026-11-16 21:54 ` Detlev Casanova
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261118193521.GD3115493@ziepe.ca \
--to=jgg@ziepe.ca \
--cc=a.hindborg@kernel.org \
--cc=abdiel.janulgue@gmail.com \
--cc=acourbot@nvidia.com \
--cc=aliceryhl@google.com \
--cc=bjorn3_gh@protonmail.com \
--cc=boqun@kernel.org \
--cc=dakr@kernel.org \
--cc=daniel.almeida@collabora.com \
--cc=detlev.casanova@collabora.com \
--cc=driver-core@lists.linux.dev \
--cc=gary@garyguo.net \
--cc=iommu@lists.linux.dev \
--cc=joro@8bytes.org \
--cc=linux-kernel@vger.kernel.org \
--cc=lossin@kernel.org \
--cc=nicolas.dufresne@collabora.com \
--cc=ojeda@kernel.org \
--cc=robin.murphy@arm.com \
--cc=rust-for-linux@vger.kernel.org \
--cc=tamird@kernel.org \
--cc=tmgross@umich.edu \
--cc=will@kernel.org \
--cc=work@onurozkan.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®