Re: [PATCH RFC v6 5/5] iommu/arm-smmu-v3: Wire up shared-SID STE ordering and feature gating

[Nicolin Chen <nicolinc@nvidia.com>]

On Tue, Oct 13, 2026 at 12:14:58PM -0300, Jason Gunthorpe wrote:
> On Tue, Oct 13, 2026 at 10:46:28PM +0800, Peng Fan wrote:
> > Hi Nicolin, Robin, Jason
> > 
> > On Tue, Oct 13, 2026 at 09:44:00AM -0300, Jason Gunthorpe wrote:
> > >On Tue, Oct 13, 2026 at 12:22:06PM +0100, Robin Murphy wrote:
> > >> If we're not going to support PASID for shared streams (which I suggest we
> > >> don't...) than we shouldn't strictly need to worry about CD tables anyway.
> > >> So rather than having to special-case CD table sharing for shared streams,
> > >> perhaps it might make sense to generalise back to a "default" per-domain CD
> > >> for non-PASID use, which can then get switched for the private per-stream
> > >> cd_table if and when a device does enable PASID?
> > >
> > >If we have stared STEs then the group becomes the per-STE data
> > >structure not the master.. We don't have any way right now to do that,
> > >but it would make some sense for the group to store the shared CD
> > 
> > Thanks for the detailed feedback, reply you all here.
> > 
> > For the CD table question: since shared-SID masters are in the same
> > group and always use the same domain, the CD content is identical.
> > With skip_shared_ste(), only stream->master's STE (and thus its
> > cd_table) is active in hardware.  The other masters' cd_tables are
> > allocated but never referenced by the STE.
> > 
> > I see two options:
> > 
> > A) Keep skip_shared_ste() and add arm_smmu_cd_owner() to redirect
> >    all cd_table operations to stream->master. This is a minimal
> >    change that works for our use case (platform devices, no PASID,
> >    no domain switching).
> > 
> > B) Drop skip_shared_ste() entirely and share the CD table at the
> >    group or domain level. This is cleaner but a larger change.
> > 
> > I'm leaning towards (A) for now to keep the series focused on the
> > simple shared-SID case.  Would that be acceptable, or do you prefer
> > we go with (B) from the start?
> 
> I think the skip is in the wrong place perhaps?
> 
> The group is always guarenteed to have all the masters in it.
> 
> So we could split them into primary masters and secondary ones. The
> secondary ones are all the devices thare share SIDs with the primary.

Yea, stream->master can be used as the marker for the primary.

> The group list is guarenteed to have all masters in it, so we can
> simply fully skip attach from the secondary. It is guarenteed by the
> group that a primary will have already attached to the STE with
> exactly the same domain so the secondary just needs to do NOP.

I was worried about the book keepings in the attach. But most of
them seems unnecessary to repeat on the secondaries. How about the
arm_smmu_master_canwbs() guard?

> If you do that then all these questions go away since there is only
> ever one thing writing the STE, it is always stable and no issue.
> 
> This does leave a problem for master remove to switch the STE to one
> of the secondaries, but that sounds like it could be managed through
> the release domain mechanism, just nominate a secondary to be primary
> and execute an attach on it in the normal way. The hitless STE
> mechanism will make it work safely.

I think the release domain can be entirely skipped on any shared
master (including primary).

ops->release_device() is called right after the release domain's
attach_dev. I think it would be cleaner to put the nomination of
the next primary into the release_device() callback -- it shares
the same cleanup helper with arm_smmu_insert_master()'s rollback
path.

Thanks
Nicolin