[PATCH RFC v4 0/4] iommu/arm-smmu-v3: Support shared Stream IDs

[PATCH RFC v4 0/4] iommu/arm-smmu-v3: Support shared Stream IDs

[Peng Fan (OSS)]

Some SoCs have a limited number of IOMMU Stream IDs and hardware
that inherently shares them.  The NXP i.MX95, for example, has
only 64 SIDs -- the internal bus carries just 6 bits of requester
ID -- serving 64+ DMA initiators spread across separate IP blocks:
MMC, SD, NET, PCI, DMA, NPU, DSP, DISPLAY and more.  Because the
SID space is exhausted, genuinely distinct platform devices (with
their own DT nodes and drivers) share a SID by hardware design.

The current ARM SMMUv3 driver rejects this situation with:
  "Aliasing StreamID unsupported, expect DMA to be broken"

This series adds support for multiple single-SID masters to share
a Stream ID, allowing such constrained SoCs to function with
IOMMU translation enabled.

Patch 1 fixes a pre-existing bug where rb_erase() is called on
duplicate SID entries that were never inserted into the RB tree,
which can corrupt the tree.  Bridged PCI devices may produce
duplicated Stream IDs; the insertion path already skips them but
the removal and error rollback paths did not.

Patch 2 adds shared-SID tracking so that when a second single-SID
master registers an already-owned SID, it joins the existing
stream rather than being rejected.  The stream structure is shared
directly between masters via a linked list. On removal, ownership
transfers to the next sharer.

Patch 3 places devices that share a SID into the same IOMMU
group, ensuring they share a single IOMMU domain (required
because one STE can only point to one cd_table).

Patch 4 wires up STE write ordering (first master writes, last
master tears down) and gates features that require unambiguous
SID-to-device mapping: SVA, IOPF/stall, and vSMMU nesting are
disabled for shared-SID masters.

Changes since RFC v3:
- Added a standalone bugfix for duplicate-SID rb_erase (new patch 1).
- Stream structures are now shared directly between masters
  instead of duplicating them (Nicolin).  No ref_count field;
  sharing is detected via list_empty(&stream->shared_masters).
- On owner removal, use rb_replace_node() + list_splice_init()
  to transfer the RB tree entry and shared_masters list to the
  next sharer, avoiding use-after-free.
- INIT_LIST_HEAD(&stream->shared_masters) moved after sort() to
  avoid list pointer corruption from bitwise memory swaps.
- Both sides of the sharing pair are checked for num_streams == 1.

Changes since RFC v2:
- Dropped the RB tree to XArray conversion patch (Nicolin).
  The RB tree is retained as-is; a new arm_smmu_find_stream()
  helper provides stream-level lookup.
- No per-master shared_sid bool; replaced by
  list_empty(&master->shared_masters_elm) (Nicolin).
- Only SID-sharing for single SID device
- Follow Nicolin's suggestion https://lore.kernel.org/linux-iommu/arG6hmng3NddGEHm@nvidia.com/
- Link to v2: https://lore.kernel.org/linux-iommu/20260921-smmu-shared-sid-v2-0-4b656ce68178@nxp.com/

Changes since RFC v1:
- Revised cover letter to drop eDMA channel example (Jason).

Signed-off-by: Peng Fan <peng.fan@nxp.com>
---
Peng Fan (4):
      iommu/arm-smmu-v3: Skip duplicate SIDs when erasing streams from RB tree
      iommu/arm-smmu-v3: Support shared SIDs in insert/remove_master
      iommu/arm-smmu-v3: Group aliasing devices into the same IOMMU group
      iommu/arm-smmu-v3: Wire up shared-SID STE ordering and feature gating

 .../iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c    |   2 +-
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c    |   3 +
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c        | 174 +++++++++++++++++----
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h        |   3 +
 4 files changed, 150 insertions(+), 32 deletions(-)
---
base-commit: 6474fa070f2b8013b4b87350b775b8c3be6e8aac
change-id: 20260930-smmu-shared-sid-ver3-96807c4f4d93

Best regards,
--  
Peng Fan <peng.fan@nxp.com>

[PATCH RFC v4 1/4] iommu/arm-smmu-v3: Skip duplicate SIDs when erasing streams from RB tree

[Peng Fan (OSS)]

From: Peng Fan <peng.fan@nxp.com>

Bridged PCI devices may end up with duplicated Stream IDs in
master->streams[].  During insertion, rb_find_add() silently
drops the duplicate node when it collides with an already-inserted
entry from the same master, leaving the duplicate's rb_node in an
uninitialized state.

However, both arm_smmu_remove_master() and the error rollback path
in arm_smmu_insert_master() unconditionally call rb_erase() on
every element of the streams array, including duplicates that were
never inserted.  Calling rb_erase() on an uninitialized rb_node
corrupts the RB tree.

The streams array is sorted by SID before insertion, so duplicates
are always adjacent.  Skip them during erasure to match the
insertion behavior.

Fixes: b00d24997a11c ("iommu/arm-smmu-v3: Fix iommu_device_probe bug due to duplicated stream ids")
Assisted-by: LLM
Signed-off-by: Peng Fan <peng.fan@nxp.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 5732f3ba0122d..a79f2250a4886 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -4153,8 +4153,11 @@ static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
 	}
 
 	if (ret) {
-		for (i--; i >= 0; i--)
+		for (i--; i >= 0; i--) {
+			if (i > 0 && master->streams[i - 1].id == master->streams[i].id)
+				continue;
 			rb_erase(&master->streams[i].node, &smmu->streams);
+		}
 		kfree(master->streams);
 		kfree(master->build_invs);
 	}
@@ -4173,8 +4176,11 @@ static void arm_smmu_remove_master(struct arm_smmu_master *master)
 		return;
 
 	mutex_lock(&smmu->streams_mutex);
-	for (i = 0; i < fwspec->num_ids; i++)
+	for (i = 0; i < fwspec->num_ids; i++) {
+		if (i > 0 && master->streams[i - 1].id == master->streams[i].id)
+			continue;
 		rb_erase(&master->streams[i].node, &smmu->streams);
+	}
 	mutex_unlock(&smmu->streams_mutex);
 
 	kfree(master->streams);

-- 
2.34.1

Re: [PATCH RFC v4 1/4] iommu/arm-smmu-v3: Skip duplicate SIDs when erasing streams from RB tree

[Nicolin Chen]

On Thu, Oct 01, 2026 at 08:49:58PM +0800, Peng Fan (OSS) wrote:
> From: Peng Fan <peng.fan@nxp.com>
> 
> Bridged PCI devices may end up with duplicated Stream IDs in
> master->streams[].  During insertion, rb_find_add() silently
> drops the duplicate node when it collides with an already-inserted
> entry from the same master, leaving the duplicate's rb_node in an
> uninitialized state.

There is already a cleaner fix:
https://lore.kernel.org/linux-iommu/76c5f9dde30269995ef842a12a3a5e1ebaa3e6df.1790188510.git.nicolinc@nvidia.com/

Hopefully Will can take it soon.

Nicolin

Re: [PATCH RFC v4 1/4] iommu/arm-smmu-v3: Skip duplicate SIDs when erasing streams from RB tree

[Peng Fan]

On Thu, Oct 01, 2026 at 09:07:18AM -0700, Nicolin Chen wrote:
>On Thu, Oct 01, 2026 at 08:49:58PM +0800, Peng Fan (OSS) wrote:
>> From: Peng Fan <peng.fan@nxp.com>
>> 
>> Bridged PCI devices may end up with duplicated Stream IDs in
>> master->streams[].  During insertion, rb_find_add() silently
>> drops the duplicate node when it collides with an already-inserted
>> entry from the same master, leaving the duplicate's rb_node in an
>> uninitialized state.
>
>There is already a cleaner fix:
>https://lore.kernel.org/linux-iommu/76c5f9dde30269995ef842a12a3a5e1ebaa3e6df.1790188510.git.nicolinc@nvidia.com/

ok. I will drop patch 1 in v5.

Thanks
Peng

>
>Hopefully Will can take it soon.
>
>Nicolin
>

[PATCH RFC v4 2/4] iommu/arm-smmu-v3: Support shared SIDs in insert/remove_master

[Peng Fan (OSS)]

From: Peng Fan <peng.fan@nxp.com>

Some SoCs have a limited number of IOMMU Stream IDs and hardware
that inherently shares them.  For example, the NXP i.MX95 has only
64 SIDs (6-bit internal bus) serving 64+ DMA initiators across
separate IP blocks - MMC, SD, NET, PCI, DMA, NPU, DSP, DISPLAY
and more.  Genuinely distinct platform devices share a SID by
hardware design because the SID space is exhausted.

When arm_smmu_insert_master() encounters a SID already owned by a
different single-SID master, instead of failing with -ENODEV, link
the new master into the existing stream's shared_masters list.
The stream structure is shared directly: the RB tree entry remains
the same, and all sharing masters are tracked via list_head.

On removal, if the departing master owns the canonical stream,
transfer ownership to the next sharer.

Sharing is restricted to single-SID masters (num_streams == 1)
since multi-SID sharing would require more complex tracking.

Suggested-by: Nicolin Chen <nicolinc@nvidia.com>
Link: https://lore.kernel.org/linux-iommu/arG6hmng3NddGEHm@nvidia.com/
Assisted-by: LLM
Signed-off-by: Peng Fan <peng.fan@nxp.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 94 +++++++++++++++++++++++------
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h |  3 +
 2 files changed, 78 insertions(+), 19 deletions(-)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index a79f2250a4886..9d98d1b2758f7 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2037,15 +2037,14 @@ static int arm_smmu_streams_cmp_key(const void *lhs, const struct rb_node *rhs)
 	return 0;
 }
 
-static int arm_smmu_streams_cmp_node(struct rb_node *lhs,
-				     const struct rb_node *rhs)
+static int arm_smmu_streams_cmp_node(struct rb_node *lhs, const struct rb_node *rhs)
 {
 	return arm_smmu_streams_cmp_key(
 		&rb_entry(lhs, struct arm_smmu_stream, node)->id, rhs);
 }
 
-static struct arm_smmu_master *
-arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid)
+static struct arm_smmu_stream *
+arm_smmu_find_stream(struct arm_smmu_device *smmu, u32 sid)
 {
 	struct rb_node *node;
 
@@ -2054,7 +2053,20 @@ arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid)
 	node = rb_find(&sid, &smmu->streams, arm_smmu_streams_cmp_key);
 	if (!node)
 		return NULL;
-	return rb_entry(node, struct arm_smmu_stream, node)->master;
+	return rb_entry(node, struct arm_smmu_stream, node);
+}
+
+static struct arm_smmu_master *
+arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid)
+{
+	struct arm_smmu_stream *stream;
+
+	stream = arm_smmu_find_stream(smmu, sid);
+	if (!stream)
+		return NULL;
+	if (!list_empty(&stream->shared_masters))
+		return NULL;
+	return stream->master;
 }
 
 /* IRQ and event handlers */
@@ -4116,6 +4128,7 @@ static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
 		new_stream->id = fwspec->ids[i];
 		new_stream->master = master;
 	}
+	INIT_LIST_HEAD(&master->shared_masters_elm);
 
 	/* Put the ids into order for sorted to_merge/to_unref arrays */
 	sort(master->streams, master->num_streams,
@@ -4128,6 +4141,8 @@ static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
 		struct rb_node *existing;
 		u32 sid = new_stream->id;
 
+		INIT_LIST_HEAD(&new_stream->shared_masters);
+
 		ret = arm_smmu_init_sid_strtab(smmu, sid);
 		if (ret)
 			break;
@@ -4136,23 +4151,33 @@ static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
 		existing = rb_find_add(&new_stream->node, &smmu->streams,
 				       arm_smmu_streams_cmp_node);
 		if (existing) {
-			struct arm_smmu_master *existing_master =
-				rb_entry(existing, struct arm_smmu_stream, node)
-					->master;
+			struct arm_smmu_stream *existing_stream =
+				rb_entry(existing, struct arm_smmu_stream, node);
 
 			/* Bridged PCI devices may end up with duplicated IDs */
-			if (existing_master == master)
+			if (existing_stream->master == master)
 				continue;
 
-			dev_warn(master->dev,
-				 "Aliasing StreamID 0x%x (from %s) unsupported, expect DMA to be broken\n",
-				 sid, dev_name(existing_master->dev));
-			ret = -ENODEV;
-			break;
+			if (master->num_streams != 1 ||
+			    existing_stream->master->num_streams != 1) {
+				dev_warn(master->dev,
+					 "Shared StreamID 0x%x not supported for multi-SID masters\n",
+					 sid);
+				ret = -ENODEV;
+				break;
+			}
+
+			if (list_empty(&existing_stream->shared_masters))
+				list_add_tail(&existing_stream->master->shared_masters_elm,
+					      &existing_stream->shared_masters);
+			list_add_tail(&master->shared_masters_elm,
+				      &existing_stream->shared_masters);
+			continue;
 		}
 	}
 
 	if (ret) {
+		list_del_init(&master->shared_masters_elm);
 		for (i--; i >= 0; i--) {
 			if (i > 0 && master->streams[i - 1].id == master->streams[i].id)
 				continue;
@@ -4170,16 +4195,47 @@ static void arm_smmu_remove_master(struct arm_smmu_master *master)
 {
 	int i;
 	struct arm_smmu_device *smmu = master->smmu;
-	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(master->dev);
 
 	if (!smmu || !master->streams)
 		return;
 
 	mutex_lock(&smmu->streams_mutex);
-	for (i = 0; i < fwspec->num_ids; i++) {
-		if (i > 0 && master->streams[i - 1].id == master->streams[i].id)
-			continue;
-		rb_erase(&master->streams[i].node, &smmu->streams);
+	if (!list_empty(&master->shared_masters_elm)) {
+		struct arm_smmu_stream *stream;
+
+		list_del_init(&master->shared_masters_elm);
+		stream = arm_smmu_find_stream(smmu, master->streams[0].id);
+		if (stream && stream->master == master) {
+			struct arm_smmu_master *next;
+			struct arm_smmu_stream *next_stream;
+
+			next = list_first_entry(&stream->shared_masters,
+					struct arm_smmu_master,
+					shared_masters_elm);
+			next_stream = &next->streams[0];
+			next_stream->ste_installed = stream->ste_installed;
+			INIT_LIST_HEAD(&next_stream->shared_masters);
+			list_splice_init(&stream->shared_masters,
+					 &next_stream->shared_masters);
+			rb_replace_node(&stream->node, &next_stream->node,
+					&smmu->streams);
+			stream = next_stream;
+		}
+
+		if (stream && list_is_singular(&stream->shared_masters)) {
+			struct arm_smmu_master *last;
+
+			last = list_first_entry(&stream->shared_masters,
+						struct arm_smmu_master,
+						shared_masters_elm);
+			list_del_init(&last->shared_masters_elm);
+		}
+	} else {
+		for (i = 0; i < master->num_streams; i++) {
+			if (i > 0 && master->streams[i - 1].id == master->streams[i].id)
+				continue;
+			rb_erase(&master->streams[i].node, &smmu->streams);
+		}
 	}
 	mutex_unlock(&smmu->streams_mutex);
 
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index dd2fee2f560e6..89177f3e7dabe 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -969,6 +969,8 @@ struct arm_smmu_stream {
 	u32				id;
 	struct arm_smmu_master		*master;
 	struct rb_node			node;
+	bool				ste_installed;
+	struct list_head		shared_masters;
 };
 
 struct arm_smmu_vmaster {
@@ -1017,6 +1019,7 @@ struct arm_smmu_master {
 	bool				ste_ats_enabled : 1;
 	bool				stall_enabled;
 	bool				ats_always_on;
+	struct list_head		shared_masters_elm;
 	unsigned int			ssid_bits;
 	unsigned int			iopf_refcount;
 };

-- 
2.34.1

Re: [PATCH RFC v4 2/4] iommu/arm-smmu-v3: Support shared SIDs in insert/remove_master

[Nicolin Chen]

On Thu, Oct 01, 2026 at 08:49:59PM +0800, Peng Fan (OSS) wrote:
> -static int arm_smmu_streams_cmp_node(struct rb_node *lhs,
> -				     const struct rb_node *rhs)
> +static int arm_smmu_streams_cmp_node(struct rb_node *lhs, const struct rb_node *rhs)

Why is this changed?

> @@ -4136,23 +4151,33 @@ static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
>  		existing = rb_find_add(&new_stream->node, &smmu->streams,
>  				       arm_smmu_streams_cmp_node);
>  		if (existing) {
> -			struct arm_smmu_master *existing_master =
> -				rb_entry(existing, struct arm_smmu_stream, node)
> -					->master;
> +			struct arm_smmu_stream *existing_stream =
> +				rb_entry(existing, struct arm_smmu_stream, node);
>  
>  			/* Bridged PCI devices may end up with duplicated IDs */
> -			if (existing_master == master)
> +			if (existing_stream->master == master)
>  				continue;
>  
> -			dev_warn(master->dev,
> -				 "Aliasing StreamID 0x%x (from %s) unsupported, expect DMA to be broken\n",
> -				 sid, dev_name(existing_master->dev));
> -			ret = -ENODEV;
> -			break;
> +			if (master->num_streams != 1 ||
> +			    existing_stream->master->num_streams != 1) {
> +				dev_warn(master->dev,
> +					 "Shared StreamID 0x%x not supported for multi-SID masters\n",
> +					 sid);
> +				ret = -ENODEV;
> +				break;
> +			}
> +
> +			if (list_empty(&existing_stream->shared_masters))
> +				list_add_tail(&existing_stream->master->shared_masters_elm,
> +					      &existing_stream->shared_masters);
> +			list_add_tail(&master->shared_masters_elm,
> +				      &existing_stream->shared_masters);

Masters still hold duplicated streams for the shared SID. And their
ste_installed flags are out of sync..

    master_a->stream[0]->a's stream0 {ste_installed=true}
    master_b->stream[0]->b's stream0 {ste_installed=false}

What I have been suggesting is to have a shared stream:
    master_a->stream[0]-->shared stream0 {ste_installed=true}
    master_b->stream[0]-|

So, the driver would need a rework first changing master:
-       struct arm_smmu_stream          *streams;
+       struct arm_smmu_stream          **streams;

Also, arm_smmu_insert_master() needs to reverse its allocation:
	for (i = 0; i < fwspec->num_ids; i++) {
		stream = arm_smmu_find_stream(smmu, sid);
		if (stream) {
			// link to the existing stream
		} else {
			// allocate new stream
		}
		master->streams[i] = stream;
	}
	if (!ret)
		sort();

Nicolin

Re: [PATCH RFC v4 2/4] iommu/arm-smmu-v3: Support shared SIDs in insert/remove_master

[Peng Fan]

On Thu, Oct 01, 2026 at 10:17:22AM -0700, Nicolin Chen wrote:
>On Thu, Oct 01, 2026 at 08:49:59PM +0800, Peng Fan (OSS) wrote:
>> -static int arm_smmu_streams_cmp_node(struct rb_node *lhs,
>> -				     const struct rb_node *rhs)
>> +static int arm_smmu_streams_cmp_node(struct rb_node *lhs, const struct rb_node *rhs)
>
>Why is this changed?

Just thought 100 chars is fine here, so ...
I will drop this change in v5.

>
>> @@ -4136,23 +4151,33 @@ static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
>>  		existing = rb_find_add(&new_stream->node, &smmu->streams,
>>  				       arm_smmu_streams_cmp_node);
>>  		if (existing) {
>> -			struct arm_smmu_master *existing_master =
>> -				rb_entry(existing, struct arm_smmu_stream, node)
>> -					->master;
>> +			struct arm_smmu_stream *existing_stream =
>> +				rb_entry(existing, struct arm_smmu_stream, node);
>>  
>>  			/* Bridged PCI devices may end up with duplicated IDs */
>> -			if (existing_master == master)
>> +			if (existing_stream->master == master)
>>  				continue;
>>  
>> -			dev_warn(master->dev,
>> -				 "Aliasing StreamID 0x%x (from %s) unsupported, expect DMA to be broken\n",
>> -				 sid, dev_name(existing_master->dev));
>> -			ret = -ENODEV;
>> -			break;
>> +			if (master->num_streams != 1 ||
>> +			    existing_stream->master->num_streams != 1) {
>> +				dev_warn(master->dev,
>> +					 "Shared StreamID 0x%x not supported for multi-SID masters\n",
>> +					 sid);
>> +				ret = -ENODEV;
>> +				break;
>> +			}
>> +
>> +			if (list_empty(&existing_stream->shared_masters))
>> +				list_add_tail(&existing_stream->master->shared_masters_elm,
>> +					      &existing_stream->shared_masters);
>> +			list_add_tail(&master->shared_masters_elm,
>> +				      &existing_stream->shared_masters);
>
>Masters still hold duplicated streams for the shared SID. And their
>ste_installed flags are out of sync..
>
>    master_a->stream[0]->a's stream0 {ste_installed=true}
>    master_b->stream[0]->b's stream0 {ste_installed=false}
>
>What I have been suggesting is to have a shared stream:
>    master_a->stream[0]-->shared stream0 {ste_installed=true}
>    master_b->stream[0]-|
>
>So, the driver would need a rework first changing master:
>-       struct arm_smmu_stream          *streams;
>+       struct arm_smmu_stream          **streams;

Ah. So I need a patch list saying:
1. iommu/arm-smmu-v3: Allocate streams individually. Change master->streams
   from an embedded array of struct arm_smmu_stream to an array of pointers,
   with each stream individually allocated, to prepare for shared-SID support
   where multiple masters will point to the same stream object
2. iommu/arm-smmu-v3: Support shared SIDs in insert/remove_master
3. iommu/arm-smmu-v3: Group aliasing devices into the same IOMMU group
4. iommu/arm-smmu-v3: Wire up shared-SID STE ordering and feature gating

>
>Also, arm_smmu_insert_master() needs to reverse its allocation:
>	for (i = 0; i < fwspec->num_ids; i++) {
>		stream = arm_smmu_find_stream(smmu, sid);
>		if (stream) {
>			// link to the existing stream
>		} else {
>			// allocate new stream
>		}
>		master->streams[i] = stream;
>	}
>	if (!ret)
>		sort();

Thanks for the details.

Thanks,
Peng

>
>Nicolin
>
>

Re: [PATCH RFC v4 2/4] iommu/arm-smmu-v3: Support shared SIDs in insert/remove_master

[Nicolin Chen]

On Fri, Oct 02, 2026 at 06:27:58PM +0800, Peng Fan wrote:
> Ah. So I need a patch list saying:
> 1. iommu/arm-smmu-v3: Allocate streams individually. Change master->streams
>    from an embedded array of struct arm_smmu_stream to an array of pointers,
>    with each stream individually allocated, to prepare for shared-SID support
>    where multiple masters will point to the same stream object
> 2. iommu/arm-smmu-v3: Support shared SIDs in insert/remove_master
> 3. iommu/arm-smmu-v3: Group aliasing devices into the same IOMMU group
> 4. iommu/arm-smmu-v3: Wire up shared-SID STE ordering and feature gating

That looks right to me.

Please also check Sashiko findings:
https://sashiko.dev/#/patchset/20261001-smmu-shared-sid-ver3-v4-0-06b34fb2e691%40nxp.com

Some of them should be gone once you fix the out-of-sync issue.
But findings like lockings would still remain.

Nicolin

[PATCH RFC v4 3/4] iommu/arm-smmu-v3: Group aliasing devices into the same IOMMU group

[Peng Fan (OSS)]

From: Peng Fan <peng.fan@nxp.com>

Devices sharing a SID must share a single IOMMU domain (and
therefore a single cd_table) because the STE can only point to one
cd_table at a time.

Detect SID aliasing at group-assignment time by looking up the RB
tree for existing owners.  arm_smmu_device_group() is called before
arm_smmu_insert_master(), so any RB tree hit belongs to a
different, already-probed master.

Assisted-by: LLM
Signed-off-by: Peng Fan <peng.fan@nxp.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 33 ++++++++++++++++++++---------
 1 file changed, 23 insertions(+), 10 deletions(-)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 9d98d1b2758f7..0f7e90af5823d 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -4379,19 +4379,32 @@ static int arm_smmu_set_dirty_tracking(struct iommu_domain *domain,
 
 static struct iommu_group *arm_smmu_device_group(struct device *dev)
 {
-	struct iommu_group *group;
+	struct arm_smmu_master *master = dev_iommu_priv_get(dev);
+	struct arm_smmu_device *smmu = master->smmu;
+	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
+	struct iommu_group *group = NULL;
+	int i;
+
+	mutex_lock(&smmu->streams_mutex);
+	for (i = 0; i < fwspec->num_ids; i++) {
+		struct arm_smmu_stream *stream;
+
+		stream = arm_smmu_find_stream(smmu, fwspec->ids[i]);
+		if (!stream)
+			continue;
+
+		group = iommu_group_get(stream->master->dev);
+		break;
+	}
+	mutex_unlock(&smmu->streams_mutex);
+
+	if (group)
+		return group;
 
-	/*
-	 * We don't support devices sharing stream IDs other than PCI RID
-	 * aliases, since the necessary ID-to-device lookup becomes rather
-	 * impractical given a potential sparse 32-bit stream ID space.
-	 */
 	if (dev_is_pci(dev))
-		group = pci_device_group(dev);
-	else
-		group = generic_device_group(dev);
+		return pci_device_group(dev);
 
-	return group;
+	return generic_device_group(dev);
 }
 
 static int arm_smmu_of_xlate(struct device *dev,

-- 
2.34.1

[PATCH RFC v4 4/4] iommu/arm-smmu-v3: Wire up shared-SID STE ordering and feature gating

[Peng Fan (OSS)]

From: Peng Fan <peng.fan@nxp.com>

For shared SIDs, only the first master to attach writes the STE
(tracked by ste_installed under streams_mutex); subsequent masters
skip the write.  On teardown, skip the ABORT STE write while other
masters still share the SID.

Fault events on shared SIDs cannot be attributed to a specific
master, so arm_smmu_find_master() returns NULL when the stream's
shared_masters list is non-empty.

Disable SVA, IOPF/stall, and vSMMU nesting for shared-SID masters
since all three require unambiguous SID-to-device mapping.

Assisted-by: LLM
Signed-off-by: Peng Fan <peng.fan@nxp.com>
---
 .../iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c    |  2 +-
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c    |  3 ++
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c        | 43 ++++++++++++++++++++--
 3 files changed, 44 insertions(+), 4 deletions(-)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c
index ab1078a97d801..73fb5fad1c181 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c
@@ -309,7 +309,7 @@ static int arm_vsmmu_vdevice_init(struct iommufd_vdevice *vdev)
 	 * arm_vsmmu_vsid_to_sid() maps a vSID to master->streams[0] alone, so
 	 * more streams would leave the rest stale and none reads out of bounds.
 	 */
-	if (master->num_streams != 1)
+	if (master->num_streams != 1 || !list_empty(&master->shared_masters_elm))
 		return -EOPNOTSUPP;
 	return 0;
 }
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
index 0a429c64fbf3e..54a0a65669ac2 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
@@ -271,6 +271,9 @@ static int arm_smmu_sva_set_dev_pasid(struct iommu_domain *domain,
 	if (!(master->smmu->features & ARM_SMMU_FEAT_SVA))
 		return -EOPNOTSUPP;
 
+	if (!list_empty(&master->shared_masters_elm))
+		return -EOPNOTSUPP;
+
 	/* Prevent arm_smmu_mm_release from being called while we are attaching */
 	if (!mmget_not_zero(domain->mm))
 		return -EINVAL;
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 0f7e90af5823d..ae33660ca79e7 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2976,11 +2976,36 @@ arm_smmu_get_step_for_sid(struct arm_smmu_device *smmu, u32 sid)
 	}
 }
 
+static bool arm_smmu_is_shared_sid(struct arm_smmu_master *master)
+{
+	return !list_empty(&master->shared_masters_elm);
+}
+
+static bool arm_smmu_skip_shared_ste(struct arm_smmu_device *smmu,
+				     u32 sid, bool is_abort)
+{
+	struct arm_smmu_stream *stream;
+
+	lockdep_assert_held(&smmu->streams_mutex);
+
+	stream = arm_smmu_find_stream(smmu, sid);
+	if (!stream || list_empty(&stream->shared_masters))
+		return false;
+
+	if (is_abort)
+		return true;
+	if (stream->ste_installed)
+		return true;
+	stream->ste_installed = true;
+	return false;
+}
+
 void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master,
 				  const struct arm_smmu_ste *target)
 {
 	int i, j;
 	struct arm_smmu_device *smmu = master->smmu;
+	bool is_abort;
 
 	master->cd_table.in_ste =
 		FIELD_GET(STRTAB_STE_0_CFG, le64_to_cpu(target->data[0])) ==
@@ -2989,10 +3014,12 @@ void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master,
 		FIELD_GET(STRTAB_STE_1_EATS, le64_to_cpu(target->data[1])) ==
 		STRTAB_STE_1_EATS_TRANS;
 
+	is_abort = FIELD_GET(STRTAB_STE_0_CFG, le64_to_cpu(target->data[0])) ==
+		   STRTAB_STE_0_CFG_ABORT;
+
 	for (i = 0; i < master->num_streams; ++i) {
 		u32 sid = master->streams[i].id;
-		struct arm_smmu_ste *step =
-			arm_smmu_get_step_for_sid(smmu, sid);
+		struct arm_smmu_ste *step;
 
 		/* Bridged PCI devices may end up with duplicated IDs */
 		for (j = 0; j < i; j++)
@@ -3001,6 +3028,16 @@ void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master,
 		if (j < i)
 			continue;
 
+		if (arm_smmu_is_shared_sid(master)) {
+			mutex_lock(&smmu->streams_mutex);
+			if (arm_smmu_skip_shared_ste(smmu, sid, is_abort)) {
+				mutex_unlock(&smmu->streams_mutex);
+				continue;
+			}
+			mutex_unlock(&smmu->streams_mutex);
+		}
+
+		step = arm_smmu_get_step_for_sid(smmu, sid);
 		arm_smmu_write_ste(master, sid, step, target);
 	}
 }
@@ -3149,7 +3186,7 @@ static int arm_smmu_enable_iopf(struct arm_smmu_master *master,
 		return 0;
 
 	/* We're not keeping track of SIDs in fault events */
-	if (master->num_streams != 1)
+	if (master->num_streams != 1 || arm_smmu_is_shared_sid(master))
 		return -EOPNOTSUPP;
 
 	if (master->iopf_refcount) {

-- 
2.34.1