[PATCH RFC v7 0/7] iommu/arm-smmu-v3: Support shared Stream IDs

[PATCH RFC v7 0/7] iommu/arm-smmu-v3: Support shared Stream IDs

[Peng Fan (OSS)]

Changes since RFC v6:
- Add "Plumb the CD table into the CD helpers" as a preparation patch, then
  introduce a per-domain CD table for non-PASID S1 streams. Normal non-PASID
  devices in one domain now share a single CD table; private tables are kept
  only for PASID, ATS-always-on, and stalling devices.
- Rework shared-SID STE handling. With the domain-owned CD table, all sharers
  build the same S1ContextPtr, so any sharer may program a normal domain STE.
  Release/blocked STE writes are skipped while the SID is shared, and the last
  master tears the STE down.
- Keep CD.S per-device rather than deriving it from the SMMU stall capability.
  Stall-capable masters use their private CD table so CD.S, STE.S1STALLD and
  STE.S2S remain consistent.
- Add arm_smmu_master_stall_enabled() and use it uniformly. Reject sharing if
  either master wants stalling, and resolve the stall policy before
  insert_master() so the incoming master's policy is visible.
- Reject sharing when the existing master has ATS, PASID, or stall enabled.
  Disable ATS, PASID, SVA, vSMMU and IOPF for shared SIDs.
- Link to v6: https://lore.kernel.org/linux-iommu/20261011-smmu-shared-sid-v6-0-cb76b89dc612@nxp.com/

Changes since RFC v5:
- Actively deduplicate Stream IDs during insertion instead of
  storing duplicate pointers. num_streams now reflects the unique
  count, eliminating pointer-equality dedup loops from all cleanup
  paths (Robin).
- Extracted arm_smmu_find_stream() into patch 2 instead of patch 3
  to clean up the find-before-allocate code (Nicolin).
- Extracted arm_smmu_free_master_streams() helper shared between
  error rollback and arm_smmu_remove_master() (Nicolin).
- Fixed remove_master to reassign stream->master when the current
  owner departs, preventing dangling pointer (Nicolin).
- Fixed remove_master to use a for loop iterating master->streams[]
  with per-stream shared vs non-shared handling (Nicolin).
- Reset ste_installed on ABORT so domain transitions (e.g. DMA ->
  VFIO) correctly re-write the STE instead of freezing it.
- Clear stall_enabled on the first master retroactively when
  sharing is detected in arm_smmu_insert_master(), not just on
  subsequent masters at probe time.  Prevents bus lockup when
  the first master probes before sharing is known.
- Link to v5: https://lore.kernel.org/linux-iommu/20261006-smmu-shared-sid-v5-0-169a59c671d3@nxp.com/

Changes since RFC v4:
- Include Nicolin's patch ("iommu/arm-smmu-v3: Don't rb_erase() a
  never-inserted stream node") to avoid conflicts in future.
  so the standalone patch 1 from v4 is dropped.
- Split old patch 2 (shared SID support) into two:
  - New patch 2: Convert streams from embedded array to individually
    allocated objects (struct arm_smmu_stream ** instead of *).
  - New patch 3: Move stream allocation inside the mutex and look up
    existing entries first (find-before-allocate), avoiding unnecessary
    allocation when the SID already exists.  Pure refactor, no
    functional change.
  - New patch 4: Add shared-SID logic on top of the refactored code.
- Duplicate-SID cleanup (bridged PCI) now uses pointer-equality dedup
  instead of RB_EMPTY_NODE checks, since streams are no longer
  pre-allocated with cleared nodes.
- Link to v4: https://lore.kernel.org/all/20261001-smmu-shared-sid-ver3-v4-0-06b34fb2e691@nxp.com/

Changes since RFC v3:
- Added a standalone bugfix for duplicate-SID rb_erase (new patch 1).
- Stream structures are now shared directly between masters
  instead of duplicating them (Nicolin).  No ref_count field;
  sharing is detected via list_empty(&stream->shared_masters).
- On owner removal, use rb_replace_node() + list_splice_init()
  to transfer the RB tree entry and shared_masters list to the
  next sharer, avoiding use-after-free.
- INIT_LIST_HEAD(&stream->shared_masters) moved after sort() to
  avoid list pointer corruption from bitwise memory swaps.
- Both sides of the sharing pair are checked for num_streams == 1.
- Link to v3: https://lore.kernel.org/all/20260930-smmu-shared-sid-v3-0-8abe6c1b4c20@nxp.com/

Changes since RFC v2:
- Dropped the RB tree to XArray conversion patch (Nicolin).
  The RB tree is retained as-is; a new arm_smmu_find_stream()
  helper provides stream-level lookup.
- No per-master shared_sid bool; replaced by
  list_empty(&master->shared_masters_elm) (Nicolin).
- Only SID-sharing for single SID device
- Follow Nicolin's suggestion https://lore.kernel.org/linux-iommu/arG6hmng3NddGEHm@nvidia.com/
- Link to v2: https://lore.kernel.org/linux-iommu/20260921-smmu-shared-sid-v2-0-4b656ce68178@nxp.com/

Changes since RFC v1:
- Revised cover letter to drop eDMA channel example (Jason).

Some SoCs have a limited number of IOMMU Stream IDs (SIDs) and
hardware that inherently shares them. For example, the NXP i.MX95
only has 64 SIDs, only 6 bits for SID in the system chip bus.
However there are many device IPs, such as SDHC, PCIE, NETC, GPU,
NPU, Display, VPU, ISP, ISI, JPEG, SATA, DMA and etc. Each Kind IP has
mutilple instances inside the SoC. So I introudce this patchset to
support shared SID.

The current ARM SMMUv3 driver rejects this with:
  "Aliasing StreamID unsupported, expect DMA to be broken"

I took comments in [1] and implement this patchset and only want to
support a very simple case that multiple platform devices share one SID.

Patch 1 converts the per-master streams array from an embedded array
of structs to an array of individually allocated pointers.  This
prepares for subsequent patches where multiple masters will point
to the same stream object.

Patch 2 moves stream allocation from upfront into the mutex-protected
loop in arm_smmu_insert_master(), using a find-before-allocate pattern:
first look up whether the SID already exists in the RB tree, only
allocate and insert a new stream when no existing entry is found.
This is a pure refactor with no functional change.

Patch 3 adds shared-SID tracking so that when a second single-SID
master registers an already-owned SID, it joins the existing
stream rather than being rejected.  The stream structure is shared
directly between masters via a linked list.  On removal, a shared
master detaches from the list; the stream and its RB tree entry
are kept alive until the last master is removed.

Patch 4 places devices that share a SID into the same IOMMU
group, ensuring they share a single IOMMU domain (required
because one STE can only point to one cd_table).

Patch 5 wires up STE write ordering (first master writes, last
master tears down) and gates features that require unambiguous
SID-to-device mapping: SVA, IOPF/stall, and vSMMU nesting are
disabled for shared-SID masters.

[1] https://lore.kernel.org/linux-iommu/DU0PR04MB94172CB3F138AD39E9B6DEEF887F9@DU0PR04MB9417.eurprd04.prod.outlook.com/

Signed-off-by: Peng Fan <peng.fan@nxp.com>
---
Peng Fan (7):
      iommu/arm-smmu-v3: Allocate streams individually
      iommu/arm-smmu-v3: Delay stream allocation to inside the mutex
      iommu/arm-smmu-v3: Plumb the CD table into the CD helpers
      iommu/arm-smmu-v3: Use a per-domain CD table for non-PASID streams
      iommu/arm-smmu-v3: Support shared SIDs in insert/remove_master
      iommu/arm-smmu-v3: Group aliasing devices into the same IOMMU group
      iommu/arm-smmu-v3: Wire up shared-SID STE ordering and feature gating

 .../iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c    |   4 +-
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c    |   3 +
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c   |   8 +-
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c        | 485 +++++++++++++++------
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h        |  35 +-
 drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c     |   2 +-
 6 files changed, 401 insertions(+), 136 deletions(-)
---
base-commit: d89e1d186ec9271bf78e62771779e9fc03bf0fc3
change-id: 20261026-smmu-shared-sid-ver7-2026-10-26-49ba8474f782

Best regards,
--  
OpenCode Assistant <opencode@Example.Com>

[PATCH RFC v7 1/7] iommu/arm-smmu-v3: Allocate streams individually

[Peng Fan (OSS)]

From: Peng Fan <peng.fan@nxp.com>

Change master->streams from an embedded array of struct arm_smmu_stream
to an array of pointers, with each stream individually allocated.

Prepare for shared-SID support where multiple masters will point to the
same stream object. With embedded structs, sharing requires duplicating
stream state and manually keeping fields like ste_installed in sync.
With individually allocated streams, a sharing master can simply point to
the existing stream.

The sort comparator is updated to dereference the pointer indirection.

No functional change.

Suggested-by: Nicolin Chen <nicolinc@nvidia.com>
Link: https://lore.kernel.org/linux-iommu/arG6hmng3NddGEHm@nvidia.com/
Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Peng Fan <peng.fan@nxp.com>
---
 .../iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c    |  2 +-
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c        | 52 ++++++++++++++--------
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h        |  2 +-
 drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c     |  2 +-
 4 files changed, 37 insertions(+), 21 deletions(-)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c
index e68b464850955..853276dd54536 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c
@@ -295,7 +295,7 @@ static int arm_vsmmu_vsid_to_sid(struct arm_vsmmu *vsmmu, u32 vsid, u32 *sid)
 
 	/* At this moment, iommufd only supports PCI device that has one SID */
 	if (sid)
-		*sid = master->streams[0].id;
+		*sid = master->streams[0]->id;
 unlock:
 	xa_unlock(&vsmmu->core.vdevs);
 	return ret;
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 6d4520c473086..5e092f51faad0 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -977,7 +977,7 @@ static void arm_smmu_page_response(struct device *dev, struct iopf_fault *unused
 	}
 
 	arm_smmu_cmdq_issue_cmd(master->smmu,
-				arm_smmu_make_cmd_resume(master->streams[0].id,
+				arm_smmu_make_cmd_resume(master->streams[0]->id,
 							 resp->grpid,
 							 resume_resp));
 	/*
@@ -1522,7 +1522,7 @@ static void arm_smmu_sync_cd(struct arm_smmu_master *master,
 	for (i = 0; i < master->num_streams; i++)
 		arm_smmu_cmdq_batch_add_cmd(
 			smmu, &cmds,
-			arm_smmu_make_cmd_cfgi_cd(master->streams[i].id, ssid,
+			arm_smmu_make_cmd_cfgi_cd(master->streams[i]->id, ssid,
 						  leaf));
 
 	arm_smmu_cmdq_batch_submit(smmu, &cmds);
@@ -2463,7 +2463,7 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master,
 	for (i = 0; i < master->num_streams; i++)
 		arm_smmu_cmdq_batch_add_cmd(
 			master->smmu, &cmds,
-			arm_smmu_make_cmd_atc_inv_all(master->streams[i].id,
+			arm_smmu_make_cmd_atc_inv_all(master->streams[i]->id,
 						      ssid));
 
 	return arm_smmu_cmdq_batch_submit(master->smmu, &cmds);
@@ -3155,13 +3155,13 @@ void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master,
 		STRTAB_STE_1_EATS_TRANS;
 
 	for (i = 0; i < master->num_streams; ++i) {
-		u32 sid = master->streams[i].id;
+		u32 sid = master->streams[i]->id;
 		struct arm_smmu_ste *step =
 			arm_smmu_get_step_for_sid(smmu, sid);
 
 		/* Bridged PCI devices may end up with duplicated IDs */
 		for (j = 0; j < i; j++)
-			if (master->streams[j].id == sid)
+			if (master->streams[j]->id == sid)
 				break;
 		if (j < i)
 			continue;
@@ -3447,7 +3447,7 @@ arm_smmu_master_build_invs(struct arm_smmu_master *master, bool ats_enabled,
 		 */
 		if (!arm_smmu_master_build_inv(
 			    master, nesting ? INV_TYPE_ATS_FULL : INV_TYPE_ATS,
-			    master->streams[i].id, ssid))
+			    master->streams[i]->id, ssid))
 			return NULL;
 	}
 
@@ -4276,10 +4276,10 @@ static int arm_smmu_init_sid_strtab(struct arm_smmu_device *smmu, u32 sid)
 
 static int arm_smmu_stream_id_cmp(const void *_l, const void *_r)
 {
-	const typeof_member(struct arm_smmu_stream, id) *l = _l;
-	const typeof_member(struct arm_smmu_stream, id) *r = _r;
+	const struct arm_smmu_stream * const *l = _l;
+	const struct arm_smmu_stream * const *r = _r;
 
-	return cmp_int(*l, *r);
+	return cmp_int((*l)->id, (*r)->id);
 }
 
 static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
@@ -4309,10 +4309,16 @@ static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
 	}
 
 	for (i = 0; i < fwspec->num_ids; i++) {
-		struct arm_smmu_stream *new_stream = &master->streams[i];
+		struct arm_smmu_stream *new_stream;
 
+		new_stream = kzalloc_obj(*new_stream, GFP_KERNEL);
+		if (!new_stream) {
+			ret = -ENOMEM;
+			goto out_free_streams;
+		}
 		new_stream->id = fwspec->ids[i];
 		new_stream->master = master;
+		master->streams[i] = new_stream;
 	}
 
 	/* Put the ids into order for sorted to_merge/to_unref arrays */
@@ -4325,11 +4331,11 @@ static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
 	 * which sort_nonatomic() invalidates by relocating the entries.
 	 */
 	for (i = 0; i < fwspec->num_ids; i++)
-		RB_CLEAR_NODE(&master->streams[i].node);
+		RB_CLEAR_NODE(&master->streams[i]->node);
 
 	mutex_lock(&smmu->streams_mutex);
 	for (i = 0; i < fwspec->num_ids; i++) {
-		struct arm_smmu_stream *new_stream = &master->streams[i];
+		struct arm_smmu_stream *new_stream = master->streams[i];
 		struct rb_node *existing;
 		u32 sid = new_stream->id;
 
@@ -4359,14 +4365,21 @@ static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
 
 	if (ret) {
 		for (i--; i >= 0; i--)
-			if (!RB_EMPTY_NODE(&master->streams[i].node))
-				rb_erase(&master->streams[i].node,
+			if (!RB_EMPTY_NODE(&master->streams[i]->node))
+				rb_erase(&master->streams[i]->node,
 					 &smmu->streams);
-		kfree(master->streams);
-		kfree(master->build_invs);
+		mutex_unlock(&smmu->streams_mutex);
+		goto out_free_streams;
 	}
 	mutex_unlock(&smmu->streams_mutex);
 
+	return 0;
+
+out_free_streams:
+	for (i = 0; i < master->num_streams; i++)
+		kfree(master->streams[i]);
+	kfree(master->streams);
+	kfree(master->build_invs);
 	return ret;
 }
 
@@ -4381,10 +4394,13 @@ static void arm_smmu_remove_master(struct arm_smmu_master *master)
 
 	mutex_lock(&smmu->streams_mutex);
 	for (i = 0; i < fwspec->num_ids; i++)
-		if (!RB_EMPTY_NODE(&master->streams[i].node))
-			rb_erase(&master->streams[i].node, &smmu->streams);
+		if (!RB_EMPTY_NODE(&master->streams[i]->node))
+			rb_erase(&master->streams[i]->node, &smmu->streams);
 	mutex_unlock(&smmu->streams_mutex);
 
+	for (i = 0; i < master->num_streams; i++)
+		kfree(master->streams[i]);
+
 	kfree(master->streams);
 	kfree(master->build_invs);
 }
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index bb048e08d613d..56ed6ad1dd523 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -1051,7 +1051,7 @@ struct arm_smmu_event {
 struct arm_smmu_master {
 	struct arm_smmu_device		*smmu;
 	struct device			*dev;
-	struct arm_smmu_stream		*streams;
+	struct arm_smmu_stream		**streams;
 	/*
 	 * Scratch memory for a to_merge or to_unref array to build a per-domain
 	 * invalidation array. It'll be pre-allocated with enough enries for all
diff --git a/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c b/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c
index 710a4c694b946..8166cece943c4 100644
--- a/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c
+++ b/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c
@@ -1257,7 +1257,7 @@ static int tegra241_vintf_init_vsid(struct iommufd_vdevice *vdev)
 	struct arm_smmu_master *master = dev_iommu_priv_get(dev);
 	struct tegra241_vintf *vintf = viommu_to_vintf(vdev->viommu);
 	struct tegra241_vintf_sid *vsid = vdev_to_vsid(vdev);
-	struct arm_smmu_stream *stream = &master->streams[0];
+	struct arm_smmu_stream *stream = master->streams[0];
 	u64 virt_sid = vdev->virt_id;
 	int sidx;
 

-- 
2.34.1

[PATCH RFC v7 2/7] iommu/arm-smmu-v3: Delay stream allocation to inside the mutex

[Peng Fan (OSS)]

From: Peng Fan <peng.fan@nxp.com>

Move arm_smmu_stream allocation from upfront (before the mutex) into
the mutex-protected loop in arm_smmu_insert_master(). Instead of
pre-allocating all stream objects and then inserting them into the RB
tree, first look up whether the SID already exists in the tree via the
new arm_smmu_find_stream() helper. Only allocate and insert a new
stream when no existing entry is found.

Duplicated Stream IDs from bridged PCI devices are now actively
skipped rather than stored as duplicate pointers. A separate write
index (n) tracks only unique entries, and master->num_streams is set
to the deduplicated count. This eliminates the need for duplicate-
pointer tracking in all cleanup paths.

Extract arm_smmu_find_stream() from arm_smmu_find_master() to provide
stream-level lookup, which will also be used by subsequent shared-SID
patches.

Extract arm_smmu_free_master_streams() to share the stream cleanup
logic between the error rollback in arm_smmu_insert_master() and
arm_smmu_remove_master().

The sort is moved after the mutex section, since streams are now
populated inside the loop rather than beforehand.

Suggested-by: Nicolin Chen <nicolinc@nvidia.com>
Suggested-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Peng Fan <peng.fan@nxp.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 124 ++++++++++++++--------------
 1 file changed, 60 insertions(+), 64 deletions(-)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 5e092f51faad0..0ea755ea82192 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2076,8 +2076,8 @@ static int arm_smmu_streams_cmp_node(struct rb_node *lhs,
 		&rb_entry(lhs, struct arm_smmu_stream, node)->id, rhs);
 }
 
-static struct arm_smmu_master *
-arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid)
+static struct arm_smmu_stream *
+arm_smmu_find_stream(struct arm_smmu_device *smmu, u32 sid)
 {
 	struct rb_node *node;
 
@@ -2086,7 +2086,18 @@ arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid)
 	node = rb_find(&sid, &smmu->streams, arm_smmu_streams_cmp_key);
 	if (!node)
 		return NULL;
-	return rb_entry(node, struct arm_smmu_stream, node)->master;
+	return rb_entry(node, struct arm_smmu_stream, node);
+}
+
+static struct arm_smmu_master *
+arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid)
+{
+	struct arm_smmu_stream *stream;
+
+	stream = arm_smmu_find_stream(smmu, sid);
+	if (!stream)
+		return NULL;
+	return stream->master;
 }
 
 /* IRQ and event handlers */
@@ -4282,10 +4293,25 @@ static int arm_smmu_stream_id_cmp(const void *_l, const void *_r)
 	return cmp_int((*l)->id, (*r)->id);
 }
 
+static void arm_smmu_free_master_streams(struct arm_smmu_master *master)
+{
+	struct arm_smmu_device *smmu = master->smmu;
+	int i;
+
+	lockdep_assert_held(&smmu->streams_mutex);
+
+	for (i = 0; i < master->num_streams; i++) {
+		if (!master->streams[i])
+			continue;
+		rb_erase(&master->streams[i]->node, &smmu->streams);
+		kfree(master->streams[i]);
+	}
+}
+
 static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
 				  struct arm_smmu_master *master)
 {
-	int i;
+	int i, n;
 	int ret = 0;
 	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(master->dev);
 	bool ats_supported = dev_is_pci(master->dev) &&
@@ -4294,7 +4320,6 @@ static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
 	master->streams = kzalloc_objs(*master->streams, fwspec->num_ids);
 	if (!master->streams)
 		return -ENOMEM;
-	master->num_streams = fwspec->num_ids;
 
 	if (!ats_supported) {
 		/* Base case has 1 ASID entry or maximum 2 VMID entries */
@@ -4308,99 +4333,70 @@ static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
 		return -ENOMEM;
 	}
 
-	for (i = 0; i < fwspec->num_ids; i++) {
-		struct arm_smmu_stream *new_stream;
-
-		new_stream = kzalloc_obj(*new_stream, GFP_KERNEL);
-		if (!new_stream) {
-			ret = -ENOMEM;
-			goto out_free_streams;
-		}
-		new_stream->id = fwspec->ids[i];
-		new_stream->master = master;
-		master->streams[i] = new_stream;
-	}
-
-	/* Put the ids into order for sorted to_merge/to_unref arrays */
-	sort(master->streams, master->num_streams,
-	     sizeof(master->streams[0]), arm_smmu_stream_id_cmp,
-	     NULL);
-
-	/*
-	 * Clear after sorting: RB_CLEAR_NODE() records the node's own address,
-	 * which sort_nonatomic() invalidates by relocating the entries.
-	 */
-	for (i = 0; i < fwspec->num_ids; i++)
-		RB_CLEAR_NODE(&master->streams[i]->node);
-
 	mutex_lock(&smmu->streams_mutex);
-	for (i = 0; i < fwspec->num_ids; i++) {
-		struct arm_smmu_stream *new_stream = master->streams[i];
-		struct rb_node *existing;
-		u32 sid = new_stream->id;
+	for (i = 0, n = 0; i < fwspec->num_ids; i++) {
+		struct arm_smmu_stream *stream;
+		u32 sid = fwspec->ids[i];
 
 		ret = arm_smmu_init_sid_strtab(smmu, sid);
 		if (ret)
 			break;
 
-		/* Insert into SID tree */
-		existing = rb_find_add(&new_stream->node, &smmu->streams,
-				       arm_smmu_streams_cmp_node);
-		if (existing) {
-			struct arm_smmu_master *existing_master =
-				rb_entry(existing, struct arm_smmu_stream, node)
-					->master;
-
+		stream = arm_smmu_find_stream(smmu, sid);
+		if (stream) {
 			/* Bridged PCI devices may end up with duplicated IDs */
-			if (existing_master == master)
+			if (stream->master == master)
 				continue;
 
 			dev_warn(master->dev,
 				 "Aliasing StreamID 0x%x (from %s) unsupported, expect DMA to be broken\n",
-				 sid, dev_name(existing_master->dev));
+				 sid, dev_name(stream->master->dev));
 			ret = -ENODEV;
 			break;
 		}
+
+		stream = kzalloc_obj(*stream, GFP_KERNEL);
+		if (!stream) {
+			ret = -ENOMEM;
+			break;
+		}
+		stream->id = sid;
+		stream->master = master;
+
+		rb_find_add(&stream->node, &smmu->streams,
+			    arm_smmu_streams_cmp_node);
+		master->streams[n++] = stream;
 	}
+	master->num_streams = n;
 
 	if (ret) {
-		for (i--; i >= 0; i--)
-			if (!RB_EMPTY_NODE(&master->streams[i]->node))
-				rb_erase(&master->streams[i]->node,
-					 &smmu->streams);
+		arm_smmu_free_master_streams(master);
 		mutex_unlock(&smmu->streams_mutex);
-		goto out_free_streams;
+		kfree(master->streams);
+		kfree(master->build_invs);
+		return ret;
 	}
 	mutex_unlock(&smmu->streams_mutex);
 
-	return 0;
+	/* Put the ids into order for sorted to_merge/to_unref arrays */
+	sort(master->streams, master->num_streams,
+	     sizeof(master->streams[0]), arm_smmu_stream_id_cmp,
+	     NULL);
 
-out_free_streams:
-	for (i = 0; i < master->num_streams; i++)
-		kfree(master->streams[i]);
-	kfree(master->streams);
-	kfree(master->build_invs);
-	return ret;
+	return 0;
 }
 
 static void arm_smmu_remove_master(struct arm_smmu_master *master)
 {
-	int i;
 	struct arm_smmu_device *smmu = master->smmu;
-	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(master->dev);
 
 	if (!smmu || !master->streams)
 		return;
 
 	mutex_lock(&smmu->streams_mutex);
-	for (i = 0; i < fwspec->num_ids; i++)
-		if (!RB_EMPTY_NODE(&master->streams[i]->node))
-			rb_erase(&master->streams[i]->node, &smmu->streams);
+	arm_smmu_free_master_streams(master);
 	mutex_unlock(&smmu->streams_mutex);
 
-	for (i = 0; i < master->num_streams; i++)
-		kfree(master->streams[i]);
-
 	kfree(master->streams);
 	kfree(master->build_invs);
 }

-- 
2.34.1

[PATCH RFC v7 3/7] iommu/arm-smmu-v3: Plumb the CD table into the CD helpers

[Peng Fan (OSS)]

From: Peng Fan <peng.fan@nxp.com>

Refactor the CD table allocation, freeing, lookup and STE-building helpers
to take a struct arm_smmu_ctx_desc_cfg pointer instead of always deriving
it from struct arm_smmu_master. Add master wrappers so existing callers
keep the same behaviour.

Prepare for domain-owned CD tables, where non-PASID streams will point
their STE at a CD table stored in the IOMMU domain rather than in the
master.

No functional change.

Signed-off-by: Peng Fan <peng.fan@nxp.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c |  3 +-
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c      | 63 +++++++++++++++---------
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h      |  5 +-
 3 files changed, 46 insertions(+), 25 deletions(-)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c
index 5a4e45e151258..864d9f2350bc1 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c
@@ -215,7 +215,8 @@ static void arm_smmu_test_make_cdtable_ste(struct arm_smmu_ste *ste,
 		.stall_enabled = stall_enabled,
 	};
 
-	arm_smmu_make_cdtable_ste(ste, &master, ats_enabled, s1dss);
+	arm_smmu_make_cdtable_ste(ste, &master, &master.cd_table, ats_enabled,
+				  s1dss);
 	if (feat & ARM_SMMU_MASTER_TEST_NESTED) {
 		struct arm_smmu_ste s2ste;
 		int i;
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 0ea755ea82192..51de3a0b8e76b 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -122,7 +122,7 @@ static const char * const event_class_str[] = {
 	[3] = "Reserved",
 };
 
-static int arm_smmu_alloc_cd_tables(struct arm_smmu_master *master);
+static int arm_smmu_alloc_master_cd_tables(struct arm_smmu_master *master);
 static bool arm_smmu_ats_supported(struct arm_smmu_master *master);
 
 static void parse_driver_options(struct arm_smmu_device *smmu)
@@ -1545,11 +1545,10 @@ static dma_addr_t arm_smmu_cd_l1_get_desc(const struct arm_smmu_cdtab_l1 *src)
 	return le64_to_cpu(src->l2ptr) & CTXDESC_L1_DESC_L2PTR_MASK;
 }
 
-struct arm_smmu_cd *arm_smmu_get_cd_ptr(struct arm_smmu_master *master,
-					u32 ssid)
+static struct arm_smmu_cd *
+arm_smmu_get_cd_ptr_in(struct arm_smmu_ctx_desc_cfg *cd_table, u32 ssid)
 {
 	struct arm_smmu_cdtab_l2 *l2;
-	struct arm_smmu_ctx_desc_cfg *cd_table = &master->cd_table;
 
 	if (!arm_smmu_cdtab_allocated(cd_table))
 		return NULL;
@@ -1563,6 +1562,12 @@ struct arm_smmu_cd *arm_smmu_get_cd_ptr(struct arm_smmu_master *master,
 	return &l2->cds[arm_smmu_cdtab_l2_idx(ssid)];
 }
 
+struct arm_smmu_cd *arm_smmu_get_cd_ptr(struct arm_smmu_master *master,
+					u32 ssid)
+{
+	return arm_smmu_get_cd_ptr_in(&master->cd_table, ssid);
+}
+
 static struct arm_smmu_cd *arm_smmu_alloc_cd_ptr(struct arm_smmu_master *master,
 						 u32 ssid)
 {
@@ -1573,7 +1578,7 @@ static struct arm_smmu_cd *arm_smmu_alloc_cd_ptr(struct arm_smmu_master *master,
 	iommu_group_mutex_assert(master->dev);
 
 	if (!arm_smmu_cdtab_allocated(cd_table)) {
-		if (arm_smmu_alloc_cd_tables(master))
+		if (arm_smmu_alloc_master_cd_tables(master))
 			return NULL;
 	}
 
@@ -1595,7 +1600,7 @@ static struct arm_smmu_cd *arm_smmu_alloc_cd_ptr(struct arm_smmu_master *master,
 			arm_smmu_sync_cd(master, ssid, false);
 		}
 	}
-	return arm_smmu_get_cd_ptr(master, ssid);
+	return arm_smmu_get_cd_ptr_in(cd_table, ssid);
 }
 
 struct arm_smmu_cd_writer {
@@ -1721,15 +1726,15 @@ void arm_smmu_clear_cd(struct arm_smmu_master *master, ioasid_t ssid)
 	arm_smmu_write_cd_entry(master, ssid, cdptr, &target);
 }
 
-static int arm_smmu_alloc_cd_tables(struct arm_smmu_master *master)
+static int arm_smmu_alloc_cd_tables(struct arm_smmu_device *smmu,
+				    struct arm_smmu_ctx_desc_cfg *cd_table,
+				    u8 ssid_bits, bool ats_always_on)
 {
 	int ret;
 	size_t l1size;
 	size_t max_contexts;
-	struct arm_smmu_device *smmu = master->smmu;
-	struct arm_smmu_ctx_desc_cfg *cd_table = &master->cd_table;
 
-	cd_table->s1cdmax = master->ssid_bits;
+	cd_table->s1cdmax = ssid_bits;
 
 	/*
 	 * When a device doesn't support PASID (non default SSID), ssid_bits is
@@ -1744,7 +1749,7 @@ static int arm_smmu_alloc_cd_tables(struct arm_smmu_master *master)
 	 * Set S1CDMAX no lower than 1. This would add a dummy substream in the
 	 * CD table but it should never be used by an actual CD.
 	 */
-	if (master->ats_always_on)
+	if (ats_always_on)
 		cd_table->s1cdmax = max_t(u8, cd_table->s1cdmax, 1);
 	max_contexts = 1 << cd_table->s1cdmax;
 
@@ -1786,11 +1791,17 @@ static int arm_smmu_alloc_cd_tables(struct arm_smmu_master *master)
 	return ret;
 }
 
-static void arm_smmu_free_cd_tables(struct arm_smmu_master *master)
+static int arm_smmu_alloc_master_cd_tables(struct arm_smmu_master *master)
+{
+	return arm_smmu_alloc_cd_tables(master->smmu, &master->cd_table,
+					master->ssid_bits,
+					master->ats_always_on);
+}
+
+static void arm_smmu_free_cd_tables(struct arm_smmu_device *smmu,
+				    struct arm_smmu_ctx_desc_cfg *cd_table)
 {
 	int i;
-	struct arm_smmu_device *smmu = master->smmu;
-	struct arm_smmu_ctx_desc_cfg *cd_table = &master->cd_table;
 
 	if (cd_table->s1fmt != STRTAB_STE_0_S1FMT_LINEAR) {
 		for (i = 0; i < cd_table->l2.num_l1_ents; i++) {
@@ -1816,6 +1827,11 @@ static void arm_smmu_free_cd_tables(struct arm_smmu_master *master)
 	}
 }
 
+static void arm_smmu_free_master_cd_tables(struct arm_smmu_master *master)
+{
+	arm_smmu_free_cd_tables(master->smmu, &master->cd_table);
+}
+
 /* Stream table manipulation functions */
 static void arm_smmu_write_strtab_l1_desc(struct arm_smmu_strtab_l1 *dst,
 					  dma_addr_t l2ptr_dma)
@@ -1900,10 +1916,10 @@ EXPORT_SYMBOL_IF_KUNIT(arm_smmu_make_bypass_ste);
 
 VISIBLE_IF_KUNIT
 void arm_smmu_make_cdtable_ste(struct arm_smmu_ste *target,
-			       struct arm_smmu_master *master, bool ats_enabled,
-			       unsigned int s1dss)
+			       struct arm_smmu_master *master,
+			       struct arm_smmu_ctx_desc_cfg *cd_table,
+			       bool ats_enabled, unsigned int s1dss)
 {
-	struct arm_smmu_ctx_desc_cfg *cd_table = &master->cd_table;
 	struct arm_smmu_device *smmu = master->smmu;
 
 	memset(target, 0, sizeof(*target));
@@ -3865,7 +3881,8 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev,
 		arm_smmu_make_s1_cd(&target_cd, master, smmu_domain);
 		arm_smmu_write_cd_entry(master, IOMMU_NO_PASID, cdptr,
 					&target_cd);
-		arm_smmu_make_cdtable_ste(&target, master, state.ats_enabled,
+		arm_smmu_make_cdtable_ste(&target, master, &master->cd_table,
+					  state.ats_enabled,
 					  STRTAB_STE_1_S1DSS_SSID0);
 		arm_smmu_install_ste_for_dev(master, &target);
 		break;
@@ -3932,7 +3949,8 @@ static void arm_smmu_update_ste(struct arm_smmu_master *master,
 	 * the S1DSS is correct and this will just update the EATS. Otherwise it
 	 * installs the entire thing. This will be hitless.
 	 */
-	arm_smmu_make_cdtable_ste(&ste, master, ats_enabled, s1dss);
+	arm_smmu_make_cdtable_ste(&ste, master, &master->cd_table, ats_enabled,
+				  s1dss);
 	arm_smmu_install_ste_for_dev(master, &ste);
 }
 
@@ -4065,7 +4083,8 @@ static void arm_smmu_attach_dev_ste(struct iommu_domain *domain,
 		 */
 		state.cd_needs_ats = true;
 		arm_smmu_attach_prepare(&state, domain);
-		arm_smmu_make_cdtable_ste(ste, master, state.ats_enabled, s1dss);
+		arm_smmu_make_cdtable_ste(ste, master, &master->cd_table,
+					  state.ats_enabled, s1dss);
 	} else {
 		arm_smmu_attach_prepare(&state, domain);
 	}
@@ -4436,7 +4455,7 @@ static int arm_smmu_master_prepare_ats(struct arm_smmu_master *master)
 
 	master->ats_always_on = true;
 
-	return arm_smmu_alloc_cd_tables(master);
+	return arm_smmu_alloc_master_cd_tables(master);
 }
 
 static struct iommu_device *arm_smmu_probe_device(struct device *dev)
@@ -4510,7 +4529,7 @@ static void arm_smmu_release_device(struct device *dev)
 	arm_smmu_disable_pasid(master);
 	arm_smmu_remove_master(master);
 	if (arm_smmu_cdtab_allocated(&master->cd_table))
-		arm_smmu_free_cd_tables(master);
+		arm_smmu_free_master_cd_tables(master);
 	kfree(master);
 }
 
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index 56ed6ad1dd523..0256119e9c137 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -1143,8 +1143,9 @@ void arm_smmu_get_cd_used(const __le64 *ent, __le64 *used_bits);
 void arm_smmu_make_bypass_ste(struct arm_smmu_device *smmu,
 			      struct arm_smmu_ste *target);
 void arm_smmu_make_cdtable_ste(struct arm_smmu_ste *target,
-			       struct arm_smmu_master *master, bool ats_enabled,
-			       unsigned int s1dss);
+			       struct arm_smmu_master *master,
+			       struct arm_smmu_ctx_desc_cfg *cd_table,
+			       bool ats_enabled, unsigned int s1dss);
 void arm_smmu_make_sva_cd(struct arm_smmu_cd *target,
 			  struct arm_smmu_master *master, struct mm_struct *mm,
 			  u16 asid);

-- 
2.34.1

[PATCH RFC v7 4/7] iommu/arm-smmu-v3: Use a per-domain CD table for non-PASID streams

[Peng Fan (OSS)]

From: Peng Fan <peng.fan@nxp.com>

The CD describes the domain context, so non-PASID devices attached to the
same S1 domain do not need private copies of it. Store a CD table in
struct arm_smmu_domain and make non-PASID streams point their STE at that
table.

The per-master CD table is now used when a device needs substreams (PASID),
for ats_always_on devices which need a CD table to use S1DSS, or when the
device uses the stall model. On the first PASID attach, seed the private
table's default substream from the RID domain and switch the STE to the
private table; after the last PASID detaches, switch the STE back to the
domain table (or the native bypass/abort STE). The private table is
retained for reuse and freed at device release.

CD.S is a property of the CD and must agree with STE.S1STALLD. On an SMMU
with STALL_MODEL==0b00, CD.S=1 together with STE.S1STALLD=1 is an illegal
CD and the SMMU aborts the stream. A shared domain CD therefore cannot
serve both stalling and terminating devices. Keep stall-enabled devices on
their private CD table, where CD.S can be set for them, while terminating
devices continue to share the domain table with CD.S=0.

Prepare for devices sharing a StreamID to naturally build the same
S1ContextPtr for the same domain, instead of depending on the stream
owner's private CD table.

Suggested-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Peng Fan <peng.fan@nxp.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 147 +++++++++++++++++++++++-----
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h |   8 ++
 2 files changed, 129 insertions(+), 26 deletions(-)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 51de3a0b8e76b..510c9028fa19f 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -1832,6 +1832,17 @@ static void arm_smmu_free_master_cd_tables(struct arm_smmu_master *master)
 	arm_smmu_free_cd_tables(master->smmu, &master->cd_table);
 }
 
+static int arm_smmu_alloc_domain_cd_tables(struct arm_smmu_domain *smmu_domain)
+{
+	return arm_smmu_alloc_cd_tables(smmu_domain->smmu,
+					&smmu_domain->cd_table, 0, false);
+}
+
+static void arm_smmu_free_domain_cd_tables(struct arm_smmu_domain *smmu_domain)
+{
+	arm_smmu_free_cd_tables(smmu_domain->smmu, &smmu_domain->cd_table);
+}
+
 /* Stream table manipulation functions */
 static void arm_smmu_write_strtab_l1_desc(struct arm_smmu_strtab_l1 *dst,
 					  dma_addr_t l2ptr_dma)
@@ -3044,6 +3055,9 @@ static void arm_smmu_domain_free_paging(struct iommu_domain *domain)
 		mutex_lock(&arm_smmu_asid_lock);
 		xa_erase(&arm_smmu_asid_xa, smmu_domain->cd.asid);
 		mutex_unlock(&arm_smmu_asid_lock);
+
+		if (arm_smmu_cdtab_allocated(&smmu_domain->cd_table))
+			arm_smmu_free_domain_cd_tables(smmu_domain);
 	} else {
 		struct arm_smmu_s2_cfg *cfg = &smmu_domain->s2_cfg;
 		if (cfg->vmid)
@@ -3135,6 +3149,15 @@ static int arm_smmu_domain_finalise(struct arm_smmu_domain *smmu_domain,
 	if (!pgtbl_ops)
 		return -ENOMEM;
 
+	smmu_domain->smmu = smmu;
+	if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) {
+		ret = arm_smmu_alloc_domain_cd_tables(smmu_domain);
+		if (ret) {
+			free_io_pgtable_ops(pgtbl_ops);
+			return ret;
+		}
+	}
+
 	smmu_domain->domain.pgsize_bitmap = pgtbl_cfg.pgsize_bitmap;
 	smmu_domain->tgsz_lg2 = __ffs(pgtbl_cfg.pgsize_bitmap);
 	smmu_domain->domain.geometry.aperture_end = (1UL << pgtbl_cfg.ias) - 1;
@@ -3144,12 +3167,13 @@ static int arm_smmu_domain_finalise(struct arm_smmu_domain *smmu_domain,
 
 	ret = finalise_stage_fn(smmu, smmu_domain);
 	if (ret < 0) {
+		if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1)
+			arm_smmu_free_domain_cd_tables(smmu_domain);
 		free_io_pgtable_ops(pgtbl_ops);
 		return ret;
 	}
 
 	smmu_domain->pgtbl_ops = pgtbl_ops;
-	smmu_domain->smmu = smmu;
 	return 0;
 }
 
@@ -3842,6 +3866,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev,
 		.ssid = IOMMU_NO_PASID,
 	};
 	struct arm_smmu_master *master;
+	struct arm_smmu_ctx_desc_cfg *cd_table = NULL;
 	struct arm_smmu_cd *cdptr;
 
 	if (!fwspec)
@@ -3854,11 +3879,27 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev,
 		return -EINVAL;
 
 	if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) {
-		cdptr = arm_smmu_alloc_cd_ptr(master, IOMMU_NO_PASID);
+		/*
+		 * Non-PASID devices use the domain's shared CD table. The
+		 * private table is needed for substreams (PASID), for
+		 * ATS-always-on (S1DSS), and for the stall model: CD.S is a
+		 * per-CD property and cannot be shared by devices that
+		 * terminate faults.
+		 */
+		if (master->ats_always_on ||
+		    master->stall_enabled ||
+		    arm_smmu_ssids_in_use(&master->cd_table)) {
+			cd_table = &master->cd_table;
+			cdptr = arm_smmu_alloc_cd_ptr(master, IOMMU_NO_PASID);
+		} else {
+			cd_table = &smmu_domain->cd_table;
+			cdptr = arm_smmu_get_cd_ptr_in(cd_table, IOMMU_NO_PASID);
+		}
 		if (!cdptr)
 			return -ENOMEM;
-	} else if (arm_smmu_ssids_in_use(&master->cd_table))
+	} else if (arm_smmu_ssids_in_use(&master->cd_table)) {
 		return -EBUSY;
+	}
 
 	/*
 	 * Prevent arm_smmu_share_asid() from trying to change the ASID
@@ -3881,7 +3922,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev,
 		arm_smmu_make_s1_cd(&target_cd, master, smmu_domain);
 		arm_smmu_write_cd_entry(master, IOMMU_NO_PASID, cdptr,
 					&target_cd);
-		arm_smmu_make_cdtable_ste(&target, master, &master->cd_table,
+		arm_smmu_make_cdtable_ste(&target, master, cd_table,
 					  state.ats_enabled,
 					  STRTAB_STE_1_S1DSS_SSID0);
 		arm_smmu_install_ste_for_dev(master, &target);
@@ -3932,22 +3973,24 @@ static void arm_smmu_update_ste(struct arm_smmu_master *master,
 				struct iommu_domain *sid_domain,
 				bool ats_enabled)
 {
-	unsigned int s1dss = STRTAB_STE_1_S1DSS_TERMINATE;
+	unsigned int s1dss;
 	struct arm_smmu_ste ste;
 
-	if (master->cd_table.in_ste && master->ste_ats_enabled == ats_enabled)
-		return;
-
 	if (sid_domain->type == IOMMU_DOMAIN_IDENTITY)
 		s1dss = STRTAB_STE_1_S1DSS_BYPASS;
-	else
-		WARN_ON(sid_domain->type != IOMMU_DOMAIN_BLOCKED);
+	else if (sid_domain->type == IOMMU_DOMAIN_BLOCKED)
+		s1dss = STRTAB_STE_1_S1DSS_TERMINATE;
+	else if (sid_domain->type & __IOMMU_DOMAIN_PAGING)
+		s1dss = STRTAB_STE_1_S1DSS_SSID0;
+	else {
+		WARN_ON(1);
+		return;
+	}
 
 	/*
-	 * Change the STE into a cdtable one with SID IDENTITY/BLOCKED behavior
-	 * using s1dss if necessary. If the cd_table is already installed then
-	 * the S1DSS is correct and this will just update the EATS. Otherwise it
-	 * installs the entire thing. This will be hitless.
+	 * The PASID entries live in the private per-master CD table, so make
+	 * sure the STE points there even when the RID domain was previously
+	 * using the domain's shared CD table.
 	 */
 	arm_smmu_make_cdtable_ste(&ste, master, &master->cd_table, ats_enabled,
 				  s1dss);
@@ -3965,7 +4008,8 @@ int arm_smmu_set_pasid(struct arm_smmu_master *master,
 		.ssid = pasid,
 		.old_domain = old,
 	};
-	struct arm_smmu_cd *cdptr;
+	struct arm_smmu_cd *cdptr, *cd0ptr = NULL;
+	bool first;
 	int ret;
 
 	/* The core code validates pasid */
@@ -3978,10 +4022,17 @@ int arm_smmu_set_pasid(struct arm_smmu_master *master,
 	    sid_domain->type != IOMMU_DOMAIN_BLOCKED)
 		return -EINVAL;
 
+	first = !arm_smmu_ssids_in_use(&master->cd_table);
 	cdptr = arm_smmu_alloc_cd_ptr(master, pasid);
 	if (!cdptr)
 		return -ENOMEM;
 
+	if (first) {
+		cd0ptr = arm_smmu_alloc_cd_ptr(master, IOMMU_NO_PASID);
+		if (!cd0ptr)
+			return -ENOMEM;
+	}
+
 	mutex_lock(&arm_smmu_asid_lock);
 	ret = arm_smmu_attach_prepare(&state, &smmu_domain->domain);
 	if (ret) {
@@ -3997,8 +4048,39 @@ int arm_smmu_set_pasid(struct arm_smmu_master *master,
 	cd->data[0] |= cpu_to_le64(
 		FIELD_PREP(CTXDESC_CD_0_ASID, smmu_domain->cd.asid));
 
+	/*
+	 * The private CD table is taking over the STE now. Make sure its
+	 * default substream matches the RID domain before the STE points at
+	 * it, since non-PASID traffic follows CD[0].
+	 */
+	if (first) {
+		if ((sid_domain->type & __IOMMU_DOMAIN_PAGING) &&
+		    to_smmu_domain(sid_domain)->stage == ARM_SMMU_DOMAIN_S1) {
+			struct arm_smmu_cd cd0;
+
+			arm_smmu_make_s1_cd(&cd0, master,
+					    to_smmu_domain(sid_domain));
+			arm_smmu_write_cd_entry(master, IOMMU_NO_PASID,
+						cd0ptr, &cd0);
+		} else if (sid_domain->type == IOMMU_DOMAIN_IDENTITY ||
+			   sid_domain->type == IOMMU_DOMAIN_BLOCKED) {
+			arm_smmu_clear_cd(master, IOMMU_NO_PASID);
+		} else {
+			WARN_ON(1);
+		}
+	}
+
 	arm_smmu_write_cd_entry(master, pasid, cdptr, cd);
-	arm_smmu_update_ste(master, sid_domain, state.ats_enabled);
+
+	/*
+	 * A repeat PASID attach leaves the STE on the private table with the
+	 * same S1DSS/EATS. Only the first attach (or an ATS change) needs to
+	 * rewrite it; arm_smmu_write_ste() would otherwise issue a redundant
+	 * CMD_PREFETCH_CFG for every stream.
+	 */
+	if (first || !master->cd_table.in_ste ||
+	    master->ste_ats_enabled != state.ats_enabled)
+		arm_smmu_update_ste(master, sid_domain, state.ats_enabled);
 
 	arm_smmu_attach_commit(&state);
 	mutex_unlock(&arm_smmu_asid_lock);
@@ -4017,35 +4099,48 @@ static int arm_smmu_blocking_set_dev_pasid(struct iommu_domain *new_domain,
 		.old_domain = old_domain,
 		.ssid = pasid,
 	};
+	int ret = 0;
 
 	mutex_lock(&arm_smmu_asid_lock);
 	arm_smmu_attach_prepare_invs(&state, NULL);
+	/*
+	 * Unlink the PASID from the domain before invalidating its CD.
+	 * arm_smmu_mm_release() walks this list with only devices_lock held
+	 * and writes a valid, faulting CD; unlinking first prevents it from
+	 * resurrecting used_ssids after the last PASID is detached.
+	 */
+	state.old_master_domain = arm_smmu_remove_master_domain(
+		master, &smmu_domain->domain, pasid);
 	arm_smmu_clear_cd(master, pasid);
 	if (master->ats_enabled)
 		arm_smmu_atc_inv_master(master, pasid);
-	state.old_master_domain = arm_smmu_remove_master_domain(
-		master, &smmu_domain->domain, pasid);
 	arm_smmu_install_old_domain_invs(&state);
 	mutex_unlock(&arm_smmu_asid_lock);
 	arm_smmu_attach_release(&state);
 
 	/*
-	 * When the last user of the CD table goes away downgrade the STE back
-	 * to a non-cd_table one, by re-attaching its sid_domain.
+	 * When the last user of the CD table goes away, switch the STE back to
+	 * the domain's shared CD table (S1) or to the native bypass/abort STE
+	 * (identity/blocked), by re-attaching its sid_domain. The private CD
+	 * table is kept allocated for reuse and freed when the device goes
+	 * away.
 	 */
 	if (!arm_smmu_ssids_in_use(&master->cd_table)) {
 		struct iommu_domain *sid_domain =
 			iommu_driver_get_domain_for_dev(master->dev);
 		bool ats_always_on = master->ats_always_on &&
 				     sid_domain->type != IOMMU_DOMAIN_BLOCKED;
-		bool downgrade = sid_domain->type == IOMMU_DOMAIN_IDENTITY ||
-				 sid_domain->type == IOMMU_DOMAIN_BLOCKED;
+		bool s1 = (sid_domain->type & __IOMMU_DOMAIN_PAGING) &&
+			  to_smmu_domain(sid_domain)->stage ==
+				  ARM_SMMU_DOMAIN_S1;
 
-		if (!ats_always_on && downgrade)
-			sid_domain->ops->attach_dev(sid_domain, dev,
-						    sid_domain);
+		if (!ats_always_on &&
+		    (s1 || sid_domain->type == IOMMU_DOMAIN_IDENTITY ||
+		     sid_domain->type == IOMMU_DOMAIN_BLOCKED))
+			ret = sid_domain->ops->attach_dev(sid_domain, dev,
+							  sid_domain);
 	}
-	return 0;
+	return ret;
 }
 
 static void arm_smmu_attach_dev_ste(struct iommu_domain *domain,
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index 0256119e9c137..a45a5a1580e0c 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -879,6 +879,7 @@ struct arm_smmu_ctx_desc_cfg {
 	};
 	dma_addr_t			cdtab_dma;
 	unsigned int			used_ssids;
+	/* The STE is S1 translate; it may point at a domain or private table */
 	u8				in_ste;
 	u8				s1fmt;
 	/* log2 of the maximum number of CDs supported by this table */
@@ -1091,6 +1092,13 @@ struct arm_smmu_domain {
 		struct arm_smmu_s2_cfg		s2_cfg;
 	};
 
+	/*
+	 * CD table shared by all non-PASID devices attached to this S1
+	 * domain. A master uses a private table only when it needs
+	 * substreams (PASID) or ATS-always-on/S1DSS.
+	 */
+	struct arm_smmu_ctx_desc_cfg	cd_table;
+
 	struct iommu_domain		domain;
 
 	struct arm_smmu_invs __rcu	*invs;

-- 
2.34.1

[PATCH RFC v7 5/7] iommu/arm-smmu-v3: Support shared SIDs in insert/remove_master

[Peng Fan (OSS)]

From: Peng Fan <peng.fan@nxp.com>

On SoCs like NXP i.MX95 where the SMMU has a limited number of Stream
IDs (e.g. 6-bit bus = 64 SIDs), multiple DMA-capable devices are
assigned the same Stream ID by SCU configuration.

When a second master attempts to use a SID that is already claimed,
link the masters together via the stream's shared_masters list instead
of rejecting the probe. The stream object is shared directly between
masters - no duplication.

Sharing is restricted to single-SID masters (num_streams == 1) to keep
the STE lifecycle simple. Multi-SID sharing is explicitly rejected.
Sharing is also rejected if the existing master has stall enabled, since
dynamically clearing stall_enabled without reprogramming the hardware
STE would leave the stall bit active and risk an unrecoverable bus
lockup when CMD_RESUME cannot be issued.

On removal, arm_smmu_free_master_streams() handles both the shared and
non-shared cases: a shared master detaches from the shared_masters list
and ownership transfers to the next sharer if the departing master was
the stream owner; the stream and its RB tree entry are kept alive until
the last master using that SID is removed.

Since fault events on shared SIDs cannot be attributed to a specific
master, arm_smmu_find_master() returns NULL when the stream's
shared_masters list is non-empty.

Suggested-by: Nicolin Chen <nicolinc@nvidia.com>
Link: https://lore.kernel.org/linux-iommu/Z3r-cBm00tVE4eKe@Asurada-Nvidia/
Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Peng Fan <peng.fan@nxp.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 55 ++++++++++++++++++++++++-----
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h |  2 ++
 2 files changed, 49 insertions(+), 8 deletions(-)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 510c9028fa19f..605d4179caaa9 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2124,6 +2124,8 @@ arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid)
 	stream = arm_smmu_find_stream(smmu, sid);
 	if (!stream)
 		return NULL;
+	if (!list_empty(&stream->shared_masters))
+		return NULL;
 	return stream->master;
 }
 
@@ -4415,10 +4417,26 @@ static void arm_smmu_free_master_streams(struct arm_smmu_master *master)
 	lockdep_assert_held(&smmu->streams_mutex);
 
 	for (i = 0; i < master->num_streams; i++) {
-		if (!master->streams[i])
+		struct arm_smmu_stream *stream = master->streams[i];
+
+		if (!stream)
+			continue;
+
+		if (!list_empty(&stream->shared_masters)) {
+			struct arm_smmu_master *next;
+
+			list_del_init(&master->shared_masters_elm);
+			next = list_first_entry(&stream->shared_masters,
+						struct arm_smmu_master,
+						shared_masters_elm);
+			if (stream->master == master)
+				stream->master = next;
+			if (list_is_singular(&stream->shared_masters))
+				list_del_init(&next->shared_masters_elm);
 			continue;
-		rb_erase(&master->streams[i]->node, &smmu->streams);
-		kfree(master->streams[i]);
+		}
+		rb_erase(&stream->node, &smmu->streams);
+		kfree(stream);
 	}
 }
 
@@ -4446,6 +4464,7 @@ static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
 		kfree(master->streams);
 		return -ENOMEM;
 	}
+	INIT_LIST_HEAD(&master->shared_masters_elm);
 
 	mutex_lock(&smmu->streams_mutex);
 	for (i = 0, n = 0; i < fwspec->num_ids; i++) {
@@ -4462,11 +4481,30 @@ static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
 			if (stream->master == master)
 				continue;
 
-			dev_warn(master->dev,
-				 "Aliasing StreamID 0x%x (from %s) unsupported, expect DMA to be broken\n",
-				 sid, dev_name(stream->master->dev));
-			ret = -ENODEV;
-			break;
+			if (fwspec->num_ids != 1 ||
+			    stream->master->num_streams != 1) {
+				dev_warn(master->dev,
+					 "Shared StreamID 0x%x not supported for multi-SID masters\n",
+					 sid);
+				ret = -ENODEV;
+				break;
+			}
+
+			if (stream->master->stall_enabled) {
+				dev_warn(master->dev,
+					 "Shared StreamID 0x%x rejected: existing master has stall enabled\n",
+					 sid);
+				ret = -EBUSY;
+				break;
+			}
+
+			if (list_empty(&stream->shared_masters))
+				list_add_tail(&stream->master->shared_masters_elm,
+					      &stream->shared_masters);
+			list_add_tail(&master->shared_masters_elm,
+				      &stream->shared_masters);
+			master->streams[n++] = stream;
+			continue;
 		}
 
 		stream = kzalloc_obj(*stream, GFP_KERNEL);
@@ -4476,6 +4514,7 @@ static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
 		}
 		stream->id = sid;
 		stream->master = master;
+		INIT_LIST_HEAD(&stream->shared_masters);
 
 		rb_find_add(&stream->node, &smmu->streams,
 			    arm_smmu_streams_cmp_node);
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index a45a5a1580e0c..c57221b311245 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -1021,6 +1021,7 @@ struct arm_smmu_stream {
 	u32				id;
 	struct arm_smmu_master		*master;
 	struct rb_node			node;
+	struct list_head		shared_masters;
 };
 
 struct arm_smmu_vmaster {
@@ -1069,6 +1070,7 @@ struct arm_smmu_master {
 	bool				ste_ats_enabled : 1;
 	bool				stall_enabled;
 	bool				ats_always_on;
+	struct list_head		shared_masters_elm;
 	unsigned int			ssid_bits;
 	unsigned int			iopf_refcount;
 };

-- 
2.34.1

[PATCH RFC v7 6/7] iommu/arm-smmu-v3: Group aliasing devices into the same IOMMU group

[Peng Fan (OSS)]

From: Peng Fan <peng.fan@nxp.com>

Devices sharing a SID must share a single IOMMU domain because the STE
can only point to one CD table at a time. With the domain-owned CD table,
non-PASID devices in that domain share that table directly.

Detect SID aliasing at group-assignment time by looking up the RB tree
for existing owners. Since probe_device() inserts the master before
device_group() is called, skip the device's own stream and look for a
different, already-probed master that shares the same SID.

Signed-off-by: Peng Fan <peng.fan@nxp.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 33 ++++++++++++++++++++---------
 1 file changed, 23 insertions(+), 10 deletions(-)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 605d4179caaa9..0ff513f9e0bc1 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -4690,19 +4690,32 @@ static int arm_smmu_set_dirty_tracking(struct iommu_domain *domain,
 
 static struct iommu_group *arm_smmu_device_group(struct device *dev)
 {
-	struct iommu_group *group;
+	struct arm_smmu_master *master = dev_iommu_priv_get(dev);
+	struct arm_smmu_device *smmu = master->smmu;
+	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
+	struct iommu_group *group = NULL;
+	int i;
+
+	mutex_lock(&smmu->streams_mutex);
+	for (i = 0; i < fwspec->num_ids; i++) {
+		struct arm_smmu_stream *stream;
+
+		stream = arm_smmu_find_stream(smmu, fwspec->ids[i]);
+		if (!stream || stream->master == master)
+			continue;
+
+		group = iommu_group_get(stream->master->dev);
+		break;
+	}
+	mutex_unlock(&smmu->streams_mutex);
+
+	if (group)
+		return group;
 
-	/*
-	 * We don't support devices sharing stream IDs other than PCI RID
-	 * aliases, since the necessary ID-to-device lookup becomes rather
-	 * impractical given a potential sparse 32-bit stream ID space.
-	 */
 	if (dev_is_pci(dev))
-		group = pci_device_group(dev);
-	else
-		group = generic_device_group(dev);
+		return pci_device_group(dev);
 
-	return group;
+	return generic_device_group(dev);
 }
 
 static int arm_smmu_of_xlate(struct device *dev,

-- 
2.34.1

[PATCH RFC v7 7/7] iommu/arm-smmu-v3: Wire up shared-SID STE ordering and feature gating

[Peng Fan (OSS)]

From: Peng Fan <peng.fan@nxp.com>

With the per-domain CD table, every non-PASID sharer of a SID builds the
same normal-domain STE: same S1ContextPtr, S1DSS and S1CDMAX. Disable
stall and ATS for shared SIDs as well, so the remaining STE fields are
common too, and let any sharer program the STE. This also handles the
case where one device's driver attaches before the stream owner's driver:
the first attach now installs a valid STE instead of leaving the boot or
firmware STE in place.

A release or blocked attach must not abort a shared STE while other
sharers remain, otherwise it would break their ongoing DMA. Skip those
writes while the SID is shared; the last master (no longer shared) still
tears the STE down. This also means removing one device from a shared
group no longer leaves the survivors aborted.

Fault events on shared SIDs cannot be attributed to a specific master,
so arm_smmu_find_master() returns NULL when the stream's shared_masters
list is non-empty.

Disable SVA, PASID, vSMMU nesting, ATS and IOPF/stall for shared-SID
masters since all of them require an unambiguous SID-to-device mapping.
Reject sharing at insert_master() time if either master has stall enabled,
or if the existing master has PASID or ATS enabled.

Introduce arm_smmu_master_stall_enabled() to describe the policy actually
programmed into the hardware: a master stalls only if it asked for it and
its StreamID is not shared. Use it consistently for CD.S, STE.S1STALLD and
STE.S2S so a shared CD always terminates faults and CD.S agrees with the
STE. Resolve stall_enabled before insert_master() so the incoming master's
policy is visible to the sharing check.

Add arm_smmu_is_shared_sid() as an inline helper in arm-smmu-v3.h so that
arm-smmu-v3-sva.c and arm-smmu-v3-iommufd.c can use it without repeating
the list_empty() pattern.

Signed-off-by: Peng Fan <peng.fan@nxp.com>
---
 .../iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c    |  2 +-
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c    |  3 +
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c   |  5 ++
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c        | 79 ++++++++++++++++++----
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h        | 18 +++++
 5 files changed, 91 insertions(+), 16 deletions(-)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c
index 853276dd54536..b435cc541aed3 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c
@@ -310,7 +310,7 @@ static int arm_vsmmu_vdevice_init(struct iommufd_vdevice *vdev)
 	 * arm_vsmmu_vsid_to_sid() maps a vSID to master->streams[0] alone, so
 	 * more streams would leave the rest stale and none reads out of bounds.
 	 */
-	if (master->num_streams != 1)
+	if (master->num_streams != 1 || arm_smmu_is_shared_sid(master))
 		return -EOPNOTSUPP;
 	return 0;
 }
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
index faeec656da9e7..b74fe609c907a 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
@@ -281,6 +281,9 @@ static int arm_smmu_sva_set_dev_pasid(struct iommu_domain *domain,
 	if (!(master->smmu->features & ARM_SMMU_FEAT_SVA))
 		return -EOPNOTSUPP;
 
+	if (arm_smmu_is_shared_sid(master))
+		return -EOPNOTSUPP;
+
 	/* Prevent arm_smmu_mm_release from being called while we are attaching */
 	if (!mmget_not_zero(domain->mm))
 		return -EINVAL;
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c
index 864d9f2350bc1..a3fbcd88675a5 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c
@@ -215,6 +215,7 @@ static void arm_smmu_test_make_cdtable_ste(struct arm_smmu_ste *ste,
 		.stall_enabled = stall_enabled,
 	};
 
+	INIT_LIST_HEAD(&master.shared_masters_elm);
 	arm_smmu_make_cdtable_ste(ste, &master, &master.cd_table, ats_enabled,
 				  s1dss);
 	if (feat & ARM_SMMU_MASTER_TEST_NESTED) {
@@ -351,6 +352,7 @@ static void arm_smmu_test_make_s2_ste(struct arm_smmu_ste *ste,
 		.pgtbl_ops = &io_pgtable.ops,
 	};
 
+	INIT_LIST_HEAD(&master.shared_masters_elm);
 	io_pgtable.cfg.arm_lpae_s2_cfg.vttbr = 0xdaedbeefdeadbeefULL;
 	io_pgtable.cfg.arm_lpae_s2_cfg.vtcr.ps = 1;
 	io_pgtable.cfg.arm_lpae_s2_cfg.vtcr.tg = 2;
@@ -506,6 +508,7 @@ static void arm_smmu_test_make_s1_cd(struct arm_smmu_cd *cd, unsigned int asid)
 		},
 	};
 
+	INIT_LIST_HEAD(&master.shared_masters_elm);
 	io_pgtable.cfg.arm_lpae_s1_cfg.ttbr = 0xdaedbeefdeadbeefULL;
 	io_pgtable.cfg.arm_lpae_s1_cfg.tcr.ips = 1;
 	io_pgtable.cfg.arm_lpae_s1_cfg.tcr.tg = 2;
@@ -549,6 +552,7 @@ static void arm_smmu_test_make_sva_cd(struct arm_smmu_cd *cd, unsigned int asid)
 		.smmu = &smmu,
 	};
 
+	INIT_LIST_HEAD(&master.shared_masters_elm);
 	arm_smmu_make_sva_cd(cd, &master, &sva_mm, asid);
 }
 
@@ -559,6 +563,7 @@ static void arm_smmu_test_make_sva_release_cd(struct arm_smmu_cd *cd,
 		.smmu = &smmu,
 	};
 
+	INIT_LIST_HEAD(&master.shared_masters_elm);
 	arm_smmu_make_sva_cd(cd, &master, NULL, asid);
 }
 
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 0ff513f9e0bc1..ee6060f05538f 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -1692,7 +1692,8 @@ void arm_smmu_make_s1_cd(struct arm_smmu_cd *target,
 		CTXDESC_CD_0_V |
 		FIELD_PREP(CTXDESC_CD_0_TCR_IPS, tcr->ips) |
 		CTXDESC_CD_0_AA64 |
-		(master->stall_enabled ? CTXDESC_CD_0_S : 0) |
+		(arm_smmu_master_stall_enabled(master) ?
+			 CTXDESC_CD_0_S : 0) |
 		CTXDESC_CD_0_R |
 		CTXDESC_CD_0_A |
 		CTXDESC_CD_0_ASET |
@@ -1947,7 +1948,7 @@ void arm_smmu_make_cdtable_ste(struct arm_smmu_ste *target,
 		FIELD_PREP(STRTAB_STE_1_S1COR, STRTAB_STE_1_S1C_CACHE_WBRA) |
 		FIELD_PREP(STRTAB_STE_1_S1CSH, ARM_SMMU_SH_ISH) |
 		((smmu->features & ARM_SMMU_FEAT_STALLS &&
-		  !master->stall_enabled) ?
+		  !arm_smmu_master_stall_enabled(master)) ?
 			 STRTAB_STE_1_S1STALLD :
 			 0) |
 		FIELD_PREP(STRTAB_STE_1_EATS,
@@ -2034,7 +2035,8 @@ void arm_smmu_make_s2_domain_ste(struct arm_smmu_ste *target,
 		STRTAB_STE_2_S2ENDI |
 #endif
 		STRTAB_STE_2_S2PTW |
-		(master->stall_enabled ? STRTAB_STE_2_S2S : 0) |
+		(arm_smmu_master_stall_enabled(master) ?
+			 STRTAB_STE_2_S2S : 0) |
 		STRTAB_STE_2_S2R);
 
 	target->data[3] = cpu_to_le64(pgtbl_cfg->arm_lpae_s2_cfg.vttbr &
@@ -3194,6 +3196,20 @@ arm_smmu_get_step_for_sid(struct arm_smmu_device *smmu, u32 sid)
 	}
 }
 
+static bool arm_smmu_ste_is_release(const struct arm_smmu_ste *target)
+{
+	u64 data0 = le64_to_cpu(target->data[0]);
+	u64 data1 = le64_to_cpu(target->data[1]);
+
+	if (FIELD_GET(STRTAB_STE_0_CFG, data0) == STRTAB_STE_0_CFG_ABORT)
+		return true;
+
+	return FIELD_GET(STRTAB_STE_0_CFG, data0) ==
+		       STRTAB_STE_0_CFG_S1_TRANS &&
+	       FIELD_GET(STRTAB_STE_1_S1DSS, data1) ==
+		       STRTAB_STE_1_S1DSS_TERMINATE;
+}
+
 void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master,
 				  const struct arm_smmu_ste *target)
 {
@@ -3209,8 +3225,7 @@ void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master,
 
 	for (i = 0; i < master->num_streams; ++i) {
 		u32 sid = master->streams[i]->id;
-		struct arm_smmu_ste *step =
-			arm_smmu_get_step_for_sid(smmu, sid);
+		struct arm_smmu_ste *step;
 
 		/* Bridged PCI devices may end up with duplicated IDs */
 		for (j = 0; j < i; j++)
@@ -3219,6 +3234,19 @@ void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master,
 		if (j < i)
 			continue;
 
+		/*
+		 * A shared STE represents the group's domain, not a single
+		 * master. Every sharer builds the same normal-domain STE, so
+		 * any of them may program it. A release/blocked attach would
+		 * instead abort the translation for the surviving sharers;
+		 * skip it until the last master (no longer shared) tears the
+		 * STE down.
+		 */
+		if (arm_smmu_is_shared_sid(master) &&
+		    arm_smmu_ste_is_release(target))
+			continue;
+
+		step = arm_smmu_get_step_for_sid(smmu, sid);
 		arm_smmu_write_ste(master, sid, step, target);
 	}
 }
@@ -3229,6 +3257,9 @@ static bool arm_smmu_ats_supported(struct arm_smmu_master *master)
 	struct arm_smmu_device *smmu = master->smmu;
 	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
 
+	if (arm_smmu_is_shared_sid(master))
+		return false;
+
 	if (!(smmu->features & ARM_SMMU_FEAT_ATS))
 		return false;
 
@@ -3366,8 +3397,7 @@ static int arm_smmu_enable_iopf(struct arm_smmu_master *master,
 	if (!master->stall_enabled)
 		return 0;
 
-	/* We're not keeping track of SIDs in fault events */
-	if (master->num_streams != 1)
+	if (master->num_streams != 1 || arm_smmu_is_shared_sid(master))
 		return -EOPNOTSUPP;
 
 	if (master->iopf_refcount) {
@@ -3889,7 +3919,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev,
 		 * terminate faults.
 		 */
 		if (master->ats_always_on ||
-		    master->stall_enabled ||
+		    arm_smmu_master_stall_enabled(master) ||
 		    arm_smmu_ssids_in_use(&master->cd_table)) {
 			cd_table = &master->cd_table;
 			cdptr = arm_smmu_alloc_cd_ptr(master, IOMMU_NO_PASID);
@@ -3962,6 +3992,9 @@ static int arm_smmu_s1_set_dev_pasid(struct iommu_domain *domain,
 	if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1)
 		return -EINVAL;
 
+	if (arm_smmu_is_shared_sid(master))
+		return -EOPNOTSUPP;
+
 	/*
 	 * We can read cd.asid outside the lock because arm_smmu_set_pasid()
 	 * will fix it
@@ -4490,9 +4523,20 @@ static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
 				break;
 			}
 
-			if (stream->master->stall_enabled) {
+			if (stream->master->stall_enabled ||
+			    master->stall_enabled) {
+				dev_warn(master->dev,
+					 "Shared StreamID 0x%x rejected: stall enabled\n",
+					 sid);
+				ret = -EBUSY;
+				break;
+			}
+
+			if (stream->master->ats_always_on ||
+			    stream->master->ats_enabled ||
+			    arm_smmu_ssids_in_use(&stream->master->cd_table)) {
 				dev_warn(master->dev,
-					 "Shared StreamID 0x%x rejected: existing master has stall enabled\n",
+					 "Shared StreamID 0x%x rejected: existing master has PASID/ATS enabled\n",
 					 sid);
 				ret = -EBUSY;
 				break;
@@ -4614,6 +4658,16 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev)
 	master->smmu = smmu;
 	dev_iommu_priv_set(dev, master);
 
+	/*
+	 * Resolve the stall policy before inserting the master. StreamID
+	 * sharing is rejected when either device wants stalling, and the
+	 * insert path needs to know the incoming master's policy.
+	 */
+	if ((smmu->features & ARM_SMMU_FEAT_STALLS &&
+	     device_property_read_bool(dev, "dma-can-stall")) ||
+	    smmu->features & ARM_SMMU_FEAT_STALL_FORCE)
+		master->stall_enabled = true;
+
 	ret = arm_smmu_insert_master(smmu, master);
 	if (ret)
 		goto err_free_master;
@@ -4635,11 +4689,6 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev)
 		master->ssid_bits = min_t(u8, master->ssid_bits,
 					  CTXDESC_LINEAR_CDMAX);
 
-	if ((smmu->features & ARM_SMMU_FEAT_STALLS &&
-	     device_property_read_bool(dev, "dma-can-stall")) ||
-	    smmu->features & ARM_SMMU_FEAT_STALL_FORCE)
-		master->stall_enabled = true;
-
 	ret = arm_smmu_master_prepare_ats(master);
 	if (ret)
 		goto err_disable_pasid;
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index c57221b311245..03fa965c4c888 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -1242,6 +1242,24 @@ int arm_smmu_init_one_queue(struct arm_smmu_device *smmu,
 int arm_smmu_cmdq_init(struct arm_smmu_device *smmu,
 		       struct arm_smmu_cmdq *cmdq);
 
+static inline bool arm_smmu_is_shared_sid(struct arm_smmu_master *master)
+{
+	return !list_empty(&master->shared_masters_elm);
+}
+
+/*
+ * The stall fault model is negotiated per device. A shared StreamID cannot
+ * represent differing per-device policies and stall events cannot be
+ * attributed to a single master, so sharing always terminates faults.
+ *
+ * This must be used consistently for CD.S, STE.S1STALLD and STE.S2S: an
+ * implementation can flag C_BAD_CD/C_BAD_STE for a mismatched CD/STE pair.
+ */
+static inline bool arm_smmu_master_stall_enabled(struct arm_smmu_master *master)
+{
+	return master->stall_enabled && !arm_smmu_is_shared_sid(master);
+}
+
 static inline bool arm_smmu_master_canwbs(struct arm_smmu_master *master)
 {
 	return dev_iommu_fwspec_get(master->dev)->flags &

-- 
2.34.1

Re: [PATCH RFC v7 3/7] iommu/arm-smmu-v3: Plumb the CD table into the CD helpers

[Nicolin Chen]

On Fri, Oct 30, 2026 at 08:09:35PM +0800, Peng Fan (OSS) wrote:
> From: Peng Fan <peng.fan@nxp.com>
> 
> Refactor the CD table allocation, freeing, lookup and STE-building helpers
> to take a struct arm_smmu_ctx_desc_cfg pointer instead of always deriving
> it from struct arm_smmu_master. Add master wrappers so existing callers
> keep the same behaviour.
> 
> Prepare for domain-owned CD tables, where non-PASID streams will point
> their STE at a CD table stored in the IOMMU domain rather than in the
> master.
> 
> No functional change.
> 
> Signed-off-by: Peng Fan <peng.fan@nxp.com>

Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>

With some nits:

> -struct arm_smmu_cd *arm_smmu_get_cd_ptr(struct arm_smmu_master *master,
> -					u32 ssid)
> +static struct arm_smmu_cd *
> +arm_smmu_get_cd_ptr_in(struct arm_smmu_ctx_desc_cfg *cd_table, u32 ssid)

Just keep "arm_smmu_get_cd_ptr", and change master to cd_table.

> +struct arm_smmu_cd *arm_smmu_get_cd_ptr(struct arm_smmu_master *master,
> +					u32 ssid)

arm_smmu_master_get_cd_ptr

> +static int arm_smmu_alloc_master_cd_tables(struct arm_smmu_master *master)

arm_smmu_master_alloc_cd_tables

And in the other patch:
arm_smmu_domain_alloc_cd_tables

> +static void arm_smmu_free_master_cd_tables(struct arm_smmu_master *master)

arm_smmu_master_free_cd_tables

> +{
> +	arm_smmu_free_cd_tables(master->smmu, &master->cd_table);
> +}

Maybe fold in the condition.

	if (arm_smmu_cdtab_allocated(&master->cd_table))
		arm_smmu_free_cd_tables(master->smmu, &master->cd_table);

Nicolin

Re: [PATCH RFC v7 4/7] iommu/arm-smmu-v3: Use a per-domain CD table for non-PASID streams

[Nicolin Chen]

On Fri, Oct 30, 2026 at 08:09:36PM +0800, Peng Fan (OSS) wrote:
> From: Peng Fan <peng.fan@nxp.com>
> 
> The CD describes the domain context, so non-PASID devices attached to the
> same S1 domain do not need private copies of it. Store a CD table in
> struct arm_smmu_domain and make non-PASID streams point their STE at that
> table.

> @@ -3044,6 +3055,9 @@ static void arm_smmu_domain_free_paging(struct iommu_domain *domain)
>  		mutex_lock(&arm_smmu_asid_lock);
>  		xa_erase(&arm_smmu_asid_xa, smmu_domain->cd.asid);
>  		mutex_unlock(&arm_smmu_asid_lock);
> +
> +		if (arm_smmu_cdtab_allocated(&smmu_domain->cd_table))
> +			arm_smmu_free_domain_cd_tables(smmu_domain);

Same comments as previous: fold arm_smmu_cdtab_allocated in.

> @@ -3854,11 +3879,27 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev,
>  		return -EINVAL;
>  
>  	if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) {
> -		cdptr = arm_smmu_alloc_cd_ptr(master, IOMMU_NO_PASID);
> +		/*
> +		 * Non-PASID devices use the domain's shared CD table. The
> +		 * private table is needed for substreams (PASID), for
> +		 * ATS-always-on (S1DSS), and for the stall model: CD.S is a
> +		 * per-CD property and cannot be shared by devices that
> +		 * terminate faults.
> +		 */
> +		if (master->ats_always_on ||
> +		    master->stall_enabled ||

For readability, let's add a helper:

static bool arm_smmu_master_needs_private_cdtab(struct arm_smmu_master *master)
{
	/* Reason 1 */
	if (master->ats_always_on)
		return true;
	/* Reason 2 */
	if (master->stall_enabled)
		return true;
	return false;
}

> @@ -3978,10 +4022,17 @@ int arm_smmu_set_pasid(struct arm_smmu_master *master,
>  	    sid_domain->type != IOMMU_DOMAIN_BLOCKED)
>  		return -EINVAL;
>  
> +	first = !arm_smmu_ssids_in_use(&master->cd_table);
>  	cdptr = arm_smmu_alloc_cd_ptr(master, pasid);
>  	if (!cdptr)
>  		return -ENOMEM;
>  
> +	if (first) {
> +		cd0ptr = arm_smmu_alloc_cd_ptr(master, IOMMU_NO_PASID);
> +		if (!cd0ptr)
> +			return -ENOMEM;
> +	}

Maybe allocate cd0ptr in arm_smmu_alloc_cd_tables()?

We can also have a helper to share with arm_smmu_alloc_cd_ptr().

static int arm_smmu_alloc_cd_ptr_l2(struct arm_smmu_master *master, u32 ssid)
{
	[...]
	*l2ptr = dma_alloc_coherent(master->smmu->dev, sizeof(**l2ptr),
				    &l2ptr_dma, GFP_KERNEL);
	if (!*l2ptr)
		return -ENOMEM;

	arm_smmu_write_cd_l1_desc(&cd_table->l2.l1tab[idx], l2ptr_dma);
	return 0;
}

> @@ -3997,8 +4048,39 @@ int arm_smmu_set_pasid(struct arm_smmu_master *master,
>  	cd->data[0] |= cpu_to_le64(
>  		FIELD_PREP(CTXDESC_CD_0_ASID, smmu_domain->cd.asid));
>  
> +	/*
> +	 * The private CD table is taking over the STE now. Make sure its
> +	 * default substream matches the RID domain before the STE points at
> +	 * it, since non-PASID traffic follows CD[0].
> +	 */
> +	if (first) {
> +		if ((sid_domain->type & __IOMMU_DOMAIN_PAGING) &&
> +		    to_smmu_domain(sid_domain)->stage == ARM_SMMU_DOMAIN_S1) {
> +			struct arm_smmu_cd cd0;
> +
> +			arm_smmu_make_s1_cd(&cd0, master,
> +					    to_smmu_domain(sid_domain));
> +			arm_smmu_write_cd_entry(master, IOMMU_NO_PASID,
> +						cd0ptr, &cd0);
> +		} else if (sid_domain->type == IOMMU_DOMAIN_IDENTITY ||
> +			   sid_domain->type == IOMMU_DOMAIN_BLOCKED) {
> +			arm_smmu_clear_cd(master, IOMMU_NO_PASID);
> +		} else {
> +			WARN_ON(1);
> +		}
> +	}
> +
>  	arm_smmu_write_cd_entry(master, pasid, cdptr, cd);
> -	arm_smmu_update_ste(master, sid_domain, state.ats_enabled);
> +
> +	/*
> +	 * A repeat PASID attach leaves the STE on the private table with the
> +	 * same S1DSS/EATS. Only the first attach (or an ATS change) needs to
> +	 * rewrite it; arm_smmu_write_ste() would otherwise issue a redundant
> +	 * CMD_PREFETCH_CFG for every stream.
> +	 */
> +	if (first || !master->cd_table.in_ste ||
> +	    master->ste_ats_enabled != state.ats_enabled)
> +		arm_smmu_update_ste(master, sid_domain, state.ats_enabled);

master->cd_table.in_ste could be set when STE walks master's CD
table. Then, you don't need to check "first". And move the guard
back into arm_smmu_update_ste.

> @@ -4017,35 +4099,48 @@ static int arm_smmu_blocking_set_dev_pasid(struct iommu_domain *new_domain,
>  		.old_domain = old_domain,
>  		.ssid = pasid,
>  	};
> +	int ret = 0;
>  
>  	mutex_lock(&arm_smmu_asid_lock);
>  	arm_smmu_attach_prepare_invs(&state, NULL);
> +	/*
> +	 * Unlink the PASID from the domain before invalidating its CD.
> +	 * arm_smmu_mm_release() walks this list with only devices_lock held
> +	 * and writes a valid, faulting CD; unlinking first prevents it from
> +	 * resurrecting used_ssids after the last PASID is detached.
> +	 */
> +	state.old_master_domain = arm_smmu_remove_master_domain(
> +		master, &smmu_domain->domain, pasid);
>  	arm_smmu_clear_cd(master, pasid);

Sashiko pointed out that it needs devices_lock here.

>  	/*
> -	 * When the last user of the CD table goes away downgrade the STE back
> -	 * to a non-cd_table one, by re-attaching its sid_domain.
> +	 * When the last user of the CD table goes away, switch the STE back to
> +	 * the domain's shared CD table (S1) or to the native bypass/abort STE
> +	 * (identity/blocked), by re-attaching its sid_domain. The private CD
> +	 * table is kept allocated for reuse and freed when the device goes
> +	 * away.
>  	 */
>  	if (!arm_smmu_ssids_in_use(&master->cd_table)) {
>  		struct iommu_domain *sid_domain =
>  			iommu_driver_get_domain_for_dev(master->dev);
>  		bool ats_always_on = master->ats_always_on &&
>  				     sid_domain->type != IOMMU_DOMAIN_BLOCKED;
> -		bool downgrade = sid_domain->type == IOMMU_DOMAIN_IDENTITY ||
> -				 sid_domain->type == IOMMU_DOMAIN_BLOCKED;
> +		bool s1 = (sid_domain->type & __IOMMU_DOMAIN_PAGING) &&
> +			  to_smmu_domain(sid_domain)->stage ==
> +				  ARM_SMMU_DOMAIN_S1;
>  
> -		if (!ats_always_on && downgrade)
> -			sid_domain->ops->attach_dev(sid_domain, dev,
> -						    sid_domain);
> +		if (!ats_always_on &&
> +		    (s1 || sid_domain->type == IOMMU_DOMAIN_IDENTITY ||
> +		     sid_domain->type == IOMMU_DOMAIN_BLOCKED))
> +			ret = sid_domain->ops->attach_dev(sid_domain, dev,
> +							  sid_domain);

This covers almost all the possible domain types here. So, just
drop that downgrade, and it should cover the other cases:

		if (!ats_always_on)

Nicolin