[PATCH v3 0/3] iommu/amd: Invalidate IRT cache for DMA aliases

[PATCH v3 0/3] iommu/amd: Invalidate IRT cache for DMA aliases

[Magnus Kalland]

DMA aliasing causes interrupt remapping table entries (IRTEs) to be shared
between multiple device IDs. See commit 3c124435e8dd
("iommu/amd: Support multiple PCI DMA aliases in IRQ Remapping") for more
information on this. However, the AMD IOMMU driver currently invalidates
IRTE cache entries on a per-device basis whenever an IRTE is updated, not
for each alias.

This approach leaves stale IRTE cache entries when an IRTE is cached under
one DMA alias but later updated and invalidated through a different alias.
In such cases, the original device ID is never invalidated, since it is
programmed via aliasing.

This incoherency bug has been observed when IRTEs are cached for one
Non-Transparent Bridge (NTB) DMA alias, later updated via another.

Fix this by invalidating the interrupt remapping table cache for all DMA
aliases when updating an IRTE.

Changes since v2:
 - Look for aliases with pci_seg->alias_table instead of
   pci_for_each_dma_alias since we can't get the pdev (lockdep).
   Track the aliases in set_remap_table_entry_alias. Invalidate IRT cache
   for each BDF sharing alias with the given devid in
   iommu_flush_irt_and_complete.
 - Make iommu_table_lock a raw spinlock to use it when invalidating
   IRT caches.
 - Rebased and applied cleanly on the IOMMU maintainers tree

Resending for visibility.

Link: https://lore.kernel.org/linux-iommu/26cfa307-6c33-41f9-a7a0-fbf202b38a00@amd.com/
Co-developed-by: Lars B. Kristiansen <larsk@dolphinics.com>
Signed-off-by: Lars B. Kristiansen <larsk@dolphinics.com>
Co-developed-by: Jonas Markussen <jonas@dolphinics.com>
Signed-off-by: Jonas Markussen <jonas@dolphinics.com>
Co-developed-by: Tore H. Larsen <torel@simula.no>
Signed-off-by: Tore H. Larsen <torel@simula.no>
Signed-off-by: Magnus Kalland <magnus@dolphinics.com>

Magnus Kalland (3):
  iommu/amd: Use raw spinlock for interrupt remapping tables
  iommu/amd: Track PCIe DMA aliases in set_remap_table_entry_alias
  iommu/amd: Invalidate IRT cache for DMA aliases

 drivers/iommu/amd/iommu.c | 52 +++++++++++++++++++++++++++++++++------
 1 file changed, 45 insertions(+), 7 deletions(-)

-- 
2.43.0

[PATCH v3 1/3] iommu/amd: Use raw spinlock for interrupt remapping tables

[Magnus Kalland]

Use raw spinlock for interrupt remapping tables since
iommu_flush_irt_and_complete is called under a raw spinlock.

Signed-off-by: Magnus Kalland <magnus@dolphinics.com>

---
 drivers/iommu/amd/iommu.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
index 81c4d7733872..f3193c6428c9 100644
--- a/drivers/iommu/amd/iommu.c
+++ b/drivers/iommu/amd/iommu.c
@@ -3164,7 +3164,8 @@ const struct iommu_ops amd_iommu_ops = {
  *****************************************************************************/
 
 static struct irq_chip amd_ir_chip;
-static DEFINE_SPINLOCK(iommu_table_lock);
+static DEFINE_RAW_SPINLOCK(iommu_table_lock);
+
 
 static void iommu_flush_irt_and_complete(struct amd_iommu *iommu, u16 devid)
 {
@@ -3310,7 +3311,7 @@ static struct irq_remap_table *alloc_irq_table(struct amd_iommu *iommu,
 	int nid = iommu->dev ? dev_to_node(&iommu->dev->dev) : NUMA_NO_NODE;
 	u16 alias;
 
-	spin_lock_irqsave(&iommu_table_lock, flags);
+	raw_spin_lock_irqsave(&iommu_table_lock, flags);
 
 	pci_seg = iommu->pci_seg;
 	table = pci_seg->irq_lookup_table[devid];
@@ -3323,14 +3324,14 @@ static struct irq_remap_table *alloc_irq_table(struct amd_iommu *iommu,
 		set_remap_table_entry(iommu, devid, table);
 		goto out_wait;
 	}
-	spin_unlock_irqrestore(&iommu_table_lock, flags);
+	raw_spin_unlock_irqrestore(&iommu_table_lock, flags);
 
 	/* Nothing there yet, allocate new irq remapping table */
 	new_table = __alloc_irq_table(nid, get_irq_table_size(max_irqs));
 	if (!new_table)
 		return NULL;
 
-	spin_lock_irqsave(&iommu_table_lock, flags);
+	raw_spin_lock_irqsave(&iommu_table_lock, flags);
 
 	table = pci_seg->irq_lookup_table[devid];
 	if (table)
@@ -3358,7 +3359,7 @@ static struct irq_remap_table *alloc_irq_table(struct amd_iommu *iommu,
 	iommu_completion_wait(iommu);
 
 out_unlock:
-	spin_unlock_irqrestore(&iommu_table_lock, flags);
+	raw_spin_unlock_irqrestore(&iommu_table_lock, flags);
 
 	if (new_table) {
 		iommu_free_pages(new_table->table);
-- 
2.43.0

[PATCH v3 2/3] iommu/amd: Track PCIe DMA aliases in set_remap_table_entry_alias

[Magnus Kalland]

Track PCIe DMA aliases in set_remap_table_entry_alias so that we can
iterate over shared IRTs by alias in iommu_flush_irt_for_aliases.

Signed-off-by: Magnus Kalland <magnus@dolphinics.com>

---
 drivers/iommu/amd/iommu.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
index f3193c6428c9..5dec3502c8b3 100644
--- a/drivers/iommu/amd/iommu.c
+++ b/drivers/iommu/amd/iommu.c
@@ -3280,12 +3280,14 @@ static int set_remap_table_entry_alias(struct pci_dev *pdev, u16 alias,
 	struct irq_remap_table *table = data;
 	struct amd_iommu_pci_seg *pci_seg;
 	struct amd_iommu *iommu = rlookup_amd_iommu(&pdev->dev);
+	u16 devid = pci_dev_id(pdev);
 
 	if (!iommu)
 		return -EINVAL;
 
 	pci_seg = iommu->pci_seg;
 	pci_seg->irq_lookup_table[alias] = table;
+	pci_seg->alias_table[alias] = devid;
 	set_dte_irq_entry(iommu, alias, table);
 	iommu_flush_dte(pci_seg->rlookup_table[alias], alias);
 
-- 
2.43.0

[PATCH v3 3/3] iommu/amd: Invalidate IRT cache for DMA aliases

[Magnus Kalland]

IRTEs may be shared between multiple device IDs when PCIe DMA
aliasing is in use. The AMD IOMMU driver currently invalidates
the interrupt remapping table cache only for the device ID used
to update the IRTE.

If the same IRTE is cached under a different DMA alias, this
leaves stale cache entries that are never invalidated.

Iterate over all device IDs sharing the same DMA alias and
invalidate the IRT cache for each of them when an IRTE is updated.

Co-developed-by: Lars B. Kristiansen <larsk@dolphinics.com>
Signed-off-by: Lars B. Kristiansen <larsk@dolphinics.com>
Co-developed-by: Jonas Markussen <jonas@dolphinics.com>
Signed-off-by: Jonas Markussen <jonas@dolphinics.com>
Co-developed-by: Tore H. Larsen <torel@simula.no>
Signed-off-by: Tore H. Larsen <torel@simula.no>
Signed-off-by: Magnus Kalland <magnus@dolphinics.com>
Link: https://lore.kernel.org/linux-iommu/26cfa307-6c33-41f9-a7a0-fbf202b38a00@amd.com/

---
 drivers/iommu/amd/iommu.c | 39 +++++++++++++++++++++++++++++++++++++--
 1 file changed, 37 insertions(+), 2 deletions(-)

diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
index 5dec3502c8b3..d9a91d1a083e 100644
--- a/drivers/iommu/amd/iommu.c
+++ b/drivers/iommu/amd/iommu.c
@@ -3166,6 +3166,31 @@ const struct iommu_ops amd_iommu_ops = {
 static struct irq_chip amd_ir_chip;
 static DEFINE_RAW_SPINLOCK(iommu_table_lock);
 
+static int iommu_flush_irt_for_aliases(struct amd_iommu *iommu,
+				       u16 alias)
+{
+	struct amd_iommu_pci_seg *pci_seg = iommu->pci_seg;
+	struct iommu_cmd cmd;
+	unsigned long flags;
+	u32 devid;
+	int ret = 0;
+
+	raw_spin_lock_irqsave(&iommu_table_lock, flags);
+
+	for (devid = 0; devid <= pci_seg->last_bdf; ++devid) {
+		if (pci_seg->alias_table[devid] != alias)
+			continue;
+
+		build_inv_irt(&cmd, devid);
+		ret = __iommu_queue_command_sync(iommu, &cmd, true);
+		if (ret)
+			goto out;
+	}
+
+out:
+	raw_spin_unlock_irqrestore(&iommu_table_lock, flags);
+	return ret;
+}
 
 static void iommu_flush_irt_and_complete(struct amd_iommu *iommu, u16 devid)
 {
@@ -3173,19 +3198,29 @@ static void iommu_flush_irt_and_complete(struct amd_iommu *iommu, u16 devid)
 	u64 data;
 	unsigned long flags;
 	struct iommu_cmd cmd, cmd2;
+	u16 alias;
 
 	if (iommu->irtcachedis_enabled)
 		return;
 
-	build_inv_irt(&cmd, devid);
+	raw_spin_lock_irqsave(&iommu_table_lock, flags);
+	alias = iommu->pci_seg->alias_table[devid];
+	raw_spin_unlock_irqrestore(&iommu_table_lock, flags);
 
 	raw_spin_lock_irqsave(&iommu->lock, flags);
 	data = get_cmdsem_val(iommu);
 	build_completion_wait(&cmd2, iommu, data);
 
-	ret = __iommu_queue_command_sync(iommu, &cmd, true);
+	if (alias == devid) {
+		build_inv_irt(&cmd, devid);
+		ret = __iommu_queue_command_sync(iommu, &cmd, true);
+	} else {
+		ret = iommu_flush_irt_for_aliases(iommu, alias);
+	}
+
 	if (ret)
 		goto out_err;
+
 	ret = __iommu_queue_command_sync(iommu, &cmd2, false);
 	if (ret)
 		goto out_err;
-- 
2.43.0

Re: [PATCH v3 0/3] iommu/amd: Invalidate IRT cache for DMA aliases

[Dheeraj Kumar Srivastava]

Hi

On 3/6/2026 2:52 PM, Magnus Kalland wrote:
> DMA aliasing causes interrupt remapping table entries (IRTEs) to be shared
> between multiple device IDs. See commit 3c124435e8dd
> ("iommu/amd: Support multiple PCI DMA aliases in IRQ Remapping") for more
> information on this. However, the AMD IOMMU driver currently invalidates
> IRTE cache entries on a per-device basis whenever an IRTE is updated, not
> for each alias.
> 
> This approach leaves stale IRTE cache entries when an IRTE is cached under
> one DMA alias but later updated and invalidated through a different alias.
> In such cases, the original device ID is never invalidated, since it is
> programmed via aliasing.
> 
> This incoherency bug has been observed when IRTEs are cached for one
> Non-Transparent Bridge (NTB) DMA alias, later updated via another.
> 
> Fix this by invalidating the interrupt remapping table cache for all DMA
> aliases when updating an IRTE.
> 
> Changes since v2:
>   - Look for aliases with pci_seg->alias_table instead of
>     pci_for_each_dma_alias since we can't get the pdev (lockdep).
>     Track the aliases in set_remap_table_entry_alias. Invalidate IRT cache
>     for each BDF sharing alias with the given devid in
>     iommu_flush_irt_and_complete.
>   - Make iommu_table_lock a raw spinlock to use it when invalidating
>     IRT caches.
>   - Rebased and applied cleanly on the IOMMU maintainers tree
> 
> Resending for visibility.

I tested the patch with lockdep (CONFIG_PROVE_LOCKING=y) enabled and
observed the following lockdep warnings in the kernel logs.

[   13.224217] kernel: =============================
[   13.224217] kernel: [ BUG: Invalid wait context ]
[   13.224217] kernel: 7.0.0-rc3-eee9d444276c-1773350434439 #1 Not tainted
[   13.224217] kernel: -----------------------------
[   13.224217] kernel: kworker/0:1/11 is trying to lock:
[   13.224217] kernel: ff364c9da9e416b8 
(&dev_data->dte_lock){....}-{3:3}, at: set_dte_irq_entry+0x84/0x150
[   13.224217] kernel: other info that might help us debug this:
[   13.224217] kernel: context-{5:5}
[   13.224217] kernel: 5 locks held by kworker/0:1/11:
[   13.224217] kernel:  #0: ff364bde58164158 
((wq_completion)sync_wq){+.+.}-{0:0}, at: process_one_work+0x4c0/0x730
[   13.224217] kernel:  #1: ff8437db001efe48 
((work_completion)(&arg.work)){+.+.}-{0:0}, at: process_one_work+0x1ee/0x730
[   13.224217] kernel:  #2: ff364bde63dabaa0 (&md->mutex){+.+.}-{4:4}, 
at: __pci_enable_msi_range+0x228/0x360
[   13.224217] kernel:  #3: ff364bde4109bca0 
(&domain->mutex){+.+.}-{4:4}, at: __irq_domain_alloc_irqs+0x3b/0xa0
[   13.224217] kernel:  #4: ffffffff886590f8 
(iommu_table_lock){....}-{2:2}, at: alloc_irq_table+0x190/0x2c0
[   13.224217] kernel: stack backtrace:
[   13.224217] kernel: CPU: 0 UID: 0 PID: 11 Comm: kworker/0:1 Not 
tainted 7.0.0-rc3-eee9d444276c-1773350434439 #1 PREEMPT(lazy)
[   13.224217] kernel: Hardware name: AMD Corporation 
Titanite_4G/Titanite_4G, BIOS RTI100FD_1 03/25/2025
[   13.224217] kernel: Workqueue: sync_wq local_pci_probe_callback
[   13.224217] kernel: Call Trace:
[   13.224217] kernel:  <TASK>
[   13.224217] kernel:  dump_stack_lvl+0x78/0xe0
[   13.224217] kernel:  __lock_acquire+0x836/0xbe0
[   13.224217] kernel:  lock_acquire+0xc7/0x300
[   13.224217] kernel:  ? set_dte_irq_entry+0x84/0x150
[   13.224217] kernel:  ? lock_acquire+0xc7/0x300
[   13.224217] kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
[   13.224217] kernel:  _raw_spin_lock+0x34/0x80
[   13.224217] kernel:  ? set_dte_irq_entry+0x84/0x150
[   13.224217] kernel:  set_dte_irq_entry+0x84/0x150
[   13.224217] kernel:  set_remap_table_entry_alias+0x64/0x90
[   13.224217] kernel:  ? __pfx_set_remap_table_entry_alias+0x10/0x10
[   13.224217] kernel:  pci_for_each_dma_alias+0x3f/0x150
[   13.224217] kernel:  alloc_irq_table+0x1f9/0x2c0
[   13.224217] kernel:  alloc_irq_index+0x2c/0x170
[   13.224217] kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
[   13.224217] kernel:  irq_remapping_alloc+0x1b6/0x520
[   13.224217] kernel:  msi_domain_alloc+0x71/0x140
[   13.224217] kernel:  irq_domain_alloc_irqs_locked+0xd2/0x370
[   13.224217] kernel:  __irq_domain_alloc_irqs+0x56/0xa0
[   13.224217] kernel:  __msi_domain_alloc_irqs+0x1f7/0x430
[   13.224217] kernel:  msi_domain_alloc_irqs_all_locked+0x5a/0xa0
[   13.224217] kernel:  __msi_capability_init+0x152/0x240
[   13.224217] kernel:  __pci_enable_msi_range+0x234/0x360
[   13.224217] kernel:  pci_alloc_irq_vectors_affinity+0xc5/0x110
[   13.224217] kernel:  pcie_port_enable_irq_vec+0x3e/0x220
[   13.224217] kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
[   13.224217] kernel:  ? __pci_set_master+0x55/0xe0
[   13.224217] kernel:  pcie_portdrv_probe+0xdf/0x2f0
[   13.224217] kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
[   13.224217] kernel:  local_pci_probe+0x41/0x90
[   13.224217] kernel:  local_pci_probe_callback+0x16/0x20
[   13.224217] kernel:  process_one_work+0x22f/0x730
[   13.224217] kernel:  worker_thread+0x1d3/0x3a0
[   13.224217] kernel:  ? __pfx_worker_thread+0x10/0x10
[   13.224217] kernel:  kthread+0xe6/0x120
[   13.224217] kernel:  ? __pfx_kthread+0x10/0x10
[   13.224217] kernel:  ret_from_fork+0x2cb/0x340
[   13.224217] kernel:  ? __pfx_kthread+0x10/0x10
[   13.224217] kernel:  ret_from_fork_asm+0x1a/0x30
[   13.224217] kernel:  </TASK>

Thanks
Dheeraj

> 
> Link: https://lore.kernel.org/linux-iommu/26cfa307-6c33-41f9-a7a0-fbf202b38a00@amd.com/
> Co-developed-by: Lars B. Kristiansen <larsk@dolphinics.com>
> Signed-off-by: Lars B. Kristiansen <larsk@dolphinics.com>
> Co-developed-by: Jonas Markussen <jonas@dolphinics.com>
> Signed-off-by: Jonas Markussen <jonas@dolphinics.com>
> Co-developed-by: Tore H. Larsen <torel@simula.no>
> Signed-off-by: Tore H. Larsen <torel@simula.no>
> Signed-off-by: Magnus Kalland <magnus@dolphinics.com>
> 
> Magnus Kalland (3):
>    iommu/amd: Use raw spinlock for interrupt remapping tables
>    iommu/amd: Track PCIe DMA aliases in set_remap_table_entry_alias
>    iommu/amd: Invalidate IRT cache for DMA aliases
> 
>   drivers/iommu/amd/iommu.c | 52 +++++++++++++++++++++++++++++++++------
>   1 file changed, 45 insertions(+), 7 deletions(-)
> 

Re: [PATCH v3 0/3] iommu/amd: Invalidate IRT cache for DMA aliases

[Magnus Kalland]

> I tested the patch with lockdep (CONFIG_PROVE_LOCKING=y) enabled and
> observed the following lockdep warnings in the kernel logs.

Thanks Dheeraj for the testing.

I believe we can make the dte_lock also a raw spinlock. What do you think?

I'll send a v4 addressing this lockdep issue and also with a fix for a
potential future lock-order inversion in iommu_flush_irt_and_complete.

Magnus

[PATCH v3 0/3] iommu/amd: Invalidate IRT cache for DMA aliases

[Magnus Kalland]

DMA aliasing causes interrupt remapping table entries (IRTEs) to be shared
between multiple device IDs. See commit 3c124435e8dd
("iommu/amd: Support multiple PCI DMA aliases in IRQ Remapping") for more
information on this. However, the AMD IOMMU driver currently invalidates
IRTE cache entries on a per-device basis whenever an IRTE is updated, not
for each alias.

This approach leaves stale IRTE cache entries when an IRTE is cached under
one DMA alias but later updated and invalidated through a different alias.
In such cases, the original device ID is never invalidated, since it is
programmed via aliasing.

This incoherency bug has been observed when IRTEs are cached for one
Non-Transparent Bridge (NTB) DMA alias, later updated via another.

Fix this by invalidating the interrupt remapping table cache for all DMA
aliases when updating an IRTE.

Changes since v2:
 - Look for aliases with pci_seg->alias_table instead of
   pci_for_each_dma_alias since we can't get the pdev (lockdep).
   Track the aliases in set_remap_table_entry_alias. Invalidate IRT cache
   for each BDF sharing alias with the given devid in
   iommu_flush_irt_and_complete.
 - Make iommu_table_lock a raw spinlock to use it when invalidating
   IRT caches.
 - Rebased and applied cleanly on the IOMMU development tree

Co-developed-by: Lars B. Kristiansen <larsk@dolphinics.com>
Signed-off-by: Lars B. Kristiansen <larsk@dolphinics.com>
Co-developed-by: Jonas Markussen <jonas@dolphinics.com>
Signed-off-by: Jonas Markussen <jonas@dolphinics.com>
Co-developed-by: Tore H. Larsen <torel@simula.no>
Signed-off-by: Tore H. Larsen <torel@simula.no>
Signed-off-by: Magnus Kalland <magnus@dolphinics.com>
Link: https://lore.kernel.org/linux-iommu/26cfa307-6c33-41f9-a7a0-fbf202b38a00@amd.com/

Magnus Kalland (3):
  iommu/amd: Use raw spinlock for interrupt remapping tables
  iommu/amd: Track PCIe DMA aliases in set_remap_table_entry_alias
  iommu/amd: Invalidate IRT cache for DMA aliases

 drivers/iommu/amd/iommu.c | 52 +++++++++++++++++++++++++++++++++------
 1 file changed, 45 insertions(+), 7 deletions(-)

-- 
2.43.0