Re: [PATCH v3 0/3] iommu/amd: Invalidate IRT cache for DMA aliases

[Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>]

Hi

On 3/6/2026 2:52 PM, Magnus Kalland wrote:
> DMA aliasing causes interrupt remapping table entries (IRTEs) to be shared
> between multiple device IDs. See commit 3c124435e8dd
> ("iommu/amd: Support multiple PCI DMA aliases in IRQ Remapping") for more
> information on this. However, the AMD IOMMU driver currently invalidates
> IRTE cache entries on a per-device basis whenever an IRTE is updated, not
> for each alias.
> 
> This approach leaves stale IRTE cache entries when an IRTE is cached under
> one DMA alias but later updated and invalidated through a different alias.
> In such cases, the original device ID is never invalidated, since it is
> programmed via aliasing.
> 
> This incoherency bug has been observed when IRTEs are cached for one
> Non-Transparent Bridge (NTB) DMA alias, later updated via another.
> 
> Fix this by invalidating the interrupt remapping table cache for all DMA
> aliases when updating an IRTE.
> 
> Changes since v2:
>   - Look for aliases with pci_seg->alias_table instead of
>     pci_for_each_dma_alias since we can't get the pdev (lockdep).
>     Track the aliases in set_remap_table_entry_alias. Invalidate IRT cache
>     for each BDF sharing alias with the given devid in
>     iommu_flush_irt_and_complete.
>   - Make iommu_table_lock a raw spinlock to use it when invalidating
>     IRT caches.
>   - Rebased and applied cleanly on the IOMMU maintainers tree
> 
> Resending for visibility.

I tested the patch with lockdep (CONFIG_PROVE_LOCKING=y) enabled and
observed the following lockdep warnings in the kernel logs.

[   13.224217] kernel: =============================
[   13.224217] kernel: [ BUG: Invalid wait context ]
[   13.224217] kernel: 7.0.0-rc3-eee9d444276c-1773350434439 #1 Not tainted
[   13.224217] kernel: -----------------------------
[   13.224217] kernel: kworker/0:1/11 is trying to lock:
[   13.224217] kernel: ff364c9da9e416b8 
(&dev_data->dte_lock){....}-{3:3}, at: set_dte_irq_entry+0x84/0x150
[   13.224217] kernel: other info that might help us debug this:
[   13.224217] kernel: context-{5:5}
[   13.224217] kernel: 5 locks held by kworker/0:1/11:
[   13.224217] kernel:  #0: ff364bde58164158 
((wq_completion)sync_wq){+.+.}-{0:0}, at: process_one_work+0x4c0/0x730
[   13.224217] kernel:  #1: ff8437db001efe48 
((work_completion)(&arg.work)){+.+.}-{0:0}, at: process_one_work+0x1ee/0x730
[   13.224217] kernel:  #2: ff364bde63dabaa0 (&md->mutex){+.+.}-{4:4}, 
at: __pci_enable_msi_range+0x228/0x360
[   13.224217] kernel:  #3: ff364bde4109bca0 
(&domain->mutex){+.+.}-{4:4}, at: __irq_domain_alloc_irqs+0x3b/0xa0
[   13.224217] kernel:  #4: ffffffff886590f8 
(iommu_table_lock){....}-{2:2}, at: alloc_irq_table+0x190/0x2c0
[   13.224217] kernel: stack backtrace:
[   13.224217] kernel: CPU: 0 UID: 0 PID: 11 Comm: kworker/0:1 Not 
tainted 7.0.0-rc3-eee9d444276c-1773350434439 #1 PREEMPT(lazy)
[   13.224217] kernel: Hardware name: AMD Corporation 
Titanite_4G/Titanite_4G, BIOS RTI100FD_1 03/25/2025
[   13.224217] kernel: Workqueue: sync_wq local_pci_probe_callback
[   13.224217] kernel: Call Trace:
[   13.224217] kernel:  <TASK>
[   13.224217] kernel:  dump_stack_lvl+0x78/0xe0
[   13.224217] kernel:  __lock_acquire+0x836/0xbe0
[   13.224217] kernel:  lock_acquire+0xc7/0x300
[   13.224217] kernel:  ? set_dte_irq_entry+0x84/0x150
[   13.224217] kernel:  ? lock_acquire+0xc7/0x300
[   13.224217] kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
[   13.224217] kernel:  _raw_spin_lock+0x34/0x80
[   13.224217] kernel:  ? set_dte_irq_entry+0x84/0x150
[   13.224217] kernel:  set_dte_irq_entry+0x84/0x150
[   13.224217] kernel:  set_remap_table_entry_alias+0x64/0x90
[   13.224217] kernel:  ? __pfx_set_remap_table_entry_alias+0x10/0x10
[   13.224217] kernel:  pci_for_each_dma_alias+0x3f/0x150
[   13.224217] kernel:  alloc_irq_table+0x1f9/0x2c0
[   13.224217] kernel:  alloc_irq_index+0x2c/0x170
[   13.224217] kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
[   13.224217] kernel:  irq_remapping_alloc+0x1b6/0x520
[   13.224217] kernel:  msi_domain_alloc+0x71/0x140
[   13.224217] kernel:  irq_domain_alloc_irqs_locked+0xd2/0x370
[   13.224217] kernel:  __irq_domain_alloc_irqs+0x56/0xa0
[   13.224217] kernel:  __msi_domain_alloc_irqs+0x1f7/0x430
[   13.224217] kernel:  msi_domain_alloc_irqs_all_locked+0x5a/0xa0
[   13.224217] kernel:  __msi_capability_init+0x152/0x240
[   13.224217] kernel:  __pci_enable_msi_range+0x234/0x360
[   13.224217] kernel:  pci_alloc_irq_vectors_affinity+0xc5/0x110
[   13.224217] kernel:  pcie_port_enable_irq_vec+0x3e/0x220
[   13.224217] kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
[   13.224217] kernel:  ? __pci_set_master+0x55/0xe0
[   13.224217] kernel:  pcie_portdrv_probe+0xdf/0x2f0
[   13.224217] kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
[   13.224217] kernel:  local_pci_probe+0x41/0x90
[   13.224217] kernel:  local_pci_probe_callback+0x16/0x20
[   13.224217] kernel:  process_one_work+0x22f/0x730
[   13.224217] kernel:  worker_thread+0x1d3/0x3a0
[   13.224217] kernel:  ? __pfx_worker_thread+0x10/0x10
[   13.224217] kernel:  kthread+0xe6/0x120
[   13.224217] kernel:  ? __pfx_kthread+0x10/0x10
[   13.224217] kernel:  ret_from_fork+0x2cb/0x340
[   13.224217] kernel:  ? __pfx_kthread+0x10/0x10
[   13.224217] kernel:  ret_from_fork_asm+0x1a/0x30
[   13.224217] kernel:  </TASK>

Thanks
Dheeraj

> 
> Link: https://lore.kernel.org/linux-iommu/26cfa307-6c33-41f9-a7a0-fbf202b38a00@amd.com/
> Co-developed-by: Lars B. Kristiansen <larsk@dolphinics.com>
> Signed-off-by: Lars B. Kristiansen <larsk@dolphinics.com>
> Co-developed-by: Jonas Markussen <jonas@dolphinics.com>
> Signed-off-by: Jonas Markussen <jonas@dolphinics.com>
> Co-developed-by: Tore H. Larsen <torel@simula.no>
> Signed-off-by: Tore H. Larsen <torel@simula.no>
> Signed-off-by: Magnus Kalland <magnus@dolphinics.com>
> 
> Magnus Kalland (3):
>    iommu/amd: Use raw spinlock for interrupt remapping tables
>    iommu/amd: Track PCIe DMA aliases in set_remap_table_entry_alias
>    iommu/amd: Invalidate IRT cache for DMA aliases
> 
>   drivers/iommu/amd/iommu.c | 52 +++++++++++++++++++++++++++++++++------
>   1 file changed, 45 insertions(+), 7 deletions(-)
>