[PATCH v1] iommu/amd: Prevent spurious IRQ disable on high-rate GALog MSIs

[PATCH v1] iommu/amd: Prevent spurious IRQ disable on high-rate GALog MSIs

[Mina Almasry]

On AMD EPYC (Turin) hosts with AVIC enabled, passed-through SR-IOV VFs
in KVM guests log interrupts targeting non-running vCPUs into the IOMMU
Guest Virtual APIC Log (GALog) ring buffer and signal the host via an
edge-triggered IntCapXT MSI (AMD-Vi*-GA).

Both __iommu_setup_intcapxt() and iommu_setup_msi() register their
handlers via request_threaded_irq(..., NULL, thread_fn, IRQF_ONESHOT,
...), using irq_default_primary_handler() (which returns
IRQ_WAKE_THREAD), while intcapxt_irqdomain_alloc() sets handle_edge_irq
which does not mask edge-triggered MSIs during oneshot thread execution.

Previously, amd_iommu_handle_irq() cleared MMIO_STATUS_OFFSET at the top
of the while (status & mask) loop before calling int_handler(iommu)
(iommu_poll_ga_log()). Under overcommitted multi-VM TCP_RR workloads
(1,000 to 10,000 flows across 4-8 VMs with vCPU overcommit), new VF
interrupts for non-running vCPUs arrive while iommu_poll_ga_log() is
draining the ring buffer, causing GALogInt to transition 0 -> 1 and fire
additional hardware edge MSIs while the IRQ thread is still inside
amd_iommu_handle_irq(). Because desc->threads_handled only increments in
irq_thread_fn() after action->thread_fn() returns, note_interrupt()
(kernel/irq/spurious.c) accounts every intermediate edge MSI as IRQ_NONE
(desc->irqs_unhandled++). Once desc->irqs_unhandled exceeds 99,900 out
of 100,000 interrupts, __report_bad_irq() logs "irq <N>: nobody cared"
and permanently disables the IOMMU GALog IRQ, breaking guest VF MSI-X
wakeups.

Fix this in two ways:
1. Drain the log ring buffer via int_handler(iommu) before writing
   status & mask to MMIO_STATUS_OFFSET (clearing only the status bits
   observed set so a concurrent overflow bit is never cleared without
   running overflow_handler()), followed by a post-clear poll pass so
   entries written before the status clear took effect are processed
   without generating redundant edge MSIs or leaving entries stranded.
2. Set IRQ_IS_POLLED on the AMD-Vi MSI and IntCapXT IRQs (clearing it on
   request_threaded_irq() failure paths) so note_interrupt() never
   spuriously disables the dedicated IOMMU event/PPR/GALog IRQ lines
   under sustained high-rate edge MSI bursts.

Discovered in production testing on AMD EPYC (Turin) hosts running
multi-VM KVM guests with passed-through SR-IOV VFs where the host
permanently disabled AMD-Vi-GA under heavy TCP_RR traffic, and verified
on the same AMD EPYC (Turin) hardware across 4-VM and 8-VM SR-IOV VF
test suites.

Fixes: 2379f3485239 ("iommu/amd: Refactor IOMMU interrupt handling logic for Event, PPR, and GA logs")
Cc: stable@vger.kernel.org
Signed-off-by: Mina Almasry <almasrymina@google.com>
---
 drivers/iommu/amd/init.c  |  4 ++++
 drivers/iommu/amd/iommu.c | 13 ++++++++++---
 2 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
index de6db5621194..5eea9204f209 100644
--- a/drivers/iommu/amd/init.c
+++ b/drivers/iommu/amd/init.c
@@ -2404,9 +2404,11 @@ static int iommu_setup_msi(struct amd_iommu *iommu)
 	if (r)
 		return r;
 
+	irq_set_status_flags(iommu->dev->irq, IRQ_IS_POLLED);
 	r = request_threaded_irq(iommu->dev->irq, NULL, amd_iommu_int_thread,
 				 IRQF_ONESHOT, "AMD-Vi", iommu);
 	if (r) {
+		irq_clear_status_flags(iommu->dev->irq, IRQ_IS_POLLED);
 		pci_disable_msi(iommu->dev);
 		return r;
 	}
@@ -2579,9 +2581,11 @@ static int __iommu_setup_intcapxt(struct amd_iommu *iommu, const char *devname,
 		return irq;
 	}
 
+	irq_set_status_flags(irq, IRQ_IS_POLLED);
 	ret = request_threaded_irq(irq, NULL, thread_fn, IRQF_ONESHOT, devname,
 				   iommu);
 	if (ret) {
+		irq_clear_status_flags(irq, IRQ_IS_POLLED);
 		irq_domain_free_irqs(irq, 1);
 		irq_domain_remove(domain);
 		return ret;
diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
index 960e1dc05ec6..ec392da008f9 100644
--- a/drivers/iommu/amd/iommu.c
+++ b/drivers/iommu/amd/iommu.c
@@ -1112,9 +1112,6 @@ static void amd_iommu_handle_irq(void *data, const char *evt_type,
 	u32 mask = int_mask | overflow_mask;
 
 	while (status & mask) {
-		/* Enable interrupt sources again */
-		writel(mask, iommu->mmio_base + MMIO_STATUS_OFFSET);
-
 		if (int_handler) {
 			pr_devel("Processing IOMMU (ivhd%d) %s Log\n",
 				 iommu->index, evt_type);
@@ -1124,6 +1121,16 @@ static void amd_iommu_handle_irq(void *data, const char *evt_type,
 		if ((status & overflow_mask) && overflow_handler)
 			overflow_handler(iommu);
 
+		/*
+		 * Clear only the status bits observed above, then poll once
+		 * more so entries written before the clear took effect are not
+		 * left unprocessed with the interrupt status bit cleared.
+		 */
+		writel(status & mask, iommu->mmio_base + MMIO_STATUS_OFFSET);
+
+		if (int_handler)
+			int_handler(iommu);
+
 		/*
 		 * Hardware bug: ERBT1312
 		 * When re-enabling interrupt (by writing 1

base-commit: 4dd16e31135ea8437a720222df34e8745f385d6d
-- 
2.56.0.385.gac671c819a-goog