mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* [PATCH] KVM: x86: Start IRQ bypass with local interrupts enabled
@ 2026-11-16 20:38 Satish Kharat
  2026-11-16 21:26 ` Sean Christopherson
  0 siblings, 1 reply; 2+ messages in thread
From: Satish Kharat @ 2026-11-16 20:38 UTC (permalink / raw)
  To: seanjc, pbonzini
  Cc: kvm, x86, linux-kernel, tglx, mingo, bp, dave.hansen, hpa,
	stable, Satish Kharat

Move the first IRQ-bypass notification after releasing irqfds.lock so
that the vendor callback runs with local interrupts enabled.

On Intel, pi_start_bypass() issues KVM_REQ_UNBLOCK to the vCPUs through
kvm_make_all_cpus_request(). If another CPU needs to be kicked, this can
reach smp_call_function_many(), which requires local interrupts enabled.
Calling the hook inside spin_lock_irq() violates that requirement and
triggered the following warning during VFIO MSI-X setup:

  WARNING: kernel/smp.c:943 at smp_call_function_many_cond
    kvm_make_all_cpus_request [kvm]
    kvm_arch_irq_bypass_add_producer [kvm]
    __connect [irqbypass]
    irq_bypass_register_producer [irqbypass]
    vfio_msi_set_vector_signal [vfio_pci_core]

The SMP API documents a possible deadlock in this context, including for
non-waiting calls. No deadlock was observed in this case.

Keep producer publication, bypass accounting and IRTE error rollback under
the existing lock. The IRQ bypass mutex still serializes producer
callbacks after irqfds.lock is released. Notify only for a successful
first attachment, after any IRTE failure has been ruled out.

An LLM assisted with implementation, code review and test automation.

Fixes: 77e1b8332d1d ("KVM: x86: Decouple device assignment from IRQ bypass")
Cc: stable@vger.kernel.org
Signed-off-by: Satish Kharat <satishkh@cisco.com>
---
Based on kvm-x86/next at cbe54fc3462b3949ab27b9ae81fb9ddb509040c7.

Validation:
- Build-tested irq.o against kvm-x86/next with W=1 and -Werror.
- Built the KVM core, Intel and AMD modules for a 7.4-rc1 kernel
  and used those modules for runtime testing.
- On Intel VT-d hardware, started a guest with an ENIC VF passed through
  using VFIO and reloaded its driver with device-posted interrupts and
  APICv enabled. Function-graph tracing captured pi_start_bypass() reaching
  the real SMP call with IRQs enabled. Posted-IRTE updates and bidirectional
  guest traffic passed.
- On Intel, selected KVM selftests passed: irqfd_test; irq_test in xAPIC and x2APIC
  routing modes; vmx_apicv_updates_test; nested_close_kvm_test; and
  xapic_ipi_test.
- On Intel, selected kvm-unit-tests passed: x2apic, xapic, ioapic, smptest and
  smptest3. The xapic run had one expected internal skip for absent
  x2APIC support in that test's CPU model.

On a Cisco C225 M8 (AMD EPYC 9555P) with AVIC/x2AVIC and
device-posted interrupts enabled, selected KVM selftests and
kvm-unit-tests passed; the VMX-only test skipped.

Physical NIC passthrough passed bidirectional traffic, link cycling,
driver reload and guest shutdown with two guest kernels. Tracing
confirmed balanced IRQ-bypass registration and successful AMD
posted-interrupt setup.

AMD has no pi_start_bypass() hook, so these are regression tests rather
than a reproduction of the Intel warning.

One guest logged an informational clocksource-check timeout, without
a BUG/WARN or unstable-clock report. Host lockdep remained enabled.
NetworkManager route-metric and profile-timestamp changes were
restored and verified separately after the tests.

On AMD EPYC 7343 with AVIC disabled, assigned ENIC VF traffic and
guest-driver reload passed using VIC forwarding between the VF
and its parent PF.

External-wire traffic was not tested on AMD.

The complete default test suites and forced IRTE-error runtime test were
not run. The public-base build check compiled the changed object; it was
not a full public-base kernel runtime test.

The IRQ-disabled warning in smp_call_function_many_cond() was observed
on multiple occasions during VFIO MSI-X setup. No deadlock was observed.
Additional diagnostic logs are available on request.
---
 arch/x86/kvm/irq.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kvm/irq.c b/arch/x86/kvm/irq.c
index 7f8b092b4479..611b3e57b09a 100644
--- a/arch/x86/kvm/irq.c
+++ b/arch/x86/kvm/irq.c
@@ -485,13 +485,13 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
 	struct kvm_kernel_irqfd *irqfd =
 		container_of(cons, struct kvm_kernel_irqfd, consumer);
 	struct kvm *kvm = irqfd->kvm;
+	bool start_bypass;
 	int ret = 0;
 
 	spin_lock_irq(&kvm->irqfds.lock);
 	irqfd->producer = prod;
 
-	if (!kvm->arch.nr_possible_bypass_irqs++)
-		kvm_x86_call(pi_start_bypass)(kvm);
+	start_bypass = !kvm->arch.nr_possible_bypass_irqs++;
 
 	if (irqfd->irq_entry.type == KVM_IRQ_ROUTING_MSI) {
 		ret = kvm_pi_update_irte(irqfd, &irqfd->irq_entry);
@@ -502,6 +502,13 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
 	}
 	spin_unlock_irq(&kvm->irqfds.lock);
 
+	/*
+	 * Wake vCPUs with IRQs enabled. The IRQ bypass mutex keeps producer
+	 * callbacks serialized after dropping irqfds.lock.
+	 */
+	if (!ret && start_bypass)
+		kvm_x86_call(pi_start_bypass)(kvm);
+
 	return ret;
 }
 

---
base-commit: cbe54fc3462b3949ab27b9ae81fb9ddb509040c7
change-id: 20261116-b4-kvm-irq-bypass-irqs-20261116-f9da0e6d8eb6

Best regards,
--  
Satish Kharat <satishkh@cisco.com>


^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2026-11-16 21:26 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-16 20:38 [PATCH] KVM: x86: Start IRQ bypass with local interrupts enabled Satish Kharat
2026-11-16 21:26 ` Sean Christopherson

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®