mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Paolo Bonzini <pbonzini@redhat.com>
To: Ankit Soni <Ankit.Soni@amd.com>,
	Sean Christopherson <seanjc@google.com>,
	Marc Zyngier <maz@kernel.org>
Cc: Oliver Upton <oliver.upton@linux.dev>,
	Joerg Roedel <joro@8bytes.org>,
	David Woodhouse <dwmw2@infradead.org>,
	Lu Baolu <baolu.lu@linux.intel.com>,
	linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev,
	kvm@vger.kernel.org, iommu@lists.linux.dev,
	linux-kernel@vger.kernel.org, Sairaj Kodilkar <sarunkod@amd.com>,
	Vasant Hegde <vasant.hegde@amd.com>,
	Maxim Levitsky <mlevitsk@redhat.com>,
	Joao Martins <joao.m.martins@oracle.com>,
	Francesco Lavra <francescolavra.fl@gmail.com>,
	David Matlack <dmatlack@google.com>,
	Naveen Rao <Naveen.Rao@amd.com>
Subject: possible deadlock due to irq_set_thread_affinity() calling into the scheduler (was Re: [PATCH v3 38/62] KVM: SVM: Take and hold ir_list_lock across IRTE updates in IOMMU)
Date: Mon, 22 Dec 2025 15:09:13 +0100	[thread overview]
Message-ID: <42513cb3-3c2e-4aa8-b748-23b6656a5096@redhat.com> (raw)
In-Reply-To: <njhjud3e6wbdftzr3ziyuh5bhyvc5ndt5qvmg7rlvh5isoop2l@f2uxctws2c7d>

On 12/22/25 10:16, Ankit Soni wrote:
>    ======================================================
>    WARNING: possible circular locking dependency detected
>    6.19.0-rc2 #20 Tainted: G            E
>    ------------------------------------------------------
>    CPU 58/KVM/28597 is trying to acquire lock:
>      ff12c47d4b1f34c0 (&irq_desc_lock_class){-.-.}-{2:2}, at: __irq_get_desc_lock+0x58/0xa0
> 
>      but task is already holding lock:
>      ff12c49b28552110 (&svm->ir_list_lock){....}-{2:2}, at: avic_pi_update_irte+0x147/0x270 [kvm_amd]
> 
>      which lock already depends on the new lock.
> 
>    Chain exists of:
>      &irq_desc_lock_class --> &rq->__lock --> &svm->ir_list_lock
> 
>    Possible unsafe locking scenario:
> 
>          CPU0                            CPU1
>          ----                            ----
>     lock(&svm->ir_list_lock);
>                                        lock(&rq->__lock);
>                                        lock(&svm->ir_list_lock);
>     lock(&irq_desc_lock_class);
> 
>          *** DEADLOCK ***
> 
> So lockdep sees:
> 
>    &irq_desc_lock_class -> &rq->__lock -> &svm->ir_list_lock
> 
> while avic_pi_update_irte() currently holds svm->ir_list_lock and then
> takes irq_desc_lock via irq_set_vcpu_affinity(), which creates the
> potential inversion.
> 
>    - Is this lockdep warning expected/benign in this code path, or does it
>      indicate a real potential deadlock between svm->ir_list_lock and
>      irq_desc_lock with AVIC + irq_bypass + VFIO?

I'd treat it as a potential (if unlikely) deadlock:

(a) irq_set_thread_affinity triggers the scheduler via wake_up_process,
while irq_desc->lock is taken

(b) the scheduler calls into KVM with rq_lock taken, and KVM uses
ir_list_lock within __avic_vcpu_load/__avic_vcpu_put

(c) KVM wants to block scheduling for a while and uses ir_list_lock for
that purpose, but then takes irq_set_vcpu_affinity takes irq_desc->lock.

I don't think there's an alternative choice of lock for (c); and there's
no easy way to pull the irq_desc->lock out of the IRQ subsystem--in fact
the stickiness of the situation comes from rq->rq_lock and
irq_desc->lock being both internal and not leaf.

Of the three, the most sketchy is (a); notably, __setup_irq() calls
wake_up_process outside desc->lock.  Therefore I'd like so much to treat
it as a kernel/irq/ bug; and the simplest (perhaps too simple...) fix is
to drop the wake_up_process().  The only cost is extra latency on the
next interrupt after an affinity change.

diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
index 8b1b4c8a4f54..fc135bd079a4 100644
--- a/kernel/irq/manage.c
+++ b/kernel/irq/manage.c
@@ -189,14 +189,10 @@ static void irq_set_thread_affinity(struct irq_desc *desc)
  	struct irqaction *action;
  
  	for_each_action_of_desc(desc, action) {
-		if (action->thread) {
+		if (action->thread)
  			set_bit(IRQTF_AFFINITY, &action->thread_flags);
-			wake_up_process(action->thread);
-		}
-		if (action->secondary && action->secondary->thread) {
+		if (action->secondary && action->secondary->thread)
  			set_bit(IRQTF_AFFINITY, &action->secondary->thread_flags);
-			wake_up_process(action->secondary->thread);
-		}
  	}
  }
  
Marc, what do you think?

Paolo


  reply	other threads:[~2025-12-22 14:09 UTC|newest]

Thread overview: 113+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-06-11 22:45 [PATCH v3 00/62] KVM: iommu: Overhaul device posted IRQs support Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 01/62] KVM: arm64: Explicitly treat routing entry type changes as changes Sean Christopherson
2025-06-13 19:43   ` Oliver Upton
2025-06-19 12:36   ` (subset) " Marc Zyngier
2025-06-11 22:45 ` [PATCH v3 02/62] KVM: arm64: WARN if unmapping vLPI fails Sean Christopherson
2025-06-12 11:59   ` Marc Zyngier
2025-06-12 14:34     ` Sean Christopherson
2025-06-13 20:47       ` Oliver Upton
2025-06-20 17:22         ` Sean Christopherson
2025-06-20 18:00           ` David Woodhouse
2025-06-20 18:48           ` Oliver Upton
2025-06-20 19:04             ` Sean Christopherson
2025-06-20 19:27               ` Oliver Upton
2025-06-20 20:31                 ` Sean Christopherson
2025-06-20 20:45                   ` Oliver Upton
2025-06-11 22:45 ` [PATCH v3 03/62] KVM: Pass new routing entries and irqfd when updating IRTEs Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 04/62] KVM: SVM: Track per-vCPU IRTEs using kvm_kernel_irqfd structure Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 05/62] KVM: SVM: Delete IRTE link from previous vCPU before setting new IRTE Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 06/62] iommu/amd: KVM: SVM: Delete now-unused cached/previous GA tag fields Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 07/62] KVM: SVM: Delete IRTE link from previous vCPU irrespective of new routing Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 08/62] KVM: SVM: Drop pointless masking of default APIC base when setting V_APIC_BAR Sean Christopherson
2025-06-13 14:15   ` Naveen N Rao
2025-06-11 22:45 ` [PATCH v3 09/62] KVM: SVM: Drop pointless masking of kernel page pa's with AVIC HPA masks Sean Christopherson
2025-06-13 14:37   ` Naveen N Rao
2025-06-11 22:45 ` [PATCH v3 10/62] KVM: SVM: Add helper to deduplicate code for getting AVIC backing page Sean Christopherson
2025-06-13 14:38   ` Naveen N Rao
2025-06-11 22:45 ` [PATCH v3 11/62] KVM: SVM: Drop vcpu_svm's pointless avic_backing_page field Sean Christopherson
2025-06-13 14:44   ` Naveen N Rao
2025-06-11 22:45 ` [PATCH v3 12/62] KVM: SVM: Inhibit AVIC if ID is too big instead of rejecting vCPU creation Sean Christopherson
2025-06-17 14:25   ` Naveen N Rao
2025-06-17 16:10     ` Sean Christopherson
2025-06-18 14:33       ` Naveen N Rao
2025-06-18 20:59         ` Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 13/62] KVM: SVM: Drop redundant check in AVIC code on ID during " Sean Christopherson
2025-06-17 14:49   ` Naveen N Rao
2025-06-17 16:33     ` Sean Christopherson
2025-06-18 14:39       ` Naveen N Rao
2025-06-11 22:45 ` [PATCH v3 14/62] KVM: SVM: Track AVIC tables as natively sized pointers, not "struct pages" Sean Christopherson
2025-06-17 15:01   ` Naveen N Rao
2025-06-11 22:45 ` [PATCH v3 15/62] KVM: SVM: Drop superfluous "cache" of AVIC Physical ID entry pointer Sean Christopherson
2025-06-19 11:09   ` Naveen N Rao
2025-06-11 22:45 ` [PATCH v3 16/62] KVM: VMX: Move enable_ipiv knob to common x86 Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 17/62] KVM: SVM: Add enable_ipiv param, never set IsRunning if disabled Sean Christopherson
2025-06-19 11:31   ` Naveen N Rao
2025-06-19 12:01     ` Naveen N Rao
2025-06-20 14:39     ` Sean Christopherson
2025-06-23 10:45       ` Naveen N Rao
2025-06-11 22:45 ` [PATCH v3 18/62] KVM: SVM: Disable (x2)AVIC IPI virtualization if CPU has erratum #1235 Sean Christopherson
2025-06-23 14:05   ` Naveen N Rao
2025-06-23 15:30     ` Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 19/62] KVM: VMX: Suppress PI notifications whenever the vCPU is put Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 20/62] KVM: SVM: Add a comment to explain why avic_vcpu_blocking() ignores IRQ blocking Sean Christopherson
2025-06-23 15:54   ` Naveen N Rao
2025-06-23 16:18     ` Sean Christopherson
2025-06-25 15:28       ` Naveen N Rao
2025-06-11 22:45 ` [PATCH v3 21/62] iommu/amd: KVM: SVM: Use pi_desc_addr to derive ga_root_ptr Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 22/62] iommu/amd: KVM: SVM: Pass NULL @vcpu_info to indicate "not guest mode" Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 23/62] KVM: SVM: Stop walking list of routing table entries when updating IRTE Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 24/62] KVM: VMX: " Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 25/62] KVM: SVM: Extract SVM specific code out of get_pi_vcpu_info() Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 26/62] KVM: x86: Move IRQ routing/delivery APIs from x86.c => irq.c Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 27/62] KVM: x86: Nullify irqfd->producer after updating IRTEs Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 28/62] KVM: x86: Dedup AVIC vs. PI code for identifying target vCPU Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 29/62] KVM: x86: Move posted interrupt tracepoint to common code Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 30/62] KVM: SVM: Clean up return handling in avic_pi_update_irte() Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 31/62] iommu: KVM: Split "struct vcpu_data" into separate AMD vs. Intel structs Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 32/62] KVM: Don't WARN if updating IRQ bypass route fails Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 33/62] KVM: Fold kvm_arch_irqfd_route_changed() into kvm_arch_update_irqfd_routing() Sean Christopherson
2025-06-13 20:50   ` Oliver Upton
2025-06-11 22:45 ` [PATCH v3 34/62] KVM: x86: Track irq_bypass_vcpu in common x86 code Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 35/62] KVM: x86: Skip IOMMU IRTE updates if there's no old or new vCPU being targeted Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 36/62] KVM: x86: Don't update IRTE entries when old and new routes were !MSI Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 37/62] KVM: SVM: Revert IRTE to legacy mode if IOMMU doesn't provide IR metadata Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 38/62] KVM: SVM: Take and hold ir_list_lock across IRTE updates in IOMMU Sean Christopherson
2025-06-17 15:42   ` Naveen N Rao
2025-12-22  9:16   ` Ankit Soni
2025-12-22 14:09     ` Paolo Bonzini [this message]
2025-12-22 19:34       ` possible deadlock due to irq_set_thread_affinity() calling into the scheduler (was Re: [PATCH v3 38/62] KVM: SVM: Take and hold ir_list_lock across IRTE updates in IOMMU) Sean Christopherson
2025-12-22 21:15         ` Paolo Bonzini
2025-12-22 22:10           ` Sean Christopherson
2025-12-23  8:59       ` Ankit Soni
2026-01-08 21:28       ` Thomas Gleixner
2026-01-08 21:53         ` Thomas Gleixner
2026-01-21 15:53           ` Paolo Bonzini
2026-01-21 18:13           ` Paolo Bonzini
2026-01-22 10:19             ` Marc Zyngier
2026-01-22 18:47             ` Thomas Gleixner
2026-01-24  7:49               ` Paolo Bonzini
2025-06-11 22:45 ` [PATCH v3 39/62] iommu/amd: Document which IRTE fields amd_iommu_update_ga() can modify Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 40/62] iommu/amd: KVM: SVM: Infer IsRun from validity of pCPU destination Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 41/62] iommu/amd: Factor out helper for manipulating IRTE GA/CPU info Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 42/62] iommu/amd: KVM: SVM: Set pCPU info in IRTE when setting vCPU affinity Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 43/62] iommu/amd: KVM: SVM: Add IRTE metadata to affined vCPU's list if AVIC is inhibited Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 44/62] KVM: SVM: Don't check for assigned device(s) when updating affinity Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 45/62] KVM: SVM: Don't check for assigned device(s) when activating AVIC Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 46/62] KVM: SVM: WARN if (de)activating guest mode in IOMMU fails Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 47/62] KVM: SVM: Process all IRTEs on affinity change even if one update fails Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 48/62] KVM: SVM: WARN if updating IRTE GA fields in IOMMU fails Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 49/62] KVM: x86: Drop superfluous "has assigned device" check in kvm_pi_update_irte() Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 50/62] KVM: x86: WARN if IRQ bypass isn't supported " Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 51/62] KVM: x86: WARN if IRQ bypass routing is updated without in-kernel local APIC Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 52/62] KVM: SVM: WARN if ir_list is non-empty at vCPU free Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 53/62] KVM: x86: Decouple device assignment from IRQ bypass Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 54/62] KVM: VMX: WARN if VT-d Posted IRQs aren't possible when starting " Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 55/62] KVM: SVM: Use vcpu_idx, not vcpu_id, for GA log tag/metadata Sean Christopherson
2025-06-11 22:45 ` [PATCH v3 56/62] iommu/amd: WARN if KVM calls GA IRTE helpers without virtual APIC support Sean Christopherson
2025-06-11 22:46 ` [PATCH v3 57/62] KVM: SVM: Fold avic_set_pi_irte_mode() into its sole caller Sean Christopherson
2025-06-11 22:46 ` [PATCH v3 58/62] KVM: SVM: Don't check vCPU's blocking status when toggling AVIC on/off Sean Christopherson
2025-06-11 22:46 ` [PATCH v3 59/62] KVM: SVM: Consolidate IRTE update " Sean Christopherson
2025-06-11 22:46 ` [PATCH v3 60/62] iommu/amd: KVM: SVM: Allow KVM to control need for GA log interrupts Sean Christopherson
2025-06-11 22:46 ` [PATCH v3 61/62] KVM: SVM: Generate GA log IRQs only if the associated vCPUs is blocking Sean Christopherson
2025-06-11 22:46 ` [PATCH v3 62/62] KVM: x86: Rename kvm_set_msi_irq() => kvm_msi_to_lapic_irq() Sean Christopherson
2025-06-24 19:38 ` [PATCH v3 00/62] KVM: iommu: Overhaul device posted IRQs support Sean Christopherson

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=42513cb3-3c2e-4aa8-b748-23b6656a5096@redhat.com \
    --to=pbonzini@redhat.com \
    --cc=Ankit.Soni@amd.com \
    --cc=Naveen.Rao@amd.com \
    --cc=baolu.lu@linux.intel.com \
    --cc=dmatlack@google.com \
    --cc=dwmw2@infradead.org \
    --cc=francescolavra.fl@gmail.com \
    --cc=iommu@lists.linux.dev \
    --cc=joao.m.martins@oracle.com \
    --cc=joro@8bytes.org \
    --cc=kvm@vger.kernel.org \
    --cc=kvmarm@lists.linux.dev \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=maz@kernel.org \
    --cc=mlevitsk@redhat.com \
    --cc=oliver.upton@linux.dev \
    --cc=sarunkod@amd.com \
    --cc=seanjc@google.com \
    --cc=vasant.hegde@amd.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®