mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Ashok Raj <ashok.raj@oss.qualcomm.com>
To: Vladimir Murzin <vladimir.murzin@arm.com>
Cc: Linux ARM Kernel <linux-arm-kernel@lists.infradead.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Will Deacon <will@kernel.org>,
	Mark Rutland <mark.rutland@arm.com>,
	Lorenzo Pieralisi <lpieralisi@kernel.org>,
	Jinjie Ruan <ruanjinjie@huawei.com>,
	Liaochang <liaochang1@huawei.com>, Marc Zyngier <maz@kernel.org>,
	Radu Rendec <radu@rendec.net>,
	Jonathan Cameron <jonathan.cameron@oss.qualcomm.com>,
	Sathvika Vasireddy <sathvika.vasireddy@oss.qualcomm.com>,
	Ashok Raj <ashok.raj@oss.qualcomm.com>,
	Ashok Raj <ashoraj@qti.qualcomm.com>
Subject: Re: [RFC PATCH v1 0/2] genirq: add migratable NMI registration and show NMI counts
Date: Mon, 16 Nov 2026 10:31:31 -0800	[thread overview]
Message-ID: <avtMA9bb9vwezquP@hu-ashoraj-lv.qualcomm.com> (raw)
In-Reply-To: <452bf82e-52ea-47c1-9074-eeb614a33747@arm.com>

Hi Vladimir

On Mon, Nov 16, 2026 at 10:38:00AM +0000, Vladimir Murzin wrote:

> Hi Ashok,
> 
> On 11/14/26 23:49, Ashok Raj wrote:
> > request_nmi() currently models NMIs as per-CPU sources. This does not
> > represent a globally routed NMI source with an affinity-capable irqchip,
> > which needs to be rerouted when its target CPU is offlined.
> > 
> > This RFC adds IRQF_NMI_MIGRATABLE as a request_nmi() registration mode
> > for such a source. It remains excluded from normal IRQ balancing, but the
> > generic startup and CPU-hotplug paths may select an online target. The
> > second patch fixes the related /proc/interrupts display: NMI-safe flow
> > handling updates the per-CPU counters rather than irq_desc::tot_count.
> > 
> > Patch 1: Add the migratable-NMI registration mode, validate that it is
> > used only through request_nmi() with an affinity-capable non-per-CPU
> > irqchip, and add IRQ KUnit coverage for registration and CPU-offline
> > migration.
> > 
> > Patch 2: Emit the per-CPU counters for all NMI descriptors, matching
> > kstat_irqs_desc().
> > 
> > Testing:
> >   - ARM64 changed-object builds with WERROR=1
> >   - ARM64 !SMP changed-object build
> >   - ARM64 IRQ KUnit run under QEMU with two CPUs, including registration
> >     validation and CPU-offline migration
> >   - local downstream GHES delivery and migration validation for the intended
> >     consumer topology (not part of this series)
> > 
> > Base and dependencies:
> > 
> > This RFC was developed and tested on v7.4-rc2 plus the FEAT_NMI v2 series
> > [1]. It depends on that series' GIC NMI support and is therefore not directly
> > applicable to mainline or linux-next as posted. This RFC is for design
> > feedback; it will need rebasing onto the upstream form of those prerequisites
> > before submission.
> > 
> > [1] https://lore.kernel.org/linux-arm-kernel/20261112132042.129062-1-vladimir.murzin@arm.com/
> > 
> 
> I'm not an expert in this area, but based on what I've learned so far,
> NMI being per-CPU like makes life much easier ;)

I'm still learning the ARM side :-), so please correct me if I've got 
something wrong.

The events I'm working on are platform RAS events described via:

	HEST -> GHES entries. 

My prototype uses an extended Type-4 GHES entry (Type-4 is
just for the RFC discussion here; I also have an alternative based on Marc's
earlier suggestion).

These are SPIs, so they can be routed to any CPU. But request_nmi() as
structured picks one CPU and pins the interrupt to it, treating it as
per-CPU. If that CPU goes offline, the interrupt is orphaned.

I see two ways to handle this:

Option 1: Retarget on CPU offline. My POC registers a hotplug callback that
picks a new target CPU. If none is available (e.g. this is the last online
CPU), it vetoes the offline. This works, but it feels like a band-aid.

Option 2: Extend the API for SPIs. Add a flag that lets the caller opt out
of the per-CPU requirement. Affinity migration and retargeting on offline
then follow the normal SPI path, which seems more natural to me.

I prefer Option 2. If it fits your series, I'll post the POC as an RFC for
feedback.

> IIUC, one of the main problems with a shared NMI (as opposed to a
> per-CPU) is handling shared state. [...] the same NMI handler could
> execute concurrently on two CPUs.
> 
> IIUC, one of the main problems with a shared NMI (as opposed to a
> per-CPU) is handling shared state. Consider:
> 
> CPU0                       CPU1                       CPU2
> 
> ---------------------------------------------------------------------------
> 
> <NMI>
> handling started
>                            set NMI affinity to CPU2
>                                                       <NMI>
>                                                       handling started
> mutate shared state
>                                                       mutate shared state
> handling finished
>                                                       handling finished
> 
> IOW, the same NMI handler could execute concurrently on two CPUs,
> potentially corrupting shared state. Once again more than happy to
> be corrected if I missed something :)

There is no risk of the same normal, non-per-CPU IRQ action handler
executing concurrently on both the previous and newly selected CPUs during
an affinity change. Genirq sets IRQD_IRQ_INPROGRESS before dropping the
descriptor lock to invoke the driver's hard-IRQ handler. If another instance
arrives on the new target while the handler remains active on the previous
target, the new CPU observes that in-progress state and does not enter the
action handler.

The affinity update is not a synchronous “wait for the old CPU” operation:
the interrupt controller can route a subsequent interrupt to the new CPU
before the old handler returns. However, handler execution remains
serialized:

Previous CPU                         New target CPU
------------                         --------------
set IRQD_IRQ_INPROGRESS
drop desc->lock
run driver's hard-IRQ handler

                                     affinity target is reprogrammed
                                     a subsequent IRQ arrives
                                     observes IRQD_IRQ_INPROGRESS
                                     does not run the action handler

handler returns
reacquire desc->lock
clear IRQD_IRQ_INPROGRESS

                                     a later or replayed IRQ may run the
                                     action handler on the new CPU

Any required pending/replay behavior for an event arriving during this
interval is handled by the applicable IRQ flow handler and irqchip semantics;
it does not cause simultaneous invocations of the driver's hard-IRQ handler.

Does this make sense? 

Cheers,
Ashok

      reply	other threads:[~2026-11-16 18:31 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-14 23:49 Ashok Raj
2026-11-14 23:49 ` [RFC PATCH v1 1/2] genirq: Add migratable NMI registration mode Ashok Raj
2026-11-14 23:49 ` [RFC PATCH v1 2/2] genirq/proc: Show NMI interrupt counts in /proc/interrupts Ashok Raj
2026-11-16 10:38 ` [RFC PATCH v1 0/2] genirq: add migratable NMI registration and show NMI counts Vladimir Murzin
2026-11-16 18:31   ` Ashok Raj [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=avtMA9bb9vwezquP@hu-ashoraj-lv.qualcomm.com \
    --to=ashok.raj@oss.qualcomm.com \
    --cc=ashoraj@qti.qualcomm.com \
    --cc=catalin.marinas@arm.com \
    --cc=jonathan.cameron@oss.qualcomm.com \
    --cc=liaochang1@huawei.com \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lpieralisi@kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=maz@kernel.org \
    --cc=radu@rendec.net \
    --cc=ruanjinjie@huawei.com \
    --cc=sathvika.vasireddy@oss.qualcomm.com \
    --cc=vladimir.murzin@arm.com \
    --cc=will@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®