From: Ashok Raj <ashok.raj@oss.qualcomm.com>
To: Vladimir Murzin <vladimir.murzin@arm.com>
Cc: Linux ARM Kernel <linux-arm-kernel@lists.infradead.org>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
Catalin Marinas <catalin.marinas@arm.com>,
Will Deacon <will@kernel.org>,
Mark Rutland <mark.rutland@arm.com>,
Lorenzo Pieralisi <lpieralisi@kernel.org>,
Jinjie Ruan <ruanjinjie@huawei.com>,
Liaochang <liaochang1@huawei.com>, Marc Zyngier <maz@kernel.org>,
Radu Rendec <radu@rendec.net>,
Jonathan Cameron <jonathan.cameron@oss.qualcomm.com>,
Sathvika Vasireddy <sathvika.vasireddy@oss.qualcomm.com>,
Ashok Raj <ashok.raj@oss.qualcomm.com>,
Ashok Raj <ashoraj@qti.qualcomm.com>
Subject: Re: [RFC PATCH v1 0/2] genirq: add migratable NMI registration and show NMI counts
Date: Mon, 16 Nov 2026 10:31:31 -0800 [thread overview]
Message-ID: <avtMA9bb9vwezquP@hu-ashoraj-lv.qualcomm.com> (raw)
In-Reply-To: <452bf82e-52ea-47c1-9074-eeb614a33747@arm.com>
Hi Vladimir
On Mon, Nov 16, 2026 at 10:38:00AM +0000, Vladimir Murzin wrote:
> Hi Ashok,
>
> On 11/14/26 23:49, Ashok Raj wrote:
> > request_nmi() currently models NMIs as per-CPU sources. This does not
> > represent a globally routed NMI source with an affinity-capable irqchip,
> > which needs to be rerouted when its target CPU is offlined.
> >
> > This RFC adds IRQF_NMI_MIGRATABLE as a request_nmi() registration mode
> > for such a source. It remains excluded from normal IRQ balancing, but the
> > generic startup and CPU-hotplug paths may select an online target. The
> > second patch fixes the related /proc/interrupts display: NMI-safe flow
> > handling updates the per-CPU counters rather than irq_desc::tot_count.
> >
> > Patch 1: Add the migratable-NMI registration mode, validate that it is
> > used only through request_nmi() with an affinity-capable non-per-CPU
> > irqchip, and add IRQ KUnit coverage for registration and CPU-offline
> > migration.
> >
> > Patch 2: Emit the per-CPU counters for all NMI descriptors, matching
> > kstat_irqs_desc().
> >
> > Testing:
> > - ARM64 changed-object builds with WERROR=1
> > - ARM64 !SMP changed-object build
> > - ARM64 IRQ KUnit run under QEMU with two CPUs, including registration
> > validation and CPU-offline migration
> > - local downstream GHES delivery and migration validation for the intended
> > consumer topology (not part of this series)
> >
> > Base and dependencies:
> >
> > This RFC was developed and tested on v7.4-rc2 plus the FEAT_NMI v2 series
> > [1]. It depends on that series' GIC NMI support and is therefore not directly
> > applicable to mainline or linux-next as posted. This RFC is for design
> > feedback; it will need rebasing onto the upstream form of those prerequisites
> > before submission.
> >
> > [1] https://lore.kernel.org/linux-arm-kernel/20261112132042.129062-1-vladimir.murzin@arm.com/
> >
>
> I'm not an expert in this area, but based on what I've learned so far,
> NMI being per-CPU like makes life much easier ;)
I'm still learning the ARM side :-), so please correct me if I've got
something wrong.
The events I'm working on are platform RAS events described via:
HEST -> GHES entries.
My prototype uses an extended Type-4 GHES entry (Type-4 is
just for the RFC discussion here; I also have an alternative based on Marc's
earlier suggestion).
These are SPIs, so they can be routed to any CPU. But request_nmi() as
structured picks one CPU and pins the interrupt to it, treating it as
per-CPU. If that CPU goes offline, the interrupt is orphaned.
I see two ways to handle this:
Option 1: Retarget on CPU offline. My POC registers a hotplug callback that
picks a new target CPU. If none is available (e.g. this is the last online
CPU), it vetoes the offline. This works, but it feels like a band-aid.
Option 2: Extend the API for SPIs. Add a flag that lets the caller opt out
of the per-CPU requirement. Affinity migration and retargeting on offline
then follow the normal SPI path, which seems more natural to me.
I prefer Option 2. If it fits your series, I'll post the POC as an RFC for
feedback.
> IIUC, one of the main problems with a shared NMI (as opposed to a
> per-CPU) is handling shared state. [...] the same NMI handler could
> execute concurrently on two CPUs.
>
> IIUC, one of the main problems with a shared NMI (as opposed to a
> per-CPU) is handling shared state. Consider:
>
> CPU0 CPU1 CPU2
>
> ---------------------------------------------------------------------------
>
> <NMI>
> handling started
> set NMI affinity to CPU2
> <NMI>
> handling started
> mutate shared state
> mutate shared state
> handling finished
> handling finished
>
> IOW, the same NMI handler could execute concurrently on two CPUs,
> potentially corrupting shared state. Once again more than happy to
> be corrected if I missed something :)
There is no risk of the same normal, non-per-CPU IRQ action handler
executing concurrently on both the previous and newly selected CPUs during
an affinity change. Genirq sets IRQD_IRQ_INPROGRESS before dropping the
descriptor lock to invoke the driver's hard-IRQ handler. If another instance
arrives on the new target while the handler remains active on the previous
target, the new CPU observes that in-progress state and does not enter the
action handler.
The affinity update is not a synchronous “wait for the old CPU” operation:
the interrupt controller can route a subsequent interrupt to the new CPU
before the old handler returns. However, handler execution remains
serialized:
Previous CPU New target CPU
------------ --------------
set IRQD_IRQ_INPROGRESS
drop desc->lock
run driver's hard-IRQ handler
affinity target is reprogrammed
a subsequent IRQ arrives
observes IRQD_IRQ_INPROGRESS
does not run the action handler
handler returns
reacquire desc->lock
clear IRQD_IRQ_INPROGRESS
a later or replayed IRQ may run the
action handler on the new CPU
Any required pending/replay behavior for an event arriving during this
interval is handled by the applicable IRQ flow handler and irqchip semantics;
it does not cause simultaneous invocations of the driver's hard-IRQ handler.
Does this make sense?
Cheers,
Ashok
prev parent reply other threads:[~2026-11-16 18:31 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-14 23:49 Ashok Raj
2026-11-14 23:49 ` [RFC PATCH v1 1/2] genirq: Add migratable NMI registration mode Ashok Raj
2026-11-14 23:49 ` [RFC PATCH v1 2/2] genirq/proc: Show NMI interrupt counts in /proc/interrupts Ashok Raj
2026-11-16 10:38 ` [RFC PATCH v1 0/2] genirq: add migratable NMI registration and show NMI counts Vladimir Murzin
2026-11-16 18:31 ` Ashok Raj [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=avtMA9bb9vwezquP@hu-ashoraj-lv.qualcomm.com \
--to=ashok.raj@oss.qualcomm.com \
--cc=ashoraj@qti.qualcomm.com \
--cc=catalin.marinas@arm.com \
--cc=jonathan.cameron@oss.qualcomm.com \
--cc=liaochang1@huawei.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=lpieralisi@kernel.org \
--cc=mark.rutland@arm.com \
--cc=maz@kernel.org \
--cc=radu@rendec.net \
--cc=ruanjinjie@huawei.com \
--cc=sathvika.vasireddy@oss.qualcomm.com \
--cc=vladimir.murzin@arm.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®