From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from fhigh-b7-smtp.messagingengine.com (fhigh-b7-smtp.messagingengine.com [202.12.124.158]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 51AD738654F for ; Wed, 17 Jun 2026 19:20:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=202.12.124.158 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781724022; cv=none; b=U/V3MNUfc9NVtOrKV/GfP5MsY2vi7cqxep5IJ67a0g8ZKrsOcEfshMCkQBDU1Psz8MPK0O2vDU/aBBiAgHdsMPevNnzqy7R5A3LM5Dzv2fe28YMc1iv8LGRRB/7l1+yteCTYQpeh9Kij6D8jGporau16vQNoGwMlsVUE5rqWyXQ= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781724022; c=relaxed/simple; bh=dyamoD+Fl1wVERjcxG+YypUK9GMYBgPNH9bBcJkALak=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=mlf+k12OnwyP9f1lXHAZ96MQEqdJ6gEkzDhd8NjLxdfnYlwZo2B7Lzym9frSFyYmEufJgzRunyD1XEjlhSoAgNrxYEZVFCFF+ARlH6sZPuEZ7uM/t3tFIbfcl1CnwtoXI5t9ZZl5cCn7t6fNJ/lfy12i4JJh35So9m/yhKr06ig= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=shutemov.name; spf=pass smtp.mailfrom=shutemov.name; dkim=pass (2048-bit key) header.d=shutemov.name header.i=@shutemov.name header.b=JMOrx2iX; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=Ee/mIF1X; arc=none smtp.client-ip=202.12.124.158 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=shutemov.name Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=shutemov.name Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=shutemov.name header.i=@shutemov.name header.b="JMOrx2iX"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="Ee/mIF1X" Received: from phl-compute-05.internal (phl-compute-05.internal [10.202.2.45]) by mailfhigh.stl.internal (Postfix) with ESMTP id 3A7E47A015B; Wed, 17 Jun 2026 15:20:20 -0400 (EDT) Received: from phl-frontend-03 ([10.202.2.162]) by phl-compute-05.internal (MEProxy); Wed, 17 Jun 2026 15:20:20 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=shutemov.name; h=cc:cc:content-transfer-encoding:content-type:content-type :date:date:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:subject:subject:to:to; s=fm2; t=1781724020; x=1781810420; bh=dkNuY4BEl6fecffLQxRP0bSeac7mm9+F WjLH7RhfLkk=; b=JMOrx2iXPcU2nxpg/Tcu+HX80LVYzOaG6GQ8HsCldwyfrhmW VNleG3cIXhXRn4xwVG8jUre51BIXXbIH5H5yL7znzmhoFWMm0zV3U8pP60JFTLJJ u+10zBmR55YIUq4a9ax17ts+eSNIKVp8DYJsCVdxit8FkFI77Z36r6N4W0/Vlp8y 1agLoTkBz2u6ICva4+9lgY+5LXlYx6Bvb6XLRPePm5UjsqvD6OCW0QUXQHd5viro ef5hNDoD9naF+6hydiJM2PrffXqr1Rz9uucZhkJfFrbXA52FToJJtWmYJa5lrvy+ tV8CM5402dkRRqG64vLm18VlfGTF8Q5Zr4bdvQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1781724020; x= 1781810420; bh=dkNuY4BEl6fecffLQxRP0bSeac7mm9+FWjLH7RhfLkk=; b=E e/mIF1XawAqy8VtByNb8gMurQ9FJd7k9RAQ/J3b5gtaCgBSzYLe/IZaPWxp7c1T7 1P3n84qbYfcuvzTQIf49PpYpsZ4OblRa3fKeuOwCucEBfkloSMFfopdMS1BI8hh5 LLCJXHBv3G5KcyNZUgLUMRrw5ML4qJb0UZjNhfHbHXplP6hlqpDBuSwwmYUvOSq1 aiL50mVVYk6Q3w+HABaGYGWiWT5tYYZGhteCwnTTi5z0CeLbi6FSw+flGVqfMh9R k5cbOX9acdYdUEnOKvs6GKTcS2R+dVmEpDC/j/Gr0g5NFOBHFtguYQyokTwscFJe ZlOlwi2MmRKIc7Pz2eT3w== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: dmFkZTGr4wEqCimFoE3RsPP1Z5greKYABweAKpg4RI45hKUFYl5vWQ7FXUaz8F/r22AqqZ omJPOaTXkHI3UZQ1hNTdCZ6m2EkUn1Dqujr/wUCpoJaDXpeTejApWWiqg3Q7uw2j7ac3Nc EkklX6fmmAmCu+ztlI9aoF3sbEDbTX9j5K3JaIt1NkLjU9JTRzcP/AliaULqrXk0beJJq7 kLOg6AY2jnCg+x4YTjy2FcmOeUguJPiBzAKb/SSa8uUvLo0J7aDdf9ww4h0IX7sZz1kcJf 2jbCfDomb5lOMpwZAEduxdAs4BawyKVsJ38lE87xkxpyZV2dHTnNzwEjdmRsX9/wYaHcZa nOWqSM77N4Qc65hQH1KHUk/HzyVyHxsDaWJ14ucii0823eUAUDHlkeCOLsTzlA7N2z4CWW rKaznVlrMxV2+xAfioZ/JPYUXtIBHkH6L39Xmi31DOb7UHvsALIRajLgiiRU0ZbzCjHss4 6eEjBcvdHZ7Tdv9/nydocZeV9uSYQZPFJmfiwim2pMjTtUU0zttMuU4pte3u7KarhCml7S DgZ/HbFAnmur+32PmcTDALVsuny4Vdzk1Tzdj6L36bjCQNb06JYaX+duUG0yO0XKqOB92l JANz3miXdvIY0hi5AM7LXiyM9zlqBMOa2tZvZMOJYe68cJ3KLPqbqKMOAXFA X-ME-Proxy: Feedback-ID: ie3994620:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 17 Jun 2026 15:20:19 -0400 (EDT) From: Kiryl Shutsemau To: Catalin Marinas , Will Deacon , James Morse Cc: Mark Rutland , Marc Zyngier , Doug Anderson , Petr Mladek , Thomas Gleixner , Andrew Morton , Baoquan He , Puranjay Mohan , Usama Arif , Breno Leitao , Julien Thierry , Lecopzer Chen , Sumit Garg , kernel-team@meta.com, kexec@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, "Kiryl Shutsemau (Meta)" Subject: [PATCH v4 3/4] drivers/firmware: add SDEI cross-CPU NMI service for arm64 Date: Wed, 17 Jun 2026 20:20:04 +0100 Message-ID: X-Mailer: git-send-email 2.54.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit From: "Kiryl Shutsemau (Meta)" Deliver an NMI-like event to an interrupt-masked arm64 CPU via the standard SDEI software-signalled event (event 0), without the pseudo-NMI hot-path cost: register a handler for event 0 and poke a target with sdei_event_signal(0, mpidr). First user is arch_trigger_cpumask_backtrace() (sysrq-l, RCU stalls, hung-task/soft-lockup dumps), which otherwise rides an IPI that can't reach a masked CPU. Falls back to the IPI path when SDEI is absent; no watchdog backend yet, so the stock detector is untouched. Signed-off-by: Kiryl Shutsemau (Meta) Reviewed-by: Douglas Anderson --- MAINTAINERS | 2 +- arch/arm64/include/asm/nmi.h | 24 +++++ arch/arm64/kernel/smp.c | 11 +++ drivers/firmware/Kconfig | 19 ++++ drivers/firmware/Makefile | 1 + drivers/firmware/arm_sdei_nmi.c | 152 ++++++++++++++++++++++++++++++++ 6 files changed, 208 insertions(+), 1 deletion(-) create mode 100644 arch/arm64/include/asm/nmi.h create mode 100644 drivers/firmware/arm_sdei_nmi.c diff --git a/MAINTAINERS b/MAINTAINERS index c8d4b913f26c..b5ddfb85dce9 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -24797,7 +24797,7 @@ M: James Morse L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) S: Maintained F: Documentation/devicetree/bindings/arm/firmware/sdei.txt -F: drivers/firmware/arm_sdei.c +F: drivers/firmware/arm_sdei* F: include/linux/arm_sdei.h F: include/uapi/linux/arm_sdei.h diff --git a/arch/arm64/include/asm/nmi.h b/arch/arm64/include/asm/nmi.h new file mode 100644 index 000000000000..9366be419d18 --- /dev/null +++ b/arch/arm64/include/asm/nmi.h @@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_NMI_H +#define __ASM_NMI_H + +#include + +/* + * Cross-CPU NMI provider hooks, consulted by the arm64 arch code before + * its regular-IRQ / pseudo-NMI IPI paths. The SDEI provider in + * drivers/firmware/arm_sdei_nmi.c implements them when active; a future + * FEAT_NMI provider could slot in here too. The stubs let callers stay + * unconditional when ARM_SDEI_NMI is off. + */ +#ifdef CONFIG_ARM_SDEI_NMI +bool sdei_nmi_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu); +#else +static inline bool sdei_nmi_trigger_cpumask_backtrace(const cpumask_t *mask, + int exclude_cpu) +{ + return false; +} +#endif + +#endif /* __ASM_NMI_H */ diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 1aa324104afb..a670434a8cae 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -45,6 +45,7 @@ #include #include #include +#include #include #include #include @@ -927,6 +928,16 @@ static void arm64_backtrace_ipi(cpumask_t *mask) void arch_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu) { + /* + * Prefer the SDEI cross-CPU NMI provider when active: firmware + * dispatches the event out of EL3 and reaches CPUs that have + * interrupts locally masked, without the per-IRQ-mask cost that + * pseudo-NMI pays for the same reach. The plain IPI path below + * can't reach such a CPU unless pseudo-NMI is enabled. + */ + if (sdei_nmi_trigger_cpumask_backtrace(mask, exclude_cpu)) + return; + /* * NOTE: though nmi_trigger_cpumask_backtrace() has "nmi_" in the name, * nothing about it truly needs to be implemented using an NMI, it's diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig index bbd2155d8483..6501087ff90d 100644 --- a/drivers/firmware/Kconfig +++ b/drivers/firmware/Kconfig @@ -36,6 +36,25 @@ config ARM_SDE_INTERFACE standard for registering callbacks from the platform firmware into the OS. This is typically used to implement RAS notifications. +config ARM_SDEI_NMI + bool "SDEI-based cross-CPU NMI service (arm64)" + depends on ARM64 && ARM_SDE_INTERFACE + help + Provides SDEI-based cross-CPU NMI delivery for hooks that need + to reach interrupt-masked CPUs on silicon that lacks FEAT_NMI: + + - arch_trigger_cpumask_backtrace() (sysrq-l, RCU stalls, + hardlockup_all_cpu_backtrace, soft-lockup secondary dumps, + hung-task auxiliary dumps) + + The driver registers a handler for the SDEI software-signalled + event (event 0) and reaches a target CPU by signalling it with + SDEI_EVENT_SIGNAL. Firmware delivers the event out of EL3 + regardless of the target's PSTATE.DAIF -- forced delivery into a + CPU wedged with interrupts locally masked. + + If unsure, say N. + config EDD tristate "BIOS Enhanced Disk Drive calls determine boot disk" depends on X86 diff --git a/drivers/firmware/Makefile b/drivers/firmware/Makefile index 4ddec2820c96..be46f1e1dc77 100644 --- a/drivers/firmware/Makefile +++ b/drivers/firmware/Makefile @@ -4,6 +4,7 @@ # obj-$(CONFIG_ARM_SCPI_PROTOCOL) += arm_scpi.o obj-$(CONFIG_ARM_SDE_INTERFACE) += arm_sdei.o +obj-$(CONFIG_ARM_SDEI_NMI) += arm_sdei_nmi.o obj-$(CONFIG_DMI) += dmi_scan.o obj-$(CONFIG_DMI_SYSFS) += dmi-sysfs.o obj-$(CONFIG_EDD) += edd.o diff --git a/drivers/firmware/arm_sdei_nmi.c b/drivers/firmware/arm_sdei_nmi.c new file mode 100644 index 000000000000..2a78cee203fb --- /dev/null +++ b/drivers/firmware/arm_sdei_nmi.c @@ -0,0 +1,152 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * arm64 SDEI-based cross-CPU NMI service. + * + * Delivering an "NMI-shaped" event to an EL1 context that has locally + * masked interrupts, on silicon without FEAT_NMI, can be done two ways: + * + * - pseudo-NMI: mask "interrupts" via the GIC priority register + * (ICC_PMR_EL1) instead of PSTATE.DAIF, leaving a high-priority band + * deliverable. Functionally this works -- but it reimplements every + * local_irq_disable()/enable() and exception entry/exit as a PMR + * write plus synchronisation, a cost paid on that hot path forever, + * whether or not an NMI is ever delivered. + * + * - SDEI: leave interrupt masking as the cheap PSTATE.DAIF operation + * and have the firmware bounce an EL3-routed Group-0 SGI back to + * NS-EL1 as an event callback. The cost is a firmware round-trip, + * but only at the rare moment delivery is actually needed. + * + * This driver takes the second path: it keeps the IRQ-mask hot path + * free and pays only when it fires, which is what makes cross-CPU NMI + * affordable on hardware where the pseudo-NMI tax isn't, until FEAT_NMI + * makes NMI masking cheap in the architecture itself. + * + * Capabilities provided: + * + * - sdei_nmi_trigger_cpumask_backtrace() — override for arm64's + * arch_trigger_cpumask_backtrace(), so sysrq-l, RCU stall dumps, + * hardlockup_all_cpu_backtrace, soft-lockup/hung-task secondary + * dumps all reach interrupt-masked CPUs. + * + * Delivery uses the standard SDEI software-signalled event (event 0) and + * SDEI_EVENT_SIGNAL. We register a handler for event 0, enable it, and + * poke a target CPU with sdei_event_signal(0, mpidr): firmware makes + * event 0 pending on that PE and dispatches the handler NMI-like, + * regardless of the target's DAIF. + * Availability is simply whether event 0 registers and enables -- if SDEI + * and its software-signalled event are present we use it, otherwise the + * driver stays inert. + */ + +#define pr_fmt(fmt) "sdei_nmi: " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +static bool sdei_nmi_available; + +#define SDEI_NMI_EVENT 0 + +static int sdei_nmi_handler(u32 event, struct pt_regs *regs, void *arg) +{ + /* + * nmi_cpu_backtrace() no-ops unless this CPU's bit is set in the + * global backtrace mask (driven by nmi_trigger_cpumask_backtrace()), + * so a fire that reaches a CPU not being backtraced is harmless. + */ + nmi_cpu_backtrace(regs); + return SDEI_EV_HANDLED; +} +NOKPROBE_SYMBOL(sdei_nmi_handler); + +static void sdei_nmi_fire(unsigned int target_cpu) +{ + int err = sdei_event_signal(SDEI_NMI_EVENT, cpu_logical_map(target_cpu)); + + if (err) + pr_warn("SDEI_EVENT_SIGNAL to CPU %u failed: %d\n", + target_cpu, err); +} + +/* + * Raise callback for nmi_trigger_cpumask_backtrace(): signal event 0 + * at every CPU still pending in @mask. The framework excludes the local + * CPU from @mask before calling us. + */ +static void sdei_nmi_raise_backtrace(cpumask_t *mask) +{ + unsigned int cpu; + + for_each_cpu(cpu, mask) + sdei_nmi_fire(cpu); +} + +/* + * Override hook for arch_trigger_cpumask_backtrace() (see + * arch/arm64/kernel/smp.c). Returns true when SDEI handled the request, + * which is the case whenever SDEI is active; on a false return the arch + * falls back to its regular-IRQ (or pseudo-NMI, if enabled) IPI. + * + * On a kernel built without paying the pseudo-NMI hot-path cost (the + * usual case for this driver's target), the IPI can't reach a CPU that + * has interrupts masked -- so the backtrace of the one CPU you care + * about comes back empty. SDEI is dispatched out of EL3 and lands + * regardless of the target's DAIF, without taxing the IRQ-mask path. + */ +bool sdei_nmi_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu) +{ + if (!sdei_nmi_available) + return false; + + nmi_trigger_cpumask_backtrace(mask, exclude_cpu, + sdei_nmi_raise_backtrace); + return true; +} + +/* + * device_initcall (after arch_initcall(sdei_init), so the SDEI subsystem + * is up): probe the firmware, register the event, and turn on the + * cross-CPU service. If the probe fails the driver stays inert and the + * override hooks decline, leaving the arch's own paths in place. + */ +static int __init sdei_nmi_init(void) +{ + int err; + + if (!sdei_is_present()) + return 0; + + err = sdei_event_register(SDEI_NMI_EVENT, sdei_nmi_handler, NULL); + if (err) { + pr_err("sdei_event_register(%u) failed: %d\n", + SDEI_NMI_EVENT, err); + return 0; + } + + err = sdei_event_enable(SDEI_NMI_EVENT); + if (err) { + pr_err("sdei_event_enable(%u) failed: %d\n", + SDEI_NMI_EVENT, err); + sdei_event_unregister(SDEI_NMI_EVENT); + return 0; + } + + sdei_nmi_available = true; + pr_info("using SDEI cross-CPU NMI (SDEI_EVENT_SIGNAL, event %u)\n", + SDEI_NMI_EVENT); + + return 0; +} +device_initcall(sdei_nmi_init); -- 2.54.0