genirq: NMI API lacks the request_mutex / dev_id matching / idempotency semantics that free_irq() has -- double free_nmi releases chip PM and module references twice (verified)

[徐晗翔 <xuhx@hust.edu.cn>]

[-- Attachment #1: Type: text/plain, Size: 5341 bytes --]

The NMI variants of the genirq registration API dropped three
guarantees that __free_irq() provides on the plain-IRQ side, all
visible in __cleanup_nmi() (kernel/irq/manage.c:2032-2065): (1) no
request_mutex -- request_nmi()'s own NMI-state setup even runs AFTER
__setup_irq() released the mutex (manage.c:2326-2330), and free_nmi()
-> __cleanup_nmi() never takes it; (2) no dev_id identity matching --
free_nmi() accepts dev_id but never compares it (manage.c:2067-2082),
so it always frees "whatever is currently installed"; (3) no
idempotency -- when the WARN at :2042 fires (desc->action == NULL,
i.e. the line was already freed) the cleanup tail
(irq_release_resources(), irq_chip_pm_put(), module_put() at
:2059-2062) STILL EXECUTES unconditionally, unlike __free_irq()'s
already-free early return which skips the tail (manage.c:1881-1887).
A double free_nmi() therefore underflows the irqchip PM reference and
the module reference and re-releases the chip resources -- I verified
this on a pristine Linux 7.3 KASAN kernel: two concurrent free_nmi()
calls on one line produced WARNING manage.c:2042 (double cleanup),
irq_nmi_teardown and irq_release_resources executed twice, and
/proc/modules showed the module refcount at -1.  The companion report
(the NMI dispatch use-after-free) covers the quiescence gap on the
same API; this mail covers the serialization/matching/idempotency
gap.  No in-tree caller today issues concurrent or double free_nmi
(probe/remove are serialized per device and each NMI line has a
single owner), so this is an API-structural defect with verified
consequences -- cheap to fix (align __cleanup_nmi() with __free_irq()
semantics), worth fixing before any driver relies on concurrent
NMI management.

The detailed description:

* The missing guarantees (Linux 7.3, kernel/irq/manage.c)

  Plain side: __setup_irq() takes request_mutex explicitly to protect
  against a concurrent __free_irq() (:1556-1564); __free_irq() holds
  it (:1868-1984), matches dev_id precisely (:1886-1889), and on
  already-free WARNs and returns early WITHOUT the release tail
  (:1881-1887); resource release happens under chip_bus_lock
  (:1967-1982).

  NMI side: __cleanup_nmi() (:2032-2065) holds only desc->lock for
  the publication removal (:2037 teardown, :2047 action=NULL,
  :2050 shutdown/deactivate); everything else -- kfree (:2057),
  irq_release_resources (:2059), irq_chip_pm_put (:2061),
  module_put (:2062) -- runs unlocked.  request_nmi()'s failure path
  reuses the same cleanup (:2333), and its success-path NMI-state
  commit (istate |= IRQS_NMI; irq_nmi_setup(), :2326-2330) runs after
  __setup_irq() released request_mutex (:2322) -- so even the setup
  half of the protocol is outside the mutex on the NMI side.

  The unconditional tail on the already-free path is the directly
  observable defect: :2042's WARN has no linkage to :2057-2062.

* Evidence (pristine 7.3 KASAN; instruments attached)

  nmi_race.c drives two threads to a rendezvous and then issues two
  concurrent free_nmi() calls on the same line (both taking the
  depth==0 WARN branch, manage.c:2078):

    WARNING manage.c:2078 at free_nmi+0x69            (x2, both threads)
    chip: irq_nmi_teardown #1
    chip: irq_release_resources #1                    (first tail: ok)
    chip: irq_nmi_teardown #2                          <- second cleanup
    WARNING manage.c:2042 at __cleanup_nmi+0x157       (desc->action
                                                          == NULL)
    chip: irq_release_resources #2                     <- tail ran again
    /proc/modules: nmi_race 24576 -1 - Live            <- module_put ran
                                                          twice: module
                                                          refcount
                                                          UNDERFLOW

  Zero KASAN reports, as the analysis predicts: the double cleanup
  path skips kfree (no UAF) -- the damage is the reference/resource
  underflow (chip runtime PM, module unload safety) and the
  double-executed irq_release_resources.

  Module usage: insmod nmi_race.ko; the two-thread rendezvous +
  concurrent free_nmi runs automatically (action=nmi2c in the plans);
  nmi_stall.ko is the companion window instrument (used by the
  companion dispatch-UAF report).

* Fix direction 

  1. free_nmi() (and the request_nmi failure path's cleanup call)
     take desc->request_mutex around the cleanup;
  2. __cleanup_nmi(): on the already-free WARN, return early and skip
     the entire release tail (idempotency, matching manage.c:
     1881-1887);
  3. free_nmi() matches dev_id (or the API drops the parameter with
     an explicit comment) -- restoring the identity semantics callers
     clearly expect (arm_pmu passes per-cpu pointers, agdi/a64fx pass
     NULL);
  4. move request_nmi()'s istate/irq_nmi_setup commit inside the
     __setup_irq() mutex-held region (or add chip_bus_lock around the
     NMI resource release, mirroring __free_irq()).

  With (2) alone, the verified W5 disappears; with (1)+(2) the NMI
  API gains the same serialization and idempotency as free_irq().
  (The dispatch-side quiescence use-after-free is a separate defect,
  reported in its own mail.)


  Please keep me in Cc; I am not subscribed to the lists.


Regards,
Hanxiang Xu
xuhx@hust.edu.cn

[-- Attachment #2: nmi_stall.c --]
[-- Type: text/plain, Size: 2814 bytes --]

// SPDX-License-Identifier: GPL-2.0
/*
 * nmi_stall.c - generic kprobe controlled-window staller for the
 * kernel/irq NMI verification (2026-11-09), following the r4_stall
 * methodology (controlled window; does not measure natural hit rates).
 *
 * Params:
 *   symbol=   function to probe (required)
 *   off=      kprobe offset inside symbol (instruction offset, hex ok)
 *   stall_us= max busy-wait duration once armed+hit (default 40s)
 *   arm=      sysfs gate (0600): stall only while armed
 *   release=  sysfs gate (0600): end the stall
 *
 * The pre_handler stalls the FIRST armed hit until release=1 (or timeout).
 * Later hits are counted only.
 */
#define pr_fmt(fmt) "NMI1STALL: " fmt

#include <linux/module.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/kprobes.h>
#include <linux/delay.h>
#include <linux/atomic.h>

static char *symbol = "";
module_param(symbol, charp, 0400);
MODULE_PARM_DESC(symbol, "kprobe target symbol");

static unsigned long off;
module_param(off, ulong, 0400);
MODULE_PARM_DESC(off, "kprobe offset inside symbol");

static int stall_us = 40000000;
module_param(stall_us, int, 0600);
MODULE_PARM_DESC(stall_us, "max stall duration in us");

static int arm;
module_param(arm, int, 0600);
MODULE_PARM_DESC(arm, "1 = stall next hit");

static int release;
module_param(release, int, 0600);
MODULE_PARM_DESC(release, "1 = end the stall");

static atomic_t hits = ATOMIC_INIT(0);
static bool consumed;

static int stall_pre(struct kprobe *p, struct pt_regs *regs)
{
	int h = atomic_inc_return(&hits);

	if (!READ_ONCE(arm) || consumed) {
		if (h <= 5)
			pr_warn("hit #%d (not stalled, arm=%d)\n", h, READ_ONCE(arm));
		return 0;
	}
	WRITE_ONCE(consumed, true);
	pr_warn("STALL HIT at %s+0x%lx stall_us=%d\n", p->symbol_name, off,
		stall_us);
	/* hard deadline so a dead orchestrator cannot hang the VM forever */
	for (int waited = 0; waited < stall_us && !READ_ONCE(release);
	     waited += 1000)
		udelay(1000);
	pr_warn("RELEASED (release=%d waited_hits=%d)\n", READ_ONCE(release),
		atomic_read(&hits));
	return 0;
}

static struct kprobe kp;

static int __init nmi_stall_init(void)
{
	if (!symbol[0]) {
		pr_err("no symbol given\n");
		return -EINVAL;
	}
	kp.symbol_name = symbol;
	kp.offset = off;
	kp.pre_handler = stall_pre;
	if (register_kprobe(&kp)) {
		pr_err("cannot register kprobe on %s+0x%lx\n", symbol, off);
		return -EINVAL;
	}
	pr_warn("armed-able: %s+0x%lx stall_us=%d (waiting for arm=1)\n",
		symbol, off, stall_us);
	return 0;
}

static void __exit nmi_stall_exit(void)
{
	unregister_kprobe(&kp);
	pr_warn("removed hits=%d stalled=%d\n", atomic_read(&hits), consumed);
}

module_init(nmi_stall_init);
module_exit(nmi_stall_exit);
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("generic kprobe stall for kernel/irq NMI verification");

[-- Attachment #3: nmi_race.c --]
[-- Type: text/plain, Size: 10724 bytes --]

// SPDX-License-Identifier: GPL-2.0
/*
 * nmi_race.c - driver module for the kernel/irq NMI race verification
 * (2026-11-09). It exercises the REAL genirq NMI core paths
 * (request_nmi/free_nmi/enable_irq + handle_fasteoi_nmi) on a fake
 * NMI-capable irq_chip, mirroring what GICv3 pseudo-NMI does on arm64.
 *
 * request_nmi()/free_nmi() are not exported, so they are resolved with the
 * kprobe-addr trick (same technique as r4_stall's symbol lookup).
 *
 * modes:
 *   1  Case-1 controlled window: dispatch thread enters handle_fasteoi_nmi
 *      and is stalled by nmi_stall.ko (kprobe at +0x56 in build-kasan-7.3,
 *      after the desc->action snapshot, before the action->handler /
 *      action->dev_id loads); free thread frees the action meanwhile; the
 *      released dispatcher then reads freed memory -> KASAN.
 *   3  natural hammer: writer request/sleep/free/sleep cycles + dense
 *      dispatcher (READ_ONCE(installed) pre-check to dodge the NULL window;
 *      residual NULL / freed reads are captured by KASAN/KCSAN).
 *      churn=1 variant: writer only cycles enable/disable (negative control,
 *      desc->action untouched).
 *   5  W5 double-free: two threads call free_nmi() concurrently on an
 *      installed line; the loser passes the irq_is_nmi() gate before the
 *      winner clears IRQS_NMI, then serializes behind desc->lock and runs
 *      a SECOND cleanup (WARN_ON(desc->action==NULL) + unconditional
 *      release tail) -> teardown/release_resources counted twice, module
 *      refcount goes negative in /proc/modules.
 */
#define pr_fmt(fmt) "NMI1RACE: " fmt

#include <linux/module.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/kprobes.h>
#include <linux/irq.h>
#include <linux/irqdesc.h>
#include <linux/interrupt.h>
#include <linux/kthread.h>
#include <linux/delay.h>
#include <linux/atomic.h>
#include <linux/sched/task.h>

typedef int (*request_nmi_t)(unsigned int, irq_handler_t, unsigned long,
			     const char *, void *);
typedef const void *(*free_nmi_t)(unsigned int, void *);

static int mode = 1;
module_param(mode, int, 0400);
MODULE_PARM_DESC(mode, "1=case1 controlled, 3=hammer, 5=W5 double-free");

static int secs = 60;
module_param(secs, int, 0400);
MODULE_PARM_DESC(secs, "hammer duration (mode 3)");

static int hold_ms = 1500;
module_param(hold_ms, int, 0400);
MODULE_PARM_DESC(hold_ms, "installed hold time per cycle (mode 3)");

static int gap_ms = 100;
module_param(gap_ms, int, 0400);
MODULE_PARM_DESC(gap_ms, "freed gap time per cycle (mode 3)");

static int churn;
module_param(churn, int, 0400);
MODULE_PARM_DESC(churn, "mode3 control: writer cycles enable/disable only");

static unsigned int cpu_disp = 2, cpu_free = 3;
module_param(cpu_disp, uint, 0400);
module_param(cpu_free, uint, 0400);

static request_nmi_t request_nmi_fn;
static free_nmi_t free_nmi_fn;

static unsigned int virq;
static void *cookie;
static atomic_t installed = ATOMIC_INIT(0);
static atomic_t go = ATOMIC_INIT(0);
static atomic_t teardown_cnt = ATOMIC_INIT(0);
static atomic_t relres_cnt = ATOMIC_INIT(0);
static atomic_t dispatches = ATOMIC_INIT(0);
static atomic_t disp_null = ATOMIC_INIT(0);
static struct task_struct *t_disp, *t_free;

/* ---- fake NMI-capable chip (core sees the same callbacks as GICv3) ---- */

static int fake_nmi_setup(struct irq_data *d)
{
	pr_warn("chip: irq_nmi_setup irq=%u\n", d->irq);
	return 0;
}

static void fake_nmi_teardown(struct irq_data *d)
{
	pr_warn("chip: irq_nmi_teardown #%d irq=%u\n",
		atomic_inc_return(&teardown_cnt), d->irq);
}

static void fake_relres(struct irq_data *d)
{
	pr_warn("chip: irq_release_resources #%d irq=%u\n",
		atomic_inc_return(&relres_cnt), d->irq);
}

static void fake_mask(struct irq_data *d) {}
static void fake_unmask(struct irq_data *d) {}
static void fake_eoi(struct irq_data *d) {}

static struct irq_chip fake_chip = {
	.name			= "nmi-fake",
	.flags			= IRQCHIP_SUPPORTS_NMI,
	.irq_mask		= fake_mask,
	.irq_unmask		= fake_unmask,
	.irq_eoi		= fake_eoi,
	.irq_nmi_setup		= fake_nmi_setup,
	.irq_nmi_teardown	= fake_nmi_teardown,
	.irq_release_resources	= fake_relres,
};

static atomic_t handler_calls = ATOMIC_INIT(0);

static irqreturn_t fake_nmi_handler(int irq, void *dev_id)
{
	/* print sparsely: printk-path accesses would dilute KCSAN sampling */
	if (atomic_inc_return(&handler_calls) % 512 == 1)
		pr_warn("HANDLER CALLED #%d irq=%d dev_id=%px\n",
			atomic_read(&handler_calls), irq, dev_id);
	return IRQ_HANDLED;
}

static void park(void)
{
	while (!kthread_should_stop())
		msleep(100);
}

static unsigned long sym_addr(const char *name)
{
	struct kprobe p = { .symbol_name = name };
	unsigned long addr = 0;

	if (!register_kprobe(&p)) {
		addr = (unsigned long)p.addr;
		unregister_kprobe(&p);
	}
	return addr;
}

static int do_request(void)
{
	int ret;

	ret = request_nmi_fn(virq, fake_nmi_handler,
			     IRQF_PERCPU | IRQF_NOBALANCING | IRQF_NO_AUTOEN |
			     IRQF_NO_THREAD,
			     "nmi-race", &cookie);
	if (!ret) {
		atomic_inc(&installed);
		enable_irq(virq); /* == enable_nmi */
		/*
		 * The GIC driver installs handle_fasteoi_nmi from its
		 * irq_nmi_setup callback (desc->lock held); the core does not
		 * do it. struct irq_desc is opaque to modules, so we install
		 * the audited flow handler via the exported API instead.
		 */
		__irq_set_handler(virq, handle_fasteoi_nmi, 0, NULL);
	}
	pr_warn("request_nmi -> %d (installed=%d)\n", ret,
		atomic_read(&installed));
	return ret;
}

static void do_free(void)
{
	const void *devname;

	atomic_dec(&installed);
	devname = free_nmi_fn(virq, &cookie);
	pr_warn("free_nmi -> %p (installed=%d)\n", devname,
		atomic_read(&installed));
}

/* mode 1: dispatcher stalls inside handle_fasteoi_nmi via nmi_stall.ko */
static int disp_thread_m1(void *arg)
{
	while (!atomic_read(&go))
		cpu_relax();
	pr_warn("DISPATCH_BEGIN (cpu=%u)\n", smp_processor_id());
	generic_handle_irq(virq);
	pr_warn("DISPATCH_DONE\n");
	park();
	return 0;
}

static int free_thread_m1(void *arg)
{
	msleep(2000);
	pr_warn("release go -> dispatch enters handle_fasteoi_nmi\n");
	atomic_inc(&go);
	msleep(3000); /* dispatcher is stalled inside handle_fasteoi_nmi now */
	do_free();
	pr_warn("FREE side finished (dispatcher still stalled on freed action)\n");
	park();
	return 0;
}

/* mode 3: natural hammer */
static int writer_thread(void *arg)
{
	unsigned long deadline = jiffies + secs * HZ;
	int cycles = 0;

	while (time_before(jiffies, deadline)) {
		if (churn) {
			/* control: no management writes at all */
			msleep_interruptible(hold_ms);
		} else {
			if (do_request())
				break;
			msleep_interruptible(hold_ms);
			do_free();
			msleep_interruptible(gap_ms);
		}
		cycles++;
	}
	if (churn)
		cycles = 0;
	pr_warn("writer done: cycles=%d churn=%d teardown=%d relres=%d\n",
		cycles, churn, atomic_read(&teardown_cnt),
		atomic_read(&relres_cnt));
	park();
	return 0;
}

static int disp_thread_m3(void *arg)
{
	unsigned long deadline = jiffies + secs * HZ;

	while (time_before(jiffies, deadline)) {
		if (atomic_read(&installed)) {
			generic_handle_irq(virq);
			atomic_inc(&dispatches);
		} else {
			atomic_inc(&disp_null);
			cpu_relax();
		}
	}
	pr_warn("dispatcher done: dispatches=%d null-skips=%d\n",
		atomic_read(&dispatches), atomic_read(&disp_null));
	park();
	return 0;
}

/* mode 5: concurrent double free_nmi (W5) */
static int w5_thread(void *arg)
{
	int id = (int)(long)arg;
	const void *ret;
	int i;

	/* rendezvous so BOTH threads pass free_nmi's irq_is_nmi() gate before
	 * either one clears IRQS_NMI inside the cleanup lock region */
	atomic_inc(&go);
	while (atomic_read(&go) < 2)
		cpu_relax();

	ret = free_nmi_fn(virq, &cookie);
	pr_warn("W5 thread %d free -> %p teardown=%d relres=%d\n", id, ret,
		atomic_read(&teardown_cnt), atomic_read(&relres_cnt));
	/* fallback retries in case this thread lost the istate race */
	for (i = 0; i < 30 && atomic_read(&teardown_cnt) < 2; i++)
		udelay(500);
	pr_warn("W5 thread %d done: teardown=%d relres=%d (expect 2/2)\n", id,
		atomic_read(&teardown_cnt), atomic_read(&relres_cnt));
	park();
	return 0;
}

static int __init nmi_race_init(void)
{
	unsigned long a_req = sym_addr("request_nmi");
	unsigned long a_free = sym_addr("free_nmi");
	int ret = -EINVAL;

	request_nmi_fn = (request_nmi_t)a_req;
	free_nmi_fn = (free_nmi_t)a_free;
	if (!request_nmi_fn || !free_nmi_fn) {
		pr_err("cannot resolve request_nmi=%lx free_nmi=%lx\n", a_req,
		       a_free);
		return -EINVAL;
	}
	pr_warn("resolved request_nmi=%px free_nmi=%px mode=%d\n",
		request_nmi_fn, free_nmi_fn, mode);

	virq = __irq_alloc_descs(-1, 0, 1, NUMA_NO_NODE, THIS_MODULE, NULL);
	if (virq < 0) {
		pr_err("irq_alloc_descs failed %d\n", virq);
		return virq;
	}
	pr_warn("allocated virq=%u (desc owner=this module)\n", virq);
	irq_set_chip(virq, &fake_chip);

	switch (mode) {
	case 1:
		ret = do_request();
		if (ret)
			return ret;
		t_free = kthread_create_on_cpu(free_thread_m1, NULL, cpu_free,
					       "nmi1_free");
		t_disp = kthread_create_on_cpu(disp_thread_m1, NULL, cpu_disp,
					       "nmi1_disp");
		if (IS_ERR(t_free) || IS_ERR(t_disp))
			return -EINVAL;
		wake_up_process(t_free);
		wake_up_process(t_disp);
		break;
	case 3:
		/* experiment: the writer installs in its first cycle.
		 * churn control: install once here so the dispatch density
		 * matches the experiment, and the writer only sleeps. */
		if (churn && do_request())
			return ret;
		t_free = kthread_create_on_cpu(writer_thread, NULL, cpu_free,
					       "nmi1_writer");
		t_disp = kthread_create_on_cpu(disp_thread_m3, NULL, cpu_disp,
					       "nmi1_disp");
		if (IS_ERR(t_free) || IS_ERR(t_disp))
			return -EINVAL;
		wake_up_process(t_free);
		wake_up_process(t_disp);
		break;
	case 5:
		ret = do_request();
		if (ret)
			return ret;
		t_free = kthread_create_on_cpu(w5_thread, (void *)1L, cpu_free,
					       "nmi_w5a");
		t_disp = kthread_create_on_cpu(w5_thread, (void *)2L, cpu_disp,
					       "nmi_w5b");
		if (IS_ERR(t_free) || IS_ERR(t_disp))
			return -EINVAL;
		wake_up_process(t_free);
		wake_up_process(t_disp);
		break;
	default:
		pr_err("bad mode %d\n", mode);
		return -EINVAL;
	}
	return 0;
}

static void __exit nmi_race_exit(void)
{
	if (t_free && !IS_ERR(t_free))
		kthread_stop(t_free);
	if (t_disp && !IS_ERR(t_disp))
		kthread_stop(t_disp);
	/* best-effort cleanup of a still-installed action (mode 3 churn off) */
	if (atomic_read(&installed) && mode == 3 && !churn)
		do_free();
	pr_warn("removed mode=%d teardown=%d relres=%d\n", mode,
		atomic_read(&teardown_cnt), atomic_read(&relres_cnt));
}

module_init(nmi_race_init);
module_exit(nmi_race_exit);
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("kernel/irq NMI race verification driver");