[PATCH v4 0/7] watchdog: qcom: Support NMI pretimeout warnings

[PATCH v4 0/7] watchdog: qcom: Support NMI pretimeout warnings

[Mayank Rungta]

On ARM64 Qualcomm SoCs, when a system freezes completely due to hard
locked CPUs with standard interrupts disabled, a standard watchdog
pretimeout warning interrupt (bark) fails to fire. To diagnose total
system lockups, we need to transition the Qualcomm hardware watchdog
pretimeout bark interrupt into an NMI (or pseudo-NMI).

Enabling NMI pretimeout handlers within a loadable driver module requires
addressing NMI teardown synchronization, allowing global interrupts in
request_nmi(), exporting NMI registration APIs, and ensuring watchdog
pretimeout governor dispatch is NMI-safe.

This 7-patch series achieves NMI pretimeout enablement for qcom-wdt:
 1) Synchronizes in-flight NMI handlers during disable and teardown via
    an atomic counter on struct irq_desc and unlazy hardware masking.
 2) Allows global interrupts in request_nmi() and updates /proc/interrupts
    to display per-CPU counts for all NMIs.
 3) Implements synchronous disable_nmi().
 4) Exports request_nmi(), free_nmi(), enable_nmi(), and disable_nmi()
    to the qcom-wdt module using EXPORT_SYMBOL_FOR_MODULES().
 5) Replaces spinlocks in watchdog_notify_pretimeout() with RCU to
    guarantee safe governor execution from NMI context.
 6) Updates the panic pretimeout governor (pretimeout_panic) to call
    nmi_panic() when invoked from NMI context.
 7) Updates qcom-wdt to request its pretimeout bark interrupt as an
    NMI (or pseudo-NMI) with fallback to standard IRQ.

Signed-off-by: Mayank Rungta <mrungta@google.com>
---
Changes in v4:
- Replaced hardware-only NMI synchronization with a lockless software counter
  (desc->nmi_active) and unlazy hardware masking via __synchronize_nmi()
  (Patches 1 & 3) (Thomas Gleixner, Marc Zyngier, Douglas Anderson).
- Allowed global interrupts in request_nmi() by dropping the mandatory
  IRQF_PERCPU check and updating /proc/interrupts for NMIs (Patches 2 & 7)
  (Thomas Gleixner, Marc Zyngier).
- Switched NMI symbol exports to EXPORT_SYMBOL_FOR_MODULES(..., "qcom-wdt")
  (Patch 4) (Thomas Gleixner).
- Added Patch 6 to invoke nmi_panic() from pretimeout_panic() when in_nmi()
  is true.
- Link to v3: https://lore.kernel.org/r/20260902-qcom-wdt-nmi-series-v3-0-f3999362a9ea@google.com
- Link to v2: https://lore.kernel.org/r/20260828-qcom-wdt-nmi-series-v2-0-363979fe6b6b@google.com
- Link to v1: https://lore.kernel.org/r/20260730-qcom-wdt-nmi-series-v1-0-3aa86d162914@google.com

Testing & Verification:
- Tested on ARM64 Qualcomm Snapdragon SoC (Google CoachZ) with GICv3
  pseudo-NMI enabled: verified NMI pretimeout bark delivery, /proc/interrupts
  accounting, runtime sysfs governor switching (noop/panic), nmi_panic()
  execution, and driver unbind/rebind teardown.

Mayank Rungta (7):
  genirq: Synchronize in-flight handlers during NMI teardown
  genirq: Allow global interrupts in request_nmi()
  genirq: Implement synchronous disable_nmi()
  genirq: Export NMI APIs for qcom-wdt
  watchdog: pretimeout: Protect governor access with RCU for NMI safety
  watchdog: pretimeout_panic: Use nmi_panic() when invoked from NMI
    context
  watchdog: qcom: Register pretimeout interrupt as NMI

 drivers/watchdog/pretimeout_panic.c    |   8 +-
 drivers/watchdog/qcom-wdt.c            |  30 ++++-
 drivers/watchdog/watchdog_pretimeout.c |  45 ++++----
 include/linux/interrupt.h              |   1 +
 include/linux/irqdesc.h                |   4 +
 include/linux/watchdog.h               |   2 +-
 kernel/irq/chip.c                      |  35 ++++--
 kernel/irq/manage.c                    | 148 +++++++++++++++++++++----
 kernel/irq/proc.c                      |   6 +-
 kernel/irq/settings.h                  |   5 +
 10 files changed, 225 insertions(+), 59 deletions(-)


base-commit: 2a500a04d6f969318c05f46aedb6e73fc1963a00
-- 
2.56.0.611.g1030f0b940-goog

[PATCH v4 1/7] genirq: Synchronize in-flight handlers during NMI teardown

[Mayank Rungta]

When freeing or disabling an interrupt requested via request_nmi(),
__cleanup_nmi() and __synchronize_hardirq() currently do not wait for
in-flight instances of the NMI handler on other CPUs to complete, which
can lead to use-after-free conditions on module unload or driver unbind.

Regular IRQs and NMIs require different synchronization mechanisms:
- Regular IRQ flow handlers already acquire desc->lock, so tracking
  in-flight execution via IRQD_IRQ_INPROGRESS under desc->lock avoids
  extra atomic operations in the interrupt fast path and extra memory
  barriers during synchronization.
- NMI handlers cannot acquire desc->lock, and querying hardware
  IRQCHIP_STATE_ACTIVE alone does not cover handler execution after EOI.
  Track in-flight NMI handlers locklessly via an atomic counter
  (desc->nmi_active) in handle_fasteoi_nmi(), paired with a dedicated
  __synchronize_nmi() helper that waits for both nmi_active and
  (when supported) IRQCHIP_STATE_ACTIVE to drain.

Because handle_fasteoi_nmi() runs locklessly and cannot mask lazily
disabled interrupts in hardware, force unlazy disable
(_IRQ_DISABLE_UNLAZY) in request_nmi() and mask the interrupt in
hardware before setting IRQD_IRQ_DISABLED in __irq_disable() (matching
irq_shutdown()). This prevents a level-triggered NMI from observing
IRQD_IRQ_DISABLED while still unmasked in hardware and livelocking on
EOI, and is safe for regular IRQs because __irq_disable() runs under
desc->lock and no chip disable/mask callbacks inspect
irqd_irq_disabled().

Update __cleanup_nmi() to serialize teardown, shut down and synchronize
the NMI before clearing desc->action and NMI state, and warn if
free_nmi() is called from interrupt context.

Signed-off-by: Mayank Rungta <mrungta@google.com>
---
 include/linux/irqdesc.h |   4 ++
 kernel/irq/chip.c       |  35 +++++++++----
 kernel/irq/manage.c     | 111 ++++++++++++++++++++++++++++++++++------
 kernel/irq/settings.h   |   5 ++
 4 files changed, 130 insertions(+), 25 deletions(-)

diff --git a/include/linux/irqdesc.h b/include/linux/irqdesc.h
index 779d6023c5c2..f1fabddf9650 100644
--- a/include/linux/irqdesc.h
+++ b/include/linux/irqdesc.h
@@ -65,6 +65,9 @@ struct irq_redirect {
  * @threads_oneshot:	bitfield to handle shared oneshot threads
  * @threads_active:	number of irqaction threads currently running
  * @wait_for_threads:	wait queue for sync_irq to wait for threaded handlers
+ * @nmi_active:		number of NMI handlers currently running (normally 0
+ *			or 1, but can briefly reach 2 if an NMI re-triggers
+ *			between hardware EOI and atomic_dec())
  * @nr_actions:		number of installed actions on this descriptor
  * @no_suspend_depth:	number of irqactions on a irq descriptor with
  *			IRQF_NO_SUSPEND set
@@ -107,6 +110,7 @@ struct irq_desc {
 	unsigned long		threads_oneshot;
 	atomic_t		threads_active;
 	wait_queue_head_t       wait_for_threads;
+	atomic_t		nmi_active;
 #ifdef CONFIG_PM_SLEEP
 	unsigned int		nr_actions;
 	unsigned int		no_suspend_depth;
diff --git a/kernel/irq/chip.c b/kernel/irq/chip.c
index de754db414d1..565cb0233fff 100644
--- a/kernel/irq/chip.c
+++ b/kernel/irq/chip.c
@@ -364,13 +364,19 @@ static void __irq_disable(struct irq_desc *desc, bool mask)
 		if (mask)
 			mask_irq(desc);
 	} else {
-		irq_state_set_disabled(desc);
 		if (desc->irq_data.chip->irq_disable) {
 			desc->irq_data.chip->irq_disable(&desc->irq_data);
 			irq_state_set_masked(desc);
 		} else if (mask) {
 			mask_irq(desc);
 		}
+		/*
+		 * Mask in hardware before setting IRQD_IRQ_DISABLED so a
+		 * lockless NMI firing before hardware masking completes does
+		 * not observe IRQD_IRQ_DISABLED, skip its handler, and EOI an
+		 * unmasked level-triggered line into an infinite NMI storm.
+		 */
+		irq_state_set_disabled(desc);
 	}
 }
 
@@ -792,21 +798,32 @@ EXPORT_SYMBOL_GPL(handle_fasteoi_irq);
 void handle_fasteoi_nmi(struct irq_desc *desc)
 {
 	struct irq_chip *chip = irq_desc_get_chip(desc);
-	struct irqaction *action = desc->action;
 	unsigned int irq = irq_desc_get_irq(desc);
+	struct irqaction *action;
 	irqreturn_t res;
 
-	__kstat_incr_irqs_this_cpu(desc);
+	atomic_inc(&desc->nmi_active);
+	/* Ensure nmi_active is visible before reading action or disabled state */
+	smp_mb__after_atomic();
 
-	trace_irq_handler_entry(irq, action);
-	/*
-	 * NMIs cannot be shared, there is only one action.
-	 */
-	res = action->handler(irq, action->dev_id);
-	trace_irq_handler_exit(irq, action, res);
+	action = READ_ONCE(desc->action);
+	if (likely(action && !irqd_irq_disabled(&desc->irq_data))) {
+		__kstat_incr_irqs_this_cpu(desc);
+
+		trace_irq_handler_entry(irq, action);
+		/*
+		 * NMIs cannot be shared, there is only one action.
+		 */
+		res = action->handler(irq, action->dev_id);
+		trace_irq_handler_exit(irq, action, res);
+	}
 
 	if (chip->irq_eoi)
 		chip->irq_eoi(&desc->irq_data);
+
+	/* Ensure all handler accesses complete before decrementing nmi_active */
+	smp_mb__before_atomic();
+	atomic_dec(&desc->nmi_active);
 }
 EXPORT_SYMBOL_GPL(handle_fasteoi_nmi);
 
diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
index ec3ecb0e48ce..3fb77a2d954d 100644
--- a/kernel/irq/manage.c
+++ b/kernel/irq/manage.c
@@ -48,11 +48,72 @@ static inline void synchronize_irqwork(struct irq_desc *desc) { }
 
 static int __irq_get_irqchip_state(struct irq_data *d, enum irqchip_irq_state which, bool *state);
 
+static void __synchronize_nmi(struct irq_desc *desc, bool sync_chip)
+{
+	struct irq_data *irqd = irq_desc_get_irq_data(desc);
+	bool inprogress;
+
+	do {
+		/*
+		 * Wait until we're out of the NMI handler before taking
+		 * desc->lock or polling the irqchip. This might give the
+		 * wrong answer due to the lack of memory barriers.
+		 */
+		while (atomic_read(&desc->nmi_active))
+			cpu_relax();
+
+		guard(raw_spinlock_irqsave)(&desc->lock);
+		/*
+		 * Ok, that indicated we're done: double-check carefully.
+		 *
+		 * Holding desc->lock serializes with prior state updates (e.g.
+		 * IRQD_IRQ_DISABLED set under desc->lock) and protects
+		 * __irq_get_irqchip_state(). Together with
+		 * smp_mb__after_spinlock(), this provides a full memory
+		 * barrier pairing with smp_mb__after_atomic() in
+		 * handle_fasteoi_nmi() so those updates are visible before
+		 * reading nmi_active.
+		 */
+		smp_mb__after_spinlock();
+
+		/*
+		 * Pair with smp_mb__before_atomic() + atomic_dec() in
+		 * handle_fasteoi_nmi() so all handler memory accesses complete
+		 * before __synchronize_nmi() returns.
+		 */
+		inprogress = atomic_read_acquire(&desc->nmi_active);
+
+		/*
+		 * Also check hardware active state when supported to bridge the
+		 * window between hardware acknowledge and atomic_inc() in
+		 * handle_fasteoi_nmi() on non-percpu_devid SPIs. Ignore the
+		 * return code; inprogress is only updated when the chip
+		 * supports it. Note that if a chip does not implement this,
+		 * disable_nmi() is still safe, and free_nmi() is as safe as we
+		 * can make it (though an NMI acknowledged just before masking
+		 * that has not yet entered handle_fasteoi_nmi() could race with
+		 * irq_nmi_teardown() resetting desc->handle_irq).
+		 */
+		if (!inprogress && sync_chip)
+			__irq_get_irqchip_state(irqd, IRQCHIP_STATE_ACTIVE,
+						&inprogress);
+	} while (inprogress);
+}
+
 static void __synchronize_hardirq(struct irq_desc *desc, bool sync_chip)
 {
 	struct irq_data *irqd = irq_desc_get_irq_data(desc);
 	bool inprogress;
 
+	/*
+	 * Generic paths such as synchronize_irq() (e.g. suspend_device_irqs())
+	 * and synchronize_hardirq() can be called on NMIs; redirect them.
+	 */
+	if (unlikely(irq_is_nmi(desc) && !irq_settings_is_per_cpu_devid(desc))) {
+		__synchronize_nmi(desc, sync_chip);
+		return;
+	}
+
 	do {
 		/*
 		 * Wait until we're out of the critical section.  This might
@@ -2044,29 +2105,38 @@ EXPORT_SYMBOL(free_irq);
 
 static const void *__cleanup_nmi(unsigned int irq, struct irq_desc *desc)
 {
-	struct irqaction *action = NULL;
-	const char *devname = NULL;
+	struct irqaction *action;
+	const char *devname;
 
-	scoped_guard(raw_spinlock_irqsave, &desc->lock) {
-		irq_nmi_teardown(desc);
+	guard(mutex)(&desc->request_mutex);
 
-		desc->istate &= ~IRQS_NMI;
-
-		if (!WARN_ON(desc->action == NULL)) {
-			action = desc->action;
-			irq_pm_remove_action(desc, action);
-			devname = action->name;
-		}
-		desc->action = NULL;
+	scoped_guard(raw_spinlock_irqsave, &desc->lock) {
+		if (WARN_ON(!desc->action))
+			return NULL;
 
 		irq_settings_clr_disable_unlazy(desc);
-		irq_shutdown_and_deactivate(desc);
+		irq_shutdown(desc);
 	}
 
-	irq_proc_update_valid(desc);
+	/*
+	 * Ensure all in-flight NMI handlers on other CPUs complete before
+	 * clearing desc->action or tearing down NMI state.
+	 */
+	__synchronize_nmi(desc, true);
+
+	scoped_guard(raw_spinlock_irqsave, &desc->lock) {
+		action = desc->action;
+		irq_pm_remove_action(desc, action);
+		devname = action->name;
+		WRITE_ONCE(desc->action, NULL);
 
-	if (action)
-		unregister_handler_proc(irq, action);
+		irq_nmi_teardown(desc);
+		desc->istate &= ~IRQS_NMI;
+		irq_domain_deactivate_irq(&desc->irq_data);
+		irq_proc_update_valid(desc);
+	}
+
+	unregister_handler_proc(irq, action);
 	kfree(action);
 
 	irq_release_resources(desc);
@@ -2081,6 +2151,8 @@ const void *free_nmi(unsigned int irq, void *dev_id)
 {
 	struct irq_desc *desc = irq_to_desc(irq);
 
+	WARN(in_interrupt(), "Trying to free NMI %d from IRQ context!\n", irq);
+
 	if (!desc || WARN_ON(!irq_is_nmi(desc)))
 		return NULL;
 
@@ -2337,6 +2409,13 @@ int request_nmi(unsigned int irq, irq_handler_t handler,
 		goto err_irq_setup;
 
 	scoped_guard(raw_spinlock_irqsave, &desc->lock) {
+		/*
+		 * Force unlazy disable so disable_nmi() / disable_nmi_nosync()
+		 * always mask the interrupt in hardware; handle_fasteoi_nmi()
+		 * runs locklessly and cannot mask lazily disabled interrupts.
+		 */
+		irq_settings_set_disable_unlazy(desc);
+
 		/* Setup NMI state */
 		desc->istate |= IRQS_NMI;
 		retval = irq_nmi_setup(desc);
diff --git a/kernel/irq/settings.h b/kernel/irq/settings.h
index 0a0c027a5d34..4fefdf062dbd 100644
--- a/kernel/irq/settings.h
+++ b/kernel/irq/settings.h
@@ -163,6 +163,11 @@ static inline bool irq_settings_disable_unlazy(struct irq_desc *desc)
 	return desc->status_use_accessors & _IRQ_DISABLE_UNLAZY;
 }
 
+static inline void irq_settings_set_disable_unlazy(struct irq_desc *desc)
+{
+	desc->status_use_accessors |= _IRQ_DISABLE_UNLAZY;
+}
+
 static inline void irq_settings_clr_disable_unlazy(struct irq_desc *desc)
 {
 	desc->status_use_accessors &= ~_IRQ_DISABLE_UNLAZY;
-- 
2.56.0.611.g1030f0b940-goog

[PATCH v4 2/7] genirq: Allow global interrupts in request_nmi()

[Mayank Rungta]

request_nmi() currently requires IRQF_PERCPU and forces
IRQF_NOBALANCING. For global system-wide interrupts (such as watchdog
pretimeout SPIs), forcing IRQF_PERCPU sets IRQD_PER_CPU and prevents CPU
hotplug teardown (migrate_one_irq()) from migrating the interrupt off an
offline CPU.

Now that in-flight NMI handlers are synchronized via
__synchronize_nmi(), drop the mandatory IRQF_PERCPU check and stop
forcing IRQF_NOBALANCING in request_nmi() (existing per-CPU callers
already pass both flags explicitly). Also check irq_is_nmi(desc) in
irq_seq_show() (matching kstat_irqs_desc() from commit c09cb1293523
("genirq: Update irq stats from NMI handlers")) so /proc/interrupts
emits per-CPU counts for non-IRQF_PERCPU NMIs.

Signed-off-by: Mayank Rungta <mrungta@google.com>
---
 kernel/irq/manage.c | 9 +++------
 kernel/irq/proc.c   | 6 ++++--
 2 files changed, 7 insertions(+), 8 deletions(-)

diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
index 3fb77a2d954d..bd7cf91719d7 100644
--- a/kernel/irq/manage.c
+++ b/kernel/irq/manage.c
@@ -2352,8 +2352,8 @@ EXPORT_SYMBOL_GPL(request_any_context_irq);
  * An interrupt line delivering NMIs cannot be shared and IRQ handling
  * cannot be threaded.
  *
- * Interrupt lines requested for NMI delivering must produce per cpu
- * interrupts and have auto enabling setting disabled.
+ * Interrupt lines requested for NMI delivering must have auto enabling
+ * setting disabled.
  *
  * @dev_id must be globally unique. Normally the address of the device data
  * structure is used as the cookie. Since the handler receives this value
@@ -2376,9 +2376,6 @@ int request_nmi(unsigned int irq, irq_handler_t handler,
 	if (irqflags & (IRQF_SHARED | IRQF_COND_SUSPEND | IRQF_IRQPOLL))
 		return -EINVAL;
 
-	if (!(irqflags & IRQF_PERCPU))
-		return -EINVAL;
-
 	if (!handler)
 		return -EINVAL;
 
@@ -2396,7 +2393,7 @@ int request_nmi(unsigned int irq, irq_handler_t handler,
 		return -ENOMEM;
 
 	action->handler = handler;
-	action->flags = irqflags | IRQF_NO_THREAD | IRQF_NOBALANCING;
+	action->flags = irqflags | IRQF_NO_THREAD;
 	action->name = name;
 	action->dev_id = dev_id;
 
diff --git a/kernel/irq/proc.c b/kernel/irq/proc.c
index 0c0df16de581..d92f456191e1 100644
--- a/kernel/irq/proc.c
+++ b/kernel/irq/proc.c
@@ -559,10 +559,12 @@ static int irq_seq_show(struct seq_file *p, void *v)
 	seq_putc(p, ':');
 
 	/*
-	 * Always output per CPU interrupts. Output device interrupts only when
-	 * desc::tot_count is not zero.
+	 * Always output per CPU and NMI interrupts, as NMI handlers only
+	 * update per-CPU kstat counters and leave desc::tot_count zero.
+	 * Output other device interrupts only when desc::tot_count is not zero.
 	 */
 	if (irq_settings_is_per_cpu(desc) || irq_settings_is_per_cpu_devid(desc) ||
+	    irq_is_nmi(desc) ||
 	    data_race(desc->tot_count))
 		irq_proc_emit_counts(p, &desc->kstat_irqs->cnt);
 	else
-- 
2.56.0.611.g1030f0b940-goog

[PATCH v4 3/7] genirq: Implement synchronous disable_nmi()

[Mayank Rungta]

Currently, disable_nmi_nosync() is the only interface to disable an NMI
line. Drivers stopping their devices need a synchronous variant that
guarantees any running instance of the NMI handler has completed on other
CPUs before returning.

Implement disable_nmi() by validating the descriptor, disabling the line
via __disable_irq_nosync(), and synchronizing in-flight NMI handlers via
__synchronize_nmi().

Signed-off-by: Mayank Rungta <mrungta@google.com>
---
 include/linux/interrupt.h |  1 +
 kernel/irq/manage.c       | 24 ++++++++++++++++++++++++
 2 files changed, 25 insertions(+)

diff --git a/include/linux/interrupt.h b/include/linux/interrupt.h
index ee2dd5349697..4a220142e16c 100644
--- a/include/linux/interrupt.h
+++ b/include/linux/interrupt.h
@@ -245,6 +245,7 @@ DEFINE_LOCK_GUARD_1(disable_irq, int,
 		    disable_irq(*_T->lock), enable_irq(*_T->lock))
 
 extern void disable_nmi_nosync(unsigned int irq);
+void disable_nmi(unsigned int irq);
 extern void disable_percpu_nmi(unsigned int irq);
 extern void enable_nmi(unsigned int irq);
 extern void enable_percpu_nmi(unsigned int irq, unsigned int type);
diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
index bd7cf91719d7..073fd3595d2c 100644
--- a/kernel/irq/manage.c
+++ b/kernel/irq/manage.c
@@ -829,6 +829,30 @@ void disable_nmi_nosync(unsigned int irq)
 	disable_irq_nosync(irq);
 }
 
+/**
+ * disable_nmi - Disable an NMI and wait for any pending handlers
+ * @irq:	Interrupt to disable
+ *
+ * Disable the selected interrupt line. Disables and enables are nested.
+ *
+ * The interrupt to disable must have been requested through request_nmi().
+ * This function ensures existing instances of the NMI handler have
+ * completed before returning.
+ *
+ * Cannot be called from NMI context.
+ */
+void disable_nmi(unsigned int irq)
+{
+	struct irq_desc *desc = irq_to_desc(irq);
+
+	if (!desc || WARN_ON(in_nmi()) || WARN_ON(!irq_is_nmi(desc)) ||
+	    WARN_ON(irq_settings_is_per_cpu_devid(desc)))
+		return;
+
+	if (!__disable_irq_nosync(irq))
+		__synchronize_nmi(desc, true);
+}
+
 void __enable_irq(struct irq_desc *desc)
 {
 	switch (desc->depth) {
-- 
2.56.0.611.g1030f0b940-goog

[PATCH v4 4/7] genirq: Export NMI APIs for qcom-wdt

[Mayank Rungta]

Currently, request_nmi(), free_nmi(), enable_nmi(), and disable_nmi() are
restricted to built-in kernel code because they are not exported.

Export these symbols with EXPORT_SYMBOL_FOR_MODULES() restricted to the
qcom-wdt module so it can register and manage its pretimeout NMI when
built as a loadable module.

Signed-off-by: Mayank Rungta <mrungta@google.com>
---
 kernel/irq/manage.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
index 073fd3595d2c..ace4d2796382 100644
--- a/kernel/irq/manage.c
+++ b/kernel/irq/manage.c
@@ -852,6 +852,7 @@ void disable_nmi(unsigned int irq)
 	if (!__disable_irq_nosync(irq))
 		__synchronize_nmi(desc, true);
 }
+EXPORT_SYMBOL_FOR_MODULES(disable_nmi, "qcom-wdt");
 
 void __enable_irq(struct irq_desc *desc)
 {
@@ -919,6 +920,7 @@ void enable_nmi(unsigned int irq)
 {
 	enable_irq(irq);
 }
+EXPORT_SYMBOL_FOR_MODULES(enable_nmi, "qcom-wdt");
 
 static int set_irq_wake_real(unsigned int irq, unsigned int on)
 {
@@ -2189,6 +2191,7 @@ const void *free_nmi(unsigned int irq, void *dev_id)
 
 	return __cleanup_nmi(irq, desc);
 }
+EXPORT_SYMBOL_FOR_MODULES(free_nmi, "qcom-wdt");
 
 /**
  * request_threaded_irq - allocate an interrupt line
@@ -2455,6 +2458,7 @@ int request_nmi(unsigned int irq, irq_handler_t handler,
 
 	return retval;
 }
+EXPORT_SYMBOL_FOR_MODULES(request_nmi, "qcom-wdt");
 
 void enable_percpu_irq(unsigned int irq, unsigned int type)
 {
-- 
2.56.0.611.g1030f0b940-goog

[PATCH v4 5/7] watchdog: pretimeout: Protect governor access with RCU for NMI safety

[Mayank Rungta]

When watchdog pretimeout interrupts are configured as NMIs (or
pseudo-NMIs), pretimeout handlers execute in NMI context. Accessing
the watchdog governor via spinlocks in watchdog_notify_pretimeout() is
unsafe in NMI context and can cause deadlocks if an NMI arrives while a
spinlock is held.

Protect governor assignment and dereference with RCU instead of
spinlocks, allowing lockless, NMI-safe governor notifications while
preserving safe runtime governor switching.

Reviewed-by: Douglas Anderson <dianders@chromium.org>
Signed-off-by: Mayank Rungta <mrungta@google.com>
---
 drivers/watchdog/watchdog_pretimeout.c | 45 ++++++++++++++------------
 include/linux/watchdog.h               |  2 +-
 2 files changed, 26 insertions(+), 21 deletions(-)

diff --git a/drivers/watchdog/watchdog_pretimeout.c b/drivers/watchdog/watchdog_pretimeout.c
index 02e09b9e396d..7fb586c77403 100644
--- a/drivers/watchdog/watchdog_pretimeout.c
+++ b/drivers/watchdog/watchdog_pretimeout.c
@@ -4,6 +4,7 @@
  */
 
 #include <linux/list.h>
+#include <linux/rcupdate.h>
 #include <linux/slab.h>
 #include <linux/spinlock.h>
 #include <linux/string.h>
@@ -67,12 +68,14 @@ int watchdog_pretimeout_available_governors_get(char *buf)
 
 int watchdog_pretimeout_governor_get(struct watchdog_device *wdd, char *buf)
 {
+	const struct watchdog_governor *gov;
 	int count = 0;
 
-	spin_lock_irq(&pretimeout_lock);
-	if (wdd->gov)
-		count = sysfs_emit(buf, "%s\n", wdd->gov->name);
-	spin_unlock_irq(&pretimeout_lock);
+	rcu_read_lock();
+	gov = rcu_dereference(wdd->gov);
+	if (gov)
+		count = sysfs_emit(buf, "%s\n", gov->name);
+	rcu_read_unlock();
 
 	return count;
 }
@@ -91,7 +94,7 @@ int watchdog_pretimeout_governor_set(struct watchdog_device *wdd,
 	}
 
 	spin_lock_irq(&pretimeout_lock);
-	wdd->gov = priv->gov;
+	rcu_assign_pointer(wdd->gov, priv->gov);
 	spin_unlock_irq(&pretimeout_lock);
 
 	mutex_unlock(&governor_lock);
@@ -101,16 +104,13 @@ int watchdog_pretimeout_governor_set(struct watchdog_device *wdd,
 
 void watchdog_notify_pretimeout(struct watchdog_device *wdd)
 {
-	unsigned long flags;
+	const struct watchdog_governor *gov;
 
-	spin_lock_irqsave(&pretimeout_lock, flags);
-	if (!wdd->gov) {
-		spin_unlock_irqrestore(&pretimeout_lock, flags);
-		return;
-	}
-
-	wdd->gov->pretimeout(wdd);
-	spin_unlock_irqrestore(&pretimeout_lock, flags);
+	rcu_read_lock();
+	gov = rcu_dereference(wdd->gov);
+	if (gov)
+		gov->pretimeout(wdd);
+	rcu_read_unlock();
 }
 EXPORT_SYMBOL_GPL(watchdog_notify_pretimeout);
 
@@ -140,8 +140,8 @@ int watchdog_register_governor(struct watchdog_governor *gov)
 		default_gov = gov;
 
 		list_for_each_entry(p, &pretimeout_list, entry)
-			if (!p->wdd->gov)
-				p->wdd->gov = default_gov;
+			if (!rcu_access_pointer(p->wdd->gov))
+				rcu_assign_pointer(p->wdd->gov, default_gov);
 		spin_unlock_irq(&pretimeout_lock);
 	}
 
@@ -170,11 +170,14 @@ void watchdog_unregister_governor(struct watchdog_governor *gov)
 	if (default_gov == gov)
 		default_gov = NULL;
 	list_for_each_entry(p, &pretimeout_list, entry)
-		if (p->wdd->gov == gov)
-			p->wdd->gov = default_gov;
+		if (rcu_dereference_protected(p->wdd->gov,
+					      lockdep_is_held(&pretimeout_lock)) == gov)
+			rcu_assign_pointer(p->wdd->gov, default_gov);
 	spin_unlock_irq(&pretimeout_lock);
 
 	mutex_unlock(&governor_lock);
+
+	synchronize_rcu();
 }
 EXPORT_SYMBOL(watchdog_unregister_governor);
 
@@ -192,7 +195,7 @@ int watchdog_register_pretimeout(struct watchdog_device *wdd)
 	spin_lock_irq(&pretimeout_lock);
 	list_add(&p->entry, &pretimeout_list);
 	p->wdd = wdd;
-	wdd->gov = default_gov;
+	rcu_assign_pointer(wdd->gov, default_gov);
 	spin_unlock_irq(&pretimeout_lock);
 
 	return 0;
@@ -206,7 +209,7 @@ void watchdog_unregister_pretimeout(struct watchdog_device *wdd)
 		return;
 
 	spin_lock_irq(&pretimeout_lock);
-	wdd->gov = NULL;
+	rcu_assign_pointer(wdd->gov, NULL);
 
 	list_for_each_entry_safe(p, t, &pretimeout_list, entry) {
 		if (p->wdd == wdd) {
@@ -216,4 +219,6 @@ void watchdog_unregister_pretimeout(struct watchdog_device *wdd)
 		}
 	}
 	spin_unlock_irq(&pretimeout_lock);
+
+	synchronize_rcu();
 }
diff --git a/include/linux/watchdog.h b/include/linux/watchdog.h
index 29cd03686154..5a3c35968cc8 100644
--- a/include/linux/watchdog.h
+++ b/include/linux/watchdog.h
@@ -105,7 +105,7 @@ struct watchdog_device {
 	const struct attribute_group **groups;
 	const struct watchdog_info *info;
 	const struct watchdog_ops *ops;
-	const struct watchdog_governor *gov;
+	const struct watchdog_governor __rcu *gov;
 	unsigned int bootstatus;
 	unsigned int timeout;
 	unsigned int pretimeout;
-- 
2.56.0.611.g1030f0b940-goog

[PATCH v4 6/7] watchdog: pretimeout_panic: Use nmi_panic() when invoked from NMI context

[Mayank Rungta]

When a watchdog pretimeout interrupt is registered as an NMI (or
pseudo-NMI), pretimeout_panic() executes in NMI context. Calling plain
panic() from NMI context is unsafe if the current CPU was already inside
vpanic() with regular interrupts disabled (where the watchdog is no
longer fed): vpanic() assumes panic_cpu == raw_smp_processor_id() only
occurs when entered via nmi_panic(), so it falls through into a
recursive panic on the same CPU, corrupting the static panic buffer,
panic notifiers, and crash dump state.

Invoke nmi_panic() when in_nmi() is true so that an NMI interrupting a
CPU that is already panicking safely returns, or stops via
nmi_panic_self_stop() if another CPU is panicking.

Signed-off-by: Mayank Rungta <mrungta@google.com>
---
 drivers/watchdog/pretimeout_panic.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/watchdog/pretimeout_panic.c b/drivers/watchdog/pretimeout_panic.c
index 8c3ac674dc45..634ae3b6c961 100644
--- a/drivers/watchdog/pretimeout_panic.c
+++ b/drivers/watchdog/pretimeout_panic.c
@@ -5,7 +5,10 @@
 
 #include <linux/kernel.h>
 #include <linux/module.h>
+#include <linux/panic.h>
+#include <linux/preempt.h>
 #include <linux/watchdog.h>
+#include <asm/irq_regs.h>
 
 #include "watchdog_pretimeout.h"
 
@@ -17,7 +20,10 @@
  */
 static void pretimeout_panic(struct watchdog_device *wdd)
 {
-	panic("watchdog pretimeout event\n");
+	if (in_nmi())
+		nmi_panic(get_irq_regs(), "watchdog pretimeout event\n");
+	else
+		panic("watchdog pretimeout event\n");
 }
 
 static struct watchdog_governor watchdog_gov_panic = {
-- 
2.56.0.611.g1030f0b940-goog

[PATCH v4 7/7] watchdog: qcom: Register pretimeout interrupt as NMI

[Mayank Rungta]

When a system is completely unresponsive due to an interrupt storm or
deadlocked CPU cores with standard interrupts disabled, a standard
watchdog pretimeout bark interrupt will fail to execute, preventing the
pretimeout governor from capturing CPU backtraces before the hardware
reset bite.

Attempt to request the Qualcomm watchdog pretimeout bark interrupt as an
NMI (or pseudo-NMI) using request_nmi(). If NMI registration is not
supported on the platform (e.g. pseudo-NMIs are disabled), gracefully
fall back to a standard interrupt via devm_request_irq().

When NMI is used, enable the NMI during probe to balance IRQF_NO_AUTOEN
and register a devres cleanup action to disable and free the NMI upon
driver unbind.

Reviewed-by: Konrad Dybcio <konrad.dybcio@oss.qualcomm.com>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Signed-off-by: Mayank Rungta <mrungta@google.com>
---
 drivers/watchdog/qcom-wdt.c | 30 ++++++++++++++++++++++++++----
 1 file changed, 26 insertions(+), 4 deletions(-)

diff --git a/drivers/watchdog/qcom-wdt.c b/drivers/watchdog/qcom-wdt.c
index 4eb1bf979012..4a99cf43c4e1 100644
--- a/drivers/watchdog/qcom-wdt.c
+++ b/drivers/watchdog/qcom-wdt.c
@@ -52,6 +52,7 @@ struct qcom_wdt {
 	unsigned long		rate;
 	void __iomem		*base;
 	const u32		*layout;
+	int			irq;
 };
 
 static void __iomem *wdt_addr(struct qcom_wdt *wdt, enum wdt_reg reg)
@@ -74,6 +75,14 @@ static irqreturn_t qcom_wdt_isr(int irq, void *arg)
 	return IRQ_HANDLED;
 }
 
+static void qcom_wdt_free_nmi(void *arg)
+{
+	struct qcom_wdt *wdt = arg;
+
+	disable_nmi(wdt->irq);
+	free_nmi(wdt->irq, &wdt->wdd);
+}
+
 static int qcom_wdt_start(struct watchdog_device *wdd)
 {
 	struct qcom_wdt *wdt = to_qcom_wdt(wdd);
@@ -311,10 +320,23 @@ static int qcom_wdt_probe(struct platform_device *pdev)
 		if (irq < 0 && irq != -ENXIO)
 			return irq;
 		if (irq > 0) {
-			ret = devm_request_irq(dev, irq, qcom_wdt_isr, 0,
-					       "wdt_bark", &wdt->wdd);
-			if (ret)
-				return ret;
+			wdt->irq = irq;
+
+			ret = request_nmi(irq, qcom_wdt_isr, IRQF_NO_AUTOEN,
+					  "wdt_bark", &wdt->wdd);
+			if (ret) {
+				/* Fallback to normal interrupt if NMI not supported */
+				ret = devm_request_irq(dev, irq, qcom_wdt_isr, 0,
+						       "wdt_bark", &wdt->wdd);
+				if (ret)
+					return ret;
+			} else {
+				enable_nmi(irq);
+				ret = devm_add_action_or_reset(dev, qcom_wdt_free_nmi,
+							       wdt);
+				if (ret)
+					return ret;
+			}
 
 			wdt->wdd.info = &qcom_wdt_pt_info;
 			wdt->wdd.pretimeout = 1;
-- 
2.56.0.611.g1030f0b940-goog