mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Jinchao Wang <wangjinchao600@gmail.com>
To: "Andrew Morton" <akpm@linux-foundation.org>,
	"Baoquan He" <bhe@redhat.com>,
	"Yury Norov" <yury.norov@gmail.com>,
	"Qianqiang Liu" <qianqiang.liu@163.com>,
	"Simona Vetter" <simona@ffwll.ch>, "Helge Deller" <deller@gmx.de>,
	"Petr Mladek" <pmladek@suse.com>,
	"Steven Rostedt" <rostedt@goodmis.org>,
	"John Ogness" <john.ogness@linutronix.de>,
	"Sergey Senozhatsky" <senozhatsky@chromium.org>,
	"Vivek Goyal" <vgoyal@redhat.com>,
	"Dave Young" <dyoung@redhat.com>, "Kees Cook" <kees@kernel.org>,
	"Tony Luck" <tony.luck@intel.com>,
	"Guilherme G. Piccoli" <gpiccoli@igalia.com>,
	"Thomas Zimmermann" <tzimmermann@suse.de>,
	"Ville Syrjälä" <ville.syrjala@linux.intel.com>,
	"Shixiong Ou" <oushixiong@kylinos.cn>,
	"Jinchao Wang" <wangjinchao600@gmail.com>,
	"Zsolt Kajtar" <soci@c64.rulez.org>,
	"Ingo Molnar" <mingo@kernel.org>,
	"Nam Cao" <namcao@linutronix.de>,
	"Jonathan Cameron" <Jonathan.Cameron@huawei.com>,
	"Joel Fernandes" <joelagnelf@nvidia.com>,
	"Joel Granados" <joel.granados@kernel.org>,
	"Jason Gunthorpe" <jgg@ziepe.ca>,
	"Sohil Mehta" <sohil.mehta@intel.com>,
	"Feng Tang" <feng.tang@linux.alibaba.com>,
	"Sravan Kumar Gundu" <sravankumarlpu@gmail.com>,
	"Douglas Anderson" <dianders@chromium.org>,
	"Thomas Gleixner" <tglx@linutronix.de>,
	"Anna Schumaker" <anna.schumaker@oracle.com>,
	"Darrick J. Wong" <djwong@kernel.org>,
	"Max Kellermann" <max.kellermann@ionos.com>,
	"Yunhui Cui" <cuiyunhui@bytedance.com>,
	"Tejun Heo" <tj@kernel.org>,
	"Luo Gengkun" <luogengkun@huaweicloud.com>,
	"Li Huafei" <lihuafei1@huawei.com>,
	"Thorsten Blum" <thorsten.blum@linux.dev>,
	"Yicong Yang" <yangyicong@hisilicon.com>,
	linux-fbdev@vger.kernel.org, dri-devel@lists.freedesktop.org,
	kexec@lists.infradead.org, linux-hardening@vger.kernel.org,
	linux-kernel@vger.kernel.org
Subject: [PATCH v2 9/9] watchdog: skip checks when panic is in progress
Date: Mon, 25 Aug 2025 10:29:37 +0800	[thread overview]
Message-ID: <20250825022947.1596226-10-wangjinchao600@gmail.com> (raw)
In-Reply-To: <20250825022947.1596226-1-wangjinchao600@gmail.com>

This issue was found when an EFI pstore was configured for kdump
logging with the NMI hard lockup detector enabled. The efi-pstore
write operation was slow, and with a large number of logs, the
pstore dump callback within kmsg_dump() took a long time.

This delay triggered the NMI watchdog, leading to a nested panic.
The call flow demonstrates how the secondary panic caused an
emergency_restart() to be triggered before the initial pstore
operation could finish, leading to a failure to dump the logs:

  real panic() {
	kmsg_dump() {
		...
		pstore_dump() {
			start_dump();
			... // long time operation triggers NMI watchdog
			nmi panic() {
				...
				emergency_restart(); // pstore unfinished
			}
			...
			finish_dump(); // never reached
		}
	}
  }

Both watchdog_buddy_check_hardlockup() and watchdog_overflow_callback() may
trigger during a panic. This can lead to recursive panic handling.

Add panic_in_progress() checks so watchdog activity is skipped once a panic
has begun.

This prevents recursive panic and keeps the panic path more reliable.

Signed-off-by: Jinchao Wang <wangjinchao600@gmail.com>
Reviewed-by: Yury Norov (NVIDIA) <yury.norov@gmail.com>
---
 kernel/watchdog.c      | 6 ++++++
 kernel/watchdog_perf.c | 4 ++++
 2 files changed, 10 insertions(+)

diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index 80b56c002c7f..597c0d947c93 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -740,6 +740,12 @@ static enum hrtimer_restart watchdog_timer_fn(struct hrtimer *hrtimer)
 	if (!watchdog_enabled)
 		return HRTIMER_NORESTART;
 
+	/*
+	 * pass the buddy check if a panic is in process
+	 */
+	if (panic_in_progress())
+		return HRTIMER_NORESTART;
+
 	watchdog_hardlockup_kick();
 
 	/* kick the softlockup detector */
diff --git a/kernel/watchdog_perf.c b/kernel/watchdog_perf.c
index 9c58f5b4381d..d3ca70e3c256 100644
--- a/kernel/watchdog_perf.c
+++ b/kernel/watchdog_perf.c
@@ -12,6 +12,7 @@
 
 #define pr_fmt(fmt) "NMI watchdog: " fmt
 
+#include <linux/panic.h>
 #include <linux/nmi.h>
 #include <linux/atomic.h>
 #include <linux/module.h>
@@ -108,6 +109,9 @@ static void watchdog_overflow_callback(struct perf_event *event,
 	/* Ensure the watchdog never gets throttled */
 	event->hw.interrupts = 0;
 
+	if (panic_in_progress())
+		return;
+
 	if (!watchdog_check_timestamp())
 		return;
 
-- 
2.43.0


      parent reply	other threads:[~2025-08-25  2:32 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-08-25  2:29 [PATCH v2 0/9] panic: introduce panic status function family Jinchao Wang
2025-08-25  2:29 ` [PATCH v2 1/9] panic: Introduce helper functions for panic state Jinchao Wang
2025-09-16  9:57   ` Petr Mladek
2025-09-17  2:09     ` Jinchao Wang
2025-08-25  2:29 ` [PATCH v2 2/9] fbdev: Use panic_in_progress() helper Jinchao Wang
2025-08-25  2:29 ` [PATCH v2 3/9] crash_core: use panic_try_start() in crash_kexec() Jinchao Wang
2025-08-29  2:30   ` Qianqiang Liu
2025-08-29  4:39     ` Jinchao Wang
2025-08-25  2:29 ` [PATCH v2 4/9] panic: use panic_try_start() in nmi_panic() Jinchao Wang
2025-08-25  2:29 ` [PATCH v2 5/9] panic: use panic_try_start() in vpanic() Jinchao Wang
2025-09-16 11:20   ` Petr Mladek
2025-08-25  2:29 ` [PATCH v2 6/9] printk/nbcon: use panic_on_this_cpu() helper Jinchao Wang
2025-09-16 11:21   ` Petr Mladek
2025-08-25  2:29 ` [PATCH v2 7/9] panic/printk: replace this_cpu_in_panic() with panic_on_this_cpu() Jinchao Wang
2025-09-16 11:22   ` Petr Mladek
2025-08-25  2:29 ` [PATCH v2 8/9] panic/printk: replace other_cpu_in_panic() with panic_on_other_cpu() Jinchao Wang
2025-09-16 11:23   ` Petr Mladek
2025-08-25  2:29 ` Jinchao Wang [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20250825022947.1596226-10-wangjinchao600@gmail.com \
    --to=wangjinchao600@gmail.com \
    --cc=Jonathan.Cameron@huawei.com \
    --cc=akpm@linux-foundation.org \
    --cc=anna.schumaker@oracle.com \
    --cc=bhe@redhat.com \
    --cc=cuiyunhui@bytedance.com \
    --cc=deller@gmx.de \
    --cc=dianders@chromium.org \
    --cc=djwong@kernel.org \
    --cc=dri-devel@lists.freedesktop.org \
    --cc=dyoung@redhat.com \
    --cc=feng.tang@linux.alibaba.com \
    --cc=gpiccoli@igalia.com \
    --cc=jgg@ziepe.ca \
    --cc=joel.granados@kernel.org \
    --cc=joelagnelf@nvidia.com \
    --cc=john.ogness@linutronix.de \
    --cc=kees@kernel.org \
    --cc=kexec@lists.infradead.org \
    --cc=lihuafei1@huawei.com \
    --cc=linux-fbdev@vger.kernel.org \
    --cc=linux-hardening@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=luogengkun@huaweicloud.com \
    --cc=max.kellermann@ionos.com \
    --cc=mingo@kernel.org \
    --cc=namcao@linutronix.de \
    --cc=oushixiong@kylinos.cn \
    --cc=pmladek@suse.com \
    --cc=qianqiang.liu@163.com \
    --cc=rostedt@goodmis.org \
    --cc=senozhatsky@chromium.org \
    --cc=simona@ffwll.ch \
    --cc=soci@c64.rulez.org \
    --cc=sohil.mehta@intel.com \
    --cc=sravankumarlpu@gmail.com \
    --cc=tglx@linutronix.de \
    --cc=thorsten.blum@linux.dev \
    --cc=tj@kernel.org \
    --cc=tony.luck@intel.com \
    --cc=tzimmermann@suse.de \
    --cc=vgoyal@redhat.com \
    --cc=ville.syrjala@linux.intel.com \
    --cc=yangyicong@hisilicon.com \
    --cc=yury.norov@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®