mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: shechenglong <shechenglong@xfusion.com>
To: <catalin.marinas@arm.com>
Cc: <will@kernel.org>, <linux-arm-kernel@lists.infradead.org>,
	<linux-kernel@vger.kernel.org>, <stone.xulei@xfusion.com>,
	<chenjialong@xfusion.com>, <yuxiating@xfusion.com>,
	shechenglong <shechenglong@xfusion.com>
Subject: [PATCH] cpu: fix hard lockup triggered during stress-ng stress testing.
Date: Thu, 18 Sep 2025 14:49:07 +0800	[thread overview]
Message-ID: <20250918064907.1832-1-shechenglong@xfusion.com> (raw)

Context of the Issue:
In an ARM64 environment, the following steps were performed:

1. Repeatedly ran stress-ng to stress the CPU, memory, and I/O.
2. Cyclically executed test case pty06 from the LTP test suite.
3. Added mitigations=off to the GRUB parameters.

After 1–2 hours of stress testing, a hardlockup occurred,
causing a system crash.

Root Cause of the Hardlockup:
Each time stress-ng starts, it invokes the /sys/kernel/debug/clear_warn_once
interface, which clears the values in the memory section from __start_once
to __end_once. This caused functions like pr_info_once() — originally
designed to print only once — to print again every time stress-ng was called.
If the pty06 test case happened to be using the serial module at that same
moment, it would sleep in waiter.list within the __down_common function.

After pr_info_once() completed its output using the serial module,
it invoked the semaphore up() function to wake up the process waiting
in waiter.list. This sequence triggered an A-A deadlock, ultimately
leading to a hardlockup and system crash.

To prevent this, a local variable should be used to control and ensure
the print operation occurs only once.

Hard lockup call stack:

_raw_spin_lock_nested+168
ttwu_queue+180 (rq_lock(rq, &rf); 2nd acquiring the rq->__lock)
try_to_wake_up+548
wake_up_process+32
__up+88
up+100
__up_console_sem+96
console_unlock+696
vprintk_emit+428
vprintk_default+64
vprintk_func+220
printk+104
spectre_v4_enable_task_mitigation+344
__switch_to+100
__schedule+1028 (rq_lock(rq, &rf); 1st acquiring the rq->__lock)
schedule_idle+48
do_idle+388
cpu_startup_entry+44
secondary_start_kernel+352

Signed-off-by: shechenglong <shechenglong@xfusion.com>
---
 arch/arm64/kernel/proton-pack.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kernel/proton-pack.c b/arch/arm64/kernel/proton-pack.c
index edf1783ffc81..f8663157e041 100644
--- a/arch/arm64/kernel/proton-pack.c
+++ b/arch/arm64/kernel/proton-pack.c
@@ -424,8 +424,10 @@ static bool spectre_v4_mitigations_off(void)
 	bool ret = cpu_mitigations_off() ||
 		   __spectre_v4_policy == SPECTRE_V4_POLICY_MITIGATION_DISABLED;
 
-	if (ret)
-		pr_info_once("spectre-v4 mitigation disabled by command-line option\n");
+	static atomic_t __printk_once = ATOMIC_INIT(0);
+
+	if (ret && !atomic_cmpxchg(&__printk_once, 0, 1))
+		pr_info("spectre-v4 mitigation disabled by command-line option\n");
 
 	return ret;
 }
-- 
2.33.0


             reply	other threads:[~2025-09-18  7:07 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-09-18  6:49 shechenglong [this message]
2025-09-18 11:28 ` Catalin Marinas
2025-09-19 12:05   ` 答复: " shechenglong
2025-09-22 16:54     ` Catalin Marinas
2025-09-22 16:08   ` Mark Rutland
2025-09-24 12:32 ` [PATCH] cpu: fix hard lockup triggered by printk calls within scheduling context shechenglong
2025-09-25 13:48   ` Catalin Marinas
2025-10-03 14:23   ` Will Deacon
2025-10-20 14:51 ` [PATCH v2 0/2] arm64: spectre: Fix hard lockup and cleanup mitigation messages shechenglong
2025-10-20 14:51   ` [PATCH v2 1/2] cpu:Remove the print when the CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY Kconfig option is disabled shechenglong
2025-10-20 14:51   ` [PATCH v2 2/2] cpu: fix hard lockup triggered by printk calls within scheduling context shechenglong
2025-10-29  3:45 ` [RESEND v2 0/2] arm64: spectre: Fix hard lockup and cleanup mitigation messages shechenglong
2025-10-29  3:45   ` [PATCH v2 1/2] cpu:Remove the print when the CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY Kconfig option is disabled shechenglong
2025-10-30 14:48     ` Will Deacon
2025-10-29  3:45   ` [PATCH v2 2/2] cpu: fix hard lockup triggered by printk calls within scheduling context shechenglong
2025-10-30 14:50     ` Will Deacon
2025-10-31  9:15 ` [PATCH v3 0/2] arm64: spectre: Fix hard lockup and cleanup mitigation messages shechenglong
2025-10-31  9:15   ` [PATCH v3 1/2] cpu:Remove the print when the CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY Kconfig option is disabled shechenglong
2025-10-31  9:15   ` [PATCH v3 2/2] cpu: fix hard lockup triggered by printk calls within scheduling context shechenglong
2025-11-07 15:53   ` [PATCH v3 0/2] arm64: spectre: Fix hard lockup and cleanup mitigation messages Will Deacon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20250918064907.1832-1-shechenglong@xfusion.com \
    --to=shechenglong@xfusion.com \
    --cc=catalin.marinas@arm.com \
    --cc=chenjialong@xfusion.com \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=stone.xulei@xfusion.com \
    --cc=will@kernel.org \
    --cc=yuxiating@xfusion.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®