From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3304E32B9A9; Mon, 29 Jun 2026 13:57:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782741467; cv=none; b=G8AD9qIqMAkZ/5Fl+vhfQl4GJBP19/FYMeMytETZs03zM2+wMLNxPel4EocusXzbf5/EeiTCIo5JyzgpMMTs5FwLrV0NtCtV7uNRQbMDQHoclB0NssWF0gv9h9z5yBM7nHbr/kPaB7Xm8ng2MbER6Kq+5gBB2E/nwHaAM+BEeQc= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782741467; c=relaxed/simple; bh=4wO9SYVs5If81prfDaLq3ORA8aK/jf2wOzwQJ3Tjk5I=; h=Date:From:To:Subject:Cc:In-Reply-To:References:MIME-Version: Message-ID:Content-Type; b=fTQwoad+zY0x7j11Pp5fKVDircP7I773dxmEDaLYr9xUeXDnorG+9sLwwsvkqhxAD4o/ckzrnbWfFFHP/5I4nvNSkUdQFJqAfpJH7axQLfhZABJjfUiIy/kIkh6Lpdh/85vy1OZacS4RuA+TKNc4NaS/p0iardh6CtfOl9RvhEg= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=PhsBE2/x; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=XrHySHBV; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="PhsBE2/x"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="XrHySHBV" Date: Mon, 29 Jun 2026 13:57:42 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1782741463; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fF5sEnfT4wWx2JCNF/W3RzRNTJ1mDmQOMX+Y/lK2SOk=; b=PhsBE2/xlyCKtKjJ0v8pe8xpJbFNW0MzOrWWEjp33whI7peRolYmen+f7Dz9QVYcnvQRAi n7u8hfOuPpdfGt3+Z6x8ReO38bA8rdilSdve0SzFrpyfl7bfKj8qx57OKxBoUIrqtNO388 qjaXmP3qBBHTVQGVT3reTW+2z4ZxDP5BndD9zTaTuPNKQ1SIXemeTmr9GMoqJDbiOyX4RB 3g7d+sqVRaNgQwfU/zYQZzad3kT/NpXv6oVC/x4YzpoD1KAXIVVCU4z8Ieu3PjZfcy1KjP 16A6IlsHQMy69orWfuLQ33P3vMofo4lOZkb3uAWMBvok/ySAvnIgTC1UBIAL4Q== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1782741463; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fF5sEnfT4wWx2JCNF/W3RzRNTJ1mDmQOMX+Y/lK2SOk=; b=XrHySHBVRfu1wDh1y4QKX2m3o6r/aQWeDRhhvLx6IQJ0CJo03E8r4vGUsiRFd9VCyYHa2A w54zJ96t5mNWzJBA== From: "tip-bot2 for Usama Arif" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: smp/core] smp: Use release stores for csd_lock_record() state Cc: Usama Arif , Thomas Gleixner , Dmitry Ilvokhin , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20260629105745.1696683-1-usama.arif@linux.dev> References: <20260629105745.1696683-1-usama.arif@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <178274146231.3843924.7768569808457911760.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Precedence: bulk Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The following commit has been merged into the smp/core branch of tip: Commit-ID: ec9f57e6fefbc0497b9f047ebe60ef9adaae7480 Gitweb: https://git.kernel.org/tip/ec9f57e6fefbc0497b9f047ebe60ef9adaa= e7480 Author: Usama Arif AuthorDate: Mon, 29 Jun 2026 03:57:45 -07:00 Committer: Thomas Gleixner CommitterDate: Mon, 29 Jun 2026 15:53:14 +02:00 smp: Use release stores for csd_lock_record() state __csd_lock_record() publishes per-CPU diagnostic state (cur_csd, cur_csd_func, cur_csd_info) that is consumed from a remote CPU by csd_lock_wait_toolong() via smp_load_acquire(&cur_csd). To order the matching cur_csd_func/cur_csd_info stores before the cur_csd publication, the producer issues smp_wmb() before writing cur_csd; to order the publication before the subsequent callback execution or CSD unlock, it issues smp_mb() after the write. The clear path mirrors this with smp_mb() before storing NULL into cur_csd so the preceding callback/unlock is observed first. The smp_mb() pair is heavier than what the consumer actually requires (on x86 each emits a locked full barrier). The consumer only needs to observe the matching cur_csd_func/cur_csd_info when it sees a non-NULL cur_csd, and to observe the preceding callback/unlock when it sees NULL -- both of which a release/acquire pair provides. The extra two-way ordering enforced by smp_mb() -- that cur_csd publication be observed before callback execution or unlock becomes visible -- would only matter if cur_csd were an exact live-state marker. csd_lock_wait_toolong() does not treat it that way: it snapshots cur_csd via smp_load_acquire() and then prints / dumps / re-IPIs without an RCU-style stall-ended recheck, so the diagnostic already tolerates the remote CPU completing its work between snapshot and report. cur_csd is best-effort context, not a precise stall boundary. Replace the smp_wmb() + plain store + smp_mb() in the publish path, and the smp_mb() + plain store in the clear path, with smp_store_release(). This pairs with the smp_load_acquire() in csd_lock_wait_toolong(): preceding cur_csd_func/cur_csd_info stores become visible before a remote reader observes the non-NULL publication, and any preceding callback/unlock becomes visible before a reader observes the NULL clear. Signed-off-by: Usama Arif Signed-off-by: Thomas Gleixner Reviewed-by: Dmitry Ilvokhin Link: https://patch.msgid.link/20260629105745.1696683-1-usama.arif@linux.dev --- kernel/smp.c | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/kernel/smp.c b/kernel/smp.c index a0bb56b..8a847a3 100644 --- a/kernel/smp.c +++ b/kernel/smp.c @@ -182,16 +182,22 @@ static atomic_t csd_bug_count =3D ATOMIC_INIT(0); static void __csd_lock_record(call_single_data_t *csd) { if (!csd) { - smp_mb(); /* NULL cur_csd after unlock. */ - __this_cpu_write(cur_csd, NULL); + /* + * Pairs with smp_load_acquire() of cur_csd in + * csd_lock_wait_toolong(): orders any preceding CSD + * callback/unlock before a remote reader observes NULL. + */ + smp_store_release(this_cpu_ptr(&cur_csd), NULL); return; } __this_cpu_write(cur_csd_func, csd->func); __this_cpu_write(cur_csd_info, csd->info); - smp_wmb(); /* func and info before csd. */ - __this_cpu_write(cur_csd, csd); - smp_mb(); /* Update cur_csd before function call. */ - /* Or before unlock, as the case may be. */ + /* + * Pairs with smp_load_acquire() of cur_csd in + * csd_lock_wait_toolong(): publishes cur_csd_func and + * cur_csd_info before the non-NULL pointer becomes visible. + */ + smp_store_release(this_cpu_ptr(&cur_csd), csd); } =20 static __always_inline void csd_lock_record(call_single_data_t *csd) @@ -272,7 +278,13 @@ static bool csd_lock_wait_toolong(call_single_data_t *cs= d, u64 ts0, u64 *ts1, in cpux =3D 0; else cpux =3D cpu; - cpu_cur_csd =3D smp_load_acquire(&per_cpu(cur_csd, cpux)); /* Before func a= nd info. */ + /* + * Pairs with smp_store_release() of cur_csd in __csd_lock_record(): + * a non-NULL cur_csd here implies cur_csd_func and cur_csd_info + * are the matching publication; a NULL value is ordered after any + * preceding CSD callback/unlock on the remote CPU. + */ + cpu_cur_csd =3D smp_load_acquire(&per_cpu(cur_csd, cpux)); /* How long since this CSD lock was stuck. */ ts_delta =3D ts2 - ts0; pr_alert("csd: %s non-responsive CSD lock (#%d) on CPU#%d, waiting %lld ns = for CPU#%02d %pS(%ps).\n",