From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4A51835FF6C; Fri, 9 Oct 2026 19:03:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791572596; cv=none; b=ZVVkuqPJ4Ve9xnF5NBae0+kwEG8LHjE3Mh4XxFJkyRPPTnbA6Z3K4sjBGv9g3Wrd3gzDEWLDKnVTqMjLFrXMAlYf0SDvCApNS2LCHZ8czJoH7dEsIxl3SfI7vOHfK7rXEo+XK5+1IyL6PqBC8KjVusAUXCR6sLfazy9nzaHao0M= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791572596; c=relaxed/simple; bh=miGuP0P2RpS2il8lVGOuOHXG1eLSz6RCzg09nOBAtvg=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=CBGmEbhTlJHkP52fvWSZMPovGvufiLtRDypeX4ne6iShj5OmsIrg3X0PtGvB4yL+OuZ1ToowyvvzawK/NIFXzo3gCiTZd7oqgNw9DdPJeyST831cX1GU4M/xBwTxPuNJITegIkCuQsnNVZeUs7Cs+SM3bCwiCil3N23zNRK6D9c= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=XFr88NQ+; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="XFr88NQ+" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E81BA1F00898; Fri, 9 Oct 2026 19:03:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1791572594; bh=TirdlCwzZ2cMnBXETSVn2wNFY8bO/3AuUeLLqbi+80w=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=XFr88NQ+TOXU9e4jM3Sku3N23cc2Z0FlIf41klvBBxFfn7opUfl6mqM7XTfAe7t66 Bwm+HBpWHvhoEAg9ztR3w5rRCOuPzlNbIRIzgw4oYZXmxm1pjIjtTuPBV7oOuqQcO+ zXK48eT943s784+SkktVSzgJyu3t4TiSOsl33stMQ+DrQr0hRyDJC178X0h/ih4QvY dCgbQl1oYdrA38feZKWPMlhj2/Wm+zeyRG3FAktp6DpXCyfadsQfC4dysoym3htxro 0PN3h65Zw8aqxuyadA8XmmfAZXNKGiRV8rY+NliN9owq2ALNty8bWTXhTGy30EI3wr 4ntoMJ+EFymgA== Received: from phl-compute-09.internal (phl-compute-09.internal [10.202.2.49]) by mailfauth.phl.internal (Postfix) with ESMTP id 26201F40066; Fri, 9 Oct 2026 15:03:12 -0400 (EDT) Received: from phl-frontend-03 ([10.202.2.162]) by phl-compute-09.internal (MEProxy); Fri, 09 Oct 2026 15:03:12 -0400 X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: dmFkZTGtq7JcenuwH9YCrRuUTTxkXWfMKXOv9XLn5AIx7CUcTLanJrTmpLXaH7tXe29Thu 4+EUJ0wq5iP3vYpD9dYfrqbBxeFyum1AHUKkNwellh8jXZyMFNUn2klB0HX8kimMTo+jCm orPPwBk7KiiZY3aSSc7xYRDc4junoAPOnmYMSLf7DQDHZr9TcNr2sh9+CyU8GFRY844FCf u9Acoo2q0GGZ4sI56HUOSyhyX7ItBVOZ3AlRYVVT/bFfOEynP3wwrCKEwYL1dzy07J4bKw tQX1exdPDTa2fjNkSiGSGlkXbHVDf2/f7FPNBSmWvIGPIGmV0Z8PRN8ClzazP6Wfe2TwlS aDo6MzGql/jk+Oha8Jymb5DSa/R2gx9s5EnNeC6U9crZ2kw4IqGZKLaO5ELmZWfPGmZy2H qNYwLnZstHYOf4nuJsyrajMKOKozOuPZrtcDeGmXzmBnRReaYe3oqb1xd7XLCQ9LXHPmtq ZWRalxK2C1eZy4tmXsAZCONcDyxk6q1wnq2EdpG9WmIv0/JcOgOreDx24hKOhZ3rttZEN9 QfLZhGmE6FTHMO9wZjN47up4l76o/u/5ALRX7gmQ+iUYuyREBhEWhmKXCd+enUvx0A7nxF mP64lTmwb8z+5VypLLfJDeDdWXFX+6NMmPQo2LnIj82OboLukcomSNJ3pxHQ X-ME-Proxy: Feedback-ID: i8dbe485b:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 9 Oct 2026 15:03:11 -0400 (EDT) Date: Fri, 9 Oct 2026 12:03:10 -0700 From: Boqun Feng To: Mathieu Desnoyers Cc: "Paul E . McKenney" , linux-kernel@vger.kernel.org, Linus Torvalds , Boqun Feng , Greg Kroah-Hartman , Sebastian Andrzej Siewior , Will Deacon , Peter Zijlstra , Alan Stern , John Stultz , Frederic Weisbecker , Joel Fernandes , Josh Triplett , Uladzislau Rezki , Steven Rostedt , Lai Jiangshan , Zqiang , Ingo Molnar , Waiman Long , Mark Rutland , Thomas Gleixner , Vlastimil Babka , maged.michael@gmail.com, Mateusz Guzik , Gary Guo , rcu@vger.kernel.org, linux-mm@kvack.org, lkmm@lists.linux.dev, Nikita Popov , llvm@lists.linux.dev, Lian Wang , Kunwu Chan Subject: Re: [PATCH hazptr v2 2/4] ptreq.h: Introduce ptr_eq() to preserve address dependency Message-ID: References: <20261009182142.6311-1-mathieu.desnoyers@efficios.com> <20261009182142.6311-3-mathieu.desnoyers@efficios.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20261009182142.6311-3-mathieu.desnoyers@efficios.com> On Fri, Oct 09, 2026 at 02:21:33PM -0400, Mathieu Desnoyers wrote: > Compiler CSE and SSA GVN optimizations can cause the address dependency > of addresses returned by rcu_dereference to be lost when comparing those > pointers with either constants or previously loaded pointers. > > Introduce ptr_eq() to compare two addresses while preserving the address > dependencies for later use of the address. It should be used when > comparing an address returned by rcu_dereference(). > > This is needed to prevent the compiler CSE and SSA GVN optimizations > from using @a (or @b) in places where the source refers to @b (or @a) > based on the fact that after the comparison, the two are known to be > equal, which does not preserve address dependencies and allows the > following misordering speculations: > > - If @b is a constant, the compiler can issue the loads which depend > on @a before loading @a. > - If @b is a register populated by a prior load, weakly-ordered > CPUs can speculate loads which depend on @a before loading @a. > > The same logic applies with @a and @b swapped. > > The header "linux/ptreq.h" is the expected include target. It > contains the documentation of the ptr_eq() API. > > An architecture specific implementation of the pointer comparison > can be implemented by each architecture as asm/ptreq.h. The x86 > implementation is provided initially. If no implementation header is > present for the architecture, an arch-agnostic fallback based on > OPTIMIZER_HIDE_VAR is included from asm-generic. > > Suggested-by: Linus Torvalds > Suggested-by: Boqun Feng > Signed-off-by: Mathieu Desnoyers > Cc: Greg Kroah-Hartman > Cc: Sebastian Andrzej Siewior > Cc: "Paul E. McKenney" > Cc: Will Deacon > Cc: Peter Zijlstra > Cc: Boqun Feng > Cc: Alan Stern > Cc: John Stultz > Cc: Linus Torvalds > Cc: Boqun Feng > Cc: Frederic Weisbecker > Cc: Joel Fernandes > Cc: Josh Triplett > Cc: Uladzislau Rezki > Cc: Steven Rostedt > Cc: Lai Jiangshan > Cc: Zqiang > Cc: Ingo Molnar > Cc: Waiman Long > Cc: Mark Rutland > Cc: Thomas Gleixner > Cc: Vlastimil Babka > Cc: maged.michael@gmail.com > Cc: Mateusz Guzik > Cc: Gary Guo > Cc: rcu@vger.kernel.org > Cc: linux-mm@kvack.org > Cc: lkmm@lists.linux.dev > Cc: Nikita Popov > Cc: llvm@lists.linux.dev > Cc: Lian Wang > Cc: Kunwu Chan > --- > Changes since v1: > - Move to linux/ptreq.h, implement x86-specific comparison I prefer the name ptr_eq.h over ptreq.h :) With that name fix or there is a strong reason of ptreq.h: Reviewed-by: Boqun Feng Regards, Boqun > and asm-generic header (fallback). > > Changes since v0: > - Include feedback from Alan Stern. > --- > arch/x86/include/asm/ptreq.h | 16 +++++++++ > include/asm-generic/Kbuild | 1 + > include/asm-generic/ptreq.h | 16 +++++++++ > include/linux/ptreq.h | 63 ++++++++++++++++++++++++++++++++++++ > 4 files changed, 96 insertions(+) > create mode 100644 arch/x86/include/asm/ptreq.h > create mode 100644 include/asm-generic/ptreq.h > create mode 100644 include/linux/ptreq.h > > diff --git a/arch/x86/include/asm/ptreq.h b/arch/x86/include/asm/ptreq.h > new file mode 100644 > index 000000000000..593b45f80451 > --- /dev/null > +++ b/arch/x86/include/asm/ptreq.h > @@ -0,0 +1,16 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > +#ifndef _ASM_X86_PTREQ_H > +#define _ASM_X86_PTREQ_H > + > +#include > +#include > + > +static __always_inline > +bool ptr_eq(const volatile void *a, const volatile void *b) > +{ > + bool ret; > + asm(__ASM_SIZE(cmp) " %1,%2" : "=@ccz" (ret) : "r" (a), "r" (b)); > + return ret; > +} > + > +#endif /* _ASM_X86_PTREQ_H */ > diff --git a/include/asm-generic/Kbuild b/include/asm-generic/Kbuild > index 2bc00c67dc54..e1d95dea09d5 100644 > --- a/include/asm-generic/Kbuild > +++ b/include/asm-generic/Kbuild > @@ -47,6 +47,7 @@ mandatory-y += percpu.h > mandatory-y += percpu_types.h > mandatory-y += pgalloc.h > mandatory-y += preempt.h > +mandatory-y += ptreq.h > mandatory-y += rqspinlock.h > mandatory-y += runtime-const.h > mandatory-y += rwonce.h > diff --git a/include/asm-generic/ptreq.h b/include/asm-generic/ptreq.h > new file mode 100644 > index 000000000000..db5a88628b1e > --- /dev/null > +++ b/include/asm-generic/ptreq.h > @@ -0,0 +1,16 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > +#ifndef __ASM_GENERIC_PTREQ_H > +#define __ASM_GENERIC_PTREQ_H > + > +#include > +#include > + > +static __always_inline > +bool ptr_eq(const volatile void *a, const volatile void *b) > +{ > + OPTIMIZER_HIDE_VAR(a); > + OPTIMIZER_HIDE_VAR(b); > + return a == b; > +} > + > +#endif /* __ASM_GENERIC_PTREQ_H */ > diff --git a/include/linux/ptreq.h b/include/linux/ptreq.h > new file mode 100644 > index 000000000000..5499f65da30a > --- /dev/null > +++ b/include/linux/ptreq.h > @@ -0,0 +1,63 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > +#ifndef _LINUX_PTREQ_H > +#define _LINUX_PTREQ_H > + > +/* > + * Compare two addresses while preserving the address dependencies for > + * later use of the address. It should be used when comparing an address > + * returned by rcu_dereference(). > + * > + * This is needed to prevent the compiler CSE and SSA GVN optimizations > + * from using @a (or @b) in places where the source refers to @b (or @a) > + * based on the fact that after the comparison, the two are known to be > + * equal, which does not preserve address dependencies and allows the > + * following misordering speculations: > + * > + * - If @b is a constant, the compiler can issue the loads which depend > + * on @a before loading @a. > + * - If @b is a register populated by a prior load, weakly-ordered > + * CPUs can speculate loads which depend on @a before loading @a. > + * > + * The same logic applies with @a and @b swapped. > + * > + * Return value: true if pointers are equal, false otherwise. > + * > + * The compiler barrier() is ineffective at fixing this issue. It does > + * not prevent the compiler CSE from losing the address dependency: > + * > + * int fct_2_volatile_barriers(void) > + * { > + * int *a, *b; > + * > + * do { > + * a = READ_ONCE(p); > + * asm volatile ("" : : : "memory"); > + * b = READ_ONCE(p); > + * } while (a != b); > + * asm volatile ("" : : : "memory"); <-- barrier() > + * return *b; > + * } > + * > + * With gcc 14.2 (arm64): > + * > + * fct_2_volatile_barriers: > + * adrp x0, .LANCHOR0 > + * add x0, x0, :lo12:.LANCHOR0 > + * .L2: > + * ldr x1, [x0] <-- x1 populated by first load. > + * ldr x2, [x0] > + * cmp x1, x2 > + * bne .L2 > + * ldr w0, [x1] <-- x1 is used for access which should depend on b. > + * ret > + * > + * On weakly-ordered architectures, this lets CPU speculation use the > + * result from the first load to speculate "ldr w0, [x1]" before > + * "ldr x2, [x0]". > + * Based on the RCU documentation, the control dependency does not > + * prevent the CPU from speculating loads. > + */ > + > +#include > + > +#endif /* _LINUX_PTREQ_H */ > -- > 2.43.0 >