From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AE8571D555 for ; Fri, 20 Feb 2026 06:33:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771569185; cv=none; b=fAKgATLsyp9wF4VlUiw58kkwaoKuON+Uk5uqgMQzZf0hOBKcM2KJwabrV551MZcdRAWVGVo0WNqRWX+yc3X0KRQjFyz0/nSPTc16sE/GbRfKNNULqedYM2UTF5QXM8zT2wQgmxxUHSd1a4HdsC5TjFasB5uVa6rfKARuYkokdnM= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771569185; c=relaxed/simple; bh=Fq+HYBDt87eoP9eXSUrmnldyYQ/rSkSpbFxYlUHf1BA=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=d++OiigQspIVsDBqdh1zECY8hPqmNn5hl7t/jTnjxbJ5Y4PGTydfauOIenW2RJJdZ7c7eAkO3Peq38TS4iAYbI+BWIF7Km7vLPDQ2/ZUbwq+6x+5uRhEFqrzMwTS8iFgWqi7w/ueHRH0oFZAoYwYeYEPoNH31p3lvnLcI7JabXE= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=MSQ82ZE3; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="MSQ82ZE3" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5F377C116D0; Fri, 20 Feb 2026 06:33:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1771569185; bh=Fq+HYBDt87eoP9eXSUrmnldyYQ/rSkSpbFxYlUHf1BA=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=MSQ82ZE30LmJrrKD9PVuqMqIuJvDK2CtgGY9C3wBZmy4pRdK/jNaqXvgw0euL+3uh TWiBisItCbT2PAjp8NXubp7LyHSMhZc91THpIb8TkiRcCLDucjDn5u7r7E//A0xZjg z2GDqgTGQVldYvJiVPtoWJVvZUOTLQ2gU7batWFj5sr4Lh5C3W7/4tYsi4+wu3mg8h FS7fGncoct/YrnchxqXEts7B03Y5RLbskDK8zxaMcNNZI6tTnGmQDj5eEwY2VwvJBX T7fGP1WxXLpgcU9LbimHWcmzf9nc3jcOl90PzPAjlS2VXpSyhqQCddFgapKvE87sBf UeXn97y9HivjA== Date: Fri, 20 Feb 2026 14:14:23 +0800 From: Jisheng Zhang To: Dev Jain Cc: Will Deacon , Catalin Marinas , Dennis Zhou , Tejun Heo , Christoph Lameter , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, maz@kernel.org Subject: Re: [PATCH] arm64: remove HAVE_CMPXCHG_LOCAL Message-ID: References: <20260215033944.16374-1-jszhang@kernel.org> <89606308-3c03-4dcf-a89d-479258b710e4@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <89606308-3c03-4dcf-a89d-479258b710e4@arm.com> On Mon, Feb 16, 2026 at 08:59:17PM +0530, Dev Jain wrote: > > On 16/02/26 4:30 pm, Will Deacon wrote: > > On Sun, Feb 15, 2026 at 11:39:44AM +0800, Jisheng Zhang wrote: > >> It turns out the generic disable/enable irq this_cpu_cmpxchg > >> implementation is faster than LL/SC or lse implementation. Remove > >> HAVE_CMPXCHG_LOCAL for better performance on arm64. > >> > >> Tested on Quad 1.9GHZ CA55 platform: > >> average mod_node_page_state() cost decreases from 167ns to 103ns > >> the spawn (30 duration) benchmark in unixbench is improved > >> from 147494 lps to 150561 lps, improved by 2.1% > >> > >> Tested on Quad 2.1GHZ CA73 platform: > >> average mod_node_page_state() cost decreases from 113ns to 85ns > >> the spawn (30 duration) benchmark in unixbench is improved > >> from 209844 lps to 212581 lps, improved by 1.3% > >> > >> Signed-off-by: Jisheng Zhang > >> --- > >> arch/arm64/Kconfig | 1 - > >> arch/arm64/include/asm/percpu.h | 24 ------------------------ > >> 2 files changed, 25 deletions(-) > > That is _entirely_ dependent on the system, so this isn't the right > > approach. I also don't think it's something we particularly want to > > micro-optimise to accomodate systems that suck at atomics. Hi Will, I read this as an implication that the cmpxchg_local version is better than generic disable/enable irq version on the newer arm64 systems. Is my understanding correct? > > Hi Will, > > As I mention in the other email, the suspect is not the atomics, but > preempt_disable(). On Apple M3, the regression reported in [1] resolves > by removing preempt_disable/enable in _pcp_protect_return. To prove > this another way, I disabled CONFIG_ARM64_HAS_LSE_ATOMICS and the > regression worsened, indicating that at least on Apple M3 the > atomics are faster. > > It may help to confirm this hypothesis on other hardware - perhaps > Jisheng can test with this change on his hardware and confirm > whether he gets the same performance improvement. Hi Dev, Thanks for the hints. I tried to remove the preempt_disable/enable from _pcp_protect_return, it improves, but the HAVE_CMPXCHG_LOCAL version is still worse than generic disable/enable irq version on CA55 and CA73. > > By coincidence, Yang Shi has been discussing the this_cpu_* overhead > at [2]. > > [1] https://lore.kernel.org/all/1052a452-9ba3-4da7-be47-7d27d27b3d1d@arm.com/ > [2] https://lore.kernel.org/all/CAHbLzkpcN-T8MH6=W3jCxcFj1gVZp8fRqe231yzZT-rV_E_org@mail.gmail.com/ > > > > > Will > > > >> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig > >> index 38dba5f7e4d2..5e7e2e65d5a5 100644 > >> --- a/arch/arm64/Kconfig > >> +++ b/arch/arm64/Kconfig > >> @@ -205,7 +205,6 @@ config ARM64 > >> select HAVE_EBPF_JIT > >> select HAVE_C_RECORDMCOUNT > >> select HAVE_CMPXCHG_DOUBLE > >> - select HAVE_CMPXCHG_LOCAL > >> select HAVE_CONTEXT_TRACKING_USER > >> select HAVE_DEBUG_KMEMLEAK > >> select HAVE_DMA_CONTIGUOUS > >> diff --git a/arch/arm64/include/asm/percpu.h b/arch/arm64/include/asm/percpu.h > >> index b57b2bb00967..70ffe566cb4b 100644 > >> --- a/arch/arm64/include/asm/percpu.h > >> +++ b/arch/arm64/include/asm/percpu.h > >> @@ -232,30 +232,6 @@ PERCPU_RET_OP(add, add, ldadd) > >> #define this_cpu_xchg_8(pcp, val) \ > >> _pcp_protect_return(xchg_relaxed, pcp, val) > >> > >> -#define this_cpu_cmpxchg_1(pcp, o, n) \ > >> - _pcp_protect_return(cmpxchg_relaxed, pcp, o, n) > >> -#define this_cpu_cmpxchg_2(pcp, o, n) \ > >> - _pcp_protect_return(cmpxchg_relaxed, pcp, o, n) > >> -#define this_cpu_cmpxchg_4(pcp, o, n) \ > >> - _pcp_protect_return(cmpxchg_relaxed, pcp, o, n) > >> -#define this_cpu_cmpxchg_8(pcp, o, n) \ > >> - _pcp_protect_return(cmpxchg_relaxed, pcp, o, n) > >> - > >> -#define this_cpu_cmpxchg64(pcp, o, n) this_cpu_cmpxchg_8(pcp, o, n) > >> - > >> -#define this_cpu_cmpxchg128(pcp, o, n) \ > >> -({ \ > >> - typedef typeof(pcp) pcp_op_T__; \ > >> - u128 old__, new__, ret__; \ > >> - pcp_op_T__ *ptr__; \ > >> - old__ = o; \ > >> - new__ = n; \ > >> - preempt_disable_notrace(); \ > >> - ptr__ = raw_cpu_ptr(&(pcp)); \ > >> - ret__ = cmpxchg128_local((void *)ptr__, old__, new__); \ > >> - preempt_enable_notrace(); \ > >> - ret__; \ > >> -}) > >> > >> #ifdef __KVM_NVHE_HYPERVISOR__ > >> extern unsigned long __hyp_per_cpu_offset(unsigned int cpu); > >> -- > >> 2.51.0 > >>