From: Heiko Carstens <hca@linux.ibm.com>
To: Alexander Gordeev <agordeev@linux.ibm.com>,
Sven Schnelle <svens@linux.ibm.com>,
Vasily Gorbik <gor@linux.ibm.com>,
Christian Borntraeger <borntraeger@linux.ibm.com>,
Mete Durlu <meted@linux.ibm.com>,
Peter Zijlstra <peterz@infradead.org>,
Mark Rutland <mark.rutland@arm.com>,
Juergen Christ <jchrist@linux.ibm.com>,
Ilya Leoshkevich <iii@linux.ibm.com>
Cc: linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org
Subject: [PATCH 09/12] s390/percpu: Use percpu code section for this_cpu_xchg()
Date: Thu, 17 Sep 2026 21:22:34 +0200 [thread overview]
Message-ID: <20260917192237.638324-10-hca@linux.ibm.com> (raw)
In-Reply-To: <20260917192237.638324-1-hca@linux.ibm.com>
Convert this_cpu_xchg() to make use of percpu code section
functionality in order to get rid of preempt_disable() and
preempt_enable() calls.
Consider only four and eight byte usages, since there are compare and
swap instructions for one and two bytes available. For one and two
byte use the previous simple implementation.
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
---
arch/s390/include/asm/percpu.h | 34 +++++++++++++++++++++++++++++-----
1 file changed, 29 insertions(+), 5 deletions(-)
diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h
index 0eae65937c0d..237bf8c80843 100644
--- a/arch/s390/include/asm/percpu.h
+++ b/arch/s390/include/asm/percpu.h
@@ -331,7 +331,7 @@ do { \
old__; \
})
-#define arch_this_cpu_xchg(pcp, nval) \
+#define arch_this_cpu_xchg_simple(pcp, nval) \
({ \
typeof(pcp) *ptr__; \
typeof(pcp) ret__; \
@@ -342,10 +342,34 @@ do { \
ret__; \
})
-#define this_cpu_xchg_1(pcp, nval) arch_this_cpu_xchg(pcp, nval)
-#define this_cpu_xchg_2(pcp, nval) arch_this_cpu_xchg(pcp, nval)
-#define this_cpu_xchg_4(pcp, nval) arch_this_cpu_xchg(pcp, nval)
-#define this_cpu_xchg_8(pcp, nval) arch_this_cpu_xchg(pcp, nval)
+#define arch_this_cpu_xchg(pcp, nval, ldop, csop) \
+({ \
+ typedef typeof(pcp) pcp_op_T__; \
+ pcp_op_T__ old__, new__ = (nval); \
+ pcp_op_T__ *ptr__; \
+ \
+ ptr__ = PERCPU_PTR(&(pcp)); \
+ asm_inline volatile( \
+ __PCPU_BEGIN("%[lcreg]","%[lcoff]","%[ptr__]") \
+ " " ldop " %[old__],0(%[ptr__])\n" \
+ "0: " csop " %[old__],%[new__],0(%[ptr__])\n"\
+ " jnz 0b\n" \
+ __PCPU_END("%[lcreg]") \
+ : [old__] "=&d" (old__), \
+ [ptr__] "+&a" (ptr__), "+m" (*ptr__), \
+ "=m" (((struct lowcore *)0)->percpu_register) \
+ : [new__] "d" (new__), \
+ [lcreg] "i" (LC_PERCPU_REGISTER), \
+ [lcoff] "i" (LC_PERCPU_OFFSET), \
+ "m" (((struct lowcore *)0)->percpu_offset) \
+ : "memory", "cc"); \
+ old__; \
+})
+
+#define this_cpu_xchg_1(pcp, nval) arch_this_cpu_xchg_simple(pcp, nval)
+#define this_cpu_xchg_2(pcp, nval) arch_this_cpu_xchg_simple(pcp, nval)
+#define this_cpu_xchg_4(pcp, nval) arch_this_cpu_xchg(pcp, nval, "l", "cs")
+#define this_cpu_xchg_8(pcp, nval) arch_this_cpu_xchg(pcp, nval, "lg", "csg")
#include <asm-generic/percpu.h>
--
2.53.0
next prev parent reply other threads:[~2026-09-17 19:22 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-17 19:22 [PATCH 00/12] s390: More this_cpu_*() changes Heiko Carstens
2026-09-17 19:22 ` [PATCH 01/12] s390/percpu: Fix comment typo Heiko Carstens
2026-09-17 19:22 ` [PATCH 02/12] s390/percpu: Add sanity check to GEN_MVIY macro Heiko Carstens
2026-09-17 19:22 ` [PATCH 03/12] s390/lowcore: Remove _AC() from LOWCORE_ALT_ADDRESS Heiko Carstens
2026-09-17 19:22 ` [PATCH 04/12] s390/percpu: Let MVIY_PERCPU() calculate alternative displacement Heiko Carstens
2026-09-17 19:22 ` [PATCH 05/12] s390/percpu/lowcore: Add and use LC_PERCPU lowcore offset defines Heiko Carstens
2026-09-17 19:22 ` [PATCH 06/12] s390/percpu: Rename inline assembly symbolic names Heiko Carstens
2026-09-17 19:22 ` [PATCH 07/12] s390/percpu: Use __PCPU_BEGIN() and __PCPU_END() for inline assemblies Heiko Carstens
2026-09-17 19:22 ` [PATCH 08/12] s390/percpu: Use percpu code section for this_cpu_cmpxchg128() Heiko Carstens
2026-09-17 19:22 ` Heiko Carstens [this message]
2026-09-17 19:22 ` [PATCH 10/12] s390/percpu: Use percpu code section for this_cpu_cmpxchg() Heiko Carstens
2026-09-17 19:22 ` [PATCH 11/12] s390: Add CC_HAS_ASM_M_FORMAT_FLAG config option Heiko Carstens
2026-09-17 19:22 ` [PATCH 12/12] s390/percpu: Rework to simplify percpu_entry() and percpu_exit() Heiko Carstens
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260917192237.638324-10-hca@linux.ibm.com \
--to=hca@linux.ibm.com \
--cc=agordeev@linux.ibm.com \
--cc=borntraeger@linux.ibm.com \
--cc=gor@linux.ibm.com \
--cc=iii@linux.ibm.com \
--cc=jchrist@linux.ibm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-s390@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=meted@linux.ibm.com \
--cc=peterz@infradead.org \
--cc=svens@linux.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®