mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* [RFC PATCH v3 0/5] Rework & improve riscv cmpxchg.h and atomic.h
@ 2023-08-04  8:48 Leonardo Bras
  2023-08-04  8:48 ` [RFC PATCH v3 1/5] riscv/cmpxchg: Deduplicate xchg() asm functions Leonardo Bras
                   ` (4 more replies)
  0 siblings, 5 replies; 13+ messages in thread
From: Leonardo Bras @ 2023-08-04  8:48 UTC (permalink / raw)
  To: Will Deacon, Peter Zijlstra, Boqun Feng, Mark Rutland,
	Paul Walmsley, Palmer Dabbelt, Albert Ou, Leonardo Bras,
	Andrea Parri, Geert Uytterhoeven, Andrzej Hajda, Palmer Dabbelt,
	Guo Ren
  Cc: linux-kernel, linux-riscv

While studying riscv's cmpxchg.h file, I got really interested in
understanding how RISCV asm implemented the different versions of
{cmp,}xchg.

When I understood the pattern, it made sense for me to remove the
duplications and create macros to make it easier to understand what exactly
changes between the versions: Instruction sufixes & barriers.

Also, did the same kind of work on atomic.c.

After that, I noted both cmpxchg and xchg only accept variables of 
size 4 and 8, compared to x86 and arm64 which do 1,2,4,8.

Now that deduplication is done, it is quite direct to implement them
for variable sizes 1 and 2, so I did it. Then Guo Ren already presented
me some possible users :)

I did compare the generated asm on a test.c that contained usage for every
changed function, and could not detect any change on patches 1 + 2 + 3 
compared with upstream.

Pathes 4 & 5 were compiled-tested, but I could not run-test them yet.

Thanks!
Leo


Changes since squashed cmpxchg RFCv2:
- Removed rc parameter from the new macro: it can be internal to the macro
- 2 new patches: cmpxchg size 1 and 2, xchg size 1 and 2
https://lore.kernel.org/all/20230803051401.710236-2-leobras@redhat.com/

Changes since squashed cmpxchg RFCv1:
- Unified with atomic.c patchset
- Rebased on top of torvalds/master (thanks Andrea Parri!)
- Removed helper macros that were not being used elsewhere in the kernel.
https://lore.kernel.org/all/20230419062505.257231-1-leobras@redhat.com/
https://lore.kernel.org/all/20230406082018.70367-1-leobras@redhat.com/

Changes since (cmpxchg) RFCv3:
- Squashed the 6 original patches in 2: one for cmpxchg and one for xchg
https://lore.kernel.org/all/20230404163741.2762165-1-leobras@redhat.com/

Changes since (cmpxchg) RFCv2:
- Fixed  macros that depend on having a local variable with a magic name
- Previous cast to (long) is now only applied on 4-bytes cmpxchg
https://lore.kernel.org/all/20230321074249.2221674-1-leobras@redhat.com/

Changes since (cmpxchg) RFCv1:
- Fixed patch 4/6 suffix from 'w.aqrl' to '.w.aqrl', to avoid build error
https://lore.kernel.org/all/20230318080059.1109286-1-leobras@redhat.com/


Leonardo Bras (5):
  riscv/cmpxchg: Deduplicate xchg() asm functions
  riscv/cmpxchg: Deduplicate cmpxchg() asm and macros
  riscv/atomic.h : Deduplicate arch_atomic.*
  riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2
  riscv/cmpxchg: Implement xchg for variables of size 1 and 2

 arch/riscv/include/asm/atomic.h  | 164 ++++++-------
 arch/riscv/include/asm/cmpxchg.h | 393 ++++++++++---------------------
 2 files changed, 195 insertions(+), 362 deletions(-)

-- 
2.41.0


^ permalink raw reply	[flat|nested] 13+ messages in thread

* [RFC PATCH v3 1/5] riscv/cmpxchg: Deduplicate xchg() asm functions
  2023-08-04  8:48 [RFC PATCH v3 0/5] Rework & improve riscv cmpxchg.h and atomic.h Leonardo Bras
@ 2023-08-04  8:48 ` Leonardo Bras
  2023-08-04  8:48 ` [RFC PATCH v3 2/5] riscv/cmpxchg: Deduplicate cmpxchg() asm and macros Leonardo Bras
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 13+ messages in thread
From: Leonardo Bras @ 2023-08-04  8:48 UTC (permalink / raw)
  To: Will Deacon, Peter Zijlstra, Boqun Feng, Mark Rutland,
	Paul Walmsley, Palmer Dabbelt, Albert Ou, Leonardo Bras,
	Andrea Parri, Geert Uytterhoeven, Andrzej Hajda, Palmer Dabbelt,
	Guo Ren
  Cc: linux-kernel, linux-riscv

In this header every xchg define (_relaxed, _acquire, _release, vanilla)
contain it's own asm file, both for 4-byte variables an 8-byte variables,
on a total of 8 versions of mostly the same asm.

This is usually bad, as it means any change may be done in up to 8
different places.

Unify those versions by creating a new define with enough parameters to
generate any version of the previous 8.

Then unify the result under a more general define, and simplify
arch_xchg* generation.

(This did not cause any change in generated asm)

Signed-off-by: Leonardo Bras <leobras@redhat.com>
Reviewed-by: Andrea Parri <parri.andrea@gmail.com>
---
 arch/riscv/include/asm/cmpxchg.h | 136 +++++--------------------------
 1 file changed, 22 insertions(+), 114 deletions(-)

diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
index 2f4726d3cfcc..ec4ea4f3f908 100644
--- a/arch/riscv/include/asm/cmpxchg.h
+++ b/arch/riscv/include/asm/cmpxchg.h
@@ -11,60 +11,30 @@
 #include <asm/barrier.h>
 #include <asm/fence.h>
 
-#define __xchg_relaxed(ptr, new, size)					\
+#define __arch_xchg(sfx, prepend, append, r, p, n)			\
 ({									\
-	__typeof__(ptr) __ptr = (ptr);					\
-	__typeof__(new) __new = (new);					\
-	__typeof__(*(ptr)) __ret;					\
-	switch (size) {							\
-	case 4:								\
-		__asm__ __volatile__ (					\
-			"	amoswap.w %0, %2, %1\n"			\
-			: "=r" (__ret), "+A" (*__ptr)			\
-			: "r" (__new)					\
-			: "memory");					\
-		break;							\
-	case 8:								\
-		__asm__ __volatile__ (					\
-			"	amoswap.d %0, %2, %1\n"			\
-			: "=r" (__ret), "+A" (*__ptr)			\
-			: "r" (__new)					\
-			: "memory");					\
-		break;							\
-	default:							\
-		BUILD_BUG();						\
-	}								\
-	__ret;								\
-})
-
-#define arch_xchg_relaxed(ptr, x)					\
-({									\
-	__typeof__(*(ptr)) _x_ = (x);					\
-	(__typeof__(*(ptr))) __xchg_relaxed((ptr),			\
-					    _x_, sizeof(*(ptr)));	\
+	__asm__ __volatile__ (						\
+		prepend							\
+		"	amoswap" sfx " %0, %2, %1\n"			\
+		append							\
+		: "=r" (r), "+A" (*(p))					\
+		: "r" (n)						\
+		: "memory");						\
 })
 
-#define __xchg_acquire(ptr, new, size)					\
+#define _arch_xchg(ptr, new, sfx, prepend, append)			\
 ({									\
-	__typeof__(ptr) __ptr = (ptr);					\
-	__typeof__(new) __new = (new);					\
+	__typeof__(*(ptr)) __new = (new);				\
 	__typeof__(*(ptr)) __ret;					\
-	switch (size) {							\
+	__typeof__(ptr) __ptr = (ptr);					\
+	switch (sizeof(*__ptr)) {					\
 	case 4:								\
-		__asm__ __volatile__ (					\
-			"	amoswap.w %0, %2, %1\n"			\
-			RISCV_ACQUIRE_BARRIER				\
-			: "=r" (__ret), "+A" (*__ptr)			\
-			: "r" (__new)					\
-			: "memory");					\
+		__arch_xchg(".w" sfx, prepend, append,			\
+			      __ret, __ptr, __new);			\
 		break;							\
 	case 8:								\
-		__asm__ __volatile__ (					\
-			"	amoswap.d %0, %2, %1\n"			\
-			RISCV_ACQUIRE_BARRIER				\
-			: "=r" (__ret), "+A" (*__ptr)			\
-			: "r" (__new)					\
-			: "memory");					\
+		__arch_xchg(".d" sfx, prepend, append,			\
+			      __ret, __ptr, __new);			\
 		break;							\
 	default:							\
 		BUILD_BUG();						\
@@ -72,79 +42,17 @@
 	__ret;								\
 })
 
-#define arch_xchg_acquire(ptr, x)					\
-({									\
-	__typeof__(*(ptr)) _x_ = (x);					\
-	(__typeof__(*(ptr))) __xchg_acquire((ptr),			\
-					    _x_, sizeof(*(ptr)));	\
-})
+#define arch_xchg_relaxed(ptr, x)					\
+	_arch_xchg(ptr, x, "", "", "")
 
-#define __xchg_release(ptr, new, size)					\
-({									\
-	__typeof__(ptr) __ptr = (ptr);					\
-	__typeof__(new) __new = (new);					\
-	__typeof__(*(ptr)) __ret;					\
-	switch (size) {							\
-	case 4:								\
-		__asm__ __volatile__ (					\
-			RISCV_RELEASE_BARRIER				\
-			"	amoswap.w %0, %2, %1\n"			\
-			: "=r" (__ret), "+A" (*__ptr)			\
-			: "r" (__new)					\
-			: "memory");					\
-		break;							\
-	case 8:								\
-		__asm__ __volatile__ (					\
-			RISCV_RELEASE_BARRIER				\
-			"	amoswap.d %0, %2, %1\n"			\
-			: "=r" (__ret), "+A" (*__ptr)			\
-			: "r" (__new)					\
-			: "memory");					\
-		break;							\
-	default:							\
-		BUILD_BUG();						\
-	}								\
-	__ret;								\
-})
+#define arch_xchg_acquire(ptr, x)					\
+	_arch_xchg(ptr, x, "", "", RISCV_ACQUIRE_BARRIER)
 
 #define arch_xchg_release(ptr, x)					\
-({									\
-	__typeof__(*(ptr)) _x_ = (x);					\
-	(__typeof__(*(ptr))) __xchg_release((ptr),			\
-					    _x_, sizeof(*(ptr)));	\
-})
-
-#define __arch_xchg(ptr, new, size)					\
-({									\
-	__typeof__(ptr) __ptr = (ptr);					\
-	__typeof__(new) __new = (new);					\
-	__typeof__(*(ptr)) __ret;					\
-	switch (size) {							\
-	case 4:								\
-		__asm__ __volatile__ (					\
-			"	amoswap.w.aqrl %0, %2, %1\n"		\
-			: "=r" (__ret), "+A" (*__ptr)			\
-			: "r" (__new)					\
-			: "memory");					\
-		break;							\
-	case 8:								\
-		__asm__ __volatile__ (					\
-			"	amoswap.d.aqrl %0, %2, %1\n"		\
-			: "=r" (__ret), "+A" (*__ptr)			\
-			: "r" (__new)					\
-			: "memory");					\
-		break;							\
-	default:							\
-		BUILD_BUG();						\
-	}								\
-	__ret;								\
-})
+	_arch_xchg(ptr, x, "", RISCV_RELEASE_BARRIER, "")
 
 #define arch_xchg(ptr, x)						\
-({									\
-	__typeof__(*(ptr)) _x_ = (x);					\
-	(__typeof__(*(ptr))) __arch_xchg((ptr), _x_, sizeof(*(ptr)));	\
-})
+	_arch_xchg(ptr, x, ".aqrl", "", "")
 
 #define xchg32(ptr, x)							\
 ({									\
-- 
2.41.0


^ permalink raw reply	[flat|nested] 13+ messages in thread

* [RFC PATCH v3 2/5] riscv/cmpxchg: Deduplicate cmpxchg() asm and macros
  2023-08-04  8:48 [RFC PATCH v3 0/5] Rework & improve riscv cmpxchg.h and atomic.h Leonardo Bras
  2023-08-04  8:48 ` [RFC PATCH v3 1/5] riscv/cmpxchg: Deduplicate xchg() asm functions Leonardo Bras
@ 2023-08-04  8:48 ` Leonardo Bras
  2023-08-04  8:48 ` [RFC PATCH v3 3/5] riscv/atomic.h : Deduplicate arch_atomic.* Leonardo Bras
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 13+ messages in thread
From: Leonardo Bras @ 2023-08-04  8:48 UTC (permalink / raw)
  To: Will Deacon, Peter Zijlstra, Boqun Feng, Mark Rutland,
	Paul Walmsley, Palmer Dabbelt, Albert Ou, Leonardo Bras,
	Andrea Parri, Geert Uytterhoeven, Andrzej Hajda, Palmer Dabbelt,
	Guo Ren
  Cc: linux-kernel, linux-riscv

In this header every cmpxchg define (_relaxed, _acquire, _release,
vanilla) contain it's own asm file, both for 4-byte variables an 8-byte
variables, on a total of 8 versions of mostly the same asm.

This is usually bad, as it means any change may be done in up to 8
different places.

Unify those versions by creating a new define with enough parameters to
generate any version of the previous 8.

Then unify the result under a more general define, and simplify
arch_cmpxchg* generation

(This did not cause any change in generated asm)

Signed-off-by: Leonardo Bras <leobras@redhat.com>
Reviewed-by: Andrea Parri <parri.andrea@gmail.com>
---
 arch/riscv/include/asm/cmpxchg.h | 187 +++++--------------------------
 1 file changed, 29 insertions(+), 158 deletions(-)

diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
index ec4ea4f3f908..5a07646fae65 100644
--- a/arch/riscv/include/asm/cmpxchg.h
+++ b/arch/riscv/include/asm/cmpxchg.h
@@ -71,127 +71,40 @@
  * store NEW in MEM.  Return the initial value in MEM.  Success is
  * indicated by comparing RETURN with OLD.
  */
-#define __cmpxchg_relaxed(ptr, old, new, size)				\
-({									\
-	__typeof__(ptr) __ptr = (ptr);					\
-	__typeof__(*(ptr)) __old = (old);				\
-	__typeof__(*(ptr)) __new = (new);				\
-	__typeof__(*(ptr)) __ret;					\
-	register unsigned int __rc;					\
-	switch (size) {							\
-	case 4:								\
-		__asm__ __volatile__ (					\
-			"0:	lr.w %0, %2\n"				\
-			"	bne  %0, %z3, 1f\n"			\
-			"	sc.w %1, %z4, %2\n"			\
-			"	bnez %1, 0b\n"				\
-			"1:\n"						\
-			: "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr)	\
-			: "rJ" ((long)__old), "rJ" (__new)		\
-			: "memory");					\
-		break;							\
-	case 8:								\
-		__asm__ __volatile__ (					\
-			"0:	lr.d %0, %2\n"				\
-			"	bne %0, %z3, 1f\n"			\
-			"	sc.d %1, %z4, %2\n"			\
-			"	bnez %1, 0b\n"				\
-			"1:\n"						\
-			: "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr)	\
-			: "rJ" (__old), "rJ" (__new)			\
-			: "memory");					\
-		break;							\
-	default:							\
-		BUILD_BUG();						\
-	}								\
-	__ret;								\
-})
 
-#define arch_cmpxchg_relaxed(ptr, o, n)					\
-({									\
-	__typeof__(*(ptr)) _o_ = (o);					\
-	__typeof__(*(ptr)) _n_ = (n);					\
-	(__typeof__(*(ptr))) __cmpxchg_relaxed((ptr),			\
-					_o_, _n_, sizeof(*(ptr)));	\
-})
 
-#define __cmpxchg_acquire(ptr, old, new, size)				\
+#define __arch_cmpxchg(lr_sfx, sc_sfx, prepend, append, r, p, co, o, n)	\
 ({									\
-	__typeof__(ptr) __ptr = (ptr);					\
-	__typeof__(*(ptr)) __old = (old);				\
-	__typeof__(*(ptr)) __new = (new);				\
-	__typeof__(*(ptr)) __ret;					\
 	register unsigned int __rc;					\
-	switch (size) {							\
-	case 4:								\
-		__asm__ __volatile__ (					\
-			"0:	lr.w %0, %2\n"				\
-			"	bne  %0, %z3, 1f\n"			\
-			"	sc.w %1, %z4, %2\n"			\
-			"	bnez %1, 0b\n"				\
-			RISCV_ACQUIRE_BARRIER				\
-			"1:\n"						\
-			: "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr)	\
-			: "rJ" ((long)__old), "rJ" (__new)		\
-			: "memory");					\
-		break;							\
-	case 8:								\
-		__asm__ __volatile__ (					\
-			"0:	lr.d %0, %2\n"				\
-			"	bne %0, %z3, 1f\n"			\
-			"	sc.d %1, %z4, %2\n"			\
-			"	bnez %1, 0b\n"				\
-			RISCV_ACQUIRE_BARRIER				\
-			"1:\n"						\
-			: "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr)	\
-			: "rJ" (__old), "rJ" (__new)			\
-			: "memory");					\
-		break;							\
-	default:							\
-		BUILD_BUG();						\
-	}								\
-	__ret;								\
-})
-
-#define arch_cmpxchg_acquire(ptr, o, n)					\
-({									\
-	__typeof__(*(ptr)) _o_ = (o);					\
-	__typeof__(*(ptr)) _n_ = (n);					\
-	(__typeof__(*(ptr))) __cmpxchg_acquire((ptr),			\
-					_o_, _n_, sizeof(*(ptr)));	\
+									\
+	__asm__ __volatile__ (						\
+		prepend							\
+		"0:	lr" lr_sfx " %0, %2\n"				\
+		"	bne  %0, %z3, 1f\n"				\
+		"	sc" sc_sfx " %1, %z4, %2\n"			\
+		"	bnez %1, 0b\n"					\
+		append							\
+		"1:\n"							\
+		: "=&r" (r), "=&r" (__rc), "+A" (*(p))			\
+		: "rJ" (co o), "rJ" (n)					\
+		: "memory");						\
 })
 
-#define __cmpxchg_release(ptr, old, new, size)				\
+#define _arch_cmpxchg(ptr, old, new, sc_sfx, prepend, append)		\
 ({									\
 	__typeof__(ptr) __ptr = (ptr);					\
 	__typeof__(*(ptr)) __old = (old);				\
 	__typeof__(*(ptr)) __new = (new);				\
 	__typeof__(*(ptr)) __ret;					\
-	register unsigned int __rc;					\
-	switch (size) {							\
+									\
+	switch (sizeof(*__ptr)) {					\
 	case 4:								\
-		__asm__ __volatile__ (					\
-			RISCV_RELEASE_BARRIER				\
-			"0:	lr.w %0, %2\n"				\
-			"	bne  %0, %z3, 1f\n"			\
-			"	sc.w %1, %z4, %2\n"			\
-			"	bnez %1, 0b\n"				\
-			"1:\n"						\
-			: "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr)	\
-			: "rJ" ((long)__old), "rJ" (__new)		\
-			: "memory");					\
+		__arch_cmpxchg(".w", ".w" sc_sfx, prepend, append,	\
+				__ret, __ptr, (long), __old, __new);	\
 		break;							\
 	case 8:								\
-		__asm__ __volatile__ (					\
-			RISCV_RELEASE_BARRIER				\
-			"0:	lr.d %0, %2\n"				\
-			"	bne %0, %z3, 1f\n"			\
-			"	sc.d %1, %z4, %2\n"			\
-			"	bnez %1, 0b\n"				\
-			"1:\n"						\
-			: "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr)	\
-			: "rJ" (__old), "rJ" (__new)			\
-			: "memory");					\
+		__arch_cmpxchg(".d", ".d" sc_sfx, prepend, append,	\
+				__ret, __ptr, /**/, __old, __new);	\
 		break;							\
 	default:							\
 		BUILD_BUG();						\
@@ -199,62 +112,20 @@
 	__ret;								\
 })
 
-#define arch_cmpxchg_release(ptr, o, n)					\
-({									\
-	__typeof__(*(ptr)) _o_ = (o);					\
-	__typeof__(*(ptr)) _n_ = (n);					\
-	(__typeof__(*(ptr))) __cmpxchg_release((ptr),			\
-					_o_, _n_, sizeof(*(ptr)));	\
-})
+#define arch_cmpxchg_relaxed(ptr, o, n)					\
+	_arch_cmpxchg((ptr), (o), (n), "", "", "")
 
-#define __cmpxchg(ptr, old, new, size)					\
-({									\
-	__typeof__(ptr) __ptr = (ptr);					\
-	__typeof__(*(ptr)) __old = (old);				\
-	__typeof__(*(ptr)) __new = (new);				\
-	__typeof__(*(ptr)) __ret;					\
-	register unsigned int __rc;					\
-	switch (size) {							\
-	case 4:								\
-		__asm__ __volatile__ (					\
-			"0:	lr.w %0, %2\n"				\
-			"	bne  %0, %z3, 1f\n"			\
-			"	sc.w.rl %1, %z4, %2\n"			\
-			"	bnez %1, 0b\n"				\
-			"	fence rw, rw\n"				\
-			"1:\n"						\
-			: "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr)	\
-			: "rJ" ((long)__old), "rJ" (__new)		\
-			: "memory");					\
-		break;							\
-	case 8:								\
-		__asm__ __volatile__ (					\
-			"0:	lr.d %0, %2\n"				\
-			"	bne %0, %z3, 1f\n"			\
-			"	sc.d.rl %1, %z4, %2\n"			\
-			"	bnez %1, 0b\n"				\
-			"	fence rw, rw\n"				\
-			"1:\n"						\
-			: "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr)	\
-			: "rJ" (__old), "rJ" (__new)			\
-			: "memory");					\
-		break;							\
-	default:							\
-		BUILD_BUG();						\
-	}								\
-	__ret;								\
-})
+#define arch_cmpxchg_acquire(ptr, o, n)					\
+	_arch_cmpxchg((ptr), (o), (n), "", "", RISCV_ACQUIRE_BARRIER)
+
+#define arch_cmpxchg_release(ptr, o, n)					\
+	_arch_cmpxchg((ptr), (o), (n), "", RISCV_RELEASE_BARRIER, "")
 
 #define arch_cmpxchg(ptr, o, n)						\
-({									\
-	__typeof__(*(ptr)) _o_ = (o);					\
-	__typeof__(*(ptr)) _n_ = (n);					\
-	(__typeof__(*(ptr))) __cmpxchg((ptr),				\
-				       _o_, _n_, sizeof(*(ptr)));	\
-})
+	_arch_cmpxchg((ptr), (o), (n), ".rl", "", "	fence rw, rw\n")
 
 #define arch_cmpxchg_local(ptr, o, n)					\
-	(__cmpxchg_relaxed((ptr), (o), (n), sizeof(*(ptr))))
+	arch_cmpxchg_relaxed((ptr), (o), (n))
 
 #define arch_cmpxchg64(ptr, o, n)					\
 ({									\
-- 
2.41.0


^ permalink raw reply	[flat|nested] 13+ messages in thread

* [RFC PATCH v3 3/5] riscv/atomic.h : Deduplicate arch_atomic.*
  2023-08-04  8:48 [RFC PATCH v3 0/5] Rework & improve riscv cmpxchg.h and atomic.h Leonardo Bras
  2023-08-04  8:48 ` [RFC PATCH v3 1/5] riscv/cmpxchg: Deduplicate xchg() asm functions Leonardo Bras
  2023-08-04  8:48 ` [RFC PATCH v3 2/5] riscv/cmpxchg: Deduplicate cmpxchg() asm and macros Leonardo Bras
@ 2023-08-04  8:48 ` Leonardo Bras
  2023-08-04  8:48 ` [RFC PATCH v3 4/5] riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2 Leonardo Bras
  2023-08-04  8:48 ` [RFC PATCH v3 5/5] riscv/cmpxchg: Implement xchg " Leonardo Bras
  4 siblings, 0 replies; 13+ messages in thread
From: Leonardo Bras @ 2023-08-04  8:48 UTC (permalink / raw)
  To: Will Deacon, Peter Zijlstra, Boqun Feng, Mark Rutland,
	Paul Walmsley, Palmer Dabbelt, Albert Ou, Leonardo Bras,
	Andrea Parri, Geert Uytterhoeven, Andrzej Hajda, Palmer Dabbelt,
	Guo Ren
  Cc: linux-kernel, linux-riscv

Some functions use mostly the same asm for 32-bit and 64-bit versions.

Make a macro that is generic enough and avoid code duplication.

(This did not cause any change in generated asm)

Signed-off-by: Leonardo Bras <leobras@redhat.com>
Reviewed-by: Guo Ren <guoren@kernel.org>
Reviewed-by: Andrea Parri <parri.andrea@gmail.com>
---
 arch/riscv/include/asm/atomic.h | 164 +++++++++++++++-----------------
 1 file changed, 76 insertions(+), 88 deletions(-)

diff --git a/arch/riscv/include/asm/atomic.h b/arch/riscv/include/asm/atomic.h
index f5dfef6c2153..80cca7ac16fd 100644
--- a/arch/riscv/include/asm/atomic.h
+++ b/arch/riscv/include/asm/atomic.h
@@ -196,22 +196,28 @@ ATOMIC_OPS(xor, xor, i)
 #undef ATOMIC_FETCH_OP
 #undef ATOMIC_OP_RETURN
 
+#define _arch_atomic_fetch_add_unless(_prev, _rc, counter, _a, _u, sfx)	\
+({									\
+	__asm__ __volatile__ (						\
+		"0:	lr." sfx "     %[p],  %[c]\n"			\
+		"	beq	       %[p],  %[u], 1f\n"		\
+		"	add            %[rc], %[p], %[a]\n"		\
+		"	sc." sfx ".rl  %[rc], %[rc], %[c]\n"		\
+		"	bnez           %[rc], 0b\n"			\
+		"	fence          rw, rw\n"			\
+		"1:\n"							\
+		: [p]"=&r" (_prev), [rc]"=&r" (_rc), [c]"+A" (counter)	\
+		: [a]"r" (_a), [u]"r" (_u)				\
+		: "memory");						\
+})
+
 /* This is required to provide a full barrier on success. */
 static __always_inline int arch_atomic_fetch_add_unless(atomic_t *v, int a, int u)
 {
        int prev, rc;
 
-	__asm__ __volatile__ (
-		"0:	lr.w     %[p],  %[c]\n"
-		"	beq      %[p],  %[u], 1f\n"
-		"	add      %[rc], %[p], %[a]\n"
-		"	sc.w.rl  %[rc], %[rc], %[c]\n"
-		"	bnez     %[rc], 0b\n"
-		"	fence    rw, rw\n"
-		"1:\n"
-		: [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter)
-		: [a]"r" (a), [u]"r" (u)
-		: "memory");
+	_arch_atomic_fetch_add_unless(prev, rc, v->counter, a, u, "w");
+
 	return prev;
 }
 #define arch_atomic_fetch_add_unless arch_atomic_fetch_add_unless
@@ -222,77 +228,86 @@ static __always_inline s64 arch_atomic64_fetch_add_unless(atomic64_t *v, s64 a,
        s64 prev;
        long rc;
 
-	__asm__ __volatile__ (
-		"0:	lr.d     %[p],  %[c]\n"
-		"	beq      %[p],  %[u], 1f\n"
-		"	add      %[rc], %[p], %[a]\n"
-		"	sc.d.rl  %[rc], %[rc], %[c]\n"
-		"	bnez     %[rc], 0b\n"
-		"	fence    rw, rw\n"
-		"1:\n"
-		: [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter)
-		: [a]"r" (a), [u]"r" (u)
-		: "memory");
+	_arch_atomic_fetch_add_unless(prev, rc, v->counter, a, u, "d");
+
 	return prev;
 }
 #define arch_atomic64_fetch_add_unless arch_atomic64_fetch_add_unless
 #endif
 
+#define _arch_atomic_inc_unless_negative(_prev, _rc, counter, sfx)	\
+({									\
+	__asm__ __volatile__ (						\
+		"0:	lr." sfx "      %[p],  %[c]\n"			\
+		"	bltz            %[p],  1f\n"			\
+		"	addi            %[rc], %[p], 1\n"		\
+		"	sc." sfx ".rl   %[rc], %[rc], %[c]\n"		\
+		"	bnez            %[rc], 0b\n"			\
+		"	fence           rw, rw\n"			\
+		"1:\n"							\
+		: [p]"=&r" (_prev), [rc]"=&r" (_rc), [c]"+A" (counter)	\
+		:							\
+		: "memory");						\
+})
+
 static __always_inline bool arch_atomic_inc_unless_negative(atomic_t *v)
 {
 	int prev, rc;
 
-	__asm__ __volatile__ (
-		"0:	lr.w      %[p],  %[c]\n"
-		"	bltz      %[p],  1f\n"
-		"	addi      %[rc], %[p], 1\n"
-		"	sc.w.rl   %[rc], %[rc], %[c]\n"
-		"	bnez      %[rc], 0b\n"
-		"	fence     rw, rw\n"
-		"1:\n"
-		: [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter)
-		:
-		: "memory");
+	_arch_atomic_inc_unless_negative(prev, rc, v->counter, "w");
+
 	return !(prev < 0);
 }
 
 #define arch_atomic_inc_unless_negative arch_atomic_inc_unless_negative
 
+#define _arch_atomic_dec_unless_positive(_prev, _rc, counter, sfx)	\
+({									\
+	__asm__ __volatile__ (						\
+		"0:	lr." sfx "      %[p],  %[c]\n"			\
+		"	bgtz            %[p],  1f\n"			\
+		"	addi            %[rc], %[p], -1\n"		\
+		"	sc." sfx ".rl   %[rc], %[rc], %[c]\n"		\
+		"	bnez            %[rc], 0b\n"			\
+		"	fence           rw, rw\n"			\
+		"1:\n"							\
+		: [p]"=&r" (_prev), [rc]"=&r" (_rc), [c]"+A" (counter)	\
+		:							\
+		: "memory");						\
+})
+
 static __always_inline bool arch_atomic_dec_unless_positive(atomic_t *v)
 {
 	int prev, rc;
 
-	__asm__ __volatile__ (
-		"0:	lr.w      %[p],  %[c]\n"
-		"	bgtz      %[p],  1f\n"
-		"	addi      %[rc], %[p], -1\n"
-		"	sc.w.rl   %[rc], %[rc], %[c]\n"
-		"	bnez      %[rc], 0b\n"
-		"	fence     rw, rw\n"
-		"1:\n"
-		: [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter)
-		:
-		: "memory");
+	_arch_atomic_dec_unless_positive(prev, rc, v->counter, "w");
+
 	return !(prev > 0);
 }
 
 #define arch_atomic_dec_unless_positive arch_atomic_dec_unless_positive
 
+#define _arch_atomic_dec_if_positive(_prev, _rc, counter, sfx)		\
+({									\
+	__asm__ __volatile__ (						\
+		"0:	lr." sfx "     %[p],  %[c]\n"			\
+		"	addi           %[rc], %[p], -1\n"		\
+		"	bltz           %[rc], 1f\n"			\
+		"	sc." sfx ".rl  %[rc], %[rc], %[c]\n"		\
+		"	bnez           %[rc], 0b\n"			\
+		"	fence          rw, rw\n"			\
+		"1:\n"							\
+		: [p]"=&r" (_prev), [rc]"=&r" (_rc), [c]"+A" (counter)	\
+		:							\
+		: "memory");						\
+})
+
 static __always_inline int arch_atomic_dec_if_positive(atomic_t *v)
 {
        int prev, rc;
 
-	__asm__ __volatile__ (
-		"0:	lr.w     %[p],  %[c]\n"
-		"	addi     %[rc], %[p], -1\n"
-		"	bltz     %[rc], 1f\n"
-		"	sc.w.rl  %[rc], %[rc], %[c]\n"
-		"	bnez     %[rc], 0b\n"
-		"	fence    rw, rw\n"
-		"1:\n"
-		: [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter)
-		:
-		: "memory");
+	_arch_atomic_dec_if_positive(prev, rc, v->counter, "w");
+
 	return prev - 1;
 }
 
@@ -304,17 +319,8 @@ static __always_inline bool arch_atomic64_inc_unless_negative(atomic64_t *v)
 	s64 prev;
 	long rc;
 
-	__asm__ __volatile__ (
-		"0:	lr.d      %[p],  %[c]\n"
-		"	bltz      %[p],  1f\n"
-		"	addi      %[rc], %[p], 1\n"
-		"	sc.d.rl   %[rc], %[rc], %[c]\n"
-		"	bnez      %[rc], 0b\n"
-		"	fence     rw, rw\n"
-		"1:\n"
-		: [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter)
-		:
-		: "memory");
+	_arch_atomic_inc_unless_negative(prev, rc, v->counter, "d");
+
 	return !(prev < 0);
 }
 
@@ -325,17 +331,8 @@ static __always_inline bool arch_atomic64_dec_unless_positive(atomic64_t *v)
 	s64 prev;
 	long rc;
 
-	__asm__ __volatile__ (
-		"0:	lr.d      %[p],  %[c]\n"
-		"	bgtz      %[p],  1f\n"
-		"	addi      %[rc], %[p], -1\n"
-		"	sc.d.rl   %[rc], %[rc], %[c]\n"
-		"	bnez      %[rc], 0b\n"
-		"	fence     rw, rw\n"
-		"1:\n"
-		: [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter)
-		:
-		: "memory");
+	_arch_atomic_dec_unless_positive(prev, rc, v->counter, "d");
+
 	return !(prev > 0);
 }
 
@@ -346,17 +343,8 @@ static __always_inline s64 arch_atomic64_dec_if_positive(atomic64_t *v)
        s64 prev;
        long rc;
 
-	__asm__ __volatile__ (
-		"0:	lr.d     %[p],  %[c]\n"
-		"	addi      %[rc], %[p], -1\n"
-		"	bltz     %[rc], 1f\n"
-		"	sc.d.rl  %[rc], %[rc], %[c]\n"
-		"	bnez     %[rc], 0b\n"
-		"	fence    rw, rw\n"
-		"1:\n"
-		: [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter)
-		:
-		: "memory");
+	_arch_atomic_dec_if_positive(prev, rc, v->counter, "d");
+
 	return prev - 1;
 }
 
-- 
2.41.0


^ permalink raw reply	[flat|nested] 13+ messages in thread

* [RFC PATCH v3 4/5] riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2
  2023-08-04  8:48 [RFC PATCH v3 0/5] Rework & improve riscv cmpxchg.h and atomic.h Leonardo Bras
                   ` (2 preceding siblings ...)
  2023-08-04  8:48 ` [RFC PATCH v3 3/5] riscv/atomic.h : Deduplicate arch_atomic.* Leonardo Bras
@ 2023-08-04  8:48 ` Leonardo Bras
  2023-08-04 17:45   ` Guo Ren
  2023-08-04  8:48 ` [RFC PATCH v3 5/5] riscv/cmpxchg: Implement xchg " Leonardo Bras
  4 siblings, 1 reply; 13+ messages in thread
From: Leonardo Bras @ 2023-08-04  8:48 UTC (permalink / raw)
  To: Will Deacon, Peter Zijlstra, Boqun Feng, Mark Rutland,
	Paul Walmsley, Palmer Dabbelt, Albert Ou, Leonardo Bras,
	Andrea Parri, Geert Uytterhoeven, Andrzej Hajda, Palmer Dabbelt,
	Guo Ren
  Cc: linux-kernel, linux-riscv

cmpxchg for variables of size 1-byte and 2-bytes is not yet available for
riscv, even though its present in other architectures such as arm64 and
x86. This could lead to not being able to implement some locking mechanisms
or requiring some rework to make it work properly.

Implement 1-byte and 2-bytes cmpxchg in order to achieve parity with other
architectures.

Signed-off-by: Leonardo Bras <leobras@redhat.com>
---
 arch/riscv/include/asm/cmpxchg.h | 35 ++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)

diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
index 5a07646fae65..dfb433ac544f 100644
--- a/arch/riscv/include/asm/cmpxchg.h
+++ b/arch/riscv/include/asm/cmpxchg.h
@@ -72,6 +72,36 @@
  * indicated by comparing RETURN with OLD.
  */
 
+#define __arch_cmpxchg_mask(sc_sfx, prepend, append, r, p, o, n)	\
+({									\
+	/* Depends on 2-byte variables being 2-byte aligned */		\
+	ulong __s = ((ulong)(p) & 0x3) * BITS_PER_BYTE;			\
+	ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0)	\
+			<< __s;						\
+	ulong __newx = (ulong)(n) << __s;				\
+	ulong __oldx = (ulong)(o) << __s;				\
+	ulong __retx;							\
+	register unsigned int __rc;					\
+									\
+	__asm__ __volatile__ (						\
+		prepend							\
+		"0:	lr.w %0, %2\n"					\
+		"	and  %0, %0, %z5\n"				\
+		"	bne  %0, %z3, 1f\n"				\
+		"	and  %1, %0, %z6\n"				\
+		"	or   %1, %1, %z4\n"				\
+		"	sc.w" sc_sfx " %1, %1, %2\n"			\
+		"	bnez %1, 0b\n"					\
+		append							\
+		"1:\n"							\
+		: "=&r" (__retx), "=&r" (__rc), "+A" (*(p))		\
+		: "rJ" ((long)__oldx), "rJ" (__newx),			\
+		  "rJ" (__mask), "rJ" (~__mask)				\
+		: "memory");						\
+									\
+	r = (__typeof__(*(p)))(__retx >> __s);				\
+})
+
 
 #define __arch_cmpxchg(lr_sfx, sc_sfx, prepend, append, r, p, co, o, n)	\
 ({									\
@@ -98,6 +128,11 @@
 	__typeof__(*(ptr)) __ret;					\
 									\
 	switch (sizeof(*__ptr)) {					\
+	case 1:								\
+	case 2:								\
+		__arch_cmpxchg_mask(sc_sfx, prepend, append,		\
+					__ret, __ptr, __old, __new);	\
+		break;							\
 	case 4:								\
 		__arch_cmpxchg(".w", ".w" sc_sfx, prepend, append,	\
 				__ret, __ptr, (long), __old, __new);	\
-- 
2.41.0


^ permalink raw reply	[flat|nested] 13+ messages in thread

* [RFC PATCH v3 5/5] riscv/cmpxchg: Implement xchg for variables of size 1 and 2
  2023-08-04  8:48 [RFC PATCH v3 0/5] Rework & improve riscv cmpxchg.h and atomic.h Leonardo Bras
                   ` (3 preceding siblings ...)
  2023-08-04  8:48 ` [RFC PATCH v3 4/5] riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2 Leonardo Bras
@ 2023-08-04  8:48 ` Leonardo Bras
  4 siblings, 0 replies; 13+ messages in thread
From: Leonardo Bras @ 2023-08-04  8:48 UTC (permalink / raw)
  To: Will Deacon, Peter Zijlstra, Boqun Feng, Mark Rutland,
	Paul Walmsley, Palmer Dabbelt, Albert Ou, Leonardo Bras,
	Andrea Parri, Geert Uytterhoeven, Andrzej Hajda, Palmer Dabbelt,
	Guo Ren
  Cc: linux-kernel, linux-riscv

xchg for variables of size 1-byte and 2-bytes is not yet available for
riscv, even though its present in other architectures such as arm64 and
x86. This could lead to not being able to implement some locking mechanisms
or requiring some rework to make it work properly.

Implement 1-byte and 2-bytes xchg in order to achieve parity with other
architectures.

Signed-off-by: Leonardo Bras <leobras@redhat.com>
---
 arch/riscv/include/asm/cmpxchg.h | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
index dfb433ac544f..943f094375c7 100644
--- a/arch/riscv/include/asm/cmpxchg.h
+++ b/arch/riscv/include/asm/cmpxchg.h
@@ -11,6 +11,31 @@
 #include <asm/barrier.h>
 #include <asm/fence.h>
 
+#define __arch_xchg_mask(prepend, append, r, p, n)			\
+({									\
+	/* Depends on 2-byte variables being 2-byte aligned */		\
+	ulong __s = ((ulong)(p) & 0x3) * BITS_PER_BYTE;			\
+	ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0)	\
+			<< __s;						\
+	ulong __newx = (ulong)(n) << __s;				\
+	ulong __retx;							\
+	register unsigned int __rc;					\
+									\
+	__asm__ __volatile__ (						\
+	       prepend							\
+	       "0:	lr.w %0, %2\n"					\
+	       "	and  %1, %0, %z4\n"				\
+	       "	or   %1, %1, %z3\n"				\
+	       "	sc.w %1, %1, %2\n"				\
+	       "	bnez %1, 0b\n"					\
+	       append							\
+	       : "=&r" (__retx), "=&r" (__rc), "+A" (*(p))		\
+	       : "rJ" (__newx), "rJ" (~__mask)				\
+	       : "memory");						\
+									\
+	r = (__typeof__(*(p)))((__retx & __mask) >> __s);		\
+})
+
 #define __arch_xchg(sfx, prepend, append, r, p, n)			\
 ({									\
 	__asm__ __volatile__ (						\
@@ -27,7 +52,13 @@
 	__typeof__(*(ptr)) __new = (new);				\
 	__typeof__(*(ptr)) __ret;					\
 	__typeof__(ptr) __ptr = (ptr);					\
+									\
 	switch (sizeof(*__ptr)) {					\
+	case 1:								\
+	case 2:								\
+		__arch_xchg_mask(prepend, append,			\
+				 __ret, __ptr, __new);			\
+		break;							\
 	case 4:								\
 		__arch_xchg(".w" sfx, prepend, append,			\
 			      __ret, __ptr, __new);			\
-- 
2.41.0


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [RFC PATCH v3 4/5] riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2
  2023-08-04  8:48 ` [RFC PATCH v3 4/5] riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2 Leonardo Bras
@ 2023-08-04 17:45   ` Guo Ren
  2023-08-05  3:14     ` Leonardo Bras Soares Passos
  0 siblings, 1 reply; 13+ messages in thread
From: Guo Ren @ 2023-08-04 17:45 UTC (permalink / raw)
  To: Leonardo Bras
  Cc: Will Deacon, Peter Zijlstra, Boqun Feng, Mark Rutland,
	Paul Walmsley, Palmer Dabbelt, Albert Ou, Andrea Parri,
	Geert Uytterhoeven, Andrzej Hajda, Palmer Dabbelt, linux-kernel,
	linux-riscv

On Fri, Aug 4, 2023 at 4:49 AM Leonardo Bras <leobras@redhat.com> wrote:
>
> cmpxchg for variables of size 1-byte and 2-bytes is not yet available for
> riscv, even though its present in other architectures such as arm64 and
> x86. This could lead to not being able to implement some locking mechanisms
> or requiring some rework to make it work properly.
>
> Implement 1-byte and 2-bytes cmpxchg in order to achieve parity with other
> architectures.
>
> Signed-off-by: Leonardo Bras <leobras@redhat.com>
> ---
>  arch/riscv/include/asm/cmpxchg.h | 35 ++++++++++++++++++++++++++++++++
>  1 file changed, 35 insertions(+)
>
> diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> index 5a07646fae65..dfb433ac544f 100644
> --- a/arch/riscv/include/asm/cmpxchg.h
> +++ b/arch/riscv/include/asm/cmpxchg.h
> @@ -72,6 +72,36 @@
>   * indicated by comparing RETURN with OLD.
>   */
>
> +#define __arch_cmpxchg_mask(sc_sfx, prepend, append, r, p, o, n)       \
> +({                                                                     \
> +       /* Depends on 2-byte variables being 2-byte aligned */          \
> +       ulong __s = ((ulong)(p) & 0x3) * BITS_PER_BYTE;                 \
> +       ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0)   \
> +                       << __s;                                         \
> +       ulong __newx = (ulong)(n) << __s;                               \
> +       ulong __oldx = (ulong)(o) << __s;                               \
> +       ulong __retx;                                                   \
> +       register unsigned int __rc;                                     \
> +                                                                       \
> +       __asm__ __volatile__ (                                          \
> +               prepend                                                 \
> +               "0:     lr.w %0, %2\n"                                  \
> +               "       and  %0, %0, %z5\n"                             \
> +               "       bne  %0, %z3, 1f\n"                             \
bug:
-               "       and  %0, %0, %z5\n"                             \
-               "       bne  %0, %z3, 1f\n"                             \
+               "       and  %1, %0, %z5\n"                             \
+               "       bne  %1, %z3, 1f\n"                             \
Your code breaks the %0.



> +               "       and  %1, %0, %z6\n"                             \
> +               "       or   %1, %1, %z4\n"                             \
> +               "       sc.w" sc_sfx " %1, %1, %2\n"                    \
> +               "       bnez %1, 0b\n"                                  \
> +               append                                                  \
> +               "1:\n"                                                  \
> +               : "=&r" (__retx), "=&r" (__rc), "+A" (*(p))             \
> +               : "rJ" ((long)__oldx), "rJ" (__newx),                   \
> +                 "rJ" (__mask), "rJ" (~__mask)                         \
> +               : "memory");                                            \
> +                                                                       \
> +       r = (__typeof__(*(p)))(__retx >> __s);                          \
> +})
> +
>
>  #define __arch_cmpxchg(lr_sfx, sc_sfx, prepend, append, r, p, co, o, n)        \
>  ({                                                                     \
> @@ -98,6 +128,11 @@
>         __typeof__(*(ptr)) __ret;                                       \
>                                                                         \
>         switch (sizeof(*__ptr)) {                                       \
> +       case 1:                                                         \
> +       case 2:                                                         \
> +               __arch_cmpxchg_mask(sc_sfx, prepend, append,            \
> +                                       __ret, __ptr, __old, __new);    \
> +               break;                                                  \
>         case 4:                                                         \
>                 __arch_cmpxchg(".w", ".w" sc_sfx, prepend, append,      \
>                                 __ret, __ptr, (long), __old, __new);    \
> --
> 2.41.0
>


-- 
Best Regards
 Guo Ren

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [RFC PATCH v3 4/5] riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2
  2023-08-04 17:45   ` Guo Ren
@ 2023-08-05  3:14     ` Leonardo Bras Soares Passos
  2023-08-05  4:24       ` Guo Ren
  0 siblings, 1 reply; 13+ messages in thread
From: Leonardo Bras Soares Passos @ 2023-08-05  3:14 UTC (permalink / raw)
  To: Guo Ren
  Cc: Will Deacon, Peter Zijlstra, Boqun Feng, Mark Rutland,
	Paul Walmsley, Palmer Dabbelt, Albert Ou, Andrea Parri,
	Geert Uytterhoeven, Andrzej Hajda, Palmer Dabbelt, linux-kernel,
	linux-riscv

Hello Guo Ren, thanks for the feedback!

On Fri, Aug 4, 2023 at 2:45 PM Guo Ren <guoren@kernel.org> wrote:
>
> On Fri, Aug 4, 2023 at 4:49 AM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > cmpxchg for variables of size 1-byte and 2-bytes is not yet available for
> > riscv, even though its present in other architectures such as arm64 and
> > x86. This could lead to not being able to implement some locking mechanisms
> > or requiring some rework to make it work properly.
> >
> > Implement 1-byte and 2-bytes cmpxchg in order to achieve parity with other
> > architectures.
> >
> > Signed-off-by: Leonardo Bras <leobras@redhat.com>
> > ---
> >  arch/riscv/include/asm/cmpxchg.h | 35 ++++++++++++++++++++++++++++++++
> >  1 file changed, 35 insertions(+)
> >
> > diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> > index 5a07646fae65..dfb433ac544f 100644
> > --- a/arch/riscv/include/asm/cmpxchg.h
> > +++ b/arch/riscv/include/asm/cmpxchg.h
> > @@ -72,6 +72,36 @@
> >   * indicated by comparing RETURN with OLD.
> >   */
> >
> > +#define __arch_cmpxchg_mask(sc_sfx, prepend, append, r, p, o, n)       \
> > +({                                                                     \
> > +       /* Depends on 2-byte variables being 2-byte aligned */          \
> > +       ulong __s = ((ulong)(p) & 0x3) * BITS_PER_BYTE;                 \
> > +       ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0)   \
> > +                       << __s;                                         \
> > +       ulong __newx = (ulong)(n) << __s;                               \
> > +       ulong __oldx = (ulong)(o) << __s;                               \
> > +       ulong __retx;                                                   \
> > +       register unsigned int __rc;                                     \
> > +                                                                       \
> > +       __asm__ __volatile__ (                                          \
> > +               prepend                                                 \
> > +               "0:     lr.w %0, %2\n"                                  \
> > +               "       and  %0, %0, %z5\n"                             \
> > +               "       bne  %0, %z3, 1f\n"                             \

> bug:
> -               "       and  %0, %0, %z5\n"                             \
> -               "       bne  %0, %z3, 1f\n"                             \
> +               "       and  %1, %0, %z5\n"                             \
> +               "       bne  %1, %z3, 1f\n"                             \
> Your code breaks the %0.

What do you mean by breaks here?

In the end of this macro, I intended  to have __retx = (*p & __mask)
which means the value is clean to be rotated at the end of the macro
(no need to apply the mask again): r = __ret >> __s;

Also, I assumed we are supposed to return the same variable type
as the pointer, so this is valid:
u8 a, *b, c;
a = xchg(b, c);

Is this correct?

> > +               append                                                  \
> > +               "1:\n"                                                  \
> > +               : "=&r" (__retx), "=&r" (__rc), "+A" (*(p))             \
> > +               : "rJ" ((long)__oldx), "rJ" (__newx),                   \
> > +                 "rJ" (__mask), "rJ" (~__mask)                         \
> > +               : "memory");                                            \
> > +                                                                       \
> > +       r = (__typeof__(*(p)))(__retx >> __s);                          \
> > +})
> > +
> >
> >  #define __arch_cmpxchg(lr_sfx, sc_sfx, prepend, append, r, p, co, o, n)        \
> >  ({                                                                     \
> > @@ -98,6 +128,11 @@
> >         __typeof__(*(ptr)) __ret;                                       \
> >                                                                         \
> >         switch (sizeof(*__ptr)) {                                       \
> > +       case 1:                                                         \
> > +       case 2:                                                         \
> > +               __arch_cmpxchg_mask(sc_sfx, prepend, append,            \
> > +                                       __ret, __ptr, __old, __new);    \
> > +               break;                                                  \
> >         case 4:                                                         \
> >                 __arch_cmpxchg(".w", ".w" sc_sfx, prepend, append,      \
> >                                 __ret, __ptr, (long), __old, __new);    \
> > --
> > 2.41.0
> >
>
>
> --
> Best Regards
>  Guo Ren
>


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [RFC PATCH v3 4/5] riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2
  2023-08-05  3:14     ` Leonardo Bras Soares Passos
@ 2023-08-05  4:24       ` Guo Ren
  2023-08-07 16:17         ` Leonardo Bras Soares Passos
  0 siblings, 1 reply; 13+ messages in thread
From: Guo Ren @ 2023-08-05  4:24 UTC (permalink / raw)
  To: Leonardo Bras Soares Passos
  Cc: Will Deacon, Peter Zijlstra, Boqun Feng, Mark Rutland,
	Paul Walmsley, Palmer Dabbelt, Albert Ou, Andrea Parri,
	Geert Uytterhoeven, Andrzej Hajda, Palmer Dabbelt, linux-kernel,
	linux-riscv

On Sat, Aug 5, 2023 at 11:14 AM Leonardo Bras Soares Passos
<leobras@redhat.com> wrote:
>
> Hello Guo Ren, thanks for the feedback!
>
> On Fri, Aug 4, 2023 at 2:45 PM Guo Ren <guoren@kernel.org> wrote:
> >
> > On Fri, Aug 4, 2023 at 4:49 AM Leonardo Bras <leobras@redhat.com> wrote:
> > >
> > > cmpxchg for variables of size 1-byte and 2-bytes is not yet available for
> > > riscv, even though its present in other architectures such as arm64 and
> > > x86. This could lead to not being able to implement some locking mechanisms
> > > or requiring some rework to make it work properly.
> > >
> > > Implement 1-byte and 2-bytes cmpxchg in order to achieve parity with other
> > > architectures.
> > >
> > > Signed-off-by: Leonardo Bras <leobras@redhat.com>
> > > ---
> > >  arch/riscv/include/asm/cmpxchg.h | 35 ++++++++++++++++++++++++++++++++
> > >  1 file changed, 35 insertions(+)
> > >
> > > diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> > > index 5a07646fae65..dfb433ac544f 100644
> > > --- a/arch/riscv/include/asm/cmpxchg.h
> > > +++ b/arch/riscv/include/asm/cmpxchg.h
> > > @@ -72,6 +72,36 @@
> > >   * indicated by comparing RETURN with OLD.
> > >   */
> > >
> > > +#define __arch_cmpxchg_mask(sc_sfx, prepend, append, r, p, o, n)       \
> > > +({                                                                     \
> > > +       /* Depends on 2-byte variables being 2-byte aligned */          \
> > > +       ulong __s = ((ulong)(p) & 0x3) * BITS_PER_BYTE;                 \
> > > +       ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0)   \
> > > +                       << __s;                                         \
> > > +       ulong __newx = (ulong)(n) << __s;                               \
> > > +       ulong __oldx = (ulong)(o) << __s;                               \
> > > +       ulong __retx;                                                   \
> > > +       register unsigned int __rc;                                     \
> > > +                                                                       \
> > > +       __asm__ __volatile__ (                                          \
> > > +               prepend                                                 \
> > > +               "0:     lr.w %0, %2\n"                                  \
> > > +               "       and  %0, %0, %z5\n"                             \
> > > +               "       bne  %0, %z3, 1f\n"                             \
>
> > bug:
> > -               "       and  %0, %0, %z5\n"                             \
> > -               "       bne  %0, %z3, 1f\n"                             \
> > +               "       and  %1, %0, %z5\n"                             \
> > +               "       bne  %1, %z3, 1f\n"                             \
> > Your code breaks the %0.
>
> What do you mean by breaks here?
>
> In the end of this macro, I intended  to have __retx = (*p & __mask)
> which means the value is clean to be rotated at the end of the macro
> (no need to apply the mask again): r = __ret >> __s;
>
> Also, I assumed we are supposed to return the same variable type
> as the pointer, so this is valid:
> u8 a, *b, c;
> a = xchg(b, c);
>
> Is this correct?
I missed your removing "__ret & mask" at the end. So this may not the problem.

Your patch can't boot. After chewing your code for several hours, I
found a problem:
diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
index 943f094375c7..67bcce63b267 100644
--- a/arch/riscv/include/asm/cmpxchg.h
+++ b/arch/riscv/include/asm/cmpxchg.h
@@ -14,6 +14,7 @@
 #define __arch_xchg_mask(prepend, append, r, p, n)                     \
 ({                                                                     \
        /* Depends on 2-byte variables being 2-byte aligned */          \
+       volatile ulong *__p = (ulong *)((ulong)(p) & ~0x3);             \
        ulong __s = ((ulong)(p) & 0x3) * BITS_PER_BYTE;                 \
        ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0)   \
                        << __s;                                         \
@@ -29,7 +30,7 @@
               "        sc.w %1, %1, %2\n"                              \
               "        bnez %1, 0b\n"                                  \
               append                                                   \
-              : "=&r" (__retx), "=&r" (__rc), "+A" (*(p))              \
+              : "=&r" (__retx), "=&r" (__rc), "+A" (*(__p))            \
               : "rJ" (__newx), "rJ" (~__mask)                          \
               : "memory");                                             \
                                                                        \
@@ -106,6 +107,7 @@
 #define __arch_cmpxchg_mask(sc_sfx, prepend, append, r, p, o, n)       \
 ({                                                                     \
        /* Depends on 2-byte variables being 2-byte aligned */          \
+       volatile ulong *__p = (ulong *)((ulong)(p) & ~0x3);             \
        ulong __s = ((ulong)(p) & 0x3) * BITS_PER_BYTE;                 \
        ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0)   \
                        << __s;                                         \
@@ -125,7 +127,7 @@
                "       bnez %1, 0b\n"                                  \
                append                                                  \
                "1:\n"                                                  \
-               : "=&r" (__retx), "=&r" (__rc), "+A" (*(p))             \
+               : "=&r" (__retx), "=&r" (__rc), "+A" (*(__p))           \
                : "rJ" ((long)__oldx), "rJ" (__newx),                   \
                  "rJ" (__mask), "rJ" (~__mask)                         \
                : "memory");                                            \

But the lkvm-static still can't boot with paravirt_spinlock .... Are
there any atomic tests in the Linux?

I found you use some "register int variables". Would it cause the problem?

You can reference this file, and it has passed the lock torture test:
https://github.com/guoren83/linux/blob/sg2042-master-qspinlock-64ilp32_v4/arch/riscv/include/asm/cmpxchg.h

I also merged your patches with the qspinlock series: (Use the above
cmpxchg.h the lkvm would run normally.)
https://github.com/guoren83/linux/tree/qspinlock_v11



>
> > > +               append                                                  \
> > > +               "1:\n"                                                  \
> > > +               : "=&r" (__retx), "=&r" (__rc), "+A" (*(p))             \
> > > +               : "rJ" ((long)__oldx), "rJ" (__newx),                   \
> > > +                 "rJ" (__mask), "rJ" (~__mask)                         \
> > > +               : "memory");                                            \
> > > +                                                                       \
> > > +       r = (__typeof__(*(p)))(__retx >> __s);                          \
> > > +})
> > > +
> > >
> > >  #define __arch_cmpxchg(lr_sfx, sc_sfx, prepend, append, r, p, co, o, n)        \
> > >  ({                                                                     \
> > > @@ -98,6 +128,11 @@
> > >         __typeof__(*(ptr)) __ret;                                       \
> > >                                                                         \
> > >         switch (sizeof(*__ptr)) {                                       \
> > > +       case 1:                                                         \
> > > +       case 2:                                                         \
> > > +               __arch_cmpxchg_mask(sc_sfx, prepend, append,            \
> > > +                                       __ret, __ptr, __old, __new);    \
> > > +               break;                                                  \
> > >         case 4:                                                         \
> > >                 __arch_cmpxchg(".w", ".w" sc_sfx, prepend, append,      \
> > >                                 __ret, __ptr, (long), __old, __new);    \
> > > --
> > > 2.41.0
> > >
> >
> >
> > --
> > Best Regards
> >  Guo Ren
> >
>


--
Best Regards
 Guo Ren

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [RFC PATCH v3 4/5] riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2
  2023-08-05  4:24       ` Guo Ren
@ 2023-08-07 16:17         ` Leonardo Bras Soares Passos
  2023-08-09  2:02           ` Leonardo Bras Soares Passos
  0 siblings, 1 reply; 13+ messages in thread
From: Leonardo Bras Soares Passos @ 2023-08-07 16:17 UTC (permalink / raw)
  To: Guo Ren
  Cc: Will Deacon, Peter Zijlstra, Boqun Feng, Mark Rutland,
	Paul Walmsley, Palmer Dabbelt, Albert Ou, Andrea Parri,
	Geert Uytterhoeven, Andrzej Hajda, Palmer Dabbelt, linux-kernel,
	linux-riscv

On Sat, Aug 5, 2023 at 1:24 AM Guo Ren <guoren@kernel.org> wrote:
>
> On Sat, Aug 5, 2023 at 11:14 AM Leonardo Bras Soares Passos
> <leobras@redhat.com> wrote:
> >
> > Hello Guo Ren, thanks for the feedback!
> >
> > On Fri, Aug 4, 2023 at 2:45 PM Guo Ren <guoren@kernel.org> wrote:
> > >
> > > On Fri, Aug 4, 2023 at 4:49 AM Leonardo Bras <leobras@redhat.com> wrote:
> > > >
> > > > cmpxchg for variables of size 1-byte and 2-bytes is not yet available for
> > > > riscv, even though its present in other architectures such as arm64 and
> > > > x86. This could lead to not being able to implement some locking mechanisms
> > > > or requiring some rework to make it work properly.
> > > >
> > > > Implement 1-byte and 2-bytes cmpxchg in order to achieve parity with other
> > > > architectures.
> > > >
> > > > Signed-off-by: Leonardo Bras <leobras@redhat.com>
> > > > ---
> > > >  arch/riscv/include/asm/cmpxchg.h | 35 ++++++++++++++++++++++++++++++++
> > > >  1 file changed, 35 insertions(+)
> > > >
> > > > diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> > > > index 5a07646fae65..dfb433ac544f 100644
> > > > --- a/arch/riscv/include/asm/cmpxchg.h
> > > > +++ b/arch/riscv/include/asm/cmpxchg.h
> > > > @@ -72,6 +72,36 @@
> > > >   * indicated by comparing RETURN with OLD.
> > > >   */
> > > >
> > > > +#define __arch_cmpxchg_mask(sc_sfx, prepend, append, r, p, o, n)       \
> > > > +({                                                                     \
> > > > +       /* Depends on 2-byte variables being 2-byte aligned */          \
> > > > +       ulong __s = ((ulong)(p) & 0x3) * BITS_PER_BYTE;                 \
> > > > +       ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0)   \
> > > > +                       << __s;                                         \
> > > > +       ulong __newx = (ulong)(n) << __s;                               \
> > > > +       ulong __oldx = (ulong)(o) << __s;                               \
> > > > +       ulong __retx;                                                   \
> > > > +       register unsigned int __rc;                                     \
> > > > +                                                                       \
> > > > +       __asm__ __volatile__ (                                          \
> > > > +               prepend                                                 \
> > > > +               "0:     lr.w %0, %2\n"                                  \
> > > > +               "       and  %0, %0, %z5\n"                             \
> > > > +               "       bne  %0, %z3, 1f\n"                             \
> >
> > > bug:
> > > -               "       and  %0, %0, %z5\n"                             \
> > > -               "       bne  %0, %z3, 1f\n"                             \
> > > +               "       and  %1, %0, %z5\n"                             \
> > > +               "       bne  %1, %z3, 1f\n"                             \
> > > Your code breaks the %0.
> >
> > What do you mean by breaks here?
> >
> > In the end of this macro, I intended  to have __retx = (*p & __mask)
> > which means the value is clean to be rotated at the end of the macro
> > (no need to apply the mask again): r = __ret >> __s;
> >
> > Also, I assumed we are supposed to return the same variable type
> > as the pointer, so this is valid:
> > u8 a, *b, c;
> > a = xchg(b, c);
> >
> > Is this correct?
> I missed your removing "__ret & mask" at the end. So this may not the problem.
>
> Your patch can't boot. After chewing your code for several hours, I
> found a problem:
> diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> index 943f094375c7..67bcce63b267 100644
> --- a/arch/riscv/include/asm/cmpxchg.h
> +++ b/arch/riscv/include/asm/cmpxchg.h
> @@ -14,6 +14,7 @@
>  #define __arch_xchg_mask(prepend, append, r, p, n)                     \
>  ({                                                                     \
>         /* Depends on 2-byte variables being 2-byte aligned */          \
> +       volatile ulong *__p = (ulong *)((ulong)(p) & ~0x3);             \

Wow, I totally missed this one: I should not assume the processor
behavior on unaligned lr/sc.
Thanks for spotting this :)

>         ulong __s = ((ulong)(p) & 0x3) * BITS_PER_BYTE;                 \
>         ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0)   \
>                         << __s;                                         \
> @@ -29,7 +30,7 @@
>                "        sc.w %1, %1, %2\n"                              \
>                "        bnez %1, 0b\n"                                  \
>                append                                                   \
> -              : "=&r" (__retx), "=&r" (__rc), "+A" (*(p))              \
> +              : "=&r" (__retx), "=&r" (__rc), "+A" (*(__p))            \
>                : "rJ" (__newx), "rJ" (~__mask)                          \
>                : "memory");                                             \
>                                                                         \
> @@ -106,6 +107,7 @@
>  #define __arch_cmpxchg_mask(sc_sfx, prepend, append, r, p, o, n)       \
>  ({                                                                     \
>         /* Depends on 2-byte variables being 2-byte aligned */          \
> +       volatile ulong *__p = (ulong *)((ulong)(p) & ~0x3);             \
>         ulong __s = ((ulong)(p) & 0x3) * BITS_PER_BYTE;                 \
>         ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0)   \
>                         << __s;                                         \
> @@ -125,7 +127,7 @@
>                 "       bnez %1, 0b\n"                                  \
>                 append                                                  \
>                 "1:\n"                                                  \
> -               : "=&r" (__retx), "=&r" (__rc), "+A" (*(p))             \
> +               : "=&r" (__retx), "=&r" (__rc), "+A" (*(__p))           \
>                 : "rJ" ((long)__oldx), "rJ" (__newx),                   \
>                   "rJ" (__mask), "rJ" (~__mask)                         \
>                 : "memory");                                            \
>
> But the lkvm-static still can't boot with paravirt_spinlock .... Are
> there any atomic tests in the Linux?

I recall reading something about 'locktorture' or so, not sure if useful:

https://docs.kernel.org/locking/locktorture.html

>
> I found you use some "register int variables". Would it cause the problem?

Honestly, not sure.
I will try inspecting the generated asm for any unexpected changes
compared to your version.

>
> You can reference this file, and it has passed the lock torture test:
> https://github.com/guoren83/linux/blob/sg2042-master-qspinlock-64ilp32_v4/arch/riscv/include/asm/cmpxchg.h
>
> I also merged your patches with the qspinlock series: (Use the above
> cmpxchg.h the lkvm would run normally.)
> https://github.com/guoren83/linux/tree/qspinlock_v11

Thanks!

I should reply as soon as I find anything.

Best regards,
Leo

>
>
>
> >
> > > > +               append                                                  \
> > > > +               "1:\n"                                                  \
> > > > +               : "=&r" (__retx), "=&r" (__rc), "+A" (*(p))             \
> > > > +               : "rJ" ((long)__oldx), "rJ" (__newx),                   \
> > > > +                 "rJ" (__mask), "rJ" (~__mask)                         \
> > > > +               : "memory");                                            \
> > > > +                                                                       \
> > > > +       r = (__typeof__(*(p)))(__retx >> __s);                          \
> > > > +})
> > > > +
> > > >
> > > >  #define __arch_cmpxchg(lr_sfx, sc_sfx, prepend, append, r, p, co, o, n)        \
> > > >  ({                                                                     \
> > > > @@ -98,6 +128,11 @@
> > > >         __typeof__(*(ptr)) __ret;                                       \
> > > >                                                                         \
> > > >         switch (sizeof(*__ptr)) {                                       \
> > > > +       case 1:                                                         \
> > > > +       case 2:                                                         \
> > > > +               __arch_cmpxchg_mask(sc_sfx, prepend, append,            \
> > > > +                                       __ret, __ptr, __old, __new);    \
> > > > +               break;                                                  \
> > > >         case 4:                                                         \
> > > >                 __arch_cmpxchg(".w", ".w" sc_sfx, prepend, append,      \
> > > >                                 __ret, __ptr, (long), __old, __new);    \
> > > > --
> > > > 2.41.0
> > > >
> > >
> > >
> > > --
> > > Best Regards
> > >  Guo Ren
> > >
> >
>
>
> --
> Best Regards
>  Guo Ren
>


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [RFC PATCH v3 4/5] riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2
  2023-08-07 16:17         ` Leonardo Bras Soares Passos
@ 2023-08-09  2:02           ` Leonardo Bras Soares Passos
  2023-08-10  4:06             ` Leonardo Bras Soares Passos
  2023-08-11  2:32             ` Guo Ren
  0 siblings, 2 replies; 13+ messages in thread
From: Leonardo Bras Soares Passos @ 2023-08-09  2:02 UTC (permalink / raw)
  To: Guo Ren
  Cc: Will Deacon, Peter Zijlstra, Boqun Feng, Mark Rutland,
	Paul Walmsley, Palmer Dabbelt, Albert Ou, Andrea Parri,
	Geert Uytterhoeven, Andrzej Hajda, Palmer Dabbelt, linux-kernel,
	linux-riscv

Hello Guo Ren,

On Mon, Aug 7, 2023 at 1:17 PM Leonardo Bras Soares Passos
<leobras@redhat.com> wrote:
>
> On Sat, Aug 5, 2023 at 1:24 AM Guo Ren <guoren@kernel.org> wrote:
> >
> > On Sat, Aug 5, 2023 at 11:14 AM Leonardo Bras Soares Passos
> > <leobras@redhat.com> wrote:
> > >
> > > Hello Guo Ren, thanks for the feedback!
> > >
> > > On Fri, Aug 4, 2023 at 2:45 PM Guo Ren <guoren@kernel.org> wrote:
> > > >
> > > > On Fri, Aug 4, 2023 at 4:49 AM Leonardo Bras <leobras@redhat.com> wrote:
> > > > >
> > > > > cmpxchg for variables of size 1-byte and 2-bytes is not yet available for
> > > > > riscv, even though its present in other architectures such as arm64 and
> > > > > x86. This could lead to not being able to implement some locking mechanisms
> > > > > or requiring some rework to make it work properly.
> > > > >
> > > > > Implement 1-byte and 2-bytes cmpxchg in order to achieve parity with other
> > > > > architectures.
> > > > >
> > > > > Signed-off-by: Leonardo Bras <leobras@redhat.com>
> > > > > ---
> > > > >  arch/riscv/include/asm/cmpxchg.h | 35 ++++++++++++++++++++++++++++++++
> > > > >  1 file changed, 35 insertions(+)
> > > > >
> > > > > diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> > > > > index 5a07646fae65..dfb433ac544f 100644
> > > > > --- a/arch/riscv/include/asm/cmpxchg.h
> > > > > +++ b/arch/riscv/include/asm/cmpxchg.h
> > > > > @@ -72,6 +72,36 @@
> > > > >   * indicated by comparing RETURN with OLD.
> > > > >   */
> > > > >
> > > > > +#define __arch_cmpxchg_mask(sc_sfx, prepend, append, r, p, o, n)       \
> > > > > +({                                                                     \
> > > > > +       /* Depends on 2-byte variables being 2-byte aligned */          \
> > > > > +       ulong __s = ((ulong)(p) & 0x3) * BITS_PER_BYTE;                 \
> > > > > +       ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0)   \
> > > > > +                       << __s;                                         \
> > > > > +       ulong __newx = (ulong)(n) << __s;                               \
> > > > > +       ulong __oldx = (ulong)(o) << __s;                               \
> > > > > +       ulong __retx;                                                   \
> > > > > +       register unsigned int __rc;                                     \
> > > > > +                                                                       \
> > > > > +       __asm__ __volatile__ (                                          \
> > > > > +               prepend                                                 \
> > > > > +               "0:     lr.w %0, %2\n"                                  \

      bne  %0, %z3, 1f\n"                             \
> > >
> > > > bug:
> > > > -               "       and  %0, %0, %z5\n"                             \
> > > > -               "       bne  %0, %z3, 1f\n"                             \
> > > > +               "       and  %1, %0, %z5\n"                             \
> > > > +               "       bne  %1, %z3, 1f\n"                             \
> > > > Your code breaks the %0.
> > >
> > > What do you mean by breaks here?
> > >
> > > In the end of this macro, I intended  to have __retx = (*p & __mask)
> > > which means the value is clean to be rotated at the end of the macro
> > > (no need to apply the mask again): r = __ret >> __s;
> > >
> > > Also, I assumed we are supposed to return the same variable type
> > > as the pointer, so this is valid:
> > > u8 a, *b, c;
> > > a = xchg(b, c);
> > >
> > > Is this correct?
> > I missed your removing "__ret & mask" at the end. So this may not the problem.

It was actually the problem :)
Even though I revised a lot, I was missing this which was made clear
by test-running on a VM:

        "0:    lr.w %0, %2\n"                    \
        "    and  %0, %0, %z5\n"                \
        "    bne  %0, %z3, 1f\n"                \
        "    and  %1, %0, %z6\n"                \
        "    or   %1, %1, %z4\n"                \
        "    sc.w" sc_sfx " %1, %1, %2\n"            \
        "    bnez %1, 0b\n"

If I do "and %0, %0, %z5" there, the line  "and  %1, %0, %z6"  will
output %1 = 0, which is not the intended.
You were right! The above fix solves the issue, and I just have to
mask it after.

Sorry it took me a few days to realize this.


> >
> > Your patch can't boot. After chewing your code for several hours, I
> > found a problem:
> > diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> > index 943f094375c7..67bcce63b267 100644
> > --- a/arch/riscv/include/asm/cmpxchg.h
> > +++ b/arch/riscv/include/asm/cmpxchg.h
> > @@ -14,6 +14,7 @@
> >  #define __arch_xchg_mask(prepend, append, r, p, n)                     \
> >  ({                                                                     \
> >         /* Depends on 2-byte variables being 2-byte aligned */          \
> > +       volatile ulong *__p = (ulong *)((ulong)(p) & ~0x3);             \
>
> Wow, I totally missed this one: I should not assume the processor
> behavior on unaligned lr/sc.
> Thanks for spotting this :)
>
> >         ulong __s = ((ulong)(p) & 0x3) * BITS_PER_BYTE;                 \
> >         ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0)   \
> >                         << __s;                                         \
> > @@ -29,7 +30,7 @@
> >                "        sc.w %1, %1, %2\n"                              \
> >                "        bnez %1, 0b\n"                                  \
> >                append                                                   \
> > -              : "=&r" (__retx), "=&r" (__rc), "+A" (*(p))              \
> > +              : "=&r" (__retx), "=&r" (__rc), "+A" (*(__p))            \
> >                : "rJ" (__newx), "rJ" (~__mask)                          \
> >                : "memory");                                             \
> >                                                                         \
> > @@ -106,6 +107,7 @@
> >  #define __arch_cmpxchg_mask(sc_sfx, prepend, append, r, p, o, n)       \
> >  ({                                                                     \
> >         /* Depends on 2-byte variables being 2-byte aligned */          \
> > +       volatile ulong *__p = (ulong *)((ulong)(p) & ~0x3);             \
> >         ulong __s = ((ulong)(p) & 0x3) * BITS_PER_BYTE;                 \
> >         ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0)   \
> >                         << __s;                                         \
> > @@ -125,7 +127,7 @@
> >                 "       bnez %1, 0b\n"                                  \
> >                 append                                                  \
> >                 "1:\n"                                                  \
> > -               : "=&r" (__retx), "=&r" (__rc), "+A" (*(p))             \
> > +               : "=&r" (__retx), "=&r" (__rc), "+A" (*(__p))           \
> >                 : "rJ" ((long)__oldx), "rJ" (__newx),                   \
> >                   "rJ" (__mask), "rJ" (~__mask)                         \
> >                 : "memory");                                            \
> >
> > But the lkvm-static still can't boot with paravirt_spinlock .... Are
> > there any atomic tests in the Linux?
>
> I recall reading something about 'locktorture' or so, not sure if useful:
>
> https://docs.kernel.org/locking/locktorture.html
>
> >
> > I found you use some "register int variables". Would it cause the problem?
>
> Honestly, not sure.
> I will try inspecting the generated asm for any unexpected changes
> compared to your version.

changed type to ulong, no changes detected in asm

>
> >
> > You can reference this file, and it has passed the lock torture test:
> > https://github.com/guoren83/linux/blob/sg2042-master-qspinlock-64ilp32_v4/arch/riscv/include/asm/cmpxchg.h
> >
> > I also merged your patches with the qspinlock series: (Use the above
> > cmpxchg.h the lkvm would run normally.)
> > https://github.com/guoren83/linux/tree/qspinlock_v11
>
> Thanks!
>
> I should reply as soon as I find anything.
>

Some changes found (for xchg16):
- pointer alignment wrong (need mask with ~0x3) : solved. Doesn't seem
to need to be volatile, though.

- AND with 0x3 for getting shift: I was trusting 2-byte vars to be
2-byte aligned , ie. p & 0x1 == 0: may not be true. Solved (it's
simple)

- on your code, IIUC you also assume ptr & 0x1 == 0, on
        ulong *__ptr = (ulong *)((ulong)ptr & ~2);
if it's not aligned, you may have __ptr = 0xYYYYYY01
(Not saying it's wrong to assume this, just listing It differs on my code)

- also verified your code generates addw and sllw on some points mine
generates add and sll, don't see any issue there

- There are extra sll and srl in my asm, because I end up casting to
typeof(*ptr) at the end.

changes found in cmpxchg (__cmpxchg_small_relaxed)
- mostly the same as above,

- the beginning of my function ends up being smaller, since I do my
masking differently.

I fixed the above issues, and  ran some userspace debugging tests on a
VM, and all seems to be correct now.

I then fetched your github repo, replaced my patches v3->v4 on your
tree, added linux boot parameter "qspinlock" on my qemu cmdline
(-machine virt): booting and working just fine.

I will send a v4 soon, please give it a test when possible.

Best regards,
Leo



> >
> >
> >
> > >
> > > > > +               append                                                  \
> > > > > +               "1:\n"                                                  \
> > > > > +               : "=&r" (__retx), "=&r" (__rc), "+A" (*(p))             \
> > > > > +               : "rJ" ((long)__oldx), "rJ" (__newx),                   \
> > > > > +                 "rJ" (__mask), "rJ" (~__mask)                         \
> > > > > +               : "memory");                                            \
> > > > > +                                                                       \
> > > > > +       r = (__typeof__(*(p)))(__retx >> __s);                          \
> > > > > +})
> > > > > +
> > > > >
> > > > >  #define __arch_cmpxchg(lr_sfx, sc_sfx, prepend, append, r, p, co, o, n)        \
> > > > >  ({                                                                     \
> > > > > @@ -98,6 +128,11 @@
> > > > >         __typeof__(*(ptr)) __ret;                                       \
> > > > >                                                                         \
> > > > >         switch (sizeof(*__ptr)) {                                       \
> > > > > +       case 1:                                                         \
> > > > > +       case 2:                                                         \
> > > > > +               __arch_cmpxchg_mask(sc_sfx, prepend, append,            \
> > > > > +                                       __ret, __ptr, __old, __new);    \
> > > > > +               break;                                                  \
> > > > >         case 4:                                                         \
> > > > >                 __arch_cmpxchg(".w", ".w" sc_sfx, prepend, append,      \
> > > > >                                 __ret, __ptr, (long), __old, __new);    \
> > > > > --
> > > > > 2.41.0
> > > > >
> > > >
> > > >
> > > > --
> > > > Best Regards
> > > >  Guo Ren
> > > >
> > >
> >
> >
> > --
> > Best Regards
> >  Guo Ren
> >


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [RFC PATCH v3 4/5] riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2
  2023-08-09  2:02           ` Leonardo Bras Soares Passos
@ 2023-08-10  4:06             ` Leonardo Bras Soares Passos
  2023-08-11  2:32             ` Guo Ren
  1 sibling, 0 replies; 13+ messages in thread
From: Leonardo Bras Soares Passos @ 2023-08-10  4:06 UTC (permalink / raw)
  To: Guo Ren
  Cc: Will Deacon, Peter Zijlstra, Boqun Feng, Mark Rutland,
	Paul Walmsley, Palmer Dabbelt, Albert Ou, Andrea Parri,
	Geert Uytterhoeven, Andrzej Hajda, Palmer Dabbelt, linux-kernel,
	linux-riscv

v5: https://lore.kernel.org/all/20230810040349.92279-2-leobras@redhat.com/

On Tue, Aug 8, 2023 at 11:02 PM Leonardo Bras Soares Passos
<leobras@redhat.com> wrote:
>
> Hello Guo Ren,
>
> On Mon, Aug 7, 2023 at 1:17 PM Leonardo Bras Soares Passos
> <leobras@redhat.com> wrote:
> >
> > On Sat, Aug 5, 2023 at 1:24 AM Guo Ren <guoren@kernel.org> wrote:
> > >
> > > On Sat, Aug 5, 2023 at 11:14 AM Leonardo Bras Soares Passos
> > > <leobras@redhat.com> wrote:
> > > >
> > > > Hello Guo Ren, thanks for the feedback!
> > > >
> > > > On Fri, Aug 4, 2023 at 2:45 PM Guo Ren <guoren@kernel.org> wrote:
> > > > >
> > > > > On Fri, Aug 4, 2023 at 4:49 AM Leonardo Bras <leobras@redhat.com> wrote:
> > > > > >
> > > > > > cmpxchg for variables of size 1-byte and 2-bytes is not yet available for
> > > > > > riscv, even though its present in other architectures such as arm64 and
> > > > > > x86. This could lead to not being able to implement some locking mechanisms
> > > > > > or requiring some rework to make it work properly.
> > > > > >
> > > > > > Implement 1-byte and 2-bytes cmpxchg in order to achieve parity with other
> > > > > > architectures.
> > > > > >
> > > > > > Signed-off-by: Leonardo Bras <leobras@redhat.com>
> > > > > > ---
> > > > > >  arch/riscv/include/asm/cmpxchg.h | 35 ++++++++++++++++++++++++++++++++
> > > > > >  1 file changed, 35 insertions(+)
> > > > > >
> > > > > > diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> > > > > > index 5a07646fae65..dfb433ac544f 100644
> > > > > > --- a/arch/riscv/include/asm/cmpxchg.h
> > > > > > +++ b/arch/riscv/include/asm/cmpxchg.h
> > > > > > @@ -72,6 +72,36 @@
> > > > > >   * indicated by comparing RETURN with OLD.
> > > > > >   */
> > > > > >
> > > > > > +#define __arch_cmpxchg_mask(sc_sfx, prepend, append, r, p, o, n)       \
> > > > > > +({                                                                     \
> > > > > > +       /* Depends on 2-byte variables being 2-byte aligned */          \
> > > > > > +       ulong __s = ((ulong)(p) & 0x3) * BITS_PER_BYTE;                 \
> > > > > > +       ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0)   \
> > > > > > +                       << __s;                                         \
> > > > > > +       ulong __newx = (ulong)(n) << __s;                               \
> > > > > > +       ulong __oldx = (ulong)(o) << __s;                               \
> > > > > > +       ulong __retx;                                                   \
> > > > > > +       register unsigned int __rc;                                     \
> > > > > > +                                                                       \
> > > > > > +       __asm__ __volatile__ (                                          \
> > > > > > +               prepend                                                 \
> > > > > > +               "0:     lr.w %0, %2\n"                                  \
>
>       bne  %0, %z3, 1f\n"                             \
> > > >
> > > > > bug:
> > > > > -               "       and  %0, %0, %z5\n"                             \
> > > > > -               "       bne  %0, %z3, 1f\n"                             \
> > > > > +               "       and  %1, %0, %z5\n"                             \
> > > > > +               "       bne  %1, %z3, 1f\n"                             \
> > > > > Your code breaks the %0.
> > > >
> > > > What do you mean by breaks here?
> > > >
> > > > In the end of this macro, I intended  to have __retx = (*p & __mask)
> > > > which means the value is clean to be rotated at the end of the macro
> > > > (no need to apply the mask again): r = __ret >> __s;
> > > >
> > > > Also, I assumed we are supposed to return the same variable type
> > > > as the pointer, so this is valid:
> > > > u8 a, *b, c;
> > > > a = xchg(b, c);
> > > >
> > > > Is this correct?
> > > I missed your removing "__ret & mask" at the end. So this may not the problem.
>
> It was actually the problem :)
> Even though I revised a lot, I was missing this which was made clear
> by test-running on a VM:
>
>         "0:    lr.w %0, %2\n"                    \
>         "    and  %0, %0, %z5\n"                \
>         "    bne  %0, %z3, 1f\n"                \
>         "    and  %1, %0, %z6\n"                \
>         "    or   %1, %1, %z4\n"                \
>         "    sc.w" sc_sfx " %1, %1, %2\n"            \
>         "    bnez %1, 0b\n"
>
> If I do "and %0, %0, %z5" there, the line  "and  %1, %0, %z6"  will
> output %1 = 0, which is not the intended.
> You were right! The above fix solves the issue, and I just have to
> mask it after.
>
> Sorry it took me a few days to realize this.
>
>
> > >
> > > Your patch can't boot. After chewing your code for several hours, I
> > > found a problem:
> > > diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> > > index 943f094375c7..67bcce63b267 100644
> > > --- a/arch/riscv/include/asm/cmpxchg.h
> > > +++ b/arch/riscv/include/asm/cmpxchg.h
> > > @@ -14,6 +14,7 @@
> > >  #define __arch_xchg_mask(prepend, append, r, p, n)                     \
> > >  ({                                                                     \
> > >         /* Depends on 2-byte variables being 2-byte aligned */          \
> > > +       volatile ulong *__p = (ulong *)((ulong)(p) & ~0x3);             \
> >
> > Wow, I totally missed this one: I should not assume the processor
> > behavior on unaligned lr/sc.
> > Thanks for spotting this :)
> >
> > >         ulong __s = ((ulong)(p) & 0x3) * BITS_PER_BYTE;                 \
> > >         ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0)   \
> > >                         << __s;                                         \
> > > @@ -29,7 +30,7 @@
> > >                "        sc.w %1, %1, %2\n"                              \
> > >                "        bnez %1, 0b\n"                                  \
> > >                append                                                   \
> > > -              : "=&r" (__retx), "=&r" (__rc), "+A" (*(p))              \
> > > +              : "=&r" (__retx), "=&r" (__rc), "+A" (*(__p))            \
> > >                : "rJ" (__newx), "rJ" (~__mask)                          \
> > >                : "memory");                                             \
> > >                                                                         \
> > > @@ -106,6 +107,7 @@
> > >  #define __arch_cmpxchg_mask(sc_sfx, prepend, append, r, p, o, n)       \
> > >  ({                                                                     \
> > >         /* Depends on 2-byte variables being 2-byte aligned */          \
> > > +       volatile ulong *__p = (ulong *)((ulong)(p) & ~0x3);             \
> > >         ulong __s = ((ulong)(p) & 0x3) * BITS_PER_BYTE;                 \
> > >         ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0)   \
> > >                         << __s;                                         \
> > > @@ -125,7 +127,7 @@
> > >                 "       bnez %1, 0b\n"                                  \
> > >                 append                                                  \
> > >                 "1:\n"                                                  \
> > > -               : "=&r" (__retx), "=&r" (__rc), "+A" (*(p))             \
> > > +               : "=&r" (__retx), "=&r" (__rc), "+A" (*(__p))           \
> > >                 : "rJ" ((long)__oldx), "rJ" (__newx),                   \
> > >                   "rJ" (__mask), "rJ" (~__mask)                         \
> > >                 : "memory");                                            \
> > >
> > > But the lkvm-static still can't boot with paravirt_spinlock .... Are
> > > there any atomic tests in the Linux?
> >
> > I recall reading something about 'locktorture' or so, not sure if useful:
> >
> > https://docs.kernel.org/locking/locktorture.html
> >
> > >
> > > I found you use some "register int variables". Would it cause the problem?
> >
> > Honestly, not sure.
> > I will try inspecting the generated asm for any unexpected changes
> > compared to your version.
>
> changed type to ulong, no changes detected in asm
>
> >
> > >
> > > You can reference this file, and it has passed the lock torture test:
> > > https://github.com/guoren83/linux/blob/sg2042-master-qspinlock-64ilp32_v4/arch/riscv/include/asm/cmpxchg.h
> > >
> > > I also merged your patches with the qspinlock series: (Use the above
> > > cmpxchg.h the lkvm would run normally.)
> > > https://github.com/guoren83/linux/tree/qspinlock_v11
> >
> > Thanks!
> >
> > I should reply as soon as I find anything.
> >
>
> Some changes found (for xchg16):
> - pointer alignment wrong (need mask with ~0x3) : solved. Doesn't seem
> to need to be volatile, though.
>
> - AND with 0x3 for getting shift: I was trusting 2-byte vars to be
> 2-byte aligned , ie. p & 0x1 == 0: may not be true. Solved (it's
> simple)
>
> - on your code, IIUC you also assume ptr & 0x1 == 0, on
>         ulong *__ptr = (ulong *)((ulong)ptr & ~2);
> if it's not aligned, you may have __ptr = 0xYYYYYY01
> (Not saying it's wrong to assume this, just listing It differs on my code)
>
> - also verified your code generates addw and sllw on some points mine
> generates add and sll, don't see any issue there
>
> - There are extra sll and srl in my asm, because I end up casting to
> typeof(*ptr) at the end.
>
> changes found in cmpxchg (__cmpxchg_small_relaxed)
> - mostly the same as above,
>
> - the beginning of my function ends up being smaller, since I do my
> masking differently.
>
> I fixed the above issues, and  ran some userspace debugging tests on a
> VM, and all seems to be correct now.
>
> I then fetched your github repo, replaced my patches v3->v4 on your
> tree, added linux boot parameter "qspinlock" on my qemu cmdline
> (-machine virt): booting and working just fine.
>
> I will send a v4 soon, please give it a test when possible.
>
> Best regards,
> Leo
>
>
>
> > >
> > >
> > >
> > > >
> > > > > > +               append                                                  \
> > > > > > +               "1:\n"                                                  \
> > > > > > +               : "=&r" (__retx), "=&r" (__rc), "+A" (*(p))             \
> > > > > > +               : "rJ" ((long)__oldx), "rJ" (__newx),                   \
> > > > > > +                 "rJ" (__mask), "rJ" (~__mask)                         \
> > > > > > +               : "memory");                                            \
> > > > > > +                                                                       \
> > > > > > +       r = (__typeof__(*(p)))(__retx >> __s);                          \
> > > > > > +})
> > > > > > +
> > > > > >
> > > > > >  #define __arch_cmpxchg(lr_sfx, sc_sfx, prepend, append, r, p, co, o, n)        \
> > > > > >  ({                                                                     \
> > > > > > @@ -98,6 +128,11 @@
> > > > > >         __typeof__(*(ptr)) __ret;                                       \
> > > > > >                                                                         \
> > > > > >         switch (sizeof(*__ptr)) {                                       \
> > > > > > +       case 1:                                                         \
> > > > > > +       case 2:                                                         \
> > > > > > +               __arch_cmpxchg_mask(sc_sfx, prepend, append,            \
> > > > > > +                                       __ret, __ptr, __old, __new);    \
> > > > > > +               break;                                                  \
> > > > > >         case 4:                                                         \
> > > > > >                 __arch_cmpxchg(".w", ".w" sc_sfx, prepend, append,      \
> > > > > >                                 __ret, __ptr, (long), __old, __new);    \
> > > > > > --
> > > > > > 2.41.0
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best Regards
> > > > >  Guo Ren
> > > > >
> > > >
> > >
> > >
> > > --
> > > Best Regards
> > >  Guo Ren
> > >


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [RFC PATCH v3 4/5] riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2
  2023-08-09  2:02           ` Leonardo Bras Soares Passos
  2023-08-10  4:06             ` Leonardo Bras Soares Passos
@ 2023-08-11  2:32             ` Guo Ren
  1 sibling, 0 replies; 13+ messages in thread
From: Guo Ren @ 2023-08-11  2:32 UTC (permalink / raw)
  To: Leonardo Bras Soares Passos
  Cc: Will Deacon, Peter Zijlstra, Boqun Feng, Mark Rutland,
	Paul Walmsley, Palmer Dabbelt, Albert Ou, Andrea Parri,
	Geert Uytterhoeven, Andrzej Hajda, Palmer Dabbelt, linux-kernel,
	linux-riscv

On Wed, Aug 9, 2023 at 10:03 AM Leonardo Bras Soares Passos
<leobras@redhat.com> wrote:
>
> Hello Guo Ren,
>
> On Mon, Aug 7, 2023 at 1:17 PM Leonardo Bras Soares Passos
> <leobras@redhat.com> wrote:
> >
> > On Sat, Aug 5, 2023 at 1:24 AM Guo Ren <guoren@kernel.org> wrote:
> > >
> > > On Sat, Aug 5, 2023 at 11:14 AM Leonardo Bras Soares Passos
> > > <leobras@redhat.com> wrote:
> > > >
> > > > Hello Guo Ren, thanks for the feedback!
> > > >
> > > > On Fri, Aug 4, 2023 at 2:45 PM Guo Ren <guoren@kernel.org> wrote:
> > > > >
> > > > > On Fri, Aug 4, 2023 at 4:49 AM Leonardo Bras <leobras@redhat.com> wrote:
> > > > > >
> > > > > > cmpxchg for variables of size 1-byte and 2-bytes is not yet available for
> > > > > > riscv, even though its present in other architectures such as arm64 and
> > > > > > x86. This could lead to not being able to implement some locking mechanisms
> > > > > > or requiring some rework to make it work properly.
> > > > > >
> > > > > > Implement 1-byte and 2-bytes cmpxchg in order to achieve parity with other
> > > > > > architectures.
> > > > > >
> > > > > > Signed-off-by: Leonardo Bras <leobras@redhat.com>
> > > > > > ---
> > > > > >  arch/riscv/include/asm/cmpxchg.h | 35 ++++++++++++++++++++++++++++++++
> > > > > >  1 file changed, 35 insertions(+)
> > > > > >
> > > > > > diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> > > > > > index 5a07646fae65..dfb433ac544f 100644
> > > > > > --- a/arch/riscv/include/asm/cmpxchg.h
> > > > > > +++ b/arch/riscv/include/asm/cmpxchg.h
> > > > > > @@ -72,6 +72,36 @@
> > > > > >   * indicated by comparing RETURN with OLD.
> > > > > >   */
> > > > > >
> > > > > > +#define __arch_cmpxchg_mask(sc_sfx, prepend, append, r, p, o, n)       \
> > > > > > +({                                                                     \
> > > > > > +       /* Depends on 2-byte variables being 2-byte aligned */          \
> > > > > > +       ulong __s = ((ulong)(p) & 0x3) * BITS_PER_BYTE;                 \
> > > > > > +       ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0)   \
> > > > > > +                       << __s;                                         \
> > > > > > +       ulong __newx = (ulong)(n) << __s;                               \
> > > > > > +       ulong __oldx = (ulong)(o) << __s;                               \
> > > > > > +       ulong __retx;                                                   \
> > > > > > +       register unsigned int __rc;                                     \
> > > > > > +                                                                       \
> > > > > > +       __asm__ __volatile__ (                                          \
> > > > > > +               prepend                                                 \
> > > > > > +               "0:     lr.w %0, %2\n"                                  \
>
>       bne  %0, %z3, 1f\n"                             \
> > > >
> > > > > bug:
> > > > > -               "       and  %0, %0, %z5\n"                             \
> > > > > -               "       bne  %0, %z3, 1f\n"                             \
> > > > > +               "       and  %1, %0, %z5\n"                             \
> > > > > +               "       bne  %1, %z3, 1f\n"                             \
> > > > > Your code breaks the %0.
> > > >
> > > > What do you mean by breaks here?
> > > >
> > > > In the end of this macro, I intended  to have __retx = (*p & __mask)
> > > > which means the value is clean to be rotated at the end of the macro
> > > > (no need to apply the mask again): r = __ret >> __s;
> > > >
> > > > Also, I assumed we are supposed to return the same variable type
> > > > as the pointer, so this is valid:
> > > > u8 a, *b, c;
> > > > a = xchg(b, c);
> > > >
> > > > Is this correct?
> > > I missed your removing "__ret & mask" at the end. So this may not the problem.
>
> It was actually the problem :)
> Even though I revised a lot, I was missing this which was made clear
> by test-running on a VM:
>
>         "0:    lr.w %0, %2\n"                    \
>         "    and  %0, %0, %z5\n"                \
>         "    bne  %0, %z3, 1f\n"                \
>         "    and  %1, %0, %z6\n"                \
>         "    or   %1, %1, %z4\n"                \
>         "    sc.w" sc_sfx " %1, %1, %2\n"            \
>         "    bnez %1, 0b\n"
>
> If I do "and %0, %0, %z5" there, the line  "and  %1, %0, %z6"  will
> output %1 = 0, which is not the intended.
> You were right! The above fix solves the issue, and I just have to
> mask it after.
>
> Sorry it took me a few days to realize this.
Great jot! I would test your 5th patch series, thx.

>
>
> > >
> > > Your patch can't boot. After chewing your code for several hours, I
> > > found a problem:
> > > diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> > > index 943f094375c7..67bcce63b267 100644
> > > --- a/arch/riscv/include/asm/cmpxchg.h
> > > +++ b/arch/riscv/include/asm/cmpxchg.h
> > > @@ -14,6 +14,7 @@
> > >  #define __arch_xchg_mask(prepend, append, r, p, n)                     \
> > >  ({                                                                     \
> > >         /* Depends on 2-byte variables being 2-byte aligned */          \
> > > +       volatile ulong *__p = (ulong *)((ulong)(p) & ~0x3);             \
> >
> > Wow, I totally missed this one: I should not assume the processor
> > behavior on unaligned lr/sc.
> > Thanks for spotting this :)
> >
> > >         ulong __s = ((ulong)(p) & 0x3) * BITS_PER_BYTE;                 \
> > >         ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0)   \
> > >                         << __s;                                         \
> > > @@ -29,7 +30,7 @@
> > >                "        sc.w %1, %1, %2\n"                              \
> > >                "        bnez %1, 0b\n"                                  \
> > >                append                                                   \
> > > -              : "=&r" (__retx), "=&r" (__rc), "+A" (*(p))              \
> > > +              : "=&r" (__retx), "=&r" (__rc), "+A" (*(__p))            \
> > >                : "rJ" (__newx), "rJ" (~__mask)                          \
> > >                : "memory");                                             \
> > >                                                                         \
> > > @@ -106,6 +107,7 @@
> > >  #define __arch_cmpxchg_mask(sc_sfx, prepend, append, r, p, o, n)       \
> > >  ({                                                                     \
> > >         /* Depends on 2-byte variables being 2-byte aligned */          \
> > > +       volatile ulong *__p = (ulong *)((ulong)(p) & ~0x3);             \
> > >         ulong __s = ((ulong)(p) & 0x3) * BITS_PER_BYTE;                 \
> > >         ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0)   \
> > >                         << __s;                                         \
> > > @@ -125,7 +127,7 @@
> > >                 "       bnez %1, 0b\n"                                  \
> > >                 append                                                  \
> > >                 "1:\n"                                                  \
> > > -               : "=&r" (__retx), "=&r" (__rc), "+A" (*(p))             \
> > > +               : "=&r" (__retx), "=&r" (__rc), "+A" (*(__p))           \
> > >                 : "rJ" ((long)__oldx), "rJ" (__newx),                   \
> > >                   "rJ" (__mask), "rJ" (~__mask)                         \
> > >                 : "memory");                                            \
> > >
> > > But the lkvm-static still can't boot with paravirt_spinlock .... Are
> > > there any atomic tests in the Linux?
> >
> > I recall reading something about 'locktorture' or so, not sure if useful:
> >
> > https://docs.kernel.org/locking/locktorture.html
> >
> > >
> > > I found you use some "register int variables". Would it cause the problem?
> >
> > Honestly, not sure.
> > I will try inspecting the generated asm for any unexpected changes
> > compared to your version.
>
> changed type to ulong, no changes detected in asm
>
> >
> > >
> > > You can reference this file, and it has passed the lock torture test:
> > > https://github.com/guoren83/linux/blob/sg2042-master-qspinlock-64ilp32_v4/arch/riscv/include/asm/cmpxchg.h
> > >
> > > I also merged your patches with the qspinlock series: (Use the above
> > > cmpxchg.h the lkvm would run normally.)
> > > https://github.com/guoren83/linux/tree/qspinlock_v11
> >
> > Thanks!
> >
> > I should reply as soon as I find anything.
> >
>
> Some changes found (for xchg16):
> - pointer alignment wrong (need mask with ~0x3) : solved. Doesn't seem
> to need to be volatile, though.
>
> - AND with 0x3 for getting shift: I was trusting 2-byte vars to be
> 2-byte aligned , ie. p & 0x1 == 0: may not be true. Solved (it's
> simple)
>
> - on your code, IIUC you also assume ptr & 0x1 == 0, on
>         ulong *__ptr = (ulong *)((ulong)ptr & ~2);
> if it's not aligned, you may have __ptr = 0xYYYYYY01
> (Not saying it's wrong to assume this, just listing It differs on my code)
>
> - also verified your code generates addw and sllw on some points mine
> generates add and sll, don't see any issue there
>
> - There are extra sll and srl in my asm, because I end up casting to
> typeof(*ptr) at the end.
>
> changes found in cmpxchg (__cmpxchg_small_relaxed)
> - mostly the same as above,
>
> - the beginning of my function ends up being smaller, since I do my
> masking differently.
>
> I fixed the above issues, and  ran some userspace debugging tests on a
> VM, and all seems to be correct now.
>
> I then fetched your github repo, replaced my patches v3->v4 on your
> tree, added linux boot parameter "qspinlock" on my qemu cmdline
> (-machine virt): booting and working just fine.
>
> I will send a v4 soon, please give it a test when possible.
>
> Best regards,
> Leo
>
>
>
> > >
> > >
> > >
> > > >
> > > > > > +               append                                                  \
> > > > > > +               "1:\n"                                                  \
> > > > > > +               : "=&r" (__retx), "=&r" (__rc), "+A" (*(p))             \
> > > > > > +               : "rJ" ((long)__oldx), "rJ" (__newx),                   \
> > > > > > +                 "rJ" (__mask), "rJ" (~__mask)                         \
> > > > > > +               : "memory");                                            \
> > > > > > +                                                                       \
> > > > > > +       r = (__typeof__(*(p)))(__retx >> __s);                          \
> > > > > > +})
> > > > > > +
> > > > > >
> > > > > >  #define __arch_cmpxchg(lr_sfx, sc_sfx, prepend, append, r, p, co, o, n)        \
> > > > > >  ({                                                                     \
> > > > > > @@ -98,6 +128,11 @@
> > > > > >         __typeof__(*(ptr)) __ret;                                       \
> > > > > >                                                                         \
> > > > > >         switch (sizeof(*__ptr)) {                                       \
> > > > > > +       case 1:                                                         \
> > > > > > +       case 2:                                                         \
> > > > > > +               __arch_cmpxchg_mask(sc_sfx, prepend, append,            \
> > > > > > +                                       __ret, __ptr, __old, __new);    \
> > > > > > +               break;                                                  \
> > > > > >         case 4:                                                         \
> > > > > >                 __arch_cmpxchg(".w", ".w" sc_sfx, prepend, append,      \
> > > > > >                                 __ret, __ptr, (long), __old, __new);    \
> > > > > > --
> > > > > > 2.41.0
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best Regards
> > > > >  Guo Ren
> > > > >
> > > >
> > >
> > >
> > > --
> > > Best Regards
> > >  Guo Ren
> > >
>


-- 
Best Regards
 Guo Ren

^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2023-08-11  2:32 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-08-04  8:48 [RFC PATCH v3 0/5] Rework & improve riscv cmpxchg.h and atomic.h Leonardo Bras
2023-08-04  8:48 ` [RFC PATCH v3 1/5] riscv/cmpxchg: Deduplicate xchg() asm functions Leonardo Bras
2023-08-04  8:48 ` [RFC PATCH v3 2/5] riscv/cmpxchg: Deduplicate cmpxchg() asm and macros Leonardo Bras
2023-08-04  8:48 ` [RFC PATCH v3 3/5] riscv/atomic.h : Deduplicate arch_atomic.* Leonardo Bras
2023-08-04  8:48 ` [RFC PATCH v3 4/5] riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2 Leonardo Bras
2023-08-04 17:45   ` Guo Ren
2023-08-05  3:14     ` Leonardo Bras Soares Passos
2023-08-05  4:24       ` Guo Ren
2023-08-07 16:17         ` Leonardo Bras Soares Passos
2023-08-09  2:02           ` Leonardo Bras Soares Passos
2023-08-10  4:06             ` Leonardo Bras Soares Passos
2023-08-11  2:32             ` Guo Ren
2023-08-04  8:48 ` [RFC PATCH v3 5/5] riscv/cmpxchg: Implement xchg " Leonardo Bras

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®