Re: wake_q memory ordering

[Davidlohr Bueso <dave@stgolabs.net>]

On Thu, 10 Oct 2019, Peter Zijlstra wrote:

>On Thu, Oct 10, 2019 at 02:13:47PM +0200, Manfred Spraul wrote:
>> Hi Peter,
>>
>> On 10/10/19 1:42 PM, Peter Zijlstra wrote:
>> > On Thu, Oct 10, 2019 at 12:41:11PM +0200, Manfred Spraul wrote:
>> > > Hi,
>> > >
>> > > Waiman Long noticed that the memory barriers in sem_lock() are not really
>> > > documented, and while adding documentation, I ended up with one case where
>> > > I'm not certain about the wake_q code:
>> > >
>> > > Questions:
>> > > - Does smp_mb__before_atomic() + a (failed) cmpxchg_relaxed provide an
>> > >    ordering guarantee?
>> > Yep. Either the atomic instruction implies ordering (eg. x86 LOCK
>> > prefix) or it doesn't (most RISC LL/SC), if it does,
>> > smp_mb__{before,after}_atomic() are a NO-OP and the ordering is
>> > unconditinoal, if it does not, then smp_mb__{before,after}_atomic() are
>> > unconditional barriers.
>>
>> And _relaxed() differs from "normal" cmpxchg only for LL/SC architectures,
>> correct?
>
>Indeed.
>
>> Therefore smp_mb__{before,after}_atomic() may be combined with
>> cmpxchg_relaxed, to form a full memory barrier, on all archs.
>
>Just so.

We might want something like this?

----8<---------------------------------------------------------

From: Davidlohr Bueso <dave@stgolabs.net>
Subject: [PATCH] Documentation/memory-barriers.txt: Mention smp_mb__{before,after}_atomic() and CAS

Explicitly mention possible usages to guarantee serialization even upon
failed cmpxchg (or similar) calls along with smp_mb__{before,after}_atomic().

Signed-off-by: Davidlohr Bueso <dbueso@suse.de>
---
 Documentation/memory-barriers.txt | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt
index 1adbb8a371c7..5d2873d4b442 100644
--- a/Documentation/memory-barriers.txt
+++ b/Documentation/memory-barriers.txt
@@ -1890,6 +1890,18 @@ There are some more advanced barrier functions:
      This makes sure that the death mark on the object is perceived to be set
      *before* the reference counter is decremented.
 
+     Similarly, these barriers can be used to guarantee serialization for atomic
+     RMW calls on architectures which may not imply memory barriers upon failure.
+
+	obj->next = NULL;
+	smp_mb__before_atomic()
+	if (cmpxchg(&obj->ptr, NULL, val))
+		return;
+
+     This makes sure that the store to the next pointer always has smp_store_mb()
+     semantics. As such, smp_mb__{before,after}_atomic() calls allow optimizing
+     the barrier usage by finer grained serialization.
+
      See Documentation/atomic_{t,bitops}.txt for more information.
 
 
-- 
2.16.4