Re: [RFC][PATCH 5/5] percpu-rwsem: Optimize readers and reduce global impact

[Peter Zijlstra <peterz@infradead.org>]

On Sat, May 30, 2015 at 10:49:00PM +0200, Oleg Nesterov wrote:
> > On Fri, May 29, 2015 at 1:09 PM, Oleg Nesterov <oleg@redhat.com> wrote:
> > >
> > > Doesn't it need mb() before "state = readers_slow" to ensure
> > > "release" semantics?

> __percpu_down_read() lacks another mb() after the "state != BLOCK"
> check for the same reason, and we can use smp_load_acquire(state)
> instead.

I made the below modification to the patch.

---
--- a/kernel/locking/percpu-rwsem.c
+++ b/kernel/locking/percpu-rwsem.c
@@ -51,7 +51,11 @@ void __percpu_down_read(struct percpu_rw
 
 	smp_mb(); /* A matches D */
 
-	if (likely(sem->state != readers_block))
+	/*
+	 * If !readers_block the critical section starts here, matched by the
+	 * release in percpu_up_write().
+	 */
+	if (likely(smp_load_acquire(sem->state) != readers_block))
 		return;
 
 	/*
@@ -154,8 +158,11 @@ void percpu_up_write(struct percpu_rw_se
 	 * One reason that we cannot just immediately flip to readers_fast is
 	 * that new readers might fail to see the results of this writer's
 	 * critical section.
+	 *
+	 * Therefore we force it through the slow path which guarantees an
+	 * acquire and thereby guarantees the critical section's consistency.
 	 */
-	sem->state = readers_slow;
+	smp_store_release(sem->state, readers_slow);
 
 	/*
 	 * Release the write lock, this will allow readers back in the game.