Re: [PATCH v3] locking/rwsem: reduce spinlock contention in wakeup after up_read/up_write

[Davidlohr Bueso <dave@stgolabs.net>]

On Mon, 2015-04-27 at 16:25 -0400, Waiman Long wrote:
> >> +	/*
> >> +	 * up_write() cleared the owner field before calling this function.
> >> +	 * If that field is now set, a writer must have stolen the lock and
> >> +	 * the wakeup operation should be aborted.
> >> +	 */
> >> +	if (rwsem_has_active_writer(sem))
> >> +		goto out;
> > We currently allow small races between rwsem owner and counter checks.
> > And __rwsem_do_wake() can be called by checking the former -- and lock
> > stealing is done with the counter as well. Please see below how we back
> > out of such cases, as it is very much considered when granting the next
> > reader. So nack to this as is, sorry.
> 
> If the first one in the queue is a writer, wake_up_process() may be 
> called directly which can be quite expensive if the lock has already 
> been stolen as the task will have to sleep again.

But how can this occur? Lock stealing takes form in two places:

1) fastpath: only if the counter is 0 -- which, since we are discussing
waking up waiter(s) code, obviously cannot occur.

2) With the cmpxchg() in rwsem_try_write_lock(), which is serialized
with the wait_lock, so again this cannot occur.

Which is why this is not considered in __rwsem_do_wake() when waking the
writer fist in the queue.

Thanks,
Davidlohr