Re: [PATCH] Documentation: Remove misleading examples of the barriers in wake_*()

[Will Deacon <will.deacon@arm.com>]

On Tue, Oct 06, 2015 at 06:24:23PM +0200, Peter Zijlstra wrote:
> On Tue, Oct 06, 2015 at 06:04:50PM +0200, Peter Zijlstra wrote:
> > On Mon, Sep 21, 2015 at 07:46:11PM +0200, Oleg Nesterov wrote:
> > > On 09/18, Peter Zijlstra wrote:
> > > >
> > > > the text is correct, right?
> > > 
> > > Yes, it looks good to me and helpful.
> > > 
> > > But damn. I forgot why exactly try_to_wake_up() needs rmb() after
> > > ->on_cpu check... It looks reasonable in any case, but I do not
> > > see any strong reason immediately.
> > 
> > I read it like the smp_rmb() we have for
> > acquire__after_spin_is_unlocked. Except, as you note below, we need to
> > need an smp_read_barrier_depends for control barriers as well....
> 
> > Yes, but I'm not sure we should go write:
> > 
> > 	while (READ_ONCE_CTRL(p->on_cpu))
> > 		cpu_relax();
> > 
> > Or:
> > 
> > 	while (p->on_cpu)
> > 		cpu_relax();
> > 
> > 	smp_read_barrier_depends();
> > 
> > It seems to me that doing the smp_mb() (for Alpha) inside the loop might
> > be sub-optimal.
> 
> And also referring to:
> 
>   lkml.kernel.org/r/20150812133109.GA8266@redhat.com
> 
> Do we want something like this?
> 
> #define smp_spin_acquire(cond) do {		\
> 	while (cond)				\
> 		cpu_relax();			\
> 	smp_read_barrier_depends(); /* ctrl */	\
> 	smp_rmb(); /* ctrl + rmb := acquire */	\
> } while (0)
> 
> And use it like:
> 
> 	smp_spin_acquire(raw_spin_is_locked(&task->pi_lock));
> 
> That might work for your task_work_run() and the scheduler case,
> although it might be somewhat awkward for sem_wait_array().

I could *really* use something like this for implementing power-saving
busy loops for arch/arm64 (i.e. in the qrwlock code). We have a WFE
instruction (wait for event) that can stop the processor clock and resume
it when the exclusive monitor is cleared (i.e. a cacheline migrates to
another CPU). That means we can implement a targetted wake-up when an
unlocker writes to a node in a queued lock, which isn't something
expressible with cpu_relax alone.

Will