Re: [PATCH] Documentation: Remove misleading examples of the barriers in wake_*()

[Peter Zijlstra <peterz@infradead.org>]

On Tue, Oct 06, 2015 at 06:04:50PM +0200, Peter Zijlstra wrote:
> On Mon, Sep 21, 2015 at 07:46:11PM +0200, Oleg Nesterov wrote:
> > On 09/18, Peter Zijlstra wrote:
> > >
> > > the text is correct, right?
> > 
> > Yes, it looks good to me and helpful.
> > 
> > But damn. I forgot why exactly try_to_wake_up() needs rmb() after
> > ->on_cpu check... It looks reasonable in any case, but I do not
> > see any strong reason immediately.
> 
> I read it like the smp_rmb() we have for
> acquire__after_spin_is_unlocked. Except, as you note below, we need to
> need an smp_read_barrier_depends for control barriers as well....

> Yes, but I'm not sure we should go write:
> 
> 	while (READ_ONCE_CTRL(p->on_cpu))
> 		cpu_relax();
> 
> Or:
> 
> 	while (p->on_cpu)
> 		cpu_relax();
> 
> 	smp_read_barrier_depends();
> 
> It seems to me that doing the smp_mb() (for Alpha) inside the loop might
> be sub-optimal.

And also referring to:

  lkml.kernel.org/r/20150812133109.GA8266@redhat.com

Do we want something like this?

#define smp_spin_acquire(cond) do {		\
	while (cond)				\
		cpu_relax();			\
	smp_read_barrier_depends(); /* ctrl */	\
	smp_rmb(); /* ctrl + rmb := acquire */	\
} while (0)

And use it like:

	smp_spin_acquire(raw_spin_is_locked(&task->pi_lock));

That might work for your task_work_run() and the scheduler case,
although it might be somewhat awkward for sem_wait_array().