Re: [PATCH] fix misplaced mb() in rcu_enter/exit_nohz()

[Oleg Nesterov <oleg@tv-sign.ru>]

On 03/17, Paul E. McKenney wrote:
>
> On Mon, Mar 17, 2008 at 11:17:41PM +0300, Oleg Nesterov wrote:
> > (to clarify: my question is completely offtopic to this patch)
> > On 03/17, Paul E. McKenney wrote:
> > > On Mon, Mar 17, 2008 at 09:30:47PM +0300, Oleg Nesterov wrote:
> > > > I'm not sure the code below is up to date, but what I have in
> > > > arch/s390/kernel/time.c is:
> > > > 
> > > > 	stop_hz_timer:
> > > > 
> > > > 		cpu_set(cpu, nohz_cpu_mask);
> > > > 		
> > > > 		if (rcu_needs_cpu(cpu) || local_softirq_pending()) {
> > > > 			cpu_clear(cpu, nohz_cpu_mask);
> > > > 			return;
> > > > 		}
> > > > 
> > > > Don't we need smp_mb() after cpu_set() ?
> > > 
> > > S390's memory model is quite strong, so it might not be needed.
> > 
> > OK, in that case we shouldn't worry.
> 
> I don't know if I would go -that- far.  ;-)
> 
> > > In any
> > > case, if needed, it goes -before- the cpu_set(), because the problems
> > > would arise if prior RCU read-side critical sections were to be reordered
> > > to follow this cpu_set(), right?
> > 
> > No, but it is very possible I missed something.
> > 
> > What if rcu_needs_cpu(cpu) is executed before cpu_set(cpu, nohz_cpu_mask)?
> > It can miss rcu_start_batch() -> rcp->cur++ and return false, but at the
> > same time rcu_start_batch() may see nohz_cpu_mask without this CPU.
> 
> If you mean that the rcu_needs_cpu() executes before the cpu_set() in
> the code fragment above, while the rcu_start_batch() executes on some
> other CPU?

Yes, and __rcu_pending() sees the old value of ->cur.

IOW. Suppose that this CPU reads rcp->cur out of order. To simplify, let's
suppose that stop_hz_timer() on CPU_0 in fact does

	xxx = rcu_needs_cpu(cpu); // false

	// ---- WINDOW ------

	cpu_set(cpu, nohz_cpu_mask);

	if (xxx || local_softirq_pending()) {
		... abort ...
	}

	...proceed...

Another CPU does rcu_start_batch() in the window above. In that case
rcp->cpumask will include CPU_0, and the grace period can't be completed
untill CPU_0 is "woken".

Oleg.