Re: per-cpu operation madness vs validation

[Peter Zijlstra <peterz@infradead.org>]

On Wed, 2011-07-27 at 09:20 -0500, Christoph Lameter wrote:
> On Wed, 27 Jul 2011, Peter Zijlstra wrote:
> 
> > > Possibly we could reduce all this percpu madness back to one form
> > > (__this_cpu_*) and require that when used a lock of the percpu_lock_t is
> > > taken.
> >
> > get_cpu_var()/put_cpu_var() were supposed to provide such delineation as
> > well, but you've been actively destroying things like that with the
> > recent per-cpu work.
> 
> The per cpu work is *not* focused on sections that access per cpu data, so
> how could it destroy that? Nothing is changed there so far. The this_cpu
> ops are introducing per cpu atomic operations that are safe and cheap
> regardless of the execution context. The primary initial motivation was
> incrementing per cpu counters without having to disabling interrupts
> and/or preemption and it grew from there.

I think you need to look at 20b876918c065818b3574a426d418f68b4f8ad19 and
try again. You removed get_cpu_var()/put_cpu_var() and replaced it with
naked preempt_disable()/preempt_enable(). That's loosing information
right there.

Also, I think if you do s/__this_cpu/this_cpu/ on that function you can
simply drop the preempt_enable/disable things.

> The atomic per cpu operations (like the this_cpu_cmpxchg) allow the
> construction of cheap sections that would satisfy your goals too (with
> some work and the use of transaction ids instead of locking). See the slub
> fastpath f.e.

The slub fastpath is a shining example of crap. Its horrid code and
there was nothing wrong with actually disabling preemption over it, even
for -rt. Sure you made it go faster, but thats not the point.

Most of the slub problems with -rt are in the slow path where you still
have per-cpu assumptions and keep preempt/irqs disabled.

I don't think it is possible, nor desirable, to wreck all per-cpu usage
in the kernel that way.