Re: [patch 04/41] cpu ops: Core piece for generic atomic per cpu operations

[Christoph Lameter <clameter@sgi.com>]

On Fri, 30 May 2008, Rusty Russell wrote:

> > No its not! In order to increment a per cpu value you need to calculate
> > the per cpu pointer address in the current per cpu segment.
> 
> Christoph, you just missed it, that's all.  Look at cpu_local_read et al in 
> include/asm-i386/local.h (ie. before the x86 mergers chose the lowest common 
> denominator one).

There is no doubt that local_t does perform an atomic vs. interrupt inc 
for example. But its not usable. Because you need to determine the address 
of the local_t belonging to the current processor first. As soon as you 
have loaded a processor specific address you can no longer be preempted 
because that may change the processor and then the wrong address may be 
increment (and then we have a race again since now we are incrementing 
counters belonging to other processors). So local_t at mininum requires 
disabling preempt.
 
Believe me I have tried to use local_t repeatedly for vm statistics etc. 
It always fails on that issue. See f.e. the patch that converts vmstat to 
cpu alloc and compare with my initial local_t based implementation 2 years 
ago that bombed out because I assumed that local_t would work right.

cpu ops does both

1. The determination of the address of the object belonging to the local 
processor.

and

2. The RMW

in one instruction. That avoids having to disable preemption or interrupts 
and it shortens the instructions significantly.