Re: per_cpu fixes

[Rusty Russell <rusty@rustcorp.com.au>]

In message <200307092120.h69LKTBH002759@napali.hpl.hp.com> you write:
> Rusty,
> 
> Care needs to be taken when taking the address of a CPU-local
> variable, because otherwise things may break when comparing addresses
> on a platform which uses virtual remapping to implement such
> variables.  In particular, it's almost always unsafe to use the
> address of a per-CPU variable which contains a "struct list", because
> the list-manipulation routines use the list-head address to detect the
> end of the list etc.

The horror.  Such rules are entirely too much problem to push on the
poor programmer 8(

When I implemented this, I imagined archs putting their per-cpu offset
inside a register, so they could get to their vars in one instruction,
but not the IA64 remapping thing.  We are now suffering because of my
limited imagination (which David has commented on before 8).

A compromise is possible.  I believe that the address of a per-cpu
variable *must* be the same everywhere, but we can provide get & set
macros which never expose an lvalue, and on IA64 could use the pinned
TLB thing:

/* Usage: set_cpu_local(myint, = 1), or set_cpu_local(mystruct,.member = 1) */
#define set_cpu_local(var, assign) ...

/* Usage: get_cpu_local(myint), or get_cpu_local(mystruct).member */
#define get_cpu_local(var) ...

I rejected such an approach before when Andi Kleen asked for it (IIRC
he wanted to use %gs as the per-cpu ptr, but couldn't easily produce
an lvalue), because I wanted a nice, clean interface.  However, recent
gcc handles the struct result of the statement expression flawlessly
AFAICT, so I'm less inclined to resist.

Thoughts?
Rusty.
PS. David, this is your revenge for making more work for you, isn't it?
--
  Anyone who quotes me in their sig is an idiot. -- Rusty Russell.