Re: queued spinlock code and results

[Linus Torvalds <torvalds@linux-foundation.org>]

On Sun, 8 Jul 2007, Andi Kleen wrote:
>
> Nick Piggin <npiggin@suse.de> writes:
> 
> > I made some tests of the queued spinlock code using userspace test code on
> > 64-bit processors. I believe the xadd based code no longer has any theoretical
> > memory ordering problems.
> 
> Linus, the background of this is that on 8 socket Opteron systems
> the current spinlocks can become very unfair to the point of severe 
> starvation. These boxes are becomming more common.

Yeah, considering the numbers, I don't have any real objections here.

I would ask that the code be given to both Intel and AMD engineers to look 
over, just to verify that the lfence is sufficient (or whether it's even 
needed), but I think the use of "xaddw" to both increment _and_ load the 
old value for the non-contention case is an obviously good (and clever) 
way to handle that one, and even if we'd have to add something heavier 
than the lfence to the contended case, it looks fine to me.

So the only remaining issue is that unfairness is probably really good for 
some loads (not just for the spinlock itself - it will likely cause much 
better cache behaviour for stuff _inside_ the lock to stay on the same 
core), but I don't think we want to optimize for the contended case 
anyway, so that's more of a "it will be interesting to see" kind of 
comment.

In short: if we can have AMD/Intel engineers look this over for any subtle 
issues, and they are happy, then I'm happy. 

		Linus