Re: SMP syncronization on AMD processors (broken?)

[Andrew Morton <akpm@osdl.org>]

Andrey Savochkin <saw@sawoct.com> wrote:
>
> I start to wonder about existing mainstream code, presumably bug-free, that
>  uses spinlocks without any problematic restart.
>  If one day some piece starts to be called too often by some legitimate
>  reasons, it might fall into the same pattern and completely block others who
>  want to take the same spinlock.

There are surely places in the kernel where we do get into a corner and do
drop a lock in the expectation that another CPU will acquire it and get us
out of the mess we got ourselves into.  One is
fs/jbd/commit.c:inverted_lock(), and I've always been afraid that it might
be vulnerable to NUMA capture effects such as you describe.

Presently inverted_lock() just does a completely pointless schedule() and
remains that way because nobody has reported it locking up yet :(

>  I'm not advocating for changing spinlock implementation, it's just a
>  thought...

It would make sense in these cases if there was some primitive which we
could call which says "hey, I expect+want another CPU to grab this lock in
preference to this CPU".

Note that inverted_lock() uses bit_spin_lock() - that's just a detail but
it does illustrate that any such primitive shouldn't be specific to just
spinlocks and rwlocks.

(And yes, I once looked at fixing JBD "for real" but it was really hard. 
Fact is, the commit code and the filesystem code really do want to take
those locks in opposite order).