[RFC 0/3] mutex: Reduce spinning contention when there is no lock owner

[Jason Low <jason.low2@hp.com>]

While optimistic spinning is beneficial to performance, I have found that
threads can potentially spin for a long time while there is no lock owner
during high contention cases. In these scenarios, too much spinning can reduce
performance. This RFC patchset attempts to address some of the issues with
spinning too much with no owner.

Patch 1 changes the mutex_can_spin_on_owner() function to address the
need_resched() case. Patch 2 modifies the way mutex spinners are queued to
reduce mspin_lock and mspin_unlock overhead when there is no owner, and is also
necessary patch for patch 3. Patch 3 limits the number of times each thread
can spin on lock->count when there is no lock owner.

This change benefit the AIM7 fserver (run on disk) at 1000-2000 users
on an 8 socket (80 core) box, with a +19.5% gain with 3.13-rc7 + patchset
compared to the baseline 3.13-rc7 kernel. At 100-900 users, the
% gain was about 4.9%, and there wasn't much performance difference
at 10-90 users. On a 2 socket (8 core) box, there was about a 5.5%
improvement at 1000-2000 users.