Re: [PATCH-tip v3 04/14] locking/rwsem: Implement lock handoff to prevent lock starvation

[Waiman Long <longman@redhat.com>]

On 04/10/2019 03:38 PM, Peter Zijlstra wrote:
> Hurph, I was still looking at v2.. I suppose I'll go stare at this
> verison, I don't think you said there were many changes, right?
>
> This version seems to still suffer that HANDOFF issue I found on v2.

It is mainly minor adjustments. I was trying to add two more patches.
While at it, make some minor changes. I will address your concern in a
separate mail later today.

>
> On Wed, Apr 10, 2019 at 02:42:21PM -0400, Waiman Long wrote:
>> Because of writer lock stealing, it is possible that a constant
>> stream of incoming writers will cause a waiting writer or reader to
>> wait indefinitely leading to lock starvation.
>>
>> The mutex code has a lock handoff mechanism to prevent lock starvation.
>> This patch implements a similar lock handoff mechanism to disable
>> lock stealing and force lock handoff to the first waiter in the queue
>> after at least a 4ms waiting period unless it is a RT writer task which
>> doesn't need to wait. The waiting period is used to avoid discouraging
>> lock stealing too much to affect performance.
>>
>> A rwsem microbenchmark was run for 5 seconds on a 2-socket 40-core
>> 80-thread Skylake system with a v5.1 based kernel and 240 write_lock
>> threads with 5us sleep critical section.
>>
>> Before the patch, the min/mean/max numbers of locking operations for
>> the locking threads were 1/7,792/173,696. After the patch, the figures
>> became 5,842/6,542/7,458.  It can be seen that the rwsem became much
>> more fair, though there was a drop of about 16% in the mean locking
>> operations done which was a tradeoff of having better fairness.
>>
>> Making the waiter set the handoff bit right after the first wakeup can
> What does 'right after the first wakeup' mean? If that the top-waiter
> setting it if it fails to acquire the lock due to steals?
Yes. It is after the first sleep.

Cheers,
Longman