Re: [PATCH -tip v2 1/2] locking/rtmutex: Support spin on owner

[Thomas Gleixner <tglx@linutronix.de>]

On Wed, 1 Jul 2015, Davidlohr Bueso wrote:

> Similar to what we have in other locks, particularly regular mutexes, the
> idea is that as long as the owner is running, there is a fair chance it'll
> release the lock soon, and thus a task trying to acquire the rtmutex will
> better off spinning instead of blocking immediately after the fastpath.
> Conditions to stop spinning and enter the slowpath are simple:
> 
> (1) Upon need_resched()
> (2) Current lock owner blocks
>  
> Because rtmutexes track the lock owner atomically, we can extend the fastpath
> to continue polling on the lock owner via cmpxchg(lock->owner, NULL, current).
> 
> However, this is a conservative approach, such that if there are any waiters
> in-line, we stop spinning and immediately take the traditional slowpath. This
> allows priority boosting to take precedence over spinning, as otherwise we
> could starve a higher priority queued-up task (ie: top waiter) if spinners
> constantly steal the lock.

I'm a bit wary about the whole approach. In the RT tree we spin AFTER
we've enqueued the waiter and run priority boosting. While I can see
the charm of your approach, i.e. avoiding the prio boost dance for the
simple case, this can introduce larger latencies.

T1 (prio = 0)  	      T2 (prio = 50)
 lock(RTM);
		      lock(RTM);
		       spin()
-->preemption	        
T3 (prio = 10) 		leave spin, because owner is not on cpu
   	   	       
		       enqueue();
		       boost();
		       schedule();
-->preemption
T1 (prio = 50)

So we trade two extra context switches in the worst case for an
enhancement of performance in the normal case. I cannot quantify the
impact of this, but we really need to evaluate that proper before
going there.

Aside of that, if the lock is really contended, then you force all
spinners off the cpu, if one of the spinners starts blocking simply
because you have no idea which one is the top prio spinner.

T1 (prio = 0)  	      T2 (prio = 50)  	  T3 (prio = 10)
 lock(RTM);
		      lock(RTM);	  lock(RTM);
		       spin()		   spin();
		               		  --> preemption
					   enqueue()
					   boost();
					   schedule();
		       sees waiter bit
		       enqueue();
		       boost();
		       schedule();

T2 could happily keep spinning despite T3 going to sleep. I'm not sure
if that's what we want to achieve.

Need to think about it some more, but I wanted to give you something
to think about as well :)

Thanks,

	tglx