Re: [rfc] "fair" rw spinlocks

[Nick Piggin <npiggin@suse.de>]

On Mon, Nov 30, 2009 at 09:05:57AM -0800, Linus Torvalds wrote:
> 
> 
> On Mon, 30 Nov 2009, Paul E. McKenney wrote:
> > 
> > My suggestion would be to put the nesting counter in the task structure
> > to avoid this problem.
> 
> It still doesn't end up being all that cheap. Now you'd need to disable 
> preemption in order to fix the race between the local counter and the real 
> lock.
> 
> That should be cheaper than cli/sti, but the downside is that now you need 
> that task struct pointer (both for the preemption disable and the 
> counter), so now you're adding some register pressure too. Of course, 
> maybe you don't even want to inline it anyway, in which case that doesn't 
> matter.
> 
> One advantage with your suggestion of using preemption is that (unlike irq 
> disables) you can keep the preemt counter over the whole lock, so you 
> don't need to re-do the preempt disable/enable in both read-lock and 
> read-unlock.
> 
> So you might end up with something like (UNTESTED!):
> 
> 	static void tasklist_write_lock(void)
> 	{
> 		spin_lock_irq(&tasklist_lock);
> 	}
> 
> 	static void tasklist_write_unlock(void)
> 	{
> 		spin_unlock_irq(&tasklist_lock);
> 	}

Two questions. Firstly, does tasklist_lock benefit much from read
side paralellism? Looking at some of the critical sections some seem
to hold it for quite a while (over task and thread iterations). So
it might not be the right thing to convert it to a spinlock?

Secondly:

> 	static void tasklist_read_lock(void)
> 	{
> 		preempt_disable();
> 		if (!current->tasklist_count++)

What happens if an interrupt and nested tasklist_read_lock() happens
here?

> 			spin_lock(&tasklist_lock);
> 	}
> 
> 	static void tasklist_read_unlock(void)
> 	{
> 		if (!--current->tasklist_count)
> 			spin_unlock(&tasklist_lock);
> 		preempt_enable();
> 	}
> 
> And the upside, of course, is that a spin_unlock() is cheaper than a 
> read_unlock() (no serializing atomic op), so while there is overhead, 
> there are also some advantages.. Maybe that atomic op advantage is enough 
> to offset the extra instructions.
> 
> And maybe we could use 'raw_spin_[un]lock()' in the above read-[un]lock 
> sequences, since tasklist_lock is pretty special, and since we do the 
> preempt disable by hand (no need to do it again in the spinlock code). 
> That looks like it might cut down the overhead of all of the above to 
> almost nothing for what is probably the common case today (ie preemption 
> enabled).
> 
> 			Linus