Re: [rfc] "fair" rw spinlocks

[Nick Piggin <npiggin@suse.de>]

On Sat, Nov 28, 2009 at 09:30:18AM -0800, Linus Torvalds wrote:
> 
> 
> On Mon, 23 Nov 2009, Nick Piggin wrote:
> > 
> > Last time this issue came up that I could see, I don't think
> > there were objections to making rwlocks fair, the main
> > difficulty seemed to be that we allow reentrant read locks
> > (so a write lock waiting must not block arbitrary read lockers).
> 
> We have at least one major rwlock user - tasklist_lock or whatever. And 
> that one definitely depends on being able to do 'rwlock()' in an 
> interrupt, without other rwlock'ers having to disable irq's (even if there 
> might be a new writer coming in on another cpu).
> 
> That usage case _might_ be turned into RCU or something similar, in which 
> case I don't think any major rwlock users remain. However, if that's the 
> case, then why should anybody care about fairness any more either?

We do have quite a large number of rwlocks really. If they are so
important as to be rwlocks, then presumably means we can expect
multiple readers in the critical sections. Therefore, we can have
a possibility of livelock. (It isn't so hard, I have a test case
which _totally_ livelocks the write-side of the tasklist lock on a
2 socket opteron, just with several threads calling wait(2)).

I'm just not sure why you don't think the other rwlocks are a problem.
Certainly they are probably not as central and visible or likely to
be a problem as tasklist_lock, but at least for a DoS, couldn't they
be an issue?

But anyway, yes I would be happy to just start with tasklist_lock
for now and if we could keep slowly trying to move away from rwlocks
in other areas, that would be good.

 
> So as far as I can tell, we have only one real user of rwlocks where 
> livelocks might be relevant, but that one real user absolutely _requires_ 
> the unfair behavior.

Yes, although the behaviour required is that it can be recursively
acquired. So we could still have a lock that disallows new non recursive
read acquires when there is a pending write locker.

RCU seems nicer, but tasklist lock locking scares me so I wanted to fix
it the easy way :)