Re: PREEMPT_RT vs bcache

[Thomas Gleixner <tglx@linutronix.de>]

On Thu, 8 Aug 2013, Christoph Hellwig wrote:

> On Thu, Aug 08, 2013 at 12:43:26AM -0700, Kent Overstreet wrote:
> > I seem to recall from looking at the logs that you just removed them
> > because all the old users could be and were converted to something
> > saner, for what they were doing (using them as completions, I want to
> > say?)
> 
> We explicitly converted them away so that we could kill it.  This was
> a joint project with Thomas.
> 
> > Bcache isn't using the rw sem as a completion though, it really is a
> > read/write lock that protects a specific data structure, and  where
> > we're taking a read lock for the duration of write IOs - and since bios
> > are asynchronous, that's why we need the non_owner() bit.
> 
> Part of this commit was to make the rw_semaphore behaviour similar to
> plain mutex, that is making sure there is exactly one owner and not
> different processes locking/unlocking it.  This is useful for PI (that's
> why the rt folks care), lock debugging and kinds of other use cases.

Right. We had to implement an anon_rw_semaphore version, which caused
more headache than it was worth the trouble.

The solution for one of the non owner use cases was something like the
below:

      read_lock(x->lock);
      atomic_inc(x->io_active);
      launch_io();
      read_unlock(x->lock);

On the writer side:

      write_lock(x->lock);
      while (atomic_read(x->io_active) {
      	    write_unlock(x->lock);
      	    wait_event(x->wait_for_io, !atomic_read(x->io_active));
	    write_lock(x->io_active);
      }
      ....

On the io side:

       complete_io()
         if (atomic_dec_and_test(x->io_active) &&
             waitqueue_active(x->wait_for_io))
                wake_up(x->wait_for_io);


That would fit into the bcache use case afacit.

Thanks,

	tglx