Re: [HACK] convert i_alloc_sem for direct_io.c craziness!

[Zach Brown <zach.brown@oracle.com>]

> Hopefully I will get some attention from those that are responsible for
> fs/direct_io.c

[ many days later, I find this amongst the lkml noise. ]

> This patch converts the i_alloc_sem into a compat_rw_semaphore for the -rt
> patch.  Seems that the code in fs/direct_io.c does some nasty logic with
> the i_alloc_sem.  For DIO_LOCKING, I'm assuming that the i_alloc_sem is
> used as a reference counter for pending requests. When the request is
> made, the down_read is performed. When the request is handled by the block
> softirq, then that softirq does an up on the request. So the owner is not
> the same between down and up. When all requests are handled, the semaphore
> counter should be zero. This keeps away any write access while requests
> are pending.

Yeah.  This is used for performing concurrent asynchronous O_DIRECT
operations.

> This patch converts (for now) the i_alloc_sem into a compat_rw_semaphore
> to give back the old features to the sem. This fixes deadlocks that we've
> been having WRT direct_io.

*nod*

> The real fix would be to get rid of the read semaphore trickery in
> direct_io.c.

Do you have any suggestions for locking constructs that RT would prefer?

The core problem is that async IO is in flight while no process holds
the usual locks in system calls.  We don't want the blocks referenced by
IOs to be freed and realocated some where else while the IO is in
flight.  Hence the i_alloc_sem acquiry in the file block modification
paths: vmtruncate - free, notify_change - a proxy for allocating writes.

The agents are:

- many tasks issuing concurrent async IO and exiting from system calls
while the IO is still in flight

- operations completed in interrupt handlers from storage devices

- tasks changing file block mapping via system calls

There's some long-term work to integrate the locking between the
buffered and O_DIRECT paths, but it's not close to ready.

- z