Re: Higher than expected disk write(2) latency

[David Dillow <dave@thedillows.org>]

On Wed, 2008-07-02 at 20:20 +0200, Martin Sustrik wrote:
> We are using librt, queueing 10240 writes, 512 bytes each.
> 
> We'll retry with libaio. Thanks for advice.

I've gotten good results going direct to the Linux AIO calls, but before
you go too far from your existing code base, are you using a single file
descriptor with the librt aio_*() routines? If so, that may be part of
your problem, as the library will serialize requests to the same fd. I
just pre-dup()'d the descriptor for each of the (max 32) requests I'd
have in flight for a toy stress-tester I wrote. With 10240 requests,
that may be a bit excessive, but it'd point out if that was part of the
problem fairly quickly.