Re: [RFC] blk-mq and I/O scheduling

[Andreas Herrmann <aherrmann@suse.com>]

On Wed, Nov 25, 2015 at 12:47:21PM -0700, Jens Axboe wrote:
> On 11/19/2015 05:02 AM, Andreas Herrmann wrote:

 --8<--

> >The latter helped to improve performance for sequential reads and
> >writes. But it's not on a par with CFQ. Increasing the time slice is
> >suboptimal (as shown with the 2ms results, see below). It might be
> >possible to get better performance when further reducing the initial
> >time slice and adapting it up to a maximum value if there are
> >repeatedly pending requests for a CPU.
> >
> >After these observations and assuming that non-rotational devices are
> >most likely fine using blk-mq without I/O scheduling support I wonder
> >whether
> >
> >- it's really a good idea to re-implement scheduling support for
> >   blk-mq that eventually behaves like CFQ for rotational devices.
> >
> >- it's technical possible to support both blk-mq and CFQ for different
> >   devices on the same host adapter. This would allow to use "good old"
> >   code for "good old" rotational devices. (But this might not be a
> >   choice if in the long run a goal is to get rid of non-blk-mq code --
> >   not sure what the plans are.)
> >
> >What do you think about this?
> 
> Sorry I did not get around to properly looking at this this week,
> I'll tend to it next week. I think the concept of tying the idling
> to a specific CPU is likely fine, though I wonder if there are cases
> where we preempt more heavily and subsequently miss breaking the
> idling properly. I don't think we want/need cfq for blk-mq, but
> basic idling could potentially be enough. That's still a far cry
> from a full cfq implementation. The long term plans are still to
> move away from the legacy IO path, though with things like
> scheduling, that's sure to take some time.

FYI, I'll plan to send an updated patch later today.  

I've slightly changed it to allow specification of a time slice in µs
(instead of ms) and to extend it for a software queue when requests
were actually put into the hardware queue for this specific software
queue. This improved performance a little bit.

> That is actually where the mq-deadline work comes in. The
> mq-deadline work is missing a test patch to limit tag allocations,
> and a bunch of other little things to truly make it functional.
> There might be some options for folding it all together, with
> idling, as that would still be important on rotating storage going
> forward.


Thanks for you comments,

Andreas