Re: [RFC] Block IO Controller V2 - some results

[Corrado Zoccolo <czoccolo@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 1031 bytes --]

Hi Vivek,
On Fri, Nov 20, 2009 at 8:50 PM, Corrado Zoccolo <czoccolo@gmail.com> wrote:
> On Fri, Nov 20, 2009 at 7:42 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
>>
>> Should we also take into account the "cfqq->dispatched" request in
>> determining whether we should enable idling on deep queue random seeky
>> readers?
>
> Probably, but I think the most important thing is to do some averaging
> or hysteresis, so the completion of a single request doesn't cause the
> switch of a workload.

can you test the new version of the idling patch?
I register the fact that the queue had a large depth in a flag, that
is reset only when the idle times out (so at the end of the burst).
Idling is enabled if that flag is set (and think time is acceptable).
This should fix the switching behaviour you observed.

I decided to not count cfqq->dispatched to determine the depth.
In this way, when all queues in the system are random the idling is
enabled only if the requests queue builds up faster than it can be
consumed.

Thanks,
Corrado

[-- Attachment #2: 0002-cfq-iosched-idling-on-deep-seeky-sync-queues.patch --]
[-- Type: application/octet-stream, Size: 2440 bytes --]

From 00fd221ab16a4eb597dff5b23f063e543f5d99b8 Mon Sep 17 00:00:00 2001
From: Corrado Zoccolo <corrado@et2.(none)>
Date: Fri, 20 Nov 2009 14:55:54 +0100
Subject: [PATCH 2/2] cfq-iosched: idling on deep seeky sync queues

Seeky sync queues with large depth can gain unfairly big share of disk
time, at the expense of other seeky queues. This patch ensures that
idling will be enabled for queues with I/O depth at least 4, and small
think time. The decision to enable idling is sticky, until an idle
window times out without seeing new request.

The reasoning behind the decision is that, if an application is using
large I/O depth, it is already optimized to make full utilization of
the hardware, and therefore we reserve a slice of exclusive use for it.

Signed-off-by: Corrado Zoccolo <czoccolo@gmail.com>
---
 block/cfq-iosched.c |   13 ++++++++++++-
 1 files changed, 12 insertions(+), 1 deletions(-)

diff --git a/block/cfq-iosched.c b/block/cfq-iosched.c
index 2a304f4..373e80f 100644
--- a/block/cfq-iosched.c
+++ b/block/cfq-iosched.c
@@ -260,6 +260,7 @@ enum cfqq_state_flags {
 	CFQ_CFQQ_FLAG_slice_new,	/* no requests dispatched in slice */
 	CFQ_CFQQ_FLAG_sync,		/* synchronous queue */
 	CFQ_CFQQ_FLAG_coop,		/* cfqq is shared */
+	CFQ_CFQQ_FLAG_deep,		/* sync cfqq experienced large depth */
 };
 
 #define CFQ_CFQQ_FNS(name)						\
@@ -286,6 +287,7 @@ CFQ_CFQQ_FNS(prio_changed);
 CFQ_CFQQ_FNS(slice_new);
 CFQ_CFQQ_FNS(sync);
 CFQ_CFQQ_FNS(coop);
+CFQ_CFQQ_FNS(deep);
 #undef CFQ_CFQQ_FNS
 
 #define cfq_log_cfqq(cfqd, cfqq, fmt, args...)	\
@@ -2359,8 +2361,12 @@ cfq_update_idle_window(struct cfq_data *cfqd, struct cfq_queue *cfqq,
 
 	enable_idle = old_idle = cfq_cfqq_idle_window(cfqq);
 
+	if (cfqq->queued[0] + cfqq->queued[1] >= 4)
+		cfq_mark_cfqq_deep(cfqq);
+
 	if (!atomic_read(&cic->ioc->nr_tasks) || !cfqd->cfq_slice_idle ||
-	    (sample_valid(cfqq->seek_samples) && CFQQ_SEEKY(cfqq)))
+	    (!cfq_cfqq_deep(cfqq) && sample_valid(cfqq->seek_samples)
+	     && CFQQ_SEEKY(cfqq)))
 		enable_idle = 0;
 	else if (sample_valid(cic->ttime_samples)) {
 		if (cic->ttime_mean > cfqd->cfq_slice_idle)
@@ -2858,6 +2864,11 @@ static void cfq_idle_slice_timer(unsigned long data)
 		 */
 		if (!RB_EMPTY_ROOT(&cfqq->sort_list))
 			goto out_kick;
+
+		/*
+		 * Queue depth flag is reset only when the idle didn't succeed
+		 */
+		cfq_clear_cfqq_deep(cfqq);
 	}
 expire:
 	cfq_slice_expired(cfqd, timed_out);
-- 
1.6.2.5