Re: [RFC] Block IO Controller V2 - some results

[Corrado Zoccolo <czoccolo@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 1578 bytes --]

Hi Vivek,
On Fri, Nov 20, 2009 at 3:18 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
> Hi Corrado,
>
> I liked the idea of putting all the sync-noidle queues together in root
> group to achieve better throughput and implemeted a small patch.
>
> It works fine for random readers. But when I do multiple direct random writers
> in one group vs a random reader in other group, I am getting strange
> behavior. Random reader moves to root group as sync-noidle workload. But
> random writers are largely sync queues in remain in other group. But many
> a times also jump into root group and preempt random reader.

can you try the attached patches?
They fix the problems you identified about no-idle preemption, and
deep seeky queues.
With those, you should not see this jumping any more.
I'll send them to Jens as soon has he comes back from vacation.

Corrado

> Anyway, with 4 random writers and 1 random reader running for 30 seconds
> in root group I get following.
>
> rw: 59,963KB/s
> rr: 66KB/s
>
> But if these are put in seprate groups test1 and test2 then
>
> rw: 30,587KB/s
> rr: 23KB/s
>
> I can understand the drop in rw throughput as it has been put under a
> group of weight 500. But rr will run in root group with weight 1000 and
> should have received much higher BW, instead it ends up loosing.
>
> Staring hard at blktrace output to figure out what's happening. One thing
> noticeable so far is that without cgroup stuff we seem to be interleaving
> dispatch from random reader and random writer much better as compared to
> with cgroup stuff.
>
> Thanks
> Vivek
>

[-- Attachment #2: 0001-cfq-iosched-fix-no-idle-preemption-logic.patch --]
[-- Type: application/octet-stream, Size: 1241 bytes --]

From 3ce2b697b9dd1bf07ba3c161eeb4264cc1a21aa3 Mon Sep 17 00:00:00 2001
From: Corrado Zoccolo <corrado@et2.(none)>
Date: Fri, 20 Nov 2009 14:43:02 +0100
Subject: [PATCH 1/2] cfq-iosched: fix no-idle preemption logic

An incoming no-idle queue should preempt the active no-idle queue
only if the active queue is idling due to service tree empty.
This change should affect only non-NCQ rotational devices, since
for those ones we introduce additional, small idles, that shouldn't
be preempted.

Reported-by: Vivek Goyal <vgoyal@redhat.com>
Signed-off-by: Corrado Zoccolo <czoccolo@gmail.com>
---
 block/cfq-iosched.c |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/block/cfq-iosched.c b/block/cfq-iosched.c
index 6925ab9..a8f7a26 100644
--- a/block/cfq-iosched.c
+++ b/block/cfq-iosched.c
@@ -2401,8 +2401,9 @@ cfq_should_preempt(struct cfq_data *cfqd, struct cfq_queue *new_cfqq,
 	if (cfq_class_idle(cfqq))
 		return true;
 
-	if (cfqd->serving_type == SYNC_NOIDLE_WORKLOAD
-	    && new_cfqq->service_tree == cfqq->service_tree)
+	if (cfqd->serving_type == SYNC_NOIDLE_WORKLOAD &&
+	    cfqq_type(new_cfqq) == SYNC_NOIDLE_WORKLOAD &&
+	    new_cfqq->service_tree-> count == 1)
 		return true;
 
 	/*
-- 
1.6.2.5


[-- Attachment #3: 0002-cfq-iosched-idling-on-deep-seeky-sync-queues.patch --]
[-- Type: application/octet-stream, Size: 1520 bytes --]

From 643dfd19e1143521e24a76d8671c1c0b9e1f90a3 Mon Sep 17 00:00:00 2001
From: Corrado Zoccolo <corrado@et2.(none)>
Date: Fri, 20 Nov 2009 14:55:54 +0100
Subject: [PATCH 2/2] cfq-iosched: idling on deep seeky sync queues

Seeky sync queues with large depth can gain unfairly big share of disk
time, at the expense of other seeky queues. This patch ensures that
idling will be enabled for queues with I/O depth at least 4.

The reasoning behind the decision is that, if an application is using
large I/O depth, it is already optimized to make full utilization of
the hardware, and therefore we reserve a slice of exclusive use for it.

Signed-off-by: Corrado Zoccolo <czoccolo@gmail.com>
---
 block/cfq-iosched.c |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/block/cfq-iosched.c b/block/cfq-iosched.c
index a8f7a26..4f67bc2 100644
--- a/block/cfq-iosched.c
+++ b/block/cfq-iosched.c
@@ -2359,8 +2359,10 @@ cfq_update_idle_window(struct cfq_data *cfqd, struct cfq_queue *cfqq,
 
 	enable_idle = old_idle = cfq_cfqq_idle_window(cfqq);
 
-	if (!atomic_read(&cic->ioc->nr_tasks) || !cfqd->cfq_slice_idle ||
-	    (sample_valid(cfqq->seek_samples) && CFQQ_SEEKY(cfqq)))
+	if (cfqq->queued[0] + cfqq->queued[1] >= 4)
+		enable_idle = 1;
+	else if (!atomic_read(&cic->ioc->nr_tasks) || !cfqd->cfq_slice_idle ||
+		 (sample_valid(cfqq->seek_samples) && CFQQ_SEEKY(cfqq)))
 		enable_idle = 0;
 	else if (sample_valid(cic->ttime_samples)) {
 		if (cic->ttime_mean > cfqd->cfq_slice_idle)
-- 
1.6.2.5