Re: fio mmap randread 64k more than 40% regression with 2.6.33-rc1

[Corrado Zoccolo <czoccolo@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 9516 bytes --]

Hi Yanmin,
On Fri, Jan 1, 2010 at 11:12 AM, Zhang, Yanmin
<yanmin_zhang@linux.intel.com> wrote:
> On Thu, 2009-12-31 at 11:34 +0100, Corrado Zoccolo wrote:
>> Hi Yanmin,
>> On Thu, Dec 31, 2009 at 10:16 AM, Zhang, Yanmin
>> <yanmin_zhang@linux.intel.com> wrote:
>> > Comparing with kernel 2.6.32, fio mmap randread 64k has more than 40% regression with
>> > 2.6.33-rc1.
>>
> Thanks for your timely reply. Some comments inlined below.
>
>> Can you compare the performance also with 2.6.31?
> We did. We run Linux kernel Performance Tracking project and run many benchmarks when a RC kernel
> is released.
>
> The result of 2.6.31 is quite similar to the one of 2.6.32. But the one of 2.6.30 is about
> 8% better than the one of 2.6.31.
>
>> I think I understand what causes your problem.
>> 2.6.32, with default settings, handled even random readers as
>> sequential ones to provide fairness. This has benefits on single disks
>> and JBODs, but causes harm on raids.
> I didn't test RAID as that machine with hardware RAID HBA is crashed now. But if we turn on
> hardware RAID in HBA, mostly we use noop io scheduler.
I think you should start testing cfq with them, too. From 2.6.33, we
have some big improvements in this area.
>
>> For 2.6.33, we changed the way in which this is handled, restoring the
>> enable_idle = 0 for seeky queues as it was in 2.6.31:
>> @@ -2218,13 +2352,10 @@ cfq_update_idle_window(struct cfq_data *cfqd,
>> struct cfq_queue *cfqq,
>>        enable_idle = old_idle = cfq_cfqq_idle_window(cfqq);
>>
>>        if (!atomic_read(&cic->ioc->nr_tasks) || !cfqd->cfq_slice_idle ||
>> -           (!cfqd->cfq_latency && cfqd->hw_tag && CFQQ_SEEKY(cfqq)))
>> +           (sample_valid(cfqq->seek_samples) && CFQQ_SEEKY(cfqq)))
>>                enable_idle = 0;
>> (compare with 2.6.31:
>>         if (!atomic_read(&cic->ioc->nr_tasks) || !cfqd->cfq_slice_idle ||
>>             (cfqd->hw_tag && CIC_SEEKY(cic)))
>>                 enable_idle = 0;
>> excluding the sample_valid check, it should be equivalent for you (I
>> assume you have NCQ disks))
>> and we provide fairness for them by servicing all seeky queues
>> together, and then idling before switching to other ones.
> As for function cfq_update_idle_window, you is right. But since
> 2.6.32, CFQ merges many patches and the patches have impact on each other.
>
>>
>> The mmap 64k randreader will have a large seek_mean, resulting in
>> being marked seeky, but will send 16 * 4k sequential requests one
>> after the other, so alternating between those seeky queues will cause
>> harm.
>>
>> I'm working on a new way to compute seekiness of queues, that should
>> fix your issue, correctly identifying those queues as non-seeky (for
>> me, a queue should be considered seeky only if it submits more than 1
>> seeky requests for 8 sequential ones).
>>
>> >
>> > The test scenario: 1 JBOD has 12 disks and every disk has 2 partitions. Create
>> > 8 1-GB files per partition and start 8 processes to do rand read on the 8 files
>> > per partitions. There are 8*24 processes totally. randread block size is 64K.
>> >
>> > We found the regression on 2 machines. One machine has 8GB memory and the other has
>> > 6GB.
>> >
>> > Bisect is very unstable. The related patches are many instead of just one.
>> >
>> >
>> > 1) commit 8e550632cccae34e265cb066691945515eaa7fb5
>> > Author: Corrado Zoccolo <czoccolo@gmail.com>
>> > Date:   Thu Nov 26 10:02:58 2009 +0100
>> >
>> >    cfq-iosched: fix corner cases in idling logic
>> >
>> >
>> > This patch introduces about less than 20% regression. I just reverted below section
>> > and this part regression disappear. It shows this regression is stable and not impacted
>> > by other patches.
>> >
>> > @@ -1253,9 +1254,9 @@ static void cfq_arm_slice_timer(struct cfq_data *cfqd)
>> >                return;
>> >
>> >        /*
>> > -        * still requests with the driver, don't idle
>> > +        * still active requests from this queue, don't idle
>> >         */
>> > -       if (rq_in_driver(cfqd))
>> > +       if (cfqq->dispatched)
>> >                return;
> Although 5 patches are related to the regression, above line is quite
> independent. Reverting above line could always improve the result for about
> 20%.
I've looked at your fio script, and it is quite complex, with lot of
things going on.
Let's keep this for last.
I've created a smaller test, that already shows some regression:
[global]
direct=0
ioengine=mmap
size=8G
bs=64k
numjobs=1
loops=5
runtime=60
#group_reporting
invalidate=0
directory=/media/hd/cfq-tests

[job0]
startdelay=0
rw=randread
filename=testfile1

[job1]
startdelay=0
rw=randread
filename=testfile2

[job2]
startdelay=0
rw=randread
filename=testfile3

[job3]
startdelay=0
rw=randread
filename=testfile4

The attached patches, in particular 0005 (that apply on top of
for-linus branch of Jen's tree
git://git.kernel.dk/linux-2.6-block.git) fix the regression on this
simplified workload.

>
>> >
>> This shouldn't affect you if all queues are marked as idle.
> Do you mean to use command ionice to mark it as idle class? I didn't try it.
No. I meant forcing enable_idle = 1, as you were almost doing with
your patch, when cfq_latency was set.
With my above patch, this should not be needed any more, since the
queues should be seen as sequential.

>
>>  Does just
>> your patch:
>> > -           (!cfq_cfqq_deep(cfqq) && sample_valid(cfqq->seek_samples)
>> > -            && CFQQ_SEEKY(cfqq)))
>> > +           (!cfqd->cfq_latency && !cfq_cfqq_deep(cfqq) &&
>> > +               sample_valid(cfqq->seek_samples) && CFQQ_SEEKY(cfqq)))
>> fix most of the regression without touching arm_slice_timer?
> No. If to fix the regression completely, I need apply above patch plus
> a debug patch. The debug patch is to just work around the 3 patches report by
> Shaohua's tiobench regression report. Without the debug patch, the regression
> isn't resolved.

Jens already merged one of Shaohua's patches, that may fix the problem
with queue combining.

> Below is the debug patch.
> diff -Nraup linux-2.6.33_rc1/block/cfq-iosched.c linux-2.6.33_rc1_randread64k/block/cfq-iosched.c
> --- linux-2.6.33_rc1/block/cfq-iosched.c        2009-12-23 14:12:03.000000000 +0800
> +++ linux-2.6.33_rc1_randread64k/block/cfq-iosched.c    2009-12-30 17:12:28.000000000 +0800
> @@ -592,6 +592,9 @@ cfq_set_prio_slice(struct cfq_data *cfqd
>        cfqq->slice_start = jiffies;
>        cfqq->slice_end = jiffies + slice;
>        cfqq->allocated_slice = slice;
> +/*YMZHANG*/
> +       cfqq->slice_end = cfq_prio_to_slice(cfqd, cfqq) + jiffies;
> +
This is disabled, on a vanilla 2.6.33 kernel, by setting low_latency = 0
>        cfq_log_cfqq(cfqd, cfqq, "set_slice=%lu", cfqq->slice_end - jiffies);
>  }
>
> @@ -1836,7 +1839,8 @@ static void cfq_arm_slice_timer(struct c
>        /*
>         * still active requests from this queue, don't idle
>         */
> -       if (cfqq->dispatched)
> +       //if (cfqq->dispatched)
> +       if (rq_in_driver(cfqd))
>                return;
>
>        /*
> @@ -1941,6 +1945,9 @@ static void cfq_setup_merge(struct cfq_q
>                new_cfqq = __cfqq;
>        }
>
> +       /* YMZHANG debug */
> +       return;
> +
This should be partially addressed by Shaohua's patch merged in Jens' tree.
But note that your 8 processes, can randomly start doing I/O on the
same file, so merging those queues is sometimes reasonable.
The patch to split them quickly was still not merged, though, so you
will still see some regression due to this. In my simplified job file,
I removed the randomness to make sure this cannot happen.

>        process_refs = cfqq_process_refs(cfqq);
>        /*
>         * If the process for the cfqq has gone away, there is no
>
>
>>
>> I guess
>> > 5db5d64277bf390056b1a87d0bb288c8b8553f96.
>> will still introduce a 10% regression, but this is needed to improve
>> latency, and you can just disable low_latency to avoid it.
> You are right. I did a quick testing. If my patch + revert 2 patches and keep
> 5db5d64, the regression is about 20%.
>
> But low_latency=0 doesn't work like what we imagined. If patch + revert 2 patches
> and keep 5db5d64 while set low_latency=0, the regression is still there. One
> reason is my patch doesn't work when low_latency=0.
Right. You can try with my patch, instead, that doesn't depend on
low_latency, and set it to 0 to remove this performance degradation.
My results:
2.6.32.2:
   READ: io=146688KB, aggrb=2442KB/s, minb=602KB/s, maxb=639KB/s,
mint=60019msec, maxt=60067msec

2.6.33 - jens:
   READ: io=128512KB, aggrb=2140KB/s, minb=526KB/s, maxb=569KB/s,
mint=60004msec, maxt=60032msec

2.6.33 - jens + my patches :
   READ: io=143232KB, aggrb=2384KB/s, minb=595KB/s, maxb=624KB/s,
mint=60003msec, maxt=60072msec

2.6.33 - jens + my patches + low_lat = 0:
   READ: io=145216KB, aggrb=2416KB/s, minb=596KB/s, maxb=632KB/s,
mint=60027msec, maxt=60087msec


>>
>> Thanks,
>> Corrado
> I attach the fio job file for your reference.
>
> I got a cold and will continue to work on it next week.
>
> Yanmin
>

Thanks,
Corrado

[-- Attachment #2: 0003-cfq-iosched-non-rot-devices-do-not-need-read-queue-m.patch --]
[-- Type: application/octet-stream, Size: 2962 bytes --]

From f7bf4db76818d6a0ce83c179cb6ae2a305af3082 Mon Sep 17 00:00:00 2001
From: Corrado Zoccolo <czoccolo@gmail.com>
Date: Wed, 30 Dec 2009 11:58:34 +0100
Subject: [PATCH 3/5] cfq-iosched: non-rot devices do not need read queue merging

Non rotational devices' performances are not affected by
distance of read requests, so there is no point in having
overhead to merge such queues.
This doesn't apply to writes, so this patch changes the
queued[] field, to be indexed by READ/WRITE instead of
SYNC/ASYNC, and only compute proximity for queues with
WRITE requests.

Signed-off-by: Corrado Zoccolo <czoccolo@gmail.com>
---
 block/cfq-iosched.c |   20 +++++++++++---------
 1 files changed, 11 insertions(+), 9 deletions(-)

diff --git a/block/cfq-iosched.c b/block/cfq-iosched.c
index 918c7fd..7da9391 100644
--- a/block/cfq-iosched.c
+++ b/block/cfq-iosched.c
@@ -108,9 +108,9 @@ struct cfq_queue {
 	struct rb_root sort_list;
 	/* if fifo isn't expired, next request to serve */
 	struct request *next_rq;
-	/* requests queued in sort_list */
+	/* requests queued in sort_list, indexed by READ/WRITE */
 	int queued[2];
-	/* currently allocated requests */
+	/* currently allocated requests, indexed by READ/WRITE */
 	int allocated[2];
 	/* fifo list of requests in sort_list */
 	struct list_head fifo;
@@ -1268,7 +1268,8 @@ static void cfq_prio_tree_add(struct cfq_data *cfqd, struct cfq_queue *cfqq)
 		return;
 	if (!cfqq->next_rq)
 		return;
-
+	if (blk_queue_nonrot(cfqd->queue) && !cfqq->queued[WRITE])
+		return;
 	cfqq->p_root = &cfqd->prio_trees[cfqq->org_ioprio];
 	__cfqq = cfq_prio_tree_lookup(cfqd, cfqq->p_root,
 				      blk_rq_pos(cfqq->next_rq), &parent, &p);
@@ -1337,10 +1338,10 @@ static void cfq_del_cfqq_rr(struct cfq_data *cfqd, struct cfq_queue *cfqq)
 static void cfq_del_rq_rb(struct request *rq)
 {
 	struct cfq_queue *cfqq = RQ_CFQQ(rq);
-	const int sync = rq_is_sync(rq);
+	const int rw = rq_data_dir(rq);
 
-	BUG_ON(!cfqq->queued[sync]);
-	cfqq->queued[sync]--;
+	BUG_ON(!cfqq->queued[rw]);
+	cfqq->queued[rw]--;
 
 	elv_rb_del(&cfqq->sort_list, rq);
 
@@ -1363,7 +1364,7 @@ static void cfq_add_rq_rb(struct request *rq)
 	struct cfq_data *cfqd = cfqq->cfqd;
 	struct request *__alias, *prev;
 
-	cfqq->queued[rq_is_sync(rq)]++;
+	cfqq->queued[rq_data_dir(rq)]++;
 
 	/*
 	 * looks a little odd, but the first insert might return an alias.
@@ -1393,7 +1394,7 @@ static void cfq_add_rq_rb(struct request *rq)
 static void cfq_reposition_rq_rb(struct cfq_queue *cfqq, struct request *rq)
 {
 	elv_rb_del(&cfqq->sort_list, rq);
-	cfqq->queued[rq_is_sync(rq)]--;
+	cfqq->queued[rq_data_dir(rq)]--;
 	cfq_add_rq_rb(rq);
 }
 
@@ -1689,7 +1690,8 @@ static struct cfq_queue *cfqq_close(struct cfq_data *cfqd,
 	struct cfq_queue *__cfqq;
 	sector_t sector = cfqd->last_position;
 
-	if (RB_EMPTY_ROOT(root))
+	if (RB_EMPTY_ROOT(root) ||
+	    (blk_queue_nonrot(cfqd->queue) && !cur_cfqq->queued[WRITE]))
 		return NULL;
 
 	/*
-- 
1.6.4.4


[-- Attachment #3: 0004-cfq-iosched-requests-in-flight-vs-in-driver-clarific.patch --]
[-- Type: application/octet-stream, Size: 4945 bytes --]

From bd47454a4381f584e79b9bb57eb0329e4b385ee5 Mon Sep 17 00:00:00 2001
From: Corrado Zoccolo <czoccolo@gmail.com>
Date: Wed, 30 Dec 2009 22:49:42 +0100
Subject: [PATCH 4/5] cfq-iosched: requests "in flight" vs "in driver" clarification

Counters for requests "in flight" and "in driver" are used asymmetrically
in cfq_may_dispatch, and have slightly different meaning.
We split the rq_in_flight counter (was sync_flight) to count both sync
and async requests, in order to use this one, which is more accurate in
some corner cases.
The rq_in_driver counter is coalesced, since individual sync/async counts
are not used any more.

Signed-off-by: Corrado Zoccolo <czoccolo@gmail.com>
---
 block/cfq-iosched.c |   44 ++++++++++++++++++--------------------------
 1 files changed, 18 insertions(+), 26 deletions(-)

diff --git a/block/cfq-iosched.c b/block/cfq-iosched.c
index 7da9391..c6d5678 100644
--- a/block/cfq-iosched.c
+++ b/block/cfq-iosched.c
@@ -227,8 +227,8 @@ struct cfq_data {
 
 	unsigned int busy_queues;
 
-	int rq_in_driver[2];
-	int sync_flight;
+	int rq_in_driver;
+	int rq_in_flight[2];
 
 	/*
 	 * queue-depth detection
@@ -419,11 +419,6 @@ static struct cfq_queue *cfq_get_queue(struct cfq_data *, bool,
 static struct cfq_io_context *cfq_cic_lookup(struct cfq_data *,
 						struct io_context *);
 
-static inline int rq_in_driver(struct cfq_data *cfqd)
-{
-	return cfqd->rq_in_driver[0] + cfqd->rq_in_driver[1];
-}
-
 static inline struct cfq_queue *cic_to_cfqq(struct cfq_io_context *cic,
 					    bool is_sync)
 {
@@ -1423,9 +1418,9 @@ static void cfq_activate_request(struct request_queue *q, struct request *rq)
 {
 	struct cfq_data *cfqd = q->elevator->elevator_data;
 
-	cfqd->rq_in_driver[rq_is_sync(rq)]++;
+	cfqd->rq_in_driver++;
 	cfq_log_cfqq(cfqd, RQ_CFQQ(rq), "activate rq, drv=%d",
-						rq_in_driver(cfqd));
+						cfqd->rq_in_driver);
 
 	cfqd->last_position = blk_rq_pos(rq) + blk_rq_sectors(rq);
 }
@@ -1433,12 +1428,11 @@ static void cfq_activate_request(struct request_queue *q, struct request *rq)
 static void cfq_deactivate_request(struct request_queue *q, struct request *rq)
 {
 	struct cfq_data *cfqd = q->elevator->elevator_data;
-	const int sync = rq_is_sync(rq);
 
-	WARN_ON(!cfqd->rq_in_driver[sync]);
-	cfqd->rq_in_driver[sync]--;
+	WARN_ON(!cfqd->rq_in_driver);
+	cfqd->rq_in_driver--;
 	cfq_log_cfqq(cfqd, RQ_CFQQ(rq), "deactivate rq, drv=%d",
-						rq_in_driver(cfqd));
+						cfqd->rq_in_driver);
 }
 
 static void cfq_remove_request(struct request *rq)
@@ -1876,8 +1870,7 @@ static void cfq_dispatch_insert(struct request_queue *q, struct request *rq)
 	cfqq->dispatched++;
 	elv_dispatch_sort(q, rq);
 
-	if (cfq_cfqq_sync(cfqq))
-		cfqd->sync_flight++;
+	cfqd->rq_in_flight[cfq_cfqq_sync(cfqq)]++;
 	cfqq->nr_sectors += blk_rq_sectors(rq);
 }
 
@@ -2224,13 +2217,13 @@ static bool cfq_may_dispatch(struct cfq_data *cfqd, struct cfq_queue *cfqq)
 	/*
 	 * Drain async requests before we start sync IO
 	 */
-	if (cfq_should_idle(cfqd, cfqq) && cfqd->rq_in_driver[BLK_RW_ASYNC])
+	if (cfq_should_idle(cfqd, cfqq) && cfqd->rq_in_flight[BLK_RW_ASYNC])
 		return false;
 
 	/*
 	 * If this is an async queue and we have sync IO in flight, let it wait
 	 */
-	if (cfqd->sync_flight && !cfq_cfqq_sync(cfqq))
+	if (cfqd->rq_in_flight[BLK_RW_SYNC] && !cfq_cfqq_sync(cfqq))
 		return false;
 
 	max_dispatch = cfqd->cfq_quantum;
@@ -3220,14 +3213,14 @@ static void cfq_update_hw_tag(struct cfq_data *cfqd)
 {
 	struct cfq_queue *cfqq = cfqd->active_queue;
 
-	if (rq_in_driver(cfqd) > cfqd->hw_tag_est_depth)
-		cfqd->hw_tag_est_depth = rq_in_driver(cfqd);
+	if (cfqd->rq_in_driver > cfqd->hw_tag_est_depth)
+		cfqd->hw_tag_est_depth = cfqd->rq_in_driver;
 
 	if (cfqd->hw_tag == 1)
 		return;
 
 	if (cfqd->rq_queued <= CFQ_HW_QUEUE_MIN &&
-	    rq_in_driver(cfqd) <= CFQ_HW_QUEUE_MIN)
+	    cfqd->rq_in_driver <= CFQ_HW_QUEUE_MIN)
 		return;
 
 	/*
@@ -3237,7 +3230,7 @@ static void cfq_update_hw_tag(struct cfq_data *cfqd)
 	 */
 	if (cfqq && cfq_cfqq_idle_window(cfqq) &&
 	    cfqq->dispatched + cfqq->queued[0] + cfqq->queued[1] <
-	    CFQ_HW_QUEUE_MIN && rq_in_driver(cfqd) < CFQ_HW_QUEUE_MIN)
+	    CFQ_HW_QUEUE_MIN && cfqd->rq_in_driver < CFQ_HW_QUEUE_MIN)
 		return;
 
 	if (cfqd->hw_tag_samples++ < 50)
@@ -3290,13 +3283,12 @@ static void cfq_completed_request(struct request_queue *q, struct request *rq)
 
 	cfq_update_hw_tag(cfqd);
 
-	WARN_ON(!cfqd->rq_in_driver[sync]);
+	WARN_ON(!cfqd->rq_in_driver);
 	WARN_ON(!cfqq->dispatched);
-	cfqd->rq_in_driver[sync]--;
+	cfqd->rq_in_driver--;
 	cfqq->dispatched--;
 
-	if (cfq_cfqq_sync(cfqq))
-		cfqd->sync_flight--;
+	cfqd->rq_in_flight[cfq_cfqq_sync(cfqq)]--;
 
 	if (sync) {
 		RQ_CIC(rq)->last_end_request = now;
@@ -3350,7 +3342,7 @@ static void cfq_completed_request(struct request_queue *q, struct request *rq)
 		}
 	}
 
-	if (!rq_in_driver(cfqd))
+	if (!cfqd->rq_in_driver)
 		cfq_schedule_dispatch(cfqd);
 }
 
-- 
1.6.4.4


[-- Attachment #4: 0005-cfq-iosched-rework-seeky-detection.patch --]
[-- Type: application/octet-stream, Size: 3405 bytes --]

From c6eb136205c0b6ebe2e9732de249ddefba26d41d Mon Sep 17 00:00:00 2001
From: Corrado Zoccolo <czoccolo@gmail.com>
Date: Thu, 31 Dec 2009 13:41:59 +0100
Subject: [PATCH 5/5] cfq-iosched: rework seeky detection

---
 block/cfq-iosched.c |   54 +++++++++++++-------------------------------------
 1 files changed, 14 insertions(+), 40 deletions(-)

diff --git a/block/cfq-iosched.c b/block/cfq-iosched.c
index c6d5678..4e203c4 100644
--- a/block/cfq-iosched.c
+++ b/block/cfq-iosched.c
@@ -133,9 +133,7 @@ struct cfq_queue {
 	unsigned short ioprio, org_ioprio;
 	unsigned short ioprio_class, org_ioprio_class;
 
-	unsigned int seek_samples;
-	u64 seek_total;
-	sector_t seek_mean;
+	u32 seek_history;
 	sector_t last_request_pos;
 	unsigned long seeky_start;
 
@@ -1658,22 +1656,13 @@ static inline sector_t cfq_dist_from_last(struct cfq_data *cfqd,
 		return cfqd->last_position - blk_rq_pos(rq);
 }
 
-#define CFQQ_SEEK_THR		8 * 1024
-#define CFQQ_SEEKY(cfqq)	((cfqq)->seek_mean > CFQQ_SEEK_THR)
+#define CFQQ_SEEK_THR		(sector_t)(8 * 100)
+#define CFQQ_SEEKY(cfqq)	(hweight32(cfqq->seek_history) > 32/8)
 
 static inline int cfq_rq_close(struct cfq_data *cfqd, struct cfq_queue *cfqq,
 			       struct request *rq, bool for_preempt)
 {
-	sector_t sdist = cfqq->seek_mean;
-
-	if (!sample_valid(cfqq->seek_samples))
-		sdist = CFQQ_SEEK_THR;
-
-	/* if seek_mean is big, using it as close criteria is meaningless */
-	if (sdist > CFQQ_SEEK_THR && !for_preempt)
-		sdist = CFQQ_SEEK_THR;
-
-	return cfq_dist_from_last(cfqd, rq) <= sdist;
+	return cfq_dist_from_last(cfqd, rq) <= CFQQ_SEEK_THR;
 }
 
 static struct cfq_queue *cfqq_close(struct cfq_data *cfqd,
@@ -2971,30 +2960,16 @@ static void
 cfq_update_io_seektime(struct cfq_data *cfqd, struct cfq_queue *cfqq,
 		       struct request *rq)
 {
-	sector_t sdist;
-	u64 total;
-
-	if (!cfqq->last_request_pos)
-		sdist = 0;
-	else if (cfqq->last_request_pos < blk_rq_pos(rq))
-		sdist = blk_rq_pos(rq) - cfqq->last_request_pos;
-	else
-		sdist = cfqq->last_request_pos - blk_rq_pos(rq);
-
-	/*
-	 * Don't allow the seek distance to get too large from the
-	 * odd fragment, pagein, etc
-	 */
-	if (cfqq->seek_samples <= 60) /* second&third seek */
-		sdist = min(sdist, (cfqq->seek_mean * 4) + 2*1024*1024);
-	else
-		sdist = min(sdist, (cfqq->seek_mean * 4) + 2*1024*64);
+	sector_t sdist = 0;
+	if (cfqq->last_request_pos) {
+		if (cfqq->last_request_pos < blk_rq_pos(rq))
+			sdist = blk_rq_pos(rq) - cfqq->last_request_pos;
+		else
+			sdist = cfqq->last_request_pos - blk_rq_pos(rq);
+	}
 
-	cfqq->seek_samples = (7*cfqq->seek_samples + 256) / 8;
-	cfqq->seek_total = (7*cfqq->seek_total + (u64)256*sdist) / 8;
-	total = cfqq->seek_total + (cfqq->seek_samples/2);
-	do_div(total, cfqq->seek_samples);
-	cfqq->seek_mean = (sector_t)total;
+	cfqq->seek_history <<= 1;
+	cfqq->seek_history |= (sdist > CFQQ_SEEK_THR);
 
 	/*
 	 * If this cfqq is shared between multiple processes, check to
@@ -3032,8 +3007,7 @@ cfq_update_idle_window(struct cfq_data *cfqd, struct cfq_queue *cfqq,
 		cfq_mark_cfqq_deep(cfqq);
 
 	if (!atomic_read(&cic->ioc->nr_tasks) || !cfqd->cfq_slice_idle ||
-	    (!cfq_cfqq_deep(cfqq) && sample_valid(cfqq->seek_samples)
-	     && CFQQ_SEEKY(cfqq)))
+	    (!cfq_cfqq_deep(cfqq) && CFQQ_SEEKY(cfqq)))
 		enable_idle = 0;
 	else if (sample_valid(cic->ttime_samples)) {
 		if (cic->ttime_mean > cfqd->cfq_slice_idle)
-- 
1.6.4.4