From: Tejun Heo <tj@kernel.org>
To: Paolo Valente <paolo.valente@unimore.it>
Cc: Jens Axboe <axboe@kernel.dk>, Li Zefan <lizefan@huawei.com>,
Fabio Checconi <fchecconi@gmail.com>,
Arianna Avanzini <avanzini.arianna@gmail.com>,
linux-kernel@vger.kernel.org,
containers@lists.linux-foundation.org, cgroups@vger.kernel.org
Subject: Re: [PATCH RFC - TAKE TWO - 11/12] block, bfq: boost the throughput on NCQ-capable flash-based devices
Date: Wed, 4 Jun 2014 09:56:27 -0400 [thread overview]
Message-ID: <20140604135627.GB5004@htj.dyndns.org> (raw)
In-Reply-To: <03CDD106-DB18-4E8F-B3D6-2AAD45782A06@unimore.it>
Hello, Paolo.
On Wed, Jun 04, 2014 at 09:29:20AM +0200, Paolo Valente wrote:
> > Shouldn't the comparison be against the benefit of "not idling
> > selectively" vs "always idling" when blkcg is in use?
> >
>
> Exactly. I’m sorry if I wrote things/sentences that did not let this
> point be clear. Maybe this lack of clarity is a further consequence
> of the annoying “not not” scheme adopted in the code and in the
> comments.
Ah, no, it was just me misreading the message.
> > I'm not really convinced about the approach. With rotating disks, we
> > know that allowing queue depth > 1 generaly lowers both throughput and
> > responsiveness and brings benefits in quite restricted cases. It
> > seems rather backwards to always allow QD > 1 and then try to optimize
> > in an attempt to recover what's lost. Wouldn't it make far more sense
> > to actively maintain QD == 1 by default and allow QD > 1 in specific
> > cases where it can be determined to be more beneficial than harmful?
>
> Although QD == 1 is not denoted explicitly as default, what you suggest is exactly what bfq does.
I see.
> >> I do not know how widespread a mechanism like ulatencyd is
> >> precisely, but in the symmetric scenario it creates, the throughput
> >> on, e.g., an HDD would drop by half if the workload is mostly random
> >> and we removed the more complex mechanism we set up. Wouldn't this
> >> be bad?
> >
> > It looks like a lot of complexity for optimization for a very
> > specific, likely unreliable (in terms of its triggering condition),
> > use case. The triggering condition is just too specific.
>
> Actually we have been asked several times to improve random-I/O
> performance on HDDs over the last years, by people recording, for
> the typical tasks performed by their machines, much lower throughput
> than with the other schedulers. Major problems have been reported
> for server workloads (database, web), and for btrfs. According to
> the feedback received after introducing this optimization in bfq,
> those problems seem to be finally gone.
I see. The equal priority part can probably work in enough cases to
be meaningful given that it just depends on the busy queues having the
same weight instead of everything in the system. It'd nice to note
that in the comment tho.
I'm still quite skeptical about the cgroup part tho. The triggering
condition is too specific and fragile. If I'm reading the bfq blkcg
implementation correctly, it seems to be applying the scheduling
algorithm recursively walking down the tree one level at a time. cfq
does it differently. cfq flattens the hierarchy by calculating the
nested weight of each active leaf queue and schedule all of them from
the same service tree. IOW, scheduling algorithm per-se doesn't care
about the hierarchy. All it sees are differing weights competing
equally regardless of the hierarchical structure.
If the same strategy can be applied to bfq, possibly the same strategy
of checking whether all the active queues have the same weight can be
used regardless of blkcg? That'd be simpler and a lot more robust.
Another thing I'm curious about is that the logic that you're using to
disable idling assumes that the disk will serve the queued commands
more or less in fair manner over time, right? If so, why does queues
having the same weight matter? Shouldn't the bandwidth scheduling
eventually make them converge to the specified weights over time?
Isn't wr_coeff > 1 test enough for maintaining reasonable
responsiveness?
> Besides, turning back to bfq, if its low-latency heuristics are
> disabled for non rotational devices, then, according to our results
> with slower devices, such as SD Cards and eMMCs, latency becomes
> easily unbearable, with no throughput gain.
Hmmm... looking at the nonrot optimizations again, so yeah assuming
the weight counting is necessary for NCQ hdds the part specific to
ssds isn't that big. You probably wanna sequence it the other way
around tho. This really should be primarily about disks at this
point.
The thing which still makes me cringe is how it scatters
blk_queue_nonrot() tests across multiple places without clear
explanation on what's going on. A bfqq being constantly seeky or not
doesn't have much to do with whether the device is rotational or not.
Its effect does and I don't think avoiding the overhead of keeping the
counters is meaningful. Things like this make the code a lot harder
to maintain in the long term as code is organized according to
seemingly arbitrary optimization rather than semantic structure. So,
let's please keep the accounting and optimization separate.
Thanks.
--
tejun
next prev parent reply other threads:[~2014-06-04 13:56 UTC|newest]
Thread overview: 107+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-05-27 12:42 [PATCH RFC RESEND 00/14] New version of the BFQ I/O Scheduler paolo
2014-05-27 12:42 ` [PATCH RFC RESEND 01/14] block: kconfig update and build bits for BFQ paolo
2014-05-28 22:19 ` Tejun Heo
2014-05-29 9:05 ` [PATCH RFC - TAKE TWO - 00/12] New version of the BFQ I/O Scheduler Paolo Valente
2014-05-29 9:05 ` [PATCH RFC - TAKE TWO - 01/12] block: introduce the BFQ-v0 I/O scheduler Paolo Valente
2014-05-30 15:36 ` Tejun Heo
2014-05-29 9:05 ` [PATCH RFC - TAKE TWO - 02/12] block, bfq: add full hierarchical scheduling and cgroups support Paolo Valente
2014-05-30 15:37 ` Tejun Heo
2014-05-30 15:39 ` Tejun Heo
2014-05-30 21:49 ` Paolo Valente
2014-05-30 21:49 ` Paolo Valente
2014-05-29 9:05 ` [PATCH RFC - TAKE TWO - 03/12] block, bfq: improve throughput boosting Paolo Valente
2014-05-29 9:05 ` [PATCH RFC - TAKE TWO - 04/12] block, bfq: modify the peak-rate estimator Paolo Valente
2014-05-29 9:05 ` [PATCH RFC - TAKE TWO - 05/12] block, bfq: add more fairness to boost throughput and reduce latency Paolo Valente
2014-05-29 9:05 ` [PATCH RFC - TAKE TWO - 06/12] block, bfq: improve responsiveness Paolo Valente
2014-05-30 15:41 ` Tejun Heo
2014-05-29 9:05 ` [PATCH RFC - TAKE TWO - 07/12] block, bfq: reduce I/O latency for soft real-time applications Paolo Valente
2014-05-29 9:05 ` [PATCH RFC - TAKE TWO - 08/12] block, bfq: preserve a low latency also with NCQ-capable drives Paolo Valente
2014-05-31 13:48 ` Tejun Heo
2014-06-02 9:58 ` Paolo Valente
2014-05-29 9:05 ` [PATCH RFC - TAKE TWO - 09/12] block, bfq: reduce latency during request-pool saturation Paolo Valente
2014-05-31 13:54 ` Tejun Heo
2014-06-02 9:54 ` Paolo Valente
2014-05-29 9:05 ` [PATCH RFC - TAKE TWO - 10/12] block, bfq: add Early Queue Merge (EQM) Paolo Valente
2014-06-01 0:03 ` Tejun Heo
2014-06-02 9:46 ` Paolo Valente
2014-06-03 16:28 ` Tejun Heo
2014-06-04 11:47 ` Paolo Valente
2014-06-04 13:04 ` Tejun Heo
2014-06-16 11:23 ` Paolo Valente
2014-05-29 9:05 ` [PATCH RFC - TAKE TWO - 11/12] block, bfq: boost the throughput on NCQ-capable flash-based devices Paolo Valente
2014-05-30 15:46 ` Tejun Heo
2014-05-30 22:01 ` Paolo Valente
2014-05-31 11:52 ` Tejun Heo
2014-06-02 9:26 ` Paolo Valente
2014-06-03 17:11 ` Tejun Heo
2014-06-04 7:29 ` Paolo Valente
2014-06-04 13:56 ` Tejun Heo [this message]
2014-06-16 10:46 ` Paolo Valente
2014-06-19 1:14 ` Tejun Heo
2014-05-29 9:05 ` [PATCH RFC - TAKE TWO - 12/12] block, bfq: boost the throughput with random I/O on NCQ-capable HDDs Paolo Valente
2014-05-30 15:51 ` Tejun Heo
2014-05-31 13:34 ` Tejun Heo
2014-05-30 16:07 ` [PATCH RFC - TAKE TWO - 00/12] New version of the BFQ I/O Scheduler Tejun Heo
2014-05-30 22:23 ` Paolo Valente
2014-05-30 23:28 ` Tejun Heo
2014-05-30 23:54 ` Paolo Valente
2014-06-02 11:14 ` Pavel Machek
2014-06-02 13:02 ` Pavel Machek
2014-06-03 16:54 ` Paolo Valente
2014-06-03 20:40 ` Pavel Machek
2014-06-04 8:39 ` Pavel Machek
2014-06-04 9:08 ` Pavel Machek
2014-06-04 10:03 ` BFQ speed tests [was Re: [PATCH RFC - TAKE TWO - 00/12] New version of the BFQ I/O Scheduler] Pavel Machek
2014-06-04 10:24 ` Paolo Valente
2014-06-04 11:59 ` Takashi Iwai
2014-06-04 12:12 ` Paolo Valente
2014-06-11 20:45 ` Paolo Valente
2014-06-13 16:21 ` Takashi Iwai
2014-06-11 20:39 ` Paolo Valente
2014-06-02 17:33 ` [PATCH RFC - TAKE TWO - 00/12] New version of the BFQ I/O Scheduler Tejun Heo
2014-06-03 4:12 ` Mike Galbraith
2014-06-04 22:31 ` Pavel Machek
2014-06-05 2:14 ` Jens Axboe
2014-05-31 0:48 ` Jens Axboe
2014-05-31 5:16 ` Tejun Heo
2014-06-02 14:29 ` Jens Axboe
2014-06-02 17:24 ` Tejun Heo
2014-06-02 17:32 ` Jens Axboe
2014-06-02 17:42 ` Tejun Heo
2014-06-02 17:46 ` Jens Axboe
2014-06-02 18:51 ` Tejun Heo
2014-06-02 20:57 ` Jens Axboe
2014-06-04 14:31 ` Christoph Hellwig
2014-06-04 14:50 ` Tejun Heo
2014-06-04 14:53 ` Christoph Hellwig
2014-06-04 14:58 ` Tejun Heo
2014-06-04 17:51 ` Christoph Hellwig
2014-06-17 15:55 ` Paolo Valente
2014-06-19 1:46 ` Tejun Heo
2014-06-19 1:49 ` Tejun Heo
2014-06-19 2:29 ` Jens Axboe
2014-06-23 13:53 ` Paolo Valente
2014-06-23 19:20 ` Tejun Heo
2014-07-09 20:54 ` Paolo Valente
2014-05-27 12:42 ` [PATCH RFC RESEND 02/14] block: introduce the BFQ-v0 I/O scheduler paolo
2014-05-27 12:42 ` [PATCH RFC RESEND 03/14] block: add hierarchical-support option to kconfig paolo
2014-05-27 12:42 ` [PATCH RFC RESEND 04/14] block, bfq: add full hierarchical scheduling and cgroups support paolo
2014-05-27 12:42 ` [PATCH RFC RESEND 05/14] block, bfq: improve throughput boosting paolo
2014-05-27 12:42 ` [PATCH RFC RESEND 06/14] block, bfq: modify the peak-rate estimator paolo
2014-05-27 12:42 ` [PATCH RFC RESEND 07/14] block, bfq: add more fairness to boost throughput and reduce latency paolo
2014-05-27 12:42 ` [PATCH RFC RESEND 08/14] block, bfq: improve responsiveness paolo
2014-05-27 12:42 ` [PATCH RFC RESEND 09/14] block, bfq: reduce I/O latency for soft real-time applications paolo
2014-05-27 12:42 ` [PATCH RFC RESEND 10/14] block, bfq: preserve a low latency also with NCQ-capable drives paolo
2014-05-27 12:42 ` [PATCH RFC RESEND 11/14] block, bfq: reduce latency during request-pool saturation paolo
2014-05-27 12:42 ` [PATCH RFC RESEND 12/14] block, bfq: add Early Queue Merge (EQM) paolo
2014-05-27 12:42 ` [PATCH RFC RESEND 13/14] block, bfq: boost the throughput on NCQ-capable flash-based devices paolo
2014-05-27 12:42 ` [PATCH RFC RESEND 14/14] block, bfq: boost the throughput with random I/O on NCQ-capable HDDs paolo
2014-05-30 15:32 ` [PATCH RFC RESEND 00/14] New version of the BFQ I/O Scheduler Vivek Goyal
2014-05-30 16:16 ` Tejun Heo
2014-05-30 17:09 ` Vivek Goyal
2014-05-30 17:26 ` Tejun Heo
2014-05-30 17:55 ` Vivek Goyal
2014-05-30 17:59 ` Tejun Heo
2014-05-30 23:33 ` Paolo Valente
2014-05-30 17:31 ` Vivek Goyal
2014-05-30 17:39 ` Tejun Heo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20140604135627.GB5004@htj.dyndns.org \
--to=tj@kernel.org \
--cc=avanzini.arianna@gmail.com \
--cc=axboe@kernel.dk \
--cc=cgroups@vger.kernel.org \
--cc=containers@lists.linux-foundation.org \
--cc=fchecconi@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=lizefan@huawei.com \
--cc=paolo.valente@unimore.it \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®