From: Nauman Rafique <nauman@google.com>
To: Vivek Goyal <vgoyal@redhat.com>
Cc: Rik van Riel <riel@redhat.com>,
linux-kernel@vger.kernel.org, jens.axboe@oracle.com,
containers@lists.linux-foundation.org, dm-devel@redhat.com,
dpshah@google.com, lizf@cn.fujitsu.com, mikew@google.com,
fchecconi@gmail.com, paolo.valente@unimore.it,
ryov@valinux.co.jp, fernando@oss.ntt.co.jp,
s-uchida@ap.jp.nec.com, taka@valinux.co.jp,
guijianfeng@cn.fujitsu.com, jmoyer@redhat.com,
dhaval@linux.vnet.ibm.com, balbir@linux.vnet.ibm.com,
righi.andrea@gmail.com, m-ikeda@ds.jp.nec.com, agk@redhat.com,
akpm@linux-foundation.org, peterz@infradead.org,
jmarchan@redhat.com, torvalds@linux-foundation.org,
mingo@elte.hu
Subject: Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
Date: Mon, 31 Aug 2009 16:51:25 -0700 [thread overview]
Message-ID: <e98e18940908311651s26de5b70ye6f4a82402956309@mail.gmail.com> (raw)
In-Reply-To: <20090831185640.GF3758@redhat.com>
On Mon, Aug 31, 2009 at 11:56 AM, Vivek Goyal<vgoyal@redhat.com> wrote:
> On Mon, Aug 31, 2009 at 01:34:54PM -0400, Rik van Riel wrote:
>> Vivek Goyal wrote:
>>> o blkio_cgroup patches from Ryo to track async bios.
>>>
>>> o This functionality is used to determine the group of async IO from page
>>> instead of context of submitting task.
>>>
>>> Signed-off-by: Hirokazu Takahashi <taka@valinux.co.jp>
>>> Signed-off-by: Ryo Tsuruta <ryov@valinux.co.jp>
>>> Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
>>
>> This seems to be the most complex part of the code so far,
>> but I see why this code is necessary.
>>
>
> Hi Rik,
>
> Thanks for reviewing the patches. I wanted to have better understanding of
> where all does it help to associate a bio to the group of process who
> created/owned the page. Hence few thoughts.
>
> When a bio is submitted to IO scheduler, it needs to determine the group
> bio belongs to and group which should be charged to. There seem to be two
> methods.
>
> - Attribute the bio to cgroup submitting process belongs to.
> - For async requests, track the original owner hence cgroup of the page
> and charge that group for the bio.
>
> One can think of pros/cons of both the approaches.
>
> - The primary use case of tracking async context seems be that if a
> process T1 in group G1 mmaps a big file and then another process T2 in
> group G2, asks for memory and triggers reclaim and generates writes of
> the file pages mapped by T1, then these writes should not be charged to
> T2, hence blkio_cgroup pages.
>
> But the flip side of this might be that group G2 is a low weight group
> and probably too busy also right now, which will delay the write out
> and possibly T2 will wait longer for memory to be allocated.
>
> - At one point of time Andrew mentioned that buffered writes are generally a
> big problem and one needs to map these to owner's group. Though I am not
> very sure what specific problem he was referring to. Can we attribute
> buffered writes to pdflush threads and move all pdflush threads in a
> cgroup to limit system wide write out activity?
>
> - Somebody also gave an example where there is a memory hogging process and
> possibly pushes out some processes to swap. It does not sound fair to
> charge those proccess for that swap writeout. These processes never
> requested swap IO.
>
> - If there are multiple buffered writers in the system, then those writers
> can also be forced to writeout some pages to disk before they are
> allowed to dirty more pages. As per the page cache design, any writer
> can pick any inode and start writing out pages. So it can happen a
> weight group task is writting out pages dirtied by a lower weight group
> task. If, async bio is mapped to owner's group, it might happen that
> higher weight group task might be made to sleep on lower weight group
> task because request descriptors are all consumed up.
>
> It looks like there does not seem to be a clean way which covers all the
> cases without issues. I am just trying to think, what is a simple way
> which covers most of the cases. Can we just stick to using submitting task
> context to determine a bio's group (as cfq does). Which can result in
> following.
>
> - Less code and reduced complexity.
>
> - Buffered writes will be charged to pdflush and its group. If one wish to
> limit buffered write activity for pdflush, one can move all the pdflush
> threads into a group and assign desired weight. Writes submitted in
> process context will continue to be charged to that process irrespective
> of the fact who dirtied that page.
What if we wanted to control buffered write activity per group? If a
group keeps dirtying pages, we wouldn't want it to dominate the disk
IO capacity at the expense of other cgroups (by dominating the writes
sent down by pdflush).
>
> - swap activity will be charged to kswapd and its group. If swap writes
> are coming from process context, it gets charged to process and its
> group.
>
> - If one is worried about the case of one process being charged for write
> out of file mapped by another process during reclaim, then we can
> probably make use of memory controller and mount memory controller and
> io controller together on same hierarchy. I am told that with memory
> controller, group's memory will be reclaimed by the process requesting
> more memory. If that's the case, then IO will automatically be charged
> to right group if we use submitting task context.
>
> I just wanted to bring this point forward for more discussions to know
> what is the right thing to do? Use bio tracking or not.
>
> Ryo, any thoughts on this?
>
> Thanks
> Vivek
>
>> Acked-by: Rik van Riel <riel@redhat.com>
>
next prev parent reply other threads:[~2009-08-31 23:51 UTC|newest]
Thread overview: 113+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-08-28 21:30 [RFC] IO scheduler based IO controller V9 Vivek Goyal
2009-08-28 21:30 ` [PATCH 01/23] io-controller: Documentation Vivek Goyal
2009-08-28 21:30 ` [PATCH 02/23] io-controller: Core of the elevator fair queuing Vivek Goyal
2009-08-28 22:26 ` Rik van Riel
2009-08-28 21:30 ` [PATCH 03/23] io-controller: Common flat fair queuing code in elevaotor layer Vivek Goyal
2009-08-29 1:29 ` Rik van Riel
2009-08-28 21:30 ` [PATCH 04/23] io-controller: Modify cfq to make use of flat elevator fair queuing Vivek Goyal
2009-08-29 1:44 ` Rik van Riel
2009-08-28 21:30 ` [PATCH 05/23] io-controller: Core scheduler changes to support hierarhical scheduling Vivek Goyal
2009-08-29 3:31 ` Rik van Riel
2009-08-28 21:30 ` [PATCH 06/23] io-controller: cgroup related changes for hierarchical group support Vivek Goyal
2009-08-29 3:37 ` Rik van Riel
2009-08-28 21:30 ` [PATCH 07/23] io-controller: Common hierarchical fair queuing code in elevaotor layer Vivek Goyal
2009-08-29 23:04 ` Rik van Riel
2009-09-03 3:08 ` Munehiro Ikeda
2009-09-10 20:11 ` Vivek Goyal
2009-08-28 21:30 ` [PATCH 08/23] io-controller: cfq changes to use " Vivek Goyal
2009-08-29 23:11 ` Rik van Riel
2009-08-28 21:30 ` [PATCH 09/23] io-controller: Export disk time used and nr sectors dipatched through cgroups Vivek Goyal
2009-08-29 23:12 ` Rik van Riel
2009-08-28 21:30 ` [PATCH 10/23] io-controller: Debug hierarchical IO scheduling Vivek Goyal
2009-08-30 0:10 ` Rik van Riel
2009-08-28 21:31 ` [PATCH 11/23] io-controller: Introduce group idling Vivek Goyal
2009-08-30 0:38 ` Rik van Riel
2009-09-18 3:56 ` [PATCH] io-controller: Fix another bug that causing system hanging Gui Jianfeng
2009-09-18 14:47 ` Vivek Goyal
2009-08-28 21:31 ` [PATCH 12/23] io-controller: Wait for requests to complete from last queue before new queue is scheduled Vivek Goyal
2009-08-30 0:40 ` Rik van Riel
2009-08-28 21:31 ` [PATCH 13/23] io-controller: Separate out queue and data Vivek Goyal
2009-08-31 15:27 ` Rik van Riel
2009-08-28 21:31 ` [PATCH 14/23] io-conroller: Prepare elevator layer for single queue schedulers Vivek Goyal
2009-08-31 2:49 ` Rik van Riel
2009-08-28 21:31 ` [PATCH 15/23] io-controller: noop changes for hierarchical fair queuing Vivek Goyal
2009-08-31 2:52 ` Rik van Riel
2009-09-10 17:32 ` Vivek Goyal
2009-08-28 21:31 ` [PATCH 16/23] io-controller: deadline " Vivek Goyal
2009-08-31 3:13 ` Rik van Riel
2009-08-31 13:46 ` Vivek Goyal
2009-08-28 21:31 ` [PATCH 17/23] io-controller: anticipatory " Vivek Goyal
2009-08-31 17:21 ` Rik van Riel
2009-08-28 21:31 ` [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios Vivek Goyal
2009-08-31 17:34 ` Rik van Riel
2009-08-31 18:56 ` Vivek Goyal
2009-08-31 23:51 ` Nauman Rafique [this message]
2009-09-01 7:00 ` Ryo Tsuruta
2009-09-01 14:11 ` Vivek Goyal
2009-09-01 14:53 ` Rik van Riel
2009-09-01 18:02 ` Nauman Rafique
2009-09-02 0:59 ` KAMEZAWA Hiroyuki
2009-09-02 3:12 ` Balbir Singh
2009-09-02 9:52 ` Ryo Tsuruta
2009-09-02 13:58 ` Vivek Goyal
2009-09-03 2:24 ` Ryo Tsuruta
2009-09-03 2:40 ` Vivek Goyal
2009-09-03 3:41 ` Ryo Tsuruta
2009-08-28 21:31 ` [PATCH 19/23] io-controller: map async requests to appropriate cgroup Vivek Goyal
2009-08-31 17:39 ` Rik van Riel
2009-08-28 21:31 ` [PATCH 20/23] io-controller: Per cgroup request descriptor support Vivek Goyal
2009-08-31 17:54 ` Rik van Riel
2009-09-14 18:33 ` Nauman Rafique
2009-09-16 18:47 ` Vivek Goyal
2009-08-28 21:31 ` [PATCH 21/23] io-controller: Per io group bdi congestion interface Vivek Goyal
2009-08-31 19:49 ` Rik van Riel
2009-08-28 21:31 ` [PATCH 22/23] io-controller: Support per cgroup per device weights and io class Vivek Goyal
2009-08-31 20:56 ` Rik van Riel
2009-08-28 21:31 ` [PATCH 23/23] io-controller: debug elevator fair queuing support Vivek Goyal
2009-08-31 20:57 ` Rik van Riel
2009-08-31 21:01 ` Vivek Goyal
2009-08-31 21:12 ` Rik van Riel
2009-08-31 1:09 ` [RFC] IO scheduler based IO controller V9 Gui Jianfeng
2009-09-02 0:58 ` Gui Jianfeng
2009-09-02 13:45 ` Vivek Goyal
2009-09-07 2:14 ` Gui Jianfeng
2009-09-08 13:55 ` Vivek Goyal
2009-09-07 7:40 ` Gui Jianfeng
2009-09-08 13:53 ` Vivek Goyal
2009-09-08 19:19 ` Vivek Goyal
2009-09-09 7:38 ` Gui Jianfeng
2009-09-09 15:05 ` Vivek Goyal
2009-09-10 3:20 ` Gui Jianfeng
2009-09-11 1:15 ` [PATCH] io-controller: Fix task hanging when there are more than one groups Gui Jianfeng
2009-09-14 2:44 ` Vivek Goyal
2009-09-15 3:37 ` Vivek Goyal
2009-09-16 0:05 ` Gui Jianfeng
2009-09-16 2:58 ` Gui Jianfeng
2009-09-16 18:09 ` Vivek Goyal
2009-09-17 6:08 ` Gui Jianfeng
2009-09-24 1:10 ` Gui Jianfeng
2009-09-09 9:41 ` [RFC] IO scheduler based IO controller V9 Jens Axboe
2009-09-08 22:28 ` Vivek Goyal
2009-09-08 22:28 ` [PATCH 24/23] io-controller: Don't leave a queue active when a disk is idle Vivek Goyal
2009-09-09 3:39 ` Rik van Riel
2009-09-08 22:28 ` [PATCH 25/23] io-controller: fix queue vs group fairness Vivek Goyal
2009-09-08 22:37 ` Daniel Walker
2009-09-09 1:09 ` Vivek Goyal
2009-09-08 23:13 ` Fabio Checconi
2009-09-09 1:32 ` Vivek Goyal
2009-09-09 2:03 ` Fabio Checconi
2009-09-09 4:44 ` Rik van Riel
2009-09-08 22:28 ` [PATCH 26/23] io-controller: fix writer preemption with in a group Vivek Goyal
2009-09-09 4:59 ` Rik van Riel
2009-09-10 15:18 ` [RFC] IO scheduler based IO controller V9 Jerome Marchand
2009-09-10 20:52 ` Vivek Goyal
2009-09-10 20:56 ` Vivek Goyal
2009-09-11 13:16 ` Jerome Marchand
2009-09-11 14:30 ` Vivek Goyal
2009-09-11 14:43 ` Vivek Goyal
2009-09-11 14:55 ` Jerome Marchand
2009-09-11 15:01 ` Vivek Goyal
2009-09-11 14:44 ` Jerome Marchand
2009-09-14 14:26 ` Jerome Marchand
2009-09-13 18:54 ` Vivek Goyal
2009-09-14 14:31 ` Jerome Marchand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e98e18940908311651s26de5b70ye6f4a82402956309@mail.gmail.com \
--to=nauman@google.com \
--cc=agk@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=balbir@linux.vnet.ibm.com \
--cc=containers@lists.linux-foundation.org \
--cc=dhaval@linux.vnet.ibm.com \
--cc=dm-devel@redhat.com \
--cc=dpshah@google.com \
--cc=fchecconi@gmail.com \
--cc=fernando@oss.ntt.co.jp \
--cc=guijianfeng@cn.fujitsu.com \
--cc=jens.axboe@oracle.com \
--cc=jmarchan@redhat.com \
--cc=jmoyer@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=lizf@cn.fujitsu.com \
--cc=m-ikeda@ds.jp.nec.com \
--cc=mikew@google.com \
--cc=mingo@elte.hu \
--cc=paolo.valente@unimore.it \
--cc=peterz@infradead.org \
--cc=riel@redhat.com \
--cc=righi.andrea@gmail.com \
--cc=ryov@valinux.co.jp \
--cc=s-uchida@ap.jp.nec.com \
--cc=taka@valinux.co.jp \
--cc=torvalds@linux-foundation.org \
--cc=vgoyal@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®