From: "Dong-Jae Kang" <baramsori72@gmail.com>
To: "Ryo Tsuruta" <ryov@valinux.co.jp>
Cc: linux-kernel@vger.kernel.org, dm-devel@redhat.com,
containers@lists.linux-foundation.org,
virtualization@lists.linux-foundation.org,
xen-devel@lists.xensource.com, agk@sourceware.org,
fernando@oss.ntt.co.jp, xemul@openvz.org,
balbir@linux.vnet.ibm.com
Subject: Re: [PATCH 0/2] dm-ioband: I/O bandwidth controller v1.7.0: Introduction
Date: Thu, 9 Oct 2008 15:15:18 +0900 [thread overview]
Message-ID: <2891419e0810082315v28f2f4cbu5f95230db3be0bc1@mail.gmail.com> (raw)
In-Reply-To: <20081008.194022.226783199.ryov@valinux.co.jp>
Hi, Ryo tsuruta.
Thank you for your fast reply.
Your comments was very helpful for me ^^
> 2. tiotest is not an appropriate tool to see how bandwith is shared
> among devices, becasue those three tiotests don't finish at the
> same time, a process which issues I/Os to a device with the highest
> weight finishes first, so you can't see how bandwidth is shared
> from the results of each tiotest.
Yes, you are right, and it is good point for correct IO testing in
dm-ioband and other controllers.
So, I tested dm-ioband and bio-cgroup patches with another IO testing
tool, xdd ver6.5(http://www.ioperformance.com/), after your reply.
Xdd supports O_DIRECT mode and time limit options.
I think, personally, it is proper tool for testing of IO controllers
in Linux Container ML.
And I found some strange points in test results. In fact, it will be
not strange for other persons^^
1. dm-ioband can control IO bandwidth well in O_DIRECT mode(read and
write), I think the result is very reasonable. but it can't control it
in Buffered mode when I checked just only output of xdd. I think
bio-cgroup patches is for solving the problems, is it right? If so,
how can I check or confirm the role of bio-cgroup patches?
2. As showed in test results, the IO performance in Buffered IO mode
is very low compared with it in O_DIRECT mode. In my opinion, the
reverse case is more natural in real life.
Can you give me a answer about it?
3. Compared with physical bandwidth(when it is checked with one
process and without dm-ioband device), the sum of the bandwidth by
dm-ioband has very considerable gap with the physical bandwidth. I
wonder the reason…. Is it overhead of dm-ioband or bio-cgroup patches?
or Are there any another reasons?
the new testing result is like below.
- Testing target : the patches of dm-ioband ver1.7.0 and bio-cgroup
latest version
- Testing Cases
1.Read and write stress test in O_DIRECT IO mode
2.Read and write stress test in not Buffered IO mode
- Testing tool : xdd ver6.5 ( http://www.ioperformance.com/ )
* Total bandwidth
Read IO
T Q Bytes Ops Time Rate IOPS Latency
%CPU OP_Type ReqSize
0 1 1048576000 128000 15.700 66.790 8153.04 0.0001
0.00 read 8192
0 1 1048576000 128000 15.700 66.790 8153.04 0.0001
0.00 read 8192
1 1 1048576000 128000 15.700 66.790 8153.04 0.0001
0.00 read 8192
Write IO
T Q Bytes Ops Time Rate IOPS Latency
%CPU OP_Type ReqSize
0 1 1048576000 128000 14.730 71.185 8689.59 0.0001
0.00 write 8192
0 1 1048576000 128000 14.730 71.185 8689.59 0.0001
0.00 write 8192
1 1 1048576000 128000 14.730 71.185 8689.59 0.0001
0.00 write 8192
* Read IO test in O_DIRECT mode
Command :
xdd.linux -op read -targets 1 /dev/mapper/ioband1 -reqsize 8 -numreqs
128000 -verbose -timelimit 30 –dio
Result :
cgroup1 (weight : 10)
T Q Bytes Ops Time Rate IOPS Latency
%CPU OP_Type ReqSize
0 1 84549632 10321 30.086 2.810 343.05 0.0029
0.00 read 8192
0 1 84549632 10321 30.086 2.810 343.05 0.0029
0.00 read 8192
1 1 84549632 10321 30.086 2.810 343.05 0.0029
0.00 read 8192
cgroup1 (weight : 30)
T Q Bytes Ops Time Rate IOPS Latency
%CPU OP_Type ReqSize
0 1 256425984 31302 30.089 8.522 1040.31 0.0010
0.00 read 8192
0 1 256425984 31302 30.089 8.522 1040.31 0.0010
0.00 read 8192
1 1 256425984 31302 30.089 8.522 1040.31 0.0010
0.00 read 8192
cgroup1 (weight : 60)
T Q Bytes Ops Time Rate IOPS Latency
%CPU OP_Type ReqSize
0 1 483467264 59017 30.000 16.116 1967.22 0.0005
0.00 read 8192
0 1 483467264 59017 30.000 16.116 1967.22 0.0005
0.00 read 8192
1 1 483467264 59017 30.000 16.116 1967.22 0.0005
0.00 read 8192
* Write IO test in O_DIRECT mode
Command :
xdd.linux -op write -targets 1 /dev/mapper/ioband1 -reqsize 8 -numreqs
128000 -verbose -timelimit 30 –dio
Result :
cgroup1 (weight : 10)
T Q Bytes Ops Time Rate IOPS Latency
%CPU OP_Type ReqSize
0 1 106790912 13036 30.034 3.556 434.04 0.0023
0.00 write 8192
0 1 106790912 13036 30.034 3.556 434.04 0.0023
0.00 write 8192
1 1 106790912 13036 30.034 3.556 434.04 0.0023
0.00 write 8192
cgroup1 (weight : 30)
T Q Bytes Ops Time Rate IOPS Latency
%CPU OP_Type ReqSize
0 1 347176960 42380 30.006 11.570 1412.40 0.0007
0.00 write 8192
0 1 347176960 42380 30.006 11.570 1412.40 0.0007
0.00 write 8192
1 1 347176960 42380 30.006 11.570 1412.40 0.0007
0.00 write 8192
cgroup1 (weight : 60)
T Q Bytes Ops Time Rate IOPS Latency
%CPU OP_Type ReqSize
0 1 636256256 77668 30.000 21.209 2588.93 0.0004
0.00 write 8192
0 1 636256256 77668 30.000 21.209 2588.93 0.0004
0.00 write 8192
1 1 636256256 77668 30.000 21.209 2588.93 0.0004
0.00 write 8192
* Read IO test in Buffered IO mode
Command :
xdd.linux -op read -targets 1 /dev/mapper/ioband1 -reqsize 8 -numreqs
128000 -verbose -timelimit 30
Result :
cgroup1 (weight : 10)
T Q Bytes Ops Time Rate IOPS Latency
%CPU OP_Type ReqSize
0 1 161284096 19688 30.012 5.374 656.00 0.0015
0.00 read 8192
0 1 161284096 19688 30.012 5.374 656.00 0.0015
0.00 read 8192
1 1 161284096 19688 30.012 5.374 656.00 0.0015
0.00 read 8192
cgroup1 (weight : 30)
T Q Bytes Ops Time Rate IOPS Latency
%CPU OP_Type ReqSize
0 1 162816000 19875 30.005 5.426 662.38 0.0015
0.00 read 8192
0 1 162816000 19875 30.005 5.426 662.38 0.0015
0.00 read 8192
1 1 162816000 19875 30.005 5.426 662.38 0.0015
0.00 read 8192
cgroup1 (weight : 60)
T Q Bytes Ops Time Rate IOPS Latency
%CPU OP_Type ReqSize
0 1 167198720 20410 30.002 5.573 680.29 0.0015
0.00 read 8192
0 1 167198720 20410 30.002 5.573 680.29 0.0015
0.00 read 8192
1 1 167198720 20410 30.002 5.573 680.29 0.0015
0.00 read 8192
* Write IO test in Buffered IO mode
Command :
xdd.linux -op write -targets 1 /dev/mapper/ioband1 -reqsize 8 -numreqs
128000 -verbose -timelimit 30
Result :
cgroup1 (weight : 10)
T Q Bytes Ops Time Rate IOPS Latency
%CPU OP_Type ReqSize
0 1 550633472 67216 30.017 18.344 2239.30 0.0004
0.00 write 8192
0 1 550633472 67216 30.017 18.344 2239.30 0.0004
0.00 write 8192
1 1 550633472 67216 30.017 18.344 2239.30 0.0004
0.00 write 8192
cgroup1 (weight : 30)
T Q Bytes Ops Time Rate IOPS Latency
%CPU OP_Type ReqSize
0 1 32768 4 32.278 0.001 0.12 8.0694
0.00 write 8192
0 1 32768 4 32.278 0.001 0.12 8.0694
0.00 write 8192
1 1 32768 4 32.278 0.001 0.12 8.0694
0.00 write 8192
cgroup1 (weight : 60)
T Q Bytes Ops Time Rate IOPS Latency
%CPU OP_Type ReqSize
0 1 4505600 550 31.875 0.141 17.25 0.0580
0.00 write 8192
0 1 4505600 550 31.875 0.141 17.25 0.0580
0.00 write 8192
1 1 4505600 550 31.875 0.141 17.25 0.0580
0.00 write 8192
> I use iostat to see the time variation of bandiwdth. The followings
> are the outputs of iostat just after starting three tiotests on the
> same setting as yours.
>
> # iostat -p dm-0 -p dm-1 -p dm-2 1
> Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
> dm-0 5430.00 0.00 10860.00 0 10860
> dm-1 16516.00 0.00 16516.00 0 16516
> dm-2 32246.00 0.00 32246.00 0 32246
>
> avg-cpu: %user %nice %system %iowait %steal %idle
> 0.51 0.00 21.83 76.14 0.00 1.52
>
> Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
> dm-0 5720.00 0.00 11440.00 0 11440
> dm-1 16138.00 0.00 16138.00 0 16138
> dm-2 32734.00 0.00 32734.00 0 32734
> ...
>
Thank you for your kindness ^^
>
> Could you give me the O_DIRECT patch?
>
Of course, if you want. But it is nothing
Tiobench tool is very simple and light source code, so I just add the
O_DIRECT option in tiotest.c of tiobench testing tool.
Anyway, after I make a patch file, I send it to you
Best Regards,
Dong-Jae Kang
next prev parent reply other threads:[~2008-10-09 6:15 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2008-10-03 5:37 Ryo Tsuruta
2008-10-03 5:38 ` [PATCH 1/2] dm-ioband: I/O bandwidth controller v1.7.0: Source code and patch Ryo Tsuruta
2008-10-03 5:39 ` [PATCH 2/2] dm-ioband: I/O bandwidth controller v1.7.0: Document Ryo Tsuruta
2008-10-08 8:29 ` [PATCH 0/2] dm-ioband: I/O bandwidth controller v1.7.0: Introduction Dong-Jae Kang
2008-10-08 10:40 ` Ryo Tsuruta
2008-10-09 6:15 ` Dong-Jae Kang [this message]
2008-10-09 12:14 ` Ryo Tsuruta
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2891419e0810082315v28f2f4cbu5f95230db3be0bc1@mail.gmail.com \
--to=baramsori72@gmail.com \
--cc=agk@sourceware.org \
--cc=balbir@linux.vnet.ibm.com \
--cc=containers@lists.linux-foundation.org \
--cc=dm-devel@redhat.com \
--cc=fernando@oss.ntt.co.jp \
--cc=linux-kernel@vger.kernel.org \
--cc=ryov@valinux.co.jp \
--cc=virtualization@lists.linux-foundation.org \
--cc=xemul@openvz.org \
--cc=xen-devel@lists.xensource.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®