Some dbench 32 results for 2.4.8-pre8, 2.4.7-ac10, and 2.4.7

Some dbench 32 results for 2.4.8-pre8, 2.4.7-ac10, and 2.4.7

[Steven Cole]

I ran dbench 32 for 2.4.8-pre8, 2.4.7-ac10, and 2.4.7.
Each set of three runs were performed right after a boot,
running vmstat, and time ./dbench 32 with no pauses in
between.  The hardware is 384 MB, 450 P3, UP, IDE disk with
ReiserFS on all partitions. The tests were done from a
transparent Konsole and KDE2.

After this summary, the more verbose results are provided.

Steven

Run #1 Throughput 5.88569 MB/sec   2.4.8-pre8
Run #2 Throughput 5.95613 MB/sec   2.4.8-pre8
Run #3 Throughput 5.8547 MB/sec    2.4.8-pre8

Run #4 Throughput 7.84171 MB/sec   2.4.7-ac10
Run #5 Throughput 7.68447 MB/sec   2.4.7-ac10
Run #6 Throughput 7.85119 MB/sec   2.4.7-ac10

Run #7 Throughput 10.2184 MB/sec   2.4.7
Run #8 Throughput 10.0105 MB/sec   2.4.7
Run #9 Throughput 10.0215 MB/sec   2.4.7

-------------------------------------------------------------------------------
2.4.8-pre8

   procs                      memory    swap          io     system         cpu
 r  b  w   swpd   free   buff  cache  si  so    bi    bo   in    cs  us  sy  id
 2  0  0      0 267028   4180  61008   0   0   176    52  572   209  10   8  82

32 clients started
[....snipped]
Throughput 5.88569 MB/sec (NB=7.35711 MB/sec  58.8569 MBit/sec)
37.16user 398.40system 11:57.77elapsed 60%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (1011major+1435minor)pagefaults 0swaps
   procs                      memory    swap          io     system         cpu
 r  b  w   swpd   free   buff  cache  si  so    bi    bo   in    cs  us  sy  id
 1  0  0      0 281788   6780  38076   0   0   131  1873 4114   235   7  59  34

32 clients started
[....snipped]
Throughput 5.95613 MB/sec (NB=7.44517 MB/sec  59.5613 MBit/sec)
36.27user 393.72system 11:50.20elapsed 60%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (1011major+1435minor)pagefaults 0swaps
   procs                      memory    swap          io     system         cpu
 r  b  w   swpd   free   buff  cache  si  so    bi    bo   in    cs  us  sy  id
 2  0  0      0 282196   6880  38088   0   0   121  2227 4799   239   7  68  25

32 clients started
[....snipped]
Throughput 5.8547 MB/sec (NB=7.31837 MB/sec  58.547 MBit/sec)
36.53user 397.94system 12:02.49elapsed 60%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (1011major+1435minor)pagefaults 0swaps
   procs                      memory    swap          io     system         cpu
 r  b  w   swpd   free   buff  cache  si  so    bi    bo   in    cs  us  sy  id
 1  0  0      0 281508   7624  38092   0   0   115  2377 5087   241   7  72  21


-------------------------------------------------------------------------------
2.4.7-ac10

   procs                      memory    swap          io     system         cpu
 r  b  w   swpd   free   buff  cache  si  so    bi    bo   in    cs  us  sy  id
 0  0  1      0 266976   4160  60952   0   0   461   115 1284   428  25  21  54

32 clients started
[....snipped]
Throughput 7.84171 MB/sec (NB=9.80214 MB/sec  78.4171 MBit/sec)
36.41user 289.55system 8:59.79elapsed 60%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (1011major+1435minor)pagefaults 0swaps
   procs                      memory    swap          io     system         cpu
 r  b  w   swpd   free   buff  cache  si  so    bi    bo   in    cs  us  sy  id
 0  0  0      0 295104   4472  24608   0   0   199  2014 4534   327  11  67  22

32 clients started
[....snipped]
Throughput 7.68447 MB/sec (NB=9.60559 MB/sec  76.8447 MBit/sec)
36.21user 287.81system 9:10.72elapsed 58%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (1011major+1435minor)pagefaults 0swaps
   procs                      memory    swap          io     system         cpu
 r  b  w   swpd   free   buff  cache  si  so    bi    bo   in    cs  us  sy  id
 1  0  0      0 296612   4768  23716   0   0   169  2242 4926   320   9  73  18

32 clients started
[....snipped]
Throughput 7.85119 MB/sec (NB=9.81399 MB/sec  78.5119 MBit/sec)
36.55user 272.83system 8:59.02elapsed 57%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (1011major+1435minor)pagefaults 0swaps
   procs                      memory    swap          io     system         cpu
 r  b  w   swpd   free   buff  cache  si  so    bi    bo   in    cs  us  sy  id
 2  0  0      0 296796   4648  23720   0   0   158  2317 5055   318   9  75  17

-------------------------------------------------------------------------------
2.4.7

   procs                      memory    swap          io     system         cpu
 r  b  w   swpd   free   buff  cache  si  so    bi    bo   in    cs  us  sy  id
 0  0  0      0 250760  20028  60940   0   0   601   109 1550   448  25  24  51

32 clients started
[....snipped]
Throughput 10.2184 MB/sec (NB=12.773 MB/sec  102.184 MBit/sec)
35.41user 286.38system 6:53.41elapsed 77%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (1011major+1435minor)pagefaults 0swaps
   procs                      memory    swap          io     system         cpu
 r  b  w   swpd   free   buff  cache  si  so    bi    bo   in    cs  us  sy  id
 1  0  0      0 299012   4208  24496   0   0   278  1964 4591   242  13  72  15

32 clients started
[....snipped]
Throughput 10.0105 MB/sec (NB=12.5131 MB/sec  100.105 MBit/sec)
36.53user 283.74system 7:02.99elapsed 75%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (1011major+1435minor)pagefaults 0swaps
   procs                      memory    swap          io     system         cpu
 r  b  w   swpd   free   buff  cache  si  so    bi    bo   in    cs  us  sy  id
 1  0  0      0 299364   4800  23628   0   0   231  2222 5010   218  11  78  10

32 clients started
[....snipped]
Throughput 10.0215 MB/sec (NB=12.5269 MB/sec  100.215 MBit/sec)
35.69user 287.32system 7:02.51elapsed 76%CPU (0avgtext+0avgdata 0maxresident)k
[....snipped]
0inputs+0outputs (1011major+1435minor)pagefaults 0swaps
   procs                      memory    swap          io     system         cpu
 r  b  w   swpd   free   buff  cache  si  so    bi    bo   in    cs  us  sy  id
 2  0  0      0 299236   4796  23748   0   0   213  2321 5171   207  11  81   9

Re: Some dbench 32 results for 2.4.8-pre8, 2.4.7-ac10, and 2.4.7

[Linus Torvalds]

In article <200108100502.f7A52Ve23324@thor.mesatop.com> you write:
>
>I ran dbench 32 for 2.4.8-pre8, 2.4.7-ac10, and 2.4.7.
>Each set of three runs were performed right after a boot,
>running vmstat, and time ./dbench 32 with no pauses in
>between.  The hardware is 384 MB, 450 P3, UP, IDE disk with
>ReiserFS on all partitions. The tests were done from a
>transparent Konsole and KDE2.

Note that dbench performs best when no writeback actually takes place:
the whole benchmark is completely optimizable. As such, the best numbers
for dbench tend to be with (a) kflushd stopped, and (b) the dirty
threshold set high.

Does the numbers change if you do something like

	killall -STOP kupdated
	echo 80 64 64 256 500 6000 90 > /proc/sys/vm/bdflush

to make it less eager to write stuff out? (That just stops the
every-five-second flush, and makes the dirty balancing numbers be 80/90%
instead of the default 30/60%)

In particular, the dirty balancing worked really badly before, and was
just fixed.  I suspect that the bdflush numbers were tuned with the
badly-working case, and they might be a bit too aggressive for dbench
these days.. 

			Linus

Re: Some dbench 32 results for 2.4.8-pre8, 2.4.7-ac10, and 2.4.7

[Steven Cole]

Linus Torvalds wrote:
>Does the numbers change if you do something like
>
>	killall -STOP kupdated
>	echo 80 64 64 256 500 6000 90 > /proc/sys/vm/bdflush
>
>to make it less eager to write stuff out? (That just stops the
>every-five-second flush, and makes the dirty balancing numbers be 80/90%
>instead of the default 30/60%)
>
>In particular, the dirty balancing worked really badly before, and was
>just fixed.  I suspect that the bdflush numbers were tuned with the
>badly-working case, and they might be a bit too aggressive for dbench
>these days.. 

Sorry Linus for the late reply, but I went to bed just before your message.
I re-ran dbench 32 with the settings above.  Here are the results:
(Same conditions as in the previous tests otherwise).  Now, off to my day job.
 
Run #1 Throughput 12.7943 MB/sec    2.4.8-pre8
Run #2 Throughput 12.667 MB/sec     2.4.8-pre8
Run #3 Throughput 12.7091 MB/sec    2.4.8-pre8
 
Run #4 Throughput 13.7765 MB/sec    2.4.7
Run #5 Throughput 13.9632 MB/sec    2.4.7
Run #6 Throughput 13.9318 MB/sec    2.4.7

Steven

Re: Some dbench 32 results for 2.4.8-pre8, 2.4.7-ac10, and 2.4.7

[Linus Torvalds]

On Fri, 10 Aug 2001, Steven Cole wrote:
> Linus Torvalds wrote:
> >Does the numbers change if you do something like
> >
> >	killall -STOP kupdated
> >	echo 80 64 64 256 500 6000 90 > /proc/sys/vm/bdflush
> >
> >In particular, the dirty balancing worked really badly before, and was
> >just fixed.  I suspect that the bdflush numbers were tuned with the
> >badly-working case, and they might be a bit too aggressive for dbench
> >these days..
>
> Sorry Linus for the late reply, but I went to bed just before your message.
> I re-ran dbench 32 with the settings above.  Here are the results:

[ Good, looks more like it ]

Now, the problem with dbench is that no way in hell should you optimize
for dbench in general, because it is a sucky kind of benchmark.

For example, waiting until the last possible minute for writeouts is
definitely the best setting for dbench, but it's a pretty horrible setting
for usability.

I suspect that for optimal dbench performance we'll always have to let teh
system admin do the above kind of horrible tweaking stuff, but at the same
time I personally absolutely detest the need for tweaks in general, and I
would like the default behaviour to be reasonable.

Killing kupdated, for example, is not really "reasonable". But I also
suspect that now that dirty balancing works sanely, the "start writeout at
30% full" is a bit early too.

So instead of the 30/60% split (the first number is "when do we start
writing things out", and the second number is "when do we start actively
waiting for it"), a 50/75% setup might be more reasonable for regular
loads, while making dbench at least a bit happier.

Are you (or others) willing to play around with the numbers a bit and look
at both dbench performance and at interactive feel?

In general

	echo x 64 64 256 500 3000 y > /proc/sys/vm/bdflush

will set the "start writeout" to 'x'%, and the "start synchronous wait" to
'y'% (and restart kupdated with "killall -CONT kupdated"). It would be
interesting to hear where the sweet spot is.

		Linus

Re: Some dbench 32 results for 2.4.8-pre8, 2.4.7-ac10, and 2.4.7

[Steven Cole]

On Friday 10 August 2001 10:45, Linus Torvalds wrote:
> On Fri, 10 Aug 2001, Steven Cole wrote:
> > Linus Torvalds wrote:
> > >Does the numbers change if you do something like
> > >
> > >	killall -STOP kupdated
> > >	echo 80 64 64 256 500 6000 90 > /proc/sys/vm/bdflush
> > >
> > >In particular, the dirty balancing worked really badly before, and was
> > >just fixed.  I suspect that the bdflush numbers were tuned with the
> > >badly-working case, and they might be a bit too aggressive for dbench
> > >these days..
> >
> > Sorry Linus for the late reply, but I went to bed just before your
> > message. I re-ran dbench 32 with the settings above.  Here are the
> > results:
>
> [ Good, looks more like it ]
>
> Now, the problem with dbench is that no way in hell should you optimize
> for dbench in general, because it is a sucky kind of benchmark.
>
> For example, waiting until the last possible minute for writeouts is
> definitely the best setting for dbench, but it's a pretty horrible setting
> for usability.
>
> I suspect that for optimal dbench performance we'll always have to let teh
> system admin do the above kind of horrible tweaking stuff, but at the same
> time I personally absolutely detest the need for tweaks in general, and I
> would like the default behaviour to be reasonable.
>
> Killing kupdated, for example, is not really "reasonable". But I also
> suspect that now that dirty balancing works sanely, the "start writeout at
> 30% full" is a bit early too.
>
> So instead of the 30/60% split (the first number is "when do we start
> writing things out", and the second number is "when do we start actively
> waiting for it"), a 50/75% setup might be more reasonable for regular
> loads, while making dbench at least a bit happier.
>
> Are you (or others) willing to play around with the numbers a bit and look
> at both dbench performance and at interactive feel?
>
> In general
>
> 	echo x 64 64 256 500 3000 y > /proc/sys/vm/bdflush
>
> will set the "start writeout" to 'x'%, and the "start synchronous wait" to
> 'y'% (and restart kupdated with "killall -CONT kupdated"). It would be
> interesting to hear where the sweet spot is.
>
> 		Linus
>

I ran dbench 32 twenty-eight times, with the results below.
The units are MB/sec throughput as reported by dbench.
The rows are the first number, "start write out";
the columns are the second number, "start synchronous wait".
For example, the lower left hand entry was obtained after
echo 30 64 64 256 500 6000 60 > /proc/sys/vm/bdflush
 
        60              70              80              90
 
90      8.693           8.690           8.853           8.875
80      8.822           8.724           8.703           8.910
70      8.558           8.557           8.785           8.420
60      8.194           8.141           8.006           7.847
50      6.662           6.738           6.904           6.892
40      6.347           6.252           6.380           6.274
30      5.687           5.802           6.209           5.898
 
I don't have a good quantifiable set of results for "interactive feel",
other than it's never very good under heavy load.  I'll try to get some
useable results for interactive feel while running dbench again this
weekend, looking for that elusive sweet spot.

Steven

Re: Some dbench 32 results for 2.4.8-pre8, 2.4.7-ac10, and 2.4.7

[Steven Cole]

On Friday 10 August 2001 16:51, Steven Cole wrote:
> I ran dbench 32 twenty-eight times, with the results below.

Sorry, I forgot to mention that this was all with 2.4.8-pre8.

Steven