ext3 multiple thread streaming write performance with 2.6.9

ext3 multiple thread streaming write performance with 2.6.9

[Shantanu Goel]

Hi,

I am seeing extremely variable and poor performance
with ext3 in the presence of multiple streaming
writers.  Below are the results of some tests I have
conducted with iozone.  XFS appears to be most
consistent performer for this workload, followed by
ext2 and finally ext3.  Has this been observed
elsewhere?  If so, is it possible to tune ext3 to
perform better on this workload?

Thanks,
Shantanu

Hardware config:
2x2.0 Xeons w/HT, 1.25GB memory, 18GB 15000 SCSI disk
attached to MPT Fusion controller.

Software config:
All SMP kernels with anticipatory I/O schedular used
throughout except for 2.4  Default values for
{pd,bd}flush used throughout.
Iozone parameters:
  -i0 -r 4k -s (256m / # threads) -e -t (# threads)

Results:
The first field is the filesystem type and how it was
mounted (for ext3).  The second field is the #
threads.  The last field is the rate observed by the
parent process in MBytes for sequential write during
each of the 3 runs followed by the average in
parenthesis.  Each filesystem was newly formatted once
before running the benchmark.  There was no other
activity on the machine during the runs.

2.6.9 (from kernel.org)

               ext2: 1: 54 54 54 (54)
               ext2: 2: 25 20 21 (22)
               ext2: 4: 29 32 40 (34)
                xfs: 1: 53 53 52 (53)
                xfs: 2: 49 49 53 (50)
                xfs: 4: 46 50 48 (48)
  ext3:data=ordered: 1: 48 48 47 (48)
  ext3:data=ordered: 2: 39 28  4 (24)
  ext3:data=ordered: 4:  4 23  4 (11)
ext3:data=writeback: 1: 48 48 48 (48)
ext3:data=writeback: 2:  4 34  4 (14)
ext3:data=writeback: 4:  5  4  5 ( 5)

2.6.8-4 (Debian 2.6.8 based kernel)

               ext2: 1: 54 54 54 (54)
               ext2: 2: 21 21 54 (32)
               ext2: 4: 28 23 22 (24)
                xfs: 1: 54 53 53 (53)
                xfs: 2: 49 52 51 (51)
                xfs: 4: 49 48 49 (49)
  ext3:data=ordered: 1: 48 48 48 (48)
  ext3:data=ordered: 2:  4  4 18 ( 9)
  ext3:data=ordered: 4:  6  5  4 ( 5)
ext3:data=writeback: 1: 48 49 50 (49)
ext3:data=writeback: 2:  4 33  4 (14)
ext3:data=writeback: 4:  4  5  4 ( 4)

2.4.27-2 (Debian 2.4.27 based kernel)

               ext2: 1: 46 47 47 (47)
               ext2: 2: 18 23 25 (22)
               ext2: 4: 17 18 23 (19)
                xfs: 1: 49 48 49 (49)
                xfs: 2: 49 48 48 (48)
                xfs: 4: 42 43 43 (43)
  ext3:data=ordered: 1: 39 37 37 (38)
  ext3:data=ordered: 2: 38 37 38 (38)
  ext3:data=ordered: 4: 45 35 35 (38)
ext3:data=writeback: 1: 37 37 37 (37)
ext3:data=writeback: 2: 27 34  9 (23)
ext3:data=writeback: 4:  8  4  6 ( 6)



	
		
__________________________________
Do you Yahoo!?
Yahoo! Mail - You care about security. So do we.
http://promotions.yahoo.com/new_mail

Re: ext3 multiple thread streaming write performance with 2.6.9

[Arjan van de Ven]

On Wed, 2004-10-27 at 21:04 -0700, Shantanu Goel wrote:
> Hi,
> 
> I am seeing extremely variable and poor performance
> with ext3 in the presence of multiple streaming
> writers.  Below are the results of some tests I have
> conducted with iozone.  XFS appears to be most
> consistent performer for this workload, followed by
> ext2 and finally ext3.  Has this been observed
> elsewhere?  If so, is it possible to tune ext3 to
> perform better on this workload?

yes you should use the reservations patch from the -mm tree;
see http://people.redhat.com/arjanv/reservations.png for a graph of the difference 

Re: ext3 multiple thread streaming write performance with 2.6.9

[Shantanu Goel]

Thanks for the pointer Arjan.  That worked out well.
Here are the numbers I got for 2.6.9 with reservation
patches applied from 2.6.9-mm1.

  ext3:data=ordered,reservation: 1: 47 48 48 (47)
  ext3:data=ordered,reservation: 2: 47 44 45 (46)
  ext3:data=ordered,reservation: 4: 45 43 45 (44)
ext3:data=writeback,reservation: 1: 47 49 48 (48)
ext3:data=writeback,reservation: 2: 48 46 44 (46)
ext3:data=writeback,reservation: 4: 46 45 45 (45)
                           ext2: 1: 53 55 54 (54)
                           ext2: 2: 42 52 51 (48)
                           ext2: 4: 21 26 25 (24)
                            xfs: 1: 53 53 53 (53)
                            xfs: 2: 48 53 51 (51)
                            xfs: 4: 43 47 46 (45)

--- Arjan van de Ven <arjan@infradead.org> wrote:

> On Wed, 2004-10-27 at 21:04 -0700, Shantanu Goel
> wrote:
> > Hi,
> > 
> > I am seeing extremely variable and poor
> performance
> > with ext3 in the presence of multiple streaming
> > writers.  Below are the results of some tests I
> have
> > conducted with iozone.  XFS appears to be most
> > consistent performer for this workload, followed
> by
> > ext2 and finally ext3.  Has this been observed
> > elsewhere?  If so, is it possible to tune ext3 to
> > perform better on this workload?
> 
> yes you should use the reservations patch from the
> -mm tree;
> see http://people.redhat.com/arjanv/reservations.png
> for a graph of the difference 
> 
> 



		
__________________________________
Do you Yahoo!?
Read only the mail you want - Yahoo! Mail SpamGuard.
http://promotions.yahoo.com/new_mail