Re: tuning for large files in xfs

[Nathan Scott <nathans@sgi.com>]

On Tue, May 23, 2006 at 06:41:36PM -0700, fitzboy wrote:
> I read online in multiple places that the largest allocation groups 
> should get is 4g,

Thats not correct (for a few years now).

> I was also thinking that the more AGs the better since I do a lot of 
> parallel reads/writes... granted it doesn't change the file system all 
> that much (the file only grows or get existing blocks get modified), so 
> I am not sure if the number of AGs matter, does it?

Yes, it can matter.  For large extents like you have here, AGs
introduce a discontinuity that you'd otherwise not have.

> Sorry, I meant that moving the Inode size to 2k (over 256bytes) gave me 
> a sizeable increase in performance... I assume that is because the 
> extent map can be smaller now (since blocks are much larger, less blocks 
> to keep track of). Of course, ideal would be to have InodeSize be large 
> and blocksize to be 32k... but I hit the limits on both...

It means that more extents/btree records fit inline in the inode,
as theres more space available after the stat data.  2k is your
best choice for inode size, stick with that.

> > - Preallocate the space in the file - i.e. before running the
> > dd you can do an "xfs_io -c 'resvsp 0 2t' /mnt/array/disk1/xxx"
> > (preallocates 2 terabytes) and then overwrite that.  Yhis will
> > give you an optimal layout.
> 
> I tried this a couple of times, but it seemed to wedge the machine... I 
> would do: 1) touch a file (just to create it), 2) do the above command 

Oh, use the -f (create) option and you won't need a touch.

> which would then show effect in du, but the file size was still 0 3) I 
> then opened that file (without O_TRUNC or O_APPEND) and started to write 
> out to it. It would work fine for a few minutes but after about 5 or 7GB 
> the machine would freeze... nothing in syslog, only a brief message on 
> console about come cpu state being bad...

Hmm - I'd be interested to hear if that happens with a recent
kernel.

> > - Your extent map is fairly large, the 2.6.17 kernel will have
> > some improvements in the way the memory management is done here
> > which may help you a bit too.
> 
> we have plenty of memory on the machines, shouldn't be an issue... I am 
> a little cautious about moving to a new kernel though...

Its not the amount of memory that was the issue here, its more the
way we were using it that was a problem for kernels of the vintage
you're using here.  You will definately see better performance in
a 2.6.17 kernel with that large extent map.

cheers.

-- 
Nathan