Re: [RFC][PATCH 0/2]Extend ext3 filesystem limit from 8TB to 16TB

[Nathan Scott <nathans@sgi.com>]

Hi Andi,

On Fri, Mar 31, 2006 at 03:33:24PM +0200, Andi Kleen wrote:
> Mingming Cao <cmm@us.ibm.com> writes:
> > > Have you done tests _near_ 8TB with a 32-bit machine, even without these
> > > patches?
> > No I haven't. The >8TB right now is attached to a 64 bit machine, but we
> > should able to move it to a 32 bit machine.
> 
> If you use XFS or JFS as backing fs you can use a holey loop device
> to simulate it.  When I tried this last time JFS worked better for me.
> XFS doesn't seem to like that many extents as will be created by 
> mkfs.ext2.

Mainline has this issue resolved now (very recently, post-.16).

This (loopback on a local file) technique will get you up to 16TB
for 32 bit platforms, where you hit the unsigned long page->index
limit (but sounds like thats fine for the testing you're doing).

A related technique we've used in the past in testing XFS on large
devices (we've successfully tested in petabyte ranges using this,
on 64 bit systems of course) is to write a tool that modifies the
values in the ondisk data structures managing the "lower" areas of
the device to say "all the space here is used", which then forces
new allocations to be done in the "higher" parts of the device
address space.  Testing then follows this recipe: mkfs-on-loop,
then run the tool, then mount, then run the usual test suites ...
perhaps thats useful here too (I dunno if the ext2/3 format lends
itself to that or not).

cheers.

-- 
Nathan