resizing of raid5?

resizing of raid5?

[Roger Abrahamsson]

Hello.

Just figured if anyone could give some information about resizing of software
raid5 systems (2.4.x kernels)? I've been looking all over for information about
if this is possible or not currently, and if not, how this system of raid
cluster blocks work in conjunction with ext2. The code is a bit tricky and not
too many comments to help one try and get a hold of how it works.
Any pointers for this would be nice.

/Roger A

Re: resizing of raid5?

[Neil Brown]

On Wednesday August 1, hyperion@gnyrf.net wrote:
> Hello.
> 
> Just figured if anyone could give some information about resizing of software
> raid5 systems (2.4.x kernels)? I've been looking all over for information about
> if this is possible or not currently, and if not, how this system of raid
> cluster blocks work in conjunction with ext2. The code is a bit tricky and not
> too many comments to help one try and get a hold of how it works.
> Any pointers for this would be nice.

The only way to resize a raid5 array is to back up, rebuild, and
re-load.  Any attempt to re-organise the data, or the linkage, to
avoid this would be more trouble that it is worth.

NeilBrown

Re: resizing of raid5?

[Jakob Østergaard]

On Wed, Aug 01, 2001 at 10:33:04PM +1000, Neil Brown wrote:
> On Wednesday August 1, hyperion@gnyrf.net wrote:
> > Hello.
> > 
> > Just figured if anyone could give some information about resizing of software
> > raid5 systems (2.4.x kernels)? I've been looking all over for information about
> > if this is possible or not currently, and if not, how this system of raid
> > cluster blocks work in conjunction with ext2. The code is a bit tricky and not
> > too many comments to help one try and get a hold of how it works.
> > Any pointers for this would be nice.
> 
> The only way to resize a raid5 array is to back up, rebuild, and
> re-load.  Any attempt to re-organise the data, or the linkage, to
> avoid this would be more trouble that it is worth.

Well, yes and no.

I wrote a resizing tool for raid0 (this is code "A").  I later re-wrote that code
so that it could convert between raid-levels, and do resizing of those levels too
(this is code "B").

However, code "B" was never finished, it had some serious bugs.  Code "A" has
been "mostly stable" (meaning, users and myself have resized several hundred
gigabytes of RAID-0, and I only know of one array that has been trashed).

Now, code "A" was shipped to one company - they re-wrote it to do RAID-5
resizing.

Code "B" was shipped to another company - they fixed the bugs.

Somewhere out there, there should be a tool that will do not only resizing of
the raid levels, but also conversion between them.  It's GPL, it's based on my
code, but I haven't seen it yet   :)

I have not had the changes back from any of those two companies - but this is
mainly because I've been too busy to care.  They seem to intend to release the
changes (after all, my code was GPL),  and I'm going to follow up on this when
I'm done with my thesis - which will be about in a month from now.

-- 
................................................................
:   jakob@unthought.net   : And I see the elder races,         :
:.........................: putrid forms of man                :
:   Jakob Østergaard      : See him rise and claim the earth,  :
:        OZ9ABN           : his downfall is at hand.           :
:.........................:............{Konkhra}...............:

Re: resizing of raid5?

[Roger Abrahamsson]

Quoting Neil Brown <neilb@cse.unsw.edu.au>:

> 
> The only way to resize a raid5 array is to back up, rebuild, and
> re-load.  Any attempt to re-organise the data, or the linkage, to
> avoid this would be more trouble that it is worth.
> 
> NeilBrown

Well, the problem is that it's not that easy when you want to backup say 200GB
of data to find anyone that can hold it for the day while you do it. Well, at
least do it without charge :)
Having things offline for up to a few days while you do the rebuild is okay for
my case, but the offloading the data is not. If I have understood md and raid5,
you create blocks of a fixed size, and calculate the checksum on each "stripe".
To rebuild this looks a bit like disk defragmentation to me, or am I totally
wrong here?

Re: resizing of raid5?

[Andreas Dilger]

Neil Brown writes:
> On Wednesday August 1, hyperion@gnyrf.net wrote:
> > Just figured if anyone could give some information about resizing of
> > software raid5 systems (2.4.x kernels)? I've been looking all over for
> > information about if this is possible or not currently, and if not, how
> > this system of raid cluster blocks work in conjunction with ext2.
> 
> The only way to resize a raid5 array is to back up, rebuild, and
> re-load.  Any attempt to re-organise the data, or the linkage, to
> avoid this would be more trouble that it is worth.

Hmm, this surprises me.  I would have thought it possible to do
"resizing" at least by adding new stripes to the end of the current
RAID 5 volume, using N+1 new "disks" to make up a new stripe group.

Adding a new disk into the stripe set and re-striping the whole thing
is definitely a lot harder.

Cheers, Andreas
-- 
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert

Re: resizing of raid5?

[Neil Brown]

On Wednesday August 1, hyperion@gnyrf.net wrote:
> Quoting Neil Brown <neilb@cse.unsw.edu.au>:
> 
> > 
> > The only way to resize a raid5 array is to back up, rebuild, and
> > re-load.  Any attempt to re-organise the data, or the linkage, to
> > avoid this would be more trouble that it is worth.
> > 
> > NeilBrown
> 
> Well, the problem is that it's not that easy when you want to backup say 200GB
> of data to find anyone that can hold it for the day while you do it. Well, at
> least do it without charge :)
> Having things offline for up to a few days while you do the rebuild is okay for
> my case, but the offloading the data is not. If I have understood md and raid5,
> you create blocks of a fixed size, and calculate the checksum on each "stripe".
> To rebuild this looks a bit like disk defragmentation to me, or am I totally
> wrong here?

Hmm.  Your backup strategy is ????

Yes, you could it.  It would probably be slower that writting to tape
and restoring, but if you don't have a tape......

You would find it very difficult to maintain safety in the event of
failure during the re-org process, but if you are willing to risk that
you could certainly write a fairly stright forward program to do it.

Suppose you are reconfiguring from a N drive array to an M drive
array, and both the old and the new array use a chunk size of C.

Then choose a number X such that the staging area that you have
(either RAM or some other drive) can contain (N-1)*(M-1)*C*X.

e.g. 64K chunks, 7 to 8 drives, 10 Gig disc space:

 X = floor (10240/6/7/64) = 3

If you don't have enough space to stage N*M*C you can still do it but
it will be much more fiddly.

Then read X*N*M*C of the drive using the N drive layout into the
staging area.  Then write it back out using the M drive layout.
Repeat until done.
You don't need to worry about calculating parity when writing out as
you can get the raid5 module to do that automatically when you tell it
about the new array.

If you (or anyone else) would like to try writting code to do this, I
would be happy to review, comment, and test.  You could probably even
do it reasonably well in perl...

NeilBrown

Re: resizing of raid5?

[Neil Brown]

On Wednesday August 1, adilger@turbolinux.com wrote:
> Neil Brown writes:
> > On Wednesday August 1, hyperion@gnyrf.net wrote:
> > > Just figured if anyone could give some information about resizing of
> > > software raid5 systems (2.4.x kernels)? I've been looking all over for
> > > information about if this is possible or not currently, and if not, how
> > > this system of raid cluster blocks work in conjunction with ext2.
> > 
> > The only way to resize a raid5 array is to back up, rebuild, and
> > re-load.  Any attempt to re-organise the data, or the linkage, to
> > avoid this would be more trouble that it is worth.
> 
> Hmm, this surprises me.  I would have thought it possible to do
> "resizing" at least by adding new stripes to the end of the current
> RAID 5 volume, using N+1 new "disks" to make up a new stripe group.

But would you bother.  just make another RAID5 set and append it to
the end using RAID-linear or LVM.  Achieve the same effect at a more
accessible level.  Or am I misunderstanding you?

NeilBrown

Re: resizing of raid5?

[Roger Abrahamsson]

Quoting Neil Brown <neilb@cse.unsw.edu.au>:

> 
> Hmm.  Your backup strategy is ????
> 
> Yes, you could it.  It would probably be slower that writting to tape
> and restoring, but if you don't have a tape......
> 
> You would find it very difficult to maintain safety in the event of
> failure during the re-org process, but if you are willing to risk that
> you could certainly write a fairly stright forward program to do it.
> 
> Suppose you are reconfiguring from a N drive array to an M drive
> array, and both the old and the new array use a chunk size of C.
> 
> Then choose a number X such that the staging area that you have
> (either RAM or some other drive) can contain (N-1)*(M-1)*C*X.
> 
> e.g. 64K chunks, 7 to 8 drives, 10 Gig disc space:
> 
>  X = floor (10240/6/7/64) = 3
> 
> If you don't have enough space to stage N*M*C you can still do it but
> it will be much more fiddly.
> 
> Then read X*N*M*C of the drive using the N drive layout into the
> staging area.  Then write it back out using the M drive layout.
> Repeat until done.
> You don't need to worry about calculating parity when writing out as
> you can get the raid5 module to do that automatically when you tell it
> about the new array.
> 
> If you (or anyone else) would like to try writting code to do this, I
> would be happy to review, comment, and test.  You could probably even
> do it reasonably well in perl...
> 
> NeilBrown
> -

Well, thats more or less how I had thought it. It would even be possible
(if you hack the md/raid5 code) to do a rebuild in a running system. If you
are really short on disk space, you only need to back up the first rows then.
Say if you go from N to M (M>N) size array then you would only  have to worry
about (N-1) / ((M-1)-(N-1)) rows before you would be having backup on disk in
the old rows..

Will look into this during the weekend a bit more and possibly begin to do some
code also, too much work at the moment to have time to play with this now
unfortunately. 

/Roger