Linux mdadm superblock question.

Linux mdadm superblock question.

[Justin Piszcz]

Hi,

I may be converting a host to ext4 and was curious, is 0.90 still the only 
superblock version for mdadm/raid-1 that you can boot from without having 
to create an initrd/etc?

Are there any benefits to using a superblock > 0.90 for a raid-1 boot 
volume < 2TB?

Justin.

Re: Linux mdadm superblock question.

[Michael Evans]

On Thu, Feb 11, 2010 at 3:00 PM, Justin Piszcz <jpiszcz@lucidpixels.com> wrote:
> Hi,
>
> I may be converting a host to ext4 and was curious, is 0.90 still the only
> superblock version for mdadm/raid-1 that you can boot from without having to
> create an initrd/etc?
>
> Are there any benefits to using a superblock > 0.90 for a raid-1 boot volume
> < 2TB?
>
> Justin.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

You need the superblock at the end of the partition:  If you read the
manual that is clearly either version 0.90 OR 1.0 (NOT 1.1 and also
NOT 1.2; those use the same superblock layout but different
locations).

Re: Linux mdadm superblock question.

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 620 bytes --]

also sprach Justin Piszcz <jpiszcz@lucidpixels.com> [2010.02.12.1200 +1300]:
> I may be converting a host to ext4 and was curious, is 0.90 still the
> only superblock version for mdadm/raid-1 that you can boot from
> without having to create an initrd/etc?
> 
> Are there any benefits to using a superblock > 0.90 for a raid-1 boot
> volume < 2TB?

FYI: http://bugs.debian.org/492897

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
it may look like i'm just sitting here doing nothing.
but i'm really actively waiting
for all my problems to go away.
 
spamtraps: madduck.bogus@madduck.net

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Linux mdadm superblock question.

[H. Peter Anvin]

On 02/11/2010 05:52 PM, Michael Evans wrote:
> On Thu, Feb 11, 2010 at 3:00 PM, Justin Piszcz <jpiszcz@lucidpixels.com> wrote:
>> Hi,
>>
>> I may be converting a host to ext4 and was curious, is 0.90 still the only
>> superblock version for mdadm/raid-1 that you can boot from without having to
>> create an initrd/etc?
>>
>> Are there any benefits to using a superblock > 0.90 for a raid-1 boot volume
>> < 2TB?
>>
>> Justin.
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
> 
> You need the superblock at the end of the partition:  If you read the
> manual that is clearly either version 0.90 OR 1.0 (NOT 1.1 and also
> NOT 1.2; those use the same superblock layout but different
> locations).

0.9 has the *serious* problem that it is hard to distinguish a whole-volume

However, apparently mdadm recently switched to a 1.1 default.  I
strongly urge Neil to change that to either 1.0 and 1.2, as I have
started to get complaints from users that they have made RAID volumes
with newer mdadm which apparently default to 1.1, and then want to boot
from them (without playing MBR games like Grub does.)  I have to tell
them that they have to regenerate their disks -- the superblock occupies
the boot sector and there is nothing I can do about it.  It's the same
pathology XFS has.

	-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

Re: Linux mdadm superblock question.

[Justin Piszcz]

On Sat, 13 Feb 2010, H. Peter Anvin wrote:

> On 02/11/2010 05:52 PM, Michael Evans wrote:
>> On Thu, Feb 11, 2010 at 3:00 PM, Justin Piszcz <jpiszcz@lucidpixels.com> wrote:
>>> Hi,
>>>
>>> I may be converting a host to ext4 and was curious, is 0.90 still the only
>>> superblock version for mdadm/raid-1 that you can boot from without having to
>>> create an initrd/etc?
>>>
>>> Are there any benefits to using a superblock > 0.90 for a raid-1 boot volume
>>> < 2TB?
>>>
>>> Justin.
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>
>> You need the superblock at the end of the partition:  If you read the
>> manual that is clearly either version 0.90 OR 1.0 (NOT 1.1 and also
>> NOT 1.2; those use the same superblock layout but different
>> locations).
>
> 0.9 has the *serious* problem that it is hard to distinguish a whole-volume
>
> However, apparently mdadm recently switched to a 1.1 default.  I
> strongly urge Neil to change that to either 1.0 and 1.2, as I have
> started to get complaints from users that they have made RAID volumes
> with newer mdadm which apparently default to 1.1, and then want to boot
> from them (without playing MBR games like Grub does.)  I have to tell
> them that they have to regenerate their disks -- the superblock occupies
> the boot sector and there is nothing I can do about it.  It's the same
> pathology XFS has.
>
> 	-hpa
>

My original question was does the newer superblock do anything special or 
offer new features *BESIDES* the quicker resync?

Justin.

Re: Linux mdadm superblock question.

[david]

On Sat, 13 Feb 2010, Justin Piszcz wrote:

> On Sat, 13 Feb 2010, H. Peter Anvin wrote:
>
>> On 02/11/2010 05:52 PM, Michael Evans wrote:
>>> On Thu, Feb 11, 2010 at 3:00 PM, Justin Piszcz <jpiszcz@lucidpixels.com> 
>>> wrote:
>>>> Hi,
>>>> 
>>>> I may be converting a host to ext4 and was curious, is 0.90 still the 
>>>> only
>>>> superblock version for mdadm/raid-1 that you can boot from without having 
>>>> to
>>>> create an initrd/etc?
>>>> 
>>>> Are there any benefits to using a superblock > 0.90 for a raid-1 boot 
>>>> volume
>>>> < 2TB?
>>>> 
>>>> Justin.
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>> 
>>> 
>>> You need the superblock at the end of the partition:  If you read the
>>> manual that is clearly either version 0.90 OR 1.0 (NOT 1.1 and also
>>> NOT 1.2; those use the same superblock layout but different
>>> locations).
>> 
>> 0.9 has the *serious* problem that it is hard to distinguish a whole-volume
>> 
>> However, apparently mdadm recently switched to a 1.1 default.  I
>> strongly urge Neil to change that to either 1.0 and 1.2, as I have
>> started to get complaints from users that they have made RAID volumes
>> with newer mdadm which apparently default to 1.1, and then want to boot
>> from them (without playing MBR games like Grub does.)  I have to tell
>> them that they have to regenerate their disks -- the superblock occupies
>> the boot sector and there is nothing I can do about it.  It's the same
>> pathology XFS has.
>>
>> 	-hpa
>> 
>
> My original question was does the newer superblock do anything special or 
> offer new features *BESIDES* the quicker resync?

the older superblocks have limits on the number of devices that can be 
part of the raid set.

David Lang

Re: Linux mdadm superblock question.

[Michael Evans]

On Sat, Feb 13, 2010 at 12:49 PM,  <david@lang.hm> wrote:
> On Sat, 13 Feb 2010, Justin Piszcz wrote:
>
>> On Sat, 13 Feb 2010, H. Peter Anvin wrote:
>>
>>> On 02/11/2010 05:52 PM, Michael Evans wrote:
>>>>
>>>> On Thu, Feb 11, 2010 at 3:00 PM, Justin Piszcz <jpiszcz@lucidpixels.com>
>>>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I may be converting a host to ext4 and was curious, is 0.90 still the
>>>>> only
>>>>> superblock version for mdadm/raid-1 that you can boot from without
>>>>> having to
>>>>> create an initrd/etc?
>>>>>
>>>>> Are there any benefits to using a superblock > 0.90 for a raid-1 boot
>>>>> volume
>>>>> < 2TB?
>>>>>
>>>>> Justin.
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe linux-raid"
>>>>> in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>
>>>>
>>>> You need the superblock at the end of the partition:  If you read the
>>>> manual that is clearly either version 0.90 OR 1.0 (NOT 1.1 and also
>>>> NOT 1.2; those use the same superblock layout but different
>>>> locations).
>>>
>>> 0.9 has the *serious* problem that it is hard to distinguish a
>>> whole-volume
>>>
>>> However, apparently mdadm recently switched to a 1.1 default.  I
>>> strongly urge Neil to change that to either 1.0 and 1.2, as I have
>>> started to get complaints from users that they have made RAID volumes
>>> with newer mdadm which apparently default to 1.1, and then want to boot
>>> from them (without playing MBR games like Grub does.)  I have to tell
>>> them that they have to regenerate their disks -- the superblock occupies
>>> the boot sector and there is nothing I can do about it.  It's the same
>>> pathology XFS has.
>>>
>>>        -hpa
>>>
>>
>> My original question was does the newer superblock do anything special or
>> offer new features *BESIDES* the quicker resync?
>
> the older superblocks have limits on the number of devices that can be part
> of the raid set.
>
> David Lang
>

The 1.1 and 1.2 formats ALSO play more nicely with stacking partition
contents.  LVM, filesystems, and partition info all begin at the start
of a block device; putting the md labels there too makes it obvious
what order to unpack the structures in.

Re: Linux mdadm superblock question.

[H. Peter Anvin]

On 02/13/2010 12:07 PM, Justin Piszcz wrote:
> 
> My original question was does the newer superblock do anything special
> or offer new features *BESIDES* the quicker resync?
> 

0.90 has a very bad problem, which is that it is hard to distinguish
between a RAID partition at the end of volume and a full RAID device.
This is because 0.90 doesn't actually tell you the start of the device.

Then, of course, there are a lot of limitations on size, number of
devices, and so on in 0.90.

	-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

Re: Linux mdadm superblock question.

[Asdo]

H. Peter Anvin wrote:
> On 02/13/2010 12:07 PM, Justin Piszcz wrote:
>   
> 0.90 has a very bad problem, which is that it is hard to distinguish
> between a RAID partition at the end of volume and a full RAID device.
> This is because 0.90 doesn't actually tell you the start of the device.
>
> Then, of course, there are a lot of limitations on size, number of
> devices, and so on in 0.90.
>
> 	-hpa
>   
I don't understand...
In a system we have, the root filesystem on a raid-6 which is on second 
(and last) partitions of many disks.
It always assembled correctly, it never tried to assemble the whole device.
(on the first partition there is a raid1 with boot)
So what's the problem exactly with not marking the beginning?

Re: Linux mdadm superblock question.

[H. Peter Anvin]

On 02/14/2010 12:25 PM, Asdo wrote:
> I don't understand...
> In a system we have, the root filesystem on a raid-6 which is on second
> (and last) partitions of many disks.
> It always assembled correctly, it never tried to assemble the whole device.
> (on the first partition there is a raid1 with boot)
> So what's the problem exactly with not marking the beginning?

In Fedora 12, for example, Dracut tries to make the distinction between
whole RAID device and a partition device, and utterly fails -- often
resulting in data loss.

With a pointer to the beginning this would have been a trivial thing to
detect.

IMO it would make sense to support autoassemble for 1.0 superblocks, and
making them the default.  The purpose would be to get everyone off 0.9.
 However, *any* default is better than 1.1.

	-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

Re: Linux mdadm superblock question.

[Henrique de Moraes Holschuh]

On Sun, 14 Feb 2010, H. Peter Anvin wrote:
> IMO it would make sense to support autoassemble for 1.0 superblocks, and
> making them the default.  The purpose would be to get everyone off 0.9.
>  However, *any* default is better than 1.1.

Well, FWIW, I would happily use (and recommend) 1.0 with auto-assemble
(after verifying all the emergency repair toolset in use where I work has
been upgraded to support it) in distros where the bootloader has enough of a
clue to not bork on md-1.0 devices.  Which should be most of the current
crop.

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh

Re: Linux mdadm superblock question.

[Rudy Zijlstra]

H. Peter Anvin wrote:
> In Fedora 12, for example, Dracut tries to make the distinction between
> whole RAID device and a partition device, and utterly fails -- often
> resulting in data loss.
>
>   
i do not use Fedora/redhat and do not intent to ever try them again... 
still, the point is valid
> With a pointer to the beginning this would have been a trivial thing to
> detect.
>
> IMO it would make sense to support autoassemble for 1.0 superblocks, and
> making them the default.  The purpose would be to get everyone off 0.9.
>  However, *any* default is better than 1.1.
>
> 	-hpa
>
>   
As long is autodetect is supported in the kernel, i am willing to 
upgrade to 1.0 superblocks. BUT i need the autodetect in the kernel, as 
i refuse to use initrd for production servers.

Cheers,


Rudy

Re: Linux mdadm superblock question.

[Mr. James W. Laferriere]

 	Hello All ,

On Mon, 15 Feb 2010, Rudy Zijlstra wrote:
> H. Peter Anvin wrote:
>> In Fedora 12, for example, Dracut tries to make the distinction between
>> whole RAID device and a partition device, and utterly fails -- often
>> resulting in data loss.
>> 
> i do not use Fedora/redhat and do not intent to ever try them again... still, 
> the point is valid
>> With a pointer to the beginning this would have been a trivial thing to
>> detect.
>> 
>> IMO it would make sense to support autoassemble for 1.0 superblocks, and
>> making them the default.  The purpose would be to get everyone off 0.9.
>>  However, *any* default is better than 1.1.
>> 	-hpa
> As long is autodetect is supported in the kernel, i am willing to upgrade to 
> 1.0 superblocks. BUT i need the autodetect in the kernel, as i refuse to use 
> initrd for production servers.
> Cheers,
> Rudy
 	I also have to agree with Rudy in this matter .

 		Tia ,  JimL
-- 
+------------------------------------------------------------------+
| James   W.   Laferriere | System    Techniques | Give me VMS     |
| Network&System Engineer | 3237     Holden Road |  Give me Linux  |
| babydr@baby-dragons.com | Fairbanks, AK. 99709 |   only  on  AXP |
+------------------------------------------------------------------+

Re: Linux mdadm superblock question.

[Neil Brown]

On Sat, 13 Feb 2010 11:58:03 -0800
"H. Peter Anvin" <hpa@zytor.com> wrote:

> On 02/11/2010 05:52 PM, Michael Evans wrote:
> > On Thu, Feb 11, 2010 at 3:00 PM, Justin Piszcz <jpiszcz@lucidpixels.com> wrote:
> >> Hi,
> >>
> >> I may be converting a host to ext4 and was curious, is 0.90 still the only
> >> superblock version for mdadm/raid-1 that you can boot from without having to
> >> create an initrd/etc?
> >>
> >> Are there any benefits to using a superblock > 0.90 for a raid-1 boot volume
> >> < 2TB?
> >>
> >> Justin.
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> >> the body of a message to majordomo@vger.kernel.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >>
> > 
> > You need the superblock at the end of the partition:  If you read the
> > manual that is clearly either version 0.90 OR 1.0 (NOT 1.1 and also
> > NOT 1.2; those use the same superblock layout but different
> > locations).
> 
> 0.9 has the *serious* problem that it is hard to distinguish a whole-volume
> 
> However, apparently mdadm recently switched to a 1.1 default.  I
> strongly urge Neil to change that to either 1.0 and 1.2, as I have
> started to get complaints from users that they have made RAID volumes
> with newer mdadm which apparently default to 1.1, and then want to boot
> from them (without playing MBR games like Grub does.)  I have to tell
> them that they have to regenerate their disks -- the superblock occupies
> the boot sector and there is nothing I can do about it.  It's the same
> pathology XFS has.

When mdadm defaults to 1.0 for a RAID1 it prints a warning to the effect that
the array might not be suitable to store '/boot', and requests confirmation.

So I assume that the people who are having this problem either do not read,
or are using some partitioning tool that runs mdadm under the hood using
"--run" to avoid the need for confirmation.  It would be nice to confirm if
that was the case, and find out what tool is being used.

If an array is not being used for /boot (or /) then I still think that 1.1 is
the better choice as it removes the possibility for confusion over partition
tables.

I guess I could try defaulting to 1.2 in a partition, and 1.1 on a
whole-device.  That might be a suitable compromise.

How do people cope with XFS??

NeilBrown

Re: Linux mdadm superblock question.

[Neil Brown]

On Thu, 11 Feb 2010 18:00:23 -0500 (EST)
Justin Piszcz <jpiszcz@lucidpixels.com> wrote:

> Hi,
> 
> I may be converting a host to ext4 and was curious, is 0.90 still the only 
> superblock version for mdadm/raid-1 that you can boot from without having 
> to create an initrd/etc?
> 
> Are there any benefits to using a superblock > 0.90 for a raid-1 boot 
> volume < 2TB?

The only noticeable differences that I can think of are:
 1/ If you reboot during recovery of a spare, then 0.90 will restart the
    recovery at the start, while 1.x will restart from where it was up to.
 2/ The /sys/class/block/mdXX/md/dev-YYY/errors counter is reset on each
    re-assembly with 0.90, but is preserved across stop/start with 1.x
 3/ If your partition starts on a multiple of 64K from the start of the
    device and is the last partition and contains 0.90 metadata, then
    mdadm can get confused by it.
 4/ If you move the devices to a host with a different arch and different
    byte-ordering, then extra effort will be needed to see the array for
    0.90, but not for 1.x

I suspect none of these is a big issue.

It is likely that future extensions will only be supported on 1.x metadata.
For example I hope to add support for storing a bad-block list, so that a
read error during recovery will only be fatal for that block, not the whole
recovery process.  This is unlikely ever to be supported on 0.90.  However
it may not be possible to hot-enable it on 1.x either, depending on how much
space has been reserved for extra metadata, so there is no guarantee that
using 1.x now makes you future-proof.

And yes, 0.90 is still the only superblock version that supports in-kernel
autodetect, and I have no intention of adding in-kernel autodetect for any
other version.

NeilBrown

Re: Linux mdadm superblock question.

[H. Peter Anvin]

On 02/15/2010 04:27 PM, Neil Brown wrote:
> 
> When mdadm defaults to 1.0 for a RAID1 it prints a warning to the effect that
> the array might not be suitable to store '/boot', and requests confirmation.
> 
> So I assume that the people who are having this problem either do not read,
> or are using some partitioning tool that runs mdadm under the hood using
> "--run" to avoid the need for confirmation.  It would be nice to confirm if
> that was the case, and find out what tool is being used.

My guess is that they are using the latter.  However, some of it is
probably also a matter of not planning ahead, or not understanding the
error message.  I'll forward one email privately (don't want to forward
a private email to a list.)

> If an array is not being used for /boot (or /) then I still think that 1.1 is
> the better choice as it removes the possibility for confusion over partition
> tables.
> 
> I guess I could try defaulting to 1.2 in a partition, and 1.1 on a
> whole-device.  That might be a suitable compromise.

In some ways, 1.1 is even more toxic on a whole-device, since that means
that it is physically impossible to boot off of it -- the hardware will
only ever read the first sector (MBR).

> How do people cope with XFS??

There are three options:

a) either don't boot from it (separate /boot);
b) use a bootloader which installs in the MBR and
hopefully-unpartitioned disk areas (e.g. Grub);
c) use a nonstandard custom MBR.

Neither (b) or (c), of course, allow for chainloading from another OS
install and thus are bad for interoperability.

	-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

Re: Linux mdadm superblock question.

[david]

On Mon, 15 Feb 2010, H. Peter Anvin wrote:

> On 02/15/2010 04:27 PM, Neil Brown wrote:
>
> There are three options:
>
> a) either don't boot from it (separate /boot);
> b) use a bootloader which installs in the MBR and
> hopefully-unpartitioned disk areas (e.g. Grub);
> c) use a nonstandard custom MBR.
>
> Neither (b) or (c), of course, allow for chainloading from another OS
> install and thus are bad for interoperability.

I have had no problems with XFS partitions and lilo as the bootloader. 
I've been doing this for a couple of years now without realizing that 
there is supposed to be a problem.

David Lang

Re: Linux mdadm superblock question.

[CaT]

On Thu, Feb 11, 2010 at 05:52:41PM -0800, Michael Evans wrote:
> You need the superblock at the end of the partition:  If you read the
> manual that is clearly either version 0.90 OR 1.0 (NOT 1.1 and also

For lilo, at least, this is not so:

http://www.sfr-fresh.com/linux/misc/lilo-22.8.src.tar.gz:a/lilo-22.8/raid.c

Line 145:

if (ioctl(md_fd,RAID_VERSION,&md_version_info) < 0)

Line 155:

if (ioctl(md_fd,GET_ARRAY_INFO,&md_array_info) < 0)

Lines 160-168:

if ((md_array_info.major_version != md_version_info.major) &&
	(md_array_info.minor_version != md_version_info.minor)) {
    die("Inconsistent Raid version information on %s   (RV=%d.%d GAI=%d.%d)",
        boot,
              (int)md_version_info.major,
              (int)md_version_info.minor,
              (int)md_array_info.major_version,
              (int)md_array_info.minor_version);
    }

It's 0.90 or nothing as md_version_info gives 0.90 due to:

/linux/drivers/md/md.c:
Line 4599:

        ver.major = MD_MAJOR_VERSION;
        ver.minor = MD_MINOR_VERSION;

linux/include/linux/raid/md_u.h:
Line 23:

#define MD_MAJOR_VERSION                0
#define MD_MINOR_VERSION                90

I got bitten by this as I was testing different raid superblocks on a new
setup. Wound up hand-making my own initramfs, which was a pain (right pain
to debug). Would prefer not to have one tbh.

-- 
  "A search of his car uncovered pornography, a homemade sex aid, women's 
  stockings and a Jack Russell terrier."
    - http://www.news.com.au/story/0%2C27574%2C24675808-421%2C00.html

Re: Linux mdadm superblock question.

[John Robinson]

On 16/02/2010 03:18, david@lang.hm wrote:
> On Mon, 15 Feb 2010, H. Peter Anvin wrote:
> 
>> On 02/15/2010 04:27 PM, Neil Brown wrote:
>>
>> There are three options:
>>
>> a) either don't boot from it (separate /boot);
>> b) use a bootloader which installs in the MBR and
>> hopefully-unpartitioned disk areas (e.g. Grub);
>> c) use a nonstandard custom MBR.
>>
>> Neither (b) or (c), of course, allow for chainloading from another OS
>> install and thus are bad for interoperability.
> 
> I have had no problems with XFS partitions and lilo as the bootloader. 
> I've been doing this for a couple of years now without realizing that 
> there is supposed to be a problem.

There isn't, if you use partitions. It could (would) go wrong if you 
tried to put an XFS filesystem, or md RAID with a v1.1 superblock, on a 
whole disc without a partition table *and* you tried to put a bootloader 
on. I can't say it's ever occurred to me to do that, because I always 
assumed that whatever I put in a partition used all of it, and I 
couldn't expect to double-book the beginning of it and have it work.

Cheers,

John.

Re: Linux mdadm superblock question.

[H. Peter Anvin]

On 02/15/2010 07:18 PM, david@lang.hm wrote:
> On Mon, 15 Feb 2010, H. Peter Anvin wrote:
>
>> On 02/15/2010 04:27 PM, Neil Brown wrote:
>>
>> There are three options:
>>
>> a) either don't boot from it (separate /boot);
>> b) use a bootloader which installs in the MBR and
>> hopefully-unpartitioned disk areas (e.g. Grub);
>> c) use a nonstandard custom MBR.
>>
>> Neither (b) or (c), of course, allow for chainloading from another OS
>> install and thus are bad for interoperability.
>
> I have had no problems with XFS partitions and lilo as the bootloader.
> I've been doing this for a couple of years now without realizing that
> there is supposed to be a problem.
>

LILO also can be stuffed in the MBR (and then uses block-pointers from 
there).  There is one more option that I didn't mention, which is to put 
the bootloader of a separate partition, OS/2 style.  Again, breaks the 
standard chainloading model.

	-hpa

Re: Linux mdadm superblock question.

[Rudy Zijlstra]

H. Peter Anvin wrote:
> On 02/15/2010 07:18 PM, david@lang.hm wrote:
>> On Mon, 15 Feb 2010, H. Peter Anvin wrote:
>>
>>> On 02/15/2010 04:27 PM, Neil Brown wrote:
>>>
>>> There are three options:
>>>
>>> a) either don't boot from it (separate /boot);
>>> b) use a bootloader which installs in the MBR and
>>> hopefully-unpartitioned disk areas (e.g. Grub);
>>> c) use a nonstandard custom MBR.
>>>
>>> Neither (b) or (c), of course, allow for chainloading from another OS
>>> install and thus are bad for interoperability.
>>
>> I have had no problems with XFS partitions and lilo as the bootloader.
>> I've been doing this for a couple of years now without realizing that
>> there is supposed to be a problem.
>>
>
> LILO also can be stuffed in the MBR (and then uses block-pointers from 
> there).  There is one more option that I didn't mention, which is to 
> put the bootloader of a separate partition, OS/2 style.  Again, breaks 
> the standard chainloading model.
There is another configuration that fails. Use partitioned md. I do not 
think it matters whether over whole device or with a partition table. 
Neither grub nor lilo will boot off from it. I've tested that exensively 
with a partition on the disks.  I am using PXE boot to boot 2 servers 
with that configuration. That adds a dependency on the pxe boot server, 
but considering the function of those servers they are moot if that pxe 
server is dead anyways....

Cheers,


Rudy

Re: Linux mdadm superblock question.

[Justin Piszcz]

On Tue, 16 Feb 2010, Neil Brown wrote:

> On Thu, 11 Feb 2010 18:00:23 -0500 (EST)
> Justin Piszcz <jpiszcz@lucidpixels.com> wrote:
>
>> Hi,
>>
>> I may be converting a host to ext4 and was curious, is 0.90 still the only
>> superblock version for mdadm/raid-1 that you can boot from without having
>> to create an initrd/etc?
>>
>> Are there any benefits to using a superblock > 0.90 for a raid-1 boot
>> volume < 2TB?
>
> The only noticeable differences that I can think of are:
> 1/ If you reboot during recovery of a spare, then 0.90 will restart the
>    recovery at the start, while 1.x will restart from where it was up to.
> 2/ The /sys/class/block/mdXX/md/dev-YYY/errors counter is reset on each
>    re-assembly with 0.90, but is preserved across stop/start with 1.x
> 3/ If your partition starts on a multiple of 64K from the start of the
>    device and is the last partition and contains 0.90 metadata, then
>    mdadm can get confused by it.
> 4/ If you move the devices to a host with a different arch and different
>    byte-ordering, then extra effort will be needed to see the array for
>    0.90, but not for 1.x
>
> I suspect none of these is a big issue.
>
> It is likely that future extensions will only be supported on 1.x metadata.
> For example I hope to add support for storing a bad-block list, so that a
> read error during recovery will only be fatal for that block, not the whole
> recovery process.  This is unlikely ever to be supported on 0.90.  However
> it may not be possible to hot-enable it on 1.x either, depending on how much
> space has been reserved for extra metadata, so there is no guarantee that
> using 1.x now makes you future-proof.
>
> And yes, 0.90 is still the only superblock version that supports in-kernel
> autodetect, and I have no intention of adding in-kernel autodetect for any
> other version.
>
> NeilBrown
>

Hi Neil,

Thanks for the response, this is exactly what I was looking for and 
probably should be put in a FAQ.

Justin.

Re: Linux mdadm superblock question.

[Bill Davidsen]

H. Peter Anvin wrote:
> On 02/15/2010 04:27 PM, Neil Brown wrote:
>   
>> When mdadm defaults to 1.0 for a RAID1 it prints a warning to the effect that
>> the array might not be suitable to store '/boot', and requests confirmation.
>>
>> So I assume that the people who are having this problem either do not read,
>> or are using some partitioning tool that runs mdadm under the hood using
>> "--run" to avoid the need for confirmation.  It would be nice to confirm if
>> that was the case, and find out what tool is being used.
>>     
>
> My guess is that they are using the latter.  However, some of it is
> probably also a matter of not planning ahead, or not understanding the
> error message.  I'll forward one email privately (don't want to forward
> a private email to a list.)
>
>   
>> If an array is not being used for /boot (or /) then I still think that 1.1 is
>> the better choice as it removes the possibility for confusion over partition
>> tables.
>>
>> I guess I could try defaulting to 1.2 in a partition, and 1.1 on a
>> whole-device.  That might be a suitable compromise.
>>     
>
> In some ways, 1.1 is even more toxic on a whole-device, since that means
> that it is physically impossible to boot off of it -- the hardware will
> only ever read the first sector (MBR).
>
>   
That is either a problem or a solution, depending which bad behavior you 
are trying hardest to avoid.
>> How do people cope with XFS??
>>     
>
> There are three options:
>
> a) either don't boot from it (separate /boot);
>   
And certainly there are other reasons to do that...
> b) use a bootloader which installs in the MBR and
> hopefully-unpartitioned disk areas (e.g. Grub);
> c) use a nonstandard custom MBR.
>
> Neither (b) or (c), of course, allow for chainloading from another OS
> install and thus are bad for interoperability.
>   

I'm not really sure what you're getting at here, I use grub in MBR and 
then add chain loader stanzas to grub.conf for many things, usually an 
alternate Linux release, or to have 32/64 of the same release handy for 
testing, and always memtest from the boot menu. Even Win98SP2 on one 
machine, since that works very poorly under KVM. (ask Avi if you care 
why, something about what it does in real mode). In any case, I don't 
see the chain loader issue, unless you mean to reboot out of some other 
OS into Linux.

-- 
Bill Davidsen <davidsen@tmr.com>
  "We can't solve today's problems by using the same thinking we
   used in creating them." - Einstein

Re: Linux mdadm superblock question.

[Bill Davidsen]

Neil Brown wrote:
> On Thu, 11 Feb 2010 18:00:23 -0500 (EST)
> Justin Piszcz <jpiszcz@lucidpixels.com> wrote:
>
>   
>> Hi,
>>
>> I may be converting a host to ext4 and was curious, is 0.90 still the only 
>> superblock version for mdadm/raid-1 that you can boot from without having 
>> to create an initrd/etc?
>>
>> Are there any benefits to using a superblock > 0.90 for a raid-1 boot 
>> volume < 2TB?
>>     
>
> The only noticeable differences that I can think of are:
>  1/ If you reboot during recovery of a spare, then 0.90 will restart the
>     recovery at the start, while 1.x will restart from where it was up to.
>  2/ The /sys/class/block/mdXX/md/dev-YYY/errors counter is reset on each
>     re-assembly with 0.90, but is preserved across stop/start with 1.x
>  3/ If your partition starts on a multiple of 64K from the start of the
>     device and is the last partition and contains 0.90 metadata, then
>     mdadm can get confused by it.
>   

Given that 4k sector drives make that a lot more likely that it used to 
be, I suspect some effort will be needed to address this sooner or later.

>  4/ If you move the devices to a host with a different arch and different
>     byte-ordering, then extra effort will be needed to see the array for
>     0.90, but not for 1.x
>
> I suspect none of these is a big issue.
>
> It is likely that future extensions will only be supported on 1.x metadata.
> For example I hope to add support for storing a bad-block list, so that a
> read error during recovery will only be fatal for that block, not the whole
> recovery process.  This is unlikely ever to be supported on 0.90.  However
> it may not be possible to hot-enable it on 1.x either, depending on how much
> space has been reserved for extra metadata, so there is no guarantee that
> using 1.x now makes you future-proof.
>
> And yes, 0.90 is still the only superblock version that supports in-kernel
> autodetect, and I have no intention of adding in-kernel autodetect for any
> other version.
>   


-- 
Bill Davidsen <davidsen@tmr.com>
  "We can't solve today's problems by using the same thinking we
   used in creating them." - Einstein

mdadm FAQ (was: Linux mdadm superblock question.)

[martin f krafft]

also sprach Justin Piszcz <jpiszcz@lucidpixels.com> [2010.02.17.0214 +1300]:
> Thanks for the response, this is exactly what I was looking for
> and probably should be put in a FAQ.

I'd be more than happy to push my FAQ[0], possibly fused with my
"recipes", upstream and would welcome anyone who wanted to help out.

0. http://git.debian.org/?p=pkg-mdadm/mdadm.git;a=blob;f=debian/FAQ;hb=HEAD
1. http://git.debian.org/?p=pkg-mdadm/mdadm.git;a=blob;f=debian/README.recipes;hb=HEAD

-- 
 .''`.   martin f. krafft <madduck@d.o>      Related projects:
: :'  :  proud Debian developer               http://debiansystem.info
`. `'`   http://people.debian.org/~madduck    http://vcs-pkg.org
  `-  Debian - when you have better things to do than fixing systems
 
"literature always anticipates life.
 it does not copy it, but moulds it to its purpose.
 the nineteenth century, as we know it,
 is largely an invention of balzac."
                                                        -- oscar wilde

Re: Linux mdadm superblock question.

[H. Peter Anvin]

On 02/16/2010 09:05 AM, Bill Davidsen wrote:
> 
> I'm not really sure what you're getting at here, I use grub in MBR and 
> then add chain loader stanzas to grub.conf for many things, usually an 
> alternate Linux release, or to have 32/64 of the same release handy for 
> testing, and always memtest from the boot menu. Even Win98SP2 on one 
> machine, since that works very poorly under KVM. (ask Avi if you care 
> why, something about what it does in real mode). In any case, I don't 
> see the chain loader issue, unless you mean to reboot out of some other 
> OS into Linux.
> 

That's one of many uses, yes

Presumably the reason you don't have problems is because the partitions
you chainload aren't RAID partitions with 1.1 superblocks, or you're
specifying an explicit offset for your chainloads (Grub syntax allows that.)

Either which way, it's a good example of the usage model.  Chainloading
is important for a lot of people.

	-hpa

Re: Linux mdadm superblock question.

[Neil Brown]

On Tue, 16 Feb 2010 08:14:21 -0500 (EST)
Justin Piszcz <jpiszcz@lucidpixels.com> wrote:

> 
> 
> On Tue, 16 Feb 2010, Neil Brown wrote:
> 
> > On Thu, 11 Feb 2010 18:00:23 -0500 (EST)
> > Justin Piszcz <jpiszcz@lucidpixels.com> wrote:
> >
> >> Hi,
> >>
> >> I may be converting a host to ext4 and was curious, is 0.90 still the only
> >> superblock version for mdadm/raid-1 that you can boot from without having
> >> to create an initrd/etc?
> >>
> >> Are there any benefits to using a superblock > 0.90 for a raid-1 boot
> >> volume < 2TB?
> >
> > The only noticeable differences that I can think of are:
> > 1/ If you reboot during recovery of a spare, then 0.90 will restart the
> >    recovery at the start, while 1.x will restart from where it was up to.
> > 2/ The /sys/class/block/mdXX/md/dev-YYY/errors counter is reset on each
> >    re-assembly with 0.90, but is preserved across stop/start with 1.x
> > 3/ If your partition starts on a multiple of 64K from the start of the
> >    device and is the last partition and contains 0.90 metadata, then
> >    mdadm can get confused by it.
> > 4/ If you move the devices to a host with a different arch and different
> >    byte-ordering, then extra effort will be needed to see the array for
> >    0.90, but not for 1.x
> >
> > I suspect none of these is a big issue.
> >
> > It is likely that future extensions will only be supported on 1.x metadata.
> > For example I hope to add support for storing a bad-block list, so that a
> > read error during recovery will only be fatal for that block, not the whole
> > recovery process.  This is unlikely ever to be supported on 0.90.  However
> > it may not be possible to hot-enable it on 1.x either, depending on how much
> > space has been reserved for extra metadata, so there is no guarantee that
> > using 1.x now makes you future-proof.
> >
> > And yes, 0.90 is still the only superblock version that supports in-kernel
> > autodetect, and I have no intention of adding in-kernel autodetect for any
> > other version.
> >
> > NeilBrown
> >
> 
> Hi Neil,
> 
> Thanks for the response, this is exactly what I was looking for and 
> probably should be put in a FAQ.
>
I believe the linux-raid wiki is open for anyone to update.  Feel free :-)

NeilBrown

Re: mdadm FAQ - see http://raid.wiki.kernel.org/

[David Greaves]

Hi all

I've not been active here for a long time - sorry :)

The linux raid wiki at OSDL (http://linux-raid.osdl.org/) was 'migrated' to a
drupal system during some Linux Foundation changes - clearly not suitable for
these kind of docs.

I spoke to maddog at kernel.org some months ago and we are now part of the
managed kernel wiki farm (which the osdl wiki pre-dated in case anyone wonders
why we didn't start out there).

I've asked osdl to redirect the current url to the kernel.org wiki but I think
this home should last us a while ;)

so:

hi martin..

martin f krafft wrote:
> also sprach Justin Piszcz <jpiszcz@lucidpixels.com> [2010.02.17.0214 +1300]:
>> Thanks for the response, this is exactly what I was looking for
>> and probably should be put in a FAQ.
> 
> I'd be more than happy to push my FAQ[0], possibly fused with my
> "recipes", upstream and would welcome anyone who wanted to help out.
> 
> 0. http://git.debian.org/?p=pkg-mdadm/mdadm.git;a=blob;f=debian/FAQ;hb=HEAD
> 1. http://git.debian.org/?p=pkg-mdadm/mdadm.git;a=blob;f=debian/README.recipes;hb=HEAD

See:
  http://raid.wiki.kernel.org/

David

-- 
"Don't worry, you'll be fine; I saw it work in a cartoon once..."