From: Peter Benie <peterb@chiark.greenend.org.uk>
To: linux-kernel@vger.kernel.org (linux-kernel)
Subject: Verifying a RAID device (Was: Are linux-fs's drive-fault-tolerant by concept?)
Date: Mon, 21 Apr 2003 15:11:14 +0100 [thread overview]
Message-ID: <16035.64514.811736.721619@chiark.greenend.org.uk> (raw)
In-Reply-To: <200304201725.h3KHP5lU000751@81-2-122-30.bradfords.org.uk>
John Bradford writes:
> [Chuck Ebbert writes]
> > I have some ugly code that forces all reads from a mirror set to
> > a specific copy, set via a global sysctl. This lets you do things
> > like make a backup from disk 0, then verify against disk 1 and take
> > action if something is wrong.
>
> That's interesting. Have you thought of making it read from _both_
> disks and check that the data matches, before passing it back?
>
> RAID1 mirrors guard against drive failiure, but if a drive returns bad
> data, but doesn't report an error, that will usually go unnoticed.
Checking the disks periodically guards against another important
failure mode. Consider this scenario:
Start with a RAID1 mirror using two apparently working disks.
The first disk develops a media fault, however, this goes unnoticed
because there happens to be no data stored on that part of the media.
Later, a fault is detected during a read of the second disk; the disk
is marked off off-line, but all the data is still readable on the
first disk so there's no need to panic yet.
You replace the known faulty disk with a new one. The md driver
automatically reconstructs the array from the first disk. Since the
md driver doesn't know about the filesystem, it reads every disk
block, regardless of whether it contains data or not.
The latent error on the first disk is now discovered, and the first
disk is now marked off-line. The replacement is only partially
reconstructed, so it remains inactive. Oops, there are no active disks
left - the md device has failed!
To guard against this, it is a good idea to periodically read all of
every disk in a RAID to detect faults early. Ideally, this should be
done as another background task, like reconstruction. Checking that
the data is valid is then a trivial extra step. If a read fails, you
mark the disk relevant disk off-line, as usual. If the parity check
fails, you just return a bad blocks list.
Peter
prev parent reply other threads:[~2003-04-21 13:59 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2003-04-20 17:03 Are linux-fs's drive-fault-tolerant by concept? Chuck Ebbert
2003-04-20 17:25 ` John Bradford
2003-04-21 14:11 ` Peter Benie [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=16035.64514.811736.721619@chiark.greenend.org.uk \
--to=peterb@chiark.greenend.org.uk \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®