mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Thorild Selen <thorild@Update.UU.SE>
To: linux-kernel@vger.kernel.org
Subject: Re: strange ext3 corruption problem on 2.6.x
Date: Tue, 16 Mar 2004 00:25:20 +0100	[thread overview]
Message-ID: <200403152325.i2FNPNnG006936@Tempo.Update.UU.SE> (raw)
In-Reply-To: <1zRh6-2V6-7@gated-at.bofh.it> (Marc Lehmann's message of "Mon, 15 Mar 2004 04:50:08 +0100")

Marc Lehmann <pcg@schmorp.de> writes:

> On Mon, Mar 15, 2004 at 08:59:29AM +1030, jpearson@oasissystems.com.au wrote:
>> 'r/o' by the RAID layer, presumably unbeknownst to VFS; are you
>> *sure* that your array is still up and 'good' when you get this
>> message?
>
> As I said, there are no other messages, so if there is a problem (cabling,
> disk-i/o etc.), then the kernel doesn't know it either (usually the kernel
> it quite loud in this condition).

I was able to repeat this (although with a somewhat different error
message) on a Xeon machine (HT, so "almost" SMP) running 2.6.3 (with
some IPv6 and NFS related patches, most likely nothing affecting
LVM/md/ext3). It took a few hours running bonnie++ on an ext3 fs on
LVM atop raid5 (four Hitachi SATA disks on a Promise SATA150 TX4
controller) until the machine got problems.

The kernel log says:

  Mar 15 06:34:40 Psilocybe kernel: EXT3-fs error (device dm-2):
    ext3_readdir: bad entry in directory #11: rec_len %% 4 != 0 - off
    set=0, inode=1061109567, rec_len=16191, name_len=63
  Mar 15 06:34:40 Psilocybe kernel: Aborting journal on device dm-2.
  Mar 15 06:34:40 Psilocybe kernel: EXT3-fs error (device dm-2) in
    ext3_ordered_writepage: IO failure
  Mar 15 06:34:40 Psilocybe kernel: EXT3-fs error (device dm-2) in
    ext3_ordered_writepage: IO failure
  Mar 15 06:34:41 Psilocybe kernel: ext3_abort called.
  Mar 15 06:34:41 Psilocybe kernel: EXT3-fs abort (device dm-2):
    ext3_journal_start: Detected aborted journal
  Mar 15 06:34:41 Psilocybe kernel: Remounting filesystem read-only
  Mar 15 06:34:42 Psilocybe kernel: EXT3-fs error (device dm-2) in
    start_transaction: Journal has aborted

And the last words from bonnie++ were:

  Writing intelligently...Can't write block.
  Bonnie: drastic I/O error (write(2)): No such file or directory

Then bonnie exited. It seems like something unrelated to this fs was
in an inconsistent state at this stage, as the machine crashed some
hours later.

The last syslog entry before the crash was at 11:05:01, then the
machine crashed quietly. The console was blank; the machine still
responded to pings, but appeared otherwise dead. The arrays were not
clean and were reconstructed at boot, also arrays that were not
involved in running the benchmark.

I can try to repeat the experiment with another fs if that is desired,
but people seem to agree already that the problem is in ext3. Any
suggestions on how to continue?


Thorild Selén
Datorföreningen Update / Update Computer Club, Uppsala, SE

       reply	other threads:[~2004-03-15 23:27 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <1zRh6-2V6-9@gated-at.bofh.it>
     [not found] ` <1zRh6-2V6-7@gated-at.bofh.it>
2004-03-15 23:25   ` Thorild Selen [this message]
2004-03-23  7:33 John Pearson
2004-03-23  7:54 ` Andrew D Kirch
     [not found] <20040314222929.GA23106@mark>
2004-03-15  3:41 ` Marc Lehmann
2004-03-18  7:25   ` Pavel Machek
  -- strict thread matches above, loose matches on Subject: below --
2004-03-13  0:47 Marc Lehmann
2004-03-13  2:34 ` Andrew Morton
2004-03-13  2:40   ` Marc Singer

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=200403152325.i2FNPNnG006936@Tempo.Update.UU.SE \
    --to=thorild@update.uu.se \
    --cc=linux-kernel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®