mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: "Robert Mueller" <robm@fastmail.fm>
To: <linux-kernel@vger.kernel.org>,
	"Reiserfs developers mail-list" <Reiserfs-Dev@namesys.com>,
	"Oleg Drokin" <green@linuxhacker.ru>,
	"Chris Mason" <mason@suse.com>
Cc: "Bron Gondwana" <brong@fastmail.fm>,
	"Jeremy Howard" <jhoward@fastmail.fm>
Subject: System lockup with processes in D state in 2.6.16.1
Date: Fri, 31 Mar 2006 08:41:38 +1000	[thread overview]
Message-ID: <13d501c6544b$23e023d0$c100a8c0@robm> (raw)

Some time ago, we were seeing a problem in the kernel in the reiserfs code 
where a lock inversion issue could cause processes to get stuck in D state, 
requiring a system reboot.

http://marc.theaimsgroup.com/?t=108932517300001&r=1&w=2

This link describes the actual call path that causes the problem.
http://marc.theaimsgroup.com/?l=linux-kernel&m=109035413201491&w=2

At the time, the solution we got was to add a patch the basically bypasses 
reiser_file_write and just calls generic_file_write. This semed to fix the 
problem and we've been running for over a year fine with that patch.

Recent we brought the issue up again with some reiser people, who mentioned 
that:

> There was a patch for the problem referenced by this link. (By Cris Mason,
> I think). This patch is long included into vanilla kernel
> (2.6.15 certainly contains it). If you still see deadlocks, I guess you 
> need
> to gather some more info again (sysrq-t and friends).

So we recently built 2.6.16.1, without the patch. However after just 1 hour 
of stress testing with cyrus again, we were able to lock up the system with 
lots of processes stuck in D state and a load running to 500+. The machine 
had about 1500 processes running on it, and the dmesg buffer was only 1M, so 
it seems we weren't able to capture all the traces with a sysrq-t, but 
there's still a lot of info. I've put the sysrq-t and kernel config output 
at the links below:

http://kernel.robm.fastmail.fm/sysrq-t-2006-03-30-1.txt
http://kernel.robm.fastmail.fm/kernel-config-2006-03-30-1.txt

Any idea if this is related to the previous problem or is something 
different?

Rob



             reply	other threads:[~2006-03-30 22:41 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2006-03-30 22:41 Robert Mueller [this message]
2006-03-31  0:58 ` Hans Reiser
2006-04-01 23:00 ` Oleg Drokin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='13d501c6544b$23e023d0$c100a8c0@robm' \
    --to=robm@fastmail.fm \
    --cc=Reiserfs-Dev@namesys.com \
    --cc=brong@fastmail.fm \
    --cc=green@linuxhacker.ru \
    --cc=jhoward@fastmail.fm \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mason@suse.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

Powered by JetHome