* 2.6.39, 3.0.0-rc7+: complete system freeze with rsync on big directory
@ 2011-07-21 9:19 Michael Brade
2011-07-21 9:28 ` Lars Täuber
0 siblings, 1 reply; 6+ messages in thread
From: Michael Brade @ 2011-07-21 9:19 UTC (permalink / raw)
To: linux-kernel
Hi,
this is perfectly reproducible: with vanilla 2.6.39.x and 3.0.0 from
yesterday's git my system completely freezes after a minute or so when I do
rsync -cavSH -n /source/ /dest/
Watching htop, it freezes only after the caches are full but not immediately
when the caches are full. No Alt-SysRq-magic is working anymore, nothing
responds at all after the freeze. This is on tty1, no X server running.
Note the -n which means dry run. I also mounted both directories read-only, so
it's not the write commands. I am using XFS on both directories, one is
mounted with iscsi, the other is local and encrypted with loop-aes. The
directory size is >20 GB, 4000-6000 files. I first noticed it when a
cp -a /source /dest froze the system.
When I go back to 2.6.37.6 it works fine. I didn't try 2.6.38 yet.
I am running this on a Dell Vostro, amd64, 6 GB RAM, 8x Intel Core i7 CPU Q
740 @ 1.73GHz.
Any ideas how to debug this despite the freeze? Do you need my kernel config?
Do you need any other info? Please CC me.
thanks,
Michael
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: 2.6.39, 3.0.0-rc7+: complete system freeze with rsync on big directory
2011-07-21 9:19 2.6.39, 3.0.0-rc7+: complete system freeze with rsync on big directory Michael Brade
@ 2011-07-21 9:28 ` Lars Täuber
2011-07-21 18:13 ` Michael Brade
2011-07-21 19:40 ` Michael Brade
0 siblings, 2 replies; 6+ messages in thread
From: Lars Täuber @ 2011-07-21 9:28 UTC (permalink / raw)
To: Michael Brade; +Cc: linux-kernel
Hi Michael,
I'm just curious, because I have troubles with storage IO too.
Am Thu, 21 Jul 2011 11:19:15 +0200
Michael Brade <brade@informatik.uni-muenchen.de> schrieb:
> Hi,
>
> this is perfectly reproducible: with vanilla 2.6.39.x and 3.0.0 from
> yesterday's git my system completely freezes after a minute or so when I do
>
> rsync -cavSH -n /source/ /dest/
>
> Watching htop, it freezes only after the caches are full but not
> immediately when the caches are full. No Alt-SysRq-magic is working
> anymore, nothing responds at all after the freeze. This is on tty1, no X
> server running.
>
> Note the -n which means dry run. I also mounted both directories read-only,
> so it's not the write commands. I am using XFS on both directories, one is
> mounted with iscsi, the other is local and encrypted with loop-aes. The
> directory size is >20 GB, 4000-6000 files. I first noticed it when a
> cp -a /source /dest froze the system.
>
> When I go back to 2.6.37.6 it works fine. I didn't try 2.6.38 yet.
>
> I am running this on a Dell Vostro, amd64, 6 GB RAM, 8x Intel Core i7 CPU Q
> 740 @ 1.73GHz.
>
> Any ideas how to debug this despite the freeze? Do you need my kernel
> config? Do you need any other info? Please CC me.
Could you try to capture the log via serial console?
(linux/Documentation/serial-console.txt)
What kind of disks/controller/driver do you use?
Lars
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: 2.6.39, 3.0.0-rc7+: complete system freeze with rsync on big directory
2011-07-21 9:28 ` Lars Täuber
@ 2011-07-21 18:13 ` Michael Brade
2011-08-13 22:17 ` [SOLVED: r8169 is the culprit] " Michael Brade
2011-07-21 19:40 ` Michael Brade
1 sibling, 1 reply; 6+ messages in thread
From: Michael Brade @ 2011-07-21 18:13 UTC (permalink / raw)
To: linux-kernel; +Cc: Lars Täuber
Hi Lars,
> > Any ideas how to debug this despite the freeze? Do you need my kernel
> > config? Do you need any other info? Please CC me.
>
> Could you try to capture the log via serial console?
> (linux/Documentation/serial-console.txt)
Hm. I don't have a serial port on my laptop and I don't have a USB dongle, so
I don't know how to do this. And really, when it freezes, everything freezes.
Even the HDD light just stops right where it is - either on of off all the
time. None of the buttons work anymore, the CPU fan keeps going the speed it
had before, no network responses anymore. Only holding the power button for 4
seconds switches it off.
But watching the normal console I get loads of
r8169 0000:13:00.0: eth0: link up
r8169 0000:13:00.0: eth0: link up
[...]
one message about every 1-2 seconds while rsync is running on 2.6.37.6.
On 2.6.39 and 3.0.0-rc7 I get about 2-3 of those per second - until it
freezes. The source partition is iSCSI so it does use the net.
> What kind of disks/controller/driver do you use?
Source HD is iSCSI over ethernet. I am using open-iscsi on the client.
Dest HD is on the laptop:
block device: bus = scsi, bus_id = 0:0:0:0 driver = sd
model = WDC WD5000BEKT-7
Driver: "ahci", "sd"
SATA controller
lspci:
00:1f.2 SATA controller: Intel Corporation 5 Series/3400 Series Chipset 6 port
SATA AHCI Controller (rev 06)
13:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI
Express Gigabit Ethernet controller (rev 03)
I guess my .config could also be helpful - if so, do you need all of it, part
of it, and how should I post this? Attach it packed with 7z?
cheers,
Michael
^ permalink raw reply [flat|nested] 6+ messages in thread
* [SOLVED: r8169 is the culprit] Re: 2.6.39, 3.0.0-rc7+: complete system freeze with rsync on big directory
2011-07-21 18:13 ` Michael Brade
@ 2011-08-13 22:17 ` Michael Brade
2011-08-13 22:49 ` David Miller
0 siblings, 1 reply; 6+ messages in thread
From: Michael Brade @ 2011-08-13 22:17 UTC (permalink / raw)
To: linux-kernel
Hi,
finally I found the reason for my system freeze: the r8169 module. The freeze
happens only on heavy network loads, i.e. > 70 MB/s. iSCSI is really fast,
that's why I only saw the freezes with iSCSI.
> But watching the normal console I get loads of
>
> r8169 0000:13:00.0: eth0: link up
> r8169 0000:13:00.0: eth0: link up
This does not happen with 2.6.32.28, and the system doesn't freeze either with
this kernel. So the r8169 is fine on 2.6.32.x. 2.6.37 and above freeze the
system. I haven't tried any kernel in between .32 and .37.
> [...]
>
> lspci:
>
> 13:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B
> PCI Express Gigabit Ethernet controller (rev 03)
The solution with 2.6.37 and above: use the r8168 module from the realtek
website. I have tested it with >30 GB at rates of 112 MB/s and experienced no
freeze anymore.
I don't know if you are interested in fixing r8169 but if so and you need
anything, let me know how I can help.
cheers,
Michael
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [SOLVED: r8169 is the culprit] Re: 2.6.39, 3.0.0-rc7+: complete system freeze with rsync on big directory
2011-08-13 22:17 ` [SOLVED: r8169 is the culprit] " Michael Brade
@ 2011-08-13 22:49 ` David Miller
0 siblings, 0 replies; 6+ messages in thread
From: David Miller @ 2011-08-13 22:49 UTC (permalink / raw)
To: brade; +Cc: linux-kernel
From: Michael Brade <brade@informatik.uni-muenchen.de>
Date: Sun, 14 Aug 2011 00:17:30 +0200
> I don't know if you are interested in fixing r8169 but if so and you need
> anything, let me know how I can help.
The first thing you need to do is send your report to
netdev@vger.kernel.org and the listed r8169 maintainers
(nic_swsd@realtek.com and romieu@fr.zoreil.com), nobody on
linux-kernel will be able to fix this.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: 2.6.39, 3.0.0-rc7+: complete system freeze with rsync on big directory
2011-07-21 9:28 ` Lars Täuber
2011-07-21 18:13 ` Michael Brade
@ 2011-07-21 19:40 ` Michael Brade
1 sibling, 0 replies; 6+ messages in thread
From: Michael Brade @ 2011-07-21 19:40 UTC (permalink / raw)
To: linux-kernel
oh dear... this isn't true:
> > When I go back to 2.6.37.6 it works fine. I didn't try 2.6.38 yet.
I just froze the system three times in a row on 2.6.37.6 as well. However, it
seems it takes longer, since I was able to copy 32 GB and 26 GB in two of
those attempts until it froze.
-- Michael
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2011-08-13 22:50 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-07-21 9:19 2.6.39, 3.0.0-rc7+: complete system freeze with rsync on big directory Michael Brade
2011-07-21 9:28 ` Lars Täuber
2011-07-21 18:13 ` Michael Brade
2011-08-13 22:17 ` [SOLVED: r8169 is the culprit] " Michael Brade
2011-08-13 22:49 ` David Miller
2011-07-21 19:40 ` Michael Brade
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®