mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* 99% System load
@ 2004-02-11 14:22 Florian Schanda
  2004-02-11 14:30 ` Florian Schanda
                   ` (2 more replies)
  0 siblings, 3 replies; 6+ messages in thread
From: Florian Schanda @ 2004-02-11 14:22 UTC (permalink / raw)
  To: Kernel Mailing List

[-- Attachment #1: Type: Text/Plain, Size: 1926 bytes --]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello,

I have experianced a strage problem with recent 2.6.{0,1,2} kernels:

Sometimes (not allways, thats the problem, otherwise I could just avoid the 
offending command) during paralel builds (for instance kde-libs or kde-base)
the build just stops, and I get a 99% system load.

For instance, just now I have (from top):

Tasks:  89 total,   5 running,  84 sleeping,   0 stopped,   0 zombie
 Cpu0 :  0.3% us, 99.7% sy,  0.0% ni,  0.0% id,  0.0% wa,  0.0% hi,  0.0% si
 Cpu1 :  0.3% us, 99.7% sy,  0.0% ni,  0.0% id,  0.0% wa,  0.0% hi,  0.0% si
 Cpu2 :  1.3% us, 98.7% sy,  0.0% ni,  0.0% id,  0.0% wa,  0.0% hi,  0.0% si
 Cpu3 :  1.0% us, 99.0% sy,  0.0% ni,  0.0% id,  0.0% wa,  0.0% hi,  0.0% si
Mem:   2071872k total,  1220772k used,   851100k free,    48080k buffers
Swap:   401616k total,        0k used,   401616k free,   892428k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 5197 ma1flfs   25   0  2256  884 2184 R 99.5  0.0   6:29.38 sh
 5199 ma1flfs   25   0  2260  888 2184 R 99.5  0.0   6:28.02 sh
 5201 ma1flfs   25   0  2260  884 2184 R 98.5  0.0   6:29.99 sh
 5208 ma1flfs   25   0  1560  440 1404 R 98.5  0.0   6:32.03 grep
<snip>

I cannot kill -9 those processes. Is there some way to get rid of them without 
a reboot?

The strange thing is, resuming the build after the reboot with exactly the 
same command (make -j4) will work fine... It's just not reproducible at all, 
but it happend often enought to make it annoying.

(BTW, this never happened with non-paralel builds)

I have no idea where to start looking... I have attached a "ps -axfu" in case 
that helps.

	Florian Schanda
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)

iD8DBQFAKjq5fCf8muQVS4cRApTDAJ9firqIMAEMYC0sWmtEYtpA+s8tKwCfXj8W
DKkrA9MxZxKXhoJcgxAuiL0=
=fV1B
-----END PGP SIGNATURE-----

[-- Attachment #2: proclist --]
[-- Type: text/plain, Size: 9572 bytes --]

USER       PID %CPU %MEM   VSZ  RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0  1484  492 ?        S    12:03   0:04 init [5]  
root         2  0.0  0.0     0    0 ?        SW   12:03   0:00 [migration/0]
root         3  0.0  0.0     0    0 ?        SWN  12:03   0:00 [ksoftirqd/0]
root         4  0.0  0.0     0    0 ?        SW   12:03   0:00 [migration/1]
root         5  0.0  0.0     0    0 ?        SWN  12:03   0:00 [ksoftirqd/1]
root         6  0.0  0.0     0    0 ?        SW   12:03   0:00 [migration/2]
root         7  0.0  0.0     0    0 ?        SWN  12:03   0:00 [ksoftirqd/2]
root         8  0.0  0.0     0    0 ?        SW   12:03   0:00 [migration/3]
root         9  0.0  0.0     0    0 ?        SWN  12:03   0:00 [ksoftirqd/3]
root        10  0.0  0.0     0    0 ?        SW<  12:03   0:00 [events/0]
root        11  0.0  0.0     0    0 ?        SW<  12:03   0:00 [events/1]
root        12  0.0  0.0     0    0 ?        SW<  12:03   0:00 [events/2]
root        13  0.0  0.0     0    0 ?        SW<  12:03   0:00 [events/3]
root        14  0.0  0.0     0    0 ?        SW<  12:03   0:00 [kblockd/0]
root        15  0.0  0.0     0    0 ?        SW<  12:03   0:00 [kblockd/1]
root        16  0.0  0.0     0    0 ?        SW<  12:03   0:00 [kblockd/2]
root        17  0.0  0.0     0    0 ?        SW<  12:03   0:00 [kblockd/3]
root        18  0.0  0.0     0    0 ?        SW   12:03   0:00 [khubd]
root        22  0.0  0.0     0    0 ?        SW   12:03   0:00 [kswapd0]
root        21  0.0  0.0     0    0 ?        SW   12:03   0:00 [pdflush]
root        19  0.0  0.0     0    0 ?        SW   12:03   0:00 [kirqd]
root        20  0.0  0.0     0    0 ?        SW   12:03   0:02 [pdflush]
root        23  0.0  0.0     0    0 ?        SW<  12:03   0:00 [aio/0]
root        24  0.0  0.0     0    0 ?        SW<  12:03   0:00 [aio/1]
root        25  0.0  0.0     0    0 ?        SW<  12:03   0:00 [aio/2]
root        26  0.0  0.0     0    0 ?        SW<  12:03   0:00 [aio/3]
root        27  0.0  0.0     0    0 ?        SW<  12:03   0:01 [xfslogd/0]
root        28  0.0  0.0     0    0 ?        SW<  12:03   0:00 [xfslogd/1]
root        29  0.0  0.0     0    0 ?        SW<  12:03   0:00 [xfslogd/2]
root        30  0.0  0.0     0    0 ?        SW<  12:03   0:00 [xfslogd/3]
root        31  0.0  0.0     0    0 ?        SW<  12:03   0:00 [xfsdatad/0]
root        32  0.0  0.0     0    0 ?        SW<  12:03   0:00 [xfsdatad/1]
root        33  0.0  0.0     0    0 ?        SW<  12:03   0:00 [xfsdatad/2]
root        34  0.0  0.0     0    0 ?        SW<  12:03   0:00 [xfsdatad/3]
root        35  0.0  0.0     0    0 ?        SW   12:03   0:00 [pagebufd]
root        36  0.0  0.0     0    0 ?        SW   12:03   0:00 [scsi_eh_0]
root        37  0.0  0.0     0    0 ?        SW   12:03   0:00 [ahc_dv_0]
root        38  0.0  0.0     0    0 ?        SW   12:03   0:00 [kseriod]
root        39  0.0  0.0     0    0 ?        SW   12:03   0:00 [xfs_syncd]
root        85  0.0  0.0     0    0 ?        SW   12:03   0:00 [xfs_syncd]
root       140  0.0  0.0  1536  596 ?        S    12:03   0:00 syslogd -m 0
root       143  0.0  0.0  2252 1344 ?        S    12:03   0:00 klogd
#1         166  0.0  0.0  1600  432 ?        S    12:03   0:00 /sbin/portmap
nobody     172  0.0  0.0  1660  716 ?        S    12:03   0:00 rpc.statd
root       178  0.0  0.0     0    0 ?        SW   12:03   0:00 [rpciod]
root       179  0.0  0.0     0    0 ?        SW   12:03   0:00 [lockd]
root       185  0.0  0.0  3116 1344 ?        S    12:03   0:00 /usr/sbin/sshd -p 50000
root       198  0.0  0.0  2808  736 ?        S    12:03   0:00 /opt/kde3/bin/kdm
root       206  1.4  0.8 281556 18176 ?      SL   12:03   1:59  \_ /usr/X11R6/bin/X vt7 -auth /var/run/xauth/A:0-AoIuu1
root       207  0.0  0.0  2972 1080 ?        S    12:03   0:00  \_ -:0              
ma1flfs    219  0.0  0.0  2280 1052 ?        S    12:04   0:00      \_ /bin/sh /opt/kde3/bin/startkde
ma1flfs    286  0.0  0.0  1480  328 ?        S    12:04   0:00          \_ kwrapper ksmserver
root       200  0.0  0.0  1476  416 tty1     S    12:03   0:00 /sbin/agetty vc/1 9600
root       201  0.0  0.0  1480  420 tty2     S    12:03   0:00 /sbin/agetty vc/2 9600
root       202  0.0  0.0  1480  420 tty3     S    12:03   0:00 /sbin/agetty vc/3 9600
root       203  0.0  0.0  1480  420 tty4     S    12:03   0:00 /sbin/agetty vc/4 9600
root       204  0.0  0.0  1480  420 tty5     S    12:03   0:00 /sbin/agetty vc/5 9600
root       205  0.0  0.0  1480  420 tty6     S    12:03   0:00 /sbin/agetty vc/6 9600
ma1flfs    240  0.0  0.4 21804 9836 ?        S    12:04   0:00 kdeinit: Running...      
ma1flfs    245  0.0  0.5 23148 11280 ?       S    12:04   0:00  \_ kdeinit: klauncher       
ma1flfs    280  1.3  0.5 40528 11536 ?       S    12:04   1:46  \_ /opt/kde3/bin/artsd -F 10 -S 4096 -a alsa -b 16 -s 60 -m artsmessage -c drkonqi -l 3 -f
ma1flfs    289  0.0  0.7 24656 14552 ?       S    12:04   0:07  \_ kdeinit: kwin            
ma1flfs    330  0.0  0.5 24624 12124 ?       S    12:06   0:01  \_ kdeinit: kio_pop3 pop3s /tmp/ksocket-ma1flfs/klauncherFD5c2b.slave-socket /tmp/ksocket-ma1flfs/kmailzuxKna.slave-socket
ma1flfs   5211  0.0  0.5 22744 10364 ?       S    14:03   0:00  \_ kdeinit: kio_file file /tmp/ksocket-ma1flfs/klauncherFD5c2b.slave-socket /tmp/ksocket-ma1flfs/kateCddIFa.slave-socket
ma1flfs   5249  0.1  0.7 26540 15656 ?       S    14:09   0:00  \_ kdeinit: konsole         
ma1flfs   5250  0.0  0.0  2412 1356 pts/1    R    14:09   0:00  |   \_ /bin/bash
ma1flfs   5263  0.0  0.0  2316  764 pts/1    R    14:17   0:00  |       \_ ps axfu
ma1flfs   5261  0.0  0.5 22752 10432 ?       S    14:17   0:00  \_ kdeinit: kio_file file /tmp/ksocket-ma1flfs/klauncherFD5c2b.slave-socket /tmp/ksocket-ma1flfs/kateYyJMVa.slave-socket
ma1flfs    243  0.0  0.4 21580 9900 ?        S    12:04   0:01 kdeinit: dcopserver --nosid
ma1flfs    248  0.0  0.7 26956 15800 ?       S    12:04   0:02 kdeinit: kded            
ma1flfs    255  0.0  0.6 24316 14040 ?       S    12:04   0:00 kdeinit: kxkb            
ma1flfs    282  0.0  0.8 29380 17656 ?       S    12:04   0:00 kdeinit: knotify         
ma1flfs    288  0.0  0.6 23164 12720 ?       S    12:04   0:00 kdeinit: ksmserver       
ma1flfs    291  0.0  0.7 25612 16160 ?       S    12:04   0:03 kdeinit: kdesktop        
ma1flfs    293  0.4  0.8 26796 17404 ?       S    12:04   0:32 kdeinit: kicker          
ma1flfs    296  0.0  0.0  1688  728 ?        S    12:04   0:00  \_ ksysguardd
ma1flfs    300  0.0  0.6 24184 13216 ?       S    12:04   0:00 kalarmd --login
ma1flfs    301  0.0  0.7 26320 15656 ?       S    12:04   0:00 kgpg
ma1flfs    302  0.0  0.7 26976 16044 ?       S    12:04   0:00 kalarm --tray
ma1flfs    311  0.0  0.6 24888 14460 ?       S    12:05   0:00 kdeinit: kio_uiserver    
ma1flfs    315  0.0  0.6 23488 13744 ?       S    12:05   0:00 kwalletmanager
ma1flfs    321  0.6  1.1 34464 23852 ?       S    12:05   0:48 juk -caption JuK -icon juk.png -miniicon juk.png
ma1flfs   5197 98.5  0.0  2256  884 ?        R    14:03  14:36 /bin/sh ../../libtool --silent --mode=compile --tag=CXX g++ -DHAVE_CONFIG_H -I. -I. -I../.. -I/opt/kde3/include -I/opt/qt/include -I/usr/X11R6/include -DQT_THREAD_SUPPORT -D_REENTRANT -Wnon-virtual-dtor -Wno-long-long -Wundef -Wall -W -Wpointer-arith -Wwrite-strings -ansi -D_XOPEN_SOURCE=500 -D_BSD_SOURCE -Wcast-align -Wconversion -Wchar-subscripts -DNDEBUG -DNO_DEBUG -O2 -O3 -march=pentium4 -mcpu=pentium4 -Wformat-security -Wmissing-format-attribute -fno-exceptions -fno-check-new -fno-common -DQT_CLEAN_NAMESPACE -DQT_NO_ASCII_CAST -DQT_NO_STL -DQT_NO_COMPAT -DQT_NO_TRANSLATION -D_GNU_SOURCE -MT kcmlocale.lo -MD -MP -MF .deps/kcmlocale.Tpo -c -o kcmlocale.lo kcmlocale.cpp
ma1flfs   5199 97.8  0.0  2260  888 ?        R    14:03  14:30 /bin/sh ../../libtool --silent --mode=compile --tag=CXX g++ -DHAVE_CONFIG_H -I. -I. -I../.. -I/opt/kde3/include -I/opt/qt/include -I/usr/X11R6/include -DQT_THREAD_SUPPORT -D_REENTRANT -Wnon-virtual-dtor -Wno-long-long -Wundef -Wall -W -Wpointer-arith -Wwrite-strings -ansi -D_XOPEN_SOURCE=500 -D_BSD_SOURCE -Wcast-align -Wconversion -Wchar-subscripts -DNDEBUG -DNO_DEBUG -O2 -O3 -march=pentium4 -mcpu=pentium4 -Wformat-security -Wmissing-format-attribute -fno-exceptions -fno-check-new -fno-common -DQT_CLEAN_NAMESPACE -DQT_NO_ASCII_CAST -DQT_NO_STL -DQT_NO_COMPAT -DQT_NO_TRANSLATION -D_GNU_SOURCE -MT toplevel.lo -MD -MP -MF .deps/toplevel.Tpo -c -o toplevel.lo toplevel.cpp
ma1flfs   5201 97.9  0.0  2260  884 ?        R    14:03  14:31 /bin/sh ../../libtool --silent --mode=compile --tag=CXX g++ -DHAVE_CONFIG_H -I. -I. -I../.. -I/opt/kde3/include -I/opt/qt/include -I/usr/X11R6/include -DQT_THREAD_SUPPORT -D_REENTRANT -Wnon-virtual-dtor -Wno-long-long -Wundef -Wall -W -Wpointer-arith -Wwrite-strings -ansi -D_XOPEN_SOURCE=500 -D_BSD_SOURCE -Wcast-align -Wconversion -Wchar-subscripts -DNDEBUG -DNO_DEBUG -O2 -O3 -march=pentium4 -mcpu=pentium4 -Wformat-security -Wmissing-format-attribute -fno-exceptions -fno-check-new -fno-common -DQT_CLEAN_NAMESPACE -DQT_NO_ASCII_CAST -DQT_NO_STL -DQT_NO_COMPAT -DQT_NO_TRANSLATION -D_GNU_SOURCE -MT klanguagebutton.lo -MD -MP -MF .deps/klanguagebutton.Tpo -c -o klanguagebutton.lo klanguagebutton.cpp
ma1flfs   5208 98.2  0.0  1560  440 ?        R    14:03  14:33 grep ^# ### BEGIN LIBTOOL TAG CONFIG: CXX$
ma1flfs   5246  1.8  4.0 97124 84304 ?       S    14:08   0:09 kmail -caption KMail -icon kmail -miniicon kmail
ma1flfs   5260  3.3  1.0 30200 20724 pts/1   S    14:17   0:01 kate proclist

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: 99% System load
  2004-02-11 14:22 99% System load Florian Schanda
@ 2004-02-11 14:30 ` Florian Schanda
  2004-02-11 19:02 ` Michel Angelo da Silva Pereira
  2004-02-12  7:44 ` Andrew Morton
  2 siblings, 0 replies; 6+ messages in thread
From: Florian Schanda @ 2004-02-11 14:30 UTC (permalink / raw)
  To: Kernel Mailing List

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wednesday February 11 2004 14:22, Florian Schanda wrote:
> The strange thing is, resuming the build after the reboot with exactly the
> same command (make -j4) will work fine... It's just not reproducible at
> all, but it happend often enought to make it annoying.

To add to this, if I do a simple "make" just now (without having rebooted) I 
end up with another unkillable process.

	Florian Schanda
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)

iD8DBQFAKjx3fCf8muQVS4cRAky8AKCdColeD7zjMnhpO5DINGDr2unQygCfe4hX
zm83dOBXU8QphGm6JYuLAic=
=wh/b
-----END PGP SIGNATURE-----

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: 99% System load
  2004-02-11 14:22 99% System load Florian Schanda
  2004-02-11 14:30 ` Florian Schanda
@ 2004-02-11 19:02 ` Michel Angelo da Silva Pereira
  2004-02-12  7:44 ` Andrew Morton
  2 siblings, 0 replies; 6+ messages in thread
From: Michel Angelo da Silva Pereira @ 2004-02-11 19:02 UTC (permalink / raw)
  To: linux-kernel

No dia 11-02-2004 12:22, Florian Schanda escreveu:
> Hello,
> 
> I have experianced a strage problem with recent 2.6.{0,1,2} kernels:
> 
> Sometimes (not allways, thats the problem, otherwise I could just avoid the 
> offending command) during paralel builds (for instance kde-libs or kde-base)
> the build just stops, and I get a 99% system load.

	I'm getting this kind of problem when I start to move a lot of emails 
in Mozilla to another Directory.
	The system go to a "Slow State" and I lost the keyboard, cannot run any 
program and only Alt+SysRq+(S,U,B) works.

-- 
===================================================================
Especialista em Segurança da Informação
Solaris Certified System Administrator - Conectiva Linux Specialist
Mondonet Network Solutions - www.mondonet.com.br - (016) 3966-3435
===================================================================

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: 99% System load
  2004-02-11 14:22 99% System load Florian Schanda
  2004-02-11 14:30 ` Florian Schanda
  2004-02-11 19:02 ` Michel Angelo da Silva Pereira
@ 2004-02-12  7:44 ` Andrew Morton
  2004-02-17 21:16   ` Florian Schanda
  2 siblings, 1 reply; 6+ messages in thread
From: Andrew Morton @ 2004-02-12  7:44 UTC (permalink / raw)
  To: Florian Schanda; +Cc: linux-kernel

Florian Schanda <ma1flfs@bath.ac.uk> wrote:
>
> Sometimes (not allways, thats the problem, otherwise I could just avoid the 
>  offending command) during paralel builds (for instance kde-libs or kde-base)
>  the build just stops, and I get a 99% system load.
> 
>  For instance, just now I have (from top):
> 
>  Tasks:  89 total,   5 running,  84 sleeping,   0 stopped,   0 zombie
>   Cpu0 :  0.3% us, 99.7% sy,  0.0% ni,  0.0% id,  0.0% wa,  0.0% hi,  0.0% si
>   Cpu1 :  0.3% us, 99.7% sy,  0.0% ni,  0.0% id,  0.0% wa,  0.0% hi,  0.0% si
>   Cpu2 :  1.3% us, 98.7% sy,  0.0% ni,  0.0% id,  0.0% wa,  0.0% hi,  0.0% si
>   Cpu3 :  1.0% us, 99.0% sy,  0.0% ni,  0.0% id,  0.0% wa,  0.0% hi,  0.0% si
>  Mem:   2071872k total,  1220772k used,   851100k free,    48080k buffers
>  Swap:   401616k total,        0k used,   401616k free,   892428k cached
> 
>    PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
>   5197 ma1flfs   25   0  2256  884 2184 R 99.5  0.0   6:29.38 sh
>   5199 ma1flfs   25   0  2260  888 2184 R 99.5  0.0   6:28.02 sh
>   5201 ma1flfs   25   0  2260  884 2184 R 98.5  0.0   6:29.99 sh
>   5208 ma1flfs   25   0  1560  440 1404 R 98.5  0.0   6:32.03 grep

Make sure that you have enabled CONFIG_KALLSYMS, and that
/proc/sys/kernel/sysrq is set to one.  Then, when it hangs, do

	echo t > /proc/sysrq-trigger

or type alt-sysrq-t

Then, run

	dmesg -s 1000000 > /tmp/foo

and make `foo' available.

It would help if you could make a note of the PIDs of the hung processes,
so they can be correlated with the sysrq output.

Thanks.


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: 99% System load
  2004-02-12  7:44 ` Andrew Morton
@ 2004-02-17 21:16   ` Florian Schanda
  2004-02-17 21:43     ` Andrew Morton
  0 siblings, 1 reply; 6+ messages in thread
From: Florian Schanda @ 2004-02-17 21:16 UTC (permalink / raw)
  To: Andrew Morton; +Cc: linux-kernel

[-- Attachment #1: Type: Text/Plain, Size: 2432 bytes --]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thursday 12 February 2004 07:44, Andrew Morton wrote:
> Florian Schanda <ma1flfs@bath.ac.uk> wrote:
> > Sometimes (not allways, thats the problem, otherwise I could just avoid
> > the offending command) during paralel builds (for instance kde-libs or
> > kde-base) the build just stops, and I get a 99% system load.
> >
> Make sure that you have enabled CONFIG_KALLSYMS, and that
> /proc/sys/kernel/sysrq is set to one.  Then, when it hangs, do
>
> 	echo t > /proc/sysrq-trigger
>
> or type alt-sysrq-t
>
> Then, run
>
> 	dmesg -s 1000000 > /tmp/foo
>
> and make `foo' available.

Finally it happened again. I attached the output of that dmesg.

> It would help if you could make a note of the PIDs of the hung processes,
> so they can be correlated with the sysrq output.

The PIDs hanging are 3303, 3305, 3306, 3307.

An advance warning: I had the nvidia driver (yes, I know, closed source makes 
debugging impossible) loaded, and before the trace begins, there are some 
messages of that module:

1: NVRM: AGPGART: unable to retrieve symbol table
atkbd.c: Unknown key released (translated set 2, code 0x7a on isa0060/serio0).
atkbd.c: This is an XFree86 bug. It shouldn't access hardware directly.
atkbd.c: Unknown key released (translated set 2, code 0x7a on isa0060/serio0).
atkbd.c: This is an XFree86 bug. It shouldn't access hardware directly.
request_module: failed /sbin/modprobe -- block-major-3-0. error = 256
3: nvidia: trying to map 0xef9f1000 to kernel space, but we're in an interrupt 
or holding a spinlock
2: nvidia: trying to map 0xef9f1000 to kernel space, but we're in an interrupt 
or holding a spinlock
3: nvidia: trying to map 0xef9f1000 to kernel space, but we're in an interrupt 
or holding a spinlock
1: nvidia: trying to map 0xef9f1000 to kernel space, but we're in an interrupt 
or holding a spinlock
2: nvidia: trying to map 0xef9f1000 to kernel space, but we're in an interrupt 
or holding a spinlock
2: nvidia: trying to map 0xef9f1000 to kernel space, but we're in an interrupt 
or holding a spinlock

- From what I understand is I get some kind of deadlock, right?

Thanks in advance,

	Florian
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFAMoSbfCf8muQVS4cRAuvPAJ4upsbqfGCuP6NgrR1wtgH99S8R7ACdG9yh
GHbC6dZIwX65HDp7sFPg3sQ=
=sy7G
-----END PGP SIGNATURE-----

[-- Attachment #2: dmesg-output --]
[-- Type: text/plain, Size: 61483 bytes --]

ive:1)
00:00:1d[B] -> 2-19 -> IRQ 19
Pin 2-18 already programmed
IOAPIC[0]: Set PCI routing entry (2-23 -> 0xc9 -> IRQ 23 Mode:1 Active:1)
00:00:1d[D] -> 2-23 -> IRQ 23
Pin 2-16 already programmed
Pin 2-17 already programmed
Pin 2-18 already programmed
Pin 2-19 already programmed
Pin 2-17 already programmed
Pin 2-18 already programmed
Pin 2-19 already programmed
Pin 2-16 already programmed
Pin 2-18 already programmed
Pin 2-19 already programmed
Pin 2-16 already programmed
Pin 2-17 already programmed
Pin 2-19 already programmed
Pin 2-16 already programmed
Pin 2-17 already programmed
Pin 2-18 already programmed
Pin 2-16 already programmed
Pin 2-17 already programmed
Pin 2-18 already programmed
Pin 2-19 already programmed
IOAPIC[0]: Set PCI routing entry (2-20 -> 0xd1 -> IRQ 20 Mode:1 Active:1)
00:02:02[A] -> 2-20 -> IRQ 20
Pin 2-18 already programmed
Pin 2-17 already programmed
Pin 2-16 already programmed
number of MP IRQ sources: 15.
number of IO-APIC #2 registers: 24.
testing the IO APIC.......................
IO APIC #2......
.... register #00: 02000000
.......    : physical APIC id: 02
.......    : Delivery Type: 0
.......    : LTS          : 0
.... register #01: 00178020
.......     : max redirection entries: 0017
.......     : PRQ implemented: 1
.......     : IO APIC version: 0020
.... register #02: 00000000
.......     : arbitration: 00
.... register #03: 00000001
.......     : Boot DT    : 1
.... IRQ redirection table:
 NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:   
 00 000 00  1    0    0   0   0    0    0    00
 01 00F 0F  0    0    0   0   0    1    1    39
 02 00F 0F  0    0    0   0   0    1    1    31
 03 00F 0F  0    0    0   0   0    1    1    41
 04 00F 0F  0    0    0   0   0    1    1    49
 05 00F 0F  0    0    0   0   0    1    1    51
 06 00F 0F  0    0    0   0   0    1    1    59
 07 00F 0F  0    0    0   0   0    1    1    61
 08 00F 0F  0    0    0   0   0    1    1    69
 09 00F 0F  0    1    0   0   0    1    1    71
 0a 00F 0F  0    0    0   0   0    1    1    79
 0b 00F 0F  0    0    0   0   0    1    1    81
 0c 00F 0F  0    0    0   0   0    1    1    89
 0d 00F 0F  0    0    0   0   0    1    1    91
 0e 00F 0F  0    0    0   0   0    1    1    99
 0f 00F 0F  0    0    0   0   0    1    1    A1
 10 00F 0F  1    1    0   1   0    1    1    B9
 11 00F 0F  1    1    0   1   0    1    1    B1
 12 00F 0F  1    1    0   1   0    1    1    A9
 13 00F 0F  1    1    0   1   0    1    1    C1
 14 00F 0F  1    1    0   1   0    1    1    D1
 15 000 00  1    0    0   0   0    0    0    00
 16 000 00  1    0    0   0   0    0    0    00
 17 00F 0F  1    1    0   1   0    1    1    C9
IRQ to pin mappings:
IRQ0 -> 0:2
IRQ1 -> 0:1
IRQ3 -> 0:3
IRQ4 -> 0:4
IRQ5 -> 0:5
IRQ6 -> 0:6
IRQ7 -> 0:7
IRQ8 -> 0:8
IRQ9 -> 0:9-> 0:9
IRQ10 -> 0:10
IRQ11 -> 0:11
IRQ12 -> 0:12
IRQ13 -> 0:13
IRQ14 -> 0:14
IRQ15 -> 0:15
IRQ16 -> 0:16
IRQ17 -> 0:17
IRQ18 -> 0:18
IRQ19 -> 0:19
IRQ20 -> 0:20
IRQ23 -> 0:23
.................................... done.
PCI: Using ACPI for IRQ routing
PCI: if you experience problems, try using option 'pci=noacpi' or even 'acpi=off'
SBF: Simple Boot Flag extension found and enabled.
SBF: Setting boot flags 0x80
Machine check exception polling timer started.
Starting balanced_irq
Total HugeTLB memory allocated, 0
ikconfig 0.7 with /proc/config*
highmem bounce pool size: 64 pages
devfs: v1.22 (20021013) Richard Gooch (rgooch@atnf.csiro.au)
devfs: boot_options: 0x0
SGI XFS with no debug enabled
Initializing Cryptographic API
pty: 256 Unix98 ptys configured
Real Time Clock Driver v1.12
loop: loaded (max 8 devices)
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
ICH4: IDE controller at PCI slot 0000:00:1f.1
ICH4: chipset revision 2
ICH4: not 100% native mode: will probe irqs later
    ide0: BM-DMA at 0x1480-0x1487, BIOS settings: hda:pio, hdb:pio
    ide1: BM-DMA at 0x1488-0x148f, BIOS settings: hdc:DMA, hdd:pio
hdc: PIONEER DVD-RW DVR-105, ATAPI CD/DVD-ROM drive
Using anticipatory io scheduler
ide1 at 0x170-0x177,0x376 on irq 15
hdc: ATAPI 47X DVD-ROM DVD-R CD-R/RW drive, 2000kB Cache, UDMA(33)
Uniform CD-ROM driver Revision: 3.20
scsi0 : Adaptec AIC7XXX EISA/VLB/PCI SCSI HBA DRIVER, Rev 6.2.36
        <Adaptec 29160 Ultra160 SCSI adapter>
        aic7892: Ultra160 Wide Channel A, SCSI Id=7, 32/253 SCBs

(scsi0:A:0): 160.000MB/s transfers (80.000MHz DT, offset 63, 16bit)
(scsi0:A:1): 80.000MB/s transfers (40.000MHz, offset 31, 16bit)
(scsi0:A:2): 80.000MB/s transfers (40.000MHz, offset 31, 16bit)
(scsi0:A:3): 160.000MB/s transfers (80.000MHz DT, offset 63, 16bit)
(scsi0:A:4): 160.000MB/s transfers (80.000MHz DT, offset 63, 16bit)
(scsi0:A:5): 20.000MB/s transfers (20.000MHz, offset 16)
(scsi0:A:6): 20.000MB/s transfers (20.000MHz, offset 16)
  Vendor: IBM       Model: DDYS-T18350N      Rev: S96H
  Type:   Direct-Access                      ANSI SCSI revision: 03
scsi0:A:0:0: Tagged Queuing enabled.  Depth 32
  Vendor: IBM       Model: DNES-318350W      Rev: SA30
  Type:   Direct-Access                      ANSI SCSI revision: 03
scsi0:A:1:0: Tagged Queuing enabled.  Depth 32
  Vendor: IBM       Model: DNES-309170W      Rev: SA30
  Type:   Direct-Access                      ANSI SCSI revision: 03
scsi0:A:2:0: Tagged Queuing enabled.  Depth 32
  Vendor: IBM       Model: DDYS-T18350N      Rev: S96H
  Type:   Direct-Access                      ANSI SCSI revision: 03
scsi0:A:3:0: Tagged Queuing enabled.  Depth 32
  Vendor: IBM       Model: DDYS-T18350N      Rev: S80D
  Type:   Direct-Access                      ANSI SCSI revision: 03
scsi0:A:4:0: Tagged Queuing enabled.  Depth 32
  Vendor: PIONEER   Model: DVD-ROM DVD-305   Rev: 1.00
  Type:   CD-ROM                             ANSI SCSI revision: 02
  Vendor: TOSHIBA   Model: DVD-ROM SD-M1201  Rev: 1R08
  Type:   CD-ROM                             ANSI SCSI revision: 02
SCSI device sda: 35843670 512-byte hdwr sectors (18352 MB)
SCSI device sda: drive cache: write back
 /dev/scsi/host0/bus0/target0/lun0: p1 p2 p3 p4
Attached scsi disk sda at scsi0, channel 0, id 0, lun 0
SCSI device sdb: 35843670 512-byte hdwr sectors (18352 MB)
SCSI device sdb: drive cache: write back
 /dev/scsi/host0/bus0/target1/lun0: p1
Attached scsi disk sdb at scsi0, channel 0, id 1, lun 0
SCSI device sdc: 17916240 512-byte hdwr sectors (9173 MB)
SCSI device sdc: drive cache: write through
 /dev/scsi/host0/bus0/target2/lun0: p1
Attached scsi disk sdc at scsi0, channel 0, id 2, lun 0
SCSI device sdd: 35843670 512-byte hdwr sectors (18352 MB)
SCSI device sdd: drive cache: write back
 /dev/scsi/host0/bus0/target3/lun0: p1
Attached scsi disk sdd at scsi0, channel 0, id 3, lun 0
SCSI device sde: 35843670 512-byte hdwr sectors (18352 MB)
SCSI device sde: drive cache: write back
 /dev/scsi/host0/bus0/target4/lun0: p1
Attached scsi disk sde at scsi0, channel 0, id 4, lun 0
sr0: scsi3-mmc drive: 16x/40x cd/rw xa/form2 cdda tray
Attached scsi CD-ROM sr0 at scsi0, channel 0, id 5, lun 0
sr1: scsi3-mmc drive: 32x/32x cd/rw xa/form2 cdda tray
Attached scsi CD-ROM sr1 at scsi0, channel 0, id 6, lun 0
Attached scsi generic sg0 at scsi0, channel 0, id 0, lun 0,  type 0
Attached scsi generic sg1 at scsi0, channel 0, id 1, lun 0,  type 0
Attached scsi generic sg2 at scsi0, channel 0, id 2, lun 0,  type 0
Attached scsi generic sg3 at scsi0, channel 0, id 3, lun 0,  type 0
Attached scsi generic sg4 at scsi0, channel 0, id 4, lun 0,  type 0
Attached scsi generic sg5 at scsi0, channel 0, id 5, lun 0,  type 5
Attached scsi generic sg6 at scsi0, channel 0, id 6, lun 0,  type 5
drivers/usb/host/uhci-hcd.c: USB Universal Host Controller Interface driver v2.1
uhci_hcd 0000:00:1d.0: UHCI Host Controller
PCI: Setting latency timer of device 0000:00:1d.0 to 64
uhci_hcd 0000:00:1d.0: irq 16, io base 00001400
uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 1
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 2 ports detected
uhci_hcd 0000:00:1d.1: UHCI Host Controller
PCI: Setting latency timer of device 0000:00:1d.1 to 64
uhci_hcd 0000:00:1d.1: irq 19, io base 00001420
uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 2
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 2 ports detected
uhci_hcd 0000:00:1d.2: UHCI Host Controller
PCI: Setting latency timer of device 0000:00:1d.2 to 64
uhci_hcd 0000:00:1d.2: irq 18, io base 00001440
uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 3
hub 3-0:1.0: USB hub found
hub 3-0:1.0: 2 ports detected
drivers/usb/core/usb.c: registered new driver hiddev
drivers/usb/core/usb.c: registered new driver hid
drivers/usb/input/hid-core.c: v2.0:USB HID core driver
mice: PS/2 mouse device common for all mice
input: PC Speaker
serio: i8042 AUX port at 0x60,0x64 irq 12
serio: i8042 KBD port at 0x60,0x64 irq 1
input: AT Translated Set 2 keyboard on isa0060/serio0
NET: Registered protocol family 2
IP: routing cache hash table of 16384 buckets, 128Kbytes
TCP: Hash tables configured (established 524288 bind 65536)
ip_conntrack version 2.1 (8192 buckets, 65536 max) - 304 bytes per conntrack
ip_tables: (C) 2000-2002 Netfilter core team
ipt_recent v0.3.1: Stephen Frost <sfrost@snowman.net>.  http://snowman.net/projects/ipt_recent/
arp_tables: (C) 2002 David S. Miller
XFS mounting filesystem sdc1
hub 1-0:1.0: new USB device on port 1, assigned address 2
Ending clean XFS mount for filesystem: sdc1
VFS: Mounted root (xfs filesystem) readonly.
Freeing unused kernel memory: 356k freed
input: USB HID v1.00 Mouse [USB Mouse] on usb-0000:00:1d.0-1
NET: Registered protocol family 1
Intel(R) PRO/100 Network Driver - version 2.3.30-k1
Copyright (c) 2003 Intel Corporation

e100: selftest OK.
e100: eth0: Intel(R) PRO/100 Network Connection
  Hardware receive checksums enabled

md: md driver 0.90.0 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: raid0 personality registered as nr 2
md: bind<sdd1>
md: nonpersistent superblock ...
md: bind<sde1>
md: nonpersistent superblock ...
md0: setting max_sectors to 32, segment boundary to 8191
raid0: looking at sde1
raid0:   comparing sde1(17920464) with sde1(17920464)
raid0:   END
raid0:   ==> UNIQUE
raid0: 1 zones
raid0: looking at sdd1
raid0:   comparing sdd1(17920464) with sde1(17920464)
raid0:   EQUAL
raid0: FINAL 1 zones
raid0: done.
raid0 : md_size is 35840928 blocks.
raid0 : conf->hash_spacing is 35840928 blocks.
raid0 : nb_zone is 1.
raid0 : Allocating 4 bytes for hash.
Adding 401616k swap on /dev/sda4.  Priority:-1 extents:1
XFS mounting filesystem md0
Ending clean XFS mount for filesystem: md0
XFS mounting filesystem sda3
Starting XFS recovery on filesystem: sda3 (dev: sda3)
Ending XFS recovery on filesystem: sda3 (dev: sda3)
e100: eth0 NIC Link is Up 100 Mbps Full duplex
request_module: failed /sbin/modprobe -- block-major-3-0. error = 256
XFS mounting filesystem sda3
Starting XFS recovery on filesystem: sda3 (dev: sda3)
Ending XFS recovery on filesystem: sda3 (dev: sda3)
nvidia: module license 'NVIDIA' taints kernel.
1: nvidia: loading NVIDIA Linux x86 NVIDIA Kernel Module  1.0-5336  Wed Jan 14 18:29:26 PST 2004
Linux agpgart interface v0.100 (c) Dave Jones
1: NVRM: AGPGART: unable to retrieve symbol table
atkbd.c: Unknown key released (translated set 2, code 0x7a on isa0060/serio0).
atkbd.c: This is an XFree86 bug. It shouldn't access hardware directly.
atkbd.c: Unknown key released (translated set 2, code 0x7a on isa0060/serio0).
atkbd.c: This is an XFree86 bug. It shouldn't access hardware directly.
1: NVRM: AGPGART: unable to retrieve symbol table
atkbd.c: Unknown key released (translated set 2, code 0x7a on isa0060/serio0).
atkbd.c: This is an XFree86 bug. It shouldn't access hardware directly.
atkbd.c: Unknown key released (translated set 2, code 0x7a on isa0060/serio0).
atkbd.c: This is an XFree86 bug. It shouldn't access hardware directly.
1: NVRM: AGPGART: unable to retrieve symbol table
atkbd.c: Unknown key released (translated set 2, code 0x7a on isa0060/serio0).
atkbd.c: This is an XFree86 bug. It shouldn't access hardware directly.
atkbd.c: Unknown key released (translated set 2, code 0x7a on isa0060/serio0).
atkbd.c: This is an XFree86 bug. It shouldn't access hardware directly.
request_module: failed /sbin/modprobe -- block-major-3-0. error = 256
3: nvidia: trying to map 0xef9f1000 to kernel space, but we're in an interrupt or holding a spinlock
2: nvidia: trying to map 0xef9f1000 to kernel space, but we're in an interrupt or holding a spinlock
3: nvidia: trying to map 0xef9f1000 to kernel space, but we're in an interrupt or holding a spinlock
1: nvidia: trying to map 0xef9f1000 to kernel space, but we're in an interrupt or holding a spinlock
2: nvidia: trying to map 0xef9f1000 to kernel space, but we're in an interrupt or holding a spinlock
2: nvidia: trying to map 0xef9f1000 to kernel space, but we're in an interrupt or holding a spinlock
SysRq : Show State

                         free                        sibling
  task             PC    stack   pid father child younger older
init          S C14F1FC8  5528     1      0     2               (NOTLB)
f7fa9eb0 00000086 00000000 c14f1fc8 c03b9980 000001d5 f7fa9f40 00000000 
       c013eccf c03b9980 00000000 00000000 ec646e80 d0a9a080 c2431c80 00000e9c 
       bbde137c 00000df6 c247f940 c247fb08 00e5e93e f7fa9ec4 0000000b 0000000b 
Call Trace:
 [<c013eccf>] __alloc_pages+0x9d/0x328
 [<c01297cd>] schedule_timeout+0x6b/0xbf
 [<c0129759>] process_timeout+0x0/0x9
 [<c016d967>] do_select+0x1a8/0x2fa
 [<c016d619>] __pollwait+0x0/0xc7
 [<c016dda5>] sys_select+0x2c7/0x4e3
 [<c0164471>] sys_stat64+0x37/0x39
 [<c0108f45>] sysenter_past_esp+0x52/0x71

migration/0   S 00000001  7888     2      1             3       (L-TLB)
c2473fc4 00000046 c2429c80 00000001 0000000f f7cef310 50b39010 00000000 
       00000000 02542ce4 f7cebf58 c2472000 f7cebf50 00000292 c2429c80 0000074b 
       02543085 00000000 c247e080 c247e248 c242a5b4 c2472000 c2429c80 c2473fec 
Call Trace:
 [<c011e046>] migration_thread+0xc9/0x137
 [<c011df7d>] migration_thread+0x0/0x137
 [<c0106e19>] kernel_thread_helper+0x5/0xb

ksoftirqd/0   S EAC55EC8  7208     3      1             4     2 (L-TLB)
f7f9ffd8 00000046 000dab44 eac55ec8 c242b674 c242b66c f7f9ff94 f7f9ff94 
       c242b67c 00000000 cedd9310 c0465664 c01250af cedd9310 c2429c80 000001ac 
       9e154195 00000dd4 c2471940 c2471b08 00000000 f7f9e000 00000000 00000000 
Call Trace:
 [<c01250af>] tasklet_action+0x65/0xae
 [<c0125351>] ksoftirqd+0xdd/0xec
 [<c0125274>] ksoftirqd+0x0/0xec
 [<c0106e19>] kernel_thread_helper+0x5/0xb

migration/1   S 00000001  7884     4      1             5     3 (L-TLB)
f7f9dfc4 00000046 c2431c80 00000001 00000003 c0464d20 c0129caf c03b897c 
       c03b8978 f7f9c000 c0121fff 00000082 c03b5510 00000008 c2431c80 0000151c 
       001025be 00000000 c2471310 c24714d8 c24325b4 f7f9c000 c2431c80 f7f9dfec 
Call Trace:
 [<c0129caf>] free_uid+0x1f/0x7d
 [<c0121fff>] reparent_to_init+0x104/0x199
 [<c011e046>] migration_thread+0xc9/0x137
 [<c011df7d>] migration_thread+0x0/0x137
 [<c0106e19>] kernel_thread_helper+0x5/0xb

ksoftirqd/1   S C54F2DC8  7784     5      1             6     4 (L-TLB)
f7f9bfd8 00000046 f7f9bf94 c54f2dc8 c2433674 c243366c f7f9bf94 f7f9bf94 
       c243367c 00000000 f7f9a000 c0465664 c01250af e1bcd310 c2431c80 00000117 
       ad6bb34d 00000dd4 c2470ce0 c2470ea8 00000000 f7f9a000 00000000 00000000 
Call Trace:
 [<c01250af>] tasklet_action+0x65/0xae
 [<c0125351>] ksoftirqd+0xdd/0xec
 [<c0125274>] ksoftirqd+0x0/0xec
 [<c0106e19>] kernel_thread_helper+0x5/0xb

migration/2   S 00000001  7884     6      1             7     5 (L-TLB)
f7f99fc4 00000046 c2439c80 00000001 00000007 c0464d20 c0129caf c03b897c 
       c03b8978 f7f98000 c0121fff 00000082 c03b5510 00000008 c2439c80 00002da8 
       001f122c 00000000 c24706b0 c2470878 c243a5b4 f7f98000 c2439c80 f7f99fec 
Call Trace:
 [<c0129caf>] free_uid+0x1f/0x7d
 [<c0121fff>] reparent_to_init+0x104/0x199
 [<c011e046>] migration_thread+0xc9/0x137
 [<c011df7d>] migration_thread+0x0/0x137
 [<c0106e19>] kernel_thread_helper+0x5/0xb

ksoftirqd/2   S 00000001  7764     7      1             8     6 (L-TLB)
f7f97fd8 00000046 c2439c80 00000001 0000000f c243b66c f7f97f94 f7f97f94 
       c243b67c 00000000 f7f96000 c0465664 c01250af d9dd6ce0 c2439c80 00000255 
       d915264a 00000dd2 c2470080 c2470248 00000000 f7f96000 00000000 00000000 
Call Trace:
 [<c01250af>] tasklet_action+0x65/0xae
 [<c0125351>] ksoftirqd+0xdd/0xec
 [<c0125274>] ksoftirqd+0x0/0xec
 [<c0106e19>] kernel_thread_helper+0x5/0xb

migration/3   S 00000001  7884     8      1             9     7 (L-TLB)
f7f93fc4 00000046 c2441c80 00000001 0000000f c0464d20 c0129caf c03b897c 
       c03b8978 f7f92000 c0121fff 00000082 c03b5510 00000008 c2441c80 00001879 
       002e594e 00000000 f7f95940 f7f95b08 c24425b4 f7f92000 c2441c80 f7f93fec 
Call Trace:
 [<c0129caf>] free_uid+0x1f/0x7d
 [<c0121fff>] reparent_to_init+0x104/0x199
 [<c011e046>] migration_thread+0xc9/0x137
 [<c011df7d>] migration_thread+0x0/0x137
 [<c0106e19>] kernel_thread_helper+0x5/0xb

ksoftirqd/3   S C7D389C8  7764     9      1            10     8 (L-TLB)
f7f91fd8 00000046 000da97e c7d389c8 c2443674 c244366c f7f91f94 f7f91f94 
       c244367c 00000000 f7f90000 c0465664 c01250af e1bcd310 c2441c80 00000114 
       adbe22f0 00000dd4 f7f95310 f7f954d8 00000000 f7f90000 00000000 00000000 
Call Trace:
 [<c01250af>] tasklet_action+0x65/0xae
 [<c0125351>] ksoftirqd+0xdd/0xec
 [<c0125274>] ksoftirqd+0x0/0xec
 [<c0106e19>] kernel_thread_helper+0x5/0xb

events/0      R C03C42E4  6904    10      1            11     9 (L-TLB)
f7f81f70 00000046 f7fc6c00 c03c42e4 c04969a0 00000286 00000001 f7f8007b 
       f7fc007b ffffff00 f7fc6c24 f7f80000 f7fc6c20 f28b4080 c2429c80 000001d8 
       feae07ce 00000df6 f7f94ce0 f7f94ea8 f7f80000 f7fc6c0c c03c42e0 00000292 
Call Trace:
 [<c0130f85>] worker_thread+0x270/0x298
 [<c026ac74>] console_callback+0x0/0xbe
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0108e6e>] ret_from_fork+0x6/0x14
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0130d15>] worker_thread+0x0/0x298
 [<c0106e19>] kernel_thread_helper+0x5/0xb

events/1      S E7AB7720  7384    11      1            12    10 (L-TLB)
c24fff70 00000046 00000000 e7ab7720 00000000 00000000 00000000 f2d46080 
       0000007b 0000007b f2d46080 c2431c80 00005436 e887b310 c2431c80 0000063b 
       327660c7 000005c9 f7f946b0 f7f94878 c24fe000 f7fc6c8c e7ab76e0 00000286 
Call Trace:
 [<c0130f85>] worker_thread+0x270/0x298
 [<c01309d1>] __call_usermodehelper+0x0/0x65
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0108e6e>] ret_from_fork+0x6/0x14
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0130d15>] worker_thread+0x0/0x298
 [<c0106e19>] kernel_thread_helper+0x5/0xb

events/2      S E8055DE8  7492    12      1            13    11 (L-TLB)
c24fdf70 00000046 00000000 e8055de8 00000000 00000000 00000000 c2680ce0 
       0000007b 0000007b c2680ce0 c2439c80 00001b89 c2681310 c2439c80 00000553 
       81239d37 000005cc f7f94080 f7f94248 c24fc000 f7fc6d0c e8055da8 00000286 
Call Trace:
 [<c0130f85>] worker_thread+0x270/0x298
 [<c01309d1>] __call_usermodehelper+0x0/0x65
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0108e6e>] ret_from_fork+0x6/0x14
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0130d15>] worker_thread+0x0/0x298
 [<c0106e19>] kernel_thread_helper+0x5/0xb

events/3      S 3CB54AB3  7724    13      1            14    12 (L-TLB)
c24f9f70 00000046 f7f95310 3cb54ab3 000005c8 f7fc6da4 c011c643 e7ab7ea8 
       00000003 00000000 f7fc6da4 3cb54ab3 000005c8 f7f95310 c2441c80 000de095 
       3cb55874 000005c8 c24fb940 c24fbb08 c24f8000 f7fc6d8c c03c42e0 00000292 
Call Trace:
 [<c011c643>] __wake_up_common+0x38/0x57
 [<c0130f85>] worker_thread+0x270/0x298
 [<c026ac74>] console_callback+0x0/0xbe
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0108e6e>] ret_from_fork+0x6/0x14
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0130d15>] worker_thread+0x0/0x298
 [<c0106e19>] kernel_thread_helper+0x5/0xb

kblockd/0     S 00000001  7116    14      1            15    13 (L-TLB)
f7d9ff70 00000046 c2429c80 00000001 0000000f f7c98000 d46f2200 f7d9e000 
       f7d9e000 f7c9de00 f7dcc024 f7d9e000 f7dcc020 c4c0c6b0 c2429c80 000000e1 
       21e611bb 00000dd2 c24fb310 c24fb4d8 f7d9e000 f7dcc00c f7c8ff1c 00000286 
Call Trace:
 [<c0130f85>] worker_thread+0x270/0x298
 [<c0279bfb>] as_work_handler+0x0/0x81
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0108e6e>] ret_from_fork+0x6/0x14
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0130d15>] worker_thread+0x0/0x298
 [<c0106e19>] kernel_thread_helper+0x5/0xb

kblockd/1     S F7C98000  7412    15      1            16    14 (L-TLB)
f7d9df70 00000046 d3dafe00 f7c98000 c0270dd3 f7c98000 d3dafe00 f7c9de00 
       d3dafe00 f7c9de00 f7dcc0a4 f7d9c000 f7dcc0a0 d0a9b310 c2431c80 000012a2 
       e74b1718 00000d7c c24face0 c24faea8 f7d9c000 f7dcc08c f7c980fc 00000286 
Call Trace:
 [<c0270dd3>] elv_next_request+0xf/0xf1
 [<c0130f85>] worker_thread+0x270/0x298
 [<c027286c>] blk_unplug_work+0x0/0x14
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0108e6e>] ret_from_fork+0x6/0x14
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0130d15>] worker_thread+0x0/0x298
 [<c0106e19>] kernel_thread_helper+0x5/0xb

kblockd/2     S F7C8CE00  7412    16      1            17    15 (L-TLB)
f7d9bf70 00000046 f7c9c080 f7c8ce00 c0270dd3 f7c8ce00 f7c9c080 f7c9de00 
       f7c9c080 f7c9de00 f7dcc124 f7d9a000 f7dcc120 f28b4080 c2439c80 00000b29 
       3683b921 00000ddd c24fa6b0 c24fa878 f7d9a000 f7dcc10c f7c8cefc 00000286 
Call Trace:
 [<c0270dd3>] elv_next_request+0xf/0xf1
 [<c0130f85>] worker_thread+0x270/0x298
 [<c027286c>] blk_unplug_work+0x0/0x14
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0108e6e>] ret_from_fork+0x6/0x14
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0130d15>] worker_thread+0x0/0x298
 [<c0106e19>] kernel_thread_helper+0x5/0xb

kblockd/3     S F7C8CE00  7424    17      1            18    16 (L-TLB)
f7d99f70 00000046 e38d3200 f7c8ce00 c0270dd3 f7c8ce00 e38d3200 f7c9de00 
       e38d3200 f7c9de00 f7dcc1a4 f7d98000 f7dcc1a0 e1bccce0 c2441c80 00001e65 
       c7cebd0f 00000dd1 c24fa080 c24fa248 f7d98000 f7dcc18c f7c8cefc 00000286 
Call Trace:
 [<c0270dd3>] elv_next_request+0xf/0xf1
 [<c0130f85>] worker_thread+0x270/0x298
 [<c027286c>] blk_unplug_work+0x0/0x14
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0108e6e>] ret_from_fork+0x6/0x14
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0130d15>] worker_thread+0x0/0x298
 [<c0106e19>] kernel_thread_helper+0x5/0xb

khubd         S 00000001  6956    18      1            19    17 (L-TLB)
f7d65fb4 00000046 c2431c80 00000001 0000000f f7c65c80 f7c32880 00000001 
       00000002 00000002 f7c32880 00000202 f7c32880 f7d65fa0 c2431c80 0001e25d 
       d7e8ef00 00000002 f7d67940 f7d67b08 f7d64000 f7d64000 f7d65fc4 00000000 
Call Trace:
 [<c02c9d25>] hub_thread+0x97/0xc3
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c02c9c8e>] hub_thread+0x0/0xc3
 [<c0106e19>] kernel_thread_helper+0x5/0xb

kirqd         S FC2EBCFD  7852    19      1            22    18 (L-TLB)
f7d63fa0 00000046 cefcf310 fc2ebcfd 00000dd5 00000000 c04675a0 00000200 
       00000000 00000000 e887b310 fc2ebcfd 00000dd5 d9dd7940 c2439c80 00000753 
       93885208 00000df6 f7d67310 f7d674d8 00e5e699 f7d63fb4 00000000 00000000 
Call Trace:
 [<c01297cd>] schedule_timeout+0x6b/0xbf
 [<c0129759>] process_timeout+0x0/0x9
 [<c01171d9>] balanced_irq+0x4b/0x71
 [<c011718e>] balanced_irq+0x0/0x71
 [<c0106e19>] kernel_thread_helper+0x5/0xb

kswapd0       S 00000100  6532    22      1            23    19 (L-TLB)
f7d4df08 00000046 000000d0 00000100 00000100 c03b9f80 00000002 00000000 
       c0146418 00000080 c03b9f80 000000d0 0000007f cefcf310 c2429c80 00008d69 
       77974d39 00000dad f7d66080 f7d66248 f7d4df20 f7d4dfc0 c03ba5d4 c03b9380 
Call Trace:
 [<c0146418>] balance_pgdat+0x19e/0x220
 [<c014657a>] kswapd+0xe0/0x115
 [<c011e87b>] autoremove_wake_function+0x0/0x4f
 [<c0108e6e>] ret_from_fork+0x6/0x14
 [<c011e87b>] autoremove_wake_function+0x0/0x4f
 [<c014649a>] kswapd+0x0/0x115
 [<c0106e19>] kernel_thread_helper+0x5/0xb

aio/0         S F7FA9F68  7804    23      1            24    22 (L-TLB)
f7d09f70 00000046 00000001 f7fa9f68 00000001 f7d09f4c c011c643 c247f940 
       00000003 00000000 f7fa9f68 f7d08000 00010000 c247f940 c2429c80 00000781 
       024f310d 00000000 f7d0b940 f7d0bb08 f7d08000 00000000 f7d09fb4 f7fa9f5c 
Call Trace:
 [<c011c643>] __wake_up_common+0x38/0x57
 [<c0130f85>] worker_thread+0x270/0x298
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0108e6e>] ret_from_fork+0x6/0x14
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0130d15>] worker_thread+0x0/0x298
 [<c0106e19>] kernel_thread_helper+0x5/0xb

aio/1         S 00000001  7724    24      1            25    23 (L-TLB)
f7d03f70 00000046 c2431c80 00000001 0000000f f7d03f4c c011c643 c247f940 
       00000003 00000000 f7fa9f68 f7d02000 00010000 f7d0b310 c2431c80 00000d26 
       024f71e9 00000000 f7d0b310 f7d0b4d8 f7d02000 00000001 f7d03fb4 f7fa9f5c 
Call Trace:
 [<c011c643>] __wake_up_common+0x38/0x57
 [<c0130f85>] worker_thread+0x270/0x298
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0108e6e>] ret_from_fork+0x6/0x14
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0130d15>] worker_thread+0x0/0x298
 [<c0106e19>] kernel_thread_helper+0x5/0xb

aio/2         S 00000001  7724    25      1            26    24 (L-TLB)
f7d01f70 00000046 c2439c80 00000001 0000000f f7d01f4c c011c643 c247f940 
       00000003 00000000 f7fa9f68 f7d00000 00010000 f7d0ace0 c2439c80 00000edc 
       024fb64d 00000000 f7d0ace0 f7d0aea8 f7d00000 00000002 f7d01fb4 f7fa9f5c 
Call Trace:
 [<c011c643>] __wake_up_common+0x38/0x57
 [<c0130f85>] worker_thread+0x270/0x298
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0108e6e>] ret_from_fork+0x6/0x14
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0130d15>] worker_thread+0x0/0x298
 [<c0106e19>] kernel_thread_helper+0x5/0xb

aio/3         S 00000001  7724    26      1            27    25 (L-TLB)
f7cfff70 00000046 c2441c80 00000001 0000000f f7cfff4c c011c643 c247f940 
       00000003 00000000 f7fa9f68 f7cfe000 00010000 f7d0a6b0 c2441c80 00000fdc 
       02500458 00000000 f7d0a6b0 f7d0a878 f7cfe000 00000003 f7cfffb4 f7fa9f5c 
Call Trace:
 [<c011c643>] __wake_up_common+0x38/0x57
 [<c0130f85>] worker_thread+0x270/0x298
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0108e6e>] ret_from_fork+0x6/0x14
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0130d15>] worker_thread+0x0/0x298
 [<c0106e19>] kernel_thread_helper+0x5/0xb

xfslogd/0     S C27DFD80  6328    27      1            28    26 (L-TLB)
f7cfdf70 00000046 f7cfc000 c27dfd80 00000000 c01e5b08 c27dfd80 00000000 
       c27d3600 c27ce580 f7d30424 f7cfc000 f7d30420 f28b4080 c2429c80 000000bf 
       2ec4ed7c 00000df6 f7d0a080 f7d0a248 f7cfc000 f7d3040c e8c2cce0 00000282 
Call Trace:
 [<c01e5b08>] xlog_state_done_syncing+0x7a/0xb1
 [<c0130f85>] worker_thread+0x270/0x298
 [<c0203a77>] pagebuf_iodone_work+0x0/0x56
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0108e6e>] ret_from_fork+0x6/0x14
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0130d15>] worker_thread+0x0/0x298
 [<c0106e19>] kernel_thread_helper+0x5/0xb

xfslogd/1     S 00000001  7724    28      1            29    27 (L-TLB)
f7cf9f70 00000046 c2431c80 00000001 0000000f f7cf9f4c c011c643 c247f940 
       00000003 00000000 f7fa9f08 f7cf8000 00010000 f7cfb940 c2431c80 00000a7b 
       0252abd9 00000000 f7cfb940 f7cfbb08 f7cf8000 00000001 f7cf9fb4 f7fa9efc 
Call Trace:
 [<c011c643>] __wake_up_common+0x38/0x57
 [<c0130f85>] worker_thread+0x270/0x298
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0108e6e>] ret_from_fork+0x6/0x14
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0130d15>] worker_thread+0x0/0x298
 [<c0106e19>] kernel_thread_helper+0x5/0xb

xfslogd/2     S 00000001  7724    29      1            30    28 (L-TLB)
f7cf7f70 00000046 c2439c80 00000001 0000000f f7cf7f4c c011c643 c247f940 
       00000003 00000000 f7fa9f08 f7cf6000 00010000 f7cfb310 c2439c80 00000ea9 
       0252f0c5 00000000 f7cfb310 f7cfb4d8 f7cf6000 00000002 f7cf7fb4 f7fa9efc 
Call Trace:
 [<c011c643>] __wake_up_common+0x38/0x57
 [<c0130f85>] worker_thread+0x270/0x298
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0108e6e>] ret_from_fork+0x6/0x14
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0130d15>] worker_thread+0x0/0x298
 [<c0106e19>] kernel_thread_helper+0x5/0xb

xfslogd/3     S 00000001  7724    30      1            31    29 (L-TLB)
f7cf5f70 00000046 c2441c80 00000001 0000000f f7cf5f4c c011c643 c247f940 
       00000003 00000000 f7fa9f08 f7cf4000 00010000 f7cface0 c2441c80 00000dd1 
       02533551 00000000 f7cface0 f7cfaea8 f7cf4000 00000003 f7cf5fb4 f7fa9efc 
Call Trace:
 [<c011c643>] __wake_up_common+0x38/0x57
 [<c0130f85>] worker_thread+0x270/0x298
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0108e6e>] ret_from_fork+0x6/0x14
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0130d15>] worker_thread+0x0/0x298
 [<c0106e19>] kernel_thread_helper+0x5/0xb

xfsdatad/0    S F7FA9F08  7804    31      1            32    30 (L-TLB)
f7cf3f70 00000046 00000001 f7fa9f08 00000001 f7cf3f4c c011c643 c247f940 
       00000003 00000000 f7fa9f08 f7cf2000 00010000 c247f940 c2429c80 000005a0 
       02536801 00000000 f7cfa6b0 f7cfa878 f7cf2000 00000000 f7cf3fb4 f7fa9efc 
Call Trace:
 [<c011c643>] __wake_up_common+0x38/0x57
 [<c0130f85>] worker_thread+0x270/0x298
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0108e6e>] ret_from_fork+0x6/0x14
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0130d15>] worker_thread+0x0/0x298
 [<c0106e19>] kernel_thread_helper+0x5/0xb

xfsdatad/1    S 00000001  7724    32      1            33    31 (L-TLB)
f7cf1f70 00000046 c2431c80 00000001 0000000f f7cf1f4c c011c643 c247f940 
       00000003 00000000 f7fa9f08 f7cf0000 00010000 f7cfa080 c2431c80 00000a50 
       0253a5ba 00000000 f7cfa080 f7cfa248 f7cf0000 00000001 f7cf1fb4 f7fa9efc 
Call Trace:
 [<c011c643>] __wake_up_common+0x38/0x57
 [<c0130f85>] worker_thread+0x270/0x298
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0108e6e>] ret_from_fork+0x6/0x14
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0130d15>] worker_thread+0x0/0x298
 [<c0106e19>] kernel_thread_helper+0x5/0xb

xfsdatad/2    S 00000001  7724    33      1            34    32 (L-TLB)
f7cedf70 00000046 c2439c80 00000001 0000000f f7cedf4c c011c643 c247f940 
       00000003 00000000 f7fa9f08 f7cec000 00010000 f7cef940 c2439c80 00001011 
       0253f34c 00000000 f7cef940 f7cefb08 f7cec000 00000002 f7cedfb4 f7fa9efc 
Call Trace:
 [<c011c643>] __wake_up_common+0x38/0x57
 [<c0130f85>] worker_thread+0x270/0x298
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0108e6e>] ret_from_fork+0x6/0x14
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0130d15>] worker_thread+0x0/0x298
 [<c0106e19>] kernel_thread_helper+0x5/0xb

xfsdatad/3    S 00000001  7724    34      1            35    33 (L-TLB)
f7cebf70 00000046 c2441c80 00000001 0000000f f7cebf4c c011c643 c247f940 
       00000003 00000000 f7fa9f08 f7cea000 00010000 f7cef310 c2441c80 00000fd2 
       02544216 00000000 f7cef310 f7cef4d8 f7cea000 00000003 f7cebfb4 f7fa9efc 
Call Trace:
 [<c011c643>] __wake_up_common+0x38/0x57
 [<c0130f85>] worker_thread+0x270/0x298
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0108e6e>] ret_from_fork+0x6/0x14
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0130d15>] worker_thread+0x0/0x298
 [<c0106e19>] kernel_thread_helper+0x5/0xb

pagebufd      S 00000000  7276    35      1            36    34 (L-TLB)
f7ce9f88 00000046 f7c68c80 00000000 00000000 f7c8fe80 00000000 f7c90c00 
       f7ce8000 c02790e4 d9dd7940 f7c9c680 f7c98000 d9dd7940 c2439c80 000000fb 
       d0e5e8ce 00000df6 f7ceece0 f7ceeea8 00e5daff f7ce9f9c f7ce8000 f7ce9fd4 
Call Trace:
 [<c02790e4>] as_next_request+0x33/0x3c
 [<c01297cd>] schedule_timeout+0x6b/0xbf
 [<c0129759>] process_timeout+0x0/0x9
 [<c0204474>] pagebuf_daemon+0x56/0x1da
 [<c020441e>] pagebuf_daemon+0x0/0x1da
 [<c0106e19>] kernel_thread_helper+0x5/0xb

scsi_eh_0     S C011A7A6     4    36      1            37    35 (L-TLB)
f7c93f70 00000046 f7c93f40 c011a7a6 c247f940 26896a00 00000000 00000000 
       0ac9a25a 00000001 c2429c80 f7c93f6c 00000096 c247f940 c2429c80 00001e09 
       0ac9a74c 00000001 f7cee6b0 f7cee878 f7c92000 f7c93fc4 00000286 f7cee6b0 
Call Trace:
 [<c011a7a6>] recalc_task_prio+0x90/0x1aa
 [<c0107d5c>] __down_interruptible+0xac/0x142
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0107e83>] __down_failed_interruptible+0x7/0xc
 [<c0296e19>] .text.lock.scsi_error+0xb9/0xc0
 [<c0296a50>] scsi_error_handler+0x0/0x141
 [<c0106e19>] kernel_thread_helper+0x5/0xb

ahc_dv_0      S 1ED01FB0  7576    37      1            38    36 (L-TLB)
f7c6bf88 00000046 c2470ce0 1ed01fb0 00000002 f7c9e400 f7c6a000 c02790e4 
       f7cc0380 f7c6a000 f7c98e00 1ed01fb0 00000002 c2470ce0 c2431c80 000005b2 
       1ed0369b 00000002 f7cee080 f7cee248 f7c6a000 f7ca50c4 00000286 f7cee080 
Call Trace:
 [<c02790e4>] as_next_request+0x33/0x3c
 [<c0107d5c>] __down_interruptible+0xac/0x142
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0297614>] scsi_run_host_queues+0x36/0x42
 [<c0107e83>] __down_failed_interruptible+0x7/0xc
 [<c02b6639>] .text.lock.aic7xxx_osm+0x71/0x1dc
 [<c02b1d58>] ahc_linux_dv_thread+0x0/0x31b
 [<c0106e19>] kernel_thread_helper+0x5/0xb

kseriod       S 00000001  7864    38      1            39    37 (L-TLB)
c27c3fb4 00000046 c2441c80 00000001 0000000f 00000000 c0129caf c03b897c 
       c03b8978 c27c2000 c0121fff c03b897c c03b5510 00000008 c2441c80 04e7fbb8 
       c239ecf5 00000002 c27c5940 c27c5b08 c27c2000 c27c2000 c27c3fc4 00000000 
Call Trace:
 [<c0129caf>] free_uid+0x1f/0x7d
 [<c0121fff>] reparent_to_init+0x104/0x199
 [<c02e1d3a>] serio_thread+0x11e/0x14a
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c02e1c1c>] serio_thread+0x0/0x14a
 [<c0106e19>] kernel_thread_helper+0x5/0xb

xfs_syncd     S 00000000  6844    39      1           133    38 (L-TLB)
c2687f9c 00000046 d3643cf0 00000000 c27dfd80 00000002 00000000 c01e3442 
       c27dfd80 00000002 c01d9137 c27c8d90 00000031 f28b4080 c2439c80 000015d5 
       50b59888 00000df4 c27c5310 c27c54d8 00e6224e c2687fb0 00000000 00000000 
Call Trace:
 [<c01e3442>] xfs_log_force+0x5d/0x8a
 [<c01d9137>] xfs_iunlock+0x37/0x71
 [<c01297cd>] schedule_timeout+0x6b/0xbf
 [<c0129759>] process_timeout+0x0/0x9
 [<c020a433>] syncd+0x46/0x8e
 [<c020a3ed>] syncd+0x0/0x8e
 [<c0106e19>] kernel_thread_helper+0x5/0xb

xfs_syncd     S 00000000    12   133      1           144    39 (L-TLB)
f7767f9c 00000046 c2439c80 00000000 f764c080 00000002 00000000 c01e3442 
       f764c080 00000002 00000000 00000031 00000031 f28b4080 c2439c80 0000022f 
       0b0a2629 00000df3 c27c46b0 c27c4878 00e60cf4 f7767fb0 00000000 00000000 
Call Trace:
 [<c01e3442>] xfs_log_force+0x5d/0x8a
 [<c01297cd>] schedule_timeout+0x6b/0xbf
 [<c0129759>] process_timeout+0x0/0x9
 [<c020a433>] syncd+0x46/0x8e
 [<c020a3ed>] syncd+0x0/0x8e
 [<c0106e19>] kernel_thread_helper+0x5/0xb

bash          S 00000001  6528   144      1 31785     145   133 (NOTLB)
f74f7f3c 00000086 c2439c80 00000001 0000000f 7abb2065 00000000 f5097800 
       f5097820 c260c340 f7764ce0 c0118e24 f5097800 c260c340 c2439c80 000061e2 
       b70403b4 00000493 f7764ce0 f7764ea8 fffffe00 f74f6000 f7764d88 f7764d88 
Call Trace:
 [<c0118e24>] do_page_fault+0x2f5/0x4df
 [<c0123cb0>] sys_wait4+0x1ac/0x272
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c012cdfd>] sigprocmask+0x64/0xf0
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0123d9d>] sys_waitpid+0x27/0x2b
 [<c0108f45>] sysenter_past_esp+0x52/0x71

bash          S 00000001  6132   145      1           146   144 (NOTLB)
f74f5e7c 00000086 c2441c80 00000001 0000000f c026b232 f762b000 00000000 
       f74f5e64 0000000b 0000000b f762b000 c025d69a f762b000 c2441c80 00005d90 
       d2a53020 000002ce f77646b0 f7764878 f762b000 7fffffff f7c49680 f75ac000 
Call Trace:
 [<c026b232>] con_write+0x39/0x47
 [<c025d69a>] opost_block+0x112/0x1da
 [<c012981f>] schedule_timeout+0xbd/0xbf
 [<c02c5474>] vgacon_cursor+0x100/0x206
 [<c025eefe>] read_chan+0x734/0x8d2
 [<c025f1f3>] write_chan+0x157/0x21a
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c025a049>] tty_write+0x1a2/0x2e5
 [<c025e7ca>] read_chan+0x0/0x8d2
 [<c0259e76>] tty_read+0x14d/0x17e
 [<c0159a72>] vfs_read+0xb0/0x119
 [<c012cf88>] sys_rt_sigprocmask+0xff/0x18a
 [<c0159ced>] sys_read+0x42/0x63
 [<c0108f45>] sysenter_past_esp+0x52/0x71

agetty        S 00000001  5872   146      1           147   145 (NOTLB)
c262fe7c 00000086 c2441c80 00000001 0000000f c2782ca0 ffffffff f77b4e00 
       00000020 c0495868 c026a7c1 f77b4e00 00000020 00000008 c2441c80 0000a771 
       32faa162 0000000c c27c4ce0 c27c4ea8 f76af000 7fffffff c2604b80 f76ae000 
Call Trace:
 [<c026a7c1>] do_con_write+0x27e/0x731
 [<c012981f>] schedule_timeout+0xbd/0xbf
 [<c025eefe>] read_chan+0x734/0x8d2
 [<c025f1f3>] write_chan+0x157/0x21a
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c025a049>] tty_write+0x1a2/0x2e5
 [<c025e7ca>] read_chan+0x0/0x8d2
 [<c0259e76>] tty_read+0x14d/0x17e
 [<c0159a72>] vfs_read+0xb0/0x119
 [<c02323c0>] __copy_to_user_ll+0x68/0x6c
 [<c0159ced>] sys_read+0x42/0x63
 [<c0108f45>] sysenter_past_esp+0x52/0x71

agetty        S 00000001  6668   147      1           160   146 (NOTLB)
f74ffe7c 00000082 c2439c80 00000001 0000000f c2782ca0 ffffffff c2621c00 
       00000020 c0495868 c026a7c1 c2621c00 00000020 00000008 c2439c80 0002320f 
       32f9f644 0000000c c2681940 c2681b08 f770a000 7fffffff c2604080 f7709000 
Call Trace:
 [<c026a7c1>] do_con_write+0x27e/0x731
 [<c012981f>] schedule_timeout+0xbd/0xbf
 [<c025eefe>] read_chan+0x734/0x8d2
 [<c025f1f3>] write_chan+0x157/0x21a
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c025a049>] tty_write+0x1a2/0x2e5
 [<c025e7ca>] read_chan+0x0/0x8d2
 [<c0259e76>] tty_read+0x14d/0x17e
 [<c0159a72>] vfs_read+0xb0/0x119
 [<c02323c0>] __copy_to_user_ll+0x68/0x6c
 [<c0159ced>] sys_read+0x42/0x63
 [<c0108f45>] sysenter_past_esp+0x52/0x71

bash          S 00000001    80   160      1 25339    2144   147 (NOTLB)
f771bf3c 00000086 c2429c80 00000001 0000000f 5d4d7065 00000000 f7c50d00 
       f7c50d20 c260ce40 c27c4080 c0118e24 f7c50d00 c260ce40 c2429c80 000039b9 
       f072f2a1 000005c8 c27c4080 c27c4248 fffffe00 f771a000 c27c4128 c27c4128 
Call Trace:
 [<c0118e24>] do_page_fault+0x2f5/0x4df
 [<c0123cb0>] sys_wait4+0x1ac/0x272
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c012cdfd>] sigprocmask+0x64/0xf0
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0123d9d>] sys_waitpid+0x27/0x2b
 [<c0108f45>] sysenter_past_esp+0x52/0x71

pdflush       S 00000001  5444  2144      1          2968   160 (L-TLB)
ed1a9fac 00000046 c2431c80 00000001 0000000f 00000000 00000000 00000000 
       00000000 00000000 00000000 00000000 00000000 00000000 c2431c80 0000079b 
       cad33e32 00000353 f7d666b0 f7d66878 ed1a9fdc ed1a8000 ed1a9fd0 ffffe000 
Call Trace:
 [<c0140603>] __pdflush+0xab/0x20c
 [<c0140764>] pdflush+0x0/0x13
 [<c0140773>] pdflush+0xf/0x13
 [<c0106e14>] kernel_thread_helper+0x0/0xb
 [<c0106e19>] kernel_thread_helper+0x5/0xb

pdflush       S 00000000  5272  2968      1         24952  2144 (L-TLB)
ed309fac 00000046 00000000 00000000 00000000 00000000 00000000 00000000 
       00000000 00000000 00000000 00000000 00000000 d9dd7940 c2439c80 0000979e 
       24cb0f6d 00000df6 f42e8080 f42e8248 ed309fdc ed308000 ed309fd0 ffffe000 
Call Trace:
 [<c0140603>] __pdflush+0xab/0x20c
 [<c0140764>] pdflush+0x0/0x13
 [<c0140773>] pdflush+0xf/0x13
 [<c0106e14>] kernel_thread_helper+0x0/0xb
 [<c0106e19>] kernel_thread_helper+0x5/0xb

bash          S 00000001  5872 24952      1          2924  2968 (NOTLB)
f74fde7c 00000082 c2431c80 00000001 0000000f c026b232 f2d2d000 00000000 
       f74fde64 00000012 00000012 f2d2d000 c025d69a f2d2d000 c2431c80 0000607d 
       40ac8c4a 0000058f f7765310 f77654d8 f2d2d000 7fffffff e866c280 e9372000 
Call Trace:
 [<c026b232>] con_write+0x39/0x47
 [<c025d69a>] opost_block+0x112/0x1da
 [<c012981f>] schedule_timeout+0xbd/0xbf
 [<c013a333>] unlock_page+0x15/0x57
 [<c025eefe>] read_chan+0x734/0x8d2
 [<c025f1f3>] write_chan+0x157/0x21a
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c025a049>] tty_write+0x1a2/0x2e5
 [<c025e7ca>] read_chan+0x0/0x8d2
 [<c0259e76>] tty_read+0x14d/0x17e
 [<c0159a72>] vfs_read+0xb0/0x119
 [<c012cf88>] sys_rt_sigprocmask+0xff/0x18a
 [<c0159ced>] sys_read+0x42/0x63
 [<c0108f45>] sysenter_past_esp+0x52/0x71

xfs_syncd     S 00000001     0  2924      1         30337 24952 (L-TLB)
e806ff9c 00000046 c2439c80 00000001 0000000f 28719838 c04675a0 00000200 
       e806ff8c 00000092 f28b4080 4e9e8332 00000b79 f28b4080 c2439c80 000000e9 
       b528a8c7 00000df4 c2680080 c2680248 00e628e4 e806ffb0 00000000 00000000 
Call Trace:
 [<c01297cd>] schedule_timeout+0x6b/0xbf
 [<c0129759>] process_timeout+0x0/0x9
 [<c020a433>] syncd+0x46/0x8e
 [<c020a3ed>] syncd+0x0/0x8e
 [<c0106e19>] kernel_thread_helper+0x5/0xb

top           S 00000000     0 31785    144                     (NOTLB)
f2675eb0 00000082 c03b9980 00000000 00000000 5b1b2020 00000000 f28b46b0 
       00000010 c03ba580 00000000 000000d0 0000004b d9dd7940 c2439c80 00039b03 
       cab2b561 00000df6 f28b46b0 f28b4878 00e5e266 f2675ec4 00000001 00000001 
Call Trace:
 [<c01297cd>] schedule_timeout+0x6b/0xbf
 [<c0129759>] process_timeout+0x0/0x9
 [<c016d967>] do_select+0x1a8/0x2fa
 [<c016d619>] __pollwait+0x0/0xc7
 [<c016dda5>] sys_select+0x2c7/0x4e3
 [<c0108f45>] sysenter_past_esp+0x52/0x71

startx        S 00000001     0 25339    160 25352               (NOTLB)
f22f1f3c 00000082 c2431c80 00000001 0000000f 3809e067 00000000 e8192080 
       e81920a0 f4a29380 f42e8ce0 c0118e24 e8192080 f4a29380 c2431c80 00013571 
       f3d7927a 000005c8 f42e8ce0 f42e8ea8 fffffe00 f22f0000 f42e8d88 f42e8d88 
Call Trace:
 [<c0118e24>] do_page_fault+0x2f5/0x4df
 [<c0123cb0>] sys_wait4+0x1ac/0x272
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0123d9d>] sys_waitpid+0x27/0x2b
 [<c0108f45>] sysenter_past_esp+0x52/0x71

xinit         S 00000001    52 25352  25339 25353               (NOTLB)
d175df50 00000082 c2429c80 00000001 0000000f d175df88 d175c000 d175c000 
       d175df68 c011c903 00000000 f2d47940 c011c5f9 00000000 c2429c80 00007885 
       97dd49df 000005c9 f2d47940 f2d47b08 fffffe00 d175c000 f2d479e8 f2d479e8 
Call Trace:
 [<c011c903>] wait_for_completion+0x88/0xea
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0123cb0>] sys_wait4+0x1ac/0x272
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0107717>] sys_vfork+0x37/0x3b
 [<c0108f45>] sysenter_past_esp+0x52/0x71

X             S 00000000     0 25353  25352         25358       (NOTLB)
e7ab7eb0 00003086 c03b9980 00000000 00000000 ffffffff 00000000 e887b310 
       00000010 c03ba580 00000000 000000d0 e887b6a0 d9dd7940 c2439c80 00000862 
       01a2d48f 00000df7 e887b310 e887b4d8 00e79f46 e7ab7ec4 00000014 00000014 
Call Trace:
 [<c01297cd>] schedule_timeout+0x6b/0xbf
 [<c0129759>] process_timeout+0x0/0x9
 [<c016d967>] do_select+0x1a8/0x2fa
 [<c016d619>] __pollwait+0x0/0xc7
 [<c016dda5>] sys_select+0x2c7/0x4e3
 [<c0108f45>] sysenter_past_esp+0x52/0x71

xterm         S 00000001     0 25358  25352 25359         25353 (NOTLB)
c7c7feb0 00000086 c2431c80 00000001 0000000f c03b9980 00000000 00000000 
       c2431c80 00000000 f2d46080 00000010 c03ba580 00000000 c2431c80 0000506a 
       fbbf743e 00000823 f2d46080 f2d46248 00000000 7fffffff 00000005 00000005 
Call Trace:
 [<c012981f>] schedule_timeout+0xbd/0xbf
 [<c025f3c0>] normal_poll+0x10a/0x150
 [<c025b5a2>] tty_poll+0x65/0x73
 [<c016d967>] do_select+0x1a8/0x2fa
 [<c016d619>] __pollwait+0x0/0xc7
 [<c016dda5>] sys_select+0x2c7/0x4e3
 [<c0108f45>] sysenter_past_esp+0x52/0x71

twm           S 00000000     0 25359  25358         25360       (NOTLB)
ce895eb0 00000086 ce895f40 00000000 c013eccf c03b9980 00000000 00000000 
       00000046 00000000 e887b940 00000010 c03ba580 d0a9a080 c2431c80 000004a3 
       57ba73e0 00000df6 e887b940 e887bb08 00000000 7fffffff 00000004 00000004 
Call Trace:
 [<c013eccf>] __alloc_pages+0x9d/0x328
 [<c012981f>] schedule_timeout+0xbd/0xbf
 [<f88686f3>] unix_poll+0x2b/0x99 [unix]
 [<c02e4ccc>] sock_poll+0x29/0x31
 [<c016d967>] do_select+0x1a8/0x2fa
 [<c016d619>] __pollwait+0x0/0xc7
 [<c016dda5>] sys_select+0x2c7/0x4e3
 [<c02e4af8>] sock_ioctl+0x114/0x2bf
 [<c0108f45>] sysenter_past_esp+0x52/0x71

xclock        S 00000000     0 25360  25358         25361 25359 (NOTLB)
f2e3beb0 00000086 f2e3bf40 00000000 c013eccf c03b9980 00000000 00000000 
       00000000 00000000 d0a9a080 00000282 c03ba580 d0a9a080 c2429c80 00000130 
       fee8bb08 00000def f2d46ce0 f2d46ea8 00e64b16 f2e3bec4 00000004 00000004 
Call Trace:
 [<c013eccf>] __alloc_pages+0x9d/0x328
 [<c01297cd>] schedule_timeout+0x6b/0xbf
 [<c0129759>] process_timeout+0x0/0x9
 [<c016d967>] do_select+0x1a8/0x2fa
 [<c016d619>] __pollwait+0x0/0xc7
 [<c016dda5>] sys_select+0x2c7/0x4e3
 [<c02e4af8>] sock_ioctl+0x114/0x2bf
 [<c0108f45>] sysenter_past_esp+0x52/0x71

xterm         S 00000000     0 25361  25358 25364   25362 25360 (NOTLB)
f72fdeb0 00000082 f72fdf40 00000000 c013eccf c03b9980 00000000 00000000 
       00000282 00000000 dd865940 00000010 c03ba580 dd865940 c2441c80 0000017f 
       43b6033f 00000df6 c26806b0 c2680878 00000000 7fffffff 00000005 00000005 
Call Trace:
 [<c013eccf>] __alloc_pages+0x9d/0x328
 [<c012981f>] schedule_timeout+0xbd/0xbf
 [<c025f3c0>] normal_poll+0x10a/0x150
 [<c025b5a2>] tty_poll+0x65/0x73
 [<c016d967>] do_select+0x1a8/0x2fa
 [<c016d619>] __pollwait+0x0/0xc7
 [<c016dda5>] sys_select+0x2c7/0x4e3
 [<c0108f45>] sysenter_past_esp+0x52/0x71

xterm         R 00000000  6668 25362  25358 25363   25365 25361 (NOTLB)
ea5bbeb0 00000082 ea5bbf40 00000000 c013eccf c03b9980 00000000 00000000 
       00000282 00000000 f28b4080 00000010 c03ba580 f28b4080 c2429c80 00000118 
       006a3eef 00000df7 e887a6b0 e887a878 00000000 7fffffff 00000005 00000005 
Call Trace:
 [<c013eccf>] __alloc_pages+0x9d/0x328
 [<c012981f>] schedule_timeout+0xbd/0xbf
 [<c025f3c0>] normal_poll+0x10a/0x150
 [<c025b5a2>] tty_poll+0x65/0x73
 [<c016d967>] do_select+0x1a8/0x2fa
 [<c016d619>] __pollwait+0x0/0xc7
 [<c016dda5>] sys_select+0x2c7/0x4e3
 [<c0108f45>] sysenter_past_esp+0x52/0x71

bash          S 00000001     0 25363  25362 24011               (NOTLB)
e78a3f3c 00000086 c2441c80 00000001 0000000f 7c645065 00000000 cf006800 
       cf006820 f357c480 f7764080 c0118e24 cf006800 f357c480 c2441c80 00004399 
       c6195e5e 00000dc4 f7764080 f7764248 fffffe00 e78a2000 f7764128 f7764128 
Call Trace:
 [<c0118e24>] do_page_fault+0x2f5/0x4df
 [<c0123cb0>] sys_wait4+0x1ac/0x272
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c012cdfd>] sigprocmask+0x64/0xf0
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0123d9d>] sys_waitpid+0x27/0x2b
 [<c0108f45>] sysenter_past_esp+0x52/0x71

bash          S 00000001     0 25365  25358               25362 (NOTLB)
cc981e7c 00000082 c2429c80 00000001 0000000f 00000000 00000003 00000008 
       00000008 d0837000 cc981e6c 00000008 c025d69a e1ce0ce0 c2429c80 0000201b 
       f6c9db7d 00000821 f7d66ce0 f7d66ea8 d0837000 7fffffff ea390080 d0675000 
Call Trace:
 [<c025d69a>] opost_block+0x112/0x1da
 [<c012981f>] schedule_timeout+0xbd/0xbf
 [<c013a333>] unlock_page+0x15/0x57
 [<c025eefe>] read_chan+0x734/0x8d2
 [<c011c285>] schedule+0x36e/0x69f
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c025a049>] tty_write+0x1a2/0x2e5
 [<c025e7ca>] read_chan+0x0/0x8d2
 [<c0259e76>] tty_read+0x14d/0x17e
 [<c0159a72>] vfs_read+0xb0/0x119
 [<c012cf88>] sys_rt_sigprocmask+0xff/0x18a
 [<c0159ced>] sys_read+0x42/0x63
 [<c0108f45>] sysenter_past_esp+0x52/0x71

bash          S 00000001     0 25364  25361 26236               (NOTLB)
d185ff3c 00000086 c2429c80 00000001 0000000f 7cd15067 00000000 c2793800 
       c2793820 f50cc180 e887a080 c0118e24 c2793800 f50cc180 c2429c80 000036f8 
       ee909e0f 00000bb9 e887a080 e887a248 fffffe00 d185e000 e887a128 e887a128 
Call Trace:
 [<c0118e24>] do_page_fault+0x2f5/0x4df
 [<c0123cb0>] sys_wait4+0x1ac/0x272
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c012cdfd>] sigprocmask+0x64/0xf0
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0123d9d>] sys_waitpid+0x27/0x2b
 [<c0108f45>] sysenter_past_esp+0x52/0x71

xterm         S 00000000     0 30337      1 30338          2924 (NOTLB)
ce931eb0 00000086 ce931f40 00000000 c013eccf c03b9980 00000000 00000000 
       c040ad88 00000000 d9dd66b0 00000010 c03ba580 d9dd7940 c2439c80 00000dc6 
       feaf377c 00000df6 d9dd66b0 d9dd6878 00000000 7fffffff 00000005 00000005 
Call Trace:
 [<c013eccf>] __alloc_pages+0x9d/0x328
 [<c012981f>] schedule_timeout+0xbd/0xbf
 [<c025f3c0>] normal_poll+0x10a/0x150
 [<c025b5a2>] tty_poll+0x65/0x73
 [<c016d967>] do_select+0x1a8/0x2fa
 [<c016d619>] __pollwait+0x0/0xc7
 [<c016dda5>] sys_select+0x2c7/0x4e3
 [<c0108f45>] sysenter_past_esp+0x52/0x71

bash          S C03B9F80     0 30338  30337                     (NOTLB)
cbf7fe7c 00000086 c013eccf c03b9f80 00000000 00000000 c03ba000 0000001f 
       0000001f d9239000 cbf7fe83 0000001f c025d69a d9dd7940 c2431c80 00001532 
       c2d7f35b 00000dee c3a0e6b0 c3a0e878 d9239000 7fffffff cd334a80 eabf7000 
Call Trace:
 [<c013eccf>] __alloc_pages+0x9d/0x328
 [<c025d69a>] opost_block+0x112/0x1da
 [<c012981f>] schedule_timeout+0xbd/0xbf
 [<c014f013>] page_remove_rmap+0x19/0x1a9
 [<c013a333>] unlock_page+0x15/0x57
 [<c025eefe>] read_chan+0x734/0x8d2
 [<c025f1f3>] write_chan+0x157/0x21a
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c025a049>] tty_write+0x1a2/0x2e5
 [<c025e7ca>] read_chan+0x0/0x8d2
 [<c0259e76>] tty_read+0x14d/0x17e
 [<c0159a72>] vfs_read+0xb0/0x119
 [<c012cf88>] sys_rt_sigprocmask+0xff/0x18a
 [<c0159ced>] sys_read+0x42/0x63
 [<c0108f45>] sysenter_past_esp+0x52/0x71

make          S 00000001     0 26236  25364 26238               (NOTLB)
cfcb9f50 00000082 c2431c80 00000001 0000000f cfcb9f88 cfcb8000 cfcb8000 
       cfcb9f68 c011c903 00000000 dde56080 c011c5f9 00000000 c2431c80 00003f1c 
       ef27ee50 00000bb9 dde56080 dde56248 fffffe00 cfcb8000 dde56128 dde56128 
Call Trace:
 [<c011c903>] wait_for_completion+0x88/0xea
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0123cb0>] sys_wait4+0x1ac/0x272
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c012cf43>] sys_rt_sigprocmask+0xba/0x18a
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0108f45>] sysenter_past_esp+0x52/0x71

make          S 00000001     0 26238  26236 26239               (NOTLB)
ca36ff50 00000082 c2431c80 00000001 0000000f daaf8ce0 c0118e24 da610080 
       c5bac140 400cbd60 00000000 00000000 400cbd60 00000000 c2431c80 00008105 
       ef92f36e 00000bb9 daaf8ce0 daaf8ea8 fffffe00 ca36e000 daaf8d88 daaf8d88 
Call Trace:
 [<c0118e24>] do_page_fault+0x2f5/0x4df
 [<c0123cb0>] sys_wait4+0x1ac/0x272
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c012cf43>] sys_rt_sigprocmask+0xba/0x18a
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0108f45>] sysenter_past_esp+0x52/0x71

sh            S 00000001     0 26239  26238  3291               (NOTLB)
c5429f3c 00000086 c2429c80 00000001 0000000f 51cd7067 00000000 cf006d00 
       cf006d20 c5bac580 c3a0f940 c0118e24 cf006d00 c5bac580 c2429c80 00011bc1 
       b5359ea7 00000dd4 c3a0f940 c3a0fb08 fffffe00 c5428000 c3a0f9e8 c3a0f9e8 
Call Trace:
 [<c0118e24>] do_page_fault+0x2f5/0x4df
 [<c0123cb0>] sys_wait4+0x1ac/0x272
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0123d9d>] sys_waitpid+0x27/0x2b
 [<c0108f45>] sysenter_past_esp+0x52/0x71

bash          S FFFDF3B4     0 24011  25363  3310               (NOTLB)
e3129f3c 00000082 080ed688 fffdf3b4 e6730080 56a73067 00000000 cf006580 
       cf0065a0 d85abec0 e13626b0 c0118e24 cf006580 f2d466b0 c2431c80 00001fb9 
       0c59223a 00000df6 e13626b0 e1362878 fffffe00 e3128000 e1362758 e1362758 
Call Trace:
 [<c0118e24>] do_page_fault+0x2f5/0x4df
 [<c0123cb0>] sys_wait4+0x1ac/0x272
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c012cdfd>] sigprocmask+0x64/0xf0
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0123d9d>] sys_waitpid+0x27/0x2b
 [<c0108f45>] sysenter_past_esp+0x52/0x71

sh            S 00000001     0  3291  26239  3292               (NOTLB)
e7e73f3c 00000086 c2441c80 00000001 0000000f 56e54067 00000000 c6045300 
       c6045320 f4a29300 c3a0e080 c0118e24 c6045300 f4a29300 c2441c80 0001acef 
       b536258b 00000dd4 c3a0e080 c3a0e248 fffffe00 e7e72000 c3a0e128 c3a0e128 
Call Trace:
 [<c0118e24>] do_page_fault+0x2f5/0x4df
 [<c0123cb0>] sys_wait4+0x1ac/0x272
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0123d9d>] sys_waitpid+0x27/0x2b
 [<c0108f45>] sysenter_past_esp+0x52/0x71

make          S 00000001    16  3292   3291  3293               (NOTLB)
ddbe1f50 00000086 c2429c80 00000001 0000000f f28b4ce0 c0118e24 f7c50800 
       e13619c0 400cbd60 00000000 00000000 400cbd60 00000000 c2429c80 00005d98 
       b5aec1f8 00000dd4 f28b4ce0 f28b4ea8 fffffe00 ddbe0000 f28b4d88 f28b4d88 
Call Trace:
 [<c0118e24>] do_page_fault+0x2f5/0x4df
 [<c0123cb0>] sys_wait4+0x1ac/0x272
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c012cf43>] sys_rt_sigprocmask+0xba/0x18a
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0108f45>] sysenter_past_esp+0x52/0x71

sh            S 00000001     0  3293   3292  3297               (NOTLB)
d3df3f3c 00000086 c2441c80 00000001 0000000f 415d2067 00000000 d3de8d00 
       d3de8d20 f48a3b80 e1bcd310 c0118e24 d3de8d00 f48a3b80 c2441c80 0001b2fa 
       b600937f 00000dd4 e1bcd310 e1bcd4d8 fffffe00 d3df2000 e1bcd3b8 e1bcd3b8 
Call Trace:
 [<c0118e24>] do_page_fault+0x2f5/0x4df
 [<c0123cb0>] sys_wait4+0x1ac/0x272
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0123d9d>] sys_waitpid+0x27/0x2b
 [<c0108f45>] sysenter_past_esp+0x52/0x71

sh            S 00000001    16  3297   3293  3298               (NOTLB)
d0d61f3c 00000086 c2431c80 00000001 0000000f 66ffa067 00000000 da610d00 
       da610d20 df58d440 f42e9940 c0118e24 da610d00 df58d440 c2431c80 0001116c 
       b60b08f0 00000dd4 f42e9940 f42e9b08 fffffe00 d0d60000 f42e99e8 f42e99e8 
Call Trace:
 [<c0118e24>] do_page_fault+0x2f5/0x4df
 [<c0123cb0>] sys_wait4+0x1ac/0x272
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0123d9d>] sys_waitpid+0x27/0x2b
 [<c0108f45>] sysenter_past_esp+0x52/0x71

make          S DF68FE74  4048  3298   3297  3300               (NOTLB)
df68feb4 00000086 cffe7380 df68fe74 00000000 00000000 f7e58a9c bf829f33 
       df68e000 f15a028c 00000001 f1613080 dfa7b000 d0a9a080 c2429c80 0002fe53 
       b73f8284 00000dd4 e1362080 e1362248 cf5659ec cf565980 df68fedc 00000001 
Call Trace:
 [<c0166da4>] pipe_wait+0x7c/0x9c
 [<c011e87b>] autoremove_wake_function+0x0/0x4f
 [<c011e87b>] autoremove_wake_function+0x0/0x4f
 [<c0164425>] cp_new_stat64+0x114/0x129
 [<c0166fa2>] pipe_readv+0x1de/0x2d1
 [<c01670cc>] pipe_read+0x37/0x3b
 [<c0159a72>] vfs_read+0xb0/0x119
 [<c0159ced>] sys_read+0x42/0x63
 [<c0108f45>] sysenter_past_esp+0x52/0x71

sh            S FFFDF384     0  3300   3298  3303    3301       (NOTLB)
dccb1f3c 00000082 080e1384 fffdf384 ca68e080 4d812067 00000000 ef993580 
       ef9935a0 d85abe00 d9dd7310 c0118e24 ef993580 d9dd7940 c2431c80 000192b8 
       b9bd1466 00000dd4 d9dd7310 d9dd74d8 fffffe00 dccb0000 d9dd73b8 d9dd73b8 
Call Trace:
 [<c0118e24>] do_page_fault+0x2f5/0x4df
 [<c0123cb0>] sys_wait4+0x1ac/0x272
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0123d9d>] sys_waitpid+0x27/0x2b
 [<c0108f45>] sysenter_past_esp+0x52/0x71

sh            S FFFC3384   188  3301   3298  3305    3302  3300 (NOTLB)
c5de1f3c 00000082 080e1384 fffc3384 dbd51080 4640b067 00000000 d3de8580 
       d3de85a0 f432d100 dd865310 c0118e24 d3de8580 dd865940 c2441c80 000226f6 
       b6fa3445 00000dd4 dd865310 dd8654d8 fffffe00 c5de0000 dd8653b8 dd8653b8 
Call Trace:
 [<c0118e24>] do_page_fault+0x2f5/0x4df
 [<c0123cb0>] sys_wait4+0x1ac/0x272
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0123d9d>] sys_waitpid+0x27/0x2b
 [<c0108f45>] sysenter_past_esp+0x52/0x71

sh            S FFFD1384     0  3302   3298  3306    3304  3301 (NOTLB)
d9f77f3c 00000086 080e1384 fffd1384 db180080 6b043067 00000000 d3de8800 
       d3de8820 f432d180 d0a9b310 c0118e24 d3de8800 f28b4080 c2439c80 00018c37 
       b788d9a8 00000dd4 d0a9b310 d0a9b4d8 fffffe00 d9f76000 d0a9b3b8 d0a9b3b8 
Call Trace:
 [<c0118e24>] do_page_fault+0x2f5/0x4df
 [<c0123cb0>] sys_wait4+0x1ac/0x272
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0123d9d>] sys_waitpid+0x27/0x2b
 [<c0108f45>] sysenter_past_esp+0x52/0x71

sh            S FFFED384     0  3304   3298  3307          3302 (NOTLB)
d6115f3c 00000086 080e1384 fffed384 d303e080 38f65067 00000000 c6729800 
       c6729820 d8512e40 f42e9310 c0118e24 c6729800 e1362080 c2429c80 000197cf 
       b73c8431 00000dd4 f42e9310 f42e94d8 fffffe00 d6114000 f42e93b8 f42e93b8 
Call Trace:
 [<c0118e24>] do_page_fault+0x2f5/0x4df
 [<c0123cb0>] sys_wait4+0x1ac/0x272
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c011c5f9>] default_wake_function+0x0/0x12
 [<c0123d9d>] sys_waitpid+0x27/0x2b
 [<c0108f45>] sysenter_past_esp+0x52/0x71

sh            R 0000001B     0  3303   3300                     (NOTLB)
ffffffff c011bf7f c243b674 0000001b d9dd7940 d0ff9d1c d0ff9d1c 00000000 
       00000001 c040ad88 0000000a 00000046 00000046 c040ad88 c0464108 00000002 
       c0464104 d0ff9d6c c0115f1d 00000000 00031324 00000000 f641ee20 00000ff0 
Call Trace:
 [<c011bf7f>] schedule+0x68/0x69f
 [<c0115f1d>] smp_apic_timer_interrupt+0xdd/0x145
 [<c0109986>] apic_timer_interrupt+0x1a/0x20
 [<c013ab45>] do_generic_mapping_read+0x78/0x554
 [<c013b021>] file_read_actor+0x0/0xec
 [<c013b2f9>] __generic_file_aio_read+0x1ec/0x21e
 [<c013b021>] file_read_actor+0x0/0xec
 [<c013eb5c>] buffered_rmqueue+0xed/0x1c3
 [<c0208b06>] xfs_read+0x174/0x2a0
 [<c0204b51>] linvfs_read+0x8d/0x9f
 [<c0159996>] do_sync_read+0x8b/0xb7
 [<c0118e24>] do_page_fault+0x2f5/0x4df
 [<c014cf8f>] do_brk+0x13d/0x21b
 [<c0159a72>] vfs_read+0xb0/0x119
 [<c012cf88>] sys_rt_sigprocmask+0xff/0x18a
 [<c0159ced>] sys_read+0x42/0x63
 [<c0108f45>] sysenter_past_esp+0x52/0x71

sh            R current      0  3306   3302                     (NOTLB)
00000000 cd58fee8 f7c22740 04000001 00000000 cd58fee8 c010b16a 00000001 
       c049bda0 cd58fee8 c0403080 cd58e000 00000001 f7c22740 c010b51d 00000001 
       cd58fee8 f7c22740 c0403098 f7c22740 00031324 00000000 f641ee20 cd58ff7c 
Call Trace:
 [<c010b16a>] handle_IRQ_event+0x3a/0x64
 [<c010b51d>] do_IRQ+0xb8/0x192
 [<c0109904>] common_interrupt+0x18/0x20
 [<c0163c62>] generic_fillattr+0x86/0xa6
 [<c020817b>] linvfs_getattr+0x27/0x3f
 [<c0163cbb>] vfs_getattr+0x39/0x9a
 [<c0163e08>] vfs_fstat+0x39/0x50
 [<c01644c7>] sys_fstat64+0x1b/0x39
 [<c0108f45>] sysenter_past_esp+0x52/0x71

sh            R 0000001B    12  3305   3301                     (NOTLB)
c01294d2 00000000 c2443674 0000001b dd865940 e3f8be98 e3f8be98 00000000 
       00000001 c040ad88 0000000a 00000046 00000046 c040ad88 c0464108 00000003 
       c0464104 e3f8bee8 c0115f1d 00000000 00031324 00000000 f641ee20 e3f8bf7c 
Call Trace:
 [<c01294d2>] run_timer_softirq+0xe5/0x1c2
 [<c0115f1d>] smp_apic_timer_interrupt+0xdd/0x145
 [<c0109986>] apic_timer_interrupt+0x1a/0x20
 [<c0163c62>] generic_fillattr+0x86/0xa6
 [<c020817b>] linvfs_getattr+0x27/0x3f
 [<c0163cbb>] vfs_getattr+0x39/0x9a
 [<c0163e08>] vfs_fstat+0x39/0x50
 [<c01644c7>] sys_fstat64+0x1b/0x39
 [<c0108f45>] sysenter_past_esp+0x52/0x71

sh            R 16D8447A    12  3307   3304                     (NOTLB)
e75ebd64 00000086 c2433674 16d8447a d0a9a080 e75ebd1c e75ebd1c 00000000 
       00000001 c040ad88 0000000a 00000046 00000046 c040ad88 c0464108 00000001 
       c0464104 e75ebd6c c0115f1d 00000000 00031324 00000000 f641ee20 00000000 
Call Trace:
 [<c0115f1d>] smp_apic_timer_interrupt+0xdd/0x145
 [<c0109986>] apic_timer_interrupt+0x1a/0x20
 [<c013ab42>] do_generic_mapping_read+0x75/0x554
 [<c013b021>] file_read_actor+0x0/0xec
 [<c013b2f9>] __generic_file_aio_read+0x1ec/0x21e
 [<c013b021>] file_read_actor+0x0/0xec
 [<c013eb5c>] buffered_rmqueue+0xed/0x1c3
 [<c0208b06>] xfs_read+0x174/0x2a0
 [<c013bb83>] filemap_nopage+0x3c2/0x456
 [<c0204b51>] linvfs_read+0x8d/0x9f
 [<c0159996>] do_sync_read+0x8b/0xb7
 [<c0118e24>] do_page_fault+0x2f5/0x4df
 [<c0205776>] linvfs_ioctl+0x41/0x80
 [<c0159a72>] vfs_read+0xb0/0x119
 [<c0159ced>] sys_read+0x42/0x63
 [<c0108f45>] sysenter_past_esp+0x52/0x71

mplayer       S F8931EAD     0  3310  24011                     (NOTLB)
d3643f50 00200082 085d6cc0 f8931ead f76e2480 d3643f70 d3643f70 00000000 
       00000009 d3642000 d0a9a080 e70b0ca8 e70b0ca4 d0a9a080 c2431c80 00000401 
       179eb5c9 00000df7 f2d466b0 f2d46878 00e5dbc5 d3643f64 0000000c 00000000 
Call Trace:
 [<f8931ead>] snd_pcm_playback_ioctl1+0x5e/0x45d [snd_pcm]
 [<c01297cd>] schedule_timeout+0x6b/0xbf
 [<f89e06e3>] snd_pcm_oss_get_space+0xab/0x160 [snd_pcm_oss]
 [<c0129759>] process_timeout+0x0/0x9
 [<c01299e1>] sys_nanosleep+0x103/0x1a9
 [<c0108f45>] sysenter_past_esp+0x52/0x71


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: 99% System load
  2004-02-17 21:16   ` Florian Schanda
@ 2004-02-17 21:43     ` Andrew Morton
  0 siblings, 0 replies; 6+ messages in thread
From: Andrew Morton @ 2004-02-17 21:43 UTC (permalink / raw)
  To: Florian Schanda; +Cc: linux-kernel

Florian Schanda <ma1flfs@bath.ac.uk> wrote:
>
> Finally it happened again. I attached the output of that dmesg.
> 
> > It would help if you could make a note of the PIDs of the hung processes,
> > so they can be correlated with the sysrq output.
> 
> The PIDs hanging are 3303, 3305, 3306, 3307.

It _looks_ like all four CPUs are madly taking timer interrupts.  Which
tends to point at some APIC problem, failing to clear the interrupt source.
But if that happened one wouldn't expect userspace to remain in a runnable
state, and you say that you can still run commands.

> An advance warning: I had the nvidia driver (yes, I know, closed source makes 
> debugging impossible) loaded, and before the trace begins, there are some 
> messages of that module:

It is probably unrelated, but I'd suggest that you not use the nvidia
driver for a while, verify that the lockup happens without it loaded.


^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2004-02-17 21:48 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2004-02-11 14:22 99% System load Florian Schanda
2004-02-11 14:30 ` Florian Schanda
2004-02-11 19:02 ` Michel Angelo da Silva Pereira
2004-02-12  7:44 ` Andrew Morton
2004-02-17 21:16   ` Florian Schanda
2004-02-17 21:43     ` Andrew Morton

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®