* Kernel hang on MX server
@ 2004-04-14 0:14 Josh Logan
2004-04-14 0:41 ` Alexander Hoogerhuis
2004-04-14 3:08 ` Andrew Morton
0 siblings, 2 replies; 3+ messages in thread
From: Josh Logan @ 2004-04-14 0:14 UTC (permalink / raw)
To: linux-kernel
[-- Attachment #1: Type: TEXT/PLAIN, Size: 2302 bytes --]
I have 2 IBM x305's which hang every few days in identical ways. Both are
running postfix (with amavis-new, spamassassin and clamav from
backports.org) and bind9. They are running debian stable.
The IDE controler is: ServerWorks CSB5 IDE
The Kernel is 2.6.4
I have tried 2.4 kernels with ext3. I have also tried 2.6.4 kernels with
ext3, jfs, and xfs, but all three FS show the same errors.
Currently trying ext2 on one of the machines.
The systems is running software raid1 between both hard drives.
/dev/hda1 and /dev/hdc1 make up md0 which is mount on /
/dev/hda3 and /dev/hdc3 make up md1 which used LVM to make /usr /home /var
and /var/spool/postfix
I have also tried mount --bind /postfix /var/spool/postfix to take LVM out
of the picture. That did not help. I have not tried to break the mirror
and only use 1 IDE drive.
Please see the attached files for better formating, and all of the
running commands.
The following processes are in uninterruptible sleep (usually IO) via ps:
root@ares:~# ps axln | grep D
F UID PID PPID PRI NI VSZ RSS WCHAN STAT TTY TIME COMMAND
1 0 103 1 15 0 0 0 11e11e D ? 0:07
[kjournald]
1 0 248 1 15 0 0 0 11e11e D ? 0:04
[kjournald]
5 0 332 1 15 0 1432 672 86e7cf Ds ? 0:24
/sbin/syslogd
4 102 9574 9572 16 0 2596 1176 109ea5 D ? 0:24 qmgr -l
-t fifo -u -c
4 102 26473 9572 17 0 2544 1112 86e7cf D ? 0:00 cleanup
-z -n pre-cleanup -t unix -u -c -o virtual_alias_maps -o canonical_maps
-o sender_canonical_maps -o recipient_canonical_maps -o masquerade_domains
4 102 26509 9572 16 0 2464 1020 109ea5 D ? 0:00 showq
-t unix -u -c
Decoding the WCHAN:
11e11e ->
c011e0f4 T yield
c011e110 T io_schedule
c011e128 T io_schedule_timeout
c011e144 T sys_sched_get_priority_max
86e7cf ->
not in System.map, but PS gave: log_wait_commit
109ea5 ->
c0109df4 t __check_printsym_format
c0109e00 T __up
c0109e18 T __down
c0109f24 T __down_interruptible
Where PS was run as:
ps -o pid,nwchan,wchan=WIDE-WCHAN-COLUMN -o comm -o state axf
I'll happily provide more information, but not sure what to try next.
Looks like it may be a driver issue, blocking IO writes to the drives.
Later, JOSH
[-- Attachment #2: Type: TEXT/plain, Size: 19584 bytes --]
root@ares:~# ./ps__axln
PID WCHAN WIDE-WCHAN-COLUMN COMMAND S
1 127a2c schedule_timeout init S
2 1240ce ksoftirqd ksoftirqd/0 S
3 12e369 worker_thread events/0 S
4 12e369 worker_thread \_ kblockd/0 S
5 13c688 pdflush \_ pdflush S
6 13c688 pdflush \_ pdflush S
8 12e369 worker_thread \_ aio/0 S
252 12e369 worker_thread \_ xfslogd/0 S
253 12e369 worker_thread \_ xfsdatad/0 S
7 1411fc kswapd kswapd0 S
10 1f0307 serio_thread kseriod S
89 84cea2 md_thread md0_raid1 S
103 86e14d kjournald kjournald S
159 9159f7 hub_thread khubd S
173 84cea2 md_thread md1_raid1 S
247 86e14d kjournald kjournald S
248 11e11e io_schedule kjournald D
249 86e14d kjournald kjournald S
254 127a2c schedule_timeout xfsbufd S
332 86e7cf log_wait_commit syslogd D
335 120608 syslog klogd S
341 10a267 rt_sigsuspend named S
342 127a2c schedule_timeout \_ named S
344 1279b8 schedule_timeout \_ named S
345 127a2c schedule_timeout \_ named S
346 1279b8 schedule_timeout \_ named S
367 1279b8 schedule_timeout clamd S
553 127a2c schedule_timeout \_ clamd S
400 12b700 pause freshclam S
407 10a267 rt_sigsuspend inetd S
512 1279b8 schedule_timeout sshd S
2029 1279b8 schedule_timeout \_ sshd S
2031 1279b8 schedule_timeout | \_ sshd S
2032 122f5e wait4 | \_ bash S
2035 - - | \_ R
26461 - - | R
26462 - - | R
3464 1279b8 schedule_timeout \_ sshd S
3467 1279b8 schedule_timeout | \_ bash S
3645 1279b8 schedule_timeout \_ sshd S
3648 1279b8 schedule_timeout | \_ bash S
11651 1279b8 schedule_timeout \_ sshd S
11653 1279b8 schedule_timeout \_ sshd S
11654 122f5e wait4 \_ bash S
11655 1279b8 schedule_timeout \_ S
515 1279b8 schedule_timeout ntpd S
522 127a2c schedule_timeout mdadm S
526 127a2c schedule_timeout atd S
529 127a2c schedule_timeout cron S
532 127a2c schedule_timeout mailgraph.pl S
536 1279b8 schedule_timeout bash S
537 1279b8 schedule_timeout getty S
538 1279b8 schedule_timeout getty S
539 1279b8 schedule_timeout getty S
540 1279b8 schedule_timeout getty S
541 1279b8 schedule_timeout getty S
9572 127a2c schedule_timeout master S
9574 127a2c schedule_timeout \_ qmgr S
27399 1279b8 schedule_timeout \_ trivial-rewr S
26052 1279b8 schedule_timeout \_ proxymap S
26332 1279b8 schedule_timeout \_ smtpd S
26337 127a2c schedule_timeout \_ lmtp S
26343 1279b8 schedule_timeout \_ smtpd S
26368 127a2c schedule_timeout \_ pickup S
26393 127a2c schedule_timeout \_ lmtp S
26400 127a2c schedule_timeout \_ cleanup S
26401 163814 flock \_ smtp S
26402 127a2c schedule_timeout \_ smtpd S
26403 127a2c schedule_timeout \_ smtp S
26407 127a2c schedule_timeout \_ bounce S
26408 127a2c schedule_timeout \_ cleanup S
26409 1279b8 schedule_timeout \_ smtpd S
26414 1279b8 schedule_timeout \_ smtpd S
26415 1279b8 schedule_timeout \_ smtpd S
26416 127a2c schedule_timeout \_ smtpd S
26417 127a2c schedule_timeout \_ smtpd S
26418 127a2c schedule_timeout \_ smtpd S
26419 127a2c schedule_timeout \_ smtpd S
26420 127a2c schedule_timeout \_ smtpd S
26421 127a2c schedule_timeout \_ smtpd S
26422 127a2c schedule_timeout \_ smtpd S
26423 127a2c schedule_timeout \_ smtpd S
26424 127a2c schedule_timeout \_ smtpd S
26425 127a2c schedule_timeout \_ smtpd S
26426 127a2c schedule_timeout \_ smtpd S
26427 127a2c schedule_timeout \_ smtpd S
26428 127a2c schedule_timeout \_ smtpd S
26429 127a2c schedule_timeout \_ smtpd S
26430 127a2c schedule_timeout \_ smtpd S
26431 127a2c schedule_timeout \_ smtpd S
26432 127a2c schedule_timeout \_ smtpd S
26433 127a2c schedule_timeout \_ smtpd S
26434 127a2c schedule_timeout \_ smtpd S
26435 127a2c schedule_timeout \_ smtpd S
26436 127a2c schedule_timeout \_ smtpd S
26437 127a2c schedule_timeout \_ smtpd S
26438 127a2c schedule_timeout \_ smtpd S
26439 127a2c schedule_timeout \_ smtpd S
26443 127a2c schedule_timeout \_ smtpd S
26444 127a2c schedule_timeout \_ smtpd S
26445 127a2c schedule_timeout \_ smtpd S
26446 127a2c schedule_timeout \_ smtpd S
26447 127a2c schedule_timeout \_ smtpd S
26451 127a2c schedule_timeout \_ smtpd S
26452 127a2c schedule_timeout \_ smtpd S
26453 127a2c schedule_timeout \_ smtpd S
26454 127a2c schedule_timeout \_ smtpd S
26456 127a2c schedule_timeout \_ smtpd S
26459 127a2c schedule_timeout \_ smtpd S
26460 127a2c schedule_timeout \_ smtpd S
16258 127a2c schedule_timeout amavisd-new S
26338 127a2c schedule_timeout \_ amavisd-new S
26377 1279b8 schedule_timeout \_ amavisd-new S
root@ares:~# ps axln
F UID PID PPID PRI NI VSZ RSS WCHAN STAT TTY TIME COMMAND
4 0 1 0 16 0 1276 488 127a2c S ? 0:00 init [2]
1 0 2 1 34 19 0 0 1240ce SN ? 0:00 [ksoftirqd/0]
1 0 3 1 5 -10 0 0 12e369 S< ? 0:00 [events/0]
1 0 4 3 5 -10 0 0 12e369 S< ? 0:00 [kblockd/0]
1 0 5 3 13 -10 0 0 13c688 S< ? 0:00 [pdflush]
1 0 6 3 5 -10 0 0 13c688 S< ? 0:00 [pdflush]
1 0 8 3 5 -10 0 0 12e369 S< ? 0:00 [aio/0]
1 0 7 1 15 0 0 0 1411fc S ? 0:00 [kswapd0]
1 0 10 1 25 0 0 0 1f0307 S ? 0:00 [kseriod]
1 0 89 1 15 0 0 0 84cea2 S ? 0:03 [md0_raid1]
1 0 103 1 15 0 0 0 11e11e D ? 0:07 [kjournald]
1 0 159 1 19 0 0 0 9159f7 S ? 0:00 [khubd]
1 0 173 1 15 0 0 0 84cea2 S ? 0:01 [md1_raid1]
1 0 247 1 15 0 0 0 86e14d S ? 0:00 [kjournald]
1 0 248 1 15 0 0 0 11e11e D ? 0:04 [kjournald]
1 0 249 1 15 0 0 0 86e14d S ? 0:00 [kjournald]
1 0 252 3 5 -10 0 0 12e369 S< ? 0:00 [xfslogd/0]
1 0 253 3 6 -10 0 0 12e369 S< ? 0:00 [xfsdatad/0]
1 0 254 1 15 0 0 0 127a2c S ? 0:00 [xfsbufd]
5 0 332 1 16 0 1432 672 11e11e Ds ? 0:24 /sbin/syslogd
5 0 335 1 15 0 2236 1408 120608 Ss ? 0:00 /sbin/klogd
1 105 341 1 18 0 18636 10732 10a267 Ss ? 0:00 /usr/sbin/nam
1 105 342 341 16 0 18636 10732 127a2c S ? 0:00 /usr/sbin/nam
5 105 344 342 16 0 18636 10732 10a267 S ? 3:48 /usr/sbin/nam
1 105 345 342 16 0 18636 10732 127a2c S ? 0:01 /usr/sbin/nam
1 105 346 342 16 0 18636 10732 1279b8 S ? 0:34 /usr/sbin/nam
1 103 367 1 16 0 13744 12800 1279b8 Ss ? 0:02 /usr/sbin/cla
1 104 400 1 15 0 2032 1004 12b700 Ss ? 0:00 /usr/bin/fres
5 0 407 1 21 0 1252 432 10a267 Ss ? 0:00 /usr/sbin/ine
5 0 512 1 16 0 2788 1212 1279b8 Ss ? 0:00 /usr/sbin/ssh
5 0 515 1 16 0 1980 1972 1279b8 SLs ? 0:00 /usr/sbin/ntp
4 0 522 1 16 0 1304 392 127a2c S ? 0:00 /sbin/mdadm -
1 1 526 1 16 0 1388 584 127a2c Ss ? 0:00 /usr/sbin/atd
5 0 529 1 16 0 1656 688 127a2c Ss ? 0:00 /usr/sbin/cro
5 0 532 1 16 0 5084 3636 127a2c Ss ? 0:20 /usr/bin/perl
4 0 536 1 16 0 2236 1268 1279b8 Ss+ tty1 0:00 -bash
4 0 537 1 16 0 1256 472 1279b8 Ss+ tty2 0:00 /sbin/getty 3
4 0 538 1 16 0 1256 472 1279b8 Ss+ tty3 0:00 /sbin/getty 3
4 0 539 1 16 0 1256 472 1279b8 Ss+ tty4 0:00 /sbin/getty 3
4 0 540 1 16 0 1256 472 1279b8 Ss+ tty5 0:00 /sbin/getty 3
4 0 541 1 16 0 1256 472 1279b8 Ss+ tty6 0:00 /sbin/getty 3
1 103 553 367 16 0 13744 12800 127a2c S ? 0:00 /usr/sbin/cla
5 0 2029 512 16 0 5716 1728 1279b8 S ? 0:00 /usr/sbin/ssh
5 1030 2031 2029 16 0 5816 1824 1279b8 S ? 0:00 /usr/sbin/ssh
0 1030 2032 2031 15 0 2216 1224 122f5e Ss pts/0 0:00 -bash
4 0 2035 2032 17 0 2236 1276 - R pts/0 0:00 bash
5 0 3464 512 16 0 5724 1780 1279b8 S ? 0:00 /usr/sbin/ssh
4 0 3467 3464 17 0 2232 1248 1279b8 Ss+ pts/1 0:00 -bash
5 0 3645 512 15 0 5724 1780 1279b8 S ? 0:00 /usr/sbin/ssh
4 0 3648 3645 15 0 2232 1260 1279b8 Ss+ pts/2 0:00 -bash
4 0 9572 1 16 0 2500 1036 127a2c Ss ? 0:20 /usr/lib/post
4 102 9574 9572 16 0 2596 1176 1279b8 S ? 0:24 qmgr -l -t fi
5 0 11651 512 16 0 5716 1728 1279b8 S ? 0:00 /usr/sbin/ssh
5 1030 11653 11651 15 0 5816 1812 1279b8 S ? 0:00 /usr/sbin/ssh
0 1030 11654 11653 15 0 2216 1224 122f5e Ss pts/5 0:00 -bash
4 0 11655 11654 15 0 2224 1264 1279b8 S+ pts/5 0:00 bash
4 102 27399 9572 16 0 2760 1380 1279b8 S ? 0:06 trivial-rewri
5 103 16258 1 16 0 27536 26100 127a2c Ss ? 0:00 amavisd (mast
4 102 26052 9572 16 0 2644 1180 1279b8 S ? 0:00 proxymap -t u
4 102 26337 9572 16 0 2504 1076 127a2c S ? 0:00 lmtp -n smtp-
1 103 26338 16258 16 0 28508 27088 127a2c S ? 0:00 amavisd (2633
4 102 26368 9572 16 0 2444 976 127a2c S ? 0:00 pickup -l -t
1 103 26377 16258 15 0 28144 26796 1279b8 S ? 0:00 amavisd (2637
4 102 26393 9572 16 0 2504 1072 127a2c S ? 0:00 lmtp -n smtp-
4 102 26400 9572 15 0 2492 1064 127a2c S ? 0:00 cleanup -z -t
4 102 26401 9572 15 0 2556 1156 163814 S ? 0:00 smtp -t unix
4 102 26402 9572 15 0 2588 1148 127a2c S ? 0:00 smtpd -n 127.
4 102 26403 9572 15 0 2556 1156 127a2c S ? 0:00 smtp -t unix
4 102 26407 9572 17 0 2472 1044 127a2c S ? 0:00 bounce -z -t
4 102 26408 9572 16 0 2544 1112 86e7cf D ? 0:00 cleanup -z -n
4 102 26409 9572 16 0 2780 1392 1279b8 S ? 0:00 smtpd -n smtp
4 102 26414 9572 16 0 2780 1408 127a2c S ? 0:00 smtpd -n smtp
4 102 26415 9572 16 0 2780 1380 1279b8 S ? 0:00 smtpd -n smtp
4 102 26416 9572 16 0 2780 1392 1279b8 S ? 0:00 smtpd -n smtp
4 102 26417 9572 16 0 2780 1380 163814 S ? 0:00 smtpd -n smtp
4 102 26418 9572 16 0 2780 1380 163814 S ? 0:00 smtpd -n smtp
4 102 26419 9572 16 0 2756 1348 127a2c S ? 0:00 smtpd -n smtp
4 102 26420 9572 16 0 2768 1368 1279b8 S ? 0:00 smtpd -n smtp
4 102 26421 9572 16 0 2780 1392 1279b8 S ? 0:00 smtpd -n smtp
4 102 26422 9572 16 0 2780 1408 127a2c S ? 0:00 smtpd -n smtp
4 102 26423 9572 16 0 2780 1380 1279b8 S ? 0:00 smtpd -n smtp
4 102 26424 9572 16 0 2780 1392 1279b8 S ? 0:00 smtpd -n smtp
4 102 26425 9572 16 0 2776 1376 127a2c S ? 0:00 smtpd -n smtp
4 102 26426 9572 16 0 2780 1380 163814 S ? 0:00 smtpd -n smtp
4 102 26427 9572 16 0 2784 1384 1279b8 S ? 0:00 smtpd -n smtp
4 102 26428 9572 16 0 2780 1380 1279b8 S ? 0:00 smtpd -n smtp
4 102 26429 9572 16 0 2780 1392 1279b8 S ? 0:00 smtpd -n smtp
4 102 26430 9572 16 0 2780 1380 163814 S ? 0:00 smtpd -n smtp
4 102 26431 9572 16 0 2780 1388 1279b8 S ? 0:00 smtpd -n smtp
4 102 26432 9572 16 0 2780 1380 163814 S ? 0:00 smtpd -n smtp
4 102 26433 9572 16 0 2780 1388 1279b8 S ? 0:00 smtpd -n smtp
4 102 26434 9572 16 0 2780 1388 1279b8 S ? 0:00 smtpd -n smtp
4 102 26435 9572 16 0 2776 1380 163814 S ? 0:00 smtpd -n smtp
4 102 26436 9572 16 0 2780 1380 163814 S ? 0:00 smtpd -n smtp
4 102 26437 9572 16 0 2780 1392 1279b8 S ? 0:00 smtpd -n smtp
4 102 26438 9572 16 0 2784 1384 1279b8 S ? 0:00 smtpd -n smtp
4 102 26439 9572 16 0 2776 1392 1279b8 S ? 0:00 smtpd -n smtp
4 102 26443 9572 16 0 2780 1388 1279b8 S ? 0:00 smtpd -n smtp
4 102 26444 9572 16 0 2784 1384 1279b8 S ? 0:00 smtpd -n smtp
4 102 26445 9572 16 0 2780 1392 1279b8 S ? 0:00 smtpd -n smtp
4 102 26446 9572 16 0 2768 1368 1279b8 S ? 0:00 smtpd -n smtp
4 102 26447 9572 16 0 2780 1388 1279b8 S ? 0:00 smtpd -n smtp
4 102 26451 9572 16 0 2780 1380 163814 S ? 0:00 smtpd -n smtp
4 102 26452 9572 16 0 2776 1392 1279b8 S ? 0:00 smtpd -n smtp
4 102 26453 9572 16 0 2784 1384 1279b8 S ? 0:00 smtpd -n smtp
4 102 26454 9572 16 0 2780 1392 1279b8 S ? 0:00 smtpd -n smtp
4 102 26456 9572 16 0 2780 1392 1279b8 S ? 0:00 smtpd -n smtp
4 102 26459 9572 16 0 2780 1396 1279b8 S ? 0:00 smtpd -n smtp
4 102 26460 9572 16 0 2792 1408 1279b8 S ? 0:00 smtpd -n smtp
4 102 26463 9572 16 0 2780 1392 1279b8 S ? 0:00 smtpd -n smtp
4 102 26464 9572 16 0 2780 1392 1279b8 S ? 0:00 smtpd -n smtp
4 102 26465 9572 16 0 2776 1380 1279b8 S ? 0:00 smtpd -n smtp
4 102 26466 9572 16 0 2780 1380 163814 S ? 0:00 smtpd -n smtp
4 102 26467 9572 16 0 2780 1392 1279b8 S ? 0:00 smtpd -n smtp
4 102 26468 9572 16 0 2776 1376 163814 S ? 0:00 smtpd -n smtp
4 102 26469 9572 16 0 2776 1376 163814 S ? 0:00 smtpd -n smtp
4 102 26470 9572 16 0 2784 1384 1279b8 S ? 0:00 smtpd -n smtp
4 102 26471 9572 16 0 2776 1376 163814 S ? 0:00 smtpd -n smtp
4 102 26472 9572 17 0 2544 1112 127a2c S ? 0:00 cleanup -z -n
4 102 26473 9572 17 0 2544 1112 11e11e D ? 0:00 cleanup -z -n
4 102 26474 9572 17 0 2524 1048 127a2c S ? 0:00 cleanup -z -n
4 102 26475 9572 17 0 2524 1048 127a2c S ? 0:00 cleanup -z -n
4 102 26476 9572 17 0 2524 1048 127a2c S ? 0:00 cleanup -z -n
4 102 26477 9572 17 0 2524 1048 127a2c S ? 0:00 cleanup -z -n
4 102 26478 9572 17 0 2524 1048 127a2c S ? 0:00 cleanup -z -n
4 102 26479 9572 17 0 2524 1048 127a2c S ? 0:00 cleanup -z -n
4 102 26480 9572 17 0 2524 1048 127a2c S ? 0:00 cleanup -z -n
4 102 26481 9572 17 0 2524 1048 127a2c S ? 0:00 cleanup -z -n
4 102 26482 9572 17 0 2524 1048 127a2c S ? 0:00 cleanup -z -n
4 102 26483 9572 17 0 2524 1048 127a2c S ? 0:00 cleanup -z -n
4 102 26484 9572 17 0 2524 1048 127a2c S ? 0:00 cleanup -z -n
4 102 26485 9572 17 0 2524 1048 127a2c S ? 0:00 cleanup -z -n
5 0 26486 512 17 0 4500 1708 1279b8 S ? 0:00 /usr/sbin/ssh
4 102 26487 9572 17 0 2524 1048 127a2c S ? 0:00 cleanup -z -n
4 102 26489 9572 17 0 2524 1048 127a2c S ? 0:00 cleanup -z -n
4 102 26490 9572 17 0 2524 1048 127a2c S ? 0:00 cleanup -z -n
4 102 26492 9572 17 0 2524 1048 127a2c S ? 0:00 cleanup -z -n
4 102 26493 9572 17 0 2524 1048 127a2c S ? 0:00 cleanup -z -n
4 102 26494 9572 17 0 2524 1048 127a2c S ? 0:00 cleanup -z -n
4 102 26495 9572 17 0 2524 1048 127a2c S ? 0:00 cleanup -z -n
4 102 26496 9572 17 0 2524 1048 127a2c S ? 0:00 cleanup -z -n
4 102 26497 9572 17 0 2524 1048 127a2c S ? 0:00 cleanup -z -n
4 102 26498 9572 17 0 2524 1048 127a2c S ? 0:00 cleanup -z -n
4 102 26499 9572 17 0 2524 1048 127a2c S ? 0:00 cleanup -z -n
4 102 26500 9572 17 0 2524 1048 127a2c S ? 0:00 cleanup -z -n
5 101 26502 26486 18 0 0 0 122841 Z ? 0:00 [sshd] <defun
4 0 26503 2035 17 0 2060 644 - R+ pts/0 0:00 ps axln
root@ares:~# ps axl | grep D
F UID PID PPID PRI NI VSZ RSS WCHAN STAT TTY TIME COMMAND
1 0 103 1 15 0 0 0 io_sch D ? 0:07 [kjournald]
1 0 248 1 15 0 0 0 io_sch D ? 0:04 [kjournald]
5 0 332 1 15 0 1432 672 log_wa Ds ? 0:24 /sbin/syslogd
4 102 9574 9572 16 0 2596 1176 down D ? 0:24 qmgr -l -t fifo -u
-c
4 102 26473 9572 17 0 2544 1112 log_wa D ? 0:00 cleanup -z -n
pre-cleanup -t unix -u -c -o virtual_alias_maps -o canonical_maps -o
sender_canonical_maps -o recipient_canonical_maps -o masquerade_domains
4 102 26509 9572 16 0 2464 1020 down D ? 0:00 showq -t unix -u -c
4 0 26517 2035 22 0 1336 440 - R+ pts/0 0:00 grep D
root@ares:~#
root@ares:~# ps axln | grep D
F UID PID PPID PRI NI VSZ RSS WCHAN STAT TTY TIME COMMAND
1 0 103 1 15 0 0 0 11e11e D ? 0:07 [kjournald]
1 0 248 1 15 0 0 0 11e11e D ? 0:04 [kjournald]
5 0 332 1 15 0 1432 672 86e7cf Ds ? 0:24 /sbin/syslogd
4 102 9574 9572 16 0 2596 1176 109ea5 D ? 0:24 qmgr -l -t fifo -u
-c
4 102 26473 9572 17 0 2544 1112 86e7cf D ? 0:00 cleanup -z -n
pre-cleanup -t unix -u -c -o virtual_alias_maps -o canonical_maps -o
sender_canonical_maps -o recipient_canonical_maps -o masquerade_domains
4 102 26509 9572 16 0 2464 1020 109ea5 D ? 0:00 showq -t unix -u -c
4 0 26519 2035 23 0 1336 440 - R+ pts/0 0:00 grep D
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Kernel hang on MX server
2004-04-14 0:14 Kernel hang on MX server Josh Logan
@ 2004-04-14 0:41 ` Alexander Hoogerhuis
2004-04-14 3:08 ` Andrew Morton
1 sibling, 0 replies; 3+ messages in thread
From: Alexander Hoogerhuis @ 2004-04-14 0:41 UTC (permalink / raw)
To: Josh Logan; +Cc: linux-kernel
I have two HP DL360G3's running a very identical load on Gentoo, both
go into doorstop-emulation mode on 2.6.5-mm1 and higher -mm-kernels
withing 24 of running with very low load. I use hardware RAID, and
ha've SCSI, but otherwise its ServerWorks chipsets and ext3.
I've only caught them a few times in seeing the actual oops; at other
times it's happened with a blanked console, but the few I've seen have
been dying while handling an interrupt, and also, the tg3-driver have
figured prominently in the stack trace.
If anyone have have a good idea on how to capture an oops that
instantly freezes the machine, then I'll go through and capture it
again; but for now I'm down on 2.6.3-mm3 and running solidly.
mvh,
A
Josh Logan <josh@wcug.wwu.edu> writes:
> I have 2 IBM x305's which hang every few days in identical ways. Both are
> running postfix (with amavis-new, spamassassin and clamav from
> backports.org) and bind9. They are running debian stable.
>
> The IDE controler is: ServerWorks CSB5 IDE
> The Kernel is 2.6.4
> I have tried 2.4 kernels with ext3. I have also tried 2.6.4 kernels with
> ext3, jfs, and xfs, but all three FS show the same errors.
> Currently trying ext2 on one of the machines.
>
> The systems is running software raid1 between both hard drives.
> /dev/hda1 and /dev/hdc1 make up md0 which is mount on /
> /dev/hda3 and /dev/hdc3 make up md1 which used LVM to make /usr /home /var
> and /var/spool/postfix
>
> I have also tried mount --bind /postfix /var/spool/postfix to take LVM out
> of the picture. That did not help. I have not tried to break the mirror
> and only use 1 IDE drive.
>
> Please see the attached files for better formating, and all of the
> running commands.
>
> The following processes are in uninterruptible sleep (usually IO) via ps:
> root@ares:~# ps axln | grep D
> F UID PID PPID PRI NI VSZ RSS WCHAN STAT TTY TIME COMMAND
> 1 0 103 1 15 0 0 0 11e11e D ? 0:07
> [kjournald]
> 1 0 248 1 15 0 0 0 11e11e D ? 0:04
> [kjournald]
> 5 0 332 1 15 0 1432 672 86e7cf Ds ? 0:24
> /sbin/syslogd
> 4 102 9574 9572 16 0 2596 1176 109ea5 D ? 0:24 qmgr -l
> -t fifo -u -c
> 4 102 26473 9572 17 0 2544 1112 86e7cf D ? 0:00 cleanup
> -z -n pre-cleanup -t unix -u -c -o virtual_alias_maps -o canonical_maps
> -o sender_canonical_maps -o recipient_canonical_maps -o masquerade_domains
> 4 102 26509 9572 16 0 2464 1020 109ea5 D ? 0:00 showq
> -t unix -u -c
>
> Decoding the WCHAN:
> 11e11e ->
> c011e0f4 T yield
> c011e110 T io_schedule
> c011e128 T io_schedule_timeout
> c011e144 T sys_sched_get_priority_max
>
>
> 86e7cf ->
> not in System.map, but PS gave: log_wait_commit
>
>
> 109ea5 ->
> c0109df4 t __check_printsym_format
> c0109e00 T __up
> c0109e18 T __down
> c0109f24 T __down_interruptible
>
> Where PS was run as:
> ps -o pid,nwchan,wchan=WIDE-WCHAN-COLUMN -o comm -o state axf
>
>
> I'll happily provide more information, but not sure what to try next.
> Looks like it may be a driver issue, blocking IO writes to the drives.
>
> Later, JOSH
>
--
Alexander Hoogerhuis | alexh@boxed.no
CCNP - CCDP - MCNE - CCSE | +47 908 21 485
"You have zero privacy anyway. Get over it." --Scott McNealy
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Kernel hang on MX server
2004-04-14 0:14 Kernel hang on MX server Josh Logan
2004-04-14 0:41 ` Alexander Hoogerhuis
@ 2004-04-14 3:08 ` Andrew Morton
1 sibling, 0 replies; 3+ messages in thread
From: Andrew Morton @ 2004-04-14 3:08 UTC (permalink / raw)
To: Josh Logan; +Cc: linux-kernel
Josh Logan <josh@wcug.wwu.edu> wrote:
>
> I have 2 IBM x305's which hang every few days in identical ways.
Can you capture a sysrq-T trace? Wait for a hang, do alt-sysrq-T?
Yo may need to arrange for klogd to write to a different disk, or
use a serial console.
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2004-04-14 3:09 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2004-04-14 0:14 Kernel hang on MX server Josh Logan
2004-04-14 0:41 ` Alexander Hoogerhuis
2004-04-14 3:08 ` Andrew Morton
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®