Bug in 2.6.8-rc3 at mm/page_alloc.c:792 and mm/rmap.c:407

Bug in 2.6.8-rc3 at mm/page_alloc.c:792 and mm/rmap.c:407

[Oskar Berggren]

Two BUG's I've been seeing, one in page_alloc.c and one in rmap.c.

Please CC any replies to me.

Everything seems to work fine, until this happens. This occured in
rc1 as well, seemed to get worse with rc2 (or there was some unrelated
problem in rc2) and then I think it got somewhat better again with rc3.
However, this still occurs several times a day.

I've been seeing this on 2.6.8-rc1 - 3. I'm running this on some
Asus Pundit-Rs, and must use 2.6.8-rc1 or later in order to have
support for the network card.

There are frequent OOPSes as well, but I don't have the output
of one right now.

I have not been able to identify any particular action to trigger
this.

Regards,
Oskar


kernel BUG at mm/page_alloc.c:792!
invalid operand: 0000 [#1]
PREEMPT 
Modules linked in: ds lp parport ipv6 nfs lockd sunrpc yenta_socket
pcmcia_core 3c59x snd_atiixp snd_ac97_codec snd_pcm
 snd_timer snd soundcore snd_page_alloc ehci_hcd tsdev mousedev usbhid
ohci_hcd usbcore shpchp pciehp pci_hotplug ati_agp agpgart eth1394 evdev
ide_s
csi scsi_mod ohci1394 ieee1394 ide_cd cdrom genrtc xfs reiserfs jfs
isofs vfat fat ext2 ext3 jbd mbcache ide_generic via82cxxx trm290
triflex slc90e6
6 sis5513 siimage serverworks sc1200 rz1000 piix pdc202xx_old opti621
ns87415 hpt366 ide_disk hpt34x generic cy82c693 cs5530 cs5520 cmd64x
atiixp amd
74xx alim15x3 aec62xx pdc202xx_new ide_core unix
CPU:    0
EIP:    0060:[__free_pages+61/71]    Not tainted
EFLAGS: 00010246   (2.6.8-rc3.p4-20040805.01) 
EIP is at __free_pages+0x3d/0x47
eax: ffffffff   ebx: cbbe8a80   ecx: c115e3a0   edx: 00000000
esi: 00000000   edi: 00000000   ebp: 00000297   esp: cda11eec
ds: 007b   es: 007b   ss: 0068
Process events/0 (pid: 3, threadinfo=cda10000 task=cda04b50)
Stack: 00000000 00000000 c0209254 cbbe8a80 c020a128 cbbe8a80 cbbe8a80
c5389380 
       c0251758 cbbe8a80 00000000 c5389380 cda10000 c02066d5 c5389380
cda3c400 
       cda10000 cda3c540 ced0ac28 c5389380 cda11f94 cda3c53c cda10000
c01299b9 
Call Trace:
 [sk_free+191/258] sk_free+0xbf/0x102
 [sk_common_release+87/203] sk_common_release+0x57/0xcb
 [inet_release+82/96] inet_release+0x52/0x60
 [sock_release+149/225] sock_release+0x95/0xe1
 [__crc_bio_get_nr_vecs+4158660/6507043] xprt_socket_autoclose+0x26/0x63
[sunrpc]
 [worker_thread+464/655] worker_thread+0x1d0/0x28f
 [__crc_bio_get_nr_vecs+4158622/6507043] xprt_socket_autoclose+0x0/0x63
[sunrpc]
 [default_wake_function+0/18] default_wake_function+0x0/0x12
 [default_wake_function+0/18] default_wake_function+0x0/0x12
 [worker_thread+0/655] worker_thread+0x0/0x28f
 [kthread+165/171] kthread+0xa5/0xab
 [kthread+0/171] kthread+0x0/0xab
 [kernel_thread_helper+5/11] kernel_thread_helper+0x5/0xb
Code: 0f 0b 18 03 9f fe 27 c0 eb cd 85 c0 74 28 05 00 00 00 40 c1 




kernel BUG at mm/rmap.c:407!
invalid operand: 0000 [#1]
PREEMPT 
Modules linked in: ds lp parport ipv6 nfs lockd sunrpc yenta_socket
pcmcia_core 3c59x snd_atiixp snd_ac97_codec snd_pcm
 snd_timer snd soundcore snd_page_alloc ehci_hcd tsdev mousedev usbhid
ohci_hcd usbcore shpchp pciehp pci_hotplug ati_agp agpgart eth1394 evdev
ide_s
csi scsi_mod ohci1394 ieee1394 ide_cd cdrom genrtc xfs reiserfs jfs
isofs vfat fat ext2 ext3 jbd mbcache ide_generic via82cxxx trm290
triflex slc90e6
6 sis5513 siimage serverworks sc1200 rz1000 piix pdc202xx_old opti621
ns87415 hpt366 ide_disk hpt34x generic cy82c693 cs5530 cs5520 cmd64x
atiixp amd
74xx alim15x3 aec62xx pdc202xx_new ide_core unix
CPU:    0
EIP:    0060:[page_remove_rmap+115/135]    Not tainted
EFLAGS: 00013246   (2.6.8-rc3.p4-20040805.01) 
EIP is at page_remove_rmap+0x73/0x87
eax: 00000000   ebx: 00023000   ecx: cda49800   edx: c10ad220
esi: c7d7348c   edi: c10ad220   ebp: 00041000   esp: c83ebe7c
ds: 007b   es: 007b   ss: 0068
Process XFree86 (pid: 2668, threadinfo=c83ea000 task=cb719770)
Stack: c0141077 c10ad220 c83cd468 c83ea000 087c49c8 05691067 08d00000
c83c908c 
       08941000 00000000 c01411d6 c034cf74 c83c9088 08900000 00041000
00000000 
       c034cf74 08900000 c83c908c 08941000 00000000 c014123d c034cf74
c83c9088 
Call Trace:
 [zap_pte_range+305/569] zap_pte_range+0x131/0x239
 [zap_pmd_range+87/115] zap_pmd_range+0x57/0x73
 [unmap_page_range+75/113] unmap_page_range+0x4b/0x71
 [unmap_vmas+248/438] unmap_vmas+0xf8/0x1b6
 [unmap_region+123/230] unmap_region+0x7b/0xe6
 [do_munmap+320/420] do_munmap+0x140/0x1a4
 [sys_brk+249/253] sys_brk+0xf9/0xfd
 [syscall_call+7/11] syscall_call+0x7/0xb
Code: 0f 0b 97 01 b3 04 28 c0 eb 95 0f 0b 96 01 b3 04 28 c0 eb 84 
 <6>note: XFree86[2668] exited with preempt_count 1
bad: scheduling while atomic!
 [schedule+1152/1157] schedule+0x480/0x485
 [call_console_drivers+105/287] call_console_drivers+0x69/0x11f
 [rwsem_down_read_failed+143/380] rwsem_down_read_failed+0x8f/0x17c
 [.text.lock.exit+107/201] .text.lock.exit+0x6b/0xc9
 [do_invalid_op+0/203] do_invalid_op+0x0/0xcb
 [do_divide_error+0/250] do_divide_error+0x0/0xfa
 [do_invalid_op+201/203] do_invalid_op+0xc9/0xcb
 [page_remove_rmap+115/135] page_remove_rmap+0x73/0x87
 [scheduler_tick+348/1049] scheduler_tick+0x15c/0x419
 [update_process_times+70/82] update_process_times+0x46/0x52
 [error_code+45/56] error_code+0x2d/0x38
 [page_remove_rmap+115/135] page_remove_rmap+0x73/0x87
 [zap_pte_range+305/569] zap_pte_range+0x131/0x239
 [zap_pmd_range+87/115] zap_pmd_range+0x57/0x73
 [unmap_page_range+75/113] unmap_page_range+0x4b/0x71
 [unmap_vmas+248/438] unmap_vmas+0xf8/0x1b6
 [unmap_region+123/230] unmap_region+0x7b/0xe6
 [do_munmap+320/420] do_munmap+0x140/0x1a4
 [sys_brk+249/253] sys_brk+0xf9/0xfd
 [syscall_call+7/11] syscall_call+0x7/0xb



-- 
Oskar Berggren <beo@sgs.o.se>

Re: Bug in 2.6.8-rc3 at mm/page_alloc.c:792 and mm/rmap.c:407

[Hugh Dickins]

On Sun, 8 Aug 2004, Oskar Berggren wrote:
> Two BUG's I've been seeing, one in page_alloc.c and one in rmap.c.

This is not the first report of an rmap.c:407,
Denis reported one a week ago (on 2.6.7-bk20).

I don't know what's behind it, but I am wondering if PageReserved
might be getting cleared while page is still mapped into userspace.

You both have sound modules in, are you using audio?

Could you mail me (privately) your /var/log/messages, Oskar, I don't
have a clear picture of the relation between your page_alloc.c:792s,
your rmap.c:407s and your other oopses.

Thanks,
Hugh

Re: Bug in 2.6.8-rc3 at mm/page_alloc.c:792 and mm/rmap.c:407

[Hugh Dickins]

On Sun, 8 Aug 2004, Hugh Dickins wrote:
> On Sun, 8 Aug 2004, Oskar Berggren wrote:
> > Two BUG's I've been seeing, one in page_alloc.c and one in rmap.c.
> 
> This is not the first report of an rmap.c:407,
> Denis reported one a week ago (on 2.6.7-bk20).
> 
> I don't know what's behind it, but I am wondering if PageReserved
> might be getting cleared while page is still mapped into userspace.

That was just a guess, I've no evidence, and now doubt that.

> You both have sound modules in, are you using audio?

You weren't actually using it (you mention in other mail),
even if Denis was, so audio no longer looks like a suspect.

> Could you mail me (privately) your /var/log/messages, Oskar, I don't
> have a clear picture of the relation between your page_alloc.c:792s,
> your rmap.c:407s and your other oopses.

Thanks a lot for the /var/log/messages.

The frustrating thing is that the most interesting lines are missing:
the "unqualified" printks, e.g. handle_BUG's "--- [ cut here ] ---" line
and stack traces do appear; but handle_BUG's KERN_ALERT "kernel BUG..."
and bad_page's very useful KERN_EMERG messages do not appear at all.

The "kernel BUG" messages you've already told us, but the missing
bad_page lines might, _might_ be really helpful.  Do you have some
klogd option set, not to print out the most important messages ;-?
I hope someone can tell us how to fix that.

I notice the page_remove_rmap BUG (which we know to be rmap.c:407
from your mail) was preceded 10 minutes earlier by a bad_page; and
one of the things bad_page will do is force page->mapcount to 0,
which would trigger the page_remove_rmap BUG, if that page being
freed was actually still in use in some process address space.

Most of the other BUGs (mostly page_alloc.c:792s, __free_pages called
from below shrink_cache or sock_release, finding page_count already 0)
were also preceded, less immediately, by bad_pages; though not all.

It's all consistent with pages being freed while still in use,
but I don't think I'm saying anything new there.  It doesn't look
to me like random corruption or bad memory, I don't think those would
show up so consistently as page freeing errors.  (Though if KERN_ERRs
aren't getting into /var/log/messages, there might be page table
corruption swap_free errors missing too.)

But I've no idea of where to look for the culprit: I'd better get
on with other things, and hope someone else can take this further.

Hugh

Re: Bug in 2.6.8-rc3 at mm/page_alloc.c:792 and mm/rmap.c:407

[Oskar Berggren]

On Mon, 2004-08-09 at 21:08, Hugh Dickins wrote:
> On Sun, 8 Aug 2004, Hugh Dickins wrote:
> > On Sun, 8 Aug 2004, Oskar Berggren wrote:
> > > Two BUG's I've been seeing, one in page_alloc.c and one in rmap.c.
> > 
> > This is not the first report of an rmap.c:407,
> > Denis reported one a week ago (on 2.6.7-bk20).
> > 

> > Could you mail me (privately) your /var/log/messages, Oskar, I don't
> > have a clear picture of the relation between your page_alloc.c:792s,
> > your rmap.c:407s and your other oopses.
> 
> Thanks a lot for the /var/log/messages.
> 
> The frustrating thing is that the most interesting lines are missing:
> the "unqualified" printks, e.g. handle_BUG's "--- [ cut here ] ---" line
> and stack traces do appear; but handle_BUG's KERN_ALERT "kernel BUG..."
> and bad_page's very useful KERN_EMERG messages do not appear at all.
> 
> The "kernel BUG" messages you've already told us, but the missing
> bad_page lines might, _might_ be really helpful.  Do you have some
> klogd option set, not to print out the most important messages ;-?
> I hope someone can tell us how to fix that.

Fresh install of Debian using debian installer beta and 'testing'.

> 
> I notice the page_remove_rmap BUG (which we know to be rmap.c:407
> from your mail) was preceded 10 minutes earlier by a bad_page; and
> one of the things bad_page will do is force page->mapcount to 0,
> which would trigger the page_remove_rmap BUG, if that page being
> freed was actually still in use in some process address space.
> 
> Most of the other BUGs (mostly page_alloc.c:792s, __free_pages called
> from below shrink_cache or sock_release, finding page_count already 0)
> were also preceded, less immediately, by bad_pages; though not all.
> 
> It's all consistent with pages being freed while still in use,
> but I don't think I'm saying anything new there.  It doesn't look
> to me like random corruption or bad memory, I don't think those would
> show up so consistently as page freeing errors.  (Though if KERN_ERRs
> aren't getting into /var/log/messages, there might be page table
> corruption swap_free errors missing too.)
> 
> But I've no idea of where to look for the culprit: I'd better get
> on with other things, and hope someone else can take this further.


I think I've managed to catch one of the missing lines that you
mention in todays messages file, and then even more in the
/var/log/syslog file:

Aug  9 13:07:59 otukt kernel:  <0>Bad page state at free_hot_cold_page
(in process 'events/0', page c1090200)
Aug  9 13:07:59 otukt kernel: flags:0x20000080 mapping:00000000
mapcount:0 count:0
Aug  9 13:07:59 otukt kernel: Backtrace:
Aug  9 13:07:59 otukt kernel:  [bad_page+109/153] bad_page+0x6d/0x99
Aug  9 13:07:59 otukt kernel:  [free_hot_cold_page+81/270]
free_hot_cold_page+0x51/0x10e
Aug  9 13:07:59 otukt kernel:  [sk_free+191/258] sk_free+0xbf/0x102
Aug  9 13:07:59 otukt kernel:  [sk_common_release+87/203]
sk_common_release+0x57/0xcb
Aug  9 13:07:59 otukt kernel:  [inet_release+82/96]
inet_release+0x52/0x60
Aug  9 13:07:59 otukt kernel:  [sock_release+149/225]
sock_release+0x95/0xe1
Aug  9 13:07:59 otukt kernel:  [__crc_bio_get_nr_vecs+4158660/6507043]
xprt_socket_autoclose+0x26/0x63 [sunrpc]
Aug  9 13:07:59 otukt kernel:  [worker_thread+464/655]
worker_thread+0x1d0/0x28f
Aug  9 13:07:59 otukt kernel:  [__crc_bio_get_nr_vecs+4158622/6507043]
xprt_socket_autoclose+0x0/0x63 [sunrpc]
Aug  9 13:07:59 otukt kernel:  [default_wake_function+0/18]
default_wake_function+0x0/0x12
Aug  9 13:07:59 otukt kernel:  [default_wake_function+0/18]
default_wake_function+0x0/0x12
Aug  9 13:07:59 otukt kernel:  [worker_thread+0/655]
worker_thread+0x0/0x28f
Aug  9 13:07:59 otukt kernel:  [kthread+165/171] kthread+0xa5/0xab
Aug  9 13:07:59 otukt kernel:  [kthread+0/171] kthread+0x0/0xab
Aug  9 13:07:59 otukt kernel:  [kernel_thread_helper+5/11]
kernel_thread_helper+0x5/0xb
Aug  9 13:07:59 otukt kernel: Trying to fix it up, but a reboot is
needed
Aug  9 13:07:59 otukt kernel: Bad page state at free_hot_cold_page (in
process 'syslogd', page c1090200)
Aug  9 13:07:59 otukt kernel: flags:0x20000080 mapping:00000000
mapcount:0 count:0
Aug  9 13:07:59 otukt kernel: Backtrace:
Aug  9 13:07:59 otukt kernel:  [bad_page+109/153] bad_page+0x6d/0x99
Aug  9 13:07:59 otukt kernel:  [free_hot_cold_page+81/270]
free_hot_cold_page+0x51/0x10e
Aug  9 13:07:59 otukt kernel:  [datagram_poll+43/202]
datagram_poll+0x2b/0xca
Aug  9 13:07:59 otukt kernel:  [poll_freewait+56/64]
poll_freewait+0x38/0x40
Aug  9 13:07:59 otukt kernel:  [do_select+438/712] do_select+0x1b6/0x2c8
Aug  9 13:07:59 otukt kernel:  [__pollwait+0/198] __pollwait+0x0/0xc6
Aug  9 13:07:59 otukt kernel:  [sys_select+691/1200]
sys_select+0x2b3/0x4b0
Aug  9 13:07:59 otukt kernel:  [syscall_call+7/11] syscall_call+0x7/0xb
Aug  9 13:07:59 otukt kernel: Trying to fix it up, but a reboot is
needed
Aug  9 13:12:52 otukt syslogd 1.4.1#15: restart.
Aug  9 13:12:52 otukt kernel: klogd 1.4.1#15, log source = /proc/kmsg
started.
Aug  9 13:12:52 otukt kernel: Inspecting
/boot/System.map-2.6.8-rc3.p4-20040805.01
Aug  9 13:12:52 otukt kernel: Loaded 27129 symbols from
/boot/System.map-2.6.8-rc3.p4-20040805.01.
Aug  9 13:12:52 otukt kernel: Symbols match kernel version 2.6.8.
Aug  9 13:12:52 otukt kernel: No module symbols loaded - kernel modules
not enabled. 
Aug  9 13:12:52 otukt kernel: Linux version 2.6.8-rc3.p4-20040805.01
(root@otukt) (gcc version 3.3.4 (Debian 1:3.3.4-3)) #1 Thu Aug 5
11:41:51 CE
ST 2004


When this happens one can observe the following:
  Machine still responds to ping, but not to SSH connection attempts.
  Mouse pointer still moves, but keyboard does NOT work.
  Window manager (ctwm) still operates with desktop/focus switching,
  moving windows, and closing some X-forwarded programs I had running.

In the end, I saw no other way than a hard reset.


Here are some additional messages I found in an earlier syslog file,
including an oops:

Aug  6 12:23:25 otukt kernel: Bad page state at free_hot_cold_page (in
process 'kswapd0', page c10f6c80)
Aug  6 12:23:25 otukt kernel: flags:0x20000000 mapping:cda3f97c
mapcount:0 count:0
Aug  6 12:23:25 otukt kernel: Backtrace:
Aug  6 12:23:25 otukt kernel:  [bad_page+109/153] bad_page+0x6d/0x99
Aug  6 12:23:25 otukt kernel:  [free_hot_cold_page+81/270]
free_hot_cold_page+0x51/0x10e
Aug  6 12:23:25 otukt kernel:  [__pagevec_free+25/33]
__pagevec_free+0x19/0x21
Aug  6 12:23:25 otukt kernel:  [release_pages+118/372]
release_pages+0x76/0x174
Aug  6 12:23:25 otukt kernel:  [__pagevec_release+40/54]
__pagevec_release+0x28/0x36
Aug  6 12:23:25 otukt kernel:  [shrink_cache+640/830]
shrink_cache+0x280/0x33e
Aug  6 12:23:25 otukt kernel:  [shrink_slab+123/390]
shrink_slab+0x7b/0x186
Aug  6 12:23:25 otukt kernel:  [shrink_zone+158/184]
shrink_zone+0x9e/0xb8
Aug  6 12:23:25 otukt kernel:  [balance_pgdat+457/557]
balance_pgdat+0x1c9/0x22d
Aug  6 12:23:25 otukt kernel:  [kswapd+199/215] kswapd+0xc7/0xd7
Aug  6 12:23:25 otukt kernel:  [autoremove_wake_function+0/87]
autoremove_wake_function+0x0/0x57
Aug  6 12:23:25 otukt kernel:  [ret_from_fork+6/20]
ret_from_fork+0x6/0x14
Aug  6 12:23:25 otukt kernel:  [autoremove_wake_function+0/87]
autoremove_wake_function+0x0/0x57
Aug  6 12:23:25 otukt kernel:  [kswapd+0/215] kswapd+0x0/0xd7
Aug  6 12:23:25 otukt kernel:  [kernel_thread_helper+5/11]
kernel_thread_helper+0x5/0xb
Aug  6 12:23:25 otukt kernel: Trying to fix it up, but a reboot is
needed
Aug  6 12:47:18 otukt -- MARK --
Aug  6 13:07:18 otukt -- MARK --
Aug  6 13:17:01 otukt /USR/SBIN/CRON[2959]: (root) CMD (   run-parts
--report /etc/cron.hourly)
Aug  6 13:23:37 otukt kernel: Bad page state at free_hot_cold_page (in
process 'events/0', page c107c120)
Aug  6 13:23:37 otukt kernel: flags:0x20100064 mapping:c0dee508
mapcount:1 count:0
Aug  6 13:23:37 otukt kernel: Backtrace:
Aug  6 13:23:37 otukt kernel:  [bad_page+109/153] bad_page+0x6d/0x99
Aug  6 13:23:37 otukt kernel:  [free_hot_cold_page+81/270]
free_hot_cold_page+0x51/0x10e
Aug  6 13:23:37 otukt kernel:  [sk_free+191/258] sk_free+0xbf/0x102
Aug  6 13:23:37 otukt kernel:  [sk_common_release+87/203]
sk_common_release+0x57/0xcb
Aug  6 13:23:37 otukt kernel:  [inet_release+82/96]
inet_release+0x52/0x60
Aug  6 13:23:37 otukt kernel:  [sock_release+149/225]
sock_release+0x95/0xe1
Aug  6 13:23:37 otukt kernel:  [__crc_bio_get_nr_vecs+4158660/6507043]
xprt_socket_autoclose+0x26/0x63 [sunrpc]
Aug  6 13:23:37 otukt kernel:  [worker_thread+464/655]
worker_thread+0x1d0/0x28f
Aug  6 13:23:37 otukt kernel:  [__crc_bio_get_nr_vecs+4158622/6507043]
xprt_socket_autoclose+0x0/0x63 [sunrpc]
Aug  6 13:23:37 otukt kernel:  [default_wake_function+0/18]
default_wake_function+0x0/0x12
Aug  6 13:23:37 otukt kernel:  [default_wake_function+0/18]
default_wake_function+0x0/0x12
Aug  6 13:23:37 otukt kernel:  [worker_thread+0/655]
worker_thread+0x0/0x28f
Aug  6 13:23:37 otukt kernel:  [kthread+165/171] kthread+0xa5/0xab
Aug  6 13:23:37 otukt kernel:  [kthread+0/171] kthread+0x0/0xab
Aug  6 13:23:37 otukt kernel:  [kernel_thread_helper+5/11]
kernel_thread_helper+0x5/0xb
Aug  6 13:23:37 otukt kernel: Trying to fix it up, but a reboot is
needed
Aug  6 13:25:07 otukt kernel: Unable to handle kernel paging request at
virtual address fffffffc
Aug  6 13:25:07 otukt kernel:  printing eip:
Aug  6 13:25:07 otukt kernel: c01634fb
Aug  6 13:25:07 otukt kernel: *pde = 00002067
Aug  6 13:25:07 otukt kernel: *pte = 00000000
Aug  6 13:25:07 otukt kernel: Oops: 0000 [#1]
Aug  6 13:25:07 otukt kernel: PREEMPT 
Aug  6 13:25:07 otukt kernel: Modules linked in: ds lp parport ipv6 nfs
lockd sunrpc yenta_socket pcmcia_core 3c59x snd_atiixp snd_ac97_codec
snd
_pcm snd_timer snd soundcore snd_page_alloc ehci_hcd tsdev mousedev
usbhid ohci_hcd usbcore shpchp pciehp pci_hotplug ati_agp agpgart
eth1394 evd
ev ide_scsi scsi_mod ohci1394 ieee1394 ide_cd cdrom genrtc xfs reiserfs
jfs isofs vfat fat ext2 ext3 jbd mbcache ide_generic via82cxxx trm290
tri
flex slc90e66 sis5513 siimage serverworks sc1200 rz1000 piix
pdc202xx_old opti621 ns87415 hpt366 ide_disk hpt34x generic cy82c693
cs5530 cs5520 c
md64x atiixp amd74xx alim15x3 aec62xx pdc202xx_new ide_core unix
Aug  6 13:25:07 otukt kernel: CPU:    0
Aug  6 13:25:07 otukt kernel: EIP:    0060:[poll_freewait+26/64]    Not
tainted
Aug  6 13:25:07 otukt kernel: EFLAGS: 00010297  
(2.6.8-rc3.p4-20040805.01) 
Aug  6 13:25:07 otukt kernel: EIP is at poll_freewait+0x1a/0x40
Aug  6 13:25:07 otukt kernel: eax: c7581f44   ebx: ffffffe4   ecx:
00000000   edx: ffffffe8
Aug  6 13:25:07 otukt kernel: esi: c3e09008   edi: c3e09000   ebp:
00000004   esp: c7581ee4
Aug  6 13:25:07 otukt kernel: ds: 007b   es: 007b   ss: 0068
Aug  6 13:25:07 otukt kernel: Process xclock (pid: 2755,
threadinfo=c7580000 task=cc39edd0)
Aug  6 13:25:07 otukt kernel: Stack: 00000000 00000000 00000004 c016387c
c7581f44 00000000 00000000 00000000 
Aug  6 13:25:07 otukt kernel:        00000008 00000000 00000000 00000000
00000304 00000008 c7580000 cb4d58ac 
Aug  6 13:25:07 otukt kernel:        cb4d58a8 cb4d58a4 cb4d58b8 cb4d58b4
cb4d58b0 00000000 00000000 00000000 
Aug  6 13:25:07 otukt kernel: Call Trace:
Aug  6 13:25:07 otukt kernel:  [do_select+438/712] do_select+0x1b6/0x2c8
Aug  6 13:25:07 otukt kernel:  [__pollwait+0/198] __pollwait+0x0/0xc6
Aug  6 13:25:07 otukt kernel:  [sys_select+691/1200]
sys_select+0x2b3/0x4b0
Aug  6 13:25:07 otukt kernel:  [syscall_call+7/11] syscall_call+0x7/0xb
Aug  6 13:25:07 otukt kernel: Code: 8b 43 18 e8 31 5e fb ff 8b 03 e8 f7
df fe ff 39 f3 77 e7 89 
Aug  6 13:47:18 otukt -- MARK --
Aug  6 14:04:23 otukt syslogd 1.4.1#15: restart.
Aug  6 14:04:23 otukt kernel: klogd 1.4.1#15, log source = /proc/kmsg
started.

-- 
Oskar Berggren <beo@sgs.o.se>

Re: Bug in 2.6.8-rc3 at mm/page_alloc.c:792 and mm/rmap.c:407

[Andrew Morton]

Oskar Berggren <beo@sgs.o.se> wrote:
>
> I think I've managed to catch one of the missing lines that you
> mention in todays messages file, and then even more in the
> /var/log/syslog file:
> 
> Aug  9 13:07:59 otukt kernel:  <0>Bad page state at free_hot_cold_page
> (in process 'events/0', page c1090200)
> Aug  9 13:07:59 otukt kernel: flags:0x20000080 mapping:00000000
> mapcount:0 count:0
> Aug  9 13:07:59 otukt kernel: Backtrace:
> Aug  9 13:07:59 otukt kernel:  [bad_page+109/153] bad_page+0x6d/0x99
> Aug  9 13:07:59 otukt kernel:  [free_hot_cold_page+81/270]
> free_hot_cold_page+0x51/0x10e
> Aug  9 13:07:59 otukt kernel:  [sk_free+191/258] sk_free+0xbf/0x102
> Aug  9 13:07:59 otukt kernel:  [sk_common_release+87/203]
> sk_common_release+0x57/0xcb
> Aug  9 13:07:59 otukt kernel:  [inet_release+82/96]
> inet_release+0x52/0x60
> Aug  9 13:07:59 otukt kernel:  [sock_release+149/225]
> sock_release+0x95/0xe1
> Aug  9 13:07:59 otukt kernel:  [__crc_bio_get_nr_vecs+4158660/6507043]
> xprt_socket_autoclose+0x26/0x63 [sunrpc]
> Aug  9 13:07:59 otukt kernel:  [worker_thread+464/655]
> worker_thread+0x1d0/0x28f
> Aug  9 13:07:59 otukt kernel:  [__crc_bio_get_nr_vecs+4158622/6507043]
> xprt_socket_autoclose+0x0/0x63 [sunrpc]

hm.  There was a page double-freeing bug in the nfs/networking code
which Dave Miller fixed just a few days ago.  I'd suggest that you retest
using the latest tree from ftp://ftp.kernel.org/pub/linux/kernel/v2.6/snapshots/