kernel oops, fast ethernet bridge, 2.4.31

[Lukasz Spaleniak <lspaleniak@wroc.zigzag.pl>]

[-- Attachment #1: Type: text/plain, Size: 919 bytes --]

Hello,

I have bridge firewall (linux box) with three fast ethernet cards (one
rtl8139 for management and two e100 for bridge). It is running 2.4.31
kernel and iptables v1.2.11. It works ok about one month. Few weeks
ago It started ooopsing. First thought was hardware, but it was
replaced with a new one and problem still exist. I'm attaching 
ksymoops analysis of ooops message. I hope I'll help.

Some facts:
Traffic over this bridge is about ~30 MBit/sec and every frame is
tagged with VID (802.1q). Problem exist also on 2.4.30 kernel.
Kernel is patched with ebtables patch. Ooops completeley freezes
machine, only hard reset takes effect. Nothing is placed into syslog.

The ksymooops analisis is attached as text file.

Anyone know this bug/problem ?

Regards,
Lukasz Spaleniak

-- 
spalek on wroc zigzag pl
GCM dpu s: a--- C++ UL++++ P+ L+++ E--- W+ N+ K- w O- M V-
PGP t--- 5 X+ R- tv-- b DI- D- G e-- h! r y+

[-- Attachment #2: lalala --]
[-- Type: application/octet-stream, Size: 6060 bytes --]

ksymoops 2.4.9 on i686 2.4.31.  Options used
     -V (default)
     -k ksyms (specified)
     -l moduly (specified)
     -o /lib/modules/2.4.31/ (default)
     -m /boot/System.map-2.4.31 (default)

skput:under: e08e7c48:1518 put:14 dev:eth2kernel BUG at skbuff.c:109!
invalid operand: 0000
CPU:    0
EIP:    0010:[<c01cd1cb>]    Not tainted
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010282
eax: 0000002a   ebx: df3b8c80   ecx: c0286000   edx: df3c3f84
esi: ddbb3ba6   edi: ddf9b810   ebp: df3b8c80   esp: c0287b48
ds: 0018   es: 0018   ss: 0018
Process swapper (pid: 0, stackpage=c0287000)
Stack: c023b4e0 e08e7c48 000005ee 0000000e dfae8c00 e08e7c51 df3b8c80 0000000e
       e08e7c48 df3b8c80 ddf9b810 ddf9b824 000005c8 c01e634c df3b8c80 00000014
       ddf9b824 000005c8 000005dc 00000014 00000000 000005c8 00000009 000005c8
Call Trace:    [<e08e7c48>] [<e08e7c51>] [<e08e7c48>] [<c01e634c>] [<e08ecec0>]
  [<e08ed020>] [<e08e7bb0>] [<e0900684>] [<e08e7bb0>] [<e08e7bb0>] [<e09073d0>]
  [<c01d9c6a>] [<e08e7bb0>] [<c01d9fd9>] [<e08e7bb0>] [<e09073d0>] [<e08ec34d>]
  [<e08e7bb0>] [<e08ed140>] [<c01d9c6a>] [<e08e7bb0>] [<c01d9fd9>] [<e08e7bb0>]
  [<e08ed140>] [<e08e7cf3>] [<e08e7bb0>] [<e08ebd0c>] [<e08e7cb0>] [<e08ebe6f>]
  [<e08ebc80>] [<e08ed110>] [<c01d9c6a>] [<e08e7cb0>] [<c01d9fd9>] [<e08e7cb0>]
  [<e08ed110>] [<e08e7dea>] [<e08e7cb0>] [<e08e88df>] [<e08e8820>] [<e08eb362>]
  [<e08e8820>] [<e08eb250>] [<e08ebade>] [<e08eb250>] [<e08ed0e0>] [<c01d9c6a>]
  [<e08e8820>] [<c01d9fd9>] [<e08e8820>] [<e08ed0e0>] [<e08e8820>] [<e08e8aa2>]
  [<e08e8820>] [<c01d1289>] [<c01d1463>] [<c01d1564>] [<c0118b05>] [<c01088fb>]
  [<c01052c0>] [<c010ac78>] [<c01052c0>] [<c01052e3>] [<c0105372>] [<c0105000>]
Code: 0f 0b 6d 00 5f a5 23 c0 83 c4 14 c3 89 f6 8d bc 27 00 00 00


>>EIP; c01cd1cb <skb_under_panic+3b/50>   <=====

>>ebx; df3b8c80 <_end+1f0f4ffc/205f83dc>
>>ecx; c0286000 <init_task_union+0/2000>
>>edx; df3c3f84 <_end+1f100300/205f83dc>
>>esi; ddbb3ba6 <_end+1d8eff22/205f83dc>
>>edi; ddf9b810 <_end+1dcd7b8c/205f83dc>
>>ebp; df3b8c80 <_end+1f0f4ffc/205f83dc>
>>esp; c0287b48 <init_task_union+1b48/2000>

Trace; e08e7c48 <[bridge]br_dev_queue_push_xmit+98/100>
Trace; e08e7c51 <[bridge]br_dev_queue_push_xmit+a1/100>
Trace; e08e7c48 <[bridge]br_dev_queue_push_xmit+98/100>
Trace; c01e634c <ip_fragment+2ec/3e0>
Trace; e08ecec0 <[bridge]__fake_net_device+0/160>
Trace; e08ed020 <[bridge]__fake_rtable+0/c0>
Trace; e08e7bb0 <[bridge]br_dev_queue_push_xmit+0/100>
Trace; e0900684 <[ip_conntrack]ip_refrag+74/80>
Trace; e08e7bb0 <[bridge]br_dev_queue_push_xmit+0/100>
Trace; e08e7bb0 <[bridge]br_dev_queue_push_xmit+0/100>
Trace; e09073d0 <[ip_conntrack]ip_conntrack_out_ops+0/18>
Trace; c01d9c6a <nf_iterate+6a/b0>
Trace; e08e7bb0 <[bridge]br_dev_queue_push_xmit+0/100>
Trace; c01d9fd9 <nf_hook_slow+99/1f0>
Trace; e08e7bb0 <[bridge]br_dev_queue_push_xmit+0/100>
Trace; e09073d0 <[ip_conntrack]ip_conntrack_out_ops+0/18>
Trace; e08ec34d <[bridge]br_nf_post_routing+13d/240>
Trace; e08e7bb0 <[bridge]br_dev_queue_push_xmit+0/100>
Trace; e08ed140 <[bridge]br_nf_ops+60/140>
Trace; c01d9c6a <nf_iterate+6a/b0>
Trace; e08e7bb0 <[bridge]br_dev_queue_push_xmit+0/100>
Trace; c01d9fd9 <nf_hook_slow+99/1f0>
Trace; e08e7bb0 <[bridge]br_dev_queue_push_xmit+0/100>
Trace; e08ed140 <[bridge]br_nf_ops+60/140>
Trace; e08e7cf3 <[bridge]br_forward_finish+43/60>
Trace; e08e7bb0 <[bridge]br_dev_queue_push_xmit+0/100>
Trace; e08ebd0c <[bridge]br_nf_forward_finish+8c/e0>
Trace; e08e7cb0 <[bridge]br_forward_finish+0/60>
Trace; e08ebe6f <[bridge]br_nf_forward_ip+10f/160>
Trace; e08ebc80 <[bridge]br_nf_forward_finish+0/e0>
Trace; e08ed110 <[bridge]br_nf_ops+30/140>
Trace; c01d9c6a <nf_iterate+6a/b0>
Trace; e08e7cb0 <[bridge]br_forward_finish+0/60>
Trace; c01d9fd9 <nf_hook_slow+99/1f0>
Trace; e08e7cb0 <[bridge]br_forward_finish+0/60>
Trace; e08ed110 <[bridge]br_nf_ops+30/140>
Trace; e08e7dea <[bridge]__br_forward+5a/70>
Trace; e08e7cb0 <[bridge]br_forward_finish+0/60>
Trace; e08e88df <[bridge]br_handle_frame_finish+bf/160>
Trace; e08e8820 <[bridge]br_handle_frame_finish+0/160>
Trace; e08eb362 <[bridge]br_nf_pre_routing_finish+112/2b0>
Trace; e08e8820 <[bridge]br_handle_frame_finish+0/160>
Trace; e08eb250 <[bridge]br_nf_pre_routing_finish+0/2b0>
Trace; e08ebade <[bridge]br_nf_pre_routing+29e/410>
Trace; e08eb250 <[bridge]br_nf_pre_routing_finish+0/2b0>
Trace; e08ed0e0 <[bridge]br_nf_ops+0/140>
Trace; c01d9c6a <nf_iterate+6a/b0>
Trace; e08e8820 <[bridge]br_handle_frame_finish+0/160>
Trace; c01d9fd9 <nf_hook_slow+99/1f0>
Trace; e08e8820 <[bridge]br_handle_frame_finish+0/160>
Trace; e08ed0e0 <[bridge]br_nf_ops+0/140>
Trace; e08e8820 <[bridge]br_handle_frame_finish+0/160>
Trace; e08e8aa2 <[bridge]br_handle_frame+122/1e0>
Trace; e08e8820 <[bridge]br_handle_frame_finish+0/160>
Trace; c01d1289 <netif_receive_skb+b9/210>
Trace; c01d1463 <process_backlog+83/110>
Trace; c01d1564 <net_rx_action+74/110>
Trace; c0118b05 <do_softirq+95/a0>
Trace; c01088fb <do_IRQ+9b/a0>
Trace; c01052c0 <default_idle+0/40>
Trace; c010ac78 <call_do_IRQ+5/d>
Trace; c01052c0 <default_idle+0/40>
Trace; c01052e3 <default_idle+23/40>
Trace; c0105372 <cpu_idle+52/70>
Trace; c0105000 <_stext+0/0>

Code;  c01cd1cb <skb_under_panic+3b/50>
00000000 <_EIP>:
Code;  c01cd1cb <skb_under_panic+3b/50>   <=====
   0:   0f 0b                     ud2a      <=====
Code;  c01cd1cd <skb_under_panic+3d/50>
   2:   6d                        insl   (%dx),%es:(%edi)
Code;  c01cd1ce <skb_under_panic+3e/50>
   3:   00 5f a5                  add    %bl,0xffffffa5(%edi)
Code;  c01cd1d1 <skb_under_panic+41/50>
   6:   23 c0                     and    %eax,%eax
Code;  c01cd1d3 <skb_under_panic+43/50>
   8:   83 c4 14                  add    $0x14,%esp
Code;  c01cd1d6 <skb_under_panic+46/50>
   b:   c3                        ret    
Code;  c01cd1d7 <skb_under_panic+47/50>
   c:   89 f6                     mov    %esi,%esi
Code;  c01cd1d9 <skb_under_panic+49/50>
   e:   8d bc 27 00 00 00 00      lea    0x0(%edi),%edi

 <0>Kernel panic: Aiee, killing interrupt handler!