Kernel 2.6.8.1: swap storm of death

[Karl Vogel <karl.vogel@seagha.com>]

I can bring down my box by running a program that does a calloc() of 512Mb 
(which is the size of my RAM). The box starts to heavily swap and never 
recovers from it. The process that calloc's the memory gets OOM killed (which 
is also strange as I have 1Gb free swap).

After the OOM kill, the shell where I started the calloc() program is alive 
but very slow. The box continues to swap and the other processes remain dead.

To gather some more statistics, I did the following:

- start 'vmstat 1|tee vmstat.txt' in 1 VT session.
- run expunge (= program that does calloc(512Mb)) in another VT.

The box freezes for some time. After a while expunge is OOM killed, the vmstat 
on the other VT remains dead. A ping over the network is still possible and I 
can still start programs on the expunge VT, albeit it is slow as the disk is 
still thrashing.



The diagnostics can be found here:

* Kernel .config
  http://users.telenet.be/kvogel/config.txt

* expunge program
  http://users.telenet.be/kvogel/expunge.c

* vmstat 1  output while executing expunge (this freezes)
  http://users.telenet.be/kvogel/vmstat.txt

* vmstat in expunge VT after the OOM kill
  http://users.telenet.be/kvogel/vmstat-after-kill.txt

* /proc/slabinfo after OOM kill
  http://users.telenet.be/kvogel/slab.txt

* swapon -s
Filename                                Type            Size    Used    
Priority
/dev/hda3                               partition       1044216 0       -1

* Kernel boot line:
       kernel /vmlinuz-2.6.8.1 ro root=/dev/compat/root elevator=cfq 
voluntary-preempt=3 preempt=1

Kernel was patched with voluntary-preempt-2.6.8.1-P7
syslogd & klogd weren't running and 'dmesg -n 1' was done beforehand.