From: Randy Dunlap <rdunlap@xenotime.net>
To: haael <haael@wp.pl>
Cc: linux-kernel@vger.kernel.org
Subject: Re: Bug: Apparent memory exhaustion WTF?
Date: Fri, 1 Apr 2011 13:22:25 -0700 [thread overview]
Message-ID: <20110401132225.a9291e25.rdunlap@xenotime.net> (raw)
In-Reply-To: <4D9632BD.1050504@wp.pl>
On Fri, 01 Apr 2011 22:17:01 +0200 haael wrote:
>
> Hello, guys. My server keeps hanging up with the error output as seen on the
> bottom of this message. I run non-patched vanilla kernel. Full system upgrade
> didn't help. Kernel upgrade from 2.6.28.7 to 2.6.35.10 didn't help. I tried
> also turning swap on/off, upgrading memory and adding new CPU. After the CPU
> upgrade things actually went worse, now my system blows up every few hours.
>
> Here you see the squid process running out of memory, but nothing changed when
> I killed squid; some other process would always cause this error. Just before a
> hangup, all processes are being killed, as if OOM killer went wild. But the OOM
> killer shouldn't disable the network stack, right?
>
> After a hangup, the system becomes completely unresponsive, doesn't answer on
> ping and even on ARP requests. The only thing that still works is the serial
> console from which I get the following error. This message is being printed
> continuously, one per second, for infinity. The only thing I can do with my
> server is to turn off the power.
>
> Tell me, guys, what to do? The server is 2x Intel dual core, 16GB RAM, HP
> Proliant. It has RAID-1 disk and a Broadcom network adapter, which is the most
> suspicious for me. Attached: lspci, /proc/meminfo, /proc/cpuinfo, kernel config
> and the actual error message from the serial console.
It would really help if you would build with KALLSYMS enabled (=y)
so that the stack trace below was meaningful/useful.
# CONFIG_KALLSYMS is not set
> [83155.708165] squid: page allocation failure. order:0, mode:0x4020
> [83155.718040] Pid: 19999, comm: squid Not tainted 2.6.35.10-server #1
> [83155.718040] Call Trace:
> [83155.718040] [<c0175d67>] ? 0xc0175d67
> [83155.718040] [<c019acdf>] ? 0xc019acdf
> [83155.718040] [<c019b275>] ? 0xc019b275
> [83155.718040] [<c030fe0b>] ? 0xc030fe0b
> [83155.718040] [<c030fe0b>] ? 0xc030fe0b
> [83155.718040] [<c030f838>] ? 0xc030f838
> [83155.718040] [<c030fe0b>] ? 0xc030fe0b
> [83155.718040] [<f81444f5>] ? 0xf81444f5
> [83155.718040] [<f8142858>] ? 0xf8142858
> [83155.718040] [<c0317a6a>] ? 0xc0317a6a
> [83155.718040] [<c0137e77>] ? 0xc0137e77
> [83155.718040] [<c0137df0>] ? 0xc0137df0
> [83155.718040] <IRQ> [<c0137c75>] ? 0xc0137c75
> [83155.718040] [<c0119ec3>] ? 0xc0119ec3
> [83155.718040] [<c0296760>] ? 0xc0296760
> [83155.718040] [<c039480a>] ? 0xc039480a
> [83155.718040] [<c01336e5>] ? 0xc01336e5
> [83155.718040] [<c01030a9>] ? 0xc01030a9
> [83155.718040] [<c0392135>] ? 0xc0392135
> [83155.718040] [<c024495a>] ? 0xc024495a
> [83155.718040] [<c0172e93>] ? 0xc0172e93
> [83155.718040] [<c0243eea>] ? 0xc0243eea
> [83155.718040] [<c0172d58>] ? 0xc0172d58
> [83155.718040] [<c0172ff3>] ? 0xc0172ff3
> [83155.718040] [<c01731b3>] ? 0xc01731b3
> [83155.718040] [<c0173274>] ? 0xc0173274
> [83155.718040] [<c0175e99>] ? 0xc0175e99
> [83155.718040] [<c0175ec4>] ? 0xc0175ec4
> [83155.718040] [<c01af563>] ? 0xc01af563
> [83155.718040] [<c0345992>] ? 0xc0345992
> [83155.718040] [<c030805c>] ? 0xc030805c
> [83155.718040] [<c01af7d7>] ? 0xc01af7d7
> [83155.718040] [<c01af4c0>] ? 0xc01af4c0
> [83155.718040] [<c01af5b0>] ? 0xc01af5b0
> [83155.718040] [<c01af5b0>] ? 0xc01af5b0
> [83155.718040] [<c01af5b0>] ? 0xc01af5b0
> [83155.718040] [<c01af5b0>] ? 0xc01af5b0
> [83155.718040] [<c01af5b0>] ? 0xc01af5b0
> [83155.718040] [<c01af5b0>] ? 0xc01af5b0
> [83155.718040] [<c01af5b0>] ? 0xc01af5b0
> [83155.718040] [<c01af5b0>] ? 0xc01af5b0
> [83155.718040] [<c01af5b0>] ? 0xc01af5b0
> [83155.718040] [<c01af5b0>] ? 0xc01af5b0
> [83155.718040] [<c01af5b0>] ? 0xc01af5b0
> [83155.718040] [<c01af5b0>] ? 0xc01af5b0
> [83155.718040] [<c01af5b0>] ? 0xc01af5b0
> [83155.718040] [<c01af5b0>] ? 0xc01af5b0
> [83155.718040] [<c01af5b0>] ? 0xc01af5b0
> [83155.718040] [<c01af5b0>] ? 0xc01af5b0
> [83155.718040] [<c01af5b0>] ? 0xc01af5b0
> [83155.718040] [<c01af5b0>] ? 0xc01af5b0
> [83155.718040] [<c01aee68>] ? 0xc01aee68
> [83155.718040] [<c01afbaa>] ? 0xc01afbaa
> [83155.718040] [<c039441d>] ? 0xc039441d
> [83155.718040] [<c0390000>] ? 0xc0390000
---
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***
next prev parent reply other threads:[~2011-04-01 20:22 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-04-01 20:17 haael
2011-04-01 20:22 ` Randy Dunlap [this message]
2011-04-02 15:01 ` Phil Turmel
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20110401132225.a9291e25.rdunlap@xenotime.net \
--to=rdunlap@xenotime.net \
--cc=haael@wp.pl \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®