From: "Torsten Kaiser" <just.for.lkml@googlemail.com>
To: "Andrew Morton" <akpm@linux-foundation.org>
Cc: linux-kernel@vger.kernel.org
Subject: Re: 2.6.23-mm1
Date: Sat, 13 Oct 2007 20:35:16 +0200 [thread overview]
Message-ID: <64bb37e0710131135s41790cafxd010d9eb04088728@mail.gmail.com> (raw)
In-Reply-To: <20071013111853.7e67c6c3.akpm@linux-foundation.org>
On 10/13/07, Andrew Morton <akpm@linux-foundation.org> wrote:
> On Sat, 13 Oct 2007 20:05:19 +0200 "Torsten Kaiser" <just.for.lkml@googlemail.com> wrote:
>
> > The only thing I noted during load testing (updating Gentoo ==
> > compiling and installing) was, that there seems to be memory leak.
> > After ~2h 2.5 of my 4Gb where gone. But there where to many things
> > going on to pinpoint it... (NFSv4 over eth1394?)
>
> Please send /proc/meminfo and /proc/slabinfo after the leak has been
> happening for a while.
>
> Sometimes `echo m > /proc/sysrq_trigger ; dmesg -s 1000000' will
> provide useful info.
I don't have the meminfo or slabinfo, only the output from SysRq+M:
SysRq : Show Memory
Mem-info:
Node 0 DMA per-cpu:
CPU 0: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0
CPU 1: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0
CPU 2: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0
CPU 3: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: Hot: hi: 186, btch: 31 usd: 173 Cold: hi: 62, btch: 15 usd: 29
CPU 1: Hot: hi: 186, btch: 31 usd: 69 Cold: hi: 62, btch: 15 usd: 4
CPU 2: Hot: hi: 186, btch: 31 usd: 82 Cold: hi: 62, btch: 15 usd: 13
CPU 3: Hot: hi: 186, btch: 31 usd: 71 Cold: hi: 62, btch: 15 usd: 3
Node 1 DMA32 per-cpu:
CPU 0: Hot: hi: 186, btch: 31 usd: 171 Cold: hi: 62, btch: 15 usd: 0
CPU 1: Hot: hi: 186, btch: 31 usd: 57 Cold: hi: 62, btch: 15 usd: 0
CPU 2: Hot: hi: 186, btch: 31 usd: 171 Cold: hi: 62, btch: 15 usd: 6
CPU 3: Hot: hi: 186, btch: 31 usd: 158 Cold: hi: 62, btch: 15 usd: 7
Node 1 Normal per-cpu:
CPU 0: Hot: hi: 186, btch: 31 usd: 0 Cold: hi: 62, btch: 15 usd: 0
CPU 1: Hot: hi: 186, btch: 31 usd: 0 Cold: hi: 62, btch: 15 usd: 0
CPU 2: Hot: hi: 186, btch: 31 usd: 170 Cold: hi: 62, btch: 15 usd: 13
CPU 3: Hot: hi: 186, btch: 31 usd: 172 Cold: hi: 62, btch: 15 usd: 19
Active:236368 inactive:63289 dirty:365 writeback:0 unstable:0
free:28366 slab:43372 mapped:13718 pagetables:2356 bounce:0
Node 0 DMA free:8048kB min:16kB low:20kB high:24kB active:0kB
inactive:0kB present:8876kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 2004 2004 2004
Node 0 DMA32 free:98364kB min:4040kB low:5048kB high:6060kB
active:527764kB inactive:107636kB present:2052320kB pages_scanned:0
all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 1 DMA32 free:5824kB min:3040kB low:3800kB high:4560kB
active:223216kB inactive:135068kB present:1544000kB pages_scanned:0
all_unreclaimable? no
lowmem_reserve[]: 0 0 505 505
Node 1 Normal free:1228kB min:1016kB low:1268kB high:1524kB
active:194492kB inactive:10452kB present:517120kB pages_scanned:0
all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 6*4kB 7*8kB 4*16kB 5*32kB 3*64kB 3*128kB 4*256kB 2*512kB
1*1024kB 2*2048kB 0*4096kB = 8048kB
Node 0 DMA32: 10629*4kB 4229*8kB 1240*16kB 4*32kB 0*64kB 0*128kB
0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 98364kB
Node 1 DMA32: 442*4kB 5*8kB 3*16kB 2*32kB 1*64kB 0*128kB 1*256kB
1*512kB 1*1024kB 1*2048kB 0*4096kB = 5824kB
Node 1 Normal: 182*4kB 17*8kB 9*16kB 1*32kB 3*64kB 0*128kB 0*256kB
0*512kB 0*1024kB 0*2048kB 0*4096kB = 1232kB
Swap cache: add 64, delete 64, find 35/40, race 0+0
Free swap = 9775416kB
Total swap = 9775416kB
Free swap: 9775416kB
1048576 pages of RAM
35172 reserved pages
106120 pages shared
0 pages swap cached
When I noticed the leak, I stopped the emerge (the Gentoo update aka
the compiling tasks) and cleared the tmpfs that was used for it. I
also stopped X.
~1Gb was used as pagecache, slubinfo showed around 200..300 Mb used
for slab, but only ~350Mb free out of 4Gb.
I did not see any tasks with abnormal memory sizes.
> The page-owner code can pinpoint a leak source. See
> ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.23/2.6.23-mm1/broken-out/page-owner-tracking-leak-detector.patch
>
> Enable CONFIG_DEBUG_SLAB_LEAK, check out /proc/slab_allocators
>
I suspected that I need more debugging options to track this.
Currently compiling more packages again, but no obvious leak.
When the leak occurred I noticed the disk throughput falling /
stalling several second and this appeared in the syslog:
ohci1394: fw-host0: Waking dma ctx=0 ... processing is probably too slow
ohci1394: fw-host0: Waking dma ctx=0 ... processing is probably too slow
I'm still using the old firewire stack because of eth1394.
I will mail again, when I have more info.
Torsten
next prev parent reply other threads:[~2007-10-13 18:35 UTC|newest]
Thread overview: 139+ messages / expand[flat|nested] mbox.gz Atom feed top
2007-10-12 4:31 2.6.23-mm1 Andrew Morton
2007-10-12 5:03 ` 2.6.23-mm1 KAMEZAWA Hiroyuki
2007-10-12 6:42 ` 2.6.23-mm1 Andrew Morton
2007-10-12 6:46 ` 2.6.23-mm1 Al Viro
2007-10-12 7:13 ` 2.6.23-mm1 Andrew Morton
2007-10-12 18:06 ` [PATCH net-2.6] uml: hard_header fix Stephen Hemminger
2007-10-12 19:04 ` 2.6.23-mm1 Al Viro
2007-10-12 19:47 ` 2.6.23-mm1 thread exit_group issue Mathieu Desnoyers
2007-10-12 20:01 ` Andrew Morton
2007-10-13 1:03 ` Andrew Morton
2007-10-13 11:48 ` Oleg Nesterov
2007-10-13 12:02 ` Oleg Nesterov
2007-10-13 17:49 ` Andrew Morton
2007-10-14 4:04 ` Mathieu Desnoyers
2007-10-12 7:25 ` 2.6.23-mm1 KAMEZAWA Hiroyuki
2007-10-12 8:36 ` 2.6.23-mm1 Sam Ravnborg
2007-10-12 8:31 ` 2.6.23-mm1 Torsten Kaiser
2007-10-12 8:37 ` 2.6.23-mm1 Andrew Morton
2007-10-12 12:46 ` 2.6.23-mm1 Torsten Kaiser
2007-10-13 8:01 ` 2.6.23-mm1 Torsten Kaiser
2007-10-13 10:55 ` 2.6.23-mm1 Jeff Garzik
2007-10-13 12:03 ` 2.6.23-mm1 Torsten Kaiser
2007-10-13 12:19 ` 2.6.23-mm1 Jeff Garzik
2007-10-13 14:32 ` 2.6.23-mm1 Torsten Kaiser
2007-10-13 14:40 ` 2.6.23-mm1 Torsten Kaiser
2007-10-13 15:13 ` 2.6.23-mm1 Torsten Kaiser
2007-10-13 17:48 ` 2.6.23-mm1 Jeff Garzik
2007-10-13 18:05 ` 2.6.23-mm1 Torsten Kaiser
2007-10-13 18:18 ` 2.6.23-mm1 Andrew Morton
2007-10-13 18:35 ` Torsten Kaiser [this message]
2007-10-14 11:54 ` 2.6.23-mm1 Torsten Kaiser
2007-10-14 18:39 ` 2.6.23-mm1 Andrew Morton
2007-10-14 19:12 ` 2.6.23-mm1 Torsten Kaiser
2007-10-14 19:26 ` 2.6.23-mm1 Andrew Morton
2007-10-14 19:40 ` 2.6.23-mm1 Torsten Kaiser
2007-10-14 22:03 ` 2.6.23-mm1 Milan Broz
2007-10-15 6:50 ` 2.6.23-mm1 Jens Axboe
2007-10-15 7:31 ` 2.6.23-mm1 Neil Brown
2007-10-15 7:45 ` 2.6.23-mm1 Jens Axboe
2007-10-13 18:41 ` 2.6.23-mm1 Jeff Garzik
2007-10-12 6:48 ` 2.6.23-mm1 Cedric Le Goater
2007-10-12 6:51 ` [PATCH] add missing parenthesis in cfe_writeblk() macro Mariusz Kozlowski
2007-10-12 7:44 ` 2.6.23-mm1 - build failure on axonram Kamalesh Babulal
2007-10-12 9:42 ` Build Failure (Was Re: 2.6.23-mm1) Dhaval Giani
2007-10-12 20:38 ` 2.6.23-mm1 Laurent Riffard
2007-10-12 21:00 ` 2.6.23-mm1 Andrew Morton
2007-10-13 9:29 ` [PATCH] Reiser4: Drop 'size' argument from bio_endio and bi_end_io Laurent Riffard
2007-10-13 10:10 ` Jens Axboe
2007-10-14 13:09 ` Edward Shishkin
2007-10-15 16:13 ` 2.6.23-mm1 Zan Lynx
2007-10-12 21:32 ` 2.6.23-mm1 Rafael J. Wysocki
2007-10-15 16:09 ` 2.6.23-mm1 Mark Gross
2007-10-15 20:40 ` 2.6.23-mm1 Rafael J. Wysocki
2007-10-16 19:58 ` 2.6.23-mm1 Mark Gross
2007-10-16 20:28 ` 2.6.23-mm1 Rafael J. Wysocki
2007-10-16 23:31 ` 2.6.23-mm1 Mark Gross
2007-10-17 21:15 ` [PATCH] static initialization with blocking notifiers. was :wqRe: 2.6.23-mm1 Mark Gross
2007-10-17 17:21 ` [PATCH] static initialization and blocking notification for pm_qos... was 2.6.23-mm1 Mark Gross
2007-10-13 4:35 ` 2.6.23-mm1 - Build failure on rgmii Kamalesh Babulal
2007-10-13 4:44 ` 2.6.23-mm1 - build failure with advansys Kamalesh Babulal
2007-10-13 6:52 ` Andrew Morton
2007-10-18 0:07 ` Paul Mackerras
2007-10-18 1:48 ` Matthew Wilcox
2007-10-13 15:50 ` 2.6.23-mm1 pm_prepare() and _finish() w/ args vs. without Joseph Fannin
2007-10-13 17:22 ` Rafael J. Wysocki
2007-10-13 18:40 ` Joseph Fannin
2007-10-13 19:13 ` Rafael J. Wysocki
2007-10-14 19:47 ` Joseph Fannin
2007-10-14 20:20 ` Rafael J. Wysocki
2007-10-15 20:55 ` Rafael J. Wysocki
2007-10-16 17:29 ` Joseph Fannin
2007-10-13 17:12 ` 2.6.23-mm1 Gabriel C
2007-10-13 18:01 ` 2.6.23-mm1 Andrew Morton
2007-10-13 18:08 ` 2.6.23-mm1 Gabriel C
2007-10-15 16:28 ` 2.6.23-mm1 Dave Hansen
2007-10-13 17:58 ` Suspend Broken (Re: 2.6.23-mm1) Dhaval Giani
2007-10-13 18:33 ` Rafael J. Wysocki
2007-10-14 4:26 ` Dhaval Giani
2007-10-14 14:19 ` Rafael J. Wysocki
2007-10-13 22:11 ` [2.6.23-mm1] CONFIG_LOCALVERSION handling broken Tilman Schmidt
2007-10-17 20:27 ` Sam Ravnborg
2007-10-17 23:06 ` Tilman Schmidt
2007-10-27 15:19 ` Tilman Schmidt
2007-10-27 15:28 ` Sam Ravnborg
2007-10-14 22:34 ` 2.6.23-mm1: BUG in reiserfs_delete_xattrs Laurent Riffard
2007-10-15 8:40 ` Christoph Hellwig
2007-10-15 18:31 ` Jeff Mahoney
2007-10-15 20:06 ` Laurent Riffard
2007-10-15 20:23 ` Jeff Mahoney
2007-10-17 8:59 ` Christoph Hellwig
2007-10-17 8:58 ` Christoph Hellwig
2007-10-17 14:55 ` Jeff Mahoney
2007-10-15 19:51 ` Laurent Riffard
2007-10-15 6:18 ` [PATCH] Add irq protection in the percpu-counters cpu-hotplug-callback path Gautham R Shenoy
2007-10-15 12:28 ` nfs mmap adventure (was: 2.6.23-mm1) Peter Zijlstra
2007-10-15 15:43 ` Trond Myklebust
2007-10-15 14:06 ` David Howells
2007-10-15 15:51 ` Trond Myklebust
2007-10-15 16:38 ` Peter Zijlstra
2007-10-15 23:27 ` David Howells
2007-10-16 1:46 ` Nick Piggin
2007-10-16 7:18 ` 2.6.23-mm1 - regression- PowerPC link failure at arch/powerpc/kernel/head_64.o Kamalesh Babulal
2007-10-16 7:28 ` Andrew Morton
2007-10-16 7:44 ` Kamalesh Babulal
2007-10-21 6:42 ` Kamalesh Babulal
2007-10-27 5:05 ` Stephen Rothwell
2007-10-17 7:01 ` 2.6.23-mm1 KAMEZAWA Hiroyuki
2007-10-17 9:02 ` 2.6.23-mm1 Andrew Morton
2007-10-17 9:10 ` 2.6.23-mm1 Jiri Kosina
2007-10-17 9:36 ` 2.6.23-mm1 KAMEZAWA Hiroyuki
2007-10-17 11:42 ` 2.6.23-mm1 Jiri Kosina
2007-10-17 12:33 ` 2.6.23-mm1 KAMEZAWA Hiroyuki
2007-10-19 9:07 ` PIE randomization (was Re: 2.6.23-mm1) Jiri Kosina
2007-10-19 21:54 ` 2.6.23-mm1 Jiri Kosina
2007-10-17 15:54 ` 2.6.23-mm1 - list_add corruption in cgroup Cedric Le Goater
2007-10-18 15:56 ` Paul Menage
2007-10-19 22:11 ` Paul Menage
2007-10-18 12:06 ` 2.6.23-mm1 - powerpc - Build fails at arch/powerpc/boot/inflate.o Kamalesh Babulal
2007-10-18 12:23 ` Paul Mackerras
2007-10-18 13:20 ` Kamalesh Babulal
2007-10-20 4:57 ` oops in lbmIODone, fails to boot [Re: 2.6.23-mm1] Mattia Dongili
2007-10-20 5:34 ` Andrew Morton
2007-10-20 12:18 ` Dave Kleikamp
2007-10-21 5:44 ` Mattia Dongili
2007-10-20 5:13 ` 2.6.23-mm1 - autofs broken Rik van Riel
2007-10-20 5:39 ` Andrew Morton
2007-10-20 5:54 ` Rik van Riel
2007-10-20 5:54 ` Rik van Riel
2007-10-20 14:56 ` Rik van Riel
2007-10-22 22:03 ` Dave Hansen
2007-10-22 3:45 ` Ian Kent
2007-10-22 16:46 ` Rik van Riel
2007-10-21 5:58 ` mysqld prevents s2ram [Re: 2.6.23-mm1] Mattia Dongili
2007-10-21 6:28 ` Mattia Dongili
2007-10-21 9:58 ` Pavel Machek
2007-10-21 11:53 ` Rafael J. Wysocki
2007-10-22 18:40 ` kernel panic when running tcpdump Mariusz Kozlowski
2007-10-22 19:03 ` Andrew Morton
2007-10-22 21:16 ` Mariusz Kozlowski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=64bb37e0710131135s41790cafxd010d9eb04088728@mail.gmail.com \
--to=just.for.lkml@googlemail.com \
--cc=akpm@linux-foundation.org \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®