From: Daniel Phillips <phillips@bonn-fries.net>
To: Sven Heinicke <sven@research.nj.nec.com>, linux-kernel@vger.kernel.org
Cc: Ben Smith <ben@google.com>, Andrea Arcangeli <andrea@suse.de>,
Rik van Riel <riel@conectiva.com.br>
Subject: Re: Google's mm problem - not reproduced on 2.4.13
Date: Fri, 2 Nov 2001 21:58:53 +0100 [thread overview]
Message-ID: <20011102205746Z16039-4784+457@humbolt.nl.linux.org> (raw)
In-Reply-To: <E15yzlQ-00021P-00@starship.berlin> <20011102181758Z16039-4784+420@humbolt.nl.linux.org> <200111022027.fA2KRwe20006@penguin.transmeta.com>
In-Reply-To: <200111022027.fA2KRwe20006@penguin.transmeta.com>
On November 2, 2001 09:27 pm, Linus Torvalds wrote:
> In article <20011102181758Z16039-4784+420@humbolt.nl.linux.org>,
> Daniel Phillips <phillips@bonn-fries.net> wrote:
> >
> >It's hard to see how that could be wrong. Plus, this test program does
> >run under 2.4.9, it just uses way too much CPU on that kernel. So I'd say
> >mm bug.
>
> So how much memory is mlocked?
I'm not sure exactly, I didn't run the test. I *think* it's just over 50% of
physical memory.
> The locked memory will stay in the inactive list (it won't even ever be
> activated, because we don't bother even scanning the mapped locked
> regions), and the inactive list fills up with pages that are completely
> worthless.
Yes, it does various things on various vms. On 2.4.9 it stays on the
inactive list until free memory gets down to rock bottom, then most of it
moves to the active list and the system reaches a steady state where it can
operate, though with kswapd grabbing 99% CPU (two processor system), but the
test does complete. On the current kernel the it dies.
> And the kernel will decide that because most of the unfreeable pages are
> mapped, it needs to do VM scanning, which obviously doesn't help.
>
> Why _does_ this thing do mlock, anyway? What's the point? And how much
> does it try to lock?
It's how the google database engine works, and keeps latency down, by mapping
big database files into memory. I didn't get more information than that on
the application.
> If root wants to shoot himself in the head by mlocking all of memory,
> that's not a VM problem, that's a stupid administrator problem.
In the tests I did, it was about 1 gig out of 2. I'm not sure how much
memory is mlocked in the 3.5 Gig test the one that's failing, but it's
certainly not anything like all of memory. Really, we should be able to
mlock 90%+ of memory without falling over.
--
Daniel
next prev parent reply other threads:[~2001-11-02 20:58 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2001-10-31 18:06 Daniel Phillips
2001-10-31 20:39 ` Daniel Phillips
2001-10-31 20:45 ` Andrea Arcangeli
2001-10-31 21:03 ` Daniel Phillips
2001-10-31 21:53 ` Andreas Dilger
2001-11-01 4:52 ` Daniel Phillips
2001-11-01 16:56 ` undefined reference in 2.2.19 build with Reiserfs (was: Google's mm problem - not reproduced on 2.4.13) Sven Heinicke
2001-11-01 22:39 ` Keith Owens
2001-10-31 22:12 ` Google's mm problem - not reproduced on 2.4.13 Ben Smith
2001-11-01 0:34 ` Andrea Arcangeli
2001-11-02 17:51 ` Sven Heinicke
2001-11-02 18:00 ` Andrea Arcangeli
2001-11-02 18:19 ` Daniel Phillips
2001-11-02 20:27 ` Linus Torvalds
2001-11-02 21:08 ` Ben Smith
2001-11-02 21:20 ` Linus Torvalds
2001-11-02 22:42 ` Ben Smith
2001-11-02 23:15 ` Daniel Phillips
2001-11-03 22:53 ` Adaptec vs Symbios performance Stephan von Krawczynski
2001-11-03 23:01 ` arjan
2001-11-02 21:12 ` Google's mm problem - not reproduced on 2.4.13 Rik van Riel
[not found] ` <200111022027.fA2KRwe20006@penguin.transmeta.com>
2001-11-02 20:58 ` Daniel Phillips [this message]
2001-11-02 18:11 ` Daniel Phillips
2001-11-02 18:48 ` Sven Heinicke
2001-11-02 18:57 ` Daniel Phillips
2001-11-01 0:19 ` Daniel Phillips
2001-11-01 0:29 ` Andrea Arcangeli
2001-11-01 1:17 ` Ben Smith
2001-11-01 1:41 ` Rik van Riel
2001-11-01 1:55 ` Ben Smith
2001-11-01 2:06 ` Andrea Arcangeli
2001-10-31 20:48 ` Rik van Riel
2001-10-31 21:04 ` Daniel Phillips
2001-10-31 21:08 ` Rik van Riel
[not found] <Pine.LNX.4.33.0111021250560.20078-100000@penguin.transmeta.com>
2001-11-02 21:13 ` Linus Torvalds
2001-11-02 21:27 ` Stephan von Krawczynski
2001-11-03 0:16 ` Linus Torvalds
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20011102205746Z16039-4784+457@humbolt.nl.linux.org \
--to=phillips@bonn-fries.net \
--cc=andrea@suse.de \
--cc=ben@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=riel@conectiva.com.br \
--cc=sven@research.nj.nec.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®