From: "Richard B. Johnson" <root@chaos.analogic.com>
To: Miquel van Smoorenburg <miquels@cistron.nl>
Cc: linux-kernel@vger.kernel.org
Subject: Re: 2.5.74-mm3 OOM killer fubared ?
Date: Thu, 10 Jul 2003 09:34:35 -0400 (EDT) [thread overview]
Message-ID: <Pine.LNX.4.53.0307100918410.203@chaos> (raw)
In-Reply-To: <bejnl9$m9l$1@news.cistron.nl>
On Thu, 10 Jul 2003, Miquel van Smoorenburg wrote:
> In article <20030710112728.GX15452@holomorphy.com>,
> William Lee Irwin III <wli@holomorphy.com> wrote:
> >On Thu, Jul 10, 2003 at 11:14:59AM +0000, Miquel van Smoorenburg wrote:
> >> Enough memory free, no problems at all .. yet every few minutes
> >> the OOM killer kills one of my innfeed processes.
> >> I notice that in -mm3 this was deleted relative to -vanilla:
> >>
> >> -
> >> - /*
> >> - * Enough swap space left? Not OOM.
> >> - */
> >> - if (nr_swap_pages > 0)
> >> - return;
> >> .. is that what causes this ? In any case, that should't vene matter -
> >> there's plenty of memory in this box, all buffers and cached, but that
> >> should be easily freed ..
> >
> >This means we're calling into it more often than we should be.
> >Basically, we hit __alloc_pages() with __GFP_WAIT set, find nothing
> >we're allowed to touch, dive into try_to_free_pages(), fall through
> >scanning there, sleep in blk_congestion_wait(), wake up again, try
> >to shrink_slab(), find nothing there either, repeat that 11 more times,
> >and then fall through to out_of_memory()... and this happens at at
> >least 10Hz.
> >
> > since = now - lastkill;
> > if (since < HZ*5)
> > goto out_unlock;
> >
> >try s/goto out_unlock/goto reset/ and let me know how it goes.
>
> But that will only change the rate at which processes are killed,
> not the fact that they are killed in the first place, right ?
>
> As I said I've got plenty memory free ... perhaps I need to tune
> /proc/sys/vm because I've got so much streaming I/O ? Possibly,
> there are too many dirty pages so cleaning them out faster might
> help (and let pflushd do it instead of my single-threaded app)
>
The problem, as I see it, is that you can dirty pages 10-15 times
faster than they can be written to disk. So, you will always
have the possibility of an OOM situation as long as you are I/O
bound. FYI, you can read/write RAM at 1,000+ megabytes/second, but
you can only write to disk at 80 megabytes/second with the fastest
SCSI around, 40 megabytes/second with ATA, 20 megabytes/second with
IDE/DMA, 10 megabytes/second with PIOW, etc. There just aren't
any disks around that will run at RAM speeds so buffered I/O will
always result in full buffers if the I/O is sustained. To completely
solve the OOM situation requires throttling the generation of data.
It is only when the data generation rate is less than or equal to
the data storage rate that you can generate data forever.
A possibility may be to not return control to the writing process
(including swap), until the write completes if RAM gets low. In
other words, stop buffering data in RAM in tight memory situations.
This forces all the tasks to wait and, therefore slows down the
dirty-page and data generation rate to match the RAM available.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.20 on an i686 machine (797.90 BogoMips).
Why is the government concerned about the lunatic fringe? Think about it.
next prev parent reply other threads:[~2003-07-10 13:19 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2003-07-10 11:14 Miquel van Smoorenburg
2003-07-10 11:27 ` William Lee Irwin III
2003-07-10 12:54 ` Miquel van Smoorenburg
2003-07-10 13:34 ` Richard B. Johnson [this message]
2003-07-10 13:45 ` Miquel van Smoorenburg
2003-07-10 14:32 ` Rik van Riel
2003-07-10 17:43 ` Mikulas Patocka
2003-07-10 15:56 ` William Lee Irwin III
2003-07-10 23:05 ` Miquel van Smoorenburg
2003-07-11 1:01 ` William Lee Irwin III
2003-07-10 17:38 ` Andrew Morton
2003-07-10 23:31 ` Miquel van Smoorenburg
2003-07-10 17:46 ` Mikulas Patocka
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Pine.LNX.4.53.0307100918410.203@chaos \
--to=root@chaos.analogic.com \
--cc=linux-kernel@vger.kernel.org \
--cc=miquels@cistron.nl \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®