mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Daniel Phillips <phillips@bonn-fries.net>
To: Stephan von Krawczynski <skraw@ithnet.com>,
	linux-kernel <linux-kernel@vger.kernel.org>
Cc: Rik van Riel <riel@conectiva.com.br>,
	Marcelo Tosatti <marcelo@conectiva.com.br>
Subject: Re: Memory Problem in 2.4.10-pre2 / __alloc_pages failed
Date: Sun, 2 Sep 2001 03:57:10 +0200	[thread overview]
Message-ID: <20010902015023Z16303-32383+2910@humbolt.nl.linux.org> (raw)
In-Reply-To: <20010901202823.303eb0eb.skraw@ithnet.com>
In-Reply-To: <20010901202823.303eb0eb.skraw@ithnet.com>

On September 1, 2001 08:28 pm, Stephan von Krawczynski wrote:
> On Fri, 31 Aug 2001 21:03:22 +0200
> Daniel Phillips <phillips@bonn-fries.net> wrote:
> 
> > > >  		/* XXX: is pages_min/4 a good amount to reserve for this? */
> > > > +		if (z->free_pages < z->pages_min / 3 && (gfp_mask & __GFP_WAIT) &&
> > > > +				!(current->flags & PF_MEMALLOC))
> > > > +			continue;
> > > Hello Daniel,
> > > 
> > > I tried this patch and it makes _no_ difference. Failures show up in same 
> > > situation and amount. Do you need traces? They look the same
> > 
> > OK, first would you confirm that the frequency of 0 order failures has
> > stayed the same?
> 
> Hello Daniel (and the rest),
> 
> I redid the test and have the following results, based on a 2.4.10-pre2 with
> above patch:
> [...]
>
> And the traces:
> 83 mostly identical errors showed up, all looking like
> 
> Sep  1 15:17:53 admin kernel: cdda2wav: __alloc_pages: 3-order allocation
> failed (gfp=0x20/0).
> [...]
> Trace; fdcf80f5 <[sg]sg_build_reserve+25/48>
> Trace; fdcf6589 <[sg]sg_ioctl+6c5/ae4>
> Trace; fdcf76bd <[sg]sg_build_indi+55/1a8>
> 
> So, there are no 0-order allocs failing in this setup.
> 
> Are you content with having no 0-order failures?

It's a start.  The important thing is to have supported my theory of what is
going on here.  What I did there is probably a good thing, it seems quite
effective for combatting 0 order atomic failures.  In this case you have a
driver that uses a fallback allocation strategy, starting with a 3 order
allocation attempt and dropping down top the next lower size on failure.  If
0 order allocation fails the whole operation fails, and maybe you will lose
a packet.  So 0 order allocations are important, we really want them to
succeed.

The next part of the theory says that the higher order allocations are
failing because of fragmentation.  I put considerable thought into this
today while wandering around in a dungeon in Berlin watching bats (really)
and I will post an article to lkml tomorrow with my findings.  To summarize
briefly here: a Linux system in steady state operation *is* going to show
physical fragmentation so that the chance of a higher order allocation
succeeding becomes very small.  The chance of failure increases
exponentially (or worse) with a) the allocation order and b) the ratio of
allocated to free memory.  The second of these you can control: the higher
you set zone->pages_min the better chance your higher order allocations
will have to succeed.  Do you want a patch for that, to see if this works
in practice?

Of course it would be much better if we had some positive mechanism for
defragging physical memory instead of just relying on chance and hoping
for the best the way we do now.  IMHO, such a mechanism can be built
practically and I'm willing to give it a try.  Basically, kswapd would try
to restore a depleted zone order list by scanning mem_map to find buddy
partners for free blocks of the next lower order.  This strategy, together
with the one used in the patch above, could largly eliminate atomic
allocation failures.  (Although as I mentioned some time ago, getting rid
of them entirely is an impossible problem.)

The question remains why we suddenly started seeing more atomic allocation
failures in the recent Linus trees.  I'll guess that the changes in
scanning strategy have caused the system to spend more time close to the
zone->pages_min amount of free memory.  This idea seems to be supported by
your memstat listings.  In some sense, it's been good to have the issue
forced so that we must come up with ways to make atomic and higher order
allocations less fragile.

--
Daniel

  reply	other threads:[~2001-09-02  1:50 UTC|newest]

Thread overview: 28+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2001-09-01 18:28 Stephan von Krawczynski
2001-09-02  1:57 ` Daniel Phillips [this message]
2001-09-02  2:21   ` Roger Larsson
2001-09-02  4:16     ` Daniel Phillips
2001-09-02 13:48     ` Alex Bligh - linux-kernel
2001-09-02 18:26       ` Daniel Phillips
2001-09-02 19:32         ` Alex Bligh - linux-kernel
2001-09-02 20:24           ` Daniel Phillips
2001-09-02 21:03             ` Alex Bligh - linux-kernel
2001-09-02 20:33           ` Daniel Phillips
2001-09-02 21:14             ` Alex Bligh - linux-kernel
2001-09-02 21:23           ` Daniel Phillips
2001-09-02 21:28             ` Alex Bligh - linux-kernel
2001-09-02 22:36               ` High order memory allocations (was Re: Memory Problem in 2.4.10-pre2 / __alloc_pages failed) Alex Bligh - linux-kernel
     [not found] <689208719.999883299@[10.132.112.53]>
     [not found] ` <20010907154801.028a48e8.skraw@ithnet.com>
2001-09-07 21:13   ` Memory Problem in 2.4.10-pre2 / __alloc_pages failed Stephan von Krawczynski
     [not found] <Pine.LNX.4.33.0109011021570.280-100000@mikeg.weiden.de>
     [not found] ` <20010901055634Z16057-32383+2785@humbolt.nl.linux.org>
2001-09-01 18:54   ` Stephan von Krawczynski
2001-09-02  3:21     ` Mike Galbraith
  -- strict thread matches above, loose matches on Subject: below --
2001-08-29 12:07 Stephan von Krawczynski
2001-08-29 16:47 ` Roger Larsson
2001-08-29 19:18 ` Stephan von Krawczynski
2001-08-30 14:16   ` Stephan von Krawczynski
2001-08-29 23:36 ` Daniel Phillips
2001-08-30 16:49   ` Roger Larsson
2001-08-30 14:46 ` Stephan von Krawczynski
2001-08-30 18:02   ` Daniel Phillips
2001-08-31 10:32   ` Stephan von Krawczynski
2001-08-31 11:06 ` Stephan von Krawczynski
2001-08-31 19:03   ` Daniel Phillips

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20010902015023Z16303-32383+2910@humbolt.nl.linux.org \
    --to=phillips@bonn-fries.net \
    --cc=linux-kernel@vger.kernel.org \
    --cc=marcelo@conectiva.com.br \
    --cc=riel@conectiva.com.br \
    --cc=skraw@ithnet.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®