mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* Kernel memory requirements and BK
@ 2004-11-05 14:46 Art Haas
  2004-11-05 14:55 ` Grzegorz Kulewski
  0 siblings, 1 reply; 4+ messages in thread
From: Art Haas @ 2004-11-05 14:46 UTC (permalink / raw)
  To: linux-kernel

Hi.

I've been having problems with 'bk pull' execution when using kernels
after the 2.6.8/2.6.8.1 releases. My machine has 192M of memory and 100M
of swap, so I believe that the memory requirements for using BK to keep
up with the kernel is sufficient, and when the machine is running with a
2.6.8.1 kernel I can 'bk pull' even if X windows is running. With the
2.6.9 and 2.6.10-rc kernels, BK bombs out with out-of-memory errors once
the repository checking begins. I've run the 'bk pull' under the newer
kernels without X running, as well as shutting down various daemons, and
still things fail with memory errors.

Here's a snippet of /var/log/messages under 2.6.10-rc1 when a
'bk pull' fails:

kernel: oom-killer: gfp_mask=0xd2
kernel: DMA per-cpu:
kernel: cpu 0 hot: low 2, high 6, batch 1
kernel: cpu 0 cold: low 0, high 2, batch 1
kernel: Normal per-cpu:
kernel: cpu 0 hot: low 22, high 66, batch 11
kernel: cpu 0 cold: low 0, high 22, batch 11
kernel: HighMem per-cpu: empty
kernel: 
kernel: Free pages:         420kB (0kB HighMem)
kernel: Active:45877 inactive:64 dirty:0 writeback:0 unstable:0 free:105 slab:1451 mapped:45846 pagetables:195
kernel: DMA free:36kB min:36kB low:72kB high:108kB active:13684kB inactive:0kB present:16384kB pages_scanned:14695 all_unreclaimable? yes
kernel: protections[]: 0 0 0
kernel: Normal free:384kB min:400kB low:800kB high:1200kB active:169824kB inactive:256kB present:180224kB pages_scanned:214215 all_unreclaimable? yes
kernel: protections[]: 0 0 0
kernel: HighMem free:0kB min:128kB low:256kB high:384kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
kernel: protections[]: 0 0 0
kernel: DMA: 1*4kB 0*8kB 0*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 36kB
kernel: Normal: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 384kB
kernel: HighMem: empty
kernel: Swap cache: add 110618, delete 108152, find 38132/44306, race 0+1

The machine used to have only 128M of memory, and 'bk pull' worked
albiet _very_ slowly as the machine went heavily into swap space usage,
but it seemed that once the number of files went over 20,000 or so I had
to add more memory, and then once again 'bk pull' worked without any
problem.

I realize that there have been many, many changes to the kernel between
2.6.8 and the current BK, but my configuration for my builds has
remained mostly the same, so I am at somewhat of a loss to figure out
just where the extra memory usage is coming from.

Is there a known set of changes that explain the additional memory
requirements?

Art Haas
-- 
Man once surrendering his reason, has no remaining guard against absurdities
the most monstrous, and like a ship without rudder, is the sport of every wind.

-Thomas Jefferson to James Smith, 1822

^ permalink raw reply	[flat|nested] 4+ messages in thread
* Re: Kernel memory requirements and BK
@ 2004-11-07  5:02 Chuck Ebbert
  0 siblings, 0 replies; 4+ messages in thread
From: Chuck Ebbert @ 2004-11-07  5:02 UTC (permalink / raw)
  To: Art Haas; +Cc: linux-kernel

Art Haas wrote:

> With the 2.6.9 and 2.6.10-rc kernels, BK bombs out with
> out-of-memory errors once the repository checking begins.

There's nothing wrong with BK's performance that can't be
solved by a terahertz processor and a terabyte of RAM. ;)

But these patches might help:

===============================================================================
# vm_pages_scanned_active_list.patch
#
#       Stop kswapd from looping.
#
#       Patch by Nick Piggin 24 Oct 2004
#       Signed-off-by: Andrew Morton <akpm@osdl.org>
#       Signed-off-by: Linus Torvalds <torvalds@osdl.org>
#       Status: in 2.6.10
#
--- 2.6.9/mm/vmscan.c
+++ 2.6.9.1/mm/vmscan.c
@@ -574,7 +574,6 @@ static void shrink_cache(struct zone *zo
                        nr_taken++;
                }
                zone->nr_inactive -= nr_taken;
-               zone->pages_scanned += nr_taken;
                spin_unlock_irq(&zone->lru_lock);
 
                if (nr_taken == 0)
@@ -675,6 +674,7 @@ refill_inactive_zone(struct zone *zone, 
                }
                pgscanned++;
        }
+       zone->pages_scanned += pgscanned;
        zone->nr_active -= pgmoved;
        spin_unlock_irq(&zone->lru_lock);
 
===============================================================================
# spurious_oomkill.patch
#
#       Prevent spurious out of memory process kills.
#       Reported to work by testers on lkml.
#
#       Patch by Rik van Riel <riel@redhat.com>
#       Status: NOT in 2.6.10
#
--- 2.6.9/mm/vmscan.c
+++ 2.6.9.1/mm/vmscan.c
@@ -379,7 +379,7 @@
 
                referenced = page_referenced(page, 1);
                /* In active use or really unfreeable?  Activate it. */
-               if (referenced && page_mapping_inuse(page))
+               if (referenced && sc->priority && page_mapping_inuse(page))
                        goto activate_locked;
 
 #ifdef CONFIG_SWAP
@@ -715,7 +715,7 @@
                if (page_mapped(page)) {
                        if (!reclaim_mapped ||
                            (total_swap_pages == 0 && PageAnon(page)) ||
-                           page_referenced(page, 0)) {
+                           (page_referenced(page, 0) && sc->priority)) {
                                list_add(&page->lru, &l_active);
                                continue;
                        }
===============================================================================

--Chuck Ebbert  06-Nov-04  23:55:23

^ permalink raw reply	[flat|nested] 4+ messages in thread
end of thread, other threads:[~2004-11-07  5:05 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2004-11-05 14:46 Kernel memory requirements and BK Art Haas
2004-11-05 14:55 ` Grzegorz Kulewski
2004-11-05 15:28   ` Art Haas
2004-11-07  5:02 Chuck Ebbert

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®