* Re: Disappearing RAM
2001-04-23 15:58 Disappearing RAM John Weber
@ 2001-04-23 16:23 ` Andi Kleen
2001-04-23 17:20 ` Richard B. Johnson
1 sibling, 0 replies; 3+ messages in thread
From: Andi Kleen @ 2001-04-23 16:23 UTC (permalink / raw)
To: John Weber; +Cc: linux-kernel
On Mon, Apr 23, 2001 at 11:58:37AM -0400, John Weber wrote:
>
> There appears to be quite a difference between the memory usage reported
> by top
> (immediately after boot and minus all user processes) and that reported
> by the kernel
> during boot. Can anyone give me any hints as to why this might happen?
>
> I am assuming that if I subtract all the memory in use by processes
> listed by top from all
> the memory in use that this equals the amount of memory used by the
> kernel.
> These numbers do not add up.
The view given by top is inaccurate because it has no way to detect shared
pages properly. This means for example that when you have glibc mapped into
all your N processes it counted N times its size, even though it only exists
once in memory. Worse it gets when you work with threaded programs; they
share their memory completely, but top/ps has no way to detect this and
counts the single shared process space for every process (I have an
experimental patch to fix the later problem)
In addition there are quite a bit of internal data structures in the kernel
which never appear in top, but eat memory. Most prominent of that is the
page table itself, which eats a few percent of your ram for a management
structure for every page (in 2.2 on 32bit ~40 bytes every 4K of memory;
in 2.4 it is much worse)
> A related question: In the 2.2 kernels, does the kernel pre-allocate
> any amount of memory
> for modules?
No.
-Andi
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Disappearing RAM
2001-04-23 15:58 Disappearing RAM John Weber
2001-04-23 16:23 ` Andi Kleen
@ 2001-04-23 17:20 ` Richard B. Johnson
1 sibling, 0 replies; 3+ messages in thread
From: Richard B. Johnson @ 2001-04-23 17:20 UTC (permalink / raw)
To: John Weber; +Cc: linux-kernel
On Mon, 23 Apr 2001, John Weber wrote:
>
> There appears to be quite a difference between the memory usage reported
> by top
> (immediately after boot and minus all user processes) and that reported
> by the kernel
> during boot. Can anyone give me any hints as to why this might happen?
>
> I am assuming that if I subtract all the memory in use by processes
> listed by top from all
> the memory in use that this equals the amount of memory used by the
> kernel.
> These numbers do not add up.
>
> A related question: In the 2.2 kernels, does the kernel pre-allocate
> any amount of memory
> for modules?
>
> Or is everything I'm seeing related to VM bugs in 2.2.<19 ?
Not VM bugs. There may be a bug (or two), but this is not it. Linux
tries to use ALL the memory that it finds. This is to speed I/O
and other operations. If you do `ls -R /`, you will see that a
lot of memory is now allocated to the directory cache. Basically,
any time you perform some I/O Linux tries to keep the result of
the last I/O operation in memory just in case you need that
data again.
In fact, when disk data are modified, the only time it's guaranteed
to have been all written to the disk is when the disk is `umounted`
during shutdown, unless the disk is mounted in a special way.
When processes need RAM, these caches are reused on, the last
time I checked, a least-used basis. So, if a process needs
memory, some pages from the cache that have not been used for awhile
are torn down (possibly written, if modified) and its RAM is used for
process RAM.
When modules are installed, the kernel dynamically allocates the
RAM for the module. There are three main types of memory that the
kernel can allocate; (1) normal virtual RAM; (2) non-paged
virtual RAM (for stuff that must be present during an interrupt),
and; (3) physical RAM (for DMA buffers). All such RAM is accessed
by its virtual address in the kernel so, once allocated, there are
no special coding provisions required. See /usr/include/linux/mm.h
for details.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.1 on an i686 machine (799.53 BogoMips).
"Memory is like gasoline. You use it up when you are running. Of
course you get it all back when you reboot..."; Actual explanation
obtained from the Micro$oft help desk.
^ permalink raw reply [flat|nested] 3+ messages in thread