Correctly determine amount of "free memory"

Correctly determine amount of "free memory"

[Gregory Giguashvili]

Hi,

Assuming that I define "free memory" as maximum memory that can be
allocated without causing swapping, is there a way I can give the best
"free memory" amount estimate?

I've tried to play with /proc/meminfo values with some progress, but I'd
like to get a qualified answer from people working with VM bits and
bytes.

Thanks a lot
Giga

Re: Correctly determine amount of "free memory"

[bert hubert]

On Mon, Dec 06, 2004 at 04:53:34PM +0200, Gregory Giguashvili wrote:
> Assuming that I define "free memory" as maximum memory that can be
> allocated without causing swapping, is there a way I can give the best
> "free memory" amount estimate?

On the side, it would not hurt to study 'overcommit' and mlock(2). The
answer to your question is not static unless care is taken.

-- 
http://www.PowerDNS.com      Open source, database driven DNS Software 
http://lartc.org           Linux Advanced Routing & Traffic Control HOWTO

RE: Correctly determine amount of "free memory"

[David Schwartz]

> Assuming that I define "free memory" as maximum memory that can be
> allocated without causing swapping, is there a way I can give the best
> "free memory" amount estimate?

	That's not really a meaningful question. In fact, technically, no matter
how much memory you allocate you won't change the amount of swapping because
allocation has no affect on the working set size.

> I've tried to play with /proc/meminfo values with some progress, but I'd
> like to get a qualified answer from people working with VM bits and
> bytes.

	The qualified answer is that /proc/meminfo gives you the available
information and what number you want depends upon what you plan to do with
it. Swapping is not necessarily the right metric for two reasons. First, a
system might swap even though it's under a perfectly normal (or even low)
load. Second, there are bad memory situations a system can get in even if
it's not swapping, for example, a system with no buffer cache for file I/O
wouldn't be swapping, but it wouldn't perform well either.

	DS

RE: Correctly determine amount of "free memory"

[Gregory Giguashvili]

> On the side, it would not hurt to study 'overcommit' and mlock(2). The
answer to your question is > not static unless care is taken. 
I'm using mlock(2), but the problem is that I need to allocate as much
memory as possible without causing swapping, so I need to have _rough_
estimate before actually allocating the memory.

Thanks,
Giga

Re: Correctly determine amount of "free memory"

[bert hubert]

On Mon, Dec 06, 2004 at 05:57:09PM +0200, Gregory Giguashvili wrote:

> I'm using mlock(2), but the problem is that I need to allocate as much
> memory as possible without causing swapping, so I need to have _rough_
> estimate before actually allocating the memory.

I think a guy called 'Ben' from google went over this a couple of years ago,
the discussion might still be relevant. His email address contained
'google.com' and he posted on lkml.

Too lazy to google for it now but this will point you in the right
direction.

-- 
http://www.PowerDNS.com      Open source, database driven DNS Software 
http://lartc.org           Linux Advanced Routing & Traffic Control HOWTO

RE: Correctly determine amount of "free memory"

[Gregory Giguashvili]

> I think a guy called 'Ben' from google went over this a couple of
years ago, the discussion might 
> still be relevant. His email address contained 'google.com' and he
posted on lkml.

Is http://www.cs.helsinki.fi/linux/linux-kernel/2001-43/0661.html what
you mean? If so, it is about mlock problems in 2.4.9 kernels.
Unfortunatelly, they don't discuss available memory issue even though
it's mentioned a couple of times.

Thanks 
Giga

RE: Correctly determine amount of "free memory"

[Gregory Giguashvili]

> That's not really a meaningful question. 
Sorry, I did my best :)

> The qualified answer is that /proc/meminfo gives you the available
information and what number 
> you want depends upon what you plan to do with it. Swapping is not
necessarily the right metric 
> for two reasons.
I plan to commit the largest chunk of memory in the quickest way. This
operation may be slowed down by swapping - that's why I don't want to
get there. If swapping is not the right metric, what is then? 

According to my humble experiments with 2.4 and 2.6 kernels, some cashed
memory reported in /proc/meminfo is reused and some is swapped. The real
problem here is that I couldn't find the right way to "predict" how much
cached memory will be discarded before starting to swap when system is
low on available RAM.

Thanks
Giga

RE: Correctly determine amount of "free memory"

[Gregory Giguashvili]

Experimenting with my simple memory allocation program for little more
time, I found that Free=MemTotal-Active gives kind of rough
approximation to the "free" memory amount. Can I make it somehow better?

What about SwapCached or Inact_target? Are there any pointers to the
information I might find useful on this subject other than
http://www.redhat.com/advice/tips/meminfo.html?

Thanks 
Giga
P.S. I need this to work with 2.4.19+ and 2.6 kernels...