Is swap == 2 * RAM a permanent thing?

Is swap == 2 * RAM a permanent thing?

[Mike A. Harris]

Is the fact that we're supposed to use double the RAM size as
swap a permanent thing or a temporary annoyance that will get
tweaked/fixed in the future at some point during 2.4.x perhaps?

What are the technical reasons behind this change?  Just curious
as I see a lot of people are complaining about having to
repartition (although a slower swap file could be used also).

I'm curious because I currently have 96Mb of RAM and 256Mb of
swap, but swap rarely if ever gets used, and performance is very
good.  This is with 2.2.18 I'm speaking.

I'm planning on upping my RAM to 256Mb or more in the near future
however, and going to 2.4.3 or 2.4.4 when released, and since
96Mb does the job for me already it would suck to have to
increase swap at the same time when it never gets used as it is
right now.

Would it be better to make part of RAM a ramdisk and swap to
that?  Sounds like we're going backwards IMHO, but I don't
understand the details, so I'll let someone that does explain
them to me.

Thanks in advance.



----------------------------------------------------------------------
    Mike A. Harris  -  Linux advocate  -  Free Software advocate
          This message is copyright 2001, all rights reserved.
  Views expressed are my own, not necessarily shared by my employer.
----------------------------------------------------------------------
If it weren't for C, we'd all be programming in BASI and OBOL.

Re: Is swap == 2 * RAM a permanent thing?

[Rik van Riel]

On Thu, 15 Mar 2001, Mike A. Harris wrote:

> Is the fact that we're supposed to use double the RAM size as
> swap a permanent thing or a temporary annoyance that will get
> tweaked/fixed in the future at some point during 2.4.x perhaps?
>
> What are the technical reasons behind this change?

The reason is that the Linux 2.4 kernel no longer reclaims swap
space on swapin (2.2 reclaimed swap space on write access, which
lead to fragmented swap space in lots of workloads).

This means that a lot of memory ends up "duplicated" in RAM and
in swap.

I plan on doing some code to reclaim swap space when we run out,
but Linus doesn't seem to like that idea very much. His argument
(when you're OOM, you should just fail instead of limp along)
makes a lot of sense, however, and the reclaiming of swap space
isn't really high on my TODO list ...

OTOH, for people who have swap < RAM and use it just as a small
overflow area, Linus' argument falls short, so I guess some time
in the future we will have code to reclaim swap space when needed.

regards,

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com.br/

Re: Is swap == 2 * RAM a permanent thing?

[Mike Harrold]

> The reason is that the Linux 2.4 kernel no longer reclaims swap
> space on swapin (2.2 reclaimed swap space on write access, which
> lead to fragmented swap space in lots of workloads).
> 
> This means that a lot of memory ends up "duplicated" in RAM and
> in swap.
> 
> I plan on doing some code to reclaim swap space when we run out,
> but Linus doesn't seem to like that idea very much. His argument
> (when you're OOM, you should just fail instead of limp along)
> makes a lot of sense, however, and the reclaiming of swap space
> isn't really high on my TODO list ...
> 
> OTOH, for people who have swap < RAM and use it just as a small
> overflow area, Linus' argument falls short, so I guess some time
> in the future we will have code to reclaim swap space when needed.

I have some questions on this.

1) If a process uses swap space and then later (after being paged
   into memory -- or even not) it completes, is killed, etc., is
   the swap space reclaimed then?

2) If a process uses swap, is paged into memory, and is then swapped
   out again, does it re-use the same swap as before?

If the answer to either question is no, then IMHO, that's a pretty
serious design flaw.

Regards,

/Mike

Re: Is swap == 2 * RAM a permanent thing?

[Rik van Riel]

On Thu, 15 Mar 2001, Mike Harrold wrote:

> 1) If a process uses swap space and then later (after being paged
>    into memory -- or even not) it completes, is killed, etc., is
>    the swap space reclaimed then?
>
> 2) If a process uses swap, is paged into memory, and is then swapped
>    out again, does it re-use the same swap as before?

Yes and yes.

Rik
--
Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml

Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: Is swap == 2 * RAM a permanent thing?

[christophe barbe]

Please Rik, could you explain what you mean with "reclaim swap space when we run out".
In my (limited) understanding, when there's no more free memory (ram and swap space), the kernel starts to kill process (and the choice is a difficult point).
Are you proposing to add an API to reclaim swap instead of killing process ?

Other thing (certainly related) what is supposed to have changed between 2.2 and 2.4.

Thanks,
Christophe

On jeu, 15 mar 2001 14:08:50 Rik van Riel wrote:
> On Thu, 15 Mar 2001, Mike A. Harris wrote:
> 
> > Is the fact that we're supposed to use double the RAM size as
> > swap a permanent thing or a temporary annoyance that will get
> > tweaked/fixed in the future at some point during 2.4.x perhaps?
> >
> > What are the technical reasons behind this change?
> 
> The reason is that the Linux 2.4 kernel no longer reclaims swap
> space on swapin (2.2 reclaimed swap space on write access, which
> lead to fragmented swap space in lots of workloads).
> 
> This means that a lot of memory ends up "duplicated" in RAM and
> in swap.
> 
> I plan on doing some code to reclaim swap space when we run out,
> but Linus doesn't seem to like that idea very much. His argument
> (when you're OOM, you should just fail instead of limp along)
> makes a lot of sense, however, and the reclaiming of swap space
> isn't really high on my TODO list ...
> 
> OTOH, for people who have swap < RAM and use it just as a small
> overflow area, Linus' argument falls short, so I guess some time
> in the future we will have code to reclaim swap space when needed.
> 
> regards,
> 
> Rik
> --
> Virtual memory is like a game you can't win;
> However, without VM there's truly nothing to lose...
> 
> 		http://www.surriel.com/
> http://www.conectiva.com/	http://distro.conectiva.com.br/
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 
-- 
Christophe Barbé
Software Engineer
Lineo High Availability Group
42-46, rue Médéric
92110 Clichy - France
phone (33).1.41.40.02.12
fax (33).1.41.40.02.01
www.lineo.com

Re: Is swap == 2 * RAM a permanent thing?

[Rik van Riel]

On Thu, 15 Mar 2001, christophe barbe wrote:

> Please Rik, could you explain what you mean with "reclaim swap
> space when we run out". In my (limited) understanding, when
> there's no more free memory (ram and swap space), the kernel
> starts to kill process (and the choice is a difficult point).
> Are you proposing to add an API to reclaim swap instead of
> killing process ?

When we swap something in from swap, it is in effect "duplicated"
in memory and swap. Freeing the swap space of these duplicates
will mean we have, effectively, more swap space.

Rik
--
Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml

Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: Is swap == 2 * RAM a permanent thing?

[christophe barbe]

Ok I understand better now.
So when we swap in, the place is still reserved in swap for the next time we swap off the same memory part.
The swap is freed only when the owner terminates.
Then when we need more memory, we need to swap off but we don't use the duplicated part.
I understand that it is to avoid fragmentation but this can lead to false OOM situation (false because we have free memory in swap but it's reserved to avoid fragmentation).

Christophe

On ven, 16 mar 2001 00:26:35 Rik van Riel wrote:
> On Thu, 15 Mar 2001, christophe barbe wrote:
> 
> > Please Rik, could you explain what you mean with "reclaim swap
> > space when we run out". In my (limited) understanding, when
> > there's no more free memory (ram and swap space), the kernel
> > starts to kill process (and the choice is a difficult point).
> > Are you proposing to add an API to reclaim swap instead of
> > killing process ?
> 
> When we swap something in from swap, it is in effect "duplicated"
> in memory and swap. Freeing the swap space of these duplicates
> will mean we have, effectively, more swap space.
> 
> Rik
> --
> Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml
> 
> Virtual memory is like a game you can't win;
> However, without VM there's truly nothing to lose...
> 
> 		http://www.surriel.com/
> http://www.conectiva.com/	http://distro.conectiva.com/
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 
-- 
Christophe Barbé
Software Engineer
Lineo High Availability Group
42-46, rue Médéric
92110 Clichy - France
phone (33).1.41.40.02.12
fax (33).1.41.40.02.01
www.lineo.com

Re: Is swap == 2 * RAM a permanent thing?

[James Lewis Nance]

On Thu, Mar 15, 2001 at 08:26:35PM -0300, Rik van Riel wrote:
> When we swap something in from swap, it is in effect "duplicated"
> in memory and swap. Freeing the swap space of these duplicates
> will mean we have, effectively, more swap space.

Hi Rik,
    Thanks for the explanation.  It brings another question to mind.  Lets
assume that I have two 16M processes and 32M of swap space.  Both the
processes have been swapped out at some point in time so the swap space is
full.  A third process is running and tries to allocate some memory, and
the kernel has no free pages.  Since swap is full, will the kernel kill my
process, or will it try and page out one of the processes that does have
space on swap?

Thanks,

Jim

Re: Is swap == 2 * RAM a permanent thing?

[Rik van Riel]

On Thu, 15 Mar 2001, James Lewis Nance wrote:
> On Thu, Mar 15, 2001 at 08:26:35PM -0300, Rik van Riel wrote:
> > When we swap something in from swap, it is in effect "duplicated"
> > in memory and swap. Freeing the swap space of these duplicates
> > will mean we have, effectively, more swap space.
>
>     Thanks for the explanation.  It brings another question to
> mind.  Lets assume that I have two 16M processes and 32M of swap
> space.  Both the processes have been swapped out at some point
> in time so the swap space is full.  A third process is running
> and tries to allocate some memory, and the kernel has no free
> pages.  Since swap is full, will the kernel kill my process, or
> will it try and page out one of the processes that does have
> space on swap?

It will end up swapping out the two processes which already have
space in swap ... even if the 3rd process is idle.

In that situation you could argue for 2 things:

1) the kernel should reclaim space when swap is full
2) you need more swap

I guess we'll want a bit of both, possibly with 1) being an
optional thing (since swap fragmentation could well be as
bad for performance as swapping out the wrong thing).

regards,

Rik
--
Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml

Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

kernel benchmark

[gis88530]

Hi,

I use kernprof+gprof to measure the 2.2.16 kernel,
but the scale is mini-second. 
So I use do_gettimeofday( ) kernel function call to measure 
the latency. (This function support micro-second scale.)

Moreover, I use SmartBits packet generator to generate
the specific network traffic load. The environment is
as follows. However, the result are very funny. I think 
that latency should increase progressively when load
increase, but the result are unable explaining.
Could you give me some hint? Thanks a lot.

1518byte packet
load    latency(us)
1%    13.1284
10%    14.1629
20%    12.6558
30%    11.1056
40%    10.7510
50%    10.4148
60%    10.3337
70%    10.1038
80%    10.1103
90%    10.3634
100%    11.2367

64byte packet
load    latency(us)
1%    3.6767
10%    2.7696
20%    4.3926
30%    2.8135
40%    8.2552
50%    5.3088
60%    9.3744
70%    23.6247
80%    8.5351
90%    9.7217
100%    13.065

Benchmark Environment:
+---smartbits<---+
|                         |
+---->Linux-----+


* The do_gettimeofday function call is as follows:
--------------
do_gettimeofday(&begin);
...
(kernel do something)
...
do_gettimeofday(&end);
if (end.tv_usec < begin.tv_usec) {
    end.tv_usec += 1000000; end.tv_sec--;
}
end.tv_sedc -= begin.tv_sec;
end.tv_usec -= begin.tv_usec;
return ((end.tv_sec*1000000) + end.tv_usec);

Re: Is swap == 2 * RAM a permanent thing?

[Ketil Froyn]

On Thu, 15 Mar 2001, Mike A. Harris wrote:

> Is the fact that we're supposed to use double the RAM size as
> swap a permanent thing or a temporary annoyance that will get
> tweaked/fixed in the future at some point during 2.4.x perhaps?

You're not supposed to do anything, that's just a general rule of thumb.
If your system hardly ever swaps, use a swapfile, because speed doesn't
matter a lot anyway.

> Would it be better to make part of RAM a ramdisk and swap to
> that?

No, don't do that. Physical memory is better than swap. Swap is a backup
for physical memory, so that you can run programs that use more memory
than you have. You don't really *need* swap, linux works fine without.

Ketil