* [PATCH] mm/mglru: deactivate anon folios into the oldest generation
@ 2026-11-12 8:19 Ridong Chen
2026-11-12 8:38 ` Kairui Song
0 siblings, 1 reply; 8+ messages in thread
From: Ridong Chen @ 2026-11-12 8:19 UTC (permalink / raw)
To: David Hildenbrand, Andrew Morton
Cc: Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Yu Zhao, Kairui Song, Qi Zheng,
Shakeel Butt, Barry Song, Axel Rasmussen, Yuanchu Xie, Wei Xu,
Baoquan He, Baolin Wang, linux-mm, linux-kernel, Ridong Chen,
Ridong Chen
From: Ridong Chen <chenridong@xiaomi.com>
folio_deactivate() is used by MADV_COLD and DAMOS_LRU_DEPRIO to move a
folio towards eviction. lru_deactivate() does that by deleting the folio
from its LRU list, clearing PG_active and PG_referenced, and adding it
back with lruvec_add_folio().
With MGLRU, lru_gen_folio_seq() places a swap-backed folio that is not
in the swap cache, i.e. an anon or shmem folio, into max_seq - 1,
regardless of its refs. That generation is active per
lru_gen_is_active(), so such a folio never reaches an inactive
generation, and a folio that has aged into min_seq + 1 is moved to a
younger generation instead. PGDEACTIVATE is still counted. Other folios
are not affected: with their refs cleared they are placed into the
oldest generation.
On a 4-generation memcg, 64MB of aged anon memory in min_seq + 1 ends up
in max_seq - 1 after MADV_COLD, while 64MB of file memory in the same
generation ends up in min_seq.
As a result, MADV_COLD can make anon memory less likely to be reclaimed.
It can be reproduced with a single process in a memcg with swap enabled:
1) map two 64MB anon regions A and B and fault them in, A first;
2) age the memcg twice through /sys/kernel/debug/lru_gen, so that both
regions are in the same generation, min_seq + 1;
3) apply MADV_COLD to B only, A is the control;
4) reclaim the size of one region from anon only:
echo "64M swappiness=max" > memory.reclaim;
5) count the swapped out pages of A and B from /proc/self/pagemap.
B is expected to be swapped out instead of A. Three runs on mm-new:
A swapped out B (MADV_COLD) swapped out
before 16384 pages 15 ~ 128 pages
after 0 ~ 157 pages 16384 pages
Use lruvec_add_folio_tail() for MGLRU, which treats the folio as
reclaiming and adds it to the tail of the oldest generation, the same
place lru_deactivate_file() puts a clean file folio. The other folios,
which were added to the head of the oldest generation, are now added to
its tail as well, so they are reclaimed first as MADV_COLD intends.
Fixes: 081488051d28 ("mm/mglru: fix underprotected page cache")
Signed-off-by: Ridong Chen <chenridong@xiaomi.com>
---
This patch was developed with the help of an AI coding assistant
(Claude).
The reproducer, run as root in a memcg with swap enabled and debugfs
mounted, e.g. "mkdir /sys/fs/cgroup/t; ./reclaim-order /sys/fs/cgroup/t":
// SPDX-License-Identifier: GPL-2.0
/*
* Two 64MB anon regions, A (hot) and B (MADV_COLD). Reclaim 64MB via
* memory.reclaim (swappiness=max -> anon only). Report how much of each
* region was swapped out (mincore can't see swap; use /proc/self/pagemap).
*/
#define _GNU_SOURCE
#include <sched.h>
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/mman.h>
static unsigned long long memcg_id;
static const char *cg_name;
/* force one aging step on node 0 of our memcg via debugfs */
static void aging(void)
{
FILE *f = fopen("/sys/kernel/debug/lru_gen", "r");
char line[512], cmd[128];
unsigned long max = 0, s, x1, x2, x3;
int in = 0, fd, len;
while (fgets(line, sizeof(line), f)) {
unsigned long long id; char path[256];
if (sscanf(line, "memcg %llu %255s", &id, path) == 2) {
in = !strcmp(path, cg_name);
if (in) memcg_id = id;
continue;
}
if (in && sscanf(line, " %lu %lu %lu %lu", &s, &x1, &x2, &x3) == 4)
max = s;
}
fclose(f);
len = snprintf(cmd, sizeof(cmd), "+ %llu 0 %lu 200 1\n", memcg_id, max);
fd = open("/sys/kernel/debug/lru_gen", O_WRONLY);
if (write(fd, cmd, len) != len) perror("aging");
close(fd);
}
static long swapped(char *p, size_t sz)
{
int fd = open("/proc/self/pagemap", O_RDONLY);
long n = 0;
for (size_t off = 0; off < sz; off += 4096) {
uint64_t e;
pread(fd, &e, 8, ((uintptr_t)(p + off) / 4096) * 8);
if (e & (1ULL << 62))
n++;
}
close(fd);
return n;
}
int main(int argc, char **argv)
{
const char *cg = argv[1];
cg_name = strrchr(cg, '/');
size_t sz = 64UL << 20;
char buf[256], *a, *b, *d;
cpu_set_t set;
int fd;
CPU_ZERO(&set); CPU_SET(0, &set);
sched_setaffinity(0, sizeof(set), &set);
snprintf(buf, sizeof(buf), "%s/cgroup.procs", cg);
fd = open(buf, O_WRONLY); dprintf(fd, "%d\n", getpid()); close(fd);
a = mmap(NULL, sz, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
b = mmap(NULL, sz, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
d = mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
madvise(a, sz, MADV_NOHUGEPAGE); madvise(b, sz, MADV_NOHUGEPAGE);
/* A is faulted in first, so without MADV_COLD it is the older one */
for (size_t i = 0; i < sz; i += 4096) a[i] = 1;
for (size_t i = 0; i < sz; i += 4096) b[i] = 1;
d[0] = 1;
madvise(d, 4096, MADV_COLD); /* drain lru_add */
/* two agings: clear young bits of both A and B, both end at min_seq+1 */
aging();
aging();
madvise(b, sz, MADV_COLD);
madvise(d, 4096, MADV_COLD);
snprintf(buf, sizeof(buf), "%s/memory.reclaim", cg);
fd = open(buf, O_WRONLY);
if (dprintf(fd, "64M swappiness=max\n") < 0)
perror("memory.reclaim");
close(fd);
printf("hot A swapped: %5ld pages cold B swapped: %5ld pages (of %zu)\n",
swapped(a, sz), swapped(b, sz), sz >> 12);
return 0;
}
mm/folio.c | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/mm/folio.c b/mm/folio.c
index ea3adb6a89d7..c412d9b398e4 100644
--- a/mm/folio.c
+++ b/mm/folio.c
@@ -583,7 +583,16 @@ static void lru_deactivate(struct lruvec *lruvec, struct folio *folio)
lruvec_del_folio(lruvec, folio);
folio_clear_active(folio);
folio_clear_referenced(folio);
- lruvec_add_folio(lruvec, folio);
+ /*
+ * For MGLRU, lruvec_add_folio() places anon folios that are not in
+ * the swap cache into the second youngest generation, which is an
+ * active one. Add the folio to the oldest generation instead, like
+ * lru_deactivate_file() does for clean folios.
+ */
+ if (lru_gen_enabled())
+ lruvec_add_folio_tail(lruvec, folio);
+ else
+ lruvec_add_folio(lruvec, folio);
__count_vm_events(PGDEACTIVATE, nr_pages);
count_memcg_events(lruvec_memcg(lruvec), PGDEACTIVATE, nr_pages);
base-commit: 2dc0136b1fdac153434ab02778fcdb32db5d0dae
--
2.34.1
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH] mm/mglru: deactivate anon folios into the oldest generation
2026-11-12 8:19 [PATCH] mm/mglru: deactivate anon folios into the oldest generation Ridong Chen
@ 2026-11-12 8:38 ` Kairui Song
2026-11-12 8:41 ` Barry Song
0 siblings, 1 reply; 8+ messages in thread
From: Kairui Song @ 2026-11-12 8:38 UTC (permalink / raw)
To: Ridong Chen
Cc: David Hildenbrand, Andrew Morton, Liam R. Howlett,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Yu Zhao, Kairui Song, Qi Zheng, Shakeel Butt, Barry Song,
Axel Rasmussen, Yuanchu Xie, Wei Xu, Baoquan He, Baolin Wang,
linux-mm, linux-kernel, Ridong Chen
On Thu, Nov 12, 2026 at 04:19:28PM +0800, Ridong Chen wrote:
> From: Ridong Chen <chenridong@xiaomi.com>
>
> folio_deactivate() is used by MADV_COLD and DAMOS_LRU_DEPRIO to move a
> folio towards eviction. lru_deactivate() does that by deleting the folio
> from its LRU list, clearing PG_active and PG_referenced, and adding it
> back with lruvec_add_folio().
>
> With MGLRU, lru_gen_folio_seq() places a swap-backed folio that is not
> in the swap cache, i.e. an anon or shmem folio, into max_seq - 1,
> regardless of its refs. That generation is active per
> lru_gen_is_active(), so such a folio never reaches an inactive
> generation, and a folio that has aged into min_seq + 1 is moved to a
> younger generation instead. PGDEACTIVATE is still counted. Other folios
> are not affected: with their refs cleared they are placed into the
> oldest generation.
>
> On a 4-generation memcg, 64MB of aged anon memory in min_seq + 1 ends up
> in max_seq - 1 after MADV_COLD, while 64MB of file memory in the same
> generation ends up in min_seq.
>
> As a result, MADV_COLD can make anon memory less likely to be reclaimed.
> It can be reproduced with a single process in a memcg with swap enabled:
>
> 1) map two 64MB anon regions A and B and fault them in, A first;
> 2) age the memcg twice through /sys/kernel/debug/lru_gen, so that both
> regions are in the same generation, min_seq + 1;
> 3) apply MADV_COLD to B only, A is the control;
> 4) reclaim the size of one region from anon only:
> echo "64M swappiness=max" > memory.reclaim;
> 5) count the swapped out pages of A and B from /proc/self/pagemap.
>
> B is expected to be swapped out instead of A. Three runs on mm-new:
>
> A swapped out B (MADV_COLD) swapped out
> before 16384 pages 15 ~ 128 pages
> after 0 ~ 157 pages 16384 pages
>
> Use lruvec_add_folio_tail() for MGLRU, which treats the folio as
> reclaiming and adds it to the tail of the oldest generation, the same
> place lru_deactivate_file() puts a clean file folio. The other folios,
> which were added to the head of the oldest generation, are now added to
> its tail as well, so they are reclaimed first as MADV_COLD intends.
>
> Fixes: 081488051d28 ("mm/mglru: fix underprotected page cache")
> Signed-off-by: Ridong Chen <chenridong@xiaomi.com>
> ---
...
>
> mm/folio.c | 11 ++++++++++-
> 1 file changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/mm/folio.c b/mm/folio.c
> index ea3adb6a89d7..c412d9b398e4 100644
> --- a/mm/folio.c
> +++ b/mm/folio.c
> @@ -583,7 +583,16 @@ static void lru_deactivate(struct lruvec *lruvec, struct folio *folio)
> lruvec_del_folio(lruvec, folio);
> folio_clear_active(folio);
> folio_clear_referenced(folio);
> - lruvec_add_folio(lruvec, folio);
> + /*
> + * For MGLRU, lruvec_add_folio() places anon folios that are not in
> + * the swap cache into the second youngest generation, which is an
> + * active one. Add the folio to the oldest generation instead, like
> + * lru_deactivate_file() does for clean folios.
> + */
> + if (lru_gen_enabled())
> + lruvec_add_folio_tail(lruvec, folio);
> + else
> + lruvec_add_folio(lruvec, folio);
Hi Ridong,
One question here, why MGLRU/CLRU have to use two different ways to
add the folio back? Will this cause over aggressive reclaim of the
folio?
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH] mm/mglru: deactivate anon folios into the oldest generation
2026-11-12 8:38 ` Kairui Song
@ 2026-11-12 8:41 ` Barry Song
2026-11-12 8:46 ` Kairui Song
2026-11-12 8:54 ` Ridong Chen
0 siblings, 2 replies; 8+ messages in thread
From: Barry Song @ 2026-11-12 8:41 UTC (permalink / raw)
To: Kairui Song
Cc: Ridong Chen, David Hildenbrand, Andrew Morton, Liam R. Howlett,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Yu Zhao, Kairui Song, Qi Zheng, Shakeel Butt, Axel Rasmussen,
Yuanchu Xie, Wei Xu, Baoquan He, Baolin Wang, linux-mm,
linux-kernel, Ridong Chen
On Thu, Nov 12, 2026 at 4:38 PM Kairui Song <ryncsn@gmail.com> wrote:
[...]
> > mm/folio.c | 11 ++++++++++-
> > 1 file changed, 10 insertions(+), 1 deletion(-)
> >
> > diff --git a/mm/folio.c b/mm/folio.c
> > index ea3adb6a89d7..c412d9b398e4 100644
> > --- a/mm/folio.c
> > +++ b/mm/folio.c
> > @@ -583,7 +583,16 @@ static void lru_deactivate(struct lruvec *lruvec, struct folio *folio)
> > lruvec_del_folio(lruvec, folio);
> > folio_clear_active(folio);
> > folio_clear_referenced(folio);
> > - lruvec_add_folio(lruvec, folio);
> > + /*
> > + * For MGLRU, lruvec_add_folio() places anon folios that are not in
> > + * the swap cache into the second youngest generation, which is an
> > + * active one. Add the folio to the oldest generation instead, like
> > + * lru_deactivate_file() does for clean folios.
> > + */
> > + if (lru_gen_enabled())
> > + lruvec_add_folio_tail(lruvec, folio);
> > + else
> > + lruvec_add_folio(lruvec, folio);
>
> Hi Ridong,
>
> One question here, why MGLRU/CLRU have to use two different ways to
> add the folio back? Will this cause over aggressive reclaim of the
> folio?
I guess that is because on mglru, that is lacing a way to add the folio to
the head of the oldest generation for anon folios while CLRU can do
that :-)
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] mm/mglru: deactivate anon folios into the oldest generation
2026-11-12 8:41 ` Barry Song
@ 2026-11-12 8:46 ` Kairui Song
2026-11-12 8:50 ` Barry Song
2026-11-12 8:54 ` Ridong Chen
1 sibling, 1 reply; 8+ messages in thread
From: Kairui Song @ 2026-11-12 8:46 UTC (permalink / raw)
To: Barry Song
Cc: Ridong Chen, David Hildenbrand, Andrew Morton, Liam R. Howlett,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Yu Zhao, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
Wei Xu, Baoquan He, Baolin Wang, linux-mm, linux-kernel,
Ridong Chen
On Thu, Nov 12, 2026 at 4:42 PM Barry Song <baohua@kernel.org> wrote:
>
> On Thu, Nov 12, 2026 at 4:38 PM Kairui Song <ryncsn@gmail.com> wrote:
> [...]
> > > mm/folio.c | 11 ++++++++++-
> > > 1 file changed, 10 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/mm/folio.c b/mm/folio.c
> > > index ea3adb6a89d7..c412d9b398e4 100644
> > > --- a/mm/folio.c
> > > +++ b/mm/folio.c
> > > @@ -583,7 +583,16 @@ static void lru_deactivate(struct lruvec *lruvec, struct folio *folio)
> > > lruvec_del_folio(lruvec, folio);
> > > folio_clear_active(folio);
> > > folio_clear_referenced(folio);
> > > - lruvec_add_folio(lruvec, folio);
> > > + /*
> > > + * For MGLRU, lruvec_add_folio() places anon folios that are not in
> > > + * the swap cache into the second youngest generation, which is an
> > > + * active one. Add the folio to the oldest generation instead, like
> > > + * lru_deactivate_file() does for clean folios.
> > > + */
> > > + if (lru_gen_enabled())
> > > + lruvec_add_folio_tail(lruvec, folio);
> > > + else
> > > + lruvec_add_folio(lruvec, folio);
> >
> > Hi Ridong,
> >
> > One question here, why MGLRU/CLRU have to use two different ways to
> > add the folio back? Will this cause over aggressive reclaim of the
> > folio?
>
> I guess that is because on mglru, that is lacing a way to add the folio to
> the head of the oldest generation for anon folios while CLRU can do
> that :-)
Right, but lruvec_add_folio_tail also added the folio to the very tail
of the inactive / oldest list, could it be a gain or problem if we do
that too for CLRU?
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] mm/mglru: deactivate anon folios into the oldest generation
2026-11-12 8:46 ` Kairui Song
@ 2026-11-12 8:50 ` Barry Song
0 siblings, 0 replies; 8+ messages in thread
From: Barry Song @ 2026-11-12 8:50 UTC (permalink / raw)
To: Kairui Song
Cc: Ridong Chen, David Hildenbrand, Andrew Morton, Liam R. Howlett,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Yu Zhao, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
Wei Xu, Baoquan He, Baolin Wang, linux-mm, linux-kernel,
Ridong Chen
On Thu, Nov 12, 2026 at 4:47 PM Kairui Song <ryncsn@gmail.com> wrote:
>
> On Thu, Nov 12, 2026 at 4:42 PM Barry Song <baohua@kernel.org> wrote:
> >
> > On Thu, Nov 12, 2026 at 4:38 PM Kairui Song <ryncsn@gmail.com> wrote:
> > [...]
> > > > mm/folio.c | 11 ++++++++++-
> > > > 1 file changed, 10 insertions(+), 1 deletion(-)
> > > >
> > > > diff --git a/mm/folio.c b/mm/folio.c
> > > > index ea3adb6a89d7..c412d9b398e4 100644
> > > > --- a/mm/folio.c
> > > > +++ b/mm/folio.c
> > > > @@ -583,7 +583,16 @@ static void lru_deactivate(struct lruvec *lruvec, struct folio *folio)
> > > > lruvec_del_folio(lruvec, folio);
> > > > folio_clear_active(folio);
> > > > folio_clear_referenced(folio);
> > > > - lruvec_add_folio(lruvec, folio);
> > > > + /*
> > > > + * For MGLRU, lruvec_add_folio() places anon folios that are not in
> > > > + * the swap cache into the second youngest generation, which is an
> > > > + * active one. Add the folio to the oldest generation instead, like
> > > > + * lru_deactivate_file() does for clean folios.
> > > > + */
> > > > + if (lru_gen_enabled())
> > > > + lruvec_add_folio_tail(lruvec, folio);
> > > > + else
> > > > + lruvec_add_folio(lruvec, folio);
> > >
> > > Hi Ridong,
> > >
> > > One question here, why MGLRU/CLRU have to use two different ways to
> > > add the folio back? Will this cause over aggressive reclaim of the
> > > folio?
> >
> > I guess that is because on mglru, that is lacing a way to add the folio to
> > the head of the oldest generation for anon folios while CLRU can do
> > that :-)
>
> Right, but lruvec_add_folio_tail also added the folio to the very tail
> of the inactive / oldest list, could it be a gain or problem if we do
> that too for CLRU?
My gut feeling is that MADV_COLD says the memory is cold. it seems to
make sense to do it all the same for both mglru and clru - to the tail
of inactive(CLRU) or oldest gen(MGLRU).
not quite sure if userspace is expecting things like this. I feel it should
expect this if it says it is cold?
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] mm/mglru: deactivate anon folios into the oldest generation
2026-11-12 8:41 ` Barry Song
2026-11-12 8:46 ` Kairui Song
@ 2026-11-12 8:54 ` Ridong Chen
2026-11-16 0:28 ` Barry Song
1 sibling, 1 reply; 8+ messages in thread
From: Ridong Chen @ 2026-11-12 8:54 UTC (permalink / raw)
To: Barry Song, Kairui Song
Cc: David Hildenbrand, Andrew Morton, Liam R. Howlett,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Yu Zhao, Kairui Song, Qi Zheng, Shakeel Butt, Axel Rasmussen,
Yuanchu Xie, Wei Xu, Baoquan He, Baolin Wang, linux-mm,
linux-kernel, Ridong Chen
On 11/12/2026 4:41 PM, Barry Song wrote:
> On Thu, Nov 12, 2026 at 4:38 PM Kairui Song <ryncsn@gmail.com> wrote:
> [...]
>>> mm/folio.c | 11 ++++++++++-
>>> 1 file changed, 10 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/mm/folio.c b/mm/folio.c
>>> index ea3adb6a89d7..c412d9b398e4 100644
>>> --- a/mm/folio.c
>>> +++ b/mm/folio.c
>>> @@ -583,7 +583,16 @@ static void lru_deactivate(struct lruvec *lruvec, struct folio *folio)
>>> lruvec_del_folio(lruvec, folio);
>>> folio_clear_active(folio);
>>> folio_clear_referenced(folio);
>>> - lruvec_add_folio(lruvec, folio);
>>> + /*
>>> + * For MGLRU, lruvec_add_folio() places anon folios that are not in
>>> + * the swap cache into the second youngest generation, which is an
>>> + * active one. Add the folio to the oldest generation instead, like
>>> + * lru_deactivate_file() does for clean folios.
>>> + */
>>> + if (lru_gen_enabled())
>>> + lruvec_add_folio_tail(lruvec, folio);
>>> + else
>>> + lruvec_add_folio(lruvec, folio);
>>
>> Hi Ridong,
>>
>> One question here, why MGLRU/CLRU have to use two different ways to
>> add the folio back? Will this cause over aggressive reclaim of the
>> folio?
>
> I guess that is because on mglru, that is lacing a way to add the folio to
> the head of the oldest generation for anon folios while CLRU can do
> that :-)
Hi Kairui, Barry,
Thanks for taking a look.
Barry is right about the reason. With the classic LRU,
lruvec_add_folio() adds the deactivated folio to the head of the
inactive list. With MGLRU, lruvec_add_folio() can't put an anon folio
into the oldest generation at all: lru_gen_folio_seq() places a
swap-backed folio that is not in the swap cache into max_seq - 1,
regardless of its refs. The only way to reach the oldest generation is
the reclaiming path of lruvec_add_folio_tail(), and that path adds the
folio to the tail.
And you are right that this makes it more aggressive than the classic
LRU. With this patch, a deactivated folio is evicted before the folios
already in the oldest generation, while the classic LRU puts it behind
the folios already on the inactive list. It also changes file folios:
before this patch they are added to the head of the oldest generation,
the same as the classic LRU, and now they go to the tail.
So the problem in the current code is that a deactivated anon folio is
promoted into an active generation, and with this patch it goes one
step further than the classic LRU in the other direction.
--
Best regards
Ridong
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] mm/mglru: deactivate anon folios into the oldest generation
2026-11-12 8:54 ` Ridong Chen
@ 2026-11-16 0:28 ` Barry Song
2026-11-16 1:48 ` Ridong Chen
0 siblings, 1 reply; 8+ messages in thread
From: Barry Song @ 2026-11-16 0:28 UTC (permalink / raw)
To: Ridong Chen
Cc: Kairui Song, David Hildenbrand, Andrew Morton, Liam R. Howlett,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Yu Zhao, Kairui Song, Qi Zheng, Shakeel Butt, Axel Rasmussen,
Yuanchu Xie, Wei Xu, Baoquan He, Baolin Wang, linux-mm,
linux-kernel, Ridong Chen
On Thu, Nov 12, 2026 at 4:55 PM Ridong Chen <ridong.chen@linux.dev> wrote:
>
>
>
> On 11/12/2026 4:41 PM, Barry Song wrote:
> > On Thu, Nov 12, 2026 at 4:38 PM Kairui Song <ryncsn@gmail.com> wrote:
> > [...]
> >>> mm/folio.c | 11 ++++++++++-
> >>> 1 file changed, 10 insertions(+), 1 deletion(-)
> >>>
> >>> diff --git a/mm/folio.c b/mm/folio.c
> >>> index ea3adb6a89d7..c412d9b398e4 100644
> >>> --- a/mm/folio.c
> >>> +++ b/mm/folio.c
> >>> @@ -583,7 +583,16 @@ static void lru_deactivate(struct lruvec *lruvec, struct folio *folio)
> >>> lruvec_del_folio(lruvec, folio);
> >>> folio_clear_active(folio);
> >>> folio_clear_referenced(folio);
> >>> - lruvec_add_folio(lruvec, folio);
> >>> + /*
> >>> + * For MGLRU, lruvec_add_folio() places anon folios that are not in
> >>> + * the swap cache into the second youngest generation, which is an
> >>> + * active one. Add the folio to the oldest generation instead, like
> >>> + * lru_deactivate_file() does for clean folios.
> >>> + */
> >>> + if (lru_gen_enabled())
> >>> + lruvec_add_folio_tail(lruvec, folio);
> >>> + else
> >>> + lruvec_add_folio(lruvec, folio);
> >>
> >> Hi Ridong,
> >>
> >> One question here, why MGLRU/CLRU have to use two different ways to
> >> add the folio back? Will this cause over aggressive reclaim of the
> >> folio?
> >
> > I guess that is because on mglru, that is lacing a way to add the folio to
> > the head of the oldest generation for anon folios while CLRU can do
> > that :-)
>
> Hi Kairui, Barry,
>
> Thanks for taking a look.
>
> Barry is right about the reason. With the classic LRU,
> lruvec_add_folio() adds the deactivated folio to the head of the
> inactive list. With MGLRU, lruvec_add_folio() can't put an anon folio
> into the oldest generation at all: lru_gen_folio_seq() places a
> swap-backed folio that is not in the swap cache into max_seq - 1,
> regardless of its refs. The only way to reach the oldest generation is
> the reclaiming path of lruvec_add_folio_tail(), and that path adds the
> folio to the tail.
>
> And you are right that this makes it more aggressive than the classic
> LRU. With this patch, a deactivated folio is evicted before the folios
> already in the oldest generation, while the classic LRU puts it behind
> the folios already on the inactive list. It also changes file folios:
> before this patch they are added to the head of the oldest generation,
> the same as the classic LRU, and now they go to the tail.
>
> So the problem in the current code is that a deactivated anon folio is
> promoted into an active generation, and with this patch it goes one
> step further than the classic LRU in the other direction.
I think MADV_COLD is fundamentally broken for anonymous folios
under MGLRU!
An anonymous folio may already be in an older generation, but
lru_deactivate() can actually promote it to the second-youngest
generation, effectively making it hotter instead of colder!
Right. For anonymous folios, we have no way to insert them at the
head of the oldest generation without introducing a new flag like
PG_cold, which is unlikely to be accepted.
If we instead insert them at the tail of the oldest generation,
I'm not sure whether that would be consistent with the historical
semantics of MADV_COLD.
But I've been thinking about how to insert a folio at the head
of the oldest generation. Why can't we simply move the folio
directly from its current generation to the head of the oldest
generation in lru_gen_deactivate_folio()?
For example, something along these lines:
lru_gen_deactivate_folio()
{
int type = folio_is_file_lru(folio);
int zone = folio_zonenum(folio);
int gen = lru_gen_from_seq(lrugen->min_seq[type]);
if (folio_lru_gen(folio) < 0)
return false;
folio_clear_active(folio);
folio_clear_referenced(folio);
list_move(&folio->lru, &lrugen->folios[gen][type][zone]);
}
Right now, lru_deactivate() seems unnecessarily complicated,
incurring additional CPU overhead and LRU lock contention:
1. Remove the folio from the LRU list.
2. Add the folio to the LRU cache.
3. Drain the LRU cache.
4. Add the folio back to the LRU list.
Why do we need all these intermediate steps instead of moving
the folio directly in a single operation, given that we're
already manipulating the LRU list?
Am I missing something?
Thanks,
Barry
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH] mm/mglru: deactivate anon folios into the oldest generation
2026-11-16 0:28 ` Barry Song
@ 2026-11-16 1:48 ` Ridong Chen
0 siblings, 0 replies; 8+ messages in thread
From: Ridong Chen @ 2026-11-16 1:48 UTC (permalink / raw)
To: Barry Song
Cc: Kairui Song, David Hildenbrand, Andrew Morton, Liam R. Howlett,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Yu Zhao, Kairui Song, Qi Zheng, Shakeel Butt, Axel Rasmussen,
Yuanchu Xie, Wei Xu, Baoquan He, Baolin Wang, linux-mm,
linux-kernel, Ridong Chen
On 11/16/2026 8:28 AM, Barry Song wrote:
> On Thu, Nov 12, 2026 at 4:55 PM Ridong Chen <ridong.chen@linux.dev> wrote:
>>
>>
>>
>> On 11/12/2026 4:41 PM, Barry Song wrote:
>>> On Thu, Nov 12, 2026 at 4:38 PM Kairui Song <ryncsn@gmail.com> wrote:
>>> [...]
>>>>> mm/folio.c | 11 ++++++++++-
>>>>> 1 file changed, 10 insertions(+), 1 deletion(-)
>>>>>
>>>>> diff --git a/mm/folio.c b/mm/folio.c
>>>>> index ea3adb6a89d7..c412d9b398e4 100644
>>>>> --- a/mm/folio.c
>>>>> +++ b/mm/folio.c
>>>>> @@ -583,7 +583,16 @@ static void lru_deactivate(struct lruvec *lruvec, struct folio *folio)
>>>>> lruvec_del_folio(lruvec, folio);
>>>>> folio_clear_active(folio);
>>>>> folio_clear_referenced(folio);
>>>>> - lruvec_add_folio(lruvec, folio);
>>>>> + /*
>>>>> + * For MGLRU, lruvec_add_folio() places anon folios that are not in
>>>>> + * the swap cache into the second youngest generation, which is an
>>>>> + * active one. Add the folio to the oldest generation instead, like
>>>>> + * lru_deactivate_file() does for clean folios.
>>>>> + */
>>>>> + if (lru_gen_enabled())
>>>>> + lruvec_add_folio_tail(lruvec, folio);
>>>>> + else
>>>>> + lruvec_add_folio(lruvec, folio);
>>>>
>>>> Hi Ridong,
>>>>
>>>> One question here, why MGLRU/CLRU have to use two different ways to
>>>> add the folio back? Will this cause over aggressive reclaim of the
>>>> folio?
>>>
>>> I guess that is because on mglru, that is lacing a way to add the folio to
>>> the head of the oldest generation for anon folios while CLRU can do
>>> that :-)
>>
>> Hi Kairui, Barry,
>>
>> Thanks for taking a look.
>>
>> Barry is right about the reason. With the classic LRU,
>> lruvec_add_folio() adds the deactivated folio to the head of the
>> inactive list. With MGLRU, lruvec_add_folio() can't put an anon folio
>> into the oldest generation at all: lru_gen_folio_seq() places a
>> swap-backed folio that is not in the swap cache into max_seq - 1,
>> regardless of its refs. The only way to reach the oldest generation is
>> the reclaiming path of lruvec_add_folio_tail(), and that path adds the
>> folio to the tail.
>>
>> And you are right that this makes it more aggressive than the classic
>> LRU. With this patch, a deactivated folio is evicted before the folios
>> already in the oldest generation, while the classic LRU puts it behind
>> the folios already on the inactive list. It also changes file folios:
>> before this patch they are added to the head of the oldest generation,
>> the same as the classic LRU, and now they go to the tail.
>>
>> So the problem in the current code is that a deactivated anon folio is
>> promoted into an active generation, and with this patch it goes one
>> step further than the classic LRU in the other direction.
>
> I think MADV_COLD is fundamentally broken for anonymous folios
> under MGLRU!
>
> An anonymous folio may already be in an older generation, but
> lru_deactivate() can actually promote it to the second-youngest
> generation, effectively making it hotter instead of colder!
>
> Right. For anonymous folios, we have no way to insert them at the
> head of the oldest generation without introducing a new flag like
> PG_cold, which is unlikely to be accepted.
>
> If we instead insert them at the tail of the oldest generation,
> I'm not sure whether that would be consistent with the historical
> semantics of MADV_COLD.
>
> But I've been thinking about how to insert a folio at the head
> of the oldest generation. Why can't we simply move the folio
> directly from its current generation to the head of the oldest
> generation in lru_gen_deactivate_folio()?
>
> For example, something along these lines:
>
> lru_gen_deactivate_folio()
> {
> int type = folio_is_file_lru(folio);
> int zone = folio_zonenum(folio);
> int gen = lru_gen_from_seq(lrugen->min_seq[type]);
>
> if (folio_lru_gen(folio) < 0)
> return false;
>
> folio_clear_active(folio);
> folio_clear_referenced(folio);
>
> list_move(&folio->lru, &lrugen->folios[gen][type][zone]);
> }
>
I think that works, and it is better than what this patch does. It
puts both anon and file folios at the head of the oldest generation,
which matches the classic LRU, and it avoids the two problems I
mentioned (too aggressive for anon, and moving file folios from the
head to the tail).
A plain list_move() is not enough though. We also need to:
- update the gen bits in folio->flags with a cmpxchg loop, as
folio_inc_gen() does, since folio_update_gen() can change them
locklessly from the page table walk;
- fix up lrugen->nr_pages and the LRU sizes. lru_gen_update_size()
warns on an active -> inactive move ("demotion requires
isolation"), so this would be done as a deletion from old_gen
followed by an addition to new_gen.
min_seq only advances under the LRU lock, so it is stable here.
Something like:
static bool lru_gen_deactivate_folio(struct lruvec *lruvec,
struct folio *folio)
{
int type = folio_is_file_lru(folio);
int zone = folio_zonenum(folio);
struct lru_gen_folio *lrugen = &lruvec->lrugen;
int new_gen = lru_gen_from_seq(lrugen->min_seq[type]);
unsigned long new_flags, old_flags = READ_ONCE(folio->flags.f);
int old_gen;
do {
old_gen = ((old_flags & LRU_GEN_MASK) >> LRU_GEN_PGOFF) - 1;
if (old_gen < 0)
return false;
new_flags = old_flags & ~LRU_GEN_MASK;
new_flags |= (new_gen + 1UL) << LRU_GEN_PGOFF;
} while (!try_cmpxchg(&folio->flags.f, &old_flags, new_flags));
if (old_gen != new_gen) {
lru_gen_update_size(lruvec, folio, old_gen, -1);
lru_gen_update_size(lruvec, folio, -1, new_gen);
}
list_move(&folio->lru, &lrugen->folios[new_gen][type][zone]);
return true;
}
> Right now, lru_deactivate() seems unnecessarily complicated,
> incurring additional CPU overhead and LRU lock contention:
>
> 1. Remove the folio from the LRU list.
> 2. Add the folio to the LRU cache.
> 3. Drain the LRU cache.
> 4. Add the folio back to the LRU list.
>
> Why do we need all these intermediate steps instead of moving
> the folio directly in a single operation, given that we're
> already manipulating the LRU list?
>
> Am I missing something?
>
I don't think the batch itself is the problem. The folio stays on the
LRU while it sits in the per-cpu batch; only when the batch is
drained do we take the LRU lock once for up to a batch of folios and
do del + add under it. So the batch reduces lock contention rather
than adding to it. What the in-place move saves is the del/add pair
inside the lock.
So I plan to keep the batching and, in lru_deactivate(), call
lru_gen_deactivate_folio() instead of lruvec_del_folio() +
lruvec_add_folio() when MGLRU is enabled. I'll test it and send v2
if that sounds good to you.
--
Best regards
Ridong
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2026-11-16 1:48 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-12 8:19 [PATCH] mm/mglru: deactivate anon folios into the oldest generation Ridong Chen
2026-11-12 8:38 ` Kairui Song
2026-11-12 8:41 ` Barry Song
2026-11-12 8:46 ` Kairui Song
2026-11-12 8:50 ` Barry Song
2026-11-12 8:54 ` Ridong Chen
2026-11-16 0:28 ` Barry Song
2026-11-16 1:48 ` Ridong Chen
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®