* [PATCH v2] mm/mglru: fix a no-op lru_gen_clear_refs() call in lru_lazyfree()
@ 2026-10-26 8:40 Hui Zhu
2026-10-26 9:47 ` Barry Song
2026-10-27 11:39 ` Kairui Song
0 siblings, 2 replies; 11+ messages in thread
From: Hui Zhu @ 2026-10-26 8:40 UTC (permalink / raw)
To: Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Yu Zhao, Kairui Song,
Barry Song, linux-mm, linux-kernel
Cc: Hui Zhu
From: Hui Zhu <zhuhui@kylinos.cn>
lru_lazyfree() calls lru_gen_clear_refs() after lruvec_del_folio(), but
lru_gen_del_folio() has already cleared LRU_GEN_MASK, so folio_lru_gen()
returns -1 and lru_gen_clear_refs() exits at its "gen < 0" check without
clearing the refs or PG_workingset.
A MADV_FREE folio therefore keeps the state it had before madvise(2),
even though madvise_free_pte_range() has already cleared the accessed and
dirty bits of its PTEs. lru_gen_folio_seq() places it one generation
younger, and PG_workingset puts it in the last tier, so sort_folio()
promotes and protects it instead of evicting it and charges it to
lrugen->protected[], which read_ctrl_pos() uses to balance anon against
file reclaim. __folio_inc_gen() clears the refs but not PG_workingset,
so this repeats every time the folio reaches the oldest generation. If
the folio is redirtied first, ttu_anon_lazyfree_folio() keeps it and
lru_gen_eviction() packs the stale state into the shadow entry, so a
later refault is promoted to the youngest generation.
Call lru_gen_clear_refs() while the folio is still on the generation
list, as deactivate_file_folio() and folio_deactivate() already do. Its
return value only says whether the folio is in the oldest generation,
which lru_lazyfree() does not need because it re-adds the folio below.
Keep folio_clear_referenced() unconditional: the classic LRU needs it,
and it also covers the folios that lru_gen_clear_refs() skips.
Fixes: cc8ec7be78ff ("mm/mglru: optimize deactivation")
Suggested-by: Barry Song <baohua@kernel.org>
Signed-off-by: Hui Zhu <zhuhui@kylinos.cn>
---
Changelog:
v2:
Call lru_gen_clear_refs() before lruvec_del_folio() instead of
open-coding the flag clears; drop the lru_gen_enabled() branch and
the comment (Barry Song).
mm/folio.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/mm/folio.c b/mm/folio.c
index 35e242b48870..ea3adb6a89d7 100644
--- a/mm/folio.c
+++ b/mm/folio.c
@@ -597,12 +597,10 @@ static void lru_lazyfree(struct lruvec *lruvec, struct folio *folio)
folio_test_swapcache(folio) || folio_test_unevictable(folio))
return;
+ lru_gen_clear_refs(folio);
lruvec_del_folio(lruvec, folio);
folio_clear_active(folio);
- if (lru_gen_enabled())
- lru_gen_clear_refs(folio);
- else
- folio_clear_referenced(folio);
+ folio_clear_referenced(folio);
/*
* Lazyfree folios are clean anonymous folios. They have
* the swapbacked flag cleared, to distinguish them from normal
--
2.53.0
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [PATCH v2] mm/mglru: fix a no-op lru_gen_clear_refs() call in lru_lazyfree() 2026-10-26 8:40 [PATCH v2] mm/mglru: fix a no-op lru_gen_clear_refs() call in lru_lazyfree() Hui Zhu @ 2026-10-26 9:47 ` Barry Song 2026-10-27 7:24 ` Hui Zhu 2026-10-27 11:39 ` Kairui Song 1 sibling, 1 reply; 11+ messages in thread From: Barry Song @ 2026-10-26 9:47 UTC (permalink / raw) To: Hui Zhu Cc: Andrew Morton, David Hildenbrand, Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko, Yu Zhao, Kairui Song, linux-mm, linux-kernel, Hui Zhu On Mon, Oct 26, 2026 at 4:40 PM Hui Zhu <hui.zhu@linux.dev> wrote: > > From: Hui Zhu <zhuhui@kylinos.cn> > > lru_lazyfree() calls lru_gen_clear_refs() after lruvec_del_folio(), but > lru_gen_del_folio() has already cleared LRU_GEN_MASK, so folio_lru_gen() > returns -1 and lru_gen_clear_refs() exits at its "gen < 0" check without > clearing the refs or PG_workingset. > > A MADV_FREE folio therefore keeps the state it had before madvise(2), > even though madvise_free_pte_range() has already cleared the accessed and > dirty bits of its PTEs. lru_gen_folio_seq() places it one generation > younger, and PG_workingset puts it in the last tier, so sort_folio() > promotes and protects it instead of evicting it and charges it to > lrugen->protected[], which read_ctrl_pos() uses to balance anon against > file reclaim. __folio_inc_gen() clears the refs but not PG_workingset, > so this repeats every time the folio reaches the oldest generation. If > the folio is redirtied first, ttu_anon_lazyfree_folio() keeps it and > lru_gen_eviction() packs the stale state into the shadow entry, so a > later refault is promoted to the youngest generation. > > Call lru_gen_clear_refs() while the folio is still on the generation > list, as deactivate_file_folio() and folio_deactivate() already do. Its > return value only says whether the folio is in the oldest generation, > which lru_lazyfree() does not need because it re-adds the folio below. > Keep folio_clear_referenced() unconditional: the classic LRU needs it, > and it also covers the folios that lru_gen_clear_refs() skips. > > Fixes: cc8ec7be78ff ("mm/mglru: optimize deactivation") > Suggested-by: Barry Song <baohua@kernel.org> > Signed-off-by: Hui Zhu <zhuhui@kylinos.cn> > --- > Changelog: > v2: > Call lru_gen_clear_refs() before lruvec_del_folio() instead of > open-coding the flag clears; drop the lru_gen_enabled() branch and > the comment (Barry Song). > > mm/folio.c | 6 ++---- > 1 file changed, 2 insertions(+), 4 deletions(-) > > diff --git a/mm/folio.c b/mm/folio.c > index 35e242b48870..ea3adb6a89d7 100644 > --- a/mm/folio.c > +++ b/mm/folio.c > @@ -597,12 +597,10 @@ static void lru_lazyfree(struct lruvec *lruvec, struct folio *folio) > folio_test_swapcache(folio) || folio_test_unevictable(folio)) > return; > > + lru_gen_clear_refs(folio); The only thing I'm not sure about is whether we need if (lru_gen_enabled()) here to keep the semantics consistent with other places where we use it. ... 2 676 mm/folio.c <<deactivate_file_folio>> if (lru_gen_enabled() && lru_gen_clear_refs(folio)) 3 695 mm/folio.c <<folio_deactivate>> if (lru_gen_enabled() ? lru_gen_clear_refs(folio) : !folio_test_active(folio)) So maybe we still need + if (lru_gen_enabled()) lru_gen_clear_refs(folio); with that, feel free to add: Reviewed-by: Barry Song <baohua@kernel.org> ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] mm/mglru: fix a no-op lru_gen_clear_refs() call in lru_lazyfree() 2026-10-26 9:47 ` Barry Song @ 2026-10-27 7:24 ` Hui Zhu 2026-10-27 11:33 ` Barry Song 0 siblings, 1 reply; 11+ messages in thread From: Hui Zhu @ 2026-10-27 7:24 UTC (permalink / raw) To: Barry Song Cc: Andrew Morton, David Hildenbrand, Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko, Yu Zhao, Kairui Song, linux-mm, linux-kernel, Hui Zhu 2026/10/26 17:47, Barry Song: > On Mon, Oct 26, 2026 at 4:40 PM Hui Zhu <hui.zhu@linux.dev> wrote: >> From: Hui Zhu <zhuhui@kylinos.cn> >> >> lru_lazyfree() calls lru_gen_clear_refs() after lruvec_del_folio(), but >> lru_gen_del_folio() has already cleared LRU_GEN_MASK, so folio_lru_gen() >> returns -1 and lru_gen_clear_refs() exits at its "gen < 0" check without >> clearing the refs or PG_workingset. >> >> A MADV_FREE folio therefore keeps the state it had before madvise(2), >> even though madvise_free_pte_range() has already cleared the accessed and >> dirty bits of its PTEs. lru_gen_folio_seq() places it one generation >> younger, and PG_workingset puts it in the last tier, so sort_folio() >> promotes and protects it instead of evicting it and charges it to >> lrugen->protected[], which read_ctrl_pos() uses to balance anon against >> file reclaim. __folio_inc_gen() clears the refs but not PG_workingset, >> so this repeats every time the folio reaches the oldest generation. If >> the folio is redirtied first, ttu_anon_lazyfree_folio() keeps it and >> lru_gen_eviction() packs the stale state into the shadow entry, so a >> later refault is promoted to the youngest generation. >> >> Call lru_gen_clear_refs() while the folio is still on the generation >> list, as deactivate_file_folio() and folio_deactivate() already do. Its >> return value only says whether the folio is in the oldest generation, >> which lru_lazyfree() does not need because it re-adds the folio below. >> Keep folio_clear_referenced() unconditional: the classic LRU needs it, >> and it also covers the folios that lru_gen_clear_refs() skips. >> >> Fixes: cc8ec7be78ff ("mm/mglru: optimize deactivation") >> Suggested-by: Barry Song <baohua@kernel.org> >> Signed-off-by: Hui Zhu <zhuhui@kylinos.cn> >> --- >> Changelog: >> v2: >> Call lru_gen_clear_refs() before lruvec_del_folio() instead of >> open-coding the flag clears; drop the lru_gen_enabled() branch and >> the comment (Barry Song). >> >> mm/folio.c | 6 ++---- >> 1 file changed, 2 insertions(+), 4 deletions(-) >> >> diff --git a/mm/folio.c b/mm/folio.c >> index 35e242b48870..ea3adb6a89d7 100644 >> --- a/mm/folio.c >> +++ b/mm/folio.c >> @@ -597,12 +597,10 @@ static void lru_lazyfree(struct lruvec *lruvec, struct folio *folio) >> folio_test_swapcache(folio) || folio_test_unevictable(folio)) >> return; >> >> + lru_gen_clear_refs(folio); > The only thing I'm not sure about is whether we need > if (lru_gen_enabled()) here to keep the semantics consistent > with other places where we use it. > > ... > 2 676 mm/folio.c <<deactivate_file_folio>> > if (lru_gen_enabled() && lru_gen_clear_refs(folio)) > 3 695 mm/folio.c <<folio_deactivate>> > if (lru_gen_enabled() ? lru_gen_clear_refs(folio) : > !folio_test_active(folio)) > > So maybe we still need > > + if (lru_gen_enabled()) > lru_gen_clear_refs(folio); I think we maybe don't need it here. The other two sites consume the return value, and that is what the check disambiguates: if (gen < 0) return true; An off-list folio has gen == -1, so without lru_gen_enabled() both deactivate_file_folio() and folio_deactivate() would take that "true" and return early, skipping the deactivation. lru_lazyfree() ignores the return value, since it re-adds the folio a few lines below anyway. The side effects gate themselves, and on something more precise than the global key: a folio on a classic list always has gen == -1, which fill_evictable() even asserts, VM_WARN_ON_ONCE_FOLIO(folio_lru_gen(folio) != -1, folio); and drain_evictable() clears LRU_GEN_MASK on the way out. So an unguarded call either clears the refs of a folio that is on a generation list, or does nothing at all - no flag writes and no lruvec access. The classic LRU cannot be affected, including with CONFIG_LRU_GEN=n, where the stub inlines away. Guarding on the key would reopen a narrower version of this bug. lru_gen_change_state() flips it before migrating any folio: if (enabled) static_branch_enable_cpuslocked(&lru_gen_caps[LRU_GEN_CORE]); else static_branch_disable_cpuslocked(&lru_gen_caps[LRU_GEN_CORE]); memcg = mem_cgroup_iter(NULL, NULL, NULL); so while MGLRU is being turned off the key is already false, but every lruvec that drain_evictable() has not reached yet still has its folios on generation lists. A MADV_FREE batch drained on another CPU in that window would keep PG_workingset. What about add a comment instead of the check: /* no-op unless the folio is on a generation list */ lru_gen_clear_refs(folio); Best, Hui > > with that, feel free to add: > > Reviewed-by: Barry Song <baohua@kernel.org> ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] mm/mglru: fix a no-op lru_gen_clear_refs() call in lru_lazyfree() 2026-10-27 7:24 ` Hui Zhu @ 2026-10-27 11:33 ` Barry Song 0 siblings, 0 replies; 11+ messages in thread From: Barry Song @ 2026-10-27 11:33 UTC (permalink / raw) To: Hui Zhu Cc: Andrew Morton, David Hildenbrand, Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko, Yu Zhao, Kairui Song, linux-mm, linux-kernel, Hui Zhu On Tue, Oct 27, 2026 at 3:25 PM Hui Zhu <hui.zhu@linux.dev> wrote: [...] > >> + lru_gen_clear_refs(folio); > > The only thing I'm not sure about is whether we need > > if (lru_gen_enabled()) here to keep the semantics consistent > > with other places where we use it. > > > > ... > > 2 676 mm/folio.c <<deactivate_file_folio>> > > if (lru_gen_enabled() && lru_gen_clear_refs(folio)) > > 3 695 mm/folio.c <<folio_deactivate>> > > if (lru_gen_enabled() ? lru_gen_clear_refs(folio) : > > !folio_test_active(folio)) > > > > So maybe we still need > > > > + if (lru_gen_enabled()) > > lru_gen_clear_refs(folio); > > I think we maybe don't need it here. The other two sites consume the > return value, and that is what the check disambiguates: > You’re right. Thanks for the thorough investigation. Reviewed-by: Barry Song <baohua@kernel.org> ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] mm/mglru: fix a no-op lru_gen_clear_refs() call in lru_lazyfree() 2026-10-26 8:40 [PATCH v2] mm/mglru: fix a no-op lru_gen_clear_refs() call in lru_lazyfree() Hui Zhu 2026-10-26 9:47 ` Barry Song @ 2026-10-27 11:39 ` Kairui Song 2026-10-31 2:42 ` Barry Song 1 sibling, 1 reply; 11+ messages in thread From: Kairui Song @ 2026-10-27 11:39 UTC (permalink / raw) To: Hui Zhu Cc: Andrew Morton, David Hildenbrand, Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko, Yu Zhao, Barry Song, linux-mm, linux-kernel, Hui Zhu On Mon, Oct 26, 2026 at 4:41 PM Hui Zhu <hui.zhu@linux.dev> wrote: > > From: Hui Zhu <zhuhui@kylinos.cn> > > lru_lazyfree() calls lru_gen_clear_refs() after lruvec_del_folio(), but > lru_gen_del_folio() has already cleared LRU_GEN_MASK, so folio_lru_gen() > returns -1 and lru_gen_clear_refs() exits at its "gen < 0" check without > clearing the refs or PG_workingset. > > A MADV_FREE folio therefore keeps the state it had before madvise(2), > even though madvise_free_pte_range() has already cleared the accessed and > dirty bits of its PTEs. lru_gen_folio_seq() places it one generation > younger, and PG_workingset puts it in the last tier, so sort_folio() > promotes and protects it instead of evicting it and charges it to > lrugen->protected[], which read_ctrl_pos() uses to balance anon against > file reclaim. __folio_inc_gen() clears the refs but not PG_workingset, > so this repeats every time the folio reaches the oldest generation. If > the folio is redirtied first, ttu_anon_lazyfree_folio() keeps it and > lru_gen_eviction() packs the stale state into the shadow entry, so a > later refault is promoted to the youngest generation. > > Call lru_gen_clear_refs() while the folio is still on the generation > list, as deactivate_file_folio() and folio_deactivate() already do. Its > return value only says whether the folio is in the oldest generation, > which lru_lazyfree() does not need because it re-adds the folio below. > Keep folio_clear_referenced() unconditional: the classic LRU needs it, > and it also covers the folios that lru_gen_clear_refs() skips. > > Fixes: cc8ec7be78ff ("mm/mglru: optimize deactivation") > Suggested-by: Barry Song <baohua@kernel.org> > Signed-off-by: Hui Zhu <zhuhui@kylinos.cn> > --- > Changelog: > v2: > Call lru_gen_clear_refs() before lruvec_del_folio() instead of > open-coding the flag clears; drop the lru_gen_enabled() branch and > the comment (Barry Song). > > mm/folio.c | 6 ++---- > 1 file changed, 2 insertions(+), 4 deletions(-) > > diff --git a/mm/folio.c b/mm/folio.c > index 35e242b48870..ea3adb6a89d7 100644 > --- a/mm/folio.c > +++ b/mm/folio.c > @@ -597,12 +597,10 @@ static void lru_lazyfree(struct lruvec *lruvec, struct folio *folio) > folio_test_swapcache(folio) || folio_test_unevictable(folio)) > return; > > + lru_gen_clear_refs(folio); > lruvec_del_folio(lruvec, folio); > folio_clear_active(folio); > - if (lru_gen_enabled()) > - lru_gen_clear_refs(folio); > - else > - folio_clear_referenced(folio); > + folio_clear_referenced(folio); Why can't we just call folio_set_lru_refs(.., 0), and should we clear PG_workingset? Keeping it helps PSI on refualt / redirty tracking (not sure if meaningful but CLRU does this), clearing it makes eviction better, but the eviction issue is MGLRU-only, CLRU keeps it. And folio_set_lru_refs(.., 0) should be a little bit faster and unify CLRU / MGLRU. And we can detect lazy freed folios in sort_folios to avoid PID form touching them? Something like: diff --git a/mm/folio.c b/mm/folio.c index 35e242b48870..416cd4beb182 100644 --- a/mm/folio.c +++ b/mm/folio.c @@ -599,10 +599,7 @@ static void lru_lazyfree(struct lruvec *lruvec, struct folio *folio) lruvec_del_folio(lruvec, folio); folio_clear_active(folio); - if (lru_gen_enabled()) - lru_gen_clear_refs(folio); - else - folio_clear_referenced(folio); + folio_set_lru_refs(folio, 0); /* * Lazyfree folios are clean anonymous folios. They have * the swapbacked flag cleared, to distinguish them from normal diff --git a/mm/vmscan.c b/mm/vmscan.c index 6a1af9b92349..e64b7f544d15 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -4835,6 +4835,10 @@ static bool sort_folio(struct lruvec *lruvec, struct folio *folio, struct scan_c if (!folio_evictable(folio)) return false; + /* No need to protect lazyfreed folios */ + if (folio_test_lazyfree(folio) && !folio_test_dirty(folio)) + return false; + /* promoted */ if (gen != lru_gen_from_seq(lrugen->min_seq[type])) { list_move(&folio->lru, &lrugen->folios[gen][type][zone]); --- BTW I also tried to cover this in MGLRU-FG series (not including the sort_folio part): https://lore.kernel.org/linux-mm/20261016-mglru-fg-v4-13-2106750c0bfa@tencent.com/ I tested keeping min(refs, LRU_REFS_WORKINGSET) / just set to 0 as in v4, nothing different from the workload (testing using Android apps as workload), so I think it's fine either way but the above patch I posted might help to impove it by a little? And if CLRU didn't actually expect PG_workingset to be kept but that's just a leftover or trivial, maybe we can always do both folio_clear_workingset and folio_set_lru_refs(0) here? ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] mm/mglru: fix a no-op lru_gen_clear_refs() call in lru_lazyfree() 2026-10-27 11:39 ` Kairui Song @ 2026-10-31 2:42 ` Barry Song 2026-10-31 7:25 ` Kairui Song 0 siblings, 1 reply; 11+ messages in thread From: Barry Song @ 2026-10-31 2:42 UTC (permalink / raw) To: Kairui Song Cc: Hui Zhu, Andrew Morton, David Hildenbrand, Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko, Yu Zhao, linux-mm, linux-kernel, Hui Zhu On Tue, Oct 27, 2026 at 7:40 PM Kairui Song <ryncsn@gmail.com> wrote: > > On Mon, Oct 26, 2026 at 4:41 PM Hui Zhu <hui.zhu@linux.dev> wrote: > > > > From: Hui Zhu <zhuhui@kylinos.cn> > > > > lru_lazyfree() calls lru_gen_clear_refs() after lruvec_del_folio(), but > > lru_gen_del_folio() has already cleared LRU_GEN_MASK, so folio_lru_gen() > > returns -1 and lru_gen_clear_refs() exits at its "gen < 0" check without > > clearing the refs or PG_workingset. > > > > A MADV_FREE folio therefore keeps the state it had before madvise(2), > > even though madvise_free_pte_range() has already cleared the accessed and > > dirty bits of its PTEs. lru_gen_folio_seq() places it one generation > > younger, and PG_workingset puts it in the last tier, so sort_folio() > > promotes and protects it instead of evicting it and charges it to > > lrugen->protected[], which read_ctrl_pos() uses to balance anon against > > file reclaim. __folio_inc_gen() clears the refs but not PG_workingset, > > so this repeats every time the folio reaches the oldest generation. If > > the folio is redirtied first, ttu_anon_lazyfree_folio() keeps it and > > lru_gen_eviction() packs the stale state into the shadow entry, so a > > later refault is promoted to the youngest generation. > > > > Call lru_gen_clear_refs() while the folio is still on the generation > > list, as deactivate_file_folio() and folio_deactivate() already do. Its > > return value only says whether the folio is in the oldest generation, > > which lru_lazyfree() does not need because it re-adds the folio below. > > Keep folio_clear_referenced() unconditional: the classic LRU needs it, > > and it also covers the folios that lru_gen_clear_refs() skips. > > > > Fixes: cc8ec7be78ff ("mm/mglru: optimize deactivation") > > Suggested-by: Barry Song <baohua@kernel.org> > > Signed-off-by: Hui Zhu <zhuhui@kylinos.cn> > > --- > > Changelog: > > v2: > > Call lru_gen_clear_refs() before lruvec_del_folio() instead of > > open-coding the flag clears; drop the lru_gen_enabled() branch and > > the comment (Barry Song). > > > > mm/folio.c | 6 ++---- > > 1 file changed, 2 insertions(+), 4 deletions(-) > > > > diff --git a/mm/folio.c b/mm/folio.c > > index 35e242b48870..ea3adb6a89d7 100644 > > --- a/mm/folio.c > > +++ b/mm/folio.c > > @@ -597,12 +597,10 @@ static void lru_lazyfree(struct lruvec *lruvec, struct folio *folio) > > folio_test_swapcache(folio) || folio_test_unevictable(folio)) > > return; > > > > + lru_gen_clear_refs(folio); > > lruvec_del_folio(lruvec, folio); > > folio_clear_active(folio); > > - if (lru_gen_enabled()) > > - lru_gen_clear_refs(folio); > > - else > > - folio_clear_referenced(folio); > > + folio_clear_referenced(folio); > > Why can't we just call folio_set_lru_refs(.., 0), and should we clear > PG_workingset? Keeping it helps PSI on refualt / redirty tracking (not > sure if meaningful but CLRU does this), clearing it makes eviction > better, but the eviction issue is MGLRU-only, CLRU keeps it. I recall that in v1 [1], Hui mentioned: "Use folio_clear_referenced() rather than folio_set_lru_refs(), which is not defined without CONFIG_LRU_GEN." I assume we can support this, but doesn't the code below also look a bit weird? diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h index 829977d399e6..66ed805a48f5 100644 --- a/include/linux/mm_inline.h +++ b/include/linux/mm_inline.h @@ -431,6 +431,15 @@ static inline void folio_migrate_lru_refs(struct folio *new, const struct folio if (folio_test_referenced(old)) folio_set_referenced(new); } + +static inline void folio_set_lru_refs(struct folio *folio, unsigned int refs) +{ + if (!refs) + folio_clear_referenced(folio); + else if (refs == 1) + folio_set_referenced(folio); +} + #endif /* CONFIG_LRU_GEN */ [1] https://lore.kernel.org/linux-mm/20261023080430.465305-1-hui.zhu@linux.dev/ Best, Barry ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] mm/mglru: fix a no-op lru_gen_clear_refs() call in lru_lazyfree() 2026-10-31 2:42 ` Barry Song @ 2026-10-31 7:25 ` Kairui Song 2026-10-31 7:26 ` Kairui Song 0 siblings, 1 reply; 11+ messages in thread From: Kairui Song @ 2026-10-31 7:25 UTC (permalink / raw) To: Barry Song Cc: Hui Zhu, Andrew Morton, David Hildenbrand, Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko, Yu Zhao, linux-mm, linux-kernel, Hui Zhu On Sat, Oct 31, 2026 at 10:42 AM Barry Song <baohua@kernel.org> wrote: > > On Tue, Oct 27, 2026 at 7:40 PM Kairui Song <ryncsn@gmail.com> wrote: > > > > On Mon, Oct 26, 2026 at 4:41 PM Hui Zhu <hui.zhu@linux.dev> wrote: > > > > > > From: Hui Zhu <zhuhui@kylinos.cn> > > > > > > lru_lazyfree() calls lru_gen_clear_refs() after lruvec_del_folio(), but > > > lru_gen_del_folio() has already cleared LRU_GEN_MASK, so folio_lru_gen() > > > returns -1 and lru_gen_clear_refs() exits at its "gen < 0" check without > > > clearing the refs or PG_workingset. > > > > > > A MADV_FREE folio therefore keeps the state it had before madvise(2), > > > even though madvise_free_pte_range() has already cleared the accessed and > > > dirty bits of its PTEs. lru_gen_folio_seq() places it one generation > > > younger, and PG_workingset puts it in the last tier, so sort_folio() > > > promotes and protects it instead of evicting it and charges it to > > > lrugen->protected[], which read_ctrl_pos() uses to balance anon against > > > file reclaim. __folio_inc_gen() clears the refs but not PG_workingset, > > > so this repeats every time the folio reaches the oldest generation. If > > > the folio is redirtied first, ttu_anon_lazyfree_folio() keeps it and > > > lru_gen_eviction() packs the stale state into the shadow entry, so a > > > later refault is promoted to the youngest generation. > > > > > > Call lru_gen_clear_refs() while the folio is still on the generation > > > list, as deactivate_file_folio() and folio_deactivate() already do. Its > > > return value only says whether the folio is in the oldest generation, > > > which lru_lazyfree() does not need because it re-adds the folio below. > > > Keep folio_clear_referenced() unconditional: the classic LRU needs it, > > > and it also covers the folios that lru_gen_clear_refs() skips. > > > > > > Fixes: cc8ec7be78ff ("mm/mglru: optimize deactivation") > > > Suggested-by: Barry Song <baohua@kernel.org> > > > Signed-off-by: Hui Zhu <zhuhui@kylinos.cn> > > > --- > > > Changelog: > > > v2: > > > Call lru_gen_clear_refs() before lruvec_del_folio() instead of > > > open-coding the flag clears; drop the lru_gen_enabled() branch and > > > the comment (Barry Song). > > > > > > mm/folio.c | 6 ++---- > > > 1 file changed, 2 insertions(+), 4 deletions(-) > > > > > > diff --git a/mm/folio.c b/mm/folio.c > > > index 35e242b48870..ea3adb6a89d7 100644 > > > --- a/mm/folio.c > > > +++ b/mm/folio.c > > > @@ -597,12 +597,10 @@ static void lru_lazyfree(struct lruvec *lruvec, struct folio *folio) > > > folio_test_swapcache(folio) || folio_test_unevictable(folio)) > > > return; > > > > > > + lru_gen_clear_refs(folio); > > > lruvec_del_folio(lruvec, folio); > > > folio_clear_active(folio); > > > - if (lru_gen_enabled()) > > > - lru_gen_clear_refs(folio); > > > - else > > > - folio_clear_referenced(folio); > > > + folio_clear_referenced(folio); > > > > Why can't we just call folio_set_lru_refs(.., 0), and should we clear > > PG_workingset? Keeping it helps PSI on refualt / redirty tracking (not > > sure if meaningful but CLRU does this), clearing it makes eviction > > better, but the eviction issue is MGLRU-only, CLRU keeps it. > > I recall that in v1 [1], Hui mentioned: > > "Use folio_clear_referenced() rather than > folio_set_lru_refs(), which is not defined without > CONFIG_LRU_GEN." > Or prehaps we can have a folio_clear_lru_refs() do in a unified way for For both CLRU / MGLRU? (Just like folio_migrate_lru_refs). PG_workingseet gets cleaned too, but if we thing carefully, it might be actually the right move, even for CLRU. Stepping back: what PG_referenced/PG_workingset encode is "this data has been accessed repeatedly, so protect it". MADV_FREE is the caller saying those bytes are garbage, the allocation that produced those accesses is gone, and whoever gets the memory next, could be a different owner with a different access pattern. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] mm/mglru: fix a no-op lru_gen_clear_refs() call in lru_lazyfree() 2026-10-31 7:25 ` Kairui Song @ 2026-10-31 7:26 ` Kairui Song 2026-11-05 21:13 ` Barry Song 0 siblings, 1 reply; 11+ messages in thread From: Kairui Song @ 2026-10-31 7:26 UTC (permalink / raw) To: Barry Song Cc: Hui Zhu, Andrew Morton, David Hildenbrand, Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko, Yu Zhao, linux-mm, linux-kernel, Hui Zhu On Sat, Oct 31, 2026 at 3:25 PM Kairui Song <ryncsn@gmail.com> wrote: > > On Sat, Oct 31, 2026 at 10:42 AM Barry Song <baohua@kernel.org> wrote: > > > > On Tue, Oct 27, 2026 at 7:40 PM Kairui Song <ryncsn@gmail.com> wrote: > > > > > > On Mon, Oct 26, 2026 at 4:41 PM Hui Zhu <hui.zhu@linux.dev> wrote: > > > > > > > > From: Hui Zhu <zhuhui@kylinos.cn> > > > > > > > > lru_lazyfree() calls lru_gen_clear_refs() after lruvec_del_folio(), but > > > > lru_gen_del_folio() has already cleared LRU_GEN_MASK, so folio_lru_gen() > > > > returns -1 and lru_gen_clear_refs() exits at its "gen < 0" check without > > > > clearing the refs or PG_workingset. > > > > > > > > A MADV_FREE folio therefore keeps the state it had before madvise(2), > > > > even though madvise_free_pte_range() has already cleared the accessed and > > > > dirty bits of its PTEs. lru_gen_folio_seq() places it one generation > > > > younger, and PG_workingset puts it in the last tier, so sort_folio() > > > > promotes and protects it instead of evicting it and charges it to > > > > lrugen->protected[], which read_ctrl_pos() uses to balance anon against > > > > file reclaim. __folio_inc_gen() clears the refs but not PG_workingset, > > > > so this repeats every time the folio reaches the oldest generation. If > > > > the folio is redirtied first, ttu_anon_lazyfree_folio() keeps it and > > > > lru_gen_eviction() packs the stale state into the shadow entry, so a > > > > later refault is promoted to the youngest generation. > > > > > > > > Call lru_gen_clear_refs() while the folio is still on the generation > > > > list, as deactivate_file_folio() and folio_deactivate() already do. Its > > > > return value only says whether the folio is in the oldest generation, > > > > which lru_lazyfree() does not need because it re-adds the folio below. > > > > Keep folio_clear_referenced() unconditional: the classic LRU needs it, > > > > and it also covers the folios that lru_gen_clear_refs() skips. > > > > > > > > Fixes: cc8ec7be78ff ("mm/mglru: optimize deactivation") > > > > Suggested-by: Barry Song <baohua@kernel.org> > > > > Signed-off-by: Hui Zhu <zhuhui@kylinos.cn> > > > > --- > > > > Changelog: > > > > v2: > > > > Call lru_gen_clear_refs() before lruvec_del_folio() instead of > > > > open-coding the flag clears; drop the lru_gen_enabled() branch and > > > > the comment (Barry Song). > > > > > > > > mm/folio.c | 6 ++---- > > > > 1 file changed, 2 insertions(+), 4 deletions(-) > > > > > > > > diff --git a/mm/folio.c b/mm/folio.c > > > > index 35e242b48870..ea3adb6a89d7 100644 > > > > --- a/mm/folio.c > > > > +++ b/mm/folio.c > > > > @@ -597,12 +597,10 @@ static void lru_lazyfree(struct lruvec *lruvec, struct folio *folio) > > > > folio_test_swapcache(folio) || folio_test_unevictable(folio)) > > > > return; > > > > > > > > + lru_gen_clear_refs(folio); > > > > lruvec_del_folio(lruvec, folio); > > > > folio_clear_active(folio); > > > > - if (lru_gen_enabled()) > > > > - lru_gen_clear_refs(folio); > > > > - else > > > > - folio_clear_referenced(folio); > > > > + folio_clear_referenced(folio); > > > > > > Why can't we just call folio_set_lru_refs(.., 0), and should we clear > > > PG_workingset? Keeping it helps PSI on refualt / redirty tracking (not > > > sure if meaningful but CLRU does this), clearing it makes eviction > > > better, but the eviction issue is MGLRU-only, CLRU keeps it. > > > > I recall that in v1 [1], Hui mentioned: > > > > "Use folio_clear_referenced() rather than > > folio_set_lru_refs(), which is not defined without > > CONFIG_LRU_GEN." > > > > Or prehaps we can have a folio_clear_lru_refs() do in a unified way > for For both CLRU / MGLRU? (Just like folio_migrate_lru_refs). And it will be better if we can keep this operation off-list just like right now :), some later works could benefit from it. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] mm/mglru: fix a no-op lru_gen_clear_refs() call in lru_lazyfree() 2026-10-31 7:26 ` Kairui Song @ 2026-11-05 21:13 ` Barry Song 2026-11-06 8:38 ` Hui Zhu 0 siblings, 1 reply; 11+ messages in thread From: Barry Song @ 2026-11-05 21:13 UTC (permalink / raw) To: Kairui Song Cc: Hui Zhu, Andrew Morton, David Hildenbrand, Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko, Yu Zhao, linux-mm, linux-kernel, Hui Zhu On Sat, Oct 31, 2026 at 3:27 PM Kairui Song <ryncsn@gmail.com> wrote: [...] > > > > > > > > > > mm/folio.c | 6 ++---- > > > > > 1 file changed, 2 insertions(+), 4 deletions(-) > > > > > > > > > > diff --git a/mm/folio.c b/mm/folio.c > > > > > index 35e242b48870..ea3adb6a89d7 100644 > > > > > --- a/mm/folio.c > > > > > +++ b/mm/folio.c > > > > > @@ -597,12 +597,10 @@ static void lru_lazyfree(struct lruvec *lruvec, struct folio *folio) > > > > > folio_test_swapcache(folio) || folio_test_unevictable(folio)) > > > > > return; > > > > > > > > > > + lru_gen_clear_refs(folio); > > > > > lruvec_del_folio(lruvec, folio); > > > > > folio_clear_active(folio); > > > > > - if (lru_gen_enabled()) > > > > > - lru_gen_clear_refs(folio); > > > > > - else > > > > > - folio_clear_referenced(folio); > > > > > + folio_clear_referenced(folio); > > > > > > > > Why can't we just call folio_set_lru_refs(.., 0), and should we clear > > > > PG_workingset? Keeping it helps PSI on refualt / redirty tracking (not > > > > sure if meaningful but CLRU does this), clearing it makes eviction > > > > better, but the eviction issue is MGLRU-only, CLRU keeps it. > > > > > > I recall that in v1 [1], Hui mentioned: > > > > > > "Use folio_clear_referenced() rather than > > > folio_set_lru_refs(), which is not defined without > > > CONFIG_LRU_GEN." > > > > > > > Or prehaps we can have a folio_clear_lru_refs() do in a unified way > > for For both CLRU / MGLRU? (Just like folio_migrate_lru_refs). > > And it will be better if we can keep this operation off-list just like > right now :), some later works could benefit from it. lru_gen_clear_refs() is currently multifunctional. It clears refs/workingset, but also returns true to skip folio_batch_add_and_move() in both deactivate_file_folio() and folio_deactivate(). This actually looks a bit weird. However, following the current logic, I've drafted a version that we can use as a starting point for discussion: diff --git a/mm/folio.c b/mm/folio.c index dc2c796a4ea7..bc17cd3b6b6c 100644 --- a/mm/folio.c +++ b/mm/folio.c @@ -444,12 +444,12 @@ static bool lru_gen_clear_refs(struct folio *folio) int type = folio_is_file_lru(folio); unsigned long seq; - if (gen < 0) - return true; - folio_set_lru_refs(folio, 0); folio_clear_workingset(folio); + if (gen < 0) + return true; + rcu_read_lock(); seq = READ_ONCE(folio_lruvec(folio)->lrugen.min_seq[type]); rcu_read_unlock(); @@ -465,6 +465,8 @@ static void lru_gen_inc_refs(struct folio *folio) static bool lru_gen_clear_refs(struct folio *folio) { + folio_clear_referenced(folio); + folio_clear_workingset(folio); return false; } The another option is: diff --git a/mm/folio.c b/mm/folio.c index dc2c796a4ea7..8d1fd788188c 100644 --- a/mm/folio.c +++ b/mm/folio.c @@ -438,6 +438,12 @@ static void lru_gen_inc_refs(struct folio *folio) } while (!try_cmpxchg(folio_flags(folio, 0), &old_flags, new_flags)); } +static inline void __lru_gen_clear_refs(struct folio *folio) +{ + folio_set_lru_refs(folio, 0); + folio_clear_workingset(folio); +} + static bool lru_gen_clear_refs(struct folio *folio) { int gen = folio_lru_gen(folio); @@ -447,8 +453,7 @@ static bool lru_gen_clear_refs(struct folio *folio) if (gen < 0) return true; - folio_set_lru_refs(folio, 0); - folio_clear_workingset(folio); + __lru_gen_clear_refs(folio); rcu_read_lock(); seq = READ_ONCE(folio_lruvec(folio)->lrugen.min_seq[type]); @@ -463,9 +468,10 @@ static void lru_gen_inc_refs(struct folio *folio) { } -static bool lru_gen_clear_refs(struct folio *folio) +static inline void __lru_gen_clear_refs(struct folio *folio) { - return false; + folio_clear_referenced(folio); + folio_clear_workingset(folio); } #endif /* CONFIG_LRU_GEN */ @@ -659,10 +665,7 @@ static void lru_lazyfree(struct lruvec *lruvec, struct folio *folio) lruvec_del_folio(lruvec, folio); folio_clear_active(folio); - if (lru_gen_enabled()) - lru_gen_clear_refs(folio); - else - folio_clear_referenced(folio); + __lru_gen_clear_refs(folio); /* * Lazyfree folios are clean anonymous folios. They have * the swapbacked flag cleared, to distinguish them from normal I would personally prefer the latter. What do you think, Hui and Kairui? Best, Barry ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] mm/mglru: fix a no-op lru_gen_clear_refs() call in lru_lazyfree() 2026-11-05 21:13 ` Barry Song @ 2026-11-06 8:38 ` Hui Zhu 2026-11-06 9:29 ` Barry Song 0 siblings, 1 reply; 11+ messages in thread From: Hui Zhu @ 2026-11-06 8:38 UTC (permalink / raw) To: Barry Song, Kairui Song Cc: Andrew Morton, David Hildenbrand, Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko, Yu Zhao, linux-mm, linux-kernel, Hui Zhu 2026/11/6 05:13, Barry Song: > On Sat, Oct 31, 2026 at 3:27 PM Kairui Song <ryncsn@gmail.com> wrote: > [...] >>>>>> mm/folio.c | 6 ++---- >>>>>> 1 file changed, 2 insertions(+), 4 deletions(-) >>>>>> >>>>>> diff --git a/mm/folio.c b/mm/folio.c >>>>>> index 35e242b48870..ea3adb6a89d7 100644 >>>>>> --- a/mm/folio.c >>>>>> +++ b/mm/folio.c >>>>>> @@ -597,12 +597,10 @@ static void lru_lazyfree(struct lruvec *lruvec, struct folio *folio) >>>>>> folio_test_swapcache(folio) || folio_test_unevictable(folio)) >>>>>> return; >>>>>> >>>>>> + lru_gen_clear_refs(folio); >>>>>> lruvec_del_folio(lruvec, folio); >>>>>> folio_clear_active(folio); >>>>>> - if (lru_gen_enabled()) >>>>>> - lru_gen_clear_refs(folio); >>>>>> - else >>>>>> - folio_clear_referenced(folio); >>>>>> + folio_clear_referenced(folio); >>>>> Why can't we just call folio_set_lru_refs(.., 0), and should we clear >>>>> PG_workingset? Keeping it helps PSI on refualt / redirty tracking (not >>>>> sure if meaningful but CLRU does this), clearing it makes eviction >>>>> better, but the eviction issue is MGLRU-only, CLRU keeps it. >>>> I recall that in v1 [1], Hui mentioned: >>>> >>>> "Use folio_clear_referenced() rather than >>>> folio_set_lru_refs(), which is not defined without >>>> CONFIG_LRU_GEN." >>>> >>> Or prehaps we can have a folio_clear_lru_refs() do in a unified way >>> for For both CLRU / MGLRU? (Just like folio_migrate_lru_refs). >> And it will be better if we can keep this operation off-list just like >> right now :), some later works could benefit from it. > lru_gen_clear_refs() is currently multifunctional. It clears > refs/workingset, but also returns true to skip > folio_batch_add_and_move() in both deactivate_file_folio() > and folio_deactivate(). > > This actually looks a bit weird. However, following the current > logic, I've drafted a version that we can use as a starting point > for discussion: > > diff --git a/mm/folio.c b/mm/folio.c > index dc2c796a4ea7..bc17cd3b6b6c 100644 > --- a/mm/folio.c > +++ b/mm/folio.c > @@ -444,12 +444,12 @@ static bool lru_gen_clear_refs(struct folio *folio) > int type = folio_is_file_lru(folio); > unsigned long seq; > > - if (gen < 0) > - return true; > - > folio_set_lru_refs(folio, 0); > folio_clear_workingset(folio); > > + if (gen < 0) > + return true; > + > rcu_read_lock(); > seq = READ_ONCE(folio_lruvec(folio)->lrugen.min_seq[type]); > rcu_read_unlock(); > @@ -465,6 +465,8 @@ static void lru_gen_inc_refs(struct folio *folio) > > static bool lru_gen_clear_refs(struct folio *folio) > { > + folio_clear_referenced(folio); > + folio_clear_workingset(folio); > return false; > } > > The another option is: > > diff --git a/mm/folio.c b/mm/folio.c > index dc2c796a4ea7..8d1fd788188c 100644 > --- a/mm/folio.c > +++ b/mm/folio.c > @@ -438,6 +438,12 @@ static void lru_gen_inc_refs(struct folio *folio) > } while (!try_cmpxchg(folio_flags(folio, 0), &old_flags, new_flags)); > } > > +static inline void __lru_gen_clear_refs(struct folio *folio) > +{ > + folio_set_lru_refs(folio, 0); > + folio_clear_workingset(folio); > +} > + > static bool lru_gen_clear_refs(struct folio *folio) > { > int gen = folio_lru_gen(folio); > @@ -447,8 +453,7 @@ static bool lru_gen_clear_refs(struct folio *folio) > if (gen < 0) > return true; > > - folio_set_lru_refs(folio, 0); > - folio_clear_workingset(folio); > + __lru_gen_clear_refs(folio); > > rcu_read_lock(); > seq = READ_ONCE(folio_lruvec(folio)->lrugen.min_seq[type]); > @@ -463,9 +468,10 @@ static void lru_gen_inc_refs(struct folio *folio) > { > } > > -static bool lru_gen_clear_refs(struct folio *folio) > +static inline void __lru_gen_clear_refs(struct folio *folio) > { > - return false; > + folio_clear_referenced(folio); > + folio_clear_workingset(folio); > } > > #endif /* CONFIG_LRU_GEN */ > @@ -659,10 +665,7 @@ static void lru_lazyfree(struct lruvec *lruvec, > struct folio *folio) > > lruvec_del_folio(lruvec, folio); > folio_clear_active(folio); > - if (lru_gen_enabled()) > - lru_gen_clear_refs(folio); > - else > - folio_clear_referenced(folio); > + __lru_gen_clear_refs(folio); > /* > * Lazyfree folios are clean anonymous folios. They have > * the swapbacked flag cleared, to distinguish them from normal Hi Barry, Kairui, > > I would personally prefer the latter. What do you think, Hui and Kairui? I prefer option B too: it keeps the flag clearing independent of the list state and leaves the two deactivate callers untouched. But I'd suggest splitting this in two: 1. Take v2 as the fix now. If that is acceptable, I will respin it with Cc: stable added. 2. Follow up with option B as a pure cleanup, no Fixes tag. The lazyfree site then becomes the off-list __lru_gen_clear_refs() call that Kairui wants for later work. The reason is backporting. Every stable from 6.14 (cc8ec7be78ff) to 7.3 carries the regression, but B is written against folio_set_lru_refs(), which only landed in this merge window (64169ceea5b1e). None of the affected trees has that helper, so backporting B would drag the refs-flags rework along with it. v2 applies to all of them with only the mm/swap.c -> mm/folio.c rename to adjust. B also starts clearing PG_workingset on the classic LRU. I agree with Kairui that this is the right semantics for MADV_FREE, but it is a change beyond this regression and deserves its own patch - it fits naturally in the MGLRU-FG flag cleanup, as does the sort_folio() lazyfree check. Best, Hui > Best, > Barry ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] mm/mglru: fix a no-op lru_gen_clear_refs() call in lru_lazyfree() 2026-11-06 8:38 ` Hui Zhu @ 2026-11-06 9:29 ` Barry Song 0 siblings, 0 replies; 11+ messages in thread From: Barry Song @ 2026-11-06 9:29 UTC (permalink / raw) To: Hui Zhu Cc: Kairui Song, Andrew Morton, David Hildenbrand, Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko, Yu Zhao, linux-mm, linux-kernel, Hui Zhu On Fri, Nov 6, 2026 at 4:38 PM Hui Zhu <hui.zhu@linux.dev> wrote: > > > 2026/11/6 05:13, Barry Song: > > On Sat, Oct 31, 2026 at 3:27 PM Kairui Song <ryncsn@gmail.com> wrote: > > [...] > >>>>>> mm/folio.c | 6 ++---- > >>>>>> 1 file changed, 2 insertions(+), 4 deletions(-) > >>>>>> > >>>>>> diff --git a/mm/folio.c b/mm/folio.c > >>>>>> index 35e242b48870..ea3adb6a89d7 100644 > >>>>>> --- a/mm/folio.c > >>>>>> +++ b/mm/folio.c > >>>>>> @@ -597,12 +597,10 @@ static void lru_lazyfree(struct lruvec *lruvec, struct folio *folio) > >>>>>> folio_test_swapcache(folio) || folio_test_unevictable(folio)) > >>>>>> return; > >>>>>> > >>>>>> + lru_gen_clear_refs(folio); > >>>>>> lruvec_del_folio(lruvec, folio); > >>>>>> folio_clear_active(folio); > >>>>>> - if (lru_gen_enabled()) > >>>>>> - lru_gen_clear_refs(folio); > >>>>>> - else > >>>>>> - folio_clear_referenced(folio); > >>>>>> + folio_clear_referenced(folio); > >>>>> Why can't we just call folio_set_lru_refs(.., 0), and should we clear > >>>>> PG_workingset? Keeping it helps PSI on refualt / redirty tracking (not > >>>>> sure if meaningful but CLRU does this), clearing it makes eviction > >>>>> better, but the eviction issue is MGLRU-only, CLRU keeps it. > >>>> I recall that in v1 [1], Hui mentioned: > >>>> > >>>> "Use folio_clear_referenced() rather than > >>>> folio_set_lru_refs(), which is not defined without > >>>> CONFIG_LRU_GEN." > >>>> > >>> Or prehaps we can have a folio_clear_lru_refs() do in a unified way > >>> for For both CLRU / MGLRU? (Just like folio_migrate_lru_refs). > >> And it will be better if we can keep this operation off-list just like > >> right now :), some later works could benefit from it. > > lru_gen_clear_refs() is currently multifunctional. It clears > > refs/workingset, but also returns true to skip > > folio_batch_add_and_move() in both deactivate_file_folio() > > and folio_deactivate(). > > > > This actually looks a bit weird. However, following the current > > logic, I've drafted a version that we can use as a starting point > > for discussion: > > > > diff --git a/mm/folio.c b/mm/folio.c > > index dc2c796a4ea7..bc17cd3b6b6c 100644 > > --- a/mm/folio.c > > +++ b/mm/folio.c > > @@ -444,12 +444,12 @@ static bool lru_gen_clear_refs(struct folio *folio) > > int type = folio_is_file_lru(folio); > > unsigned long seq; > > > > - if (gen < 0) > > - return true; > > - > > folio_set_lru_refs(folio, 0); > > folio_clear_workingset(folio); > > > > + if (gen < 0) > > + return true; > > + > > rcu_read_lock(); > > seq = READ_ONCE(folio_lruvec(folio)->lrugen.min_seq[type]); > > rcu_read_unlock(); > > @@ -465,6 +465,8 @@ static void lru_gen_inc_refs(struct folio *folio) > > > > static bool lru_gen_clear_refs(struct folio *folio) > > { > > + folio_clear_referenced(folio); > > + folio_clear_workingset(folio); > > return false; > > } > > > > The another option is: > > > > diff --git a/mm/folio.c b/mm/folio.c > > index dc2c796a4ea7..8d1fd788188c 100644 > > --- a/mm/folio.c > > +++ b/mm/folio.c > > @@ -438,6 +438,12 @@ static void lru_gen_inc_refs(struct folio *folio) > > } while (!try_cmpxchg(folio_flags(folio, 0), &old_flags, new_flags)); > > } > > > > +static inline void __lru_gen_clear_refs(struct folio *folio) > > +{ > > + folio_set_lru_refs(folio, 0); > > + folio_clear_workingset(folio); > > +} > > + > > static bool lru_gen_clear_refs(struct folio *folio) > > { > > int gen = folio_lru_gen(folio); > > @@ -447,8 +453,7 @@ static bool lru_gen_clear_refs(struct folio *folio) > > if (gen < 0) > > return true; > > > > - folio_set_lru_refs(folio, 0); > > - folio_clear_workingset(folio); > > + __lru_gen_clear_refs(folio); > > > > rcu_read_lock(); > > seq = READ_ONCE(folio_lruvec(folio)->lrugen.min_seq[type]); > > @@ -463,9 +468,10 @@ static void lru_gen_inc_refs(struct folio *folio) > > { > > } > > > > -static bool lru_gen_clear_refs(struct folio *folio) > > +static inline void __lru_gen_clear_refs(struct folio *folio) > > { > > - return false; > > + folio_clear_referenced(folio); > > + folio_clear_workingset(folio); > > } > > > > #endif /* CONFIG_LRU_GEN */ > > @@ -659,10 +665,7 @@ static void lru_lazyfree(struct lruvec *lruvec, > > struct folio *folio) > > > > lruvec_del_folio(lruvec, folio); > > folio_clear_active(folio); > > - if (lru_gen_enabled()) > > - lru_gen_clear_refs(folio); > > - else > > - folio_clear_referenced(folio); > > + __lru_gen_clear_refs(folio); > > /* > > * Lazyfree folios are clean anonymous folios. They have > > * the swapbacked flag cleared, to distinguish them from normal > > > Hi Barry, Kairui, > > > > > I would personally prefer the latter. What do you think, Hui and Kairui? > > I prefer option B too: it keeps the flag clearing independent of the > list state and leaves the two deactivate callers untouched. > > But I'd suggest splitting this in two: > > 1. Take v2 as the fix now. If that is acceptable, I will respin it > with Cc: stable added. > > 2. Follow up with option B as a pure cleanup, no Fixes tag. The > lazyfree site then becomes the off-list __lru_gen_clear_refs() > call that Kairui wants for later work. Makes sense to me. I also personally hope this can be merged into mainline soon so that we can backport it to the Android common kernel. Having lazyfree folios with hotness information sitting on the file LRU really makes little sense. I recall I once did the below to ensure that lazyfree folios get higher reclaim priority: https://lore.kernel.org/linux-mm/20240914063746.46290-1-21cnbao@gmail.com/ And it significantly improves Android performance, according to Gaoxu from Honor: https://lore.kernel.org/linux-mm/7dcd3446cd8c4da69242e5d6680c1429@honor.com/ I didn't pursue that approach further, but at least yours should address part of the problem. Best, Barry ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2026-11-06 9:29 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2026-10-26 8:40 [PATCH v2] mm/mglru: fix a no-op lru_gen_clear_refs() call in lru_lazyfree() Hui Zhu 2026-10-26 9:47 ` Barry Song 2026-10-27 7:24 ` Hui Zhu 2026-10-27 11:33 ` Barry Song 2026-10-27 11:39 ` Kairui Song 2026-10-31 2:42 ` Barry Song 2026-10-31 7:25 ` Kairui Song 2026-10-31 7:26 ` Kairui Song 2026-11-05 21:13 ` Barry Song 2026-11-06 8:38 ` Hui Zhu 2026-11-06 9:29 ` Barry Song
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®