mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Lian Wang <lianux.mm@gmail.com>
To: Kairui Song <kasong@tencent.com>
Cc: "wang lian(ByteDance)" <lianux.mm@gmail.com>,
	linux-mm@kvack.org, "Johannes Weiner" <hannes@cmpxchg.org>,
	"Muchun Song" <muchun.song@linux.dev>,
	"Qi Zheng" <qi.zheng@linux.dev>,
	"Ying Huang" <ying.huang@linux.alibaba.com>,
	"Chris Li" <chrisl@kernel.org>,
	"Baoquan He" <baoquan.he@linux.dev>,
	"Nico Pache" <nico.pache@linux.dev>,
	"Usama Arif" <usama.arif@linux.dev>,
	"Michal Hocko" <mhocko@kernel.org>,
	"Roman Gushchin" <roman.gushchin@linux.dev>,
	"Shakeel Butt" <shakeel.butt@linux.dev>,
	"David Hildenbrand" <david@kernel.org>,
	"Lorenzo Stoakes" <ljs@kernel.org>,
	"Liam R. Howlett" <liam@infradead.org>,
	"Barry Song" <baohua@kernel.org>,
	"Axel Rasmussen" <axelrasmussen@google.com>,
	"Yuanchu Xie" <yuanchu@google.com>, "Wei Xu" <weixugc@google.com>,
	"Vlastimil Babka" <vbabka@kernel.org>,
	"Suren Baghdasaryan" <surenb@google.com>,
	"Nhat Pham" <nphamcs@gmail.com>,
	"Youngjun Park" <youngjun.park@lge.com>,
	"Zi Yan" <ziy@nvidia.com>, "Gregory Price" <gourry@gourry.net>,
	"Matthew Wilcox (Oracle)" <willy@infradead.org>,
	"Baolin Wang" <baolin.wang@linux.alibaba.com>,
	"Ryan Roberts" <ryan.roberts@arm.com>,
	"Dev Jain" <dev.jain@arm.com>,
	"Lance Yang" <lance.yang@linux.dev>,
	"Hugh Dickins" <hughd@google.com>, "SJ Park" <sj@kernel.org>,
	"David Rientjes" <rientjes@google.com>,
	"Yu Zhao" <yuzhao@google.com>,
	"Vernon Yang" <vernon2gm@gmail.com>,
	"Zicheng Wang" <wangzicheng@honor.com>,
	"Chen Ridong" <chenridong@xiaomi.com>,
	"Tal Zussman" <tz2294@columbia.edu>,
	"Kairui Song" <ryncsn@gmail.com>,
	"Xiang Liu" <liuxiang.277@bytedance.com>,
	"Ehab Ababneh" <ehab.ababneh@intel.com>,
	"Zhao Zhenzhuo" <zhaozhengzhuo@uniontech.com>,
	"Joanne Koong" <joannelkoong@gmail.com>,
	"Kunwu Chan" <kunwu.chan@gmail.com>,
	"Hui Zhu" <hui.zhu@linux.dev>,
	linux-kernel@vger.kernel.org, cgroups@vger.kernel.org,
	damon@lists.linux.dev, linux-doc@vger.kernel.org
Subject: Re: [PATCH v5 00/17] mm/mglru: frequency guided promotion (MGLRU-FG) and flag cleanup
Date: Thu, 19 Nov 2026 13:13:33 +0800	[thread overview]
Message-ID: <20261119051404.14533-1-lianux.mm@gmail.com> (raw)
In-Reply-To: <20261119-mglru-fg-v5-0-9eee2fe97a2d@tencent.com>

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="us-ascii", Size: 3520 bytes --]

From: wang lian(ByteDance) <lianux.mm@gmail.com>

Hi Kairui,

On Thu, 19 Nov 2026 05:03:32 +0800 Kairui Song <kasong@tencent.com> wrote:

> I also retested the LevelDB benchmark from the cache_ext paper [4].
> Interestingly, mainline MGLRU already beats CLRU on this one after a
> recent change in lru_gen_folio_seq that bumps new folios with refs == 1
> to the second-oldest generation as discussed. That change accidentally
> gave random reads a higher hotness level while making sequential reads
> much colder as described before [13]. However it's unreliable and
> incorrect because it depends on LRU drain timing, and hurts some workloads
> as reported, or where the sequential part is actually hotter (any workload
> involving hotter large file and many small cold files will be affected)
> which is reproduced below.
> 
> This series improves it while covering ordinary workloads too and fixing
> more regressions:
> 
> LevelDB Scan / Get, Throughput Total:
> CLRU:         4708.3 ops/s
> MGLRU:        5026.9 ops/s (faster than CLRU, but hurts other workloads)
> MGLRU After:  5179.2 ops/s (even better in all cases, and no regression)
>

We did observe a LevelDB regression in our earlier tests on our
machine. The fixes and results discussed in your cover look promising,
and may address our case too. I haven't retested v5 locally yet, so I'll
rerun the workload on the current base to check whether the regression
is resolved. I'll also look more closely at the access patterns and how
MGLRU behaves with them.

The results on file-backed workloads have been especially encouraging.
Next, I'd like to discuss some anonymous-memory workloads and potential
fixes with you.

> Others
> ======
> 
> Additionally, PSI, smaps, and readahead should all benefit from better
> accuracy since this series unifies the flag usage between classical
> LRU and MGLRU.
> 
> Other tests such as MySQL are looking fine, with no regressions. There is
> also community test report [7].
> 
> Refault distance is not included yet, so MGLRU may respond more slowly to
> workingset shifts. That can be added later, as previously demonstrated
> [8], [9].
> 
> Extra note about future development: this series is actually highly
> compatible with ideas like workingset reporting [10]. The "gen climbing
> folio" design may appear to conflict with workingset reporting's idea,
> but it doesn't. The solution is simple and straightforward: once we can
> extend the generation number to a larger value (e.g. 64 or 128), the
> refs-driven promotion can stop at a lower gen (e.g. oldest_gen + 16),
> leaving the remaining newer generations as perfectly time-gap-separated
> bins. These older gens can be considered as expiring gen.
> 
> The tier count is not fixed either, we'll need to find a way to tune it
> if tiers go beyond 4 though, but that shouldn't be hard, not a blocker
> either.
> 
> More details are in the individual commit messages. LLM is used to help
> improve the comments and tests, all results are double checked.
>

Thank you for this outstanding work! We've seen some really exciting
gains in our tests, and we're very happy with the results. From our
earlier private conversations, I know how much time and effort you've
invested in this. You deserve all the praise for that dedication.

I'll continue reviewing and testing the series and work with you on
further improvements. I'll go through the patches one by one, and I'll
share the workload scenarios and requirements from our side as we go.

Thanks,
Lian

      parent reply	other threads:[~2026-11-19  5:14 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-18 21:03 Kairui Song via B4 Relay
2026-11-18 21:03 ` [PATCH v5 01/17] mm/mglru: rename the generation offset in lru_gen_folio_seq() Kairui Song via B4 Relay
2026-11-18 23:48   ` Barry Song
2026-11-19  0:27   ` Ridong Chen
2026-11-19  2:45   ` lianux.mm
2026-11-18 21:03 ` [PATCH v5 02/17] mm/memcontrol: allow update of LRU statistic without holding LRU lock Kairui Song via B4 Relay
2026-11-18 21:03 ` [PATCH v5 03/17] mm/mglru: make generation page counters atomic Kairui Song via B4 Relay
2026-11-18 21:03 ` [PATCH v5 04/17] mm/memcg: add folio-based lruvec live helper Kairui Song via B4 Relay
2026-11-18 21:03 ` [PATCH v5 05/17] mm/mglru: skip the flags cmpxchg when nothing changes Kairui Song via B4 Relay
2026-11-18 21:03 ` [PATCH v5 06/17] mm/mglru: frequency guided workingset promotion (MGLRU-FG) Kairui Song via B4 Relay
2026-11-18 21:03 ` [PATCH v5 07/17] mm/mglru: make folio lru referenced times count a generic API Kairui Song via B4 Relay
2026-11-18 21:03 ` [PATCH v5 08/17] mm/mglru: move add/del LRU size accounting out of lru_gen_update_size() Kairui Song via B4 Relay
2026-11-18 21:03 ` [PATCH v5 09/17] mm/mglru, gup: mark folios referenced via a fast helper Kairui Song via B4 Relay
2026-11-18 21:03 ` [PATCH v5 10/17] mm/mglru, smaps: convert to LRU refs based operations for MGLRU Kairui Song via B4 Relay
2026-11-18 21:03 ` [PATCH v5 11/17] mm/damon: " Kairui Song via B4 Relay
2026-11-18 21:03 ` [PATCH v5 12/17] mm/khugepaged: " Kairui Song via B4 Relay
2026-11-18 21:03 ` [PATCH v5 13/17] mm/madvise: adapt for " Kairui Song via B4 Relay
2026-11-18 21:03 ` [PATCH v5 14/17] mm/huge_memory: mark folio as accessed more accurately on split Kairui Song via B4 Relay
2026-11-18 21:03 ` [PATCH v5 15/17] mm/mglru: folio LRU refs based active/inactive number accounting Kairui Song via B4 Relay
2026-11-18 21:03 ` [PATCH v5 16/17] mm/mglru: make folio_inc_lru_refs lruvec lockless Kairui Song via B4 Relay
2026-11-18 21:03 ` [PATCH v5 17/17] mm/mglru: make folio_test_workingset() work based on folio LRU refs Kairui Song via B4 Relay
2026-11-19  5:13 ` Lian Wang [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261119051404.14533-1-lianux.mm@gmail.com \
    --to=lianux.mm@gmail.com \
    --cc=axelrasmussen@google.com \
    --cc=baohua@kernel.org \
    --cc=baolin.wang@linux.alibaba.com \
    --cc=baoquan.he@linux.dev \
    --cc=cgroups@vger.kernel.org \
    --cc=chenridong@xiaomi.com \
    --cc=chrisl@kernel.org \
    --cc=damon@lists.linux.dev \
    --cc=david@kernel.org \
    --cc=dev.jain@arm.com \
    --cc=ehab.ababneh@intel.com \
    --cc=gourry@gourry.net \
    --cc=hannes@cmpxchg.org \
    --cc=hughd@google.com \
    --cc=hui.zhu@linux.dev \
    --cc=joannelkoong@gmail.com \
    --cc=kasong@tencent.com \
    --cc=kunwu.chan@gmail.com \
    --cc=lance.yang@linux.dev \
    --cc=liam@infradead.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=liuxiang.277@bytedance.com \
    --cc=ljs@kernel.org \
    --cc=mhocko@kernel.org \
    --cc=muchun.song@linux.dev \
    --cc=nico.pache@linux.dev \
    --cc=nphamcs@gmail.com \
    --cc=qi.zheng@linux.dev \
    --cc=rientjes@google.com \
    --cc=roman.gushchin@linux.dev \
    --cc=ryan.roberts@arm.com \
    --cc=ryncsn@gmail.com \
    --cc=shakeel.butt@linux.dev \
    --cc=sj@kernel.org \
    --cc=surenb@google.com \
    --cc=tz2294@columbia.edu \
    --cc=usama.arif@linux.dev \
    --cc=vbabka@kernel.org \
    --cc=vernon2gm@gmail.com \
    --cc=wangzicheng@honor.com \
    --cc=weixugc@google.com \
    --cc=willy@infradead.org \
    --cc=ying.huang@linux.alibaba.com \
    --cc=youngjun.park@lge.com \
    --cc=yuanchu@google.com \
    --cc=yuzhao@google.com \
    --cc=zhaozhengzhuo@uniontech.com \
    --cc=ziy@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®