Re: [PATCH v2 2/2] mm/memcg: migrate per-node hugetlb lruvec stat together with hugetlb folio

[Muchun Song <muchun.song@linux.dev>]

> On Sep 23, 2026, at 17:55, Hongfu Li <hongfu.li@linux.dev> wrote:
> 
> 
> On 9/23/26 3:56 PM, Muchun Song wrote:
>> 
>>> On Sep 23, 2026, at 11:40, Hongfu Li <hongfu.li@linux.dev> wrote:
>>> 
>>> 
>>> On 9/23/26 10:40 AM, Muchun Song wrote:
>>>> 
>>>> On 2026/9/23 10:05, Hongfu Li wrote:
>>>>> From: Hongfu Li <lihongfu@kylinos.cn>
>>>>> 
>>>>> memory.numa_stat exposes per-node hugetlb counters from per-node lruvec
>>>>> stats. These stats are accounted against folio_nid(): incremented on
>>>>> the folio's node when handed to a user, decremented when the folio is
>>>>> returned to the pool.
>>>>> 
>>>>> During hugetlb folio migration, mem_cgroup_migrate() moves the charge
>>>>> to the new folio and drops the memcg data of the old one, so the free
>>>>> of the old folio right after the migration skips the memcg per-node
>>>>> lruvec decrement. The hugetlb count stays attributed to the old node
>>>>> for the rest of the life of the charge, while the target folio gets no
>>>>> increment on the new node; its later free decrements a counter that
>>>>> was never incremented.
>>>>> 
>>>>> Migrate the per-node lruvec accounting alongside migration. Global
>>>>> memcg totals remain balanced because they track resource consumption,
>>>>> not node placement.
>>>>> 
>>>>> Fixes: 05d4532b60e3 ("memcg/hugetlb: add hugeTLB counters to memcg")
>>>>> Cc: stable@vger.kernel.org
>>>>> Signed-off-by: Hongfu Li <lihongfu@kylinos.cn>
>>>>> Tested-by: Joshua Hahn <joshua.hahnjy@gmail.com>
>>>>> Reviewed-by: Joshua Hahn <joshua.hahnjy@gmail.com>
>>>>> Reviewed-by: Oscar Salvador <osalvador@suse.de>
>>>>> ---
>>>>>   include/linux/memcontrol.h |  8 ++++++++
>>>>>   mm/hugetlb.c               | 25 +++++++++++++++++++++++++
>>>>>   mm/memcontrol.c            |  5 ++---
>>>>>   3 files changed, 35 insertions(+), 3 deletions(-)
>>>>> 
>>>>> diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
>>>>> index a8358f297b65..74110a324f9e 100644
>>>>> --- a/include/linux/memcontrol.h
>>>>> +++ b/include/linux/memcontrol.h
>>>>> @@ -984,6 +984,9 @@ unsigned long lruvec_page_state_monotonic(const struct lruvec *lruvec,
>>>>>   unsigned long lruvec_page_state_local(const struct lruvec *lruvec,
>>>>>                         enum node_stat_item idx);
>>>>>   +void mod_memcg_lruvec_state(struct lruvec *lruvec,
>>>>> +                enum node_stat_item idx, int val);
>>>>> +
>>>>>   void mem_cgroup_flush_stats(struct mem_cgroup *memcg);
>>>>>   void mem_cgroup_flush_stats_ratelimited(struct mem_cgroup *memcg);
>>>>>   @@ -1452,6 +1455,11 @@ static inline unsigned long lruvec_page_state_local(const struct lruvec *lruvec,
>>>>>       return node_page_state(lruvec_pgdat(lruvec), idx);
>>>>>   }
>>>>>   +static inline void mod_memcg_lruvec_state(struct lruvec *lruvec,
>>>>> +                      enum node_stat_item idx, int val)
>>>>> +{
>>>>> +}
>>>>> +
>>>>>   static inline void mem_cgroup_flush_stats(struct mem_cgroup *memcg)
>>>>>   {
>>>>>   }
>>>>> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
>>>>> index 519c30b338a8..76d019594b39 100644
>>>>> --- a/mm/hugetlb.c
>>>>> +++ b/mm/hugetlb.c
>>>>> @@ -23,6 +23,7 @@
>>>>>   #include <linux/mmdebug.h>
>>>>>   #include <linux/sched/signal.h>
>>>>>   #include <linux/rmap.h>
>>>>> +#include <linux/rcupdate.h>
>>>>>   #include <linux/string_choices.h>
>>>>>   #include <linux/string_helpers.h>
>>>>>   #include <linux/swap.h>
>>>>> @@ -7378,12 +7379,36 @@ void folio_putback_hugetlb(struct folio *folio)
>>>>>       folio_put(folio);
>>>>>   }
>>>>>   +static void move_hugetlb_lruvec_stat(struct folio *old_folio,
>>>>> +                     struct folio *new_folio)
>>>>> +{
>>>>> +    struct mem_cgroup *memcg;
>>>>> +    long nr_pages = folio_nr_pages(old_folio);
>>>>> +    int old_nid = folio_nid(old_folio);
>>>>> +    int new_nid = folio_nid(new_folio);
>>>>> +
>>>>> +    if (old_nid == new_nid)
>>>>> +        return;
>>>>> +
>>>>> +    guard(rcu)();
>>>>> +
>>>>> +    memcg = folio_memcg(new_folio);
>>>>> +    if (!memcg)
>>>>> +        return;
>>>>> +
>>>>> +    mod_memcg_lruvec_state(mem_cgroup_lruvec(memcg, NODE_DATA(old_nid)),
>>>>> +                   NR_HUGETLB, -nr_pages);
>>>> Why not use mod_lruvec_state? mod_memcg_lruvec_state is an internal
>>>> API for memcg, I don't want it to be exported.
>>> Thank you for the review.
>>> 
>>> mod_lruvec_state() would update the node counter a second time.  It calls
>>> mod_node_page_state() as well, and the target's node counter is already
>>> updated in alloc_hugetlb_folio_nodemask() (patch 1/2):
>>> 
>>>     lruvec_stat_mod_folio(folio, NR_HUGETLB, folio_nr_pages(folio));
>>> 
>>> For an uncharged folio lruvec_stat_mod_folio() only updates the node
>>> counter.  The target folio is not charged to any memcg at that point; its
>>> charge only appears later in mem_cgroup_migrate().  So the node side is
>>> already covered and only the per-memcg attribution has to follow the
>>> charge here.
>> Looking at the first patch again, I do not think the
>> overall accounting is incorrect. My concern is where NR_HUGETLB
>> accounting is performed.
>> 
>> The low-level allocation helpers only take a folio from the
>> hugetlb pool. They do not have enough context to decide how
>> the folio should be accounted. That decision should be left
>> to their callers, once the folio enters the relevant usage
>> lifecycle.
>> 
>> hugetlb_alloc_folio() is a good example. It first obtains a
>> folio, then calls mem_cgroup_charge_hugetlb(), and only
>> afterwards calls lruvec_stat_mod_folio(). At that point,
>> the folio has the correct memcg and lruvec, so all relevant
>> counters can be updated together.
>> 
>> After this series, alloc_hugetlb_folio_reserve() and
>> alloc_hugetlb_folio_nodemask() account NR_HUGETLB before
>> returning the folio. This makes them unsuitable for callers
>> that need to charge the folio afterwards. Before the charge,
>> lruvec_stat_mod_folio() can update only the node counter.
>> The later charge does not associate that earlier update with
>> the new memcg.
>> 
>> If a caller accounts again after charging, the node counter is
>> updated twice. If it does not, the memcg and per-node lruvec
>> counters remain missing. The caller then needs a memcg-only
>> correction, creating a special accounting protocol between
>> the helper and its callers.
>> 
>> Migration exposes the same issue. The target gets node
>> accounting before it inherits the source memcg. The second patch
>> must update the lruvec state and then cancel its node changes,
>> because the target allocation already updated the node counter.
>> This works, but makes the accounting lifecycle harder to follow.
>> 
>> Would it be cleaner to keep these helpers focused on allocation
>> and let each caller account at the correct lifecycle point?
>> Normal allocation can account after a successful memcg charge.
>> Migration can move the complete lruvec state after it succeeds.
> 
> Hi Muchun,

Hi,

> 
> Thanks a lot for the detailed explanation.
> 
> One thing keeps the target accounting in alloc_hugetlb_folio_nodemask(): the
> node counter is paired with the folio's free, and a failed migration frees its
> target right away (put_new_folio()/folio_put() -> free_huge_folio(), which
> always decrements the node counter). If the target were accounted only after a
> successful migration, every failed migration would decrement a node counter
> that was never incremented.

You're right, and HugeTLB is really tricky. I think HugeTLB might need some cleanup
in the future to better handle the accounting here. As a bug fix, the current
changes look good to me.

Acked-by: Muchun Song <muchun.song@linux.dev>

A future cleanup, might be to have all the underlying HugeTLB allocation functions
update only the global node statistics (use mod_node_page_state instead	of
lruvec_stat_mod_folio). Its caller, if it needs to charge memory
cgroup, can then call mod_memcg_lruvec_state that updates only the memory cgroup
statistics.

Thanks.

> 
> Unless there is a way to let free_huge_folio() tell that the folio comes
> from a failed migration, and skip the decrement for it.
> 
>> An uncharged path can explicitly update only the node counter
>> if global accounting is still required.
>> 
>> This would keep policy out of the low-level allocator and make
>> the accounting lifecycle easier to follow.
>> 
>> Thanks.
>> 
>>>> Thanks.
>>>> 
>>>>> + mod_memcg_lruvec_state(mem_cgroup_lruvec(memcg, NODE_DATA(new_nid)),
>>>>> +                   NR_HUGETLB, nr_pages);
>>>>> +}
>>>>> +
>>>>>   void move_hugetlb_state(struct folio *old_folio, struct folio *new_folio,
>>>>>               enum migrate_reason reason)
>>>>>   {
>>>>>       struct hstate *h = folio_hstate(old_folio);
>>>>>         hugetlb_cgroup_migrate(old_folio, new_folio);
>>>>> +    move_hugetlb_lruvec_stat(old_folio, new_folio);
>>>>>       folio_set_owner_migrate_reason(new_folio, reason);
>>>>>         /*
>>>>> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
>>>>> index 88824f783571..a5335da5d425 100644
>>>>> --- a/mm/memcontrol.c
>>>>> +++ b/mm/memcontrol.c
>>>>> @@ -1015,9 +1015,8 @@ static void __mod_memcg_lruvec_state(struct mem_cgroup_per_node *pn,
>>>>>       put_cpu();
>>>>>   }
>>>>>   -static void mod_memcg_lruvec_state(struct lruvec *lruvec,
>>>>> -                     enum node_stat_item idx,
>>>>> -                     int val)
>>>>> +void mod_memcg_lruvec_state(struct lruvec *lruvec,
>>>>> +                enum node_stat_item idx, int val)
>>>>>   {
>>>>>       struct pglist_data *pgdat = lruvec_pgdat(lruvec);
>>>>>       struct mem_cgroup_per_node *pn;
>>>>> 
>>> -- 
>>> Best regards,
>>> Hongfu
> 
> -- 
> Best regards,
> Hongfu