[PATCH v2 0/2] mm: fix hugetlb NR_HUGETLB accounting on folio migration

[PATCH v2 0/2] mm: fix hugetlb NR_HUGETLB accounting on folio migration

[Hongfu Li]

The hugeTLB counters added by 05d4532b60e3 ("memcg/hugetlb: add hugeTLB
counters to memcg") are maintained in two per-node places:
  - the per-node vmstat counter NR_HUGETLB, exposed as nr_hugetlb in
    /proc/vmstat;
  - the per-node memcg lruvec stat, exposed via memory.numa_stat.

Both are accounted against the folio's node, and both drift when a
hugetlb folio is migrated, though in different ways.

A migration target folio is allocated by alloc_hugetlb_folio_nodemask()
and inherits the old folio's state without ever being accounted, while
the old folio is freed right after and its free is accounted. That
alone loses vmstat accounting: the target node has no matching increment
for the decrement on the old node, so /proc/vmstat's nr_hugetlb shrinks
by nr_pages per migration. Patch 1 accounts the folio where it is
obtained, so the increment pairs with the free in free_huge_folio() on
the successful as well as the failed migration path. The memfd page
cache preallocation helper has the same asymmetry and is fixed in the
same patch.

The per-node lruvec stat breaks differently. mem_cgroup_migrate()
moves the charge to the new folio and drops the old folio's memcg data,
so the old folio's free right after migration skips the memcg per-node
decrement; the count stays attributed to the old node for the rest of
the charge's life, and the target folio never gets an increment on its
new node. Patch 2 moves that per-node accounting along with the charge,
in move_hugetlb_state().

---
Changes in v2:
- In patch 2/2, move_hugetlb_lruvec_stat() no longer takes the objcg
  parameter; it takes the memcg from the new folio and uses guard(rcu)().
- In patch 2/2, move_hugetlb_lruvec_stat() is moved to mm/hugetlb.c and is
  now called from move_hugetlb_state() instead of mem_cgroup_migrate().
- In patch 2/2, make mod_memcg_lruvec_state() non-static (declared in
  include/linux/memcontrol.h).
- Collected tags on both patches.

To: Muchun Song <muchun.song@linux.dev>
To: Oscar Salvador <osalvador@suse.de>
To: David Hildenbrand <david@kernel.org>
To: Andrew Morton <akpm@linux-foundation.org>
To: Chris Down <chris@chrisdown.name>
To: Johannes Weiner <hannes@cmpxchg.org>
To: Shakeel Butt <shakeel.butt@linux.dev>
To: Nhat Pham <nphamcs@gmail.com>
To: Michal Hocko <mhocko@kernel.org>
To: Roman Gushchin <roman.gushchin@linux.dev>
Cc: hongfu.li@linux.dev
Cc: Michal Hocko <mhocko@suse.com>
Cc: linux-mm@kvack.org
Cc: linux-kernel@vger.kernel.org
Cc: Joshua Hahn <joshua.hahnjy@gmail.com>
Cc: cgroups@vger.kernel.org

---
Hongfu Li (2):
      mm/hugetlb: account migration target folio in per-node NR_HUGETLB vmstat
      mm/memcg: migrate per-node hugetlb lruvec stat together with hugetlb folio

 include/linux/memcontrol.h |  8 +++++++
 mm/hugetlb.c               | 60 ++++++++++++++++++++++++++++++++++++----------
 mm/memcontrol.c            |  5 ++--
 3 files changed, 58 insertions(+), 15 deletions(-)
---
base-commit: 8d29b5365d528da545c1fe0768a55babadee95d0
change-id: 20260922-for-hugetlb_state3-573f27215d4e

Best regards,
--  
Hongfu Li <lihongfu@kylinos.cn>

[PATCH v2 1/2] mm/hugetlb: account migration target folio in per-node NR_HUGETLB vmstat

[Hongfu Li]

From: Hongfu Li <lihongfu@kylinos.cn>

The NR_HUGETLB vmstat counter is maintained per folio's node: incremented
when a huge page is handed to a user via hugetlb_alloc_folio() and
decremented when it is returned to the pool via free_huge_folio().

A folio obtained by alloc_hugetlb_folio_nodemask() never goes through
hugetlb_alloc_folio(), so it is never accounted, while its free always
is. For a migration target this means the target node gets no matching
increment for the decrement on the old node, so the global nr_hugetlb in
/proc/vmstat drops by nr_pages for each migration. The same asymmetry
affects the failed migration path, which frees the target again right
away, and the temporary folio hugetlb_mfill_atomic_pte() takes from the
same helper.

alloc_hugetlb_folio_reserve(), used to preallocate the memfd page cache
folios, has the same asymmetry: the folio is handed to a user without
being accounted, while its free is accounted through free_huge_folio().

Account the folio where it is obtained, in alloc_hugetlb_folio_nodemask()
and alloc_hugetlb_folio_reserve(), so that the increment pairs with the
decrement in free_huge_folio(): a successful migration hands the folio
to a user, a failed one frees it again.

Fixes: 05d4532b60e3 ("memcg/hugetlb: add hugeTLB counters to memcg")
Cc: stable@vger.kernel.org
Signed-off-by: Hongfu Li <lihongfu@kylinos.cn>
Tested-by: Joshua Hahn <joshua.hahnjy@gmail.com>
Reviewed-by: Joshua Hahn <joshua.hahnjy@gmail.com>
Acked-by: Muchun Song <muchun.song@linux.dev>
Acked-by: Oscar Salvador <osalvador@suse.de>
---
 mm/hugetlb.c | 35 +++++++++++++++++++++++------------
 1 file changed, 23 insertions(+), 12 deletions(-)

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index da980377d353..519c30b338a8 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -2206,6 +2206,11 @@ struct folio *alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_nid,
 	}
 
 	spin_unlock_irq(&hugetlb_lock);
+
+	if (folio)
+		lruvec_stat_mod_folio(folio, NR_HUGETLB,
+				      folio_nr_pages(folio));
+
 	return folio;
 }
 
@@ -2213,24 +2218,30 @@ struct folio *alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_nid,
 struct folio *alloc_hugetlb_folio_nodemask(struct hstate *h, int preferred_nid,
 		nodemask_t *nmask, gfp_t gfp_mask, bool allow_alloc_fallback)
 {
-	spin_lock_irq(&hugetlb_lock);
-	if (available_huge_pages(h)) {
-		struct folio *folio;
+	struct folio *folio = NULL;
 
+	spin_lock_irq(&hugetlb_lock);
+	if (available_huge_pages(h))
 		folio = dequeue_hugetlb_folio_nodemask(h, gfp_mask,
 						preferred_nid, nmask);
-		if (folio) {
-			spin_unlock_irq(&hugetlb_lock);
-			return folio;
-		}
-	}
 	spin_unlock_irq(&hugetlb_lock);
 
-	/* We cannot fallback to other nodes, as we could break the per-node pool. */
-	if (!allow_alloc_fallback)
-		gfp_mask |= __GFP_THISNODE;
+	if (!folio) {
+		/*
+		 * We cannot fallback to other nodes, as we could break the
+		 * per-node pool.
+		 */
+		if (!allow_alloc_fallback)
+			gfp_mask |= __GFP_THISNODE;
 
-	return alloc_migrate_hugetlb_folio(h, gfp_mask, preferred_nid, nmask);
+		folio = alloc_migrate_hugetlb_folio(h, gfp_mask, preferred_nid,
+						    nmask);
+	}
+
+	if (folio)
+		lruvec_stat_mod_folio(folio, NR_HUGETLB, folio_nr_pages(folio));
+
+	return folio;
 }
 
 static nodemask_t *policy_mbind_nodemask(gfp_t gfp)

-- 
2.54.0

[PATCH v2 2/2] mm/memcg: migrate per-node hugetlb lruvec stat together with hugetlb folio

[Hongfu Li]

From: Hongfu Li <lihongfu@kylinos.cn>

memory.numa_stat exposes per-node hugetlb counters from per-node lruvec
stats. These stats are accounted against folio_nid(): incremented on
the folio's node when handed to a user, decremented when the folio is
returned to the pool.

During hugetlb folio migration, mem_cgroup_migrate() moves the charge
to the new folio and drops the memcg data of the old one, so the free
of the old folio right after the migration skips the memcg per-node
lruvec decrement. The hugetlb count stays attributed to the old node
for the rest of the life of the charge, while the target folio gets no
increment on the new node; its later free decrements a counter that
was never incremented.

Migrate the per-node lruvec accounting alongside migration. Global
memcg totals remain balanced because they track resource consumption,
not node placement.

Fixes: 05d4532b60e3 ("memcg/hugetlb: add hugeTLB counters to memcg")
Cc: stable@vger.kernel.org
Signed-off-by: Hongfu Li <lihongfu@kylinos.cn>
Tested-by: Joshua Hahn <joshua.hahnjy@gmail.com>
Reviewed-by: Joshua Hahn <joshua.hahnjy@gmail.com>
Reviewed-by: Oscar Salvador <osalvador@suse.de>
---
 include/linux/memcontrol.h |  8 ++++++++
 mm/hugetlb.c               | 25 +++++++++++++++++++++++++
 mm/memcontrol.c            |  5 ++---
 3 files changed, 35 insertions(+), 3 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index a8358f297b65..74110a324f9e 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -984,6 +984,9 @@ unsigned long lruvec_page_state_monotonic(const struct lruvec *lruvec,
 unsigned long lruvec_page_state_local(const struct lruvec *lruvec,
 				      enum node_stat_item idx);
 
+void mod_memcg_lruvec_state(struct lruvec *lruvec,
+			    enum node_stat_item idx, int val);
+
 void mem_cgroup_flush_stats(struct mem_cgroup *memcg);
 void mem_cgroup_flush_stats_ratelimited(struct mem_cgroup *memcg);
 
@@ -1452,6 +1455,11 @@ static inline unsigned long lruvec_page_state_local(const struct lruvec *lruvec,
 	return node_page_state(lruvec_pgdat(lruvec), idx);
 }
 
+static inline void mod_memcg_lruvec_state(struct lruvec *lruvec,
+					  enum node_stat_item idx, int val)
+{
+}
+
 static inline void mem_cgroup_flush_stats(struct mem_cgroup *memcg)
 {
 }
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 519c30b338a8..76d019594b39 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -23,6 +23,7 @@
 #include <linux/mmdebug.h>
 #include <linux/sched/signal.h>
 #include <linux/rmap.h>
+#include <linux/rcupdate.h>
 #include <linux/string_choices.h>
 #include <linux/string_helpers.h>
 #include <linux/swap.h>
@@ -7378,12 +7379,36 @@ void folio_putback_hugetlb(struct folio *folio)
 	folio_put(folio);
 }
 
+static void move_hugetlb_lruvec_stat(struct folio *old_folio,
+				     struct folio *new_folio)
+{
+	struct mem_cgroup *memcg;
+	long nr_pages = folio_nr_pages(old_folio);
+	int old_nid = folio_nid(old_folio);
+	int new_nid = folio_nid(new_folio);
+
+	if (old_nid == new_nid)
+		return;
+
+	guard(rcu)();
+
+	memcg = folio_memcg(new_folio);
+	if (!memcg)
+		return;
+
+	mod_memcg_lruvec_state(mem_cgroup_lruvec(memcg, NODE_DATA(old_nid)),
+			       NR_HUGETLB, -nr_pages);
+	mod_memcg_lruvec_state(mem_cgroup_lruvec(memcg, NODE_DATA(new_nid)),
+			       NR_HUGETLB, nr_pages);
+}
+
 void move_hugetlb_state(struct folio *old_folio, struct folio *new_folio,
 			enum migrate_reason reason)
 {
 	struct hstate *h = folio_hstate(old_folio);
 
 	hugetlb_cgroup_migrate(old_folio, new_folio);
+	move_hugetlb_lruvec_stat(old_folio, new_folio);
 	folio_set_owner_migrate_reason(new_folio, reason);
 
 	/*
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 88824f783571..a5335da5d425 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -1015,9 +1015,8 @@ static void __mod_memcg_lruvec_state(struct mem_cgroup_per_node *pn,
 	put_cpu();
 }
 
-static void mod_memcg_lruvec_state(struct lruvec *lruvec,
-				     enum node_stat_item idx,
-				     int val)
+void mod_memcg_lruvec_state(struct lruvec *lruvec,
+			    enum node_stat_item idx, int val)
 {
 	struct pglist_data *pgdat = lruvec_pgdat(lruvec);
 	struct mem_cgroup_per_node *pn;

-- 
2.54.0

Re: [PATCH v2 2/2] mm/memcg: migrate per-node hugetlb lruvec stat together with hugetlb folio

[Muchun Song]

On 2026/9/23 10:05, Hongfu Li wrote:
> From: Hongfu Li <lihongfu@kylinos.cn>
>
> memory.numa_stat exposes per-node hugetlb counters from per-node lruvec
> stats. These stats are accounted against folio_nid(): incremented on
> the folio's node when handed to a user, decremented when the folio is
> returned to the pool.
>
> During hugetlb folio migration, mem_cgroup_migrate() moves the charge
> to the new folio and drops the memcg data of the old one, so the free
> of the old folio right after the migration skips the memcg per-node
> lruvec decrement. The hugetlb count stays attributed to the old node
> for the rest of the life of the charge, while the target folio gets no
> increment on the new node; its later free decrements a counter that
> was never incremented.
>
> Migrate the per-node lruvec accounting alongside migration. Global
> memcg totals remain balanced because they track resource consumption,
> not node placement.
>
> Fixes: 05d4532b60e3 ("memcg/hugetlb: add hugeTLB counters to memcg")
> Cc: stable@vger.kernel.org
> Signed-off-by: Hongfu Li <lihongfu@kylinos.cn>
> Tested-by: Joshua Hahn <joshua.hahnjy@gmail.com>
> Reviewed-by: Joshua Hahn <joshua.hahnjy@gmail.com>
> Reviewed-by: Oscar Salvador <osalvador@suse.de>
> ---
>   include/linux/memcontrol.h |  8 ++++++++
>   mm/hugetlb.c               | 25 +++++++++++++++++++++++++
>   mm/memcontrol.c            |  5 ++---
>   3 files changed, 35 insertions(+), 3 deletions(-)
>
> diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
> index a8358f297b65..74110a324f9e 100644
> --- a/include/linux/memcontrol.h
> +++ b/include/linux/memcontrol.h
> @@ -984,6 +984,9 @@ unsigned long lruvec_page_state_monotonic(const struct lruvec *lruvec,
>   unsigned long lruvec_page_state_local(const struct lruvec *lruvec,
>   				      enum node_stat_item idx);
>   
> +void mod_memcg_lruvec_state(struct lruvec *lruvec,
> +			    enum node_stat_item idx, int val);
> +
>   void mem_cgroup_flush_stats(struct mem_cgroup *memcg);
>   void mem_cgroup_flush_stats_ratelimited(struct mem_cgroup *memcg);
>   
> @@ -1452,6 +1455,11 @@ static inline unsigned long lruvec_page_state_local(const struct lruvec *lruvec,
>   	return node_page_state(lruvec_pgdat(lruvec), idx);
>   }
>   
> +static inline void mod_memcg_lruvec_state(struct lruvec *lruvec,
> +					  enum node_stat_item idx, int val)
> +{
> +}
> +
>   static inline void mem_cgroup_flush_stats(struct mem_cgroup *memcg)
>   {
>   }
> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> index 519c30b338a8..76d019594b39 100644
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -23,6 +23,7 @@
>   #include <linux/mmdebug.h>
>   #include <linux/sched/signal.h>
>   #include <linux/rmap.h>
> +#include <linux/rcupdate.h>
>   #include <linux/string_choices.h>
>   #include <linux/string_helpers.h>
>   #include <linux/swap.h>
> @@ -7378,12 +7379,36 @@ void folio_putback_hugetlb(struct folio *folio)
>   	folio_put(folio);
>   }
>   
> +static void move_hugetlb_lruvec_stat(struct folio *old_folio,
> +				     struct folio *new_folio)
> +{
> +	struct mem_cgroup *memcg;
> +	long nr_pages = folio_nr_pages(old_folio);
> +	int old_nid = folio_nid(old_folio);
> +	int new_nid = folio_nid(new_folio);
> +
> +	if (old_nid == new_nid)
> +		return;
> +
> +	guard(rcu)();
> +
> +	memcg = folio_memcg(new_folio);
> +	if (!memcg)
> +		return;
> +
> +	mod_memcg_lruvec_state(mem_cgroup_lruvec(memcg, NODE_DATA(old_nid)),
> +			       NR_HUGETLB, -nr_pages);

Why not use mod_lruvec_state? mod_memcg_lruvec_state is an internal
API for memcg, I don't want it to be exported.

Thanks.

> +	mod_memcg_lruvec_state(mem_cgroup_lruvec(memcg, NODE_DATA(new_nid)),
> +			       NR_HUGETLB, nr_pages);
> +}
> +
>   void move_hugetlb_state(struct folio *old_folio, struct folio *new_folio,
>   			enum migrate_reason reason)
>   {
>   	struct hstate *h = folio_hstate(old_folio);
>   
>   	hugetlb_cgroup_migrate(old_folio, new_folio);
> +	move_hugetlb_lruvec_stat(old_folio, new_folio);
>   	folio_set_owner_migrate_reason(new_folio, reason);
>   
>   	/*
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index 88824f783571..a5335da5d425 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -1015,9 +1015,8 @@ static void __mod_memcg_lruvec_state(struct mem_cgroup_per_node *pn,
>   	put_cpu();
>   }
>   
> -static void mod_memcg_lruvec_state(struct lruvec *lruvec,
> -				     enum node_stat_item idx,
> -				     int val)
> +void mod_memcg_lruvec_state(struct lruvec *lruvec,
> +			    enum node_stat_item idx, int val)
>   {
>   	struct pglist_data *pgdat = lruvec_pgdat(lruvec);
>   	struct mem_cgroup_per_node *pn;
>

Re: [PATCH v2 2/2] mm/memcg: migrate per-node hugetlb lruvec stat together with hugetlb folio

[Hongfu Li]

On 9/23/26 10:40 AM, Muchun Song wrote:
>
>
> On 2026/9/23 10:05, Hongfu Li wrote:
>> From: Hongfu Li <lihongfu@kylinos.cn>
>>
>> memory.numa_stat exposes per-node hugetlb counters from per-node lruvec
>> stats. These stats are accounted against folio_nid(): incremented on
>> the folio's node when handed to a user, decremented when the folio is
>> returned to the pool.
>>
>> During hugetlb folio migration, mem_cgroup_migrate() moves the charge
>> to the new folio and drops the memcg data of the old one, so the free
>> of the old folio right after the migration skips the memcg per-node
>> lruvec decrement. The hugetlb count stays attributed to the old node
>> for the rest of the life of the charge, while the target folio gets no
>> increment on the new node; its later free decrements a counter that
>> was never incremented.
>>
>> Migrate the per-node lruvec accounting alongside migration. Global
>> memcg totals remain balanced because they track resource consumption,
>> not node placement.
>>
>> Fixes: 05d4532b60e3 ("memcg/hugetlb: add hugeTLB counters to memcg")
>> Cc: stable@vger.kernel.org
>> Signed-off-by: Hongfu Li <lihongfu@kylinos.cn>
>> Tested-by: Joshua Hahn <joshua.hahnjy@gmail.com>
>> Reviewed-by: Joshua Hahn <joshua.hahnjy@gmail.com>
>> Reviewed-by: Oscar Salvador <osalvador@suse.de>
>> ---
>>   include/linux/memcontrol.h |  8 ++++++++
>>   mm/hugetlb.c               | 25 +++++++++++++++++++++++++
>>   mm/memcontrol.c            |  5 ++---
>>   3 files changed, 35 insertions(+), 3 deletions(-)
>>
>> diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
>> index a8358f297b65..74110a324f9e 100644
>> --- a/include/linux/memcontrol.h
>> +++ b/include/linux/memcontrol.h
>> @@ -984,6 +984,9 @@ unsigned long lruvec_page_state_monotonic(const 
>> struct lruvec *lruvec,
>>   unsigned long lruvec_page_state_local(const struct lruvec *lruvec,
>>                         enum node_stat_item idx);
>>   +void mod_memcg_lruvec_state(struct lruvec *lruvec,
>> +                enum node_stat_item idx, int val);
>> +
>>   void mem_cgroup_flush_stats(struct mem_cgroup *memcg);
>>   void mem_cgroup_flush_stats_ratelimited(struct mem_cgroup *memcg);
>>   @@ -1452,6 +1455,11 @@ static inline unsigned long 
>> lruvec_page_state_local(const struct lruvec *lruvec,
>>       return node_page_state(lruvec_pgdat(lruvec), idx);
>>   }
>>   +static inline void mod_memcg_lruvec_state(struct lruvec *lruvec,
>> +                      enum node_stat_item idx, int val)
>> +{
>> +}
>> +
>>   static inline void mem_cgroup_flush_stats(struct mem_cgroup *memcg)
>>   {
>>   }
>> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
>> index 519c30b338a8..76d019594b39 100644
>> --- a/mm/hugetlb.c
>> +++ b/mm/hugetlb.c
>> @@ -23,6 +23,7 @@
>>   #include <linux/mmdebug.h>
>>   #include <linux/sched/signal.h>
>>   #include <linux/rmap.h>
>> +#include <linux/rcupdate.h>
>>   #include <linux/string_choices.h>
>>   #include <linux/string_helpers.h>
>>   #include <linux/swap.h>
>> @@ -7378,12 +7379,36 @@ void folio_putback_hugetlb(struct folio *folio)
>>       folio_put(folio);
>>   }
>>   +static void move_hugetlb_lruvec_stat(struct folio *old_folio,
>> +                     struct folio *new_folio)
>> +{
>> +    struct mem_cgroup *memcg;
>> +    long nr_pages = folio_nr_pages(old_folio);
>> +    int old_nid = folio_nid(old_folio);
>> +    int new_nid = folio_nid(new_folio);
>> +
>> +    if (old_nid == new_nid)
>> +        return;
>> +
>> +    guard(rcu)();
>> +
>> +    memcg = folio_memcg(new_folio);
>> +    if (!memcg)
>> +        return;
>> +
>> +    mod_memcg_lruvec_state(mem_cgroup_lruvec(memcg, 
>> NODE_DATA(old_nid)),
>> +                   NR_HUGETLB, -nr_pages);
>
> Why not use mod_lruvec_state? mod_memcg_lruvec_state is an internal
> API for memcg, I don't want it to be exported.

Thank you for the review.

mod_lruvec_state() would update the node counter a second time.  It calls
mod_node_page_state() as well, and the target's node counter is already
updated in alloc_hugetlb_folio_nodemask() (patch 1/2):

     lruvec_stat_mod_folio(folio, NR_HUGETLB, folio_nr_pages(folio));

For an uncharged folio lruvec_stat_mod_folio() only updates the node
counter.  The target folio is not charged to any memcg at that point; its
charge only appears later in mem_cgroup_migrate().  So the node side is
already covered and only the per-memcg attribution has to follow the
charge here.

>
> Thanks.
>
>> + mod_memcg_lruvec_state(mem_cgroup_lruvec(memcg, NODE_DATA(new_nid)),
>> +                   NR_HUGETLB, nr_pages);
>> +}
>> +
>>   void move_hugetlb_state(struct folio *old_folio, struct folio 
>> *new_folio,
>>               enum migrate_reason reason)
>>   {
>>       struct hstate *h = folio_hstate(old_folio);
>>         hugetlb_cgroup_migrate(old_folio, new_folio);
>> +    move_hugetlb_lruvec_stat(old_folio, new_folio);
>>       folio_set_owner_migrate_reason(new_folio, reason);
>>         /*
>> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
>> index 88824f783571..a5335da5d425 100644
>> --- a/mm/memcontrol.c
>> +++ b/mm/memcontrol.c
>> @@ -1015,9 +1015,8 @@ static void __mod_memcg_lruvec_state(struct 
>> mem_cgroup_per_node *pn,
>>       put_cpu();
>>   }
>>   -static void mod_memcg_lruvec_state(struct lruvec *lruvec,
>> -                     enum node_stat_item idx,
>> -                     int val)
>> +void mod_memcg_lruvec_state(struct lruvec *lruvec,
>> +                enum node_stat_item idx, int val)
>>   {
>>       struct pglist_data *pgdat = lruvec_pgdat(lruvec);
>>       struct mem_cgroup_per_node *pn;
>>
>
-- 
Best regards,
Hongfu

Re: [PATCH v2 2/2] mm/memcg: migrate per-node hugetlb lruvec stat together with hugetlb folio

[Muchun Song]

> On Sep 23, 2026, at 11:40, Hongfu Li <hongfu.li@linux.dev> wrote:
> 
> 
> On 9/23/26 10:40 AM, Muchun Song wrote:
>> 
>> 
>> On 2026/9/23 10:05, Hongfu Li wrote:
>>> From: Hongfu Li <lihongfu@kylinos.cn>
>>> 
>>> memory.numa_stat exposes per-node hugetlb counters from per-node lruvec
>>> stats. These stats are accounted against folio_nid(): incremented on
>>> the folio's node when handed to a user, decremented when the folio is
>>> returned to the pool.
>>> 
>>> During hugetlb folio migration, mem_cgroup_migrate() moves the charge
>>> to the new folio and drops the memcg data of the old one, so the free
>>> of the old folio right after the migration skips the memcg per-node
>>> lruvec decrement. The hugetlb count stays attributed to the old node
>>> for the rest of the life of the charge, while the target folio gets no
>>> increment on the new node; its later free decrements a counter that
>>> was never incremented.
>>> 
>>> Migrate the per-node lruvec accounting alongside migration. Global
>>> memcg totals remain balanced because they track resource consumption,
>>> not node placement.
>>> 
>>> Fixes: 05d4532b60e3 ("memcg/hugetlb: add hugeTLB counters to memcg")
>>> Cc: stable@vger.kernel.org
>>> Signed-off-by: Hongfu Li <lihongfu@kylinos.cn>
>>> Tested-by: Joshua Hahn <joshua.hahnjy@gmail.com>
>>> Reviewed-by: Joshua Hahn <joshua.hahnjy@gmail.com>
>>> Reviewed-by: Oscar Salvador <osalvador@suse.de>
>>> ---
>>>   include/linux/memcontrol.h |  8 ++++++++
>>>   mm/hugetlb.c               | 25 +++++++++++++++++++++++++
>>>   mm/memcontrol.c            |  5 ++---
>>>   3 files changed, 35 insertions(+), 3 deletions(-)
>>> 
>>> diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
>>> index a8358f297b65..74110a324f9e 100644
>>> --- a/include/linux/memcontrol.h
>>> +++ b/include/linux/memcontrol.h
>>> @@ -984,6 +984,9 @@ unsigned long lruvec_page_state_monotonic(const struct lruvec *lruvec,
>>>   unsigned long lruvec_page_state_local(const struct lruvec *lruvec,
>>>                         enum node_stat_item idx);
>>>   +void mod_memcg_lruvec_state(struct lruvec *lruvec,
>>> +                enum node_stat_item idx, int val);
>>> +
>>>   void mem_cgroup_flush_stats(struct mem_cgroup *memcg);
>>>   void mem_cgroup_flush_stats_ratelimited(struct mem_cgroup *memcg);
>>>   @@ -1452,6 +1455,11 @@ static inline unsigned long lruvec_page_state_local(const struct lruvec *lruvec,
>>>       return node_page_state(lruvec_pgdat(lruvec), idx);
>>>   }
>>>   +static inline void mod_memcg_lruvec_state(struct lruvec *lruvec,
>>> +                      enum node_stat_item idx, int val)
>>> +{
>>> +}
>>> +
>>>   static inline void mem_cgroup_flush_stats(struct mem_cgroup *memcg)
>>>   {
>>>   }
>>> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
>>> index 519c30b338a8..76d019594b39 100644
>>> --- a/mm/hugetlb.c
>>> +++ b/mm/hugetlb.c
>>> @@ -23,6 +23,7 @@
>>>   #include <linux/mmdebug.h>
>>>   #include <linux/sched/signal.h>
>>>   #include <linux/rmap.h>
>>> +#include <linux/rcupdate.h>
>>>   #include <linux/string_choices.h>
>>>   #include <linux/string_helpers.h>
>>>   #include <linux/swap.h>
>>> @@ -7378,12 +7379,36 @@ void folio_putback_hugetlb(struct folio *folio)
>>>       folio_put(folio);
>>>   }
>>>   +static void move_hugetlb_lruvec_stat(struct folio *old_folio,
>>> +                     struct folio *new_folio)
>>> +{
>>> +    struct mem_cgroup *memcg;
>>> +    long nr_pages = folio_nr_pages(old_folio);
>>> +    int old_nid = folio_nid(old_folio);
>>> +    int new_nid = folio_nid(new_folio);
>>> +
>>> +    if (old_nid == new_nid)
>>> +        return;
>>> +
>>> +    guard(rcu)();
>>> +
>>> +    memcg = folio_memcg(new_folio);
>>> +    if (!memcg)
>>> +        return;
>>> +
>>> +    mod_memcg_lruvec_state(mem_cgroup_lruvec(memcg, NODE_DATA(old_nid)),
>>> +                   NR_HUGETLB, -nr_pages);
>> 
>> Why not use mod_lruvec_state? mod_memcg_lruvec_state is an internal
>> API for memcg, I don't want it to be exported.
> 
> Thank you for the review.
> 
> mod_lruvec_state() would update the node counter a second time.  It calls
> mod_node_page_state() as well, and the target's node counter is already
> updated in alloc_hugetlb_folio_nodemask() (patch 1/2):
> 
>     lruvec_stat_mod_folio(folio, NR_HUGETLB, folio_nr_pages(folio));
> 
> For an uncharged folio lruvec_stat_mod_folio() only updates the node
> counter.  The target folio is not charged to any memcg at that point; its
> charge only appears later in mem_cgroup_migrate().  So the node side is
> already covered and only the per-memcg attribution has to follow the
> charge here.

Looking at the first patch again, I do not think the
overall accounting is incorrect. My concern is where NR_HUGETLB
accounting is performed.

The low-level allocation helpers only take a folio from the
hugetlb pool. They do not have enough context to decide how
the folio should be accounted. That decision should be left
to their callers, once the folio enters the relevant usage
lifecycle.

hugetlb_alloc_folio() is a good example. It first obtains a
folio, then calls mem_cgroup_charge_hugetlb(), and only
afterwards calls lruvec_stat_mod_folio(). At that point,
the folio has the correct memcg and lruvec, so all relevant
counters can be updated together.

After this series, alloc_hugetlb_folio_reserve() and
alloc_hugetlb_folio_nodemask() account NR_HUGETLB before
returning the folio. This makes them unsuitable for callers
that need to charge the folio afterwards. Before the charge,
lruvec_stat_mod_folio() can update only the node counter.
The later charge does not associate that earlier update with
the new memcg.

If a caller accounts again after charging, the node counter is
updated twice. If it does not, the memcg and per-node lruvec
counters remain missing. The caller then needs a memcg-only
correction, creating a special accounting protocol between
the helper and its callers.

Migration exposes the same issue. The target gets node
accounting before it inherits the source memcg. The second patch
must update the lruvec state and then cancel its node changes,
because the target allocation already updated the node counter.
This works, but makes the accounting lifecycle harder to follow.

Would it be cleaner to keep these helpers focused on allocation
and let each caller account at the correct lifecycle point?
Normal allocation can account after a successful memcg charge.
Migration can move the complete lruvec state after it succeeds.
An uncharged path can explicitly update only the node counter
if global accounting is still required.

This would keep policy out of the low-level allocator and make
the accounting lifecycle easier to follow.

Thanks.

> 
>> 
>> Thanks.
>> 
>>> + mod_memcg_lruvec_state(mem_cgroup_lruvec(memcg, NODE_DATA(new_nid)),
>>> +                   NR_HUGETLB, nr_pages);
>>> +}
>>> +
>>>   void move_hugetlb_state(struct folio *old_folio, struct folio *new_folio,
>>>               enum migrate_reason reason)
>>>   {
>>>       struct hstate *h = folio_hstate(old_folio);
>>>         hugetlb_cgroup_migrate(old_folio, new_folio);
>>> +    move_hugetlb_lruvec_stat(old_folio, new_folio);
>>>       folio_set_owner_migrate_reason(new_folio, reason);
>>>         /*
>>> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
>>> index 88824f783571..a5335da5d425 100644
>>> --- a/mm/memcontrol.c
>>> +++ b/mm/memcontrol.c
>>> @@ -1015,9 +1015,8 @@ static void __mod_memcg_lruvec_state(struct mem_cgroup_per_node *pn,
>>>       put_cpu();
>>>   }
>>>   -static void mod_memcg_lruvec_state(struct lruvec *lruvec,
>>> -                     enum node_stat_item idx,
>>> -                     int val)
>>> +void mod_memcg_lruvec_state(struct lruvec *lruvec,
>>> +                enum node_stat_item idx, int val)
>>>   {
>>>       struct pglist_data *pgdat = lruvec_pgdat(lruvec);
>>>       struct mem_cgroup_per_node *pn;
>>> 
>> 
> -- 
> Best regards,
> Hongfu

Re: [PATCH v2 2/2] mm/memcg: migrate per-node hugetlb lruvec stat together with hugetlb folio

[Hongfu Li]

On 9/23/26 3:56 PM, Muchun Song wrote:
>
>> On Sep 23, 2026, at 11:40, Hongfu Li <hongfu.li@linux.dev> wrote:
>>
>>
>> On 9/23/26 10:40 AM, Muchun Song wrote:
>>>
>>> On 2026/9/23 10:05, Hongfu Li wrote:
>>>> From: Hongfu Li <lihongfu@kylinos.cn>
>>>>
>>>> memory.numa_stat exposes per-node hugetlb counters from per-node lruvec
>>>> stats. These stats are accounted against folio_nid(): incremented on
>>>> the folio's node when handed to a user, decremented when the folio is
>>>> returned to the pool.
>>>>
>>>> During hugetlb folio migration, mem_cgroup_migrate() moves the charge
>>>> to the new folio and drops the memcg data of the old one, so the free
>>>> of the old folio right after the migration skips the memcg per-node
>>>> lruvec decrement. The hugetlb count stays attributed to the old node
>>>> for the rest of the life of the charge, while the target folio gets no
>>>> increment on the new node; its later free decrements a counter that
>>>> was never incremented.
>>>>
>>>> Migrate the per-node lruvec accounting alongside migration. Global
>>>> memcg totals remain balanced because they track resource consumption,
>>>> not node placement.
>>>>
>>>> Fixes: 05d4532b60e3 ("memcg/hugetlb: add hugeTLB counters to memcg")
>>>> Cc: stable@vger.kernel.org
>>>> Signed-off-by: Hongfu Li <lihongfu@kylinos.cn>
>>>> Tested-by: Joshua Hahn <joshua.hahnjy@gmail.com>
>>>> Reviewed-by: Joshua Hahn <joshua.hahnjy@gmail.com>
>>>> Reviewed-by: Oscar Salvador <osalvador@suse.de>
>>>> ---
>>>>    include/linux/memcontrol.h |  8 ++++++++
>>>>    mm/hugetlb.c               | 25 +++++++++++++++++++++++++
>>>>    mm/memcontrol.c            |  5 ++---
>>>>    3 files changed, 35 insertions(+), 3 deletions(-)
>>>>
>>>> diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
>>>> index a8358f297b65..74110a324f9e 100644
>>>> --- a/include/linux/memcontrol.h
>>>> +++ b/include/linux/memcontrol.h
>>>> @@ -984,6 +984,9 @@ unsigned long lruvec_page_state_monotonic(const struct lruvec *lruvec,
>>>>    unsigned long lruvec_page_state_local(const struct lruvec *lruvec,
>>>>                          enum node_stat_item idx);
>>>>    +void mod_memcg_lruvec_state(struct lruvec *lruvec,
>>>> +                enum node_stat_item idx, int val);
>>>> +
>>>>    void mem_cgroup_flush_stats(struct mem_cgroup *memcg);
>>>>    void mem_cgroup_flush_stats_ratelimited(struct mem_cgroup *memcg);
>>>>    @@ -1452,6 +1455,11 @@ static inline unsigned long lruvec_page_state_local(const struct lruvec *lruvec,
>>>>        return node_page_state(lruvec_pgdat(lruvec), idx);
>>>>    }
>>>>    +static inline void mod_memcg_lruvec_state(struct lruvec *lruvec,
>>>> +                      enum node_stat_item idx, int val)
>>>> +{
>>>> +}
>>>> +
>>>>    static inline void mem_cgroup_flush_stats(struct mem_cgroup *memcg)
>>>>    {
>>>>    }
>>>> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
>>>> index 519c30b338a8..76d019594b39 100644
>>>> --- a/mm/hugetlb.c
>>>> +++ b/mm/hugetlb.c
>>>> @@ -23,6 +23,7 @@
>>>>    #include <linux/mmdebug.h>
>>>>    #include <linux/sched/signal.h>
>>>>    #include <linux/rmap.h>
>>>> +#include <linux/rcupdate.h>
>>>>    #include <linux/string_choices.h>
>>>>    #include <linux/string_helpers.h>
>>>>    #include <linux/swap.h>
>>>> @@ -7378,12 +7379,36 @@ void folio_putback_hugetlb(struct folio *folio)
>>>>        folio_put(folio);
>>>>    }
>>>>    +static void move_hugetlb_lruvec_stat(struct folio *old_folio,
>>>> +                     struct folio *new_folio)
>>>> +{
>>>> +    struct mem_cgroup *memcg;
>>>> +    long nr_pages = folio_nr_pages(old_folio);
>>>> +    int old_nid = folio_nid(old_folio);
>>>> +    int new_nid = folio_nid(new_folio);
>>>> +
>>>> +    if (old_nid == new_nid)
>>>> +        return;
>>>> +
>>>> +    guard(rcu)();
>>>> +
>>>> +    memcg = folio_memcg(new_folio);
>>>> +    if (!memcg)
>>>> +        return;
>>>> +
>>>> +    mod_memcg_lruvec_state(mem_cgroup_lruvec(memcg, NODE_DATA(old_nid)),
>>>> +                   NR_HUGETLB, -nr_pages);
>>> Why not use mod_lruvec_state? mod_memcg_lruvec_state is an internal
>>> API for memcg, I don't want it to be exported.
>> Thank you for the review.
>>
>> mod_lruvec_state() would update the node counter a second time.  It calls
>> mod_node_page_state() as well, and the target's node counter is already
>> updated in alloc_hugetlb_folio_nodemask() (patch 1/2):
>>
>>      lruvec_stat_mod_folio(folio, NR_HUGETLB, folio_nr_pages(folio));
>>
>> For an uncharged folio lruvec_stat_mod_folio() only updates the node
>> counter.  The target folio is not charged to any memcg at that point; its
>> charge only appears later in mem_cgroup_migrate().  So the node side is
>> already covered and only the per-memcg attribution has to follow the
>> charge here.
> Looking at the first patch again, I do not think the
> overall accounting is incorrect. My concern is where NR_HUGETLB
> accounting is performed.
>
> The low-level allocation helpers only take a folio from the
> hugetlb pool. They do not have enough context to decide how
> the folio should be accounted. That decision should be left
> to their callers, once the folio enters the relevant usage
> lifecycle.
>
> hugetlb_alloc_folio() is a good example. It first obtains a
> folio, then calls mem_cgroup_charge_hugetlb(), and only
> afterwards calls lruvec_stat_mod_folio(). At that point,
> the folio has the correct memcg and lruvec, so all relevant
> counters can be updated together.
>
> After this series, alloc_hugetlb_folio_reserve() and
> alloc_hugetlb_folio_nodemask() account NR_HUGETLB before
> returning the folio. This makes them unsuitable for callers
> that need to charge the folio afterwards. Before the charge,
> lruvec_stat_mod_folio() can update only the node counter.
> The later charge does not associate that earlier update with
> the new memcg.
>
> If a caller accounts again after charging, the node counter is
> updated twice. If it does not, the memcg and per-node lruvec
> counters remain missing. The caller then needs a memcg-only
> correction, creating a special accounting protocol between
> the helper and its callers.
>
> Migration exposes the same issue. The target gets node
> accounting before it inherits the source memcg. The second patch
> must update the lruvec state and then cancel its node changes,
> because the target allocation already updated the node counter.
> This works, but makes the accounting lifecycle harder to follow.
>
> Would it be cleaner to keep these helpers focused on allocation
> and let each caller account at the correct lifecycle point?
> Normal allocation can account after a successful memcg charge.
> Migration can move the complete lruvec state after it succeeds.

Hi Muchun,

Thanks a lot for the detailed explanation.

One thing keeps the target accounting in alloc_hugetlb_folio_nodemask(): the
node counter is paired with the folio's free, and a failed migration 
frees its
target right away (put_new_folio()/folio_put() -> free_huge_folio(), which
always decrements the node counter). If the target were accounted only 
after a
successful migration, every failed migration would decrement a node counter
that was never incremented.

Unless there is a way to let free_huge_folio() tell that the folio comes
from a failed migration, and skip the decrement for it.

> An uncharged path can explicitly update only the node counter
> if global accounting is still required.
>
> This would keep policy out of the low-level allocator and make
> the accounting lifecycle easier to follow.
>
> Thanks.
>
>>> Thanks.
>>>
>>>> + mod_memcg_lruvec_state(mem_cgroup_lruvec(memcg, NODE_DATA(new_nid)),
>>>> +                   NR_HUGETLB, nr_pages);
>>>> +}
>>>> +
>>>>    void move_hugetlb_state(struct folio *old_folio, struct folio *new_folio,
>>>>                enum migrate_reason reason)
>>>>    {
>>>>        struct hstate *h = folio_hstate(old_folio);
>>>>          hugetlb_cgroup_migrate(old_folio, new_folio);
>>>> +    move_hugetlb_lruvec_stat(old_folio, new_folio);
>>>>        folio_set_owner_migrate_reason(new_folio, reason);
>>>>          /*
>>>> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
>>>> index 88824f783571..a5335da5d425 100644
>>>> --- a/mm/memcontrol.c
>>>> +++ b/mm/memcontrol.c
>>>> @@ -1015,9 +1015,8 @@ static void __mod_memcg_lruvec_state(struct mem_cgroup_per_node *pn,
>>>>        put_cpu();
>>>>    }
>>>>    -static void mod_memcg_lruvec_state(struct lruvec *lruvec,
>>>> -                     enum node_stat_item idx,
>>>> -                     int val)
>>>> +void mod_memcg_lruvec_state(struct lruvec *lruvec,
>>>> +                enum node_stat_item idx, int val)
>>>>    {
>>>>        struct pglist_data *pgdat = lruvec_pgdat(lruvec);
>>>>        struct mem_cgroup_per_node *pn;
>>>>
>> -- 
>> Best regards,
>> Hongfu

-- 
Best regards,
Hongfu

Re: [PATCH v2 2/2] mm/memcg: migrate per-node hugetlb lruvec stat together with hugetlb folio

[Muchun Song]

> On Sep 23, 2026, at 17:55, Hongfu Li <hongfu.li@linux.dev> wrote:
> 
> 
> On 9/23/26 3:56 PM, Muchun Song wrote:
>> 
>>> On Sep 23, 2026, at 11:40, Hongfu Li <hongfu.li@linux.dev> wrote:
>>> 
>>> 
>>> On 9/23/26 10:40 AM, Muchun Song wrote:
>>>> 
>>>> On 2026/9/23 10:05, Hongfu Li wrote:
>>>>> From: Hongfu Li <lihongfu@kylinos.cn>
>>>>> 
>>>>> memory.numa_stat exposes per-node hugetlb counters from per-node lruvec
>>>>> stats. These stats are accounted against folio_nid(): incremented on
>>>>> the folio's node when handed to a user, decremented when the folio is
>>>>> returned to the pool.
>>>>> 
>>>>> During hugetlb folio migration, mem_cgroup_migrate() moves the charge
>>>>> to the new folio and drops the memcg data of the old one, so the free
>>>>> of the old folio right after the migration skips the memcg per-node
>>>>> lruvec decrement. The hugetlb count stays attributed to the old node
>>>>> for the rest of the life of the charge, while the target folio gets no
>>>>> increment on the new node; its later free decrements a counter that
>>>>> was never incremented.
>>>>> 
>>>>> Migrate the per-node lruvec accounting alongside migration. Global
>>>>> memcg totals remain balanced because they track resource consumption,
>>>>> not node placement.
>>>>> 
>>>>> Fixes: 05d4532b60e3 ("memcg/hugetlb: add hugeTLB counters to memcg")
>>>>> Cc: stable@vger.kernel.org
>>>>> Signed-off-by: Hongfu Li <lihongfu@kylinos.cn>
>>>>> Tested-by: Joshua Hahn <joshua.hahnjy@gmail.com>
>>>>> Reviewed-by: Joshua Hahn <joshua.hahnjy@gmail.com>
>>>>> Reviewed-by: Oscar Salvador <osalvador@suse.de>
>>>>> ---
>>>>>   include/linux/memcontrol.h |  8 ++++++++
>>>>>   mm/hugetlb.c               | 25 +++++++++++++++++++++++++
>>>>>   mm/memcontrol.c            |  5 ++---
>>>>>   3 files changed, 35 insertions(+), 3 deletions(-)
>>>>> 
>>>>> diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
>>>>> index a8358f297b65..74110a324f9e 100644
>>>>> --- a/include/linux/memcontrol.h
>>>>> +++ b/include/linux/memcontrol.h
>>>>> @@ -984,6 +984,9 @@ unsigned long lruvec_page_state_monotonic(const struct lruvec *lruvec,
>>>>>   unsigned long lruvec_page_state_local(const struct lruvec *lruvec,
>>>>>                         enum node_stat_item idx);
>>>>>   +void mod_memcg_lruvec_state(struct lruvec *lruvec,
>>>>> +                enum node_stat_item idx, int val);
>>>>> +
>>>>>   void mem_cgroup_flush_stats(struct mem_cgroup *memcg);
>>>>>   void mem_cgroup_flush_stats_ratelimited(struct mem_cgroup *memcg);
>>>>>   @@ -1452,6 +1455,11 @@ static inline unsigned long lruvec_page_state_local(const struct lruvec *lruvec,
>>>>>       return node_page_state(lruvec_pgdat(lruvec), idx);
>>>>>   }
>>>>>   +static inline void mod_memcg_lruvec_state(struct lruvec *lruvec,
>>>>> +                      enum node_stat_item idx, int val)
>>>>> +{
>>>>> +}
>>>>> +
>>>>>   static inline void mem_cgroup_flush_stats(struct mem_cgroup *memcg)
>>>>>   {
>>>>>   }
>>>>> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
>>>>> index 519c30b338a8..76d019594b39 100644
>>>>> --- a/mm/hugetlb.c
>>>>> +++ b/mm/hugetlb.c
>>>>> @@ -23,6 +23,7 @@
>>>>>   #include <linux/mmdebug.h>
>>>>>   #include <linux/sched/signal.h>
>>>>>   #include <linux/rmap.h>
>>>>> +#include <linux/rcupdate.h>
>>>>>   #include <linux/string_choices.h>
>>>>>   #include <linux/string_helpers.h>
>>>>>   #include <linux/swap.h>
>>>>> @@ -7378,12 +7379,36 @@ void folio_putback_hugetlb(struct folio *folio)
>>>>>       folio_put(folio);
>>>>>   }
>>>>>   +static void move_hugetlb_lruvec_stat(struct folio *old_folio,
>>>>> +                     struct folio *new_folio)
>>>>> +{
>>>>> +    struct mem_cgroup *memcg;
>>>>> +    long nr_pages = folio_nr_pages(old_folio);
>>>>> +    int old_nid = folio_nid(old_folio);
>>>>> +    int new_nid = folio_nid(new_folio);
>>>>> +
>>>>> +    if (old_nid == new_nid)
>>>>> +        return;
>>>>> +
>>>>> +    guard(rcu)();
>>>>> +
>>>>> +    memcg = folio_memcg(new_folio);
>>>>> +    if (!memcg)
>>>>> +        return;
>>>>> +
>>>>> +    mod_memcg_lruvec_state(mem_cgroup_lruvec(memcg, NODE_DATA(old_nid)),
>>>>> +                   NR_HUGETLB, -nr_pages);
>>>> Why not use mod_lruvec_state? mod_memcg_lruvec_state is an internal
>>>> API for memcg, I don't want it to be exported.
>>> Thank you for the review.
>>> 
>>> mod_lruvec_state() would update the node counter a second time.  It calls
>>> mod_node_page_state() as well, and the target's node counter is already
>>> updated in alloc_hugetlb_folio_nodemask() (patch 1/2):
>>> 
>>>     lruvec_stat_mod_folio(folio, NR_HUGETLB, folio_nr_pages(folio));
>>> 
>>> For an uncharged folio lruvec_stat_mod_folio() only updates the node
>>> counter.  The target folio is not charged to any memcg at that point; its
>>> charge only appears later in mem_cgroup_migrate().  So the node side is
>>> already covered and only the per-memcg attribution has to follow the
>>> charge here.
>> Looking at the first patch again, I do not think the
>> overall accounting is incorrect. My concern is where NR_HUGETLB
>> accounting is performed.
>> 
>> The low-level allocation helpers only take a folio from the
>> hugetlb pool. They do not have enough context to decide how
>> the folio should be accounted. That decision should be left
>> to their callers, once the folio enters the relevant usage
>> lifecycle.
>> 
>> hugetlb_alloc_folio() is a good example. It first obtains a
>> folio, then calls mem_cgroup_charge_hugetlb(), and only
>> afterwards calls lruvec_stat_mod_folio(). At that point,
>> the folio has the correct memcg and lruvec, so all relevant
>> counters can be updated together.
>> 
>> After this series, alloc_hugetlb_folio_reserve() and
>> alloc_hugetlb_folio_nodemask() account NR_HUGETLB before
>> returning the folio. This makes them unsuitable for callers
>> that need to charge the folio afterwards. Before the charge,
>> lruvec_stat_mod_folio() can update only the node counter.
>> The later charge does not associate that earlier update with
>> the new memcg.
>> 
>> If a caller accounts again after charging, the node counter is
>> updated twice. If it does not, the memcg and per-node lruvec
>> counters remain missing. The caller then needs a memcg-only
>> correction, creating a special accounting protocol between
>> the helper and its callers.
>> 
>> Migration exposes the same issue. The target gets node
>> accounting before it inherits the source memcg. The second patch
>> must update the lruvec state and then cancel its node changes,
>> because the target allocation already updated the node counter.
>> This works, but makes the accounting lifecycle harder to follow.
>> 
>> Would it be cleaner to keep these helpers focused on allocation
>> and let each caller account at the correct lifecycle point?
>> Normal allocation can account after a successful memcg charge.
>> Migration can move the complete lruvec state after it succeeds.
> 
> Hi Muchun,

Hi,

> 
> Thanks a lot for the detailed explanation.
> 
> One thing keeps the target accounting in alloc_hugetlb_folio_nodemask(): the
> node counter is paired with the folio's free, and a failed migration frees its
> target right away (put_new_folio()/folio_put() -> free_huge_folio(), which
> always decrements the node counter). If the target were accounted only after a
> successful migration, every failed migration would decrement a node counter
> that was never incremented.

You're right, and HugeTLB is really tricky. I think HugeTLB might need some cleanup
in the future to better handle the accounting here. As a bug fix, the current
changes look good to me.

Acked-by: Muchun Song <muchun.song@linux.dev>

A future cleanup, might be to have all the underlying HugeTLB allocation functions
update only the global node statistics (use mod_node_page_state instead	of
lruvec_stat_mod_folio). Its caller, if it needs to charge memory
cgroup, can then call mod_memcg_lruvec_state that updates only the memory cgroup
statistics.

Thanks.

> 
> Unless there is a way to let free_huge_folio() tell that the folio comes
> from a failed migration, and skip the decrement for it.
> 
>> An uncharged path can explicitly update only the node counter
>> if global accounting is still required.
>> 
>> This would keep policy out of the low-level allocator and make
>> the accounting lifecycle easier to follow.
>> 
>> Thanks.
>> 
>>>> Thanks.
>>>> 
>>>>> + mod_memcg_lruvec_state(mem_cgroup_lruvec(memcg, NODE_DATA(new_nid)),
>>>>> +                   NR_HUGETLB, nr_pages);
>>>>> +}
>>>>> +
>>>>>   void move_hugetlb_state(struct folio *old_folio, struct folio *new_folio,
>>>>>               enum migrate_reason reason)
>>>>>   {
>>>>>       struct hstate *h = folio_hstate(old_folio);
>>>>>         hugetlb_cgroup_migrate(old_folio, new_folio);
>>>>> +    move_hugetlb_lruvec_stat(old_folio, new_folio);
>>>>>       folio_set_owner_migrate_reason(new_folio, reason);
>>>>>         /*
>>>>> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
>>>>> index 88824f783571..a5335da5d425 100644
>>>>> --- a/mm/memcontrol.c
>>>>> +++ b/mm/memcontrol.c
>>>>> @@ -1015,9 +1015,8 @@ static void __mod_memcg_lruvec_state(struct mem_cgroup_per_node *pn,
>>>>>       put_cpu();
>>>>>   }
>>>>>   -static void mod_memcg_lruvec_state(struct lruvec *lruvec,
>>>>> -                     enum node_stat_item idx,
>>>>> -                     int val)
>>>>> +void mod_memcg_lruvec_state(struct lruvec *lruvec,
>>>>> +                enum node_stat_item idx, int val)
>>>>>   {
>>>>>       struct pglist_data *pgdat = lruvec_pgdat(lruvec);
>>>>>       struct mem_cgroup_per_node *pn;
>>>>> 
>>> -- 
>>> Best regards,
>>> Hongfu
> 
> -- 
> Best regards,
> Hongfu