[PATCH v2 0/2] mm/hugetlb: fix surplus accounting and availability checks during demotion

[PATCH v2 0/2] mm/hugetlb: fix surplus accounting and availability checks during demotion

[Longlong Xia]

From: Longlong Xia <xialonglong@kylinos.cn>

This is v2 of the two-patch series fixing surplus accounting and
availability checks in the hugetlb demote path.

Patch 1 fixes source hstate accounting when the free folio selected for
demotion accounts for a surplus page. Patch 2 prevents demotion from
removing free huge pages that back reservations.

Both fixes were tested with x86_64 QEMU guests. The commands below use:

  hstate=/sys/kernel/mm/hugepages/hugepages-1048576kB

Patch 1: surplus accounting

A vmemmap restoration failure is difficult to trigger deterministically.
For this test only, add a one-shot fault injection that makes the first
attempt to restore the vmemmap of an optimized 1 GiB folio fail:

  /* TEST ONLY: fail the first optimized 1G folio restore. */
  static atomic_t fail_next_1g_restore = ATOMIC_INIT(1);

  /* In __hugetlb_vmemmap_restore_folio(). */
  if (huge_page_size(h) == SZ_1G &&
      atomic_cmpxchg(&fail_next_1g_restore, 1, 0) == 1) {
          pr_info("TEST ONLY: forcing one 1G vmemmap "
                  "restore failure\n");
          return -ENOMEM;
  }

The injection does not modify the demotion or accounting code. It is
one-shot so that the later restore performed during demotion can succeed.

1. Boot QEMU with:

     hugepagesz=1G hugepages=0 hugetlb_cma=1G
     hugetlb_free_vmemmap=on

2. Enable overcommit:

     echo 1 > "$hstate/nr_overcommit_hugepages"

3. Allocate one 1 GiB huge page:

     nr=1 surplus=1 free=0 resv=0

4. Unmap it. The forced restoration failure leaves the folio on the
   freelist while it is still accounted as surplus:

     nr=1 surplus=1 free=1 resv=0

5. Demote one page:

     echo 1 > "$hstate/demote"

   Before this fix:

     nr=0 surplus=1 free=0 resv=0
     surplus > nr

   After this fix:

     nr=0 surplus=0 free=0 resv=0

Patch 2: cap demotion

This reproducer requires no kernel instrumentation.

1. Boot QEMU with:

     hugepagesz=1G hugepages=2
     nr=2 surplus=0 free=2 resv=0

2. Reserve one 1 GiB huge page with an untouched hugetlbfs mapping:

     nr=2 surplus=0 free=2 resv=1

3. Request demotion of two pages:

     echo 2 > "$hstate/demote"

   Before this fix:

     nr=0 surplus=0 free=0 resv=1
     resv > free

   After this fix:

     nr=1 surplus=0 free=1 resv=1
     resv == free

4. Touch the reserved page and let the process exit.

   Before this fix, the access fails with SIGBUS and leaves:

     nr=0 surplus=0 free=0 resv=0

   After this fix, the access succeeds and leaves:

     nr=1 surplus=0 free=1 resv=0

Changes in v2:
- Add reproducer details and test results suggested by Andrew Morton.

Link: https://lore.kernel.org/all/20260823034307.1072415-1-xialonglong2025@163.com/

Longlong Xia (2):
  mm/hugetlb: preserve source surplus accounting during demotion
  mm/hugetlb: cap demotion at currently available free pages

 mm/hugetlb.c       | 57 ++++++++++++++++++++++++++++++++++++++++++----
 mm/hugetlb_sysfs.c | 10 ++++----
 2 files changed, 57 insertions(+), 10 deletions(-)


base-commit: a4ff2be345d0abc943da8dd8da98151843b750dc
-- 
2.43.0

[PATCH v2 1/2] mm/hugetlb: preserve source surplus accounting during demotion

[Longlong Xia]

From: Longlong Xia <xialonglong@kylinos.cn>

demote_pool_huge_page() currently removes every source folio as a
persistent folio. A free folio can instead account for one of the source
hstate's surplus pages, for example after a vmemmap restoration failure.
Removing such a folio without adjusting surplus_huge_pages makes the
persistent count underflow, and later subtracting it from max_huge_pages
can underflow that counter as well.

Classify selected folios against the node's surplus count while holding
hugetlb_lock, and preserve that classification on rollback. Track the
number of successfully demoted persistent folios separately so only those
folios reduce the source max_huge_pages target. All successfully demoted
folios still increase the destination target because the new destination
folios are added as persistent pages.

Testing:
Tested on an x86_64 QEMU guest booted with:

  hugepagesz=1G hugepages=0 hugetlb_cma=1G
  hugetlb_free_vmemmap=on

For testing only, add a one-shot fault injection that makes the first call
to __hugetlb_vmemmap_restore_folio() for an optimized 1 GiB folio return
-ENOMEM. Set nr_overcommit_hugepages to 1, then allocate one 1 GiB huge
page:

  nr=1 surplus=1 free=0 resv=0

Unmap it. The failed restoration leaves the folio on the freelist while it
is still accounted as surplus:

  nr=1 surplus=1 free=1 resv=0

Demote one page. Before this fix, the result is:

  nr=0 surplus=1 free=0 resv=0

After this fix, the result is:

  nr=0 surplus=0 free=0 resv=0

The fault injection is one-shot, so the restore performed during demotion
can succeed.

Fixes: 8531fc6f52f5 ("hugetlb: add hugetlb demote page support")
Assisted-by: Codex:gpt-5.6-sol
Signed-off-by: Longlong Xia <xialonglong@kylinos.cn>
---
 mm/hugetlb.c | 35 +++++++++++++++++++++++++++++++----
 1 file changed, 31 insertions(+), 4 deletions(-)

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index ed26105b84de..640df58be4e5 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -3983,6 +3983,7 @@ long demote_pool_huge_page(struct hstate *src, nodemask_t *nodes_allowed,
 	struct hstate *dst;
 	long rc = 0;
 	long nr_demoted = 0;
+	long nr_persistent = 0;
 
 	lockdep_assert_held(&hugetlb_lock);
 
@@ -3995,22 +3996,40 @@ long demote_pool_huge_page(struct hstate *src, nodemask_t *nodes_allowed,
 
 	for_each_node_mask_to_free(src, nr_nodes, node, nodes_allowed) {
 		LIST_HEAD(list);
+		LIST_HEAD(surplus_list);
 		struct folio *folio, *next;
 
 		list_for_each_entry_safe(folio, next, &src->hugepage_freelists[node], lru) {
+			bool adjust_surplus;
+
 			if (folio_test_hwpoison(folio))
 				continue;
 
-			remove_hugetlb_folio(src, folio, false);
-			list_add(&folio->lru, &list);
+			/* Surplus accounting is maintained per node, not per folio. */
+			adjust_surplus = src->surplus_huge_pages_node[node] > 0;
+			remove_hugetlb_folio(src, folio, adjust_surplus);
+			list_add(&folio->lru, adjust_surplus ? &surplus_list : &list);
+			if (!adjust_surplus)
+				nr_persistent++;
 
 			if (++nr_demoted == nr_to_demote)
 				break;
 		}
 
+		if (list_empty(&list) && list_empty(&surplus_list))
+			continue;
+
 		spin_unlock_irq(&hugetlb_lock);
 
-		rc = demote_free_hugetlb_folios(src, dst, &list);
+		if (!list_empty(&list))
+			rc = demote_free_hugetlb_folios(src, dst, &list);
+		if (!list_empty(&surplus_list)) {
+			long tmp_rc;
+
+			tmp_rc = demote_free_hugetlb_folios(src, dst, &surplus_list);
+			if (rc >= 0)
+				rc = tmp_rc;
+		}
 
 		spin_lock_irq(&hugetlb_lock);
 
@@ -4018,6 +4037,14 @@ long demote_pool_huge_page(struct hstate *src, nodemask_t *nodes_allowed,
 			list_del(&folio->lru);
 			add_hugetlb_folio(src, folio, false);
 
+			nr_demoted--;
+			nr_persistent--;
+		}
+
+		list_for_each_entry_safe(folio, next, &surplus_list, lru) {
+			list_del(&folio->lru);
+			add_hugetlb_folio(src, folio, true);
+
 			nr_demoted--;
 		}
 
@@ -4029,7 +4056,7 @@ long demote_pool_huge_page(struct hstate *src, nodemask_t *nodes_allowed,
 	 * Not absolutely necessary, but for consistency update max_huge_pages
 	 * based on pool changes for the demoted page.
 	 */
-	src->max_huge_pages -= nr_demoted;
+	src->max_huge_pages -= nr_persistent;
 	dst->max_huge_pages += nr_demoted << (huge_page_order(src) - huge_page_order(dst));
 
 	if (rc < 0)
-- 
2.43.0

[PATCH v2 2/2] mm/hugetlb: cap demotion at currently available free pages

[Longlong Xia]

From: Longlong Xia <xialonglong@kylinos.cn>

Demotion must not remove free huge pages that back existing reservations.
The sysfs path checks whether any page is available, but passes the entire
request to demote_pool_huge_page(). For example, with two free pages and
one reservation, a request for two pages removes both and leaves the
reservation without a backing page.

Cap the sysfs request by both global availability and the selected node's
free pages. Recheck global availability in demote_pool_huge_page() before
each node batch because that function drops hugetlb_lock while restoring
vmemmap and reservations can change before the next batch.

Testing:
Tested on an x86_64 QEMU guest booted with:

  hugepagesz=1G hugepages=2

Reserve one 1 GiB huge page with an untouched hugetlbfs mapping:

  nr=2 surplus=0 free=2 resv=1

Request demotion of two pages. Before this fix, both free pages are
demoted:

  nr=0 surplus=0 free=0 resv=1

Touching the reserved mapping then fails with SIGBUS. After this fix, the
request is capped at the single available page:

  nr=1 surplus=0 free=1 resv=1

Touching the reserved mapping succeeds. After the process exits, the
counters are:

  nr=1 surplus=0 free=1 resv=0

Fixes: c0f398c3b2cf ("mm/hugetlb_vmemmap: batch HVO work when demoting")
Assisted-by: Codex:gpt-5.6-sol
Signed-off-by: Longlong Xia <xialonglong@kylinos.cn>
---
 mm/hugetlb.c       | 22 +++++++++++++++++++++-
 mm/hugetlb_sysfs.c | 10 +++++-----
 2 files changed, 26 insertions(+), 6 deletions(-)

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 640df58be4e5..ae26d400ad31 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -3998,6 +3998,26 @@ long demote_pool_huge_page(struct hstate *src, nodemask_t *nodes_allowed,
 		LIST_HEAD(list);
 		LIST_HEAD(surplus_list);
 		struct folio *folio, *next;
+		unsigned long nr_available, nr_target;
+
+		/*
+		 * Re-check available each node batch: the previous
+		 * batch released hugetlb_lock for vmemmap restore/split,
+		 * and a new reservation could have been added in that
+		 * window, shrinking the budget.  available is global
+		 * (resv is not per-node), so 0 means no node can
+		 * contribute -- stop the whole scan.
+		 */
+		nr_available = available_huge_pages(src);
+		if (!nr_available)
+			break;
+
+		/*
+		 * Cap this batch at the current budget; expressed as a
+		 * cumulative stop point because nr_demoted is running.
+		 */
+		nr_target = nr_demoted + min_t(unsigned long,
+				nr_to_demote - nr_demoted, nr_available);
 
 		list_for_each_entry_safe(folio, next, &src->hugepage_freelists[node], lru) {
 			bool adjust_surplus;
@@ -4012,7 +4032,7 @@ long demote_pool_huge_page(struct hstate *src, nodemask_t *nodes_allowed,
 			if (!adjust_surplus)
 				nr_persistent++;
 
-			if (++nr_demoted == nr_to_demote)
+			if (++nr_demoted == nr_target)
 				break;
 		}
 
diff --git a/mm/hugetlb_sysfs.c b/mm/hugetlb_sysfs.c
index 79ece91406bf..326a54b4d991 100644
--- a/mm/hugetlb_sysfs.c
+++ b/mm/hugetlb_sysfs.c
@@ -211,15 +211,15 @@ static ssize_t demote_store(struct kobject *kobj,
 		 * Check for available pages to demote each time thorough the
 		 * loop as demote_pool_huge_page will drop hugetlb_lock.
 		 */
+		nr_available = h->free_huge_pages - h->resv_huge_pages;
 		if (nid != NUMA_NO_NODE)
-			nr_available = h->free_huge_pages_node[nid];
-		else
-			nr_available = h->free_huge_pages;
-		nr_available -= h->resv_huge_pages;
+			nr_available = min(nr_available,
+					   h->free_huge_pages_node[nid]);
 		if (!nr_available)
 			break;
 
-		rc = demote_pool_huge_page(h, n_mask, nr_demote);
+		rc = demote_pool_huge_page(h, n_mask,
+					   min(nr_demote, nr_available));
 		if (rc < 0) {
 			err = rc;
 			break;
-- 
2.43.0

Re: [PATCH v2 0/2] mm/hugetlb: fix surplus accounting and availability checks during demotion

[Andrew Morton]

On Mon, 31 Aug 2026 21:35:17 +0800 Longlong Xia <xialonglong2025@163.com> wrote:

> From: Longlong Xia <xialonglong@kylinos.cn>
> 
> This is v2 of the two-patch series fixing surplus accounting and
> availability checks in the hugetlb demote path.
> 
> Patch 1 fixes source hstate accounting when the free folio selected for
> demotion accounts for a surplus page. Patch 2 prevents demotion from
> removing free huge pages that back reservations.
> 
> Both fixes were tested with x86_64 QEMU guests. The commands below use:
> 
>   hstate=/sys/kernel/mm/hugepages/hugepages-1048576kB
> 
> Patch 1: surplus accounting
> 
> A vmemmap restoration failure is difficult to trigger deterministically.
> For this test only, add a one-shot fault injection that makes the first
> attempt to restore the vmemmap of an optimized 1 GiB folio fail:

OK, so hard to hit from userspace but not impossible.

> Patch 2: cap demotion
> 
> This reproducer requires no kernel instrumentation.
> 
> ...
> 
>    Before this fix, the access fails with SIGBUS and leaves:
> 
>      nr=0 surplus=0 free=0 resv=0
> 
>    After this fix, the access succeeds and leaves:
> 
>      nr=1 surplus=0 free=1 resv=0

OK, that's bad behavior.

I asked my friendly neighborhood LLM and was told

: I'd phrase it like this, keeping the stable justification concrete
: without overstating the exact failure mode:
: 
: On architectures where gigantic HugeTLB pages cannot be allocated or
: freed at runtime, memory hot-remove or hwpoison can attempt to dissolve
: a boot-allocated gigantic page.  The lower-level removal helpers
: silently reject such pages, but dissolve_free_hugetlb_folio() continues
: and can free the folio while it is still on the HugeTLB free list.  If
: vmemmap restoration fails, the rollback can additionally add the
: already-listed folio to the free list again.
: 
: This corrupts the HugeTLB free-page state.  The corruption can persist
: beyond the operation which triggered it and be encountered by a later
: HugeTLB allocation, potentially resulting in kernel warnings, crashes
: or memory corruption.  Avoid the corruption by rejecting gigantic pages
: without runtime support before modifying the HugeTLB pool.

So, with a statement like that, I'm thinking that we should backport
these fixes (or something else that fixes these issues!)

Anyway, these are matters for maintainers to consider, please.  For now
I'll get these patches under test.