mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Ackerley Tng via B4 Relay <devnull+ackerleytng.google.com@kernel.org>
To: Muchun Song <muchun.song@linux.dev>,
	Oscar Salvador <osalvador@suse.de>,
	 David Hildenbrand <david@kernel.org>,
	Joshua Hahn <joshua.hahnjy@gmail.com>,
	 Shakeel Butt <shakeel.butt@linux.dev>,
	Nhat Pham <nphamcs@gmail.com>,
	 Andrew Morton <akpm@linux-foundation.org>,
	Peter Xu <peterx@redhat.com>,  Wupeng Ma <mawupeng1@huawei.com>,
	fvdl@google.com, rientjes@google.com,  jthoughton@google.com,
	Mike Kravetz <mike.kravetz@oracle.com>,
	 Johannes Weiner <hannes@cmpxchg.org>,
	Michal Hocko <mhocko@kernel.org>,
	 Roman Gushchin <roman.gushchin@linux.dev>,
	Lorenzo Stoakes <ljs@kernel.org>,
	 "Liam R. Howlett" <liam@infradead.org>,
	Vlastimil Babka <vbabka@kernel.org>,
	 Mike Rapoport <rppt@kernel.org>,
	Suren Baghdasaryan <surenb@google.com>,
	 Michal Hocko <mhocko@suse.com>, Jonathan Corbet <corbet@lwn.net>,
	 Shuah Khan <skhan@linuxfoundation.org>,
	Alex Shi <alexs@kernel.org>,  Yanteng Si <si.yanteng@linux.dev>,
	Dongliang Mu <dzm91@hust.edu.cn>,
	 Hongxiang Lou <louhongxiang@huawei.com>,
	Miaohe Lin <linmiaohe@huawei.com>
Cc: vannapurve@google.com, erdemaktas@google.com, linux-mm@kvack.org,
	 linux-kernel@vger.kernel.org, cgroups@vger.kernel.org,
	 linux-doc@vger.kernel.org, Ackerley Tng <ackerleytng@google.com>,
	 stable@vger.kernel.org
Subject: [PATCH v4 05/16] mm: hugetlb: Fix subpool usage leak on allocation failure
Date: Wed, 22 Jul 2026 16:41:13 -0700	[thread overview]
Message-ID: <20260722-hugetlb-alloc-failure-fixes-v4-5-88e8b81970dc@google.com> (raw)
In-Reply-To: <20260722-hugetlb-alloc-failure-fixes-v4-0-88e8b81970dc@google.com>

From: Ackerley Tng <ackerleytng@google.com>

When alloc_hugetlb_folio() fails early (e.g. buddy allocation failure or
hugetlb cgroup charging failure) and gbl_chg == 1 (meaning a reservation
was not used, but a global page was allocated instead), the subpool page
acquired via hugepage_subpool_get_pages() must still be returned.

Currently, the error path out_subpool_put: only calls
hugepage_subpool_put_pages() if !gbl_chg is true. If gbl_chg is 1, it
skips it, permanently leaking the subpool's used_hpages counter.

With the earlier patch to always track used_hpages in the subpool, always
call hugepage_subpool_put_pages() if map_chg is true to consistently
restore the page to the subpool. Condition on map_chg because map_chg also
gates hugepage_subpool_get_pages().

Opportunistically rename gbl_chg to gbl_resv_get, which is the number of
global pages needed (because neither resv_map nor subpool reservations
could be used).

If the number of global pages needed is 0, this allocation uses a
reservation somewhere, hence proceed to consume a reservation by
decrementing h->resv_huge_pages. If the number of global pages needed is 1,
reservations are neither created nor consumed.

Also rename gbl_reserve to gbl_resv_put, which is the number of
pages the subpool could not absorb into its reservations.

Adjust global reservations using hugetlb_acct_memory() with the difference
between gbl_resv_get and gbl_resv_put to take care of possible races where
another thread might have performed get or put with the same subpool, hence
perhaps requiring updates to global reservation counts. (If gbl_resv_get ==
0 because a resv_map reservation was used, map_chg == 0 so the entire
subpool returning is skipped - still correct.)

Fixes: a833a693a490e ("mm: hugetlb: fix incorrect fallback for subpool")
Cc: stable@vger.kernel.org
Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 mm/hugetlb.c | 23 ++++++++++-------------
 1 file changed, 10 insertions(+), 13 deletions(-)

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 5ee1bc5c00bfe..879e4640dc50d 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -2872,7 +2872,7 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma,
 	struct hugepage_subpool *spool = subpool_vma(vma);
 	struct hstate *h = hstate_vma(vma);
 	struct folio *folio;
-	long retval, gbl_chg, gbl_reserve;
+	long retval, gbl_resv_get;
 	map_chg_state map_chg;
 	int ret, idx;
 	struct hugetlb_cgroup *h_cg = NULL;
@@ -2912,15 +2912,15 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma,
 	 * Or if it can get one from the pool reservation directly.
 	 */
 	if (map_chg) {
-		gbl_chg = hugepage_subpool_get_pages(spool, 1);
-		if (gbl_chg < 0)
+		gbl_resv_get = hugepage_subpool_get_pages(spool, 1);
+		if (gbl_resv_get < 0)
 			goto out_end_reservation;
 	} else {
 		/*
 		 * If we have the vma reservation ready, no need for extra
 		 * global reservation.
 		 */
-		gbl_chg = 0;
+		gbl_resv_get = 0;
 	}
 
 	/*
@@ -2949,7 +2949,7 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma,
 	 * from the global free pool (global change).  gbl_chg == 0 indicates
 	 * a reservation exists for the allocation.
 	 */
-	folio = dequeue_hugetlb_folio_vma(h, vma, addr, gbl_chg);
+	folio = dequeue_hugetlb_folio_vma(h, vma, addr, gbl_resv_get);
 	if (!folio) {
 		spin_unlock_irq(&hugetlb_lock);
 		folio = alloc_buddy_hugetlb_folio_with_mpol(h, vma, addr);
@@ -2965,7 +2965,7 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma,
 	 * Either dequeued or buddy-allocated folio needs to add special
 	 * mark to the folio when it consumes a global reservation.
 	 */
-	if (!gbl_chg) {
+	if (!gbl_resv_get) {
 		folio_set_hugetlb_restore_reserve(folio);
 		h->resv_huge_pages--;
 	}
@@ -3022,13 +3022,10 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma,
 		hugetlb_cgroup_uncharge_cgroup_rsvd(idx, pages_per_huge_page(h),
 						    h_cg_rsvd);
 out_subpool_put:
-	/*
-	 * put page to subpool iff the quota of subpool's rsv_hpages is used
-	 * during hugepage_subpool_get_pages.
-	 */
-	if (map_chg && !gbl_chg) {
-		gbl_reserve = hugepage_subpool_put_pages(spool, 1);
-		hugetlb_acct_memory(h, -gbl_reserve);
+	if (map_chg) {
+		long gbl_resv_put = hugepage_subpool_put_pages(spool, 1);
+
+		hugetlb_acct_memory(h, gbl_resv_get - gbl_resv_put);
 	}
 
 

-- 
2.55.0.229.g6434b31f56-goog



  parent reply	other threads:[~2026-07-22 23:41 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-07-22 23:41 [PATCH v4 00/16] Fix bugs on HugeTLB folio allocation failure paths Ackerley Tng via B4 Relay
2026-07-22 23:41 ` [PATCH v4 01/16] mm: hugetlb: Track used_hpages when getting/putting pages from subpool Ackerley Tng via B4 Relay
2026-07-29 17:14   ` Ackerley Tng
2026-07-22 23:41 ` [PATCH v4 02/16] mm: hugetlb: Return -ENOSPC on memcg charge failure Ackerley Tng via B4 Relay
2026-07-22 23:41 ` [PATCH v4 03/16] mm: hugetlb: Use try-commit-cancel protocol for memcg charge of folios Ackerley Tng via B4 Relay
2026-07-22 23:41 ` [PATCH v4 04/16] mm: hugetlb: Remove unused mem_cgroup_charge_hugetlb function Ackerley Tng via B4 Relay
2026-07-22 23:41 ` Ackerley Tng via B4 Relay [this message]
2026-07-22 23:41 ` [PATCH v4 06/16] mm: hugetlb: Rename local variables for clarity in hugetlb_reserve_pages() Ackerley Tng via B4 Relay
2026-07-22 23:41 ` [PATCH v4 07/16] mm: hugetlb: Fix Use-After-Free in unlock_or_release_subpool() Ackerley Tng via B4 Relay
2026-07-22 23:41 ` [PATCH v4 08/16] fs: hugetlbfs: Fix global reservation leak in hugetlbfs_fill_super() Ackerley Tng via B4 Relay
2026-07-22 23:41 ` [PATCH v4 09/16] WIP: mm: hugetlb: Move subpool functions to hugetlb_subpool.c Ackerley Tng via B4 Relay
2026-07-22 23:41 ` [PATCH v4 10/16] WIP: fs: hugetlbfs: Refactor subpool getters and integrate with hugetlb_subpool API Ackerley Tng via B4 Relay
2026-07-22 23:41 ` [PATCH v4 11/16] WIP: mm: hugetlb: Make struct hugepage_subpool private to hugetlb_subpool.c Ackerley Tng via B4 Relay
2026-07-22 23:41 ` [PATCH v4 12/16] WIP: tools: testing: Add unit tests for HugeTLB subpool functions Ackerley Tng via B4 Relay
2026-07-22 23:41 ` [PATCH v4 13/16] WIP: Reproducer for allocation failure due to cgroup v2 memory limits Ackerley Tng via B4 Relay
2026-07-22 23:41 ` [PATCH v4 14/16] WIP: Reproducer for subpool usage leak Ackerley Tng via B4 Relay
2026-07-22 23:41 ` [PATCH v4 15/16] WIP: Reproducer for false restoration on shared HugeTLB mappings Ackerley Tng via B4 Relay
2026-07-22 23:41 ` [PATCH v4 16/16] WIP: Reproducer for out_put_pages subpool reserve leakage Ackerley Tng via B4 Relay

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260722-hugetlb-alloc-failure-fixes-v4-5-88e8b81970dc@google.com \
    --to=devnull+ackerleytng.google.com@kernel.org \
    --cc=ackerleytng@google.com \
    --cc=akpm@linux-foundation.org \
    --cc=alexs@kernel.org \
    --cc=cgroups@vger.kernel.org \
    --cc=corbet@lwn.net \
    --cc=david@kernel.org \
    --cc=dzm91@hust.edu.cn \
    --cc=erdemaktas@google.com \
    --cc=fvdl@google.com \
    --cc=hannes@cmpxchg.org \
    --cc=joshua.hahnjy@gmail.com \
    --cc=jthoughton@google.com \
    --cc=liam@infradead.org \
    --cc=linmiaohe@huawei.com \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=ljs@kernel.org \
    --cc=louhongxiang@huawei.com \
    --cc=mawupeng1@huawei.com \
    --cc=mhocko@kernel.org \
    --cc=mhocko@suse.com \
    --cc=mike.kravetz@oracle.com \
    --cc=muchun.song@linux.dev \
    --cc=nphamcs@gmail.com \
    --cc=osalvador@suse.de \
    --cc=peterx@redhat.com \
    --cc=rientjes@google.com \
    --cc=roman.gushchin@linux.dev \
    --cc=rppt@kernel.org \
    --cc=shakeel.butt@linux.dev \
    --cc=si.yanteng@linux.dev \
    --cc=skhan@linuxfoundation.org \
    --cc=stable@vger.kernel.org \
    --cc=surenb@google.com \
    --cc=vannapurve@google.com \
    --cc=vbabka@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®