From: Ackerley Tng via B4 Relay <devnull+ackerleytng.google.com@kernel.org>
To: Muchun Song <muchun.song@linux.dev>,
Oscar Salvador <osalvador@suse.de>,
David Hildenbrand <david@kernel.org>,
Joshua Hahn <joshua.hahnjy@gmail.com>,
Shakeel Butt <shakeel.butt@linux.dev>,
Nhat Pham <nphamcs@gmail.com>,
Andrew Morton <akpm@linux-foundation.org>,
Peter Xu <peterx@redhat.com>, Wupeng Ma <mawupeng1@huawei.com>,
fvdl@google.com, rientjes@google.com, jthoughton@google.com,
Mike Kravetz <mike.kravetz@oracle.com>,
Johannes Weiner <hannes@cmpxchg.org>,
Michal Hocko <mhocko@kernel.org>,
Roman Gushchin <roman.gushchin@linux.dev>,
Lorenzo Stoakes <ljs@kernel.org>,
"Liam R. Howlett" <liam@infradead.org>,
Vlastimil Babka <vbabka@kernel.org>,
Mike Rapoport <rppt@kernel.org>,
Suren Baghdasaryan <surenb@google.com>,
Michal Hocko <mhocko@suse.com>, Jonathan Corbet <corbet@lwn.net>,
Shuah Khan <skhan@linuxfoundation.org>,
Alex Shi <alexs@kernel.org>, Yanteng Si <si.yanteng@linux.dev>,
Dongliang Mu <dzm91@hust.edu.cn>,
Hongxiang Lou <louhongxiang@huawei.com>,
Miaohe Lin <linmiaohe@huawei.com>
Cc: vannapurve@google.com, erdemaktas@google.com, linux-mm@kvack.org,
linux-kernel@vger.kernel.org, cgroups@vger.kernel.org,
linux-doc@vger.kernel.org, Ackerley Tng <ackerleytng@google.com>,
stable@vger.kernel.org
Subject: [PATCH v4 05/16] mm: hugetlb: Fix subpool usage leak on allocation failure
Date: Wed, 22 Jul 2026 16:41:13 -0700 [thread overview]
Message-ID: <20260722-hugetlb-alloc-failure-fixes-v4-5-88e8b81970dc@google.com> (raw)
In-Reply-To: <20260722-hugetlb-alloc-failure-fixes-v4-0-88e8b81970dc@google.com>
From: Ackerley Tng <ackerleytng@google.com>
When alloc_hugetlb_folio() fails early (e.g. buddy allocation failure or
hugetlb cgroup charging failure) and gbl_chg == 1 (meaning a reservation
was not used, but a global page was allocated instead), the subpool page
acquired via hugepage_subpool_get_pages() must still be returned.
Currently, the error path out_subpool_put: only calls
hugepage_subpool_put_pages() if !gbl_chg is true. If gbl_chg is 1, it
skips it, permanently leaking the subpool's used_hpages counter.
With the earlier patch to always track used_hpages in the subpool, always
call hugepage_subpool_put_pages() if map_chg is true to consistently
restore the page to the subpool. Condition on map_chg because map_chg also
gates hugepage_subpool_get_pages().
Opportunistically rename gbl_chg to gbl_resv_get, which is the number of
global pages needed (because neither resv_map nor subpool reservations
could be used).
If the number of global pages needed is 0, this allocation uses a
reservation somewhere, hence proceed to consume a reservation by
decrementing h->resv_huge_pages. If the number of global pages needed is 1,
reservations are neither created nor consumed.
Also rename gbl_reserve to gbl_resv_put, which is the number of
pages the subpool could not absorb into its reservations.
Adjust global reservations using hugetlb_acct_memory() with the difference
between gbl_resv_get and gbl_resv_put to take care of possible races where
another thread might have performed get or put with the same subpool, hence
perhaps requiring updates to global reservation counts. (If gbl_resv_get ==
0 because a resv_map reservation was used, map_chg == 0 so the entire
subpool returning is skipped - still correct.)
Fixes: a833a693a490e ("mm: hugetlb: fix incorrect fallback for subpool")
Cc: stable@vger.kernel.org
Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
mm/hugetlb.c | 23 ++++++++++-------------
1 file changed, 10 insertions(+), 13 deletions(-)
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 5ee1bc5c00bfe..879e4640dc50d 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -2872,7 +2872,7 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma,
struct hugepage_subpool *spool = subpool_vma(vma);
struct hstate *h = hstate_vma(vma);
struct folio *folio;
- long retval, gbl_chg, gbl_reserve;
+ long retval, gbl_resv_get;
map_chg_state map_chg;
int ret, idx;
struct hugetlb_cgroup *h_cg = NULL;
@@ -2912,15 +2912,15 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma,
* Or if it can get one from the pool reservation directly.
*/
if (map_chg) {
- gbl_chg = hugepage_subpool_get_pages(spool, 1);
- if (gbl_chg < 0)
+ gbl_resv_get = hugepage_subpool_get_pages(spool, 1);
+ if (gbl_resv_get < 0)
goto out_end_reservation;
} else {
/*
* If we have the vma reservation ready, no need for extra
* global reservation.
*/
- gbl_chg = 0;
+ gbl_resv_get = 0;
}
/*
@@ -2949,7 +2949,7 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma,
* from the global free pool (global change). gbl_chg == 0 indicates
* a reservation exists for the allocation.
*/
- folio = dequeue_hugetlb_folio_vma(h, vma, addr, gbl_chg);
+ folio = dequeue_hugetlb_folio_vma(h, vma, addr, gbl_resv_get);
if (!folio) {
spin_unlock_irq(&hugetlb_lock);
folio = alloc_buddy_hugetlb_folio_with_mpol(h, vma, addr);
@@ -2965,7 +2965,7 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma,
* Either dequeued or buddy-allocated folio needs to add special
* mark to the folio when it consumes a global reservation.
*/
- if (!gbl_chg) {
+ if (!gbl_resv_get) {
folio_set_hugetlb_restore_reserve(folio);
h->resv_huge_pages--;
}
@@ -3022,13 +3022,10 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma,
hugetlb_cgroup_uncharge_cgroup_rsvd(idx, pages_per_huge_page(h),
h_cg_rsvd);
out_subpool_put:
- /*
- * put page to subpool iff the quota of subpool's rsv_hpages is used
- * during hugepage_subpool_get_pages.
- */
- if (map_chg && !gbl_chg) {
- gbl_reserve = hugepage_subpool_put_pages(spool, 1);
- hugetlb_acct_memory(h, -gbl_reserve);
+ if (map_chg) {
+ long gbl_resv_put = hugepage_subpool_put_pages(spool, 1);
+
+ hugetlb_acct_memory(h, gbl_resv_get - gbl_resv_put);
}
--
2.55.0.229.g6434b31f56-goog
next prev parent reply other threads:[~2026-07-22 23:41 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-07-22 23:41 [PATCH v4 00/16] Fix bugs on HugeTLB folio allocation failure paths Ackerley Tng via B4 Relay
2026-07-22 23:41 ` [PATCH v4 01/16] mm: hugetlb: Track used_hpages when getting/putting pages from subpool Ackerley Tng via B4 Relay
2026-07-29 17:14 ` Ackerley Tng
2026-07-22 23:41 ` [PATCH v4 02/16] mm: hugetlb: Return -ENOSPC on memcg charge failure Ackerley Tng via B4 Relay
2026-07-22 23:41 ` [PATCH v4 03/16] mm: hugetlb: Use try-commit-cancel protocol for memcg charge of folios Ackerley Tng via B4 Relay
2026-07-22 23:41 ` [PATCH v4 04/16] mm: hugetlb: Remove unused mem_cgroup_charge_hugetlb function Ackerley Tng via B4 Relay
2026-07-22 23:41 ` Ackerley Tng via B4 Relay [this message]
2026-07-22 23:41 ` [PATCH v4 06/16] mm: hugetlb: Rename local variables for clarity in hugetlb_reserve_pages() Ackerley Tng via B4 Relay
2026-07-22 23:41 ` [PATCH v4 07/16] mm: hugetlb: Fix Use-After-Free in unlock_or_release_subpool() Ackerley Tng via B4 Relay
2026-07-22 23:41 ` [PATCH v4 08/16] fs: hugetlbfs: Fix global reservation leak in hugetlbfs_fill_super() Ackerley Tng via B4 Relay
2026-07-22 23:41 ` [PATCH v4 09/16] WIP: mm: hugetlb: Move subpool functions to hugetlb_subpool.c Ackerley Tng via B4 Relay
2026-07-22 23:41 ` [PATCH v4 10/16] WIP: fs: hugetlbfs: Refactor subpool getters and integrate with hugetlb_subpool API Ackerley Tng via B4 Relay
2026-07-22 23:41 ` [PATCH v4 11/16] WIP: mm: hugetlb: Make struct hugepage_subpool private to hugetlb_subpool.c Ackerley Tng via B4 Relay
2026-07-22 23:41 ` [PATCH v4 12/16] WIP: tools: testing: Add unit tests for HugeTLB subpool functions Ackerley Tng via B4 Relay
2026-07-22 23:41 ` [PATCH v4 13/16] WIP: Reproducer for allocation failure due to cgroup v2 memory limits Ackerley Tng via B4 Relay
2026-07-22 23:41 ` [PATCH v4 14/16] WIP: Reproducer for subpool usage leak Ackerley Tng via B4 Relay
2026-07-22 23:41 ` [PATCH v4 15/16] WIP: Reproducer for false restoration on shared HugeTLB mappings Ackerley Tng via B4 Relay
2026-07-22 23:41 ` [PATCH v4 16/16] WIP: Reproducer for out_put_pages subpool reserve leakage Ackerley Tng via B4 Relay
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260722-hugetlb-alloc-failure-fixes-v4-5-88e8b81970dc@google.com \
--to=devnull+ackerleytng.google.com@kernel.org \
--cc=ackerleytng@google.com \
--cc=akpm@linux-foundation.org \
--cc=alexs@kernel.org \
--cc=cgroups@vger.kernel.org \
--cc=corbet@lwn.net \
--cc=david@kernel.org \
--cc=dzm91@hust.edu.cn \
--cc=erdemaktas@google.com \
--cc=fvdl@google.com \
--cc=hannes@cmpxchg.org \
--cc=joshua.hahnjy@gmail.com \
--cc=jthoughton@google.com \
--cc=liam@infradead.org \
--cc=linmiaohe@huawei.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=ljs@kernel.org \
--cc=louhongxiang@huawei.com \
--cc=mawupeng1@huawei.com \
--cc=mhocko@kernel.org \
--cc=mhocko@suse.com \
--cc=mike.kravetz@oracle.com \
--cc=muchun.song@linux.dev \
--cc=nphamcs@gmail.com \
--cc=osalvador@suse.de \
--cc=peterx@redhat.com \
--cc=rientjes@google.com \
--cc=roman.gushchin@linux.dev \
--cc=rppt@kernel.org \
--cc=shakeel.butt@linux.dev \
--cc=si.yanteng@linux.dev \
--cc=skhan@linuxfoundation.org \
--cc=stable@vger.kernel.org \
--cc=surenb@google.com \
--cc=vannapurve@google.com \
--cc=vbabka@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®