mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Miaohe Lin <linmiaohe@huawei.com>
To: Jiaqi Yan <jiaqiyan@google.com>
Cc: <osalvador@kernel.org>, <harry.yoo@oracle.com>,
	<willy@infradead.org>, <osalvador@suse.de>, <jackmanb@google.com>,
	<hannes@cmpxchg.org>, <nao.horiguchi@gmail.com>,
	<david@kernel.org>, <william.roche@oracle.com>,
	<tony.luck@intel.com>, <wangkefeng.wang@huawei.com>,
	<jane.chu@oracle.com>, <akpm@linux-foundation.org>,
	<muchun.song@linux.dev>, <liam@infradead.org>,
	<rientjes@google.com>, <duenwen@google.com>,
	<jthoughton@google.com>, <linux-mm@kvack.org>,
	<linux-kernel@vger.kernel.org>, <vbabka@suse.cz>,
	<rppt@kernel.org>, <shuah@kernel.org>, <surenb@google.com>,
	<mhocko@suse.com>, <boudewijn@delta-utec.com>, <ljs@kernel.org>,
	<ziy@nvidia.com>, <vbabka@kernel.org>
Subject: Re: [PATCH v6 2/5] mm/page_alloc: only free healthy pages in high-order has_hwpoisoned folio
Date: Fri, 17 Jul 2026 15:19:16 +0800	[thread overview]
Message-ID: <d46c6393-c516-4fc0-31d4-b2e9d6de0942@huawei.com> (raw)
In-Reply-To: <20260705180714.3708947-3-jiaqiyan@google.com>

On 2026/7/6 2:07, Jiaqi Yan wrote:
> At the end of dissolve_free_hugetlb_folio(), a free HugeTLB folio
> becomes non-HugeTLB, and it is released to buddy allocator
> as a high-order folio, e.g. a folio that contains 262144 pages
> if the folio was a 1G HugeTLB hugepage.
> 
> This is problematic if the HugeTLB hugepage contained HWPoison
> subpages. In that case, since buddy allocator does not check
> HWPoison for non-zero-order folio, the raw HWPoison page can
> be given out with its buddy page and be re-used by either
> kernel or userspace.
> 
> Memory failure recovery (MFR) in kernel does attempt to take
> raw HWPoison page off buddy allocator after
> dissolve_free_hugetlb_folio(). However, there is always a time
> window between dissolve_free_hugetlb_folio() frees a HWPoison
> high-order folio to buddy allocator and MFR takes HWPoison
> raw page off buddy allocator.
> 
> Another similar situation is when a transparent huge page (THP)
> runs into memory failure but splitting failed. Such THP will
> eventually be released to buddy allocator when owning userspace
> processes are gone, but with certain subpages having HWPoison.
> 
> One obvious way to avoid both problems is to add page sanity
> checks in page allocate or free path. However, it is against
> the past efforts to reduce sanity check overhead [1,2,3].
> 
> Introduce free_has_hwpoisoned() to only free the healthy pages
> and to exclude the HWPoison ones in the high-order folio.
> The idea is to iterate through the sub-pages of the folio to
> identify contiguous ranges of healthy pages.
> 
> free_has_hwpoisoned() is added at the end of __free_pages_prepare()
> as a shortcut and only if PG_has_hwpoisoned indicates HWPoison page
> exists and after checks and preparations in __free_pages_prepare()
> all succeeded. It then use __free_prepared_contig_range() to
> decompose healthy range into the largest possible chunks of
> different orders, then freed via __free_frozen_pages().
> 
> free_has_hwpoisoned() has linear time complexity wrt the number
> of pages in the folio. While the power-of-two decomposition
> ensures that the number of calls to the buddy allocator is
> logarithmic for each contiguous healthy range, the mandatory
> linear scan of pages to identify PageHWPoison() defines the
> overall time complexity. For a 1G hugepage having 8 HWPoison
> pages, free_has_hwpoisoned() takes around 1ms on average on
> a system having 56 Intel Skylake physical cores. This is
> 15x to the case of freeing no HWPoison page. The cost is far
> from triggering soft lockup, and fair for handling exceptional
> hardware memory errors.
> 
> [1] https://lore.kernel.org/linux-mm/1460711275-1130-15-git-send-email-mgorman@techsingularity.net
> [2] https://lore.kernel.org/linux-mm/1460711275-1130-16-git-send-email-mgorman@techsingularity.net
> [3] https://lore.kernel.org/all/20230216095131.17336-1-vbabka@suse.cz
> 
> Signed-off-by: Jiaqi Yan <jiaqiyan@google.com>

Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>

Nits:

> ---
>  mm/page_alloc.c | 165 ++++++++++++++++++++++++++++++++++++++++--------
>  1 file changed, 140 insertions(+), 25 deletions(-)
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 7d27aff48b15..6418896c9df6 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -93,6 +93,13 @@ typedef int __bitwise fpi_t;
>  /* free_pages_prepare() has already been called for page(s) being freed. */
>  #define FPI_PREPARED		((__force fpi_t)BIT(3))
>  
> +/*
> + * Page(s) needs to go through free_pages_sanitize(), for example, because
> + * free_pages_prepare() cannot sanitize a high-order page block due to
> + * hardware error in some page(s).
> + */
> +#define FPI_SANITIZE		((__force fpi_t)BIT(4))
> +
>  /* prevent >1 _updater_ of zone percpu pageset ->high and ->batch fields */
>  static DEFINE_MUTEX(pcp_batch_high_lock);
>  #define MIN_PERCPU_PAGELIST_HIGH_FRACTION (8)
> @@ -210,6 +217,8 @@ gfp_t gfp_allowed_mask __read_mostly = GFP_BOOT_MASK;
>  unsigned int pageblock_order __read_mostly;
>  #endif
>  
> +static void free_has_hwpoisoned(struct page *page, unsigned int order,
> +				fpi_t fpi_flags);
>  static void __free_pages_ok(struct page *page, unsigned int order,
>  			    fpi_t fpi_flags);
>  static void reserve_highatomic_pageblock(struct page *page, int order,
> @@ -1311,14 +1320,73 @@ static inline void pgalloc_tag_sub_pages(struct alloc_tag *tag, unsigned int nr)
>  
>  #endif /* CONFIG_MEM_ALLOC_PROFILING */
>  
> +/*
> + * Sanitize, which requires writing, a block of pages at the last moment of
> + * preparing to freeing them, i.e. __free_pages_prepare().

s/freeing/free/ ?

> + */
> +static void free_pages_sanitize(struct page *page, unsigned int order)
> +{
> +	bool init = want_init_on_free();
> +	/*
> +	 * __kasan_unpoison_pages() sets kasan tag on every tail page, so
> +	 * it is fine to use should_skip_kasan_poison() when pages here
> +	 * were a set of tail pages from a compound folio.
> +	 */
> +	bool skip_kasan_poison = should_skip_kasan_poison(page);
> +
> +	kernel_poison_pages(page, 1 << order);
> +
> +	/*
> +	 * As memory initialization might be integrated into KASAN,
> +	 * KASAN poisoning and memory initialization code must be
> +	 * kept together to avoid discrepancies in behavior.
> +	 *
> +	 * With hardware tag-based KASAN, memory tags must be set before the
> +	 * page becomes unavailable via debug_pagealloc or arch_free_page.
> +	 */
> +	if (!skip_kasan_poison) {
> +		kasan_poison_pages(page, order, init);
> +
> +		/* Memory is already initialized if KASAN did it internally. */
> +		if (kasan_has_integrated_init())
> +			init = false;
> +	}
> +	if (init)
> +		clear_highpages_kasan_tagged(page, 1 << order);
> +
> +	/*
> +	 * arch_free_page() can make the page's contents inaccessible.  s390
> +	 * does this.  So nothing which can access the page's contents should
> +	 * happen after this.
> +	 */
> +	arch_free_page(page, order);
> +
> +	debug_pagealloc_unmap_pages(page, 1 << order);
> +}
> +
> +/*
> + * Returns
> + * - true: checks and preparations all good, caller can proceed freeing.
> + * - false: do not proceed freeing for one of the following reasons:
> + *   1. Some check failed so it is not safe to proceed freeing.
> + *   2. A compound page has some HWPoison pages. The healthy pages
> + *      are already safely freed, and the HWPoison ones isolated.
> + */
>  static __always_inline bool __free_pages_prepare(struct page *page,
>  		unsigned int order, fpi_t fpi_flags)
>  {
>  	int bad = 0;
> -	bool skip_kasan_poison = should_skip_kasan_poison(page);
> -	bool init = want_init_on_free();
>  	bool compound = PageCompound(page);
>  	struct folio *folio = page_folio(page);
> +	/*
> +	 * When dealing with compound page, PG_has_hwpoisoned is cleared
> +	 * with PAGE_FLAGS_SECOND. So the check must be done first.
> +	 *
> +	 * Note we can't exclude PG_has_hwpoisoned from PAGE_FLAGS_SECOND.
> +	 * Because PG_has_hwpoisoned == PG_active, free_page_is_bad() will
> +	 * confuse and complaint that the first tail page is still active.
> +	 */
> +	bool should_fhh = compound && folio_test_has_hwpoisoned(folio);
>  
>  	if (fpi_flags & FPI_PREPARED)
>  		return true;
> @@ -1416,34 +1484,19 @@ static __always_inline bool __free_pages_prepare(struct page *page,
>  					   PAGE_SIZE << order);
>  	}
>  
> -	kernel_poison_pages(page, 1 << order);
> -
>  	/*
> -	 * As memory initialization might be integrated into KASAN,
> -	 * KASAN poisoning and memory initialization code must be
> -	 * kept together to avoid discrepancies in behavior.
> +	 * After breaking down compound page and dealing with page metadata
> +	 * (e.g. page owner and page alloc tags), take a shortcut if this
> +	 * was a compound page containing certain HWPoison subpages.
>  	 *
> -	 * With hardware tag-based KASAN, memory tags must be set before the
> -	 * page becomes unavailable via debug_pagealloc or arch_free_page.
> +	 * FPI_SANITIZE to remember free_pages_sanitize() healthy pages.
>  	 */
> -	if (!skip_kasan_poison) {
> -		kasan_poison_pages(page, order, init);
> -
> -		/* Memory is already initialized if KASAN did it internally. */
> -		if (kasan_has_integrated_init())
> -			init = false;
> +	if (should_fhh) {
> +		free_has_hwpoisoned(page, order, fpi_flags | FPI_SANITIZE);
> +		return false;
>  	}
> -	if (init)
> -		clear_highpages_kasan_tagged(page, 1 << order);
> -
> -	/*
> -	 * arch_free_page() can make the page's contents inaccessible.  s390
> -	 * does this.  So nothing which can access the page's contents should
> -	 * happen after this.
> -	 */
> -	arch_free_page(page, order);
>  
> -	debug_pagealloc_unmap_pages(page, 1 << order);
> +	free_pages_sanitize(page, order);
>  
>  	return true;
>  }
> @@ -6869,7 +6922,14 @@ static void __free_prepared_contig_range(struct page *page,
>  		/*
>  		 * Free the chunk as a single block. Our caller has already
>  		 * called free_pages_prepare() for each order-0 page.
> +		 *
> +		 * If the original compound page has HWPoison page,
> +		 * free_pages_prepare() has to skip sanitize at that time,
> +		 * but now it is good time to do that.
>  		 */
> +		if (fpi_flags & FPI_SANITIZE)
> +			free_pages_sanitize(page, order);
> +
>  		__free_frozen_pages(page, order, fpi_flags | FPI_PREPARED);
>  
>  		pfn += 1UL << order;
> @@ -6956,6 +7016,61 @@ void __free_contig_range(unsigned long pfn, unsigned long nr_pages)
>  	__free_contig_range_common(pfn, nr_pages, /* is_frozen= */ false);
>  }
>  
> +/*
> + * Given some contiguous pages that have certain number of HWPoison page(s),
> + * free only the healthy ones.
> + *
> + * Used at the end of __free_pages_prepare(). Even if having HWPoison pages,
> + * breaking down compound page and clearing metadata (e.g. page owner, alloc
> + * tag) can be done together during __free_pages_prepare(), which simplifies
> + * the splitting here: unlike __split_unmapped_folio(), there is no need to
> + * turn split pages into a compound page or to carry metadata.
> + *
> + * It scans every raw page of the compound page and causes nontrivial overhead.
> + * So only use this when the compound page contains HWPoison page(s).
> + *
> + * It also works when order == 0, regardless of PageHWPoison() or not.

It works but won't be called when order == 0?

Thanks.
.

  reply	other threads:[~2026-07-17  7:19 UTC|newest]

Thread overview: 40+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-07-05 18:07 [PATCH v6 0/5] Only " Jiaqi Yan
2026-07-05 18:07 ` [PATCH v6 1/5] mm/page_alloc: introduce __free_prepared_contig_range() with fpi_t Jiaqi Yan
2026-07-17  7:17   ` Miaohe Lin
2026-07-22  9:38   ` Vlastimil Babka (SUSE)
2026-07-25  3:54   ` Matthew Wilcox
2026-08-17  0:28     ` Jiaqi Yan
2026-07-05 18:07 ` [PATCH v6 2/5] mm/page_alloc: only free healthy pages in high-order has_hwpoisoned folio Jiaqi Yan
2026-07-17  7:19   ` Miaohe Lin [this message]
2026-07-22  9:40   ` Vlastimil Babka (SUSE)
2026-07-05 18:07 ` [PATCH v6 3/5] mm/memory-failure: set has_hwpoisoned flags on dissolved HugeTLB folio Jiaqi Yan
2026-07-25  3:05   ` Jiaqi Yan
2026-07-05 18:07 ` [PATCH v6 4/5] mm/memory-failure: skip take_page_off_buddy after dissolving HWPoison HugeTLB page Jiaqi Yan
2026-07-17  7:37   ` Miaohe Lin
2026-08-17  0:29     ` Jiaqi Yan
2026-08-17  7:23       ` Miaohe Lin
2026-08-18  3:30         ` Jiaqi Yan
2026-08-18  9:06           ` Miaohe Lin
2026-11-03 17:49             ` Jiaqi Yan
2026-07-05 18:07 ` [PATCH v6 5/5] selftests/mm: add hard memory failure anonymous HugeTLB test Jiaqi Yan
2026-07-17  7:37   ` Miaohe Lin
2026-07-25  3:05   ` Jiaqi Yan
2026-07-05 18:50 ` [PATCH v6 0/5] Only free healthy pages in high-order has_hwpoisoned folio Andrew Morton
2026-07-06  9:03   ` David Hildenbrand (Arm)
2026-07-25  3:06   ` Jiaqi Yan
2026-07-17 10:18 ` David Hildenbrand (Arm)
2026-07-17 13:06   ` William Roche
2026-07-22  8:27     ` Vlastimil Babka (SUSE)
2026-07-22  9:03       ` Vlastimil Babka (SUSE)
2026-07-25  6:00         ` Jiaqi Yan
2026-07-25  6:52       ` Jiaqi Yan
2026-07-27 14:20       ` David Hildenbrand (Arm)
2026-07-27 18:20         ` Matthew Wilcox
2026-07-28 17:18         ` Vlastimil Babka (SUSE)
2026-08-04 19:51           ` David Hildenbrand (Arm)
2026-08-05  8:58             ` Vlastimil Babka (SUSE)
2026-08-05  9:09               ` David Hildenbrand (Arm)
2026-11-03 17:49                 ` Jiaqi Yan
2026-08-28 23:36           ` Linux MM Alignment Session on hwpoison (was: Re: [PATCH v6 0/5] Only free healthy pages in high-order has_hwpoisoned folio) David Rientjes
2026-08-31 19:26             ` David Rientjes
2026-09-07 13:46             ` Linux MM Alignment Session on hwpoison David Hildenbrand (Arm)

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=d46c6393-c516-4fc0-31d4-b2e9d6de0942@huawei.com \
    --to=linmiaohe@huawei.com \
    --cc=akpm@linux-foundation.org \
    --cc=boudewijn@delta-utec.com \
    --cc=david@kernel.org \
    --cc=duenwen@google.com \
    --cc=hannes@cmpxchg.org \
    --cc=harry.yoo@oracle.com \
    --cc=jackmanb@google.com \
    --cc=jane.chu@oracle.com \
    --cc=jiaqiyan@google.com \
    --cc=jthoughton@google.com \
    --cc=liam@infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=ljs@kernel.org \
    --cc=mhocko@suse.com \
    --cc=muchun.song@linux.dev \
    --cc=nao.horiguchi@gmail.com \
    --cc=osalvador@kernel.org \
    --cc=osalvador@suse.de \
    --cc=rientjes@google.com \
    --cc=rppt@kernel.org \
    --cc=shuah@kernel.org \
    --cc=surenb@google.com \
    --cc=tony.luck@intel.com \
    --cc=vbabka@kernel.org \
    --cc=vbabka@suse.cz \
    --cc=wangkefeng.wang@huawei.com \
    --cc=william.roche@oracle.com \
    --cc=willy@infradead.org \
    --cc=ziy@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®