From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7ED9519E97B for ; Thu, 6 Aug 2026 02:29:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785983390; cv=none; b=crLzEvetA+HgQQQ+AZXaSNsSIpH2t1OFeL/kRUSAUReB/ErfWbQSi+r0k+9+61iCraxpU1SvEhrWOXin3ycJ/qOZH/oI5we1PWVb6juu5HOBj9uRfxQ9Ph+5BvJQDSwbDYFSXMm8Ub7LBFRCGU4iW+NzL0uoCy8vEPC5b3qD740= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785983390; c=relaxed/simple; bh=ArFlT2tb5Jahy6dugTO0uQc8HH6hvHWLFH/ow788NfM=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=H/CWd4lkBpCvZP9Db3Ze9WsO2pQl0aPvsrnRx34ggWwSCV9brunyTeASjbHBmXS29bInUoH2CadT983nMDO2myoSMSzLo5riCAyBgays63ucx96oMKet5c/kdHcK39eErNPS4OkvIOSiKHCW4/82lkuvd4LYOnHiFzuJPtuwNUc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=fTKut1P7; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="fTKut1P7" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1785983387; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9jY7+oBUrKHkFO4iBtP2E++Vts1iHCUswnJVsbsvky4=; b=fTKut1P7QK6DCrS7ZG6DNc+7y4OCLy+sx07nJslDevYXnwI7Xne5WVx8fD0IfvJ19hsNRd QpYUyXprzVkTS6lLQMQ8qlBxhRokdNh3pdnIFL9eaakJ9MVtR4+3QoK6/k1avgMutNsm0G 9xiUyAdVxCYrhQDJJ6EHpWF4trYH3hI= Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-526-NKr_pi2VNpe-6TMbwZSN8Q-1; Wed, 05 Aug 2026 22:29:45 -0400 X-MC-Unique: NKr_pi2VNpe-6TMbwZSN8Q-1 X-Mimecast-MFC-AGG-ID: NKr_pi2VNpe-6TMbwZSN8Q_1785983385 Received: by mail-qt1-f200.google.com with SMTP id d75a77b69052e-51c0e702df8so17036121cf.2 for ; Wed, 05 Aug 2026 19:29:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785983385; x=1786588185; h=content-transfer-encoding:content-type:in-reply-to:from :content-language:references:cc:to:subject:user-agent:mime-version :date:message-id:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=9jY7+oBUrKHkFO4iBtP2E++Vts1iHCUswnJVsbsvky4=; b=MDdxjDJ4HuWrJMp/rMr28OI5GT/DrGg4pM/YvXjplkXqq1nrJBMcH9Wn8j6uPXHgLe frWkfAItr9Y9/ETXzpU5Icbq8Elr1w7X/xIEw9P628H0G8yuQ1wA9/0tT9A7Ktd52Vdt 6TgJWsOrLP72e2mksLflcenxvBqq2a7nGjPoVMXFA+vizJyqSAczEPm8plObq3yl4wTi UWW/u5sI/BFt67u2eZhIWFyQQ92ZZBD0eG2ROD/wOO92Y6VmQRQvQPAym23cWehphDtR bqiheA5ZQv5UT22Ug28+2NYxpcYpaUGHs9FXmapvrryUmWTJexjo8qAFeRdvJdxlnd9t e4Qw== X-Forwarded-Encrypted: i=1; AHgh+RqI1CmLqMEiq0G2zEdpuPEJSQUJdlc1/kvGFbXy1paxQBdg6NdTelsyif+BDMhN+1blA2sphfb+rnrRJOg=@vger.kernel.org X-Gm-Message-State: AOJu0YzRAPRjGrya36feny11QjFUlscGRlqpjvrdlyhkX3FgF6MWLrhE F+dvDc2SkdsOqZlVI6jEiSNnaqD6R3zg1V21t1s6J/xJ0pwPRgYKKtZSdzkz3CYIivf/OZTBW80 FXGlADtu/D8UgyeM+xdDu4zdge7pgRHHLbzMHXbdtxxJWEngX+97+KxLEP0HYQMfrpw== X-Gm-Gg: AR+sD12vb9ShBzp9XL6jva84Rq5f5e41Rpv9KYZyEHb2ezImxT6QOW1a0jGOc2fdYCh 4jYt0WUNQp/f+cfNronafE22njruxw4hBvC9T+CSM4z0RpS5z1ECY/rTjlLdh3UldOvcfIgiQo2 rTrCEmSbPtAl88bSd6jQh3PluZRQh+1GfGqJP/lfc92QoNa1dzwnHuklYsxYDCgYGN3TDUV4RN9 +J1bEv1O9LC+Tg/5oIRT/w46eEBvb1Hq6rk1dhFGzMjaQWzNv8yBsevgN+Ov/3vI8efXIr8E9Jr lkJnmzfgzBKAj/CERjrmGL6aYI1NefZGcM0yCJ1qRtGcJYnsWCn/p/woXkrkJ7mF67VLD96MjVV sj8LTXZ36eNAhY5eliVr2ps8whPbTdnCxQTxkObskNrkBTgDrFD632deTqnEofA== X-Received: by 2002:a05:622a:4ccf:b0:50d:e471:2d1e with SMTP id d75a77b69052e-52ce61b4ef7mr119776871cf.35.1785983384913; Wed, 05 Aug 2026 19:29:44 -0700 (PDT) X-Received: by 2002:a05:622a:4ccf:b0:50d:e471:2d1e with SMTP id d75a77b69052e-52ce61b4ef7mr119776491cf.35.1785983384251; Wed, 05 Aug 2026 19:29:44 -0700 (PDT) Received: from [192.168.2.110] (bras-base-aylmpq0104w-grc-61-70-49-81-60.dsl.bell.ca. [70.49.81.60]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-52ce892a067sm31848491cf.17.2026.08.05.19.29.42 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 05 Aug 2026 19:29:43 -0700 (PDT) Message-ID: <3374e351-f6ad-44b2-9ee1-dd3b9c33fbc3@redhat.com> Date: Wed, 5 Aug 2026 22:29:42 -0400 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v5 10/11] mm: install PMD swap entries on swap-out To: Usama Arif , Andrew Morton , david@kernel.org, chrisl@kernel.org, kasong@tencent.com, ljs@kernel.org, ziy@nvidia.com, linux-mm@kvack.org Cc: ying.huang@linux.alibaba.com, Baoquan He , willy@infradead.org, youngjun.park@lge.com, hannes@cmpxchg.org, riel@surriel.com, shakeel.butt@linux.dev, alex@ghiti.fr, kas@kernel.org, baohua@kernel.org, dev.jain@arm.com, baolin.wang@linux.alibaba.com, Nico Pache , "Liam R. Howlett" , ryan.roberts@arm.com, Vlastimil Babka , lance.yang@linux.dev, linux-kernel@vger.kernel.org, nphamcs@gmail.com, shikemeng@huaweicloud.com, yosry@kernel.org, kernel-team@meta.com References: <20260722152043.2273289-1-usama.arif@linux.dev> <20260722152043.2273289-11-usama.arif@linux.dev> Content-Language: en-US, en-CA From: Luiz Capitulino In-Reply-To: <20260722152043.2273289-11-usama.arif@linux.dev> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 2026-07-22 11:19, Usama Arif wrote: > Reclaim today splits a PMD-mapped anonymous THP into 512 PTE swap > entries before unmap, losing the huge mapping across the swap > round-trip and forcing khugepaged to rebuild it later. The contiguous > swap range was already secured when the folio was added to the swap > cache (a non-contiguous allocation would have split the folio earlier), > so the PMD can be replaced by a single PMD-level swap entry instead. > > This patch mirrors the existing PTE swap-out path at PMD granularity: > - shrink_folio_list() drops TTU_SPLIT_HUGE_PMD for PMD-mappable > swapcache folios. zswap is handled by the PMD swap-in users: if any > covered slot currently has a zswap entry, they split the PMD swap > entry and fall back to the per-PTE path. > - try_to_unmap_one() now has a PMD branch that calls > set_pmd_swap_entry() and adjusts MM_ANONPAGES / MM_SWAPENTS by > HPAGE_PMD_NR before walk_done. TTU_SPLIT_HUGE_PMD remains the > fallback. > - set_pmd_swap_entry() is the installer. Mirroring the PTE swap-out > sequence at PMD granularity, it clears the present mapping (keeping > the original for rollback), bumps the swap_map refcount for the > folio's 512 slots, transfers the exclusive state in the swap entry, > propagates the dirty bit to the folio so writeback is not lost, > and installs a swap PMD that preserves the original > soft-dirty / uffd-wp / exclusive bits. Any failing step rolls back > the present mapping. > > The swap entry value matches what 512 PTE swap entries would encode, so > swap_map refcounting is unchanged: each of the 512 slots carries a > count of 1, released individually on later split or together on swap-in. > > Add thp_swpout_pmd to count each PMD mapping replaced by a PMD-level > swap entry. Unlike the folio-level thp_swpout counter, a fork-shared THP > can increment this counter once for each mapping; document that > distinction. > > Signed-off-by: Usama Arif > --- > Documentation/admin-guide/mm/transhuge.rst | 5 ++ > include/linux/huge_mm.h | 2 + > include/linux/vm_event_item.h | 1 + > mm/huge_memory.c | 80 ++++++++++++++++++++++ > mm/rmap.c | 19 +++++ > mm/vmscan.c | 9 ++- > mm/vmstat.c | 1 + > 7 files changed, 116 insertions(+), 1 deletion(-) > > diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst > index 16f37135ed80..b421d7982db6 100644 > --- a/Documentation/admin-guide/mm/transhuge.rst > +++ b/Documentation/admin-guide/mm/transhuge.rst > @@ -630,6 +630,11 @@ thp_swpout > is incremented every time a huge page is swapout in one > piece without splitting. > > +thp_swpout_pmd > + is incremented every time a PMD mapping is replaced by a PMD-level > + swap entry. A fork-shared THP can increment this counter once for each > + PMD mapping that is swapped out. > + > thp_swpout_fallback > is incremented if a huge page has to be split before swapout. > Usually because failed to allocate some continuous swap space > diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h > index e7107e0991ad..41cf643a3f55 100644 > --- a/include/linux/huge_mm.h > +++ b/include/linux/huge_mm.h > @@ -554,6 +554,8 @@ vm_fault_t do_huge_pmd_device_private(struct vm_fault *vmf); > > #ifdef CONFIG_THP_SWAP > vm_fault_t do_huge_pmd_swap_page(struct vm_fault *vmf); > +int set_pmd_swap_entry(struct page_vma_mapped_walk *pvmw, > + struct folio *folio); > #else > static inline vm_fault_t do_huge_pmd_swap_page(struct vm_fault *vmf) > { > diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h > index 2628ccda076a..f8fd4e13698c 100644 > --- a/include/linux/vm_event_item.h > +++ b/include/linux/vm_event_item.h > @@ -108,6 +108,7 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT, > THP_ZERO_PAGE_ALLOC_FAILED, > THP_SWPOUT, > THP_SWPOUT_FALLBACK, > + THP_SWPOUT_PMD, > #endif > #ifdef CONFIG_BALLOON > BALLOON_INFLATE, > diff --git a/mm/huge_memory.c b/mm/huge_memory.c > index 6ef56936ced4..c014631e5a26 100644 > --- a/mm/huge_memory.c > +++ b/mm/huge_memory.c > @@ -5561,3 +5561,83 @@ void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new) > trace_remove_migration_pmd(address, pmd_val(pmde)); > } > #endif > + > +#ifdef CONFIG_THP_SWAP > +/** > + * set_pmd_swap_entry() - Replace a PMD mapping with a PMD-level swap entry. > + * @pvmw: Page vma mapped walk context, must have pvmw->pmd set and > + * pvmw->pte NULL (i.e. PMD-mapped). > + * @folio: The folio being swapped out. Must be in the swap cache. > + * > + * This installs a PMD-level swap entry in place of a present PMD mapping, > + * avoiding the need to split the PMD into PTE-level swap entries. > + * > + * Return: 0 on success, negative error code on failure. > + */ > +int set_pmd_swap_entry(struct page_vma_mapped_walk *pvmw, > + struct folio *folio) > +{ > + struct vm_area_struct *vma = pvmw->vma; > + struct mm_struct *mm = vma->vm_mm; > + unsigned long address = pvmw->address; > + unsigned long haddr = address & HPAGE_PMD_MASK; > + struct page *page = folio_page(folio, 0); > + bool anon_exclusive; > + pmd_t pmdval; > + swp_entry_t entry; > + pmd_t pmdswp; > + > + if (!(pvmw->pmd && !pvmw->pte)) > + return 0; Should we call VM_WARN_ON_ONCE() and return an error instead? This function is only called by try_to_unmap_one() under this condition. In addition, returning zero would cause try_to_unmap_one() to assume that the PMD swap entry was installed, which is not the case. > + > + VM_BUG_ON_FOLIO(!folio_test_swapcache(folio), folio); > + VM_BUG_ON_FOLIO(!folio_test_anon(folio), folio); > + > + if (unlikely(folio_test_swapbacked(folio) != > + folio_test_swapcache(folio))) { > + WARN_ON_ONCE(1); > + return -EBUSY; > + } > + > + flush_cache_range(vma, haddr, haddr + HPAGE_PMD_SIZE); > + > + pmdval = pmdp_invalidate(vma, haddr, pvmw->pmd); > + > + /* Update high watermark before we lower rss */ > + update_hiwater_rss(mm); > + > + if (folio_dup_swap(folio, NULL) < 0) { > + set_pmd_at(mm, haddr, pvmw->pmd, pmdval); > + return -ENOMEM; > + } > + > + /* See folio_try_share_anon_rmap_pmd(): invalidate PMD first. */ > + anon_exclusive = PageAnonExclusive(page); > + if (anon_exclusive && folio_try_share_anon_rmap_pmd(folio, page)) { > + folio_put_swap(folio, NULL); > + set_pmd_at(mm, haddr, pvmw->pmd, pmdval); > + return -EBUSY; > + } > + > + mm_prepare_for_swap_entries(mm); > + > + if (pmd_dirty(pmdval)) > + folio_mark_dirty(folio); > + > + entry = folio->swap; > + pmdswp = softleaf_to_pmd(entry); > + if (pmd_soft_dirty(pmdval)) > + pmdswp = pmd_swp_mksoft_dirty(pmdswp); > + if (pmd_uffd(pmdval)) > + pmdswp = pmd_swp_mkuffd(pmdswp); > + if (anon_exclusive) > + pmdswp = pmd_swp_mkexclusive(pmdswp); > + set_pmd_at(mm, haddr, pvmw->pmd, pmdswp); > + > + folio_remove_rmap_pmd(folio, page, vma); > + folio_put(folio); > + > + count_vm_event(THP_SWPOUT_PMD); > + return 0; > +} > +#endif /* CONFIG_THP_SWAP */ > diff --git a/mm/rmap.c b/mm/rmap.c > index b7ead3e9f064..4574f7b969b6 100644 > --- a/mm/rmap.c > +++ b/mm/rmap.c > @@ -2282,6 +2282,25 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, > goto walk_abort; > } > > +#ifdef CONFIG_THP_SWAP > + /* > + * If the folio is in the swap cache and we're not > + * asked to split, install a PMD-level swap entry. > + */ > + if (!(flags & TTU_SPLIT_HUGE_PMD) && > + folio_test_anon(folio) && > + folio_test_swapcache(folio)) { > + if (set_pmd_swap_entry(&pvmw, folio)) > + goto walk_abort; > + > + add_mm_counter(mm, MM_ANONPAGES, > + -HPAGE_PMD_NR); > + add_mm_counter(mm, MM_SWAPENTS, > + HPAGE_PMD_NR); > + goto walk_done; > + } > +#endif > + > if (flags & TTU_SPLIT_HUGE_PMD) { > /* > * We temporarily have to drop the PTL and > diff --git a/mm/vmscan.c b/mm/vmscan.c > index 9b8ee902f972..83bb34403e42 100644 > --- a/mm/vmscan.c > +++ b/mm/vmscan.c > @@ -1333,7 +1333,14 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, > enum ttu_flags flags = TTU_BATCH_FLUSH; > bool was_swapbacked = folio_test_swapbacked(folio); > > - if (folio_test_pmd_mappable(folio)) > + /* > + * With THP_SWAP, PMD-mappable folios already in the > + * swap cache can be unmapped with a PMD-level swap > + * entry, avoiding the cost of splitting the PMD. > + */ > + if (folio_test_pmd_mappable(folio) && > + !(IS_ENABLED(CONFIG_THP_SWAP) && > + folio_test_swapcache(folio))) > flags |= TTU_SPLIT_HUGE_PMD; > /* > * Without TTU_SYNC, try_to_unmap will only begin to > diff --git a/mm/vmstat.c b/mm/vmstat.c > index 4e26e5fd6666..68e6efc0dabd 100644 > --- a/mm/vmstat.c > +++ b/mm/vmstat.c > @@ -1422,6 +1422,7 @@ const char * const vmstat_text[] = { > [I(THP_ZERO_PAGE_ALLOC_FAILED)] = "thp_zero_page_alloc_failed", > [I(THP_SWPOUT)] = "thp_swpout", > [I(THP_SWPOUT_FALLBACK)] = "thp_swpout_fallback", > + [I(THP_SWPOUT_PMD)] = "thp_swpout_pmd", > #endif > #ifdef CONFIG_BALLOON > [I(BALLOON_INFLATE)] = "balloon_inflate",