From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mta0.migadu.com (out-28.mta0.migadu.com [91.218.175.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 45F3737DEBB for ; Tue, 18 Aug 2026 09:20:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.28 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787044842; cv=none; b=lJeA1ClJzS8K/UuzswsIna4tIF1VywiDt1BIuX8AoBix/oWXjlHCamRDv8I9wIm6HRy/SsaTyrBm0G1QKDN2muCODyX+WwSQL8pq+fTfYbzh13uA4561/pIlzFcU9JoW3tYIVu9k8RWvWrpShrE5RqtCQSAg47ww9cPg81a9Juw= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787044842; c=relaxed/simple; bh=B1m6agvmeh0+Uq3KM3O2ABateFoLYnXjxIldxiUU5DY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=hHtfqOkNl1cM8YsaPt04znBKeqfRvA8stJ+upKAm6yh0pTlZ+zKVlASd/lYS078GDSfQ7BGf7JF/GWDUXsUYZwapPlpxULXYBS+QQykmEHXtE7KytTunCCpq2TWfzBIQaBrJvnVv7aT6unlwXbvd9sXAf/d4ecAjl7ZbsLndVFU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=W1/rOEwN; arc=none smtp.client-ip=91.218.175.28 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="W1/rOEwN" X-Envelope-To: linux-kernel@vger.kernel.org DKIM-Signature: a=rsa-sha256; bh=B1m6agvmeh0+Uq3KM3O2ABateFoLYnXjxIldxiUU5DY=; c=simple/simple; d=linux.dev; h=from:to:subject:date:message-id:mime-version:content-type; s=key1; t=1787044838; v=1; x=1787649638; b=W1/rOEwNCrcocXF0rGLJGMXPVcd488WUGrrF0fuvY8JZ+a6JcJqi03JP9hixULMVWTbONSRR +KnhQMa/+Hl4QENdc/PwNW6Smpu3O48n6KwbCQFgFm9Rw5GuTEMi6JjUc2+589AN013X7M8A+9a VBP8A9i7OKOMuy+PSkYLL/Vw= X-Envelope-To: linux-kernel@vger.kernel.org Received: from localhost (2602:fce1:44f:115e::) by smtp.migadu.com with ESMTPS id 178fe688d0370b01; Tue, 18 Aug 2026 09:20:38 +0000 X-Migadu-Flow: FLOW_OUT From: Lance Yang To: linmiaohe@huawei.com Cc: shivankg@amd.com, akpm@linux-foundation.org, david@kernel.org, ljs@kernel.org, riel@surriel.com, liam@infradead.org, vbabka@kernel.org, harry@kernel.org, jannh@google.com, rppt@kernel.org, surenb@google.com, mhocko@suse.com, ziy@nvidia.com, matthew.brost@intel.com, joshua.hahnjy@gmail.com, rakie.kim@sk.com, byungchul@sk.com, gourry@gourry.net, ying.huang@linux.alibaba.com, apopple@nvidia.com, kmanaouil.dev@gmail.com, fvdl@google.com, kinseyho@google.com, weixugc@google.com, bharata@amd.com, rientjes@google.com, dev.jain@arm.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Lance Yang Subject: Re: [PATCH v2 7/7] mm/rmap: batch the unmap of large folios in try_to_migrate_one() Date: Tue, 18 Aug 2026 17:20:32 +0800 Message-Id: <20260818092032.47670-1-lance.yang@linux.dev> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <860941c2-287f-f88f-920c-6d99e2cd4483@huawei.com> References: <860941c2-287f-f88f-920c-6d99e2cd4483@huawei.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit On Tue, Aug 18, 2026 at 04:55:06PM +0800, Miaohe Lin wrote: >On 2026/8/17 17:14, Lance Yang wrote: >> +Cc Miaohe >> >> On Thu, Aug 13, 2026 at 04:23:18AM +0000, Shivank Garg wrote: >>> try_to_migrate_one() converts present PTEs to migration entries one at a >>> time. For a PTE-mapped large folio, this repeat calls to ptep clear+flush, >>> the migration entry build and set, folio_remove_rmap_pte() and folio_put(), >>> each re-entering page_vma_mapped_walk() once per base page (256 times for >>> 1M folio). >>> >>> Mirror try_to_unmap_one() to introduce folio_migrate_pte_batch() to detect >>> eligible batch for PTEs mapping conseuctive subpages of a large folios, >>> and convert the whole batch in one shot using the batched helpers. >>> >>> A side-effect of this change is trace_set_migration_pte() will record >>> one event per batched run instead of earlier behavior of one per base page. >>> >>> Signed-off-by: Shivank Garg >>> --- >>> mm/rmap.c | 115 ++++++++++++++++++++++++++++++++++++++++++++++---------------- >>> 1 file changed, 86 insertions(+), 29 deletions(-) >>> >>> diff --git a/mm/rmap.c b/mm/rmap.c >>> index 35752a70f3a0..63b885c0b7ef 100644 >>> --- a/mm/rmap.c >>> +++ b/mm/rmap.c >>> @@ -2675,6 +2675,44 @@ static bool try_to_migrate_hugetlb_one(struct folio *folio, >>> return ret; >>> } >>> >>> +static inline unsigned int folio_migrate_pte_batch(struct folio *folio, >>> + struct page_vma_mapped_walk *pvmw, pte_t pte, >>> + struct page *subpage, bool anon_exclusive) >>> +{ >>> + unsigned long end_addr, addr = pvmw->address; >>> + struct vm_area_struct *vma = pvmw->vma; >>> + unsigned int max_nr, nr; >>> + >>> +#ifdef __HAVE_ARCH_UNMAP_ONE >>> + /* Cannot batch unmap if arch_unmap_one() is defined. */ >>> + return 1; >>> +#endif >>> + >>> + if (!folio_test_large(folio)) >>> + return 1; >>> + if (folio_is_zone_device(folio) || folio_test_has_hwpoisoned(folio)) >>> + return 1; >>> + if (pte_unused(pte)) >>> + return 1; >>> + >>> + /* We may only batch within a single VMA and a single page table. */ >>> + end_addr = pmd_addr_end(addr, vma->vm_end); >>> + max_nr = (end_addr - addr) >> PAGE_SHIFT; >> >> Hmm ... can this still batch over a poisoned tail page? >> >> memory_failure() sets PageHWPoison() before taking folio lock, but >> cannot set PG_has_hwpoisoned until it acquires and releases that lock. >> >> So tail page can already be poisoned while folio_test_has_hwpoisoned() >> still returns false ... no? > >When memory error hits thp pages, memory_failure() first set PG_has_hwpoisoned and >then tries to split thp pages. And try_to_migrate() will be called to set migration >entries for anon pages. Does folio_migrate_pte_batch() work on this case? If so, the >folio_test_has_hwpoisoned() check above could catch the bad pages? > >Or do you worry about the scene that meory error hits a thp while it's under migration? Yeah, latter case is exactly what I meant. try_to_migrate() is called with folio lock held. If migration already owns the lock, memory_failure() can set PageHWPoison() on a tail page and then block in folio_lock(), before reaching folio_set_has_hwpoisoned(). try_to_migrate_one() may meanwhile start from a healthy subpage, see PageHWPoison(subpage) clear and PG_has_hwpoisoned still clear, then batch across poisoned tail and install a normal migration entry for it. Once memory_failure() publishes PG_has_hwpoisoned, the folio-level check does stop batching, as you said. It's just this window before publication that worries me ... Thanks, Lance >Thanks both. >. > >> >> Starting from a healthy first subpage, folio_migrate_pte_batch() can >> then batch across poisoned tail page. hwpoison only describes first >> subpage, so set_softleaf_ptes() installs a normal migration entry for >> poisoned page instead of an HWPoison entry ... >> >> Should folio_migrate_pte_batch() check PageHWPoison() on every candidate >> subpage and stop before a poisoned one? >> >> Cheers, Lance >> >> >>> + /* >>> + * If unmap fails, we need to restore the ptes. To avoid accidentally >>> + * upgrading write permissions for ptes that were not originally writable, >>> + * and to avoid losing the soft-dirty bit, use the appropriate FPB flags. >>> + */ >>> + nr = folio_pte_batch_flags(folio, vma, pvmw->pte, &pte, max_nr, >>> + FPB_RESPECT_WRITE | FPB_RESPECT_SOFT_DIRTY); >>> + >>> + /* Limit possible batch count to a uniform PageAnonExclusive value */ >>> + if (folio_test_anon(folio)) >>> + nr = page_anon_exclusive_batch(0, nr, subpage, anon_exclusive); >>> + >>> + return nr; >>> +} >>> + >>> /* >>> * @arg: enum ttu_flags will be passed to this argument. >>> * >>> @@ -2686,12 +2724,12 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, >>> { >>> struct mm_struct *mm = vma->vm_mm; >>> DEFINE_FOLIO_VMA_WALK(pvmw, folio, vma, address, 0); >>> - bool anon_exclusive, writable, ret = true; >>> + bool anon_exclusive, hwpoison, writable, ret = true; >>> pte_t pteval; >>> struct page *subpage; >>> struct mmu_notifier_range range; >>> enum ttu_flags flags = (enum ttu_flags)(long)arg; >>> - unsigned long pfn; >>> + unsigned long pfn, end_addr, nr_pages; >>> >>> /* >>> * When racing against e.g. zap_pte_range() on another cpu, >>> @@ -2744,11 +2782,8 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, >>> VM_BUG_ON_FOLIO(folio_test_hugetlb(folio) || >>> !folio_test_pmd_mappable(folio), folio); >>> >>> - if (set_pmd_migration_entry(&pvmw, subpage)) { >>> - ret = false; >>> - page_vma_mapped_walk_done(&pvmw); >>> - break; >>> - } >>> + if (set_pmd_migration_entry(&pvmw, subpage)) >>> + goto walk_abort; >>> continue; >>> #endif >>> } >>> @@ -2773,10 +2808,25 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, >>> subpage = folio_page(folio, pfn - folio_pfn(folio)); >>> anon_exclusive = folio_test_anon(folio) && >>> PageAnonExclusive(subpage); >>> + /* >>> + * memory_failure() can set PageHWPoison concurrently without holding >>> + * the folio lock. Snapshot the flag here to decide whether to batch >>> + * PTEs or install hwpoison entry. >>> + */ >>> + hwpoison = PageHWPoison(subpage); >>> >>> + nr_pages = 1; >>> if (likely(pte_present(pteval))) { >>> - flush_cache_page(vma, address, pfn); >>> - /* Nuke the page table entry. */ >>> + if (!hwpoison) >>> + nr_pages = folio_migrate_pte_batch(folio, &pvmw, >>> + pteval, subpage, >>> + anon_exclusive); >>> + >>> + end_addr = address + nr_pages * PAGE_SIZE; >>> + flush_cache_range(vma, address, end_addr); >>> + >>> + /* Nuke the page table entries. */ >>> + pteval = get_and_clear_ptes(mm, address, pvmw.pte, nr_pages); >>> if (should_defer_flush(mm, flags)) { >>> /* >>> * We clear the PTE but do not flush so potentially >>> @@ -2786,11 +2836,9 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, >>> * transition on a cached TLB entry is written through >>> * and traps if the PTE is unmapped. >>> */ >>> - pteval = ptep_get_and_clear(mm, address, pvmw.pte); >>> - >>> - set_tlb_ubc_flush_pending(mm, pteval, address, address + PAGE_SIZE); >>> + set_tlb_ubc_flush_pending(mm, pteval, address, end_addr); >>> } else { >>> - pteval = ptep_clear_flush(vma, address, pvmw.pte); >>> + flush_tlb_range(vma, address, end_addr); >>> } >>> if (pte_dirty(pteval)) >>> folio_mark_dirty(folio); >>> @@ -2809,7 +2857,8 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, >>> /* Update high watermark before we lower rss */ >>> update_hiwater_rss(mm); >>> >>> - if (PageHWPoison(subpage)) { >>> + if (hwpoison) { >>> + VM_WARN_ON_ONCE(nr_pages != 1); >>> VM_WARN_ON_FOLIO(folio_is_device_private(folio), folio); >>> >>> pteval = swp_entry_to_pte(make_hwpoison_entry(subpage)); >>> @@ -2837,19 +2886,15 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, >>> * so we'll not check/care. >>> */ >>> if (arch_unmap_one(mm, vma, address, pteval) < 0) { >>> - set_pte_at(mm, address, pvmw.pte, pteval); >>> - ret = false; >>> - page_vma_mapped_walk_done(&pvmw); >>> - break; >>> + set_ptes(mm, address, pvmw.pte, pteval, nr_pages); >>> + goto walk_abort; >>> } >>> >>> - /* See folio_try_share_anon_rmap_pte(): clear PTE first. */ >>> + /* See folio_try_share_anon_rmap_ptes(): clear PTE first. */ >>> if (anon_exclusive && >>> - folio_try_share_anon_rmap_pte(folio, subpage)) { >>> - set_pte_at(mm, address, pvmw.pte, pteval); >>> - ret = false; >>> - page_vma_mapped_walk_done(&pvmw); >>> - break; >>> + folio_try_share_anon_rmap_ptes(folio, subpage, nr_pages)) { >>> + set_ptes(mm, address, pvmw.pte, pteval, nr_pages); >>> + goto walk_abort; >>> } >>> >>> /* >>> @@ -2859,19 +2904,31 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, >>> */ >>> swp_pte = make_migration_pte(subpage, pteval, >>> writable, anon_exclusive); >>> - set_pte_at(mm, address, pvmw.pte, swp_pte); >>> trace_set_migration_pte(address, pte_val(swp_pte), >>> folio_order(folio)); >>> + >>> + /* Set nr_pages migration entries, advancing the PFN. */ >>> + set_softleaf_ptes(mm, address, pvmw.pte, swp_pte, nr_pages); >>> /* >>> * No need to invalidate here it will synchronize on >>> * against the special swap migration pte. >>> */ >>> } >>> >>> - folio_remove_rmap_pte(folio, subpage, vma); >>> - if (vma->vm_flags & VM_LOCKED) >>> - mlock_drain_local(); >>> - folio_put(folio); >>> + finish_folio_unmap(vma, folio, subpage, nr_pages); >>> + >>> + /* >>> + * If we batched the entire folio, there is nothing left to >>> + * walk; stop right here. >>> + */ >>> + if (nr_pages == folio_nr_pages(folio)) >>> + goto walk_done; >>> + continue; >>> +walk_abort: >>> + ret = false; >>> +walk_done: >>> + page_vma_mapped_walk_done(&pvmw); >>> + break; >>> } >>> >>> mmu_notifier_invalidate_range_end(&range); >>> >>> -- >>> 2.43.0 >>> >>> >> . >> > >