From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from canpmsgout06.his.huawei.com (canpmsgout06.his.huawei.com [113.46.200.221]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB3BE377AB7 for ; Tue, 18 Aug 2026 08:55:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=113.46.200.221 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787043318; cv=none; b=HbL8rSETkERDXBF/6t5iFZk5oG+C1ADplAEnd/FKn39PWtzCch82Jploh0WUEVi46gH0LjEmmiaIPUczM36Iz6/RQtgPiNALINCvYXf4b7cyTADrD0TjwR6GmGvXIAVYNM3xR1bReBe2El9xJwyFlmovzYbE7oSwfk1DHvJNd08= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787043318; c=relaxed/simple; bh=UV6D/yowIE+MkIxhUiz9Qa6aR+BZeZYquILIcg7iQl0=; h=Subject:To:CC:References:From:Message-ID:Date:MIME-Version: In-Reply-To:Content-Type; b=myJFI1FrPyJ20Yg2SA92OP807jVUvW+nCbf+NMSFL7QJg/oQXm4HhDzyrE8OnWp+MbWbbZjnMVw1pzeaTgPPgxUfSd3O4eWVFG/HIJv5Tv8w7aWqhwicunOnhwl+zjT6cpOefkmpaYA/NDlyiHu+0v4Ye+4LfOry8C8eg7Z5Nk4= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b=khK6KdnF; arc=none smtp.client-ip=113.46.200.221 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b="khK6KdnF" dkim-signature: v=1; a=rsa-sha256; d=huawei.com; s=dkim; c=relaxed/relaxed; q=dns/txt; h=From; bh=N4mZniCDA8t7+WaShjbdSaKjsh0mLEC6shWNiqjBdH4=; b=khK6KdnF9jQJ1MFIVA0f6gyRBr687iqbT2CxAuZF/Jzmz87t6A7YIJN4NU+W6PanhZsDqwBgU kf1ip0l6bQQSOeX7oHAwkrkl8ZEbumYCxNiFA5/zSynw0SGG0strcpvQuwG6vxkhFky4O9Kyy6v JOWRKKQivdzgoVNvU8GRpEA= Received: from mail.maildlp.com (unknown [172.19.163.104]) by canpmsgout06.his.huawei.com (SkyGuard) with ESMTPS id 4hPNYB1HLzzRhRG; Tue, 18 Aug 2026 16:44:30 +0800 (CST) Received: from dggemv712-chm.china.huawei.com (unknown [10.1.198.32]) by mail.maildlp.com (Postfix) with ESMTPS id 102A64058C; Tue, 18 Aug 2026 16:55:09 +0800 (CST) Received: from kwepemq500010.china.huawei.com (7.202.194.235) by dggemv712-chm.china.huawei.com (10.1.198.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Tue, 18 Aug 2026 16:55:08 +0800 Received: from [10.173.124.160] (10.173.124.160) by kwepemq500010.china.huawei.com (7.202.194.235) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Tue, 18 Aug 2026 16:55:07 +0800 Subject: Re: [PATCH v2 7/7] mm/rmap: batch the unmap of large folios in try_to_migrate_one() To: Lance Yang , CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , References: <20260813-migrate-rmap-batch-v2-7-3c5424c555c7@amd.com> <20260817091444.58876-1-lance.yang@linux.dev> From: Miaohe Lin Message-ID: <860941c2-287f-f88f-920c-6d99e2cd4483@huawei.com> Date: Tue, 18 Aug 2026 16:55:06 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.6.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20260817091444.58876-1-lance.yang@linux.dev> Content-Type: text/plain; charset="utf-8" Content-Language: en-US Content-Transfer-Encoding: 7bit X-ClientProxiedBy: kwepems500001.china.huawei.com (7.221.188.70) To kwepemq500010.china.huawei.com (7.202.194.235) On 2026/8/17 17:14, Lance Yang wrote: > +Cc Miaohe > > On Thu, Aug 13, 2026 at 04:23:18AM +0000, Shivank Garg wrote: >> try_to_migrate_one() converts present PTEs to migration entries one at a >> time. For a PTE-mapped large folio, this repeat calls to ptep clear+flush, >> the migration entry build and set, folio_remove_rmap_pte() and folio_put(), >> each re-entering page_vma_mapped_walk() once per base page (256 times for >> 1M folio). >> >> Mirror try_to_unmap_one() to introduce folio_migrate_pte_batch() to detect >> eligible batch for PTEs mapping conseuctive subpages of a large folios, >> and convert the whole batch in one shot using the batched helpers. >> >> A side-effect of this change is trace_set_migration_pte() will record >> one event per batched run instead of earlier behavior of one per base page. >> >> Signed-off-by: Shivank Garg >> --- >> mm/rmap.c | 115 ++++++++++++++++++++++++++++++++++++++++++++++---------------- >> 1 file changed, 86 insertions(+), 29 deletions(-) >> >> diff --git a/mm/rmap.c b/mm/rmap.c >> index 35752a70f3a0..63b885c0b7ef 100644 >> --- a/mm/rmap.c >> +++ b/mm/rmap.c >> @@ -2675,6 +2675,44 @@ static bool try_to_migrate_hugetlb_one(struct folio *folio, >> return ret; >> } >> >> +static inline unsigned int folio_migrate_pte_batch(struct folio *folio, >> + struct page_vma_mapped_walk *pvmw, pte_t pte, >> + struct page *subpage, bool anon_exclusive) >> +{ >> + unsigned long end_addr, addr = pvmw->address; >> + struct vm_area_struct *vma = pvmw->vma; >> + unsigned int max_nr, nr; >> + >> +#ifdef __HAVE_ARCH_UNMAP_ONE >> + /* Cannot batch unmap if arch_unmap_one() is defined. */ >> + return 1; >> +#endif >> + >> + if (!folio_test_large(folio)) >> + return 1; >> + if (folio_is_zone_device(folio) || folio_test_has_hwpoisoned(folio)) >> + return 1; >> + if (pte_unused(pte)) >> + return 1; >> + >> + /* We may only batch within a single VMA and a single page table. */ >> + end_addr = pmd_addr_end(addr, vma->vm_end); >> + max_nr = (end_addr - addr) >> PAGE_SHIFT; > > Hmm ... can this still batch over a poisoned tail page? > > memory_failure() sets PageHWPoison() before taking folio lock, but > cannot set PG_has_hwpoisoned until it acquires and releases that lock. > > So tail page can already be poisoned while folio_test_has_hwpoisoned() > still returns false ... no? When memory error hits thp pages, memory_failure() first set PG_has_hwpoisoned and then tries to split thp pages. And try_to_migrate() will be called to set migration entries for anon pages. Does folio_migrate_pte_batch() work on this case? If so, the folio_test_has_hwpoisoned() check above could catch the bad pages? Or do you worry about the scene that meory error hits a thp while it's under migration? Thanks both. . > > Starting from a healthy first subpage, folio_migrate_pte_batch() can > then batch across poisoned tail page. hwpoison only describes first > subpage, so set_softleaf_ptes() installs a normal migration entry for > poisoned page instead of an HWPoison entry ... > > Should folio_migrate_pte_batch() check PageHWPoison() on every candidate > subpage and stop before a poisoned one? > > Cheers, Lance > > >> + /* >> + * If unmap fails, we need to restore the ptes. To avoid accidentally >> + * upgrading write permissions for ptes that were not originally writable, >> + * and to avoid losing the soft-dirty bit, use the appropriate FPB flags. >> + */ >> + nr = folio_pte_batch_flags(folio, vma, pvmw->pte, &pte, max_nr, >> + FPB_RESPECT_WRITE | FPB_RESPECT_SOFT_DIRTY); >> + >> + /* Limit possible batch count to a uniform PageAnonExclusive value */ >> + if (folio_test_anon(folio)) >> + nr = page_anon_exclusive_batch(0, nr, subpage, anon_exclusive); >> + >> + return nr; >> +} >> + >> /* >> * @arg: enum ttu_flags will be passed to this argument. >> * >> @@ -2686,12 +2724,12 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, >> { >> struct mm_struct *mm = vma->vm_mm; >> DEFINE_FOLIO_VMA_WALK(pvmw, folio, vma, address, 0); >> - bool anon_exclusive, writable, ret = true; >> + bool anon_exclusive, hwpoison, writable, ret = true; >> pte_t pteval; >> struct page *subpage; >> struct mmu_notifier_range range; >> enum ttu_flags flags = (enum ttu_flags)(long)arg; >> - unsigned long pfn; >> + unsigned long pfn, end_addr, nr_pages; >> >> /* >> * When racing against e.g. zap_pte_range() on another cpu, >> @@ -2744,11 +2782,8 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, >> VM_BUG_ON_FOLIO(folio_test_hugetlb(folio) || >> !folio_test_pmd_mappable(folio), folio); >> >> - if (set_pmd_migration_entry(&pvmw, subpage)) { >> - ret = false; >> - page_vma_mapped_walk_done(&pvmw); >> - break; >> - } >> + if (set_pmd_migration_entry(&pvmw, subpage)) >> + goto walk_abort; >> continue; >> #endif >> } >> @@ -2773,10 +2808,25 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, >> subpage = folio_page(folio, pfn - folio_pfn(folio)); >> anon_exclusive = folio_test_anon(folio) && >> PageAnonExclusive(subpage); >> + /* >> + * memory_failure() can set PageHWPoison concurrently without holding >> + * the folio lock. Snapshot the flag here to decide whether to batch >> + * PTEs or install hwpoison entry. >> + */ >> + hwpoison = PageHWPoison(subpage); >> >> + nr_pages = 1; >> if (likely(pte_present(pteval))) { >> - flush_cache_page(vma, address, pfn); >> - /* Nuke the page table entry. */ >> + if (!hwpoison) >> + nr_pages = folio_migrate_pte_batch(folio, &pvmw, >> + pteval, subpage, >> + anon_exclusive); >> + >> + end_addr = address + nr_pages * PAGE_SIZE; >> + flush_cache_range(vma, address, end_addr); >> + >> + /* Nuke the page table entries. */ >> + pteval = get_and_clear_ptes(mm, address, pvmw.pte, nr_pages); >> if (should_defer_flush(mm, flags)) { >> /* >> * We clear the PTE but do not flush so potentially >> @@ -2786,11 +2836,9 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, >> * transition on a cached TLB entry is written through >> * and traps if the PTE is unmapped. >> */ >> - pteval = ptep_get_and_clear(mm, address, pvmw.pte); >> - >> - set_tlb_ubc_flush_pending(mm, pteval, address, address + PAGE_SIZE); >> + set_tlb_ubc_flush_pending(mm, pteval, address, end_addr); >> } else { >> - pteval = ptep_clear_flush(vma, address, pvmw.pte); >> + flush_tlb_range(vma, address, end_addr); >> } >> if (pte_dirty(pteval)) >> folio_mark_dirty(folio); >> @@ -2809,7 +2857,8 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, >> /* Update high watermark before we lower rss */ >> update_hiwater_rss(mm); >> >> - if (PageHWPoison(subpage)) { >> + if (hwpoison) { >> + VM_WARN_ON_ONCE(nr_pages != 1); >> VM_WARN_ON_FOLIO(folio_is_device_private(folio), folio); >> >> pteval = swp_entry_to_pte(make_hwpoison_entry(subpage)); >> @@ -2837,19 +2886,15 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, >> * so we'll not check/care. >> */ >> if (arch_unmap_one(mm, vma, address, pteval) < 0) { >> - set_pte_at(mm, address, pvmw.pte, pteval); >> - ret = false; >> - page_vma_mapped_walk_done(&pvmw); >> - break; >> + set_ptes(mm, address, pvmw.pte, pteval, nr_pages); >> + goto walk_abort; >> } >> >> - /* See folio_try_share_anon_rmap_pte(): clear PTE first. */ >> + /* See folio_try_share_anon_rmap_ptes(): clear PTE first. */ >> if (anon_exclusive && >> - folio_try_share_anon_rmap_pte(folio, subpage)) { >> - set_pte_at(mm, address, pvmw.pte, pteval); >> - ret = false; >> - page_vma_mapped_walk_done(&pvmw); >> - break; >> + folio_try_share_anon_rmap_ptes(folio, subpage, nr_pages)) { >> + set_ptes(mm, address, pvmw.pte, pteval, nr_pages); >> + goto walk_abort; >> } >> >> /* >> @@ -2859,19 +2904,31 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, >> */ >> swp_pte = make_migration_pte(subpage, pteval, >> writable, anon_exclusive); >> - set_pte_at(mm, address, pvmw.pte, swp_pte); >> trace_set_migration_pte(address, pte_val(swp_pte), >> folio_order(folio)); >> + >> + /* Set nr_pages migration entries, advancing the PFN. */ >> + set_softleaf_ptes(mm, address, pvmw.pte, swp_pte, nr_pages); >> /* >> * No need to invalidate here it will synchronize on >> * against the special swap migration pte. >> */ >> } >> >> - folio_remove_rmap_pte(folio, subpage, vma); >> - if (vma->vm_flags & VM_LOCKED) >> - mlock_drain_local(); >> - folio_put(folio); >> + finish_folio_unmap(vma, folio, subpage, nr_pages); >> + >> + /* >> + * If we batched the entire folio, there is nothing left to >> + * walk; stop right here. >> + */ >> + if (nr_pages == folio_nr_pages(folio)) >> + goto walk_done; >> + continue; >> +walk_abort: >> + ret = false; >> +walk_done: >> + page_vma_mapped_walk_done(&pvmw); >> + break; >> } >> >> mmu_notifier_invalidate_range_end(&range); >> >> -- >> 2.43.0 >> >> > . >