From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from out30-111.freemail.mail.aliyun.com (out30-111.freemail.mail.aliyun.com [115.124.30.111]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8F1C28C1F for ; Thu, 25 Dec 2025 01:01:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.111 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766624521; cv=none; b=hWxw/vDp2bZaqu0w1MyyFB0yj0O97CK6qfpuxzrrKSyPjGqppnQC+1WqLP7/qpz4aOyMpoB4XqPXeD68yNfiKgLXDimGjYpo6GApXSF8kd1oaCsdTQG1F2gAQ1/wDrMcENy5drcO1F7kF7g0usEJh8HR/angAo70TmeU9C4j+sk= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766624521; c=relaxed/simple; bh=dH2i8i2ASIxkI24TFRzVUf6ip5/gYimckROkF5rBggs=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=IXB7fGBb6NYxu2lGD/Io2Kb+YTpfcoECZg6zwO6/TXbRewIG4I6Ca5CEYPQY8Za8R5gZLMHe65SK/p05kgKL9tibMdrq9UqM2dTxIAppc8DTHGoHEnamxLE/o21jHVZPRHUqR2qvbg1u+8dqVH0UGMaMsg0xjXvbcjNUAWXvWrs= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=GREM+rTI; arc=none smtp.client-ip=115.124.30.111 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="GREM+rTI" DKIM-Signature:v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1766624510; h=Message-ID:Date:MIME-Version:Subject:To:From:Content-Type; bh=2fy7EXMdCyEH27NzJMvO2tdqkUATW53yXrUXnni0WAs=; b=GREM+rTI4/0Do6Yvi99rN2SIMHDTfy2XkDqdpNz6HPtCo9LFx3kVw9UzCSU5MaAwCX9SkOhrNdEMgx+YKKaB+1h4Ii6EzWc257LCDVleKSXYYNb7bBiSCUfSRCiHAa4ugNb7XV0U8QsKuY6zoss9s2DcT/jA4lo5oDxUANcvEqk= Received: from 30.74.144.121(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0Wvci210_1766624507 cluster:ay36) by smtp.aliyun-inc.com; Thu, 25 Dec 2025 09:01:48 +0800 Message-ID: <43b6ef51-c88f-4b0c-b512-c7195afdd31b@linux.alibaba.com> Date: Thu, 25 Dec 2025 09:01:47 +0800 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v4 1/5] mm: rmap: support batched checks of the references for large folios To: Ryan Roberts , akpm@linux-foundation.org, david@kernel.org, catalin.marinas@arm.com, will@kernel.org Cc: lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, vbabka@suse.cz, rppt@kernel.org, surenb@google.com, mhocko@suse.com, riel@surriel.com, harry.yoo@oracle.com, jannh@google.com, willy@infradead.org, baohua@kernel.org, dev.jain@arm.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org References: From: Baolin Wang In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hi Ryan, On 2025/12/24 21:24, Ryan Roberts wrote: > On 23/12/2025 05:48, Baolin Wang wrote: >> Currently, folio_referenced_one() always checks the young flag for each PTE >> sequentially, which is inefficient for large folios. This inefficiency is >> especially noticeable when reclaiming clean file-backed large folios, where >> folio_referenced() is observed as a significant performance hotspot. >> >> Moreover, on Arm64 architecture, which supports contiguous PTEs, there is already >> an optimization to clear the young flags for PTEs within a contiguous range. >> However, this is not sufficient. We can extend this to perform batched operations >> for the entire large folio (which might exceed the contiguous range: CONT_PTE_SIZE). >> >> Introduce a new API: clear_flush_young_ptes() to facilitate batched checking >> of the young flags and flushing TLB entries, thereby improving performance >> during large folio reclamation. And it will be overridden by the architecture >> that implements a more efficient batch operation in the following patches. >> >> Signed-off-by: Baolin Wang > > With the 2 niggles below addressed: > > Reviewed-by: Ryan Roberts Thanks. >> --- >> include/linux/mmu_notifier.h | 9 +++++---- >> include/linux/pgtable.h | 35 +++++++++++++++++++++++++++++++++++ >> mm/rmap.c | 29 +++++++++++++++++++++++++++-- >> 3 files changed, 67 insertions(+), 6 deletions(-) >> >> diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h >> index d1094c2d5fb6..dbbdcef4abf1 100644 >> --- a/include/linux/mmu_notifier.h >> +++ b/include/linux/mmu_notifier.h >> @@ -515,16 +515,17 @@ static inline void mmu_notifier_range_init_owner( >> range->owner = owner; >> } >> >> -#define ptep_clear_flush_young_notify(__vma, __address, __ptep) \ >> +#define ptep_clear_flush_young_notify(__vma, __address, __ptep, __nr) \ > > I think I previously suggested that this should be renamed to > clear_flush_young_ptes_notify() given that it is now a batch operation. Were > others against that or did you forget? Sorry, I missed this comment. Yes, sounds reasonable to me. Will do in the next version. >> ({ \ >> int __young; \ >> struct vm_area_struct *___vma = __vma; \ >> unsigned long ___address = __address; \ >> - __young = ptep_clear_flush_young(___vma, ___address, __ptep); \ >> + unsigned int ___nr = __nr; \ >> + __young = clear_flush_young_ptes(___vma, ___address, __ptep, ___nr); \ >> __young |= mmu_notifier_clear_flush_young(___vma->vm_mm, \ >> ___address, \ >> ___address + \ >> - PAGE_SIZE); \ >> + ___nr * PAGE_SIZE); \ >> __young; \ >> }) >> >> @@ -650,7 +651,7 @@ static inline void mmu_notifier_subscriptions_destroy(struct mm_struct *mm) >> >> #define mmu_notifier_range_update_to_read_only(r) false >> >> -#define ptep_clear_flush_young_notify ptep_clear_flush_young >> +#define ptep_clear_flush_young_notify clear_flush_young_ptes >> #define pmdp_clear_flush_young_notify pmdp_clear_flush_young >> #define ptep_clear_young_notify ptep_test_and_clear_young >> #define pmdp_clear_young_notify pmdp_test_and_clear_young >> diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h >> index 2f0dd3a4ace1..fcf7a7820061 100644 >> --- a/include/linux/pgtable.h >> +++ b/include/linux/pgtable.h >> @@ -1087,6 +1087,41 @@ static inline void wrprotect_ptes(struct mm_struct *mm, unsigned long addr, >> } >> #endif >> >> +#ifndef clear_flush_young_ptes >> +/** >> + * clear_flush_young_ptes - Clear the access bit and perform a TLB flush for PTEs >> + * that map consecutive pages of the same folio. >> + * @vma: The virtual memory area the pages are mapped into. >> + * @addr: Address the first page is mapped at. >> + * @ptep: Page table pointer for the first entry. >> + * @nr: Number of entries to clear access bit. >> + * >> + * May be overridden by the architecture; otherwise, implemented as a simple >> + * loop over ptep_clear_flush_young(). >> + * >> + * Note that PTE bits in the PTE range besides the PFN can differ. For example, >> + * some PTEs might be write-protected. >> + * >> + * Context: The caller holds the page table lock. The PTEs map consecutive >> + * pages that belong to the same folio. The PTEs are all in the same PMD. >> + */ >> +static inline int clear_flush_young_ptes(struct vm_area_struct *vma, >> + unsigned long addr, pte_t *ptep, >> + unsigned int nr) >> +{ >> + int young; >> + >> + young = ptep_clear_flush_young(vma, addr, ptep); >> + while (--nr) { >> + ptep++; >> + addr += PAGE_SIZE; >> + young |= ptep_clear_flush_young(vma, addr, ptep); >> + } > > I think it's better to avoid the two ptep_clear_flush_young() calls if we can. > Personally I think we should just go for the simple: > > for (i = 0; i < nr; ++i, ++ptep, addr += PAGE_SIZE) > young |= ptep_clear_flush_young(vma, addr, ptep); ACK. Thanks for reviewing.