[PATCH] riscv: mm: Limit per-page TLB flushes for large PTE batches

[PATCH] riscv: mm: Limit per-page TLB flushes for large PTE batches

[hu.shengming]

From: Shengming Hu <hu.shengming@zte.com.cn>

On systems without Svvptc, update_mmu_cache_range() performs one local
TLB invalidation per PTE. With 4 KiB pages, installing a PTE-mapped
1 MiB anonymous folio requires 256 SFENCE.VMA instructions, or 256
SINVAL.VMA instructions bracketed by SFENCE.W.INVAL and
SFENCE.INVAL.IR.

Apply the existing local TLB range-flush policy to PTE batch updates.
When nr exceeds tlb_flush_all_threshold, use local_flush_tlb_all_asid()
to replace per-page invalidations with a single ASID-wide SFENCE.VMA.
This orders the PTE writes before subsequent implicit page-table
accesses and invalidates translations for the entire batch. As with
range flushes, this may also evict unrelated translations for the same
ASID.

Benchmark results from will-it-scale's page_fault1_processes under
QEMU 9.1.93 (qemu-9.2.0-0.3.rc3.fc40), with svvptc=false and
CONFIG_TRANSPARENT_HUGEPAGE=y (higher is better):

Folio size      Before       After     Change
=============================================
4 KiB           116537      113503      -2.6%
512 KiB        1431962     1977519     +38.1%
1 MiB          1528457     2208209     +44.5%

Signed-off-by: Shengming Hu <hu.shengming@zte.com.cn>
---
 arch/riscv/include/asm/pgtable.h | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index 2ebd5fa5adc5..ebe42fcf35ae 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -571,6 +571,11 @@ static inline void update_mmu_cache_range(struct vm_fault *vmf,
 	if (riscv_has_extension_unlikely(RISCV_ISA_EXT_SVVPTC))
 		return;

+	if (nr > tlb_flush_all_threshold) {
+		local_flush_tlb_all_asid(asid);
+		return;
+	}
+
 	if (riscv_has_extension_unlikely(RISCV_ISA_EXT_SVINVAL)) {
 		local_sfence_w_inval();
 		while (nr--)
-- 
2.25.1

Re: [PATCH] riscv: mm: Limit per-page TLB flushes for large PTE batches

[Yicong Yang]

On 11/12/26 11:05 AM, hu.shengming@zte.com.cn wrote:
> From: Shengming Hu <hu.shengming@zte.com.cn>
> 
> On systems without Svvptc, update_mmu_cache_range() performs one local
> TLB invalidation per PTE. With 4 KiB pages, installing a PTE-mapped
> 1 MiB anonymous folio requires 256 SFENCE.VMA instructions, or 256
> SINVAL.VMA instructions bracketed by SFENCE.W.INVAL and
> SFENCE.INVAL.IR.
> 
> Apply the existing local TLB range-flush policy to PTE batch updates.
> When nr exceeds tlb_flush_all_threshold, use local_flush_tlb_all_asid()
> to replace per-page invalidations with a single ASID-wide SFENCE.VMA.
> This orders the PTE writes before subsequent implicit page-table
> accesses and invalidates translations for the entire batch. As with
> range flushes, this may also evict unrelated translations for the same
> ASID.
> 
> Benchmark results from will-it-scale's page_fault1_processes under
> QEMU 9.1.93 (qemu-9.2.0-0.3.rc3.fc40), with svvptc=false and
> CONFIG_TRANSPARENT_HUGEPAGE=y (higher is better):
> 
> Folio size      Before       After     Change
> =============================================
> 4 KiB           116537      113503      -2.6%
> 512 KiB        1431962     1977519     +38.1%
> 1 MiB          1528457     2208209     +44.5%
> 
> Signed-off-by: Shengming Hu <hu.shengming@zte.com.cn>
> ---
>  arch/riscv/include/asm/pgtable.h | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
> index 2ebd5fa5adc5..ebe42fcf35ae 100644
> --- a/arch/riscv/include/asm/pgtable.h
> +++ b/arch/riscv/include/asm/pgtable.h
> @@ -571,6 +571,11 @@ static inline void update_mmu_cache_range(struct vm_fault *vmf,
>  	if (riscv_has_extension_unlikely(RISCV_ISA_EXT_SVVPTC))
>  		return;
> 
> +	if (nr > tlb_flush_all_threshold) {
> +		local_flush_tlb_all_asid(asid);
> +		return;
> +	}
> +

could we reuse local_flush_tlb_range_threshold_asid() or
local_flush_tlb_range_asid()? now the logic is exactly
the same.

thanks.

>  	if (riscv_has_extension_unlikely(RISCV_ISA_EXT_SVINVAL)) {
>  		local_sfence_w_inval();
>  		while (nr--)

Re: [PATCH] riscv: mm: Limit per-page TLB flushes for large PTE batches

[Jisheng Zhang]

On Thu, Nov 12, 2026 at 11:05:25AM +0800, hu.shengming@zte.com.cn wrote:
> From: Shengming Hu <hu.shengming@zte.com.cn>
> 
> On systems without Svvptc, update_mmu_cache_range() performs one local
> TLB invalidation per PTE. With 4 KiB pages, installing a PTE-mapped
> 1 MiB anonymous folio requires 256 SFENCE.VMA instructions, or 256
> SINVAL.VMA instructions bracketed by SFENCE.W.INVAL and
> SFENCE.INVAL.IR.
> 
> Apply the existing local TLB range-flush policy to PTE batch updates.
> When nr exceeds tlb_flush_all_threshold, use local_flush_tlb_all_asid()
> to replace per-page invalidations with a single ASID-wide SFENCE.VMA.
> This orders the PTE writes before subsequent implicit page-table
> accesses and invalidates translations for the entire batch. As with
> range flushes, this may also evict unrelated translations for the same
> ASID.
> 
> Benchmark results from will-it-scale's page_fault1_processes under
> QEMU 9.1.93 (qemu-9.2.0-0.3.rc3.fc40), with svvptc=false and
> CONFIG_TRANSPARENT_HUGEPAGE=y (higher is better):

Can you test on real hw rather than on qemu to get the performance numbers?

> 
> Folio size      Before       After     Change
> =============================================
> 4 KiB           116537      113503      -2.6%
> 512 KiB        1431962     1977519     +38.1%
> 1 MiB          1528457     2208209     +44.5%

Since one page is the most case, if we really regressed about 2.6%, then I
think we need to evaluate whether the gain deserves the pay.

> 
> Signed-off-by: Shengming Hu <hu.shengming@zte.com.cn>
> ---
>  arch/riscv/include/asm/pgtable.h | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
> index 2ebd5fa5adc5..ebe42fcf35ae 100644
> --- a/arch/riscv/include/asm/pgtable.h
> +++ b/arch/riscv/include/asm/pgtable.h
> @@ -571,6 +571,11 @@ static inline void update_mmu_cache_range(struct vm_fault *vmf,
>  	if (riscv_has_extension_unlikely(RISCV_ISA_EXT_SVVPTC))
>  		return;
> 
> +	if (nr > tlb_flush_all_threshold) {

unlikely(...)

> +		local_flush_tlb_all_asid(asid);
> +		return;
> +	}
> +
>  	if (riscv_has_extension_unlikely(RISCV_ISA_EXT_SVINVAL)) {
>  		local_sfence_w_inval();
>  		while (nr--)
> -- 
> 2.25.1
> 
> _______________________________________________
> linux-riscv mailing list
> linux-riscv@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-riscv