[PATCH] arm64: mm: Fix the break-before-make flush range for erratum 2645198

[PATCH] arm64: mm: Fix the break-before-make flush range for erratum 2645198

[Andrea Parri]

modify_prot_start_ptes() performs the break-before-make TLB invalidation
required by erratum 2645198 with __flush_tlb_range(), whose third
argument is the end address of the range.  It passes nr * PAGE_SIZE
instead of addr + nr * PAGE_SIZE, so __do_flush_tlb_range() computes the
page count as (nr * PAGE_SIZE - addr) >> PAGE_SHIFT.

For addr > nr * PAGE_SIZE that subtraction underflows, the page count
exceeds the batching limit and the flush degenerates to flush_tlb_mm(),
so a single-page mprotect broadcasts an ASID-wide invalidation and a
full-range mmu notifier call.

For addr <= nr * PAGE_SIZE only [addr, nr * PAGE_SIZE) is invalidated,
and when the cleared batch starts below nr * PAGE_SIZE the tail is left
in the TLB.  The workaround then no longer covers the whole batch, and
for addr == nr * PAGE_SIZE the flush is empty.

On affected Cortex-A715 CPUs, this can corrupt ESR_ELx and FAR_ELx on the
next instruction abort caused by a permission fault.

Pass addr + nr * PAGE_SIZE as the end address.

Fixes: 7efa1cd5f89b5 ("arm64: add batched versions of ptep_modify_prot_start/commit")
Cc: stable@vger.kernel.org
Assisted-by: LLM
Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
---
 arch/arm64/mm/mmu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
index 79d90226fd5dc..d4384131e10d8 100644
--- a/arch/arm64/mm/mmu.c
+++ b/arch/arm64/mm/mmu.c
@@ -2295,7 +2295,7 @@ pte_t modify_prot_start_ptes(struct vm_area_struct *vma, unsigned long addr,
 		 * in cases where cpu is affected with errata #2645198.
 		 */
 		if (pte_accessible(vma->vm_mm, pte) && pte_user_exec(pte))
-			__flush_tlb_range(vma, addr, nr * PAGE_SIZE,
+			__flush_tlb_range(vma, addr, addr + nr * PAGE_SIZE,
 					  PAGE_SIZE, 3, TLBF_NOWALKCACHE);
 	}
 
-- 
2.53.0

Re: [PATCH] arm64: mm: Fix the break-before-make flush range for erratum 2645198

[Dev Jain]

On 23/09/26 8:06 pm, Andrea Parri wrote:
> modify_prot_start_ptes() performs the break-before-make TLB invalidation
> required by erratum 2645198 with __flush_tlb_range(), whose third
> argument is the end address of the range.  It passes nr * PAGE_SIZE
> instead of addr + nr * PAGE_SIZE, so __do_flush_tlb_range() computes the
> page count as (nr * PAGE_SIZE - addr) >> PAGE_SHIFT.
> 
> For addr > nr * PAGE_SIZE that subtraction underflows, the page count
> exceeds the batching limit and the flush degenerates to flush_tlb_mm(),
> so a single-page mprotect broadcasts an ASID-wide invalidation and a
> full-range mmu notifier call.
> 
> For addr <= nr * PAGE_SIZE only [addr, nr * PAGE_SIZE) is invalidated,
> and when the cleared batch starts below nr * PAGE_SIZE the tail is left
> in the TLB.  The workaround then no longer covers the whole batch, and
> for addr == nr * PAGE_SIZE the flush is empty.
> 
> On affected Cortex-A715 CPUs, this can corrupt ESR_ELx and FAR_ELx on the
> next instruction abort caused by a permission fault.
> 
> Pass addr + nr * PAGE_SIZE as the end address.
> 
> Fixes: 7efa1cd5f89b5 ("arm64: add batched versions of ptep_modify_prot_start/commit")
> Cc: stable@vger.kernel.org
> Assisted-by: LLM
> Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
> ---

Thanks, such typos should be filtered out by LLMs nowadays : )

Reviewed-by: Dev Jain <dev.jain@arm.com>


>  arch/arm64/mm/mmu.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
> index 79d90226fd5dc..d4384131e10d8 100644
> --- a/arch/arm64/mm/mmu.c
> +++ b/arch/arm64/mm/mmu.c
> @@ -2295,7 +2295,7 @@ pte_t modify_prot_start_ptes(struct vm_area_struct *vma, unsigned long addr,
>  		 * in cases where cpu is affected with errata #2645198.
>  		 */
>  		if (pte_accessible(vma->vm_mm, pte) && pte_user_exec(pte))
> -			__flush_tlb_range(vma, addr, nr * PAGE_SIZE,
> +			__flush_tlb_range(vma, addr, addr + nr * PAGE_SIZE,
>  					  PAGE_SIZE, 3, TLBF_NOWALKCACHE);
>  	}
>  

Re: [PATCH] arm64: mm: Fix the break-before-make flush range for erratum 2645198

[Catalin Marinas]

On Wed, Sep 23, 2026 at 04:36:32PM +0200, Andrea Parri wrote:
> modify_prot_start_ptes() performs the break-before-make TLB invalidation
> required by erratum 2645198 with __flush_tlb_range(), whose third
> argument is the end address of the range.  It passes nr * PAGE_SIZE
> instead of addr + nr * PAGE_SIZE, so __do_flush_tlb_range() computes the
> page count as (nr * PAGE_SIZE - addr) >> PAGE_SHIFT.
> 
> For addr > nr * PAGE_SIZE that subtraction underflows, the page count
> exceeds the batching limit and the flush degenerates to flush_tlb_mm(),
> so a single-page mprotect broadcasts an ASID-wide invalidation and a
> full-range mmu notifier call.
> 
> For addr <= nr * PAGE_SIZE only [addr, nr * PAGE_SIZE) is invalidated,
> and when the cleared batch starts below nr * PAGE_SIZE the tail is left
> in the TLB.  The workaround then no longer covers the whole batch, and
> for addr == nr * PAGE_SIZE the flush is empty.
> 
> On affected Cortex-A715 CPUs, this can corrupt ESR_ELx and FAR_ELx on the
> next instruction abort caused by a permission fault.
> 
> Pass addr + nr * PAGE_SIZE as the end address.
> 
> Fixes: 7efa1cd5f89b5 ("arm64: add batched versions of ptep_modify_prot_start/commit")
> Cc: stable@vger.kernel.org
> Assisted-by: LLM
> Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
> ---
>  arch/arm64/mm/mmu.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
> index 79d90226fd5dc..d4384131e10d8 100644
> --- a/arch/arm64/mm/mmu.c
> +++ b/arch/arm64/mm/mmu.c
> @@ -2295,7 +2295,7 @@ pte_t modify_prot_start_ptes(struct vm_area_struct *vma, unsigned long addr,
>  		 * in cases where cpu is affected with errata #2645198.
>  		 */
>  		if (pte_accessible(vma->vm_mm, pte) && pte_user_exec(pte))
> -			__flush_tlb_range(vma, addr, nr * PAGE_SIZE,
> +			__flush_tlb_range(vma, addr, addr + nr * PAGE_SIZE,
>  					  PAGE_SIZE, 3, TLBF_NOWALKCACHE);

Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>

Re: [PATCH] arm64: mm: Fix the break-before-make flush range for erratum 2645198

[Will Deacon]

On Wed, 23 Sep 2026 16:36:32 +0200, Andrea Parri wrote:
> modify_prot_start_ptes() performs the break-before-make TLB invalidation
> required by erratum 2645198 with __flush_tlb_range(), whose third
> argument is the end address of the range.  It passes nr * PAGE_SIZE
> instead of addr + nr * PAGE_SIZE, so __do_flush_tlb_range() computes the
> page count as (nr * PAGE_SIZE - addr) >> PAGE_SHIFT.
> 
> For addr > nr * PAGE_SIZE that subtraction underflows, the page count
> exceeds the batching limit and the flush degenerates to flush_tlb_mm(),
> so a single-page mprotect broadcasts an ASID-wide invalidation and a
> full-range mmu notifier call.
> 
> [...]

Applied to arm64 (for-next/fixes), thanks!

[1/1] arm64: mm: Fix the break-before-make flush range for erratum 2645198
      https://git.kernel.org/arm64/c/1fef81669147

Cheers,
-- 
Will

https://fixes.arm64.dev
https://next.arm64.dev
https://will.arm64.dev