* [PATCH 0/3] arm64: Missing barriers in kernel page-table updates
@ 2026-11-20 12:16 Fuad Tabba
2026-11-20 12:16 ` [PATCH 1/3] arm64: pi: Add an ISB after mapping the FDT in the initial ID map Fuad Tabba
` (2 more replies)
0 siblings, 3 replies; 7+ messages in thread
From: Fuad Tabba @ 2026-11-20 12:16 UTC (permalink / raw)
To: Catalin Marinas, Will Deacon
Cc: Mark Rutland, Ard Biesheuvel, Ryan Roberts, Yang Shi,
Kevin Brodsky, Anshuman Khandual, linux-arm-kernel, linux-kernel,
Fuad Tabba
Hi folks,
Three fixes for kernel mappings that are made valid and then used on
the same CPU before an ISB, so the access can take a translation fault.
Tested on QEMU. The runtime changes to the linear map that patch 3
covers were not exercised, since they require a CPU with BBML3.
Based on Linux 7.4-rc1 (6628e7d4c6b25).
Cheers,
/fuad
Fuad Tabba (3):
arm64: pi: Add an ISB after mapping the FDT in the initial ID map
arm64: mm: Add an ISB after remapping the KPTI page-table slot
arm64: mm: Don't defer the barriers when installing a fixmap entry
arch/arm64/kernel/pi/map_kernel.c | 1 +
arch/arm64/mm/fixmap.c | 4 +++-
arch/arm64/mm/proc.S | 1 +
3 files changed, 5 insertions(+), 1 deletion(-)
base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
--
2.47.3
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 1/3] arm64: pi: Add an ISB after mapping the FDT in the initial ID map
2026-11-20 12:16 [PATCH 0/3] arm64: Missing barriers in kernel page-table updates Fuad Tabba
@ 2026-11-20 12:16 ` Fuad Tabba
2026-11-21 10:42 ` Ryan Roberts
2026-11-20 12:16 ` [PATCH 2/3] arm64: mm: Add an ISB after remapping the KPTI page-table slot Fuad Tabba
2026-11-20 12:16 ` [PATCH 3/3] arm64: mm: Don't defer the barriers when installing a fixmap entry Fuad Tabba
2 siblings, 1 reply; 7+ messages in thread
From: Fuad Tabba @ 2026-11-20 12:16 UTC (permalink / raw)
To: Catalin Marinas, Will Deacon
Cc: Mark Rutland, Ard Biesheuvel, Ryan Roberts, Yang Shi,
Kevin Brodsky, Anshuman Khandual, linux-arm-kernel, linux-kernel,
Fuad Tabba, stable
map_fdt() maps the FDT into the live initial ID map with only a DSB
before the caller reads it, so the read can fault on a translation made
speculatively before the mapping was written. This early in boot,
VBAR_EL1 doesn't yet point at the kernel's vectors, and the kernel fails
to boot.
Add an ISB after the DSB.
Fixes: 84b04d3e6bdbc ("arm64: kernel: Create initial ID map from C code")
Cc: stable@vger.kernel.org
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
arch/arm64/kernel/pi/map_kernel.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/arm64/kernel/pi/map_kernel.c b/arch/arm64/kernel/pi/map_kernel.c
index fb44cbdd2f292..f9338257adb7d 100644
--- a/arch/arm64/kernel/pi/map_kernel.c
+++ b/arch/arm64/kernel/pi/map_kernel.c
@@ -211,6 +211,7 @@ static void *__init map_fdt(phys_addr_t fdt)
fdt, PAGE_KERNEL, IDMAP_ROOT_LEVEL,
(pte_t *)init_idmap_pg_dir, false, 0);
dsb(ishst);
+ isb();
return (void *)fdt;
}
--
2.47.3
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 2/3] arm64: mm: Add an ISB after remapping the KPTI page-table slot
2026-11-20 12:16 [PATCH 0/3] arm64: Missing barriers in kernel page-table updates Fuad Tabba
2026-11-20 12:16 ` [PATCH 1/3] arm64: pi: Add an ISB after mapping the FDT in the initial ID map Fuad Tabba
@ 2026-11-20 12:16 ` Fuad Tabba
2026-11-21 10:48 ` Ryan Roberts
2026-11-20 12:16 ` [PATCH 3/3] arm64: mm: Don't defer the barriers when installing a fixmap entry Fuad Tabba
2 siblings, 1 reply; 7+ messages in thread
From: Fuad Tabba @ 2026-11-20 12:16 UTC (permalink / raw)
To: Catalin Marinas, Will Deacon
Cc: Mark Rutland, Ard Biesheuvel, Ryan Roberts, Yang Shi,
Kevin Brodsky, Anshuman Khandual, linux-arm-kernel, linux-kernel,
Fuad Tabba, stable
kpti_map_pgtbl remaps a temporary slot with break-before-make, and
kpti_mk_tbl_ng then reads the page tables through it. The make ends in
a DSB alone, so the reads can fault on a translation of the slot made
speculatively before the new entry was written. With the temporary page
tables in TTBR1 the kernel's vectors are unmapped, and the boot hangs.
Add an ISB after the DSB.
Fixes: 47546a1912fc4 ("arm64: mm: install KPTI nG mappings with MMU enabled")
Cc: stable@vger.kernel.org
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
arch/arm64/mm/proc.S | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index 12aacc74f7647..7ca36d59bf77f 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -262,6 +262,7 @@ SYM_FUNC_ALIAS(__pi_idmap_cpu_replace_ttbr1, idmap_cpu_replace_ttbr1)
orr pte, pte, pte_flags
str pte, [temp_pte, #8 * (\level + 2)]
dsb nshst
+ isb
.endm
/*
--
2.47.3
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 3/3] arm64: mm: Don't defer the barriers when installing a fixmap entry
2026-11-20 12:16 [PATCH 0/3] arm64: Missing barriers in kernel page-table updates Fuad Tabba
2026-11-20 12:16 ` [PATCH 1/3] arm64: pi: Add an ISB after mapping the FDT in the initial ID map Fuad Tabba
2026-11-20 12:16 ` [PATCH 2/3] arm64: mm: Add an ISB after remapping the KPTI page-table slot Fuad Tabba
@ 2026-11-20 12:16 ` Fuad Tabba
2026-11-21 11:01 ` Ryan Roberts
2 siblings, 1 reply; 7+ messages in thread
From: Fuad Tabba @ 2026-11-20 12:16 UTC (permalink / raw)
To: Catalin Marinas, Will Deacon
Cc: Mark Rutland, Ard Biesheuvel, Ryan Roberts, Yang Shi,
Kevin Brodsky, Anshuman Khandual, linux-arm-kernel, linux-kernel,
Fuad Tabba, stable
In lazy MMU mode, __set_pte(), which __set_fixmap() uses, only queues
its DSB and ISB, but a caller can dereference the slot as soon as
__set_fixmap() returns. On a CPU with BBML3, runtime changes to the
linear map are made in lazy MMU mode. With 4K pages and 39-bit VAs, for
example, such a change can update a top-level entry, which
set_swapper_pgd() writes through FIX_PGD. That store can fault, and the
kernel then warns about a spurious translation fault. When kgdb stops a
CPU in lazy MMU mode, its breakpoint writes through FIX_TEXT_POKE0 can
fault the same way, and fail.
Emit the barriers directly in __set_fixmap().
Fixes: 5fdd05efa1cd3 ("arm64/mm: Batch barriers when updating kernel mappings")
Cc: stable@vger.kernel.org
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
arch/arm64/mm/fixmap.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/mm/fixmap.c b/arch/arm64/mm/fixmap.c
index f66a0016dd02d..951004ad700c3 100644
--- a/arch/arm64/mm/fixmap.c
+++ b/arch/arm64/mm/fixmap.c
@@ -127,7 +127,9 @@ void __set_fixmap(enum fixed_addresses idx,
ptep = fixmap_pte(addr);
if (pgprot_val(flags)) {
- __set_pte(ptep, pfn_pte(phys >> PAGE_SHIFT, flags));
+ /* The slot must be usable on return, even in lazy MMU mode. */
+ __set_pte_nosync(ptep, pfn_pte(phys >> PAGE_SHIFT, flags));
+ emit_pte_barriers();
} else {
__pte_clear(&init_mm, addr, ptep);
flush_tlb_kernel_range(addr, addr+PAGE_SIZE);
--
2.47.3
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 1/3] arm64: pi: Add an ISB after mapping the FDT in the initial ID map
2026-11-20 12:16 ` [PATCH 1/3] arm64: pi: Add an ISB after mapping the FDT in the initial ID map Fuad Tabba
@ 2026-11-21 10:42 ` Ryan Roberts
0 siblings, 0 replies; 7+ messages in thread
From: Ryan Roberts @ 2026-11-21 10:42 UTC (permalink / raw)
To: Fuad Tabba, Catalin Marinas, Will Deacon
Cc: Mark Rutland, Ard Biesheuvel, Yang Shi, Kevin Brodsky,
Anshuman Khandual, linux-arm-kernel, linux-kernel, Fuad Tabba,
stable
On 20/11/2026 12:16, Fuad Tabba wrote:
> map_fdt() maps the FDT into the live initial ID map with only a DSB
> before the caller reads it, so the read can fault on a translation made
> speculatively before the mapping was written. This early in boot,
> VBAR_EL1 doesn't yet point at the kernel's vectors, and the kernel fails
> to boot.
>
> Add an ISB after the DSB.
>
> Fixes: 84b04d3e6bdbc ("arm64: kernel: Create initial ID map from C code")
> Cc: stable@vger.kernel.org
> Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
I did a quick audit of all the other DSBs without ISBs in the pi code and I
think the rest are all correct because they are not synchronising stores to
_live_ tables so there can't have been any speculation with the old invalid value.
Reviewed-by: Ryan Roberts <ryan.roberts@arm.com>
> ---
> arch/arm64/kernel/pi/map_kernel.c | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/arch/arm64/kernel/pi/map_kernel.c b/arch/arm64/kernel/pi/map_kernel.c
> index fb44cbdd2f292..f9338257adb7d 100644
> --- a/arch/arm64/kernel/pi/map_kernel.c
> +++ b/arch/arm64/kernel/pi/map_kernel.c
> @@ -211,6 +211,7 @@ static void *__init map_fdt(phys_addr_t fdt)
> fdt, PAGE_KERNEL, IDMAP_ROOT_LEVEL,
> (pte_t *)init_idmap_pg_dir, false, 0);
> dsb(ishst);
> + isb();
>
> return (void *)fdt;
> }
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 2/3] arm64: mm: Add an ISB after remapping the KPTI page-table slot
2026-11-20 12:16 ` [PATCH 2/3] arm64: mm: Add an ISB after remapping the KPTI page-table slot Fuad Tabba
@ 2026-11-21 10:48 ` Ryan Roberts
0 siblings, 0 replies; 7+ messages in thread
From: Ryan Roberts @ 2026-11-21 10:48 UTC (permalink / raw)
To: Fuad Tabba, Catalin Marinas, Will Deacon
Cc: Mark Rutland, Ard Biesheuvel, Yang Shi, Kevin Brodsky,
Anshuman Khandual, linux-arm-kernel, linux-kernel, Fuad Tabba,
stable
On 20/11/2026 12:16, Fuad Tabba wrote:
> kpti_map_pgtbl remaps a temporary slot with break-before-make, and
> kpti_mk_tbl_ng then reads the page tables through it. The make ends in
> a DSB alone, so the reads can fault on a translation of the slot made
> speculatively before the new entry was written. With the temporary page
> tables in TTBR1 the kernel's vectors are unmapped, and the boot hangs.
>
> Add an ISB after the DSB.
Reviewed-by: Ryan Roberts <ryan.roberts@arm.com>
>
> Fixes: 47546a1912fc4 ("arm64: mm: install KPTI nG mappings with MMU enabled")
> Cc: stable@vger.kernel.org
> Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
> ---
> arch/arm64/mm/proc.S | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
> index 12aacc74f7647..7ca36d59bf77f 100644
> --- a/arch/arm64/mm/proc.S
> +++ b/arch/arm64/mm/proc.S
> @@ -262,6 +262,7 @@ SYM_FUNC_ALIAS(__pi_idmap_cpu_replace_ttbr1, idmap_cpu_replace_ttbr1)
> orr pte, pte, pte_flags
> str pte, [temp_pte, #8 * (\level + 2)]
> dsb nshst
> + isb
> .endm
>
> /*
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 3/3] arm64: mm: Don't defer the barriers when installing a fixmap entry
2026-11-20 12:16 ` [PATCH 3/3] arm64: mm: Don't defer the barriers when installing a fixmap entry Fuad Tabba
@ 2026-11-21 11:01 ` Ryan Roberts
0 siblings, 0 replies; 7+ messages in thread
From: Ryan Roberts @ 2026-11-21 11:01 UTC (permalink / raw)
To: Fuad Tabba, Catalin Marinas, Will Deacon
Cc: Mark Rutland, Ard Biesheuvel, Yang Shi, Kevin Brodsky,
Anshuman Khandual, linux-arm-kernel, linux-kernel, Fuad Tabba,
stable
On 20/11/2026 12:16, Fuad Tabba wrote:
> In lazy MMU mode, __set_pte(), which __set_fixmap() uses, only queues
> its DSB and ISB, but a caller can dereference the slot as soon as
> __set_fixmap() returns. On a CPU with BBML3, runtime changes to the
> linear map are made in lazy MMU mode. With 4K pages and 39-bit VAs, for
> example, such a change can update a top-level entry, which
> set_swapper_pgd() writes through FIX_PGD. That store can fault, and the
> kernel then warns about a spurious translation fault. When kgdb stops a
> CPU in lazy MMU mode, its breakpoint writes through FIX_TEXT_POKE0 can
> fault the same way, and fail.
>
> Emit the barriers directly in __set_fixmap().
>
> Fixes: 5fdd05efa1cd3 ("arm64/mm: Batch barriers when updating kernel mappings")
My memory might be betraying me, but I think this change went in before BBML3
(BBML2_NOABORT)? In which case, the bug is only exposed by the latter series?
> Cc: stable@vger.kernel.org
> Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
> ---> arch/arm64/mm/fixmap.c | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/mm/fixmap.c b/arch/arm64/mm/fixmap.c
> index f66a0016dd02d..951004ad700c3 100644
> --- a/arch/arm64/mm/fixmap.c
> +++ b/arch/arm64/mm/fixmap.c
> @@ -127,7 +127,9 @@ void __set_fixmap(enum fixed_addresses idx,
> ptep = fixmap_pte(addr);
>
> if (pgprot_val(flags)) {
> - __set_pte(ptep, pfn_pte(phys >> PAGE_SHIFT, flags));
> + /* The slot must be usable on return, even in lazy MMU mode. */
> + __set_pte_nosync(ptep, pfn_pte(phys >> PAGE_SHIFT, flags));
> + emit_pte_barriers();
I wonder if it would be clearer to wrap __set_ptes():
lazy_mmu_mode_pause()
__set_ptes()
lazy_mmu_mode_resume()
I notice that ptep_try_set() is also using emit_pte_barriers() directly, but
that feels like a layering violation to me.
Thanks,
Ryan
> } else {
> __pte_clear(&init_mm, addr, ptep);
> flush_tlb_kernel_range(addr, addr+PAGE_SIZE);
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2026-11-21 11:01 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-20 12:16 [PATCH 0/3] arm64: Missing barriers in kernel page-table updates Fuad Tabba
2026-11-20 12:16 ` [PATCH 1/3] arm64: pi: Add an ISB after mapping the FDT in the initial ID map Fuad Tabba
2026-11-21 10:42 ` Ryan Roberts
2026-11-20 12:16 ` [PATCH 2/3] arm64: mm: Add an ISB after remapping the KPTI page-table slot Fuad Tabba
2026-11-21 10:48 ` Ryan Roberts
2026-11-20 12:16 ` [PATCH 3/3] arm64: mm: Don't defer the barriers when installing a fixmap entry Fuad Tabba
2026-11-21 11:01 ` Ryan Roberts
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®