* [PATCH 0/3] arm64: Missing barriers in kernel page-table updates
@ 2026-11-20 12:16 Fuad Tabba
2026-11-20 12:16 ` [PATCH 1/3] arm64: pi: Add an ISB after mapping the FDT in the initial ID map Fuad Tabba
` (2 more replies)
0 siblings, 3 replies; 7+ messages in thread
From: Fuad Tabba @ 2026-11-20 12:16 UTC (permalink / raw)
To: Catalin Marinas, Will Deacon
Cc: Mark Rutland, Ard Biesheuvel, Ryan Roberts, Yang Shi,
Kevin Brodsky, Anshuman Khandual, linux-arm-kernel, linux-kernel,
Fuad Tabba
Hi folks,
Three fixes for kernel mappings that are made valid and then used on
the same CPU before an ISB, so the access can take a translation fault.
Tested on QEMU. The runtime changes to the linear map that patch 3
covers were not exercised, since they require a CPU with BBML3.
Based on Linux 7.4-rc1 (6628e7d4c6b25).
Cheers,
/fuad
Fuad Tabba (3):
arm64: pi: Add an ISB after mapping the FDT in the initial ID map
arm64: mm: Add an ISB after remapping the KPTI page-table slot
arm64: mm: Don't defer the barriers when installing a fixmap entry
arch/arm64/kernel/pi/map_kernel.c | 1 +
arch/arm64/mm/fixmap.c | 4 +++-
arch/arm64/mm/proc.S | 1 +
3 files changed, 5 insertions(+), 1 deletion(-)
base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
--
2.47.3
^ permalink raw reply [flat|nested] 7+ messages in thread* [PATCH 1/3] arm64: pi: Add an ISB after mapping the FDT in the initial ID map 2026-11-20 12:16 [PATCH 0/3] arm64: Missing barriers in kernel page-table updates Fuad Tabba @ 2026-11-20 12:16 ` Fuad Tabba 2026-11-21 10:42 ` Ryan Roberts 2026-11-20 12:16 ` [PATCH 2/3] arm64: mm: Add an ISB after remapping the KPTI page-table slot Fuad Tabba 2026-11-20 12:16 ` [PATCH 3/3] arm64: mm: Don't defer the barriers when installing a fixmap entry Fuad Tabba 2 siblings, 1 reply; 7+ messages in thread From: Fuad Tabba @ 2026-11-20 12:16 UTC (permalink / raw) To: Catalin Marinas, Will Deacon Cc: Mark Rutland, Ard Biesheuvel, Ryan Roberts, Yang Shi, Kevin Brodsky, Anshuman Khandual, linux-arm-kernel, linux-kernel, Fuad Tabba, stable map_fdt() maps the FDT into the live initial ID map with only a DSB before the caller reads it, so the read can fault on a translation made speculatively before the mapping was written. This early in boot, VBAR_EL1 doesn't yet point at the kernel's vectors, and the kernel fails to boot. Add an ISB after the DSB. Fixes: 84b04d3e6bdbc ("arm64: kernel: Create initial ID map from C code") Cc: stable@vger.kernel.org Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev> --- arch/arm64/kernel/pi/map_kernel.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/arm64/kernel/pi/map_kernel.c b/arch/arm64/kernel/pi/map_kernel.c index fb44cbdd2f292..f9338257adb7d 100644 --- a/arch/arm64/kernel/pi/map_kernel.c +++ b/arch/arm64/kernel/pi/map_kernel.c @@ -211,6 +211,7 @@ static void *__init map_fdt(phys_addr_t fdt) fdt, PAGE_KERNEL, IDMAP_ROOT_LEVEL, (pte_t *)init_idmap_pg_dir, false, 0); dsb(ishst); + isb(); return (void *)fdt; } -- 2.47.3 ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 1/3] arm64: pi: Add an ISB after mapping the FDT in the initial ID map 2026-11-20 12:16 ` [PATCH 1/3] arm64: pi: Add an ISB after mapping the FDT in the initial ID map Fuad Tabba @ 2026-11-21 10:42 ` Ryan Roberts 0 siblings, 0 replies; 7+ messages in thread From: Ryan Roberts @ 2026-11-21 10:42 UTC (permalink / raw) To: Fuad Tabba, Catalin Marinas, Will Deacon Cc: Mark Rutland, Ard Biesheuvel, Yang Shi, Kevin Brodsky, Anshuman Khandual, linux-arm-kernel, linux-kernel, Fuad Tabba, stable On 20/11/2026 12:16, Fuad Tabba wrote: > map_fdt() maps the FDT into the live initial ID map with only a DSB > before the caller reads it, so the read can fault on a translation made > speculatively before the mapping was written. This early in boot, > VBAR_EL1 doesn't yet point at the kernel's vectors, and the kernel fails > to boot. > > Add an ISB after the DSB. > > Fixes: 84b04d3e6bdbc ("arm64: kernel: Create initial ID map from C code") > Cc: stable@vger.kernel.org > Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev> I did a quick audit of all the other DSBs without ISBs in the pi code and I think the rest are all correct because they are not synchronising stores to _live_ tables so there can't have been any speculation with the old invalid value. Reviewed-by: Ryan Roberts <ryan.roberts@arm.com> > --- > arch/arm64/kernel/pi/map_kernel.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/arch/arm64/kernel/pi/map_kernel.c b/arch/arm64/kernel/pi/map_kernel.c > index fb44cbdd2f292..f9338257adb7d 100644 > --- a/arch/arm64/kernel/pi/map_kernel.c > +++ b/arch/arm64/kernel/pi/map_kernel.c > @@ -211,6 +211,7 @@ static void *__init map_fdt(phys_addr_t fdt) > fdt, PAGE_KERNEL, IDMAP_ROOT_LEVEL, > (pte_t *)init_idmap_pg_dir, false, 0); > dsb(ishst); > + isb(); > > return (void *)fdt; > } ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 2/3] arm64: mm: Add an ISB after remapping the KPTI page-table slot 2026-11-20 12:16 [PATCH 0/3] arm64: Missing barriers in kernel page-table updates Fuad Tabba 2026-11-20 12:16 ` [PATCH 1/3] arm64: pi: Add an ISB after mapping the FDT in the initial ID map Fuad Tabba @ 2026-11-20 12:16 ` Fuad Tabba 2026-11-21 10:48 ` Ryan Roberts 2026-11-20 12:16 ` [PATCH 3/3] arm64: mm: Don't defer the barriers when installing a fixmap entry Fuad Tabba 2 siblings, 1 reply; 7+ messages in thread From: Fuad Tabba @ 2026-11-20 12:16 UTC (permalink / raw) To: Catalin Marinas, Will Deacon Cc: Mark Rutland, Ard Biesheuvel, Ryan Roberts, Yang Shi, Kevin Brodsky, Anshuman Khandual, linux-arm-kernel, linux-kernel, Fuad Tabba, stable kpti_map_pgtbl remaps a temporary slot with break-before-make, and kpti_mk_tbl_ng then reads the page tables through it. The make ends in a DSB alone, so the reads can fault on a translation of the slot made speculatively before the new entry was written. With the temporary page tables in TTBR1 the kernel's vectors are unmapped, and the boot hangs. Add an ISB after the DSB. Fixes: 47546a1912fc4 ("arm64: mm: install KPTI nG mappings with MMU enabled") Cc: stable@vger.kernel.org Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev> --- arch/arm64/mm/proc.S | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 12aacc74f7647..7ca36d59bf77f 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -262,6 +262,7 @@ SYM_FUNC_ALIAS(__pi_idmap_cpu_replace_ttbr1, idmap_cpu_replace_ttbr1) orr pte, pte, pte_flags str pte, [temp_pte, #8 * (\level + 2)] dsb nshst + isb .endm /* -- 2.47.3 ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 2/3] arm64: mm: Add an ISB after remapping the KPTI page-table slot 2026-11-20 12:16 ` [PATCH 2/3] arm64: mm: Add an ISB after remapping the KPTI page-table slot Fuad Tabba @ 2026-11-21 10:48 ` Ryan Roberts 0 siblings, 0 replies; 7+ messages in thread From: Ryan Roberts @ 2026-11-21 10:48 UTC (permalink / raw) To: Fuad Tabba, Catalin Marinas, Will Deacon Cc: Mark Rutland, Ard Biesheuvel, Yang Shi, Kevin Brodsky, Anshuman Khandual, linux-arm-kernel, linux-kernel, Fuad Tabba, stable On 20/11/2026 12:16, Fuad Tabba wrote: > kpti_map_pgtbl remaps a temporary slot with break-before-make, and > kpti_mk_tbl_ng then reads the page tables through it. The make ends in > a DSB alone, so the reads can fault on a translation of the slot made > speculatively before the new entry was written. With the temporary page > tables in TTBR1 the kernel's vectors are unmapped, and the boot hangs. > > Add an ISB after the DSB. Reviewed-by: Ryan Roberts <ryan.roberts@arm.com> > > Fixes: 47546a1912fc4 ("arm64: mm: install KPTI nG mappings with MMU enabled") > Cc: stable@vger.kernel.org > Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev> > --- > arch/arm64/mm/proc.S | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S > index 12aacc74f7647..7ca36d59bf77f 100644 > --- a/arch/arm64/mm/proc.S > +++ b/arch/arm64/mm/proc.S > @@ -262,6 +262,7 @@ SYM_FUNC_ALIAS(__pi_idmap_cpu_replace_ttbr1, idmap_cpu_replace_ttbr1) > orr pte, pte, pte_flags > str pte, [temp_pte, #8 * (\level + 2)] > dsb nshst > + isb > .endm > > /* ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 3/3] arm64: mm: Don't defer the barriers when installing a fixmap entry 2026-11-20 12:16 [PATCH 0/3] arm64: Missing barriers in kernel page-table updates Fuad Tabba 2026-11-20 12:16 ` [PATCH 1/3] arm64: pi: Add an ISB after mapping the FDT in the initial ID map Fuad Tabba 2026-11-20 12:16 ` [PATCH 2/3] arm64: mm: Add an ISB after remapping the KPTI page-table slot Fuad Tabba @ 2026-11-20 12:16 ` Fuad Tabba 2026-11-21 11:01 ` Ryan Roberts 2 siblings, 1 reply; 7+ messages in thread From: Fuad Tabba @ 2026-11-20 12:16 UTC (permalink / raw) To: Catalin Marinas, Will Deacon Cc: Mark Rutland, Ard Biesheuvel, Ryan Roberts, Yang Shi, Kevin Brodsky, Anshuman Khandual, linux-arm-kernel, linux-kernel, Fuad Tabba, stable In lazy MMU mode, __set_pte(), which __set_fixmap() uses, only queues its DSB and ISB, but a caller can dereference the slot as soon as __set_fixmap() returns. On a CPU with BBML3, runtime changes to the linear map are made in lazy MMU mode. With 4K pages and 39-bit VAs, for example, such a change can update a top-level entry, which set_swapper_pgd() writes through FIX_PGD. That store can fault, and the kernel then warns about a spurious translation fault. When kgdb stops a CPU in lazy MMU mode, its breakpoint writes through FIX_TEXT_POKE0 can fault the same way, and fail. Emit the barriers directly in __set_fixmap(). Fixes: 5fdd05efa1cd3 ("arm64/mm: Batch barriers when updating kernel mappings") Cc: stable@vger.kernel.org Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev> --- arch/arm64/mm/fixmap.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/arm64/mm/fixmap.c b/arch/arm64/mm/fixmap.c index f66a0016dd02d..951004ad700c3 100644 --- a/arch/arm64/mm/fixmap.c +++ b/arch/arm64/mm/fixmap.c @@ -127,7 +127,9 @@ void __set_fixmap(enum fixed_addresses idx, ptep = fixmap_pte(addr); if (pgprot_val(flags)) { - __set_pte(ptep, pfn_pte(phys >> PAGE_SHIFT, flags)); + /* The slot must be usable on return, even in lazy MMU mode. */ + __set_pte_nosync(ptep, pfn_pte(phys >> PAGE_SHIFT, flags)); + emit_pte_barriers(); } else { __pte_clear(&init_mm, addr, ptep); flush_tlb_kernel_range(addr, addr+PAGE_SIZE); -- 2.47.3 ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 3/3] arm64: mm: Don't defer the barriers when installing a fixmap entry 2026-11-20 12:16 ` [PATCH 3/3] arm64: mm: Don't defer the barriers when installing a fixmap entry Fuad Tabba @ 2026-11-21 11:01 ` Ryan Roberts 0 siblings, 0 replies; 7+ messages in thread From: Ryan Roberts @ 2026-11-21 11:01 UTC (permalink / raw) To: Fuad Tabba, Catalin Marinas, Will Deacon Cc: Mark Rutland, Ard Biesheuvel, Yang Shi, Kevin Brodsky, Anshuman Khandual, linux-arm-kernel, linux-kernel, Fuad Tabba, stable On 20/11/2026 12:16, Fuad Tabba wrote: > In lazy MMU mode, __set_pte(), which __set_fixmap() uses, only queues > its DSB and ISB, but a caller can dereference the slot as soon as > __set_fixmap() returns. On a CPU with BBML3, runtime changes to the > linear map are made in lazy MMU mode. With 4K pages and 39-bit VAs, for > example, such a change can update a top-level entry, which > set_swapper_pgd() writes through FIX_PGD. That store can fault, and the > kernel then warns about a spurious translation fault. When kgdb stops a > CPU in lazy MMU mode, its breakpoint writes through FIX_TEXT_POKE0 can > fault the same way, and fail. > > Emit the barriers directly in __set_fixmap(). > > Fixes: 5fdd05efa1cd3 ("arm64/mm: Batch barriers when updating kernel mappings") My memory might be betraying me, but I think this change went in before BBML3 (BBML2_NOABORT)? In which case, the bug is only exposed by the latter series? > Cc: stable@vger.kernel.org > Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev> > ---> arch/arm64/mm/fixmap.c | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/arch/arm64/mm/fixmap.c b/arch/arm64/mm/fixmap.c > index f66a0016dd02d..951004ad700c3 100644 > --- a/arch/arm64/mm/fixmap.c > +++ b/arch/arm64/mm/fixmap.c > @@ -127,7 +127,9 @@ void __set_fixmap(enum fixed_addresses idx, > ptep = fixmap_pte(addr); > > if (pgprot_val(flags)) { > - __set_pte(ptep, pfn_pte(phys >> PAGE_SHIFT, flags)); > + /* The slot must be usable on return, even in lazy MMU mode. */ > + __set_pte_nosync(ptep, pfn_pte(phys >> PAGE_SHIFT, flags)); > + emit_pte_barriers(); I wonder if it would be clearer to wrap __set_ptes(): lazy_mmu_mode_pause() __set_ptes() lazy_mmu_mode_resume() I notice that ptep_try_set() is also using emit_pte_barriers() directly, but that feels like a layering violation to me. Thanks, Ryan > } else { > __pte_clear(&init_mm, addr, ptep); > flush_tlb_kernel_range(addr, addr+PAGE_SIZE); ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2026-11-21 11:01 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2026-11-20 12:16 [PATCH 0/3] arm64: Missing barriers in kernel page-table updates Fuad Tabba 2026-11-20 12:16 ` [PATCH 1/3] arm64: pi: Add an ISB after mapping the FDT in the initial ID map Fuad Tabba 2026-11-21 10:42 ` Ryan Roberts 2026-11-20 12:16 ` [PATCH 2/3] arm64: mm: Add an ISB after remapping the KPTI page-table slot Fuad Tabba 2026-11-21 10:48 ` Ryan Roberts 2026-11-20 12:16 ` [PATCH 3/3] arm64: mm: Don't defer the barriers when installing a fixmap entry Fuad Tabba 2026-11-21 11:01 ` Ryan Roberts
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®