From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1E5273E008F for ; Wed, 3 Jun 2026 16:11:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780503082; cv=none; b=G0kxbtvlojospRKLQE/5kfPFh+RDtpQX5zlm8uHXdgcxH866i0E53mltMlCjFQKGHR487rY3w7c9HlYBMwWl7WwiLcfV9JO692OUe5d6YT9OtJYpucI2MDD05z/mFTvK5I+JbvpqhP7fd0BU9ZIN51Falek1Dax5Ply03DzpB0s= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780503082; c=relaxed/simple; bh=eaNupvnA0Lqw/MOrFohHOEsrKK+Vfr9jUBx2HbnfPRI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=lasdT5vOeTzb6cmSxgEnMaFknwXTM8EkwGjFU5UwdjERuycG3AS3BJcQcK9sZG8JlWucE+uN5a8x4rZ4wyyM+8aEoyHq8pAU432mYcQ9JUGFluETBfI/5BnRIR+0RKzkkEQrO6kMG/537ZKfVIFkd4fbtayZ8SMdjQYhE6sn+Cc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=tIShb6Qe; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="tIShb6Qe" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-490a767c7dcso31983485e9.2 for ; Wed, 03 Jun 2026 09:11:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1780503079; x=1781107879; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=WbgvnoAHr1VZwp/TGprKI6jwxgm2Tuk9bwrWQUAuGn8=; b=tIShb6Qe294qeaOGuxPopMUjQjFP6wrZ92gIbwYjL23K3MJZOcllb9rLANe7S7wc+j rsK7lHTgEOtL2zDJd0O94C4z2anbx+3xtSbZi/av5UaYTMjaJsjH+/lw8BfCRxwVkb7v YT5NUGYyZqWlS7bVfc4mAbx8de7fCnZl5zakaHOoxMaU8tHjgPRfWR4vwSacQXdclsg4 StMXdiQOlq5VRuxvydeZxBMjg015zgvujCHPGUwjHKreBwTLDQ0kfCTM+hoBiEcwfzt0 fZgHzFfejYg/UiG3Xp2eXckocu5iXNVEQ7TjtZv6D2SDeWrDLv4Nl2UM3tWajHkxB2GC R/YQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780503079; x=1781107879; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=WbgvnoAHr1VZwp/TGprKI6jwxgm2Tuk9bwrWQUAuGn8=; b=RIk0UdULBODsUH+gBiZpulKmxi45JYgNZz8VPRzPYINtA1zWQ8XD5rxn7WrNAYD31S UnzQ4MzN6HKwYvPQXAv7aLp9RYmz/zfFUIoUsYzI89VYA1chKAaX7BRfX0wu338o5CsA xAWoGwXQmMXZ0TcLesu0g2wKXy4qEf/zqy5fg9o0R1nSEKZnOiyxhxbHNVNfNEUxFW0A h0sXRW9w7Rc/kV31tubL4b07hldDjT4TkD8Mj1167wtopjBDNWfjpgD7j7HZphLTZSfo G2HQbFls+ITzB54LYJpiVb7uSJyznTz68rjKY8CxgxsIGTXRLYqrAJU6Rjq0sMey4vFC v6MQ== X-Gm-Message-State: AOJu0Yz4vzsD4qEmYD6VXPHx8tByKSU36b9PWo1kg/Ms58r4LYtS+LVZ f1vIkmceFpm6hax7XyothQHB8jBXbUh+35Uib7izBQSyZ7kB+f4dC01iWWZjVn6C921fMN1vtA= = X-Received: from wruh4.prod.google.com ([2002:a5d:6884:0:b0:43d:7e53:c76a]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:414f:b0:490:b645:3213 with SMTP id 5b1f17b1804b1-490b645343bmr42462745e9.19.1780503079435; Wed, 03 Jun 2026 09:11:19 -0700 (PDT) Date: Wed, 3 Jun 2026 18:09:51 +0200 In-Reply-To: <20260603160949.3372482-6-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260603160949.3372482-6-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=5738; i=ardb@kernel.org; h=from:subject; bh=H9XZxM8XBIS3dkH+KdJkynk6oFDR1zV1Vb5pNRNFzRU=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIUsh8DzDwfV/5vOnvllwZIk0Y+nb+pOhh2KSuA8vEr+Rz qBsVavdUcrCIMbFICumyCIw+++7nacnStU6z5KFmcPKBDaEi1MAJnLmIcN/rw1nuH8kfxSp4ReY xBokMWvuuo68ZWwbW60S2H7vzZvoxshwgW3XwzULLl7eGtkw+d2zAtWQBxclN1Qqb9kQ/pKdb1M 6FwA= X-Mailer: git-send-email 2.54.0.1013.g208068f2d8-goog Message-ID: <20260603160949.3372482-7-ardb+git@google.com> Subject: [PATCH 1/4] arm64: Move generic KASAN page tables out of BSS too From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, maz@kernel.org, Ard Biesheuvel , Kevin Brodsky , Mark Brown Content-Type: text/plain; charset="UTF-8" From: Ard Biesheuvel Make sure that all KASAN page tables are emitted into the .pgdir section (provided that the arch has one - otherwise, fall back to page aligned BSS) This is needed because BSS itself is no longer accessible via the linear map. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/linkage.h | 2 ++ arch/arm64/include/asm/mmu.h | 2 -- arch/arm64/kernel/vmlinux.lds.S | 8 ++++---- arch/arm64/mm/fixmap.c | 6 +++--- arch/arm64/mm/kasan_init.c | 2 +- include/linux/linkage.h | 4 ++++ mm/kasan/init.c | 10 +++++----- 7 files changed, 19 insertions(+), 15 deletions(-) diff --git a/arch/arm64/include/asm/linkage.h b/arch/arm64/include/asm/linkage.h index 40bd17add539..8637f667667c 100644 --- a/arch/arm64/include/asm/linkage.h +++ b/arch/arm64/include/asm/linkage.h @@ -43,4 +43,6 @@ SYM_TYPED_START(name, SYM_L_GLOBAL, SYM_A_ALIGN) \ bti c ; +#define __bss_pgtbl __section(".bss..pgtbl") __aligned(PAGE_SIZE) + #endif diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h index fb95754f2876..5e1211c540ab 100644 --- a/arch/arm64/include/asm/mmu.h +++ b/arch/arm64/include/asm/mmu.h @@ -13,8 +13,6 @@ #ifndef __ASSEMBLER__ -#define __pgtbl_bss __section(".pgdir.bss") __aligned(PAGE_SIZE) - #include #include diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 2b0ebfb30c63..d3ed59abab38 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -352,11 +352,11 @@ SECTIONS BSS_SECTION(SBSS_ALIGN, 0, PAGE_SIZE) __pi___bss_start = __bss_start; - /* fixmap BSS starts here - preceding data/BSS is omitted from the linear map */ - .pgdir.bss (NOLOAD) : ALIGN(PAGE_SIZE) { - *(.pgdir.bss) + /* page table BSS starts here - preceding data/BSS is omitted from the linear map */ + .pgtbl : ALIGN(PAGE_SIZE) { + *(.bss..pgtbl) } - ASSERT(ADDR(.pgdir.bss) == __bss_stop, ".pgdir.bss must follow BSS") + ASSERT(ADDR(.pgtbl) == __bss_stop, ".pgtbl must follow BSS") . = ALIGN(PAGE_SIZE); __pi_init_pg_dir = .; diff --git a/arch/arm64/mm/fixmap.c b/arch/arm64/mm/fixmap.c index 1a3bbd67dd76..f66a0016dd02 100644 --- a/arch/arm64/mm/fixmap.c +++ b/arch/arm64/mm/fixmap.c @@ -31,9 +31,9 @@ static_assert(NR_BM_PMD_TABLES == 1); #define BM_PTE_TABLE_IDX(addr) __BM_TABLE_IDX(addr, PMD_SHIFT) -static pte_t bm_pte[NR_BM_PTE_TABLES][PTRS_PER_PTE] __pgtbl_bss; -static pmd_t bm_pmd[PTRS_PER_PMD] __pgtbl_bss __maybe_unused; -static pud_t bm_pud[PTRS_PER_PUD] __pgtbl_bss __maybe_unused; +static pte_t bm_pte[NR_BM_PTE_TABLES][PTRS_PER_PTE] __bss_pgtbl; +static pmd_t bm_pmd[PTRS_PER_PMD] __bss_pgtbl __maybe_unused; +static pud_t bm_pud[PTRS_PER_PUD] __bss_pgtbl __maybe_unused; static inline pte_t *fixmap_pte(unsigned long addr) { diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c index dbf22cae82ee..3fcad956fdf7 100644 --- a/arch/arm64/mm/kasan_init.c +++ b/arch/arm64/mm/kasan_init.c @@ -214,7 +214,7 @@ asmlinkage void __init kasan_early_init(void) * shadow pud_t[]/p4d_t[], which could end up getting corrupted * when the linear region is mapped. */ - static pte_t tbl[PTRS_PER_PTE] __pgtbl_bss; + static pte_t tbl[PTRS_PER_PTE] __bss_pgtbl; pgd_t *pgdp = pgd_offset_k(KASAN_SHADOW_START); set_pgd(pgdp, __pgd(__pa_symbol(tbl) | PGD_TYPE_TABLE)); diff --git a/include/linux/linkage.h b/include/linux/linkage.h index b11660b706c5..53fe1f48fd28 100644 --- a/include/linux/linkage.h +++ b/include/linux/linkage.h @@ -39,6 +39,10 @@ #define __page_aligned_data __section(".data..page_aligned") __aligned(PAGE_SIZE) #define __page_aligned_bss __section(".bss..page_aligned") __aligned(PAGE_SIZE) +#ifndef __bss_pgtbl +#define __bss_pgtbl __page_aligned_bss +#endif + /* * For assembly routines. * diff --git a/mm/kasan/init.c b/mm/kasan/init.c index 9c880f607c6a..66a883887987 100644 --- a/mm/kasan/init.c +++ b/mm/kasan/init.c @@ -26,10 +26,10 @@ * - Latter it reused it as zero shadow to cover large ranges of memory * that allowed to access, but not handled by kasan (vmalloc/vmemmap ...). */ -unsigned char kasan_early_shadow_page[PAGE_SIZE] __page_aligned_bss; +unsigned char kasan_early_shadow_page[PAGE_SIZE] __bss_pgtbl; #if CONFIG_PGTABLE_LEVELS > 4 -p4d_t kasan_early_shadow_p4d[MAX_PTRS_PER_P4D] __page_aligned_bss; +p4d_t kasan_early_shadow_p4d[MAX_PTRS_PER_P4D] __bss_pgtbl; static inline bool kasan_p4d_table(pgd_t pgd) { return pgd_page(pgd) == virt_to_page(lm_alias(kasan_early_shadow_p4d)); @@ -41,7 +41,7 @@ static inline bool kasan_p4d_table(pgd_t pgd) } #endif #if CONFIG_PGTABLE_LEVELS > 3 -pud_t kasan_early_shadow_pud[MAX_PTRS_PER_PUD] __page_aligned_bss; +pud_t kasan_early_shadow_pud[MAX_PTRS_PER_PUD] __bss_pgtbl; static inline bool kasan_pud_table(p4d_t p4d) { return p4d_page(p4d) == virt_to_page(lm_alias(kasan_early_shadow_pud)); @@ -53,7 +53,7 @@ static inline bool kasan_pud_table(p4d_t p4d) } #endif #if CONFIG_PGTABLE_LEVELS > 2 -pmd_t kasan_early_shadow_pmd[MAX_PTRS_PER_PMD] __page_aligned_bss; +pmd_t kasan_early_shadow_pmd[MAX_PTRS_PER_PMD] __bss_pgtbl; static inline bool kasan_pmd_table(pud_t pud) { return pud_page(pud) == virt_to_page(lm_alias(kasan_early_shadow_pmd)); @@ -65,7 +65,7 @@ static inline bool kasan_pmd_table(pud_t pud) } #endif pte_t kasan_early_shadow_pte[MAX_PTRS_PER_PTE + PTE_HWTABLE_PTRS] - __page_aligned_bss; + __bss_pgtbl; static inline bool kasan_pte_table(pmd_t pmd) { -- 2.54.0.1013.g208068f2d8-goog