From: Will Deacon <will@kernel.org>
To: linux-arm-kernel@lists.infradead.org
Cc: linux-kernel@vger.kernel.org, Will Deacon <will@kernel.org>,
Arnd Bergmann <arnd@arndb.de>, Ard Biesheuvel <ardb@kernel.org>,
Ada Couprie Diaz <ada.coupriediaz@arm.com>,
David Hildenbrand <david@kernel.org>,
Catalin Marinas <catalin.marinas@arm.com>,
Vladimir Murzin <vladimir.murzin@arm.com>,
Mark Rutland <mark.rutland@arm.com>,
Mostafa Saleh <smostafa@google.com>,
Lorenzo Stoakes <ljs@kernel.org>,
Oliver Upton <oupton@kernel.org>,
Linus Walleij <linusw@kernel.org>, Marc Zyngier <maz@kernel.org>
Subject: [PATCH 05/21] KVM: arm64: Protect TPIDRRO_EL0 across guest entry/exit
Date: Mon, 7 Sep 2026 17:42:30 +0100 [thread overview]
Message-ID: <20260907164247.17223-6-will@kernel.org> (raw)
In-Reply-To: <20260907164247.17223-1-will@kernel.org>
From: Mostafa Saleh <smostafa@google.com>
To prepare for using TPIDRRO_EL0 to hold 'current' and repurposing
SP_EL0 as an overflow stack pointer, eagerly switch both registers when
entering and exiting the guest.
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Will Deacon <will@kernel.org>
---
arch/arm64/include/asm/kvm_asm.h | 10 ++++++++++
arch/arm64/kernel/asm-offsets.c | 1 +
arch/arm64/kvm/hyp/entry.S | 12 ++++++++----
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 2 --
4 files changed, 19 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
index e5b92ac09e69..1cfe9216a2ff 100644
--- a/arch/arm64/include/asm/kvm_asm.h
+++ b/arch/arm64/include/asm/kvm_asm.h
@@ -408,6 +408,16 @@ void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr, u64 elr_virt,
msr sp_el0, \tmp
.endm
+.macro save_tpidrro_el0 ctxt, tmp
+ mrs \tmp, tpidrro_el0
+ str \tmp, [\ctxt, #CPU_TPIDRRO_EL0_OFFSET]
+.endm
+
+.macro restore_tpidrro_el0 ctxt, tmp
+ ldr \tmp, [\ctxt, #CPU_TPIDRRO_EL0_OFFSET]
+ msr tpidrro_el0, \tmp
+.endm
+
#endif
#endif /* __ARM_KVM_ASM_H__ */
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 6038ab3beb25..cb8448cc5f51 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -108,6 +108,7 @@ int main(void)
DEFINE(VCPU_HCR_EL2, offsetof(struct kvm_vcpu, arch.hcr_el2));
DEFINE(CPU_USER_PT_REGS, offsetof(struct kvm_cpu_context, regs));
DEFINE(CPU_ELR_EL2, offsetof(struct kvm_cpu_context, sys_regs[ELR_EL2]));
+ DEFINE(CPU_TPIDRRO_EL0_OFFSET, offsetof(struct kvm_cpu_context, sys_regs[TPIDRRO_EL0]));
DEFINE(CPU_RGSR_EL1, offsetof(struct kvm_cpu_context, sys_regs[RGSR_EL1]));
DEFINE(CPU_GCR_EL1, offsetof(struct kvm_cpu_context, sys_regs[GCR_EL1]));
DEFINE(CPU_APIAKEYLO_EL1, offsetof(struct kvm_cpu_context, sys_regs[APIAKEYLO_EL1]));
diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
index 308100ed25de..4c89931a6a92 100644
--- a/arch/arm64/kvm/hyp/entry.S
+++ b/arch/arm64/kvm/hyp/entry.S
@@ -30,8 +30,9 @@ SYM_FUNC_START(__guest_enter)
// Store the hyp regs
save_callee_saved_regs x1
- // Save hyp's sp_el0
+ // Save hyp's sp_el0 and tpidrro_el0
save_sp_el0 x1, x2
+ save_tpidrro_el0 x1, x2
// Now the hyp state is stored if we have a pending RAS SError it must
// affect the host or hyp. If any asynchronous exception is pending we
@@ -66,8 +67,9 @@ alternative_else_nop_endif
// when this feature is enabled for kernel code.
ptrauth_switch_to_guest x29, x0, x1, x2
- // Restore the guest's sp_el0
+ // Restore the guest's sp_el0 and tpidrro_el0
restore_sp_el0 x29, x0
+ restore_tpidrro_el0 x29, x0
// Restore guest regs x0-x17
ldp x0, x1, [x29, #CPU_XREG_OFFSET(0)]
@@ -146,8 +148,9 @@ SYM_INNER_LABEL(__guest_exit, SYM_L_GLOBAL)
// Store the guest regs x18-x29, lr
save_callee_saved_regs x1
- // Store the guest's sp_el0
+ // Store the guest's sp_el0 and tpidrro_el0
save_sp_el0 x1, x2
+ save_tpidrro_el0 x1, x2
adr_this_cpu x2, kvm_hyp_ctxt, x3
@@ -161,8 +164,9 @@ SYM_INNER_LABEL(__guest_exit, SYM_L_GLOBAL)
// mte_switch_to_hyp(g_ctxt, h_ctxt, reg1)
mte_switch_to_hyp x1, x2, x3
- // Restore hyp's sp_el0
+ // Restore hyp's sp_el0 and tpidrro_el0
restore_sp_el0 x2, x3
+ restore_tpidrro_el0 x2, x3
// Now restore the hyp regs
restore_callee_saved_regs x2
diff --git a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
index c382848d3194..b5f7ef8e6371 100644
--- a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
+++ b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
@@ -66,7 +66,6 @@ static inline void __sysreg_save_common_state(struct kvm_cpu_context *ctxt)
static inline void __sysreg_save_user_state(struct kvm_cpu_context *ctxt)
{
ctxt_sys_reg(ctxt, TPIDR_EL0) = read_sysreg(tpidr_el0);
- ctxt_sys_reg(ctxt, TPIDRRO_EL0) = read_sysreg(tpidrro_el0);
}
static inline bool ctxt_has_mte(struct kvm_cpu_context *ctxt)
@@ -209,7 +208,6 @@ static inline void __sysreg_restore_common_state(struct kvm_cpu_context *ctxt)
static inline void __sysreg_restore_user_state(struct kvm_cpu_context *ctxt)
{
write_sysreg(ctxt_sys_reg(ctxt, TPIDR_EL0), tpidr_el0);
- write_sysreg(ctxt_sys_reg(ctxt, TPIDRRO_EL0), tpidrro_el0);
}
static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt,
--
2.55.0.979.g7e5102b832-goog
next prev parent reply other threads:[~2026-09-07 16:43 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-07 16:42 [PATCH 00/21] arm64: Move overflow sp into SP_EL1 and kernel sp into SP_EL0 Will Deacon
2026-09-07 16:42 ` [PATCH 01/21] arm64: entry: Defer setting of TPIDRRO_EL0 until exit to userspace Will Deacon
2026-09-11 7:53 ` Jinjie Ruan
2026-09-07 16:42 ` [PATCH 02/21] arm64: entry: Only check for stack overflow on exceptions from EL1 Will Deacon
2026-09-07 16:42 ` [PATCH 03/21] arm64: stackprotector: Temporarily disable per-task stackprotector Will Deacon
2026-09-07 16:42 ` [PATCH 04/21] arm64: bpf: Add support for generating reads of TPIDRRO_EL0 Will Deacon
2026-09-07 16:42 ` Will Deacon [this message]
2026-09-07 16:42 ` [PATCH 06/21] arm64: Store 'current' in TPIDRRO_EL0 instead of SP_EL0 Will Deacon
2026-09-08 13:19 ` David Laight
2026-09-11 12:57 ` Will Deacon
2026-09-07 16:42 ` [PATCH 07/21] selftests/bpf: arm64: Use TPIDRRO_EL0 instead of SP_EL0 for 'current' Will Deacon
2026-09-07 16:42 ` [PATCH 08/21] scripts/gdb: " Will Deacon
2026-09-07 16:42 ` [PATCH 09/21] arm64: stackprotector: Re-enable per-task stackprotector Will Deacon
2026-09-07 16:42 ` [PATCH 10/21] arm64: percpu: Specialise set_my_cpu_offset() for the primary CPU Will Deacon
2026-09-07 16:42 ` [PATCH 11/21] arm64: percpu: Annotate __kern_my_cpu_offset() as '__always_inline' Will Deacon
2026-09-07 16:42 ` [PATCH 12/21] KVM: arm64: Preserve handler/thread bit of EL1 mode in __finalise_el2() Will Deacon
2026-09-07 16:42 ` [PATCH 13/21] arm64: sdei: Guard most of asm/sdei.h with CONFIG_ARM_SDE_INTERFACE Will Deacon
2026-09-07 16:42 ` [PATCH 14/21] arm64: sdei: Support SDEI events from kernel handler and thread modes Will Deacon
2026-09-07 16:42 ` [PATCH 15/21] arm64: entry: Point SP_EL0 at the overflow stack Will Deacon
2026-09-07 16:42 ` [PATCH 16/21] arm64: entry: Implement EL1t exception handlers for " Will Deacon
2026-09-07 16:42 ` [PATCH 17/21] arm64: entry: Use SPSel to switch to " Will Deacon
2026-09-07 16:42 ` [PATCH 18/21] arm64: entry: Split up kernel_ventry macro into separate helper macros Will Deacon
2026-09-07 16:42 ` [PATCH 19/21] arm64: entry: The great stack switcheroo Will Deacon
2026-09-08 11:30 ` Will Deacon
2026-09-07 16:42 ` [PATCH 20/21] arm64: tracing: Advertise a mode of EL1t in synthetic kernel regs Will Deacon
2026-09-07 16:42 ` [PATCH 21/21] arm64: Rename 'overflow_stack' and OVERFLOW_STACK_SIZE Will Deacon
2026-09-09 10:39 ` [PATCH 00/21] arm64: Move overflow sp into SP_EL1 and kernel sp into SP_EL0 Vladimir Murzin
2026-09-09 11:29 ` Will Deacon
2026-09-10 13:49 ` Vladimir Murzin
2026-09-11 12:57 ` Will Deacon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260907164247.17223-6-will@kernel.org \
--to=will@kernel.org \
--cc=ada.coupriediaz@arm.com \
--cc=ardb@kernel.org \
--cc=arnd@arndb.de \
--cc=catalin.marinas@arm.com \
--cc=david@kernel.org \
--cc=linusw@kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=ljs@kernel.org \
--cc=mark.rutland@arm.com \
--cc=maz@kernel.org \
--cc=oupton@kernel.org \
--cc=smostafa@google.com \
--cc=vladimir.murzin@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®