From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 633E22EA75A for ; Thu, 4 Sep 2025 22:21:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757024493; cv=none; b=NZqHccXu5vtB6a4r5izbWfbSDS7gpYf6S+5MrbJgvxBkNGz+kUfrVzh9zKL6Kh8xPZL902+L03MOuVO+6BrnfZ8heOmf9iWIjGw8CBJoMqlYN7clAR/byJ8Ma2yFp0ptkBiVbq7Tmt4yUfRgucTH/8bdyA+HqRhzPpVMfSLTR9A= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757024493; c=relaxed/simple; bh=j/2KYHKwMUKHzSF9eJA6I9CmNXA9iNyyoRgUC+QBP0I=; h=Message-ID:From:To:Cc:Subject:References:MIME-Version: Content-Type:Date; b=n0Q75B6btaCr1GzviRZzdHXlLqKy9y+vr7E2TZhKy+4YQvtb5rPQiaL9spmU5b/vEDF8MATMGYtoikVdyRRAvYk75pMptk056c7Q6A9e2xyQ/qvtLYXg7vGTZALxTjc/qAtOFFIKi+dN1sKZzHIpY3T33GsYeglv87CSC8eebjM= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=fEBItZu6; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=EC1LkyVb; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="fEBItZu6"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="EC1LkyVb" Message-ID: <20250904185720.974711741@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1757024489; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=uEpXb4MYm4+O+MxEi6/Nv49MCNNEk4A00lo72YqNxnw=; b=fEBItZu6R0avNLsor7srW8gvLnPKwaeonJ+l4X6J+phOCPn4DQZDt2c5fPHD5OWqR5/gl1 DVAEYbwtgOW6LC+WYEhgjKFKYI18gea13RoG3v77bSZt6DI2nEItNry9w1R8L/sih6fNIL d3PSOFnRM+y1Ms4PAIXh1eA8dsWU++p42WIZttSti5SysSVu4LulGI8ZPyZSOFC56EHTFr U2NNrQsZuNA9tNcwAMAsKZEHh/NKBQ3V6FToTFdzU8IH81ID0E93xTgig941Rx2d0loXbL ZyZwheED6Q7tLY6DVJ31Wx4dGAfbcwFvvB6W2AUExRTmYAd76/xmW82bwJ9x5g== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1757024489; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=uEpXb4MYm4+O+MxEi6/Nv49MCNNEk4A00lo72YqNxnw=; b=EC1LkyVbDXjtWAWmWmp10+60UMpGBMMNwcCH0qRzc1JdFkhT2W8QhB1Ids3+l1LfGU71eG ruevL2SPRLJoqNAw== From: Thomas Gleixner To: LKML Cc: Michael Jeanson , Jens Axboe , Mathieu Desnoyers , Peter Zijlstra , "Paul E. McKenney" , Boqun Feng , Paolo Bonzini , Sean Christopherson , Wei Liu , Dexuan Cui , x86@kernel.org, Arnd Bergmann , Heiko Carstens , Christian Borntraeger , Sven Schnelle , Huacai Chen , Paul Walmsley , Palmer Dabbelt Subject: [patch V3 29/37] entry: Split up exit_to_user_mode_prepare() References: <20250904185336.943880027@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Date: Fri, 5 Sep 2025 00:21:28 +0200 (CEST) exit_to_user_mode_prepare() is used for both interrupts and syscalls, but there is extra rseq work, which is only required for in the interrupt exit case. Split up the function and provide wrappers for syscalls and interrupts, which allows to separate the rseq exit work in the next step. Signed-off-by: Thomas Gleixner Reviewed-by: Mathieu Desnoyers --- include/linux/entry-common.h | 2 - include/linux/irq-entry-common.h | 42 ++++++++++++++++++++++++++++++++++----- 2 files changed, 38 insertions(+), 6 deletions(-) --- a/include/linux/entry-common.h +++ b/include/linux/entry-common.h @@ -156,7 +156,7 @@ static __always_inline void syscall_exit if (unlikely(work & SYSCALL_WORK_EXIT)) syscall_exit_work(regs, work); local_irq_disable_exit_to_user(); - exit_to_user_mode_prepare(regs); + syscall_exit_to_user_mode_prepare(regs); } /** --- a/include/linux/irq-entry-common.h +++ b/include/linux/irq-entry-common.h @@ -201,7 +201,7 @@ void arch_do_signal_or_restart(struct pt unsigned long exit_to_user_mode_loop(struct pt_regs *regs, unsigned long ti_work); /** - * exit_to_user_mode_prepare - call exit_to_user_mode_loop() if required + * __exit_to_user_mode_prepare - call exit_to_user_mode_loop() if required * @regs: Pointer to pt_regs on entry stack * * 1) check that interrupts are disabled @@ -209,8 +209,10 @@ unsigned long exit_to_user_mode_loop(str * 3) call exit_to_user_mode_loop() if any flags from * EXIT_TO_USER_MODE_WORK are set * 4) check that interrupts are still disabled + * + * Don't invoke directly, use the syscall/irqentry_ prefixed variants below */ -static __always_inline void exit_to_user_mode_prepare(struct pt_regs *regs) +static __always_inline void __exit_to_user_mode_prepare(struct pt_regs *regs) { unsigned long ti_work; @@ -224,15 +226,45 @@ static __always_inline void exit_to_user ti_work = exit_to_user_mode_loop(regs, ti_work); arch_exit_to_user_mode_prepare(regs, ti_work); +} - rseq_exit_to_user_mode(); - +static __always_inline void __exit_to_user_mode_validate(void) +{ /* Ensure that kernel state is sane for a return to userspace */ kmap_assert_nomap(); lockdep_assert_irqs_disabled(); lockdep_sys_exit(); } + +/** + * syscall_exit_to_user_mode_prepare - call exit_to_user_mode_loop() if required + * @regs: Pointer to pt_regs on entry stack + * + * Wrapper around __exit_to_user_mode_prepare() to separate the exit work for + * syscalls and interrupts. + */ +static __always_inline void syscall_exit_to_user_mode_prepare(struct pt_regs *regs) +{ + __exit_to_user_mode_prepare(regs); + rseq_exit_to_user_mode(); + __exit_to_user_mode_validate(); +} + +/** + * irqentry_exit_to_user_mode_prepare - call exit_to_user_mode_loop() if required + * @regs: Pointer to pt_regs on entry stack + * + * Wrapper around __exit_to_user_mode_prepare() to separate the exit work for + * syscalls and interrupts. + */ +static __always_inline void irqentry_exit_to_user_mode_prepare(struct pt_regs *regs) +{ + __exit_to_user_mode_prepare(regs); + rseq_exit_to_user_mode(); + __exit_to_user_mode_validate(); +} + /** * exit_to_user_mode - Fixup state when exiting to user mode * @@ -297,7 +329,7 @@ static __always_inline void irqentry_ent static __always_inline void irqentry_exit_to_user_mode(struct pt_regs *regs) { instrumentation_begin(); - exit_to_user_mode_prepare(regs); + irqentry_exit_to_user_mode_prepare(regs); instrumentation_end(); exit_to_user_mode(); }