mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Josh Poimboeuf <jpoimboe@kernel.org>
To: x86@kernel.org
Cc: linux-kernel@vger.kernel.org,
	Thomas Gleixner <tglx@linutronix.de>,
	Borislav Petkov <bp@alien8.de>,
	Peter Zijlstra <peterz@infradead.org>,
	Pawan Gupta <pawan.kumar.gupta@linux.intel.com>,
	Waiman Long <longman@redhat.com>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	Ingo Molnar <mingo@redhat.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Michael Ellerman <mpe@ellerman.id.au>,
	linuxppc-dev@lists.ozlabs.org,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Mark Rutland <mark.rutland@arm.com>,
	"Kirill A . Shutemov" <kirill@shutemov.name>
Subject: [PATCH v3 2/6] x86/uaccess: Avoid barrier_nospec() in 64-bit __get_user()
Date: Mon, 28 Oct 2024 18:56:15 -0700	[thread overview]
Message-ID: <82cbb9983fef5ecf6f1cb33661e977172d40a7e6.1730166635.git.jpoimboe@kernel.org> (raw)
In-Reply-To: <cover.1730166635.git.jpoimboe@kernel.org>

The barrier_nospec() in 64-bit __get_user() is slow.  Instead use
pointer masking to force the user pointer to all 1's if a previous
access_ok() mispredicted true for an invalid address.

Note that for safety on some AMD CPUs, this relies on recent commit
86e6b1547b3d ("x86: fix user address masking non-canonical speculation
issue").

Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
 arch/x86/lib/getuser.S | 24 ++++++++++++++++++++----
 1 file changed, 20 insertions(+), 4 deletions(-)

diff --git a/arch/x86/lib/getuser.S b/arch/x86/lib/getuser.S
index 4357ec2a0bfc..998d5be6b794 100644
--- a/arch/x86/lib/getuser.S
+++ b/arch/x86/lib/getuser.S
@@ -112,8 +112,12 @@ EXPORT_SYMBOL(__get_user_8)
 
 /* .. and the same for __get_user, just without the range checks */
 SYM_FUNC_START(__get_user_nocheck_1)
-	ASM_STAC
+#ifdef CONFIG_X86_64
+	check_range size=1
+#else
 	ASM_BARRIER_NOSPEC
+#endif
+	ASM_STAC
 	UACCESS movzbl (%_ASM_AX),%edx
 	xor %eax,%eax
 	ASM_CLAC
@@ -122,8 +126,12 @@ SYM_FUNC_END(__get_user_nocheck_1)
 EXPORT_SYMBOL(__get_user_nocheck_1)
 
 SYM_FUNC_START(__get_user_nocheck_2)
-	ASM_STAC
+#ifdef CONFIG_X86_64
+	check_range size=2
+#else
 	ASM_BARRIER_NOSPEC
+#endif
+	ASM_STAC
 	UACCESS movzwl (%_ASM_AX),%edx
 	xor %eax,%eax
 	ASM_CLAC
@@ -132,8 +140,12 @@ SYM_FUNC_END(__get_user_nocheck_2)
 EXPORT_SYMBOL(__get_user_nocheck_2)
 
 SYM_FUNC_START(__get_user_nocheck_4)
-	ASM_STAC
+#ifdef CONFIG_X86_64
+	check_range size=4
+#else
 	ASM_BARRIER_NOSPEC
+#endif
+	ASM_STAC
 	UACCESS movl (%_ASM_AX),%edx
 	xor %eax,%eax
 	ASM_CLAC
@@ -142,8 +154,12 @@ SYM_FUNC_END(__get_user_nocheck_4)
 EXPORT_SYMBOL(__get_user_nocheck_4)
 
 SYM_FUNC_START(__get_user_nocheck_8)
-	ASM_STAC
+#ifdef CONFIG_X86_64
+	check_range size=8
+#else
 	ASM_BARRIER_NOSPEC
+#endif
+	ASM_STAC
 #ifdef CONFIG_X86_64
 	UACCESS movq (%_ASM_AX),%rdx
 #else
-- 
2.47.0


  parent reply	other threads:[~2024-10-29  1:56 UTC|newest]

Thread overview: 29+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-10-29  1:56 [PATCH v3 0/6] x86/uaccess: avoid barrier_nospec() Josh Poimboeuf
2024-10-29  1:56 ` [PATCH v3 1/6] x86/uaccess: Avoid barrier_nospec() in 64-bit copy_from_user() Josh Poimboeuf
2024-10-29  8:13   ` Kirill A . Shutemov
2024-10-30  2:03   ` Linus Torvalds
2024-10-30  4:59     ` Josh Poimboeuf
2024-10-29  1:56 ` Josh Poimboeuf [this message]
2024-10-29  3:27   ` [PATCH v3 2/6] x86/uaccess: Avoid barrier_nospec() in 64-bit __get_user() Josh Poimboeuf
2024-11-08 17:12     ` David Laight
2024-11-15 23:06       ` 'Josh Poimboeuf'
2024-11-16  1:27         ` Linus Torvalds
2024-11-16 21:38           ` David Laight
2024-11-16 23:08             ` Linus Torvalds
2024-11-21 21:40           ` Josh Poimboeuf
2024-11-21 22:16             ` Linus Torvalds
2024-11-22  0:12               ` Josh Poimboeuf
2024-11-22  1:02                 ` Linus Torvalds
2024-11-22  3:11                   ` Josh Poimboeuf
2024-11-22  3:57                     ` Linus Torvalds
2024-11-22  9:06                       ` Christophe Leroy
2024-11-22 18:16                         ` Linus Torvalds
2024-11-22 19:13                       ` Linus Torvalds
2024-11-22 19:35                         ` Linus Torvalds
2024-11-24 16:11                           ` David Laight
2024-11-24 18:16                             ` Linus Torvalds
2024-11-22  9:38               ` David Laight
2024-10-29  1:56 ` [PATCH v3 3/6] x86/uaccess: Avoid barrier_nospec() in 32-bit copy_from_user() Josh Poimboeuf
2024-10-29  1:56 ` [PATCH v3 4/6] x86/uaccess: Convert 32-bit get_user() to unconditional pointer masking Josh Poimboeuf
2024-10-29  1:56 ` [PATCH v3 5/6] x86/uaccess: Avoid barrier_nospec() in 32-bit __get_user() Josh Poimboeuf
2024-10-29  1:56 ` [PATCH v3 6/6] x86/uaccess: Converge [__]get_user() implementations Josh Poimboeuf

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=82cbb9983fef5ecf6f1cb33661e977172d40a7e6.1730166635.git.jpoimboe@kernel.org \
    --to=jpoimboe@kernel.org \
    --cc=andrew.cooper3@citrix.com \
    --cc=bp@alien8.de \
    --cc=dave.hansen@linux.intel.com \
    --cc=kirill@shutemov.name \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linuxppc-dev@lists.ozlabs.org \
    --cc=longman@redhat.com \
    --cc=mark.rutland@arm.com \
    --cc=mingo@redhat.com \
    --cc=mpe@ellerman.id.au \
    --cc=pawan.kumar.gupta@linux.intel.com \
    --cc=peterz@infradead.org \
    --cc=tglx@linutronix.de \
    --cc=torvalds@linux-foundation.org \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®