[PATCH] x86/build/64: Prevent native builds from generating SSE/AVX code

[PATCH] x86/build/64: Prevent native builds from generating SSE/AVX code

[Antoni Kuczyński]

With CONFIG_X86_NATIVE_CPU=y, GCC is invoked with -march=native after
the -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -mno-sse4a flags
that keep FP/SIMD code out of the kernel. The GCC driver expands
-march=native into an explicit list of ISA options for the build host
(e.g. "-march=ivybridge ... -msse4.2 -mavx ..."), and because these come
later on the command line they override the earlier -mno-* flags.

As a result, on any build host with AVX the compiler is free to use
SSE/AVX registers throughout the kernel, for instance to zero or copy
structures. A trivial reproducer:

  struct s { long a, b, c, d; };
  void f(struct s *p) { struct s z = {0}; *p = z; }

  $ gcc -mno-sse -mno-sse2 -mno-avx -march=native -Os -c t.c
  vpxor   %xmm0,%xmm0,%xmm0
  vmovdqu %ymm0,(%rdi)

On an Ivy Bridge machine (i5-3317U, GCC 14.2) a v7.3 kernel built this
way resets immediately after the bootloader hands over control. Booting
the same image in QEMU with "-d int,cpu_reset -no-reboot" shows:

  v=06 (#UD) IP=0010:000000000125e47b
  v=0d (#GP) -> v=08 (#DF) -> Triple fault

  __pi_startup_64_load_idt+0xb:
  c5 f9 ef c0    vpxor  %xmm0,%xmm0,%xmm0

The startup code executes an AVX instruction before CR4.OSXSAVE/XCR0
are set up and before an IDT is installed, so the #UD escalates to a
triple fault. Even past early boot, compiler-generated SIMD code would
silently clobber user space FPU state, since the kernel does not save
it on kernel entry outside of kernel_fpu_begin()/kernel_fpu_end().

Re-apply the -mno-* flags after -march=native, the same way EGPR use
is already suppressed for native builds.

Fixes: ea1dcca1de12 ("x86/kbuild/64: Add the CONFIG_X86_NATIVE_CPU option to locally optimize the kernel with '-march=native'")
Cc: stable@vger.kernel.org
Signed-off-by: Antoni Kuczyński <antoni.kucz@gmail.com>
---

Notes:
    Tested: v7.3 + this patch, CONFIG_X86_NATIVE_CPU=y, GCC 14.2.0 (Debian),
    boots on i5-3317U. arch/x86/boot/startup/gdt_idt.o contains no xmm/ymm
    instructions after the change (previously 2).

 arch/x86/Makefile | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/arch/x86/Makefile b/arch/x86/Makefile
index b7e08519b8c1..2c0575fd7bbc 100644
--- a/arch/x86/Makefile
+++ b/arch/x86/Makefile
@@ -161,6 +161,9 @@ else
 
 ifdef CONFIG_X86_NATIVE_CPU
         KBUILD_CFLAGS += -march=native
+        # -march=native expands to explicit -msse/-mavx/... flags that
+        # override the earlier -mno-* ones; disable FP/SIMD code again.
+        KBUILD_CFLAGS += -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -mno-sse4a
         # Prevent the compiler from generating EGPR use. The kernel is
         # not yet prepared for general in-kernel use.
         KBUILD_CFLAGS += $(call cc-option,-mno-apx-features=egpr)
-- 
2.47.3

Re: [PATCH] x86/build/64: Prevent native builds from generating SSE/AVX code

[Antoni Kuczyński]

Adding Nathan, who reviewed the related EGPR fix for native builds
(commit 63edf5a009ae), and the x86 maintainers explicitly - sorry,
the Cc list got dropped when sending.

Patch: https://lore.kernel.org/all/20261025174336.8857-1-antoni.kucz@gmail.com/

Thanks,
Antoni

Re: [PATCH] x86/build/64: Prevent native builds from generating SSE/AVX code

[Nathan Chancellor]

Hi Antoni,

On Sun, Oct 25, 2026 at 06:43:35PM +0100, Antoni Kuczyński wrote:
> With CONFIG_X86_NATIVE_CPU=y, GCC is invoked with -march=native after
> the -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -mno-sse4a flags
> that keep FP/SIMD code out of the kernel. The GCC driver expands
> -march=native into an explicit list of ISA options for the build host
> (e.g. "-march=ivybridge ... -msse4.2 -mavx ..."), and because these come
> later on the command line they override the earlier -mno-* flags.
> 
> As a result, on any build host with AVX the compiler is free to use
> SSE/AVX registers throughout the kernel, for instance to zero or copy
> structures. A trivial reproducer:
> 
>   struct s { long a, b, c, d; };
>   void f(struct s *p) { struct s z = {0}; *p = z; }
> 
>   $ gcc -mno-sse -mno-sse2 -mno-avx -march=native -Os -c t.c
>   vpxor   %xmm0,%xmm0,%xmm0
>   vmovdqu %ymm0,(%rdi)
> 
> On an Ivy Bridge machine (i5-3317U, GCC 14.2) a v7.3 kernel built this
> way resets immediately after the bootloader hands over control. Booting
> the same image in QEMU with "-d int,cpu_reset -no-reboot" shows:
> 
>   v=06 (#UD) IP=0010:000000000125e47b
>   v=0d (#GP) -> v=08 (#DF) -> Triple fault
> 
>   __pi_startup_64_load_idt+0xb:
>   c5 f9 ef c0    vpxor  %xmm0,%xmm0,%xmm0
> 
> The startup code executes an AVX instruction before CR4.OSXSAVE/XCR0
> are set up and before an IDT is installed, so the #UD escalates to a
> triple fault. Even past early boot, compiler-generated SIMD code would
> silently clobber user space FPU state, since the kernel does not save
> it on kernel entry outside of kernel_fpu_begin()/kernel_fpu_end().
> 
> Re-apply the -mno-* flags after -march=native, the same way EGPR use
> is already suppressed for native builds.
> 
> Fixes: ea1dcca1de12 ("x86/kbuild/64: Add the CONFIG_X86_NATIVE_CPU option to locally optimize the kernel with '-march=native'")

This option has been in the kernel for quite some time and I am sure
people are using it. I wonder why this has not come up sooner. Just
unlucky timing or configuration?

> Cc: stable@vger.kernel.org
> Signed-off-by: Antoni Kuczyński <antoni.kucz@gmail.com>
> ---
> 
> Notes:
>     Tested: v7.3 + this patch, CONFIG_X86_NATIVE_CPU=y, GCC 14.2.0 (Debian),
>     boots on i5-3317U. arch/x86/boot/startup/gdt_idt.o contains no xmm/ymm
>     instructions after the change (previously 2).
> 
>  arch/x86/Makefile | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/arch/x86/Makefile b/arch/x86/Makefile
> index b7e08519b8c1..2c0575fd7bbc 100644
> --- a/arch/x86/Makefile
> +++ b/arch/x86/Makefile
> @@ -161,6 +161,9 @@ else
>  
>  ifdef CONFIG_X86_NATIVE_CPU
>          KBUILD_CFLAGS += -march=native
> +        # -march=native expands to explicit -msse/-mavx/... flags that
> +        # override the earlier -mno-* ones; disable FP/SIMD code again.
> +        KBUILD_CFLAGS += -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -mno-sse4a

Maintaining two copies of these flags (and having them appear in the
command line twice) seems objectively worse than just adding an extra
'ifdef CONFIG_X86_64' and moving the block up so that this happens in
the naturally expected order?

diff --git a/arch/x86/Makefile b/arch/x86/Makefile
index b7e08519b8c1..c18f660f6dc8 100644
--- a/arch/x86/Makefile
+++ b/arch/x86/Makefile
@@ -66,6 +66,21 @@ export REALMODE_CFLAGS
 # e.g.: obj-y += foo_$(BITS).o
 export BITS
 
+ifdef CONFIG_X86_64
+ifdef CONFIG_X86_NATIVE_CPU
+KBUILD_CFLAGS += -march=native
+# Prevent the compiler from generating EGPR use. The kernel is
+# not yet prepared for general in-kernel use.
+KBUILD_CFLAGS += $(call cc-option,-mno-apx-features=egpr)
+
+# generate_rust_target.rs handles Rust APX gating.
+KBUILD_RUSTFLAGS += -Ctarget-cpu=native
+else
+KBUILD_CFLAGS += -march=x86-64 -mtune=generic
+KBUILD_RUSTFLAGS += -Ctarget-cpu=x86-64 -Ztune-cpu=generic
+endif
+endif
+
 #
 # Prevent GCC from generating any FP code by mistake.
 #
@@ -73,6 +88,8 @@ export BITS
 #
 #    https://gcc.gnu.org/bugzilla/show_bug.cgi?id=53383
 #
+# and after -march=native.
+#
 KBUILD_CFLAGS += -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -mno-sse4a
 KBUILD_RUSTFLAGS += --target=$(objtree)/scripts/target.json
 KBUILD_RUSTFLAGS += -Ctarget-feature=-sse,-sse2,-sse3,-ssse3,-sse4.1,-sse4.2,-avx,-avx2
@@ -159,19 +176,6 @@ else
 	# Use -mskip-rax-setup if supported.
 	KBUILD_CFLAGS += -mskip-rax-setup
 
-ifdef CONFIG_X86_NATIVE_CPU
-        KBUILD_CFLAGS += -march=native
-        # Prevent the compiler from generating EGPR use. The kernel is
-        # not yet prepared for general in-kernel use.
-        KBUILD_CFLAGS += $(call cc-option,-mno-apx-features=egpr)
-
-        # generate_rust_target.rs handles Rust APX gating.
-        KBUILD_RUSTFLAGS += -Ctarget-cpu=native
-else
-        KBUILD_CFLAGS += -march=x86-64 -mtune=generic
-        KBUILD_RUSTFLAGS += -Ctarget-cpu=x86-64 -Ztune-cpu=generic
-endif
-
         KBUILD_CFLAGS += -mno-red-zone
         KBUILD_CFLAGS += -mcmodel=kernel
         KBUILD_RUSTFLAGS += -Cno-redzone=y
-- 
Cheers,
Nathan

Re: [PATCH] x86/build/64: Prevent native builds from generating SSE/AVX code

[Antoni Kuczyński]

Hi Nathan,

On Sun, Oct 25, 2026 at 12:44:11PM -0700, Nathan Chancellor wrote:
> This option has been in the kernel for quite some time and I am sure
> people are using it. I wonder why this has not come up sooner. Just
> unlucky timing or configuration?

Configuration, on my side, sorry for the noise. Following up on your
question I found that "gcc" in my PATH was the icecc (icecream 1.4.0)
wrapper. To be able to distribute the compilation, icecc replaces
-march=native with an explicit list of options for the local CPU:

  /usr/bin/gcc -mno-sse -mno-avx -Os -march=ivybridge -mmmx -mpopcnt
    -msse -msse2 -msse3 -mssse3 -msse4.1 -msse4.2 -mavx ...

so the expanded -msse/-mavx come after the kernel's -mno-* flags and
win. Plain GCC 14.2 does not have this problem: the reproducer from
the changelog built with /usr/bin/gcc-14 emits no SIMD instructions,
as explicit -mno-* options take precedence over what -march=native
enables.

So the changelog's explanation and the Fixes:/stable tags are wrong,
please drop this patch. I will report the issue to icecream instead.

> Maintaining two copies of these flags (and having them appear in the
> command line twice) seems objectively worse than just adding an extra
> 'ifdef CONFIG_X86_64' and moving the block up so that this happens in
> the naturally expected order?

Agreed, and FWIW your diff on top of current master also fixes the
icecc build here: -march=native now comes before the -mno-* flags and
arch/x86/boot/startup/gdt_idt.o contains no SIMD instructions. If you
think that ordering is worth having on its own merits, I am happy to
send it as a cleanup without Fixes:/stable, but I understand if it is
not worth the churn for an icecc-only problem.

Thanks for the review,
Antoni

Re: [PATCH] x86/build/64: Prevent native builds from generating SSE/AVX code

[Nathan Chancellor]

On Sun, Oct 25, 2026 at 09:07:35PM +0100, Antoni Kuczyński wrote:
> On Sun, Oct 25, 2026 at 12:44:11PM -0700, Nathan Chancellor wrote:
> > This option has been in the kernel for quite some time and I am sure
> > people are using it. I wonder why this has not come up sooner. Just
> > unlucky timing or configuration?
> 
> Configuration, on my side, sorry for the noise. Following up on your
> question I found that "gcc" in my PATH was the icecc (icecream 1.4.0)
> wrapper. To be able to distribute the compilation, icecc replaces
> -march=native with an explicit list of options for the local CPU:
> 
>   /usr/bin/gcc -mno-sse -mno-avx -Os -march=ivybridge -mmmx -mpopcnt
>     -msse -msse2 -msse3 -mssse3 -msse4.1 -msse4.2 -mavx ...
> 
> so the expanded -msse/-mavx come after the kernel's -mno-* flags and
> win. Plain GCC 14.2 does not have this problem: the reproducer from
> the changelog built with /usr/bin/gcc-14 emits no SIMD instructions,
> as explicit -mno-* options take precedence over what -march=native
> enables.
> 
> So the changelog's explanation and the Fixes:/stable tags are wrong,
> please drop this patch. I will report the issue to icecream instead.

Ah yeah, that makes a lot more sense. I guess the easiest solution is
probably something like this patch internally where the expanded value
of '-march=native' is moved to the front of the GCC command so that any
overrides can happen naturally.

> > Maintaining two copies of these flags (and having them appear in the
> > command line twice) seems objectively worse than just adding an extra
> > 'ifdef CONFIG_X86_64' and moving the block up so that this happens in
> > the naturally expected order?
> 
> Agreed, and FWIW your diff on top of current master also fixes the
> icecc build here: -march=native now comes before the -mno-* flags and
> arch/x86/boot/startup/gdt_idt.o contains no SIMD instructions. If you
> think that ordering is worth having on its own merits, I am happy to
> send it as a cleanup without Fixes:/stable, but I understand if it is
> not worth the churn for an icecc-only problem.

I guess that is up to the x86 maintainers. I would personally be fine
with it since the workaround is basically an extra ifdef and comment,
which I do not see an substantial maintenance burden, but they might
feel otherwise.

-- 
Cheers,
Nathan