Re: [PATCH] x86: add hintable NOPs emulation

["H. Peter Anvin" <hpa@zytor.com>]

On August 19, 2025 6:34:46 PM PDT, Marcos Del Sol Vives <marcos@orca.pet> wrote:
>Hintable NOPs are a series of instructions introduced by Intel with the
>Pentium Pro (i686), and described in US patent US5701442A.
>
>These instructions were reserved to allow backwards-compatible changes
>in the instruction set possible, by having old processors treat them as
>variable-length NOPs, while having other semantics in modern processors.
>
>Some modern uses are:
> - Multi-byte/long NOPs
> - Indirect Branch Tracking (ENDBR32)
> - Shadow Stack (part of CET)
>
>Some processors advertising i686 compatibility lack full support for
>them, which may cause #UD to be incorrectly triggered, crashing software
>that uses then with an unexpected SIGILL.
>
>One such software is sudo in Debian bookworm, which is compiled with
>GCC -fcf-protection=branch and contains ENDBR32 instructions. It crashes
>on my Vortex86DX3 processor and VIA C3 Nehalem processors [1].
>
>This patch is a much simplified version of my previous patch for x86
>instruction emulation [2], that only emulates hintable NOPs.
>
>When #UD is raised, it checks if the opcode corresponds to a hintable NOP
>in user space. If true, it warns the user via the dmesg and advances the
>instruction pointer, thus emulating its expected NOP behaviour.
>
>[1]: https://lists.debian.org/debian-devel/2023/10/msg00118.html
>[2]: https://lore.kernel.org/all/20210626130313.1283485-1-marcos@orca.pet/
>
>Signed-off-by: Marcos Del Sol Vives <marcos@orca.pet>
>---
> arch/x86/Kconfig                 | 29 +++++++++++++++++++++++++
> arch/x86/include/asm/processor.h |  4 ++++
> arch/x86/kernel/process.c        |  3 +++
> arch/x86/kernel/traps.c          | 36 ++++++++++++++++++++++++++++++++
> 4 files changed, 72 insertions(+)
>
>diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
>index 58d890fe2100..a6daebdc2573 100644
>--- a/arch/x86/Kconfig
>+++ b/arch/x86/Kconfig
>@@ -1286,6 +1286,35 @@ config X86_IOPL_IOPERM
> 	  ability to disable interrupts from user space which would be
> 	  granted if the hardware IOPL mechanism would be used.
> 
>+config X86_HNOP_EMU
>+	bool "Hintable NOPs emulation"
>+	depends on X86_32
>+	default y
>+	help
>+	  Hintable NOPs are a series of instructions introduced by Intel with
>+	  the Pentium Pro (i686), and described in US patent US5701442A.
>+
>+	  These instructions were reserved to allow backwards-compatible
>+	  changes in the instruction set possible, by having old processors
>+	  treat them as variable-length NOPs, while having other semantics in
>+	  modern processors.
>+
>+	  Some modern uses are:
>+	   - Multi-byte/long NOPs
>+	   - Indirect Branch Tracking (ENDBR32)
>+	   - Shadow Stack (part of CET)
>+
>+	  Some processors advertising i686 compatibility (such as Cyrix MII,
>+	  VIA C3 Nehalem or DM&P Vortex86DX3) lack full support for them,
>+	  which may cause SIGILL to be incorrectly raised in user space when
>+	  a hintable NOP is encountered.
>+
>+	  Say Y here if you want the kernel to emulate them, allowing programs
>+	  that make use of them to run transparently on such processors.
>+
>+	  This emulation has no performance penalty for processors that
>+	  properly support them, so if unsure, enable it.
>+
> config TOSHIBA
> 	tristate "Toshiba Laptop support"
> 	depends on X86_32
>diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
>index bde58f6510ac..c34fb678c4de 100644
>--- a/arch/x86/include/asm/processor.h
>+++ b/arch/x86/include/asm/processor.h
>@@ -499,6 +499,10 @@ struct thread_struct {
> 
> 	unsigned int		iopl_warn:1;
> 
>+#ifdef CONFIG_X86_HNOP_EMU
>+	unsigned int		hnop_warn:1;
>+#endif
>+
> 	/*
> 	 * Protection Keys Register for Userspace.  Loaded immediately on
> 	 * context switch. Store it in thread_struct to avoid a lookup in
>diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
>index 1b7960cf6eb0..6ec8021638d0 100644
>--- a/arch/x86/kernel/process.c
>+++ b/arch/x86/kernel/process.c
>@@ -178,6 +178,9 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
> 	p->thread.io_bitmap = NULL;
> 	clear_tsk_thread_flag(p, TIF_IO_BITMAP);
> 	p->thread.iopl_warn = 0;
>+#ifdef CONFIG_X86_HNOP_EMU
>+	p->thread.hnop_warn = 0;
>+#endif
> 	memset(p->thread.ptrace_bps, 0, sizeof(p->thread.ptrace_bps));
> 
> #ifdef CONFIG_X86_64
>diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c
>index 36354b470590..2dcb7d7edf8a 100644
>--- a/arch/x86/kernel/traps.c
>+++ b/arch/x86/kernel/traps.c
>@@ -295,12 +295,48 @@ DEFINE_IDTENTRY(exc_overflow)
> 	do_error_trap(regs, 0, "overflow", X86_TRAP_OF, SIGSEGV, 0, NULL);
> }
> 
>+#ifdef CONFIG_X86_HNOP_EMU
>+static bool handle_hnop(struct pt_regs *regs)
>+{
>+	struct thread_struct *t = &current->thread;
>+	unsigned char buf[MAX_INSN_SIZE];
>+	unsigned long nr_copied;
>+	struct insn insn;
>+
>+	nr_copied = insn_fetch_from_user(regs, buf);
>+	if (nr_copied <= 0)
>+		return false;
>+
>+	if (!insn_decode_from_regs(&insn, regs, buf, nr_copied))
>+		return false;
>+
>+	/* Hintable NOPs cover 0F 18 to 0F 1F */
>+	if (insn.opcode.bytes[0] != 0x0F ||
>+		insn.opcode.bytes[1] < 0x18 || insn.opcode.bytes[1] > 0x1F)
>+		return false;
>+
>+	if (!t->hnop_warn) {
>+		pr_warn_ratelimited("%s[%d] emulating hintable NOP, ip:%lx\n",
>+		       current->comm, task_pid_nr(current), regs->ip);
>+		t->hnop_warn = 1;
>+	}
>+
>+	regs->ip += insn.length;
>+	return true;
>+}
>+#endif
>+
> #ifdef CONFIG_X86_F00F_BUG
> void handle_invalid_op(struct pt_regs *regs)
> #else
> static inline void handle_invalid_op(struct pt_regs *regs)
> #endif
> {
>+#ifdef CONFIG_X86_HNOP_EMU
>+	if (user_mode(regs) && handle_hnop(regs))
>+		return;
>+#endif
>+
> 	do_error_trap(regs, 0, "invalid opcode", X86_TRAP_UD, SIGILL,
> 		      ILL_ILLOPN, error_get_trap_addr(regs));
> }

Do those processors support FCOMI?