[PATCH 0/2] x86/shstk: don't leak a shadow stack when clone TLS fails

[PATCH 0/2] x86/shstk: don't leak a shadow stack when clone TLS fails

[Iván Ezequiel Rodriguez]

copy_thread() maps a shadow stack and set_new_tls() can still fail.
That error goes to bad_fork_cleanup_io and skips exit_thread().
bad_fork_cleanup_thread calls exit_thread() only after copy_thread()
has returned 0.

commit 748c90c69336 ("x86/shstk: Remove useless clone error handling")
removed the shstk_free() on the set_new_tls() failure. Restoring it
only there puts back the single free. The allocation-error return
still aliases the parent, and the exit_thread() path is left alone,
so the double unmap that commit avoided does not come back.

The selftest assumes /proc is mounted. A clone3() with a valid TLS
must succeed before EPERM counts as the TLS failure. No CPUID support,
no kernel support, or a locked feature skips the test. Any other
ARCH_SHSTK_ENABLE error fails it.

Sixteen clone3() calls with CLONE_VM|CLONE_SETTLS and an out-of-range
TLS leaked 16 shadow-stack VMAs without the fix. With it, the count
stayed unchanged and call/return on the parent still used the
original shadow stack. Tested with user shadow stack on the host and
in a one-vCPU KVM guest (-cpu host), same configuration with and
without the fix.

The diagnosis and the selftest were drafted with Grok 4.7.

Iván Ezequiel Rodriguez (2):
  x86/shstk: free shadow stack when copy_thread() fails
  selftests/x86: shadow stack must survive a bad TLS

 arch/x86/kernel/process.c                     |  12 +-
 tools/testing/selftests/x86/Makefile          |   3 +-
 tools/testing/selftests/x86/shstk_clone_tls.c | 364 ++++++++++++++++++
 3 files changed, 376 insertions(+), 3 deletions(-)
 create mode 100644 tools/testing/selftests/x86/shstk_clone_tls.c

-- 
2.43.0

[PATCH 1/2] x86/shstk: free shadow stack when copy_thread() fails

[Iván Ezequiel Rodriguez]

copy_thread() maps a shadow stack for a CLONE_VM child and can still
fail in set_new_tls(). That error returns through bad_fork_cleanup_io,
which does not call exit_thread(). bad_fork_cleanup_thread does, but
only after copy_thread() has returned 0.

commit 748c90c69336 ("x86/shstk: Remove useless clone error handling")
removed the shstk_free() on this path, assuming exit_thread() would
run. It does not. Each rejected clone leaves one shadow-stack VMA in
the parent's mm until the mm dies: one map_count slot, pages not
populated. The size follows the clone stack, or RLIMIT_STACK when
that size is zero.

Free only after set_new_tls() fails. The allocation error returns
earlier, while the child's base still aliases the parent.
shstk_free() ignores a cleared base and a different mm, so
CLONE_VFORK and !CLONE_VM stay no-ops. The exit_thread() path is
left alone, so the double unmap that commit avoided does not come
back. The io bitmap is still shared only when TLS setup succeeds.

Tested with user shadow stack enabled, on one config with and without
this change. Sixteen clone3() calls with CLONE_VM|CLONE_SETTLS and an
out-of-range TLS leaked 16 shadow-stack VMAs without the fix. With it,
the VMA count was unchanged and call/return on the parent still used
the original shadow stack.

Fixes: 748c90c69336 ("x86/shstk: Remove useless clone error handling")
Cc: stable@vger.kernel.org
Assisted-by: LLM
Signed-off-by: Iván Ezequiel Rodriguez <ivanrwcm25@gmail.com>
---
 arch/x86/kernel/process.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
index 346c438ac880..be840652d055 100644
--- a/arch/x86/kernel/process.c
+++ b/arch/x86/kernel/process.c
@@ -265,11 +265,19 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
 	/* Set a new TLS for the child thread? */
 	if (clone_flags & CLONE_SETTLS)
 		ret = set_new_tls(p, tls);
+	if (ret) {
+		/*
+		 * copy_process() skips exit_thread() when copy_thread()
+		 * fails. Free the shadow stack mapped before this error.
+		 */
+		shstk_free(p);
+		return ret;
+	}
 
-	if (!ret && unlikely(test_tsk_thread_flag(current, TIF_IO_BITMAP)))
+	if (unlikely(test_tsk_thread_flag(current, TIF_IO_BITMAP)))
 		io_bitmap_share(p);
 
-	return ret;
+	return 0;
 }
 
 static void pkru_flush_thread(void)
-- 
2.43.0

[PATCH 2/2] selftests/x86: shadow stack must survive a bad TLS

[Iván Ezequiel Rodriguez]

clone3(CLONE_VM|CLONE_SETTLS) with an out-of-range TLS fails in
set_new_tls() after copy_thread() has mapped a shadow stack.
copy_process() does not call exit_thread() on that failure, so the
VMA stays in the parent's mm.

Enable user shadow stack, reject 16 such clones, and require the
shadow-stack VMA count to stay unchanged. The original mapping must
remain, and a call/return on that stack must succeed, so the cleanup
cannot be unmapping the parent's shadow stack.

A clone3 with the same flags and a valid TLS has to succeed first.
EPERM from the rejected calls is not evidence on its own: a policy
can reject every clone3(). The child exits with a raw syscall because
it resumes on an empty shadow stack.

Skip when CPUID has no shadow stack, when the kernel does not provide
the feature, or when it is locked. Any other ARCH_SHSTK_ENABLE error
fails the test. The test expects /proc already mounted, as in a
normal kselftest run.

The recursive call goes through an indirect call. A visible recursive
body is optimized into a closed form and does not execute ret.

Assisted-by: LLM
Signed-off-by: Iván Ezequiel Rodriguez <ivanrwcm25@gmail.com>
---
 tools/testing/selftests/x86/Makefile          |   3 +-
 tools/testing/selftests/x86/shstk_clone_tls.c | 364 ++++++++++++++++++
 2 files changed, 366 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/x86/shstk_clone_tls.c

diff --git a/tools/testing/selftests/x86/Makefile b/tools/testing/selftests/x86/Makefile
index d478b13cc8d5..4f8a771b0839 100644
--- a/tools/testing/selftests/x86/Makefile
+++ b/tools/testing/selftests/x86/Makefile
@@ -19,7 +19,8 @@ TARGETS_C_32BIT_ONLY := entry_from_vm86 test_syscall_vdso unwind_vdso \
 			test_FCMOV test_FCOMI test_FISTTP \
 			vdso_restorer
 TARGETS_C_64BIT_ONLY := fsgsbase sysret_rip syscall_numbering \
-			corrupt_xstate_header amx lam test_shadow_stack avx apx
+			corrupt_xstate_header amx lam test_shadow_stack avx apx \
+			shstk_clone_tls
 # Some selftests require 32bit support enabled also on 64bit systems
 TARGETS_C_32BIT_NEEDED := ldt_gdt ptrace_syscall
 
diff --git a/tools/testing/selftests/x86/shstk_clone_tls.c b/tools/testing/selftests/x86/shstk_clone_tls.c
new file mode 100644
index 000000000000..6b2f1001b084
--- /dev/null
+++ b/tools/testing/selftests/x86/shstk_clone_tls.c
@@ -0,0 +1,364 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * clone3(CLONE_VM|CLONE_SETTLS) with an out-of-range TLS must not leak a
+ * shadow-stack VMA. copy_thread() maps one before set_new_tls() can fail,
+ * and copy_process() does not call exit_thread() on that failure.
+ *
+ * Pass: shadow-stack VMA count is unchanged, the original mapping is
+ * still there, and a call/return on this thread still works.
+ * Fail: one new shadow-stack VMA per rejected clone.
+ *
+ * Enable shadow stack with an inlined syscall. A normal function return
+ * across that point has no address on the shadow stack, so main() must
+ * not return after a successful enable.
+ */
+#define _GNU_SOURCE
+
+#include <errno.h>
+#include <linux/sched.h>
+#include <sched.h>
+#include <signal.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/mman.h>
+#include <sys/syscall.h>
+#include <sys/wait.h>
+#include <unistd.h>
+
+#ifndef __NR_clone3
+#define __NR_clone3 435
+#endif
+
+#ifndef ARCH_SHSTK_ENABLE
+#define ARCH_SHSTK_ENABLE	0x5001
+#endif
+#ifndef ARCH_SHSTK_STATUS
+#define ARCH_SHSTK_STATUS	0x5005
+#endif
+#ifndef ARCH_SHSTK_SHSTK
+#define ARCH_SHSTK_SHSTK	1ULL /* bit 0 */
+#endif
+
+/* Same value as KSFT_SKIP in kselftest.h. */
+#define KSFT_SKIP		4
+
+#define NCLONE			16
+#define STACK_SIZE		(64 * 1024)
+
+/* CPUID.(EAX=07H, ECX=0):ECX[7], CET shadow stack. */
+#define CPUID_SHSTK		0x80
+
+#define ARCH_PRCTL(arg1, arg2)					\
+({								\
+	long _ret;						\
+	register long _num  asm("eax") = __NR_arch_prctl;	\
+	register long _arg1 asm("rdi") = (long)(arg1);		\
+	register long _arg2 asm("rsi") = (long)(arg2);		\
+								\
+	asm volatile (						\
+		"syscall\n"					\
+		: "=a"(_ret)					\
+		: "r"(_arg1), "r"(_arg2),			\
+		  "0"(_num)					\
+		: "rcx", "r11", "memory", "cc"			\
+	);							\
+	_ret;							\
+})
+
+#define get_ssp()							\
+({									\
+	unsigned long _ssp = 0;						\
+									\
+	asm volatile("rdsspq %0" : "+r" (_ssp));			\
+	_ssp;								\
+})
+
+static int cpu_has_shstk(void)
+{
+	unsigned int eax, ebx, ecx, edx;
+
+	asm volatile("cpuid"
+		     : "=a" (eax), "=b" (ebx), "=c" (ecx), "=d" (edx)
+		     : "a" (7), "c" (0));
+	return ecx & CPUID_SHSTK;
+}
+
+static void on_fault(int sig)
+{
+	static const char msg[] = "[FAIL]\tparent shadow stack faulted\n";
+
+	(void)sig;
+	if (write(2, msg, sizeof(msg) - 1) < 0)
+		_exit(2);
+	_exit(2);
+}
+
+struct ss_span {
+	unsigned long start;
+	unsigned long end;
+};
+
+static void ss_scan(int *vmas, int *kb, unsigned long addr, struct ss_span *hit,
+		    const struct ss_span *want, int *kept)
+{
+	FILE *f;
+	char line[256];
+	unsigned long start = 0, end = 0;
+	int size = 0;
+	int saw_size = 0;
+
+	*vmas = 0;
+	*kb = 0;
+	if (hit) {
+		hit->start = 0;
+		hit->end = 0;
+	}
+	if (kept)
+		*kept = 0;
+	f = fopen("/proc/self/smaps", "r");
+	if (!f) {
+		perror("smaps");
+		_exit(1);
+	}
+	while (fgets(line, sizeof(line), f)) {
+		unsigned long a, b;
+
+		if (sscanf(line, "%lx-%lx", &a, &b) == 2) {
+			start = a;
+			end = b;
+			saw_size = 0;
+			continue;
+		}
+		if (sscanf(line, "Size: %d kB", &size) == 1) {
+			saw_size = 1;
+			continue;
+		}
+		if (strncmp(line, "VmFlags:", 8) || !saw_size)
+			continue;
+		if (!strstr(line, " ss")) {
+			saw_size = 0;
+			continue;
+		}
+		(*vmas)++;
+		*kb += size;
+		/* Ordinary membership is [start, end). */
+		if (hit && addr >= start && addr < end) {
+			hit->start = start;
+			hit->end = end;
+		}
+		/*
+		 * An empty shadow stack leaves SSP at the exclusive end.
+		 * That address is not inside [start, end).
+		 */
+		if (hit && addr == end) {
+			hit->start = start;
+			hit->end = end;
+		}
+		if (kept && want && start == want->start && end == want->end)
+			*kept = 1;
+		saw_size = 0;
+	}
+	fclose(f);
+}
+
+/*
+ * clone3 carries tls in the struct. 0 is a valid FS base. -1 is rejected
+ * by ARCH_SET_FS before any FS write. No CLONE_THREAD: a created child is
+ * a separate process and _exit() does not tear this one down.
+ *
+ * Returns 0 if the child exited 0, -errno if clone3 failed, 1 otherwise.
+ */
+static int clone_tls(void *stack, size_t stack_size, unsigned long tls)
+{
+	struct clone_args args = {
+		.flags = CLONE_VM | CLONE_SETTLS,
+		.exit_signal = SIGCHLD,
+		.stack = (uint64_t)(uintptr_t)stack,
+		.stack_size = stack_size,
+		.tls = tls,
+	};
+	long ret;
+	int st;
+
+	/*
+	 * Inline syscall. The child resumes here on an empty shadow stack,
+	 * so it must not return into the libc wrapper.
+	 */
+	asm volatile("syscall"
+		     : "=a" (ret)
+		     : "0" (__NR_clone3),
+		       "D" (&args),
+		       "S" (sizeof(args))
+		     : "rcx", "r11", "memory");
+	if (ret == 0) {
+		asm volatile("syscall"
+			     :
+			     : "a" (__NR_exit),
+			       "D" (0)
+			     : "memory");
+		__builtin_unreachable();
+	}
+	if (ret < 0)
+		return ret;
+	if (waitpid((pid_t)ret, &st, 0) < 0)
+		return -errno;
+	if (!WIFEXITED(st) || WEXITSTATUS(st) != 0)
+		return 1;
+	return 0;
+}
+
+/*
+ * GCC 13 turns a visible recursive body into "return n + 1". Hide the
+ * callee in an asm output so each level stays a real indirect call/ret.
+ */
+static int depth(int n);
+static int (*depth_fn)(int) = depth;
+
+static int depth(int n)
+{
+	int (*go)(int n);
+
+	if (n <= 0)
+		return 1;
+	asm volatile("" : "=r" (go) : "0" (depth_fn) : "memory");
+	return go(n - 1) + 1;
+}
+
+static int skip_enable(long rc)
+{
+	if (rc == -EOPNOTSUPP) {
+		printf("[SKIP]\tkernel has no user shadow stack\n");
+	} else if (rc == -EINVAL) {
+		printf("[SKIP]\tkernel built without user shadow stack\n");
+	} else if (rc == -EPERM) {
+		printf("[SKIP]\tuser shadow stack is locked\n");
+	} else {
+		printf("[FAIL]\tARCH_SHSTK_ENABLE returned %ld\n", rc);
+		return 1;
+	}
+	return KSFT_SKIP;
+}
+
+int main(void)
+{
+	struct sigaction sa;
+	struct ss_span parent;
+	void *stack;
+	unsigned long ssp, ssp_mid;
+	long rc;
+	int vmas0, kb0, vmas1, kb1;
+	int i, kept, d, cret;
+
+	setbuf(stdout, NULL);
+
+	if (!cpu_has_shstk()) {
+		printf("[SKIP]\tCPUID has no shadow stack\n");
+		return KSFT_SKIP;
+	}
+
+	stack = mmap(NULL, STACK_SIZE, PROT_READ | PROT_WRITE,
+		     MAP_PRIVATE | MAP_ANONYMOUS | MAP_STACK, -1, 0);
+	if (stack == MAP_FAILED) {
+		perror("mmap");
+		return 1;
+	}
+
+	/*
+	 * rdsspq #UDs while CET is off, so enable before the first read.
+	 * Already enabled is a no-op. Locked-and-enabled returns EPERM
+	 * before that no-op; ARCH_SHSTK_STATUS tells the two apart.
+	 */
+	rc = ARCH_PRCTL(ARCH_SHSTK_ENABLE, ARCH_SHSTK_SHSTK);
+	if (rc == -EPERM) {
+		unsigned long status = 0;
+
+		if (ARCH_PRCTL(ARCH_SHSTK_STATUS, &status) ||
+		    !(status & ARCH_SHSTK_SHSTK))
+			return skip_enable(rc);
+	} else if (rc) {
+		return skip_enable(rc);
+	}
+	if (!get_ssp()) {
+		printf("[FAIL]\tenable left SSP at 0\n");
+		_exit(1);
+	}
+
+	/*
+	 * Same flags as the rejected calls, with a TLS that set_new_tls()
+	 * accepts. EPERM from the rejected calls is only meaningful if this
+	 * one is allowed: a policy can reject every clone3().
+	 */
+	cret = clone_tls(stack, STACK_SIZE, 0);
+	if (cret < 0) {
+		printf("[SKIP]\tclone3 with a valid TLS failed: %s\n",
+		       strerror(-cret));
+		_exit(KSFT_SKIP);
+	}
+	if (cret) {
+		printf("[FAIL]\tclone3 with a valid TLS did not exit 0\n");
+		_exit(1);
+	}
+
+	memset(&sa, 0, sizeof(sa));
+	sa.sa_handler = on_fault;
+	sigaction(SIGSEGV, &sa, NULL);
+	sigaction(SIGBUS, &sa, NULL);
+
+	ssp = get_ssp();
+	ss_scan(&vmas0, &kb0, ssp, &parent, NULL, NULL);
+	if (!parent.start) {
+		printf("[FAIL]\tSSP %lx matches no shadow-stack VMA\n", ssp);
+		_exit(1);
+	}
+
+	for (i = 0; i < NCLONE; i++) {
+		cret = clone_tls(stack, STACK_SIZE, (unsigned long)-1);
+		if (cret == -EPERM)
+			continue;
+		if (cret < 0)
+			printf("[FAIL]\tbad TLS errno %d, want EPERM\n", -cret);
+		else
+			printf("[FAIL]\tbad TLS was accepted\n");
+		_exit(1);
+	}
+
+	ss_scan(&vmas1, &kb1, 0, NULL, &parent, &kept);
+	ssp_mid = get_ssp();
+	d = -1;
+	if (kept && ssp_mid)
+		d = depth(16);
+	ssp = get_ssp();
+
+	printf("ss_vmas=%d->%d delta=%d ss_kb=%d->%d\n",
+	       vmas0, vmas1, vmas1 - vmas0, kb0, kb1);
+	printf("parent=%lx-%lx kept=%d ssp_before_depth=%lx depth=%d ssp=%lx\n",
+	       parent.start, parent.end, kept, ssp_mid, d, ssp);
+
+	if (vmas1 != vmas0 || kb1 != kb0) {
+		printf("[FAIL]\tleaked %d shadow-stack VMA(s), %d kB\n",
+		       vmas1 - vmas0, kb1 - kb0);
+		_exit(1);
+	}
+	if (!kept) {
+		printf("[FAIL]\tparent shadow stack %lx-%lx was unmapped\n",
+		       parent.start, parent.end);
+		_exit(2);
+	}
+	if (!ssp_mid) {
+		printf("[FAIL]\tparent SSP was 0 before call/return\n");
+		_exit(2);
+	}
+	if (d != 17) {
+		printf("[FAIL]\tparent call/return returned %d, want 17\n", d);
+		_exit(2);
+	}
+	if (!ssp) {
+		printf("[FAIL]\tparent SSP is 0 after call/return\n");
+		_exit(2);
+	}
+
+	printf("[OK]\tno shadow-stack VMA leak; parent stack still works\n");
+	_exit(0);
+}
-- 
2.43.0