mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
To: tglx@kernel.org, peterz@infradead.org
Cc: linux-kernel@vger.kernel.org,
	Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
	mingo@redhat.com, juri.lelli@redhat.com,
	vincent.guittot@linaro.org, dietmar.eggemann@arm.com,
	rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de,
	vschneid@redhat.com, kprateek.nayak@amd.com, paulmck@kernel.org,
	boqun@kernel.org, shuah@kernel.org, kees@kernel.org,
	viro@zeniv.linux.org.uk, brauner@kernel.org, jack@suse.cz,
	oleg@redhat.com, bestswngs@gmail.com, xmei5@asu.edu,
	xujiakai24@mails.ucas.ac.cn, linux-mm@kvack.org,
	linux-fsdevel@vger.kernel.org, linux-kselftest@vger.kernel.org
Subject: [PATCH 16/17] selftests/rseq: Test affinity changes while a task replaces or drops its mm
Date: Tue, 17 Nov 2026 11:17:16 -0500	[thread overview]
Message-ID: <ccc2a6c50429b1ad9c8d5cc45ec8e58c12df7fde.1794932121.git.mathieu.desnoyers@efficios.com> (raw)
In-Reply-To: <cover.1794932121.git.mathieu.desnoyers@efficios.com>

sched_setaffinity() on another task updates the allowed CPUs of the mm
of the target. set_cpus_allowed_common() reads p->mm under p->pi_lock
for that. The commit "sched/mmcid: Synchronize MM teardown with affinity
updates" takes the same lock once p->mm has changed, before the old mm
is dropped.

No test runs an affinity change against such a teardown. Weiming Shi
reported the use after free with a reproducer: one process calls execve()
in a loop while other processes change its affinity.

Add mm_cid_teardown_test. Its first case follows that reproducer. One
process calls execve() on the test in a loop on its own CPU. Up to four
other processes change its affinity between one and two CPUs.

The second case drops an mm instead of replacing one. Short lived
processes exit on one CPU. Another process changes their affinity from a
second CPU. No case covers kthread_unuse_mm().

User space cannot see a read of the freed mm. Only KASAN reports it. So
both cases check that the two sides keep making progress.

The test starts processes, not threads. glibc needs futex(2) to create a
thread. In the exec case, the use after free is only reachable without
CONFIG_FUTEX. With CONFIG_FUTEX, futex_exec_done() already takes
p->pi_lock after the switch to the new mm.

Both cases pass on guests with 2 to 64 CPUs. The first case needs 3 CPUs
and is skipped below that. On the 8-CPU guest:

  # 1629 execve calls, 427211 affinity changes
  ok 1 affinity changes while a task replaces its mm
  # 2140 processes exited, 86658 affinity changes
  ok 2 affinity changes while a task drops its mm

A guest with KASAN and without CONFIG_FUTEX ran ten seconds per case on
8 CPUs. The kernel reported nothing. This kernel had that commit. The
test was not run on a kernel without it.

Suggested-by: Weiming Shi <bestswngs@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Link: https://lore.kernel.org/all/20261110183809.3308868-2-bestswngs@gmail.com/
---
 tools/testing/selftests/rseq/Makefile         |   2 +-
 .../selftests/rseq/mm_cid_teardown_test.c     | 396 ++++++++++++++++++
 2 files changed, 397 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/rseq/mm_cid_teardown_test.c

diff --git a/tools/testing/selftests/rseq/Makefile b/tools/testing/selftests/rseq/Makefile
index d72502fc7652..2735704f516b 100644
--- a/tools/testing/selftests/rseq/Makefile
+++ b/tools/testing/selftests/rseq/Makefile
@@ -22,7 +22,7 @@ TEST_GEN_PROGS = basic_test basic_percpu_ops_test basic_percpu_ops_mm_cid_test \
 		 param_test_benchmark param_test_mm_cid_benchmark \
 		 mm_cid_execve_test mm_cid_transition_test \
 		 mm_cid_bound_test mm_cid_exec_affinity_test \
-		 mm_cid_window_test
+		 mm_cid_window_test mm_cid_teardown_test
 
 TEST_GEN_PROGS_EXTENDED = librseq.so \
 	param_test \
diff --git a/tools/testing/selftests/rseq/mm_cid_teardown_test.c b/tools/testing/selftests/rseq/mm_cid_teardown_test.c
new file mode 100644
index 000000000000..f07a566901f8
--- /dev/null
+++ b/tools/testing/selftests/rseq/mm_cid_teardown_test.c
@@ -0,0 +1,396 @@
+// SPDX-License-Identifier: LGPL-2.1
+/*
+ * Affinity changes from another task while a task replaces or drops its mm.
+ *
+ * sched_setaffinity(2) on another task updates the allowed CPUs of the mm
+ * of that task. It reads that mm under the pi_lock of the target. The
+ * target can replace its mm in execve(2), or drop it when it exits, on
+ * another CPU at the same time. The old mm is freed right after.
+ *
+ * Case 1: one process calls execve(2) on this test in a loop. Processes on
+ * the other CPUs change its affinity as fast as they can.
+ *
+ * Case 2: short lived processes exit while another process changes their
+ * affinity.
+ *
+ * A read of a freed mm stays invisible to user space. Both cases check
+ * that both sides keep making progress, and that the kernel stays alive.
+ * Under KASAN the kernel reports the read.
+ *
+ * The test uses no thread, so that it also runs on a kernel built without
+ * CONFIG_FUTEX.
+ *
+ * Each case runs for one second. The first argument sets another number of
+ * seconds.
+ */
+
+#define _GNU_SOURCE
+#include <errno.h>
+#include <poll.h>
+#include <sched.h>
+#include <signal.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/eventfd.h>
+#include <sys/mman.h>
+#include <sys/prctl.h>
+#include <sys/wait.h>
+#include <time.h>
+#include <unistd.h>
+
+#include "rseq.h"
+#include "kselftest.h"
+
+#define MAX_CPUS		256
+#define MAX_SETTERS		4
+#define RUNTIME_S		1
+#define FIRST_EXEC_MS		5000
+#define EXIT_DELAY_STEP_US	7
+#define MAX_EXIT_DELAY_US	200
+
+#define CHILD_PASS		0
+#define CHILD_ERROR		2
+
+/* Shared with the processes that change the affinity. */
+struct shared {
+	unsigned long long	changes[MAX_SETTERS];
+	pid_t			target;		/* the process that exits */
+};
+
+static const char *exec_case = "affinity changes while a task replaces its mm";
+static const char *exit_case = "affinity changes while a task drops its mm";
+
+static int cpus[MAX_CPUS], nr_cpus;
+
+static long long now_us(void)
+{
+	struct timespec ts;
+
+	clock_gettime(CLOCK_MONOTONIC, &ts);
+	return ts.tv_sec * 1000000LL + ts.tv_nsec / 1000;
+}
+
+static void busy_wait_us(int us)
+{
+	long long end = now_us() + us;
+
+	while (now_us() < end)
+		;
+}
+
+static int pin(int cpu)
+{
+	cpu_set_t set;
+
+	CPU_ZERO(&set);
+	CPU_SET(cpu, &set);
+	return sched_setaffinity(0, sizeof(set), &set);
+}
+
+/* Set @one to the CPU @a, and @two to the CPUs @a and @b. */
+static void init_masks(cpu_set_t *one, cpu_set_t *two, int a, int b)
+{
+	CPU_ZERO(one);
+	CPU_SET(a, one);
+	CPU_ZERO(two);
+	CPU_SET(a, two);
+	CPU_SET(b, two);
+}
+
+/*
+ * Change the affinity of @pid, or of the process that exits when @pid is
+ * zero, until @deadline. The two masks alternate, so that every change
+ * reaches the mm of the target. Count the changes and exit.
+ */
+static void setter_main(struct shared *sh, int index, pid_t pid, int cpu,
+			int a, int b, long long deadline)
+{
+	unsigned long long changes = 0;
+	cpu_set_t one, two;
+	int i = 0;
+
+	if (prctl(PR_SET_PDEATHSIG, SIGKILL, 0, 0, 0)) {
+		fprintf(stderr, "prctl(PR_SET_PDEATHSIG): %s\n", strerror(errno));
+		_exit(CHILD_ERROR);
+	}
+	init_masks(&one, &two, a, b);
+	if (pin(cpu)) {
+		fprintf(stderr, "sched_setaffinity: %s\n", strerror(errno));
+		_exit(CHILD_ERROR);
+	}
+	while (now_us() < deadline) {
+		const cpu_set_t *mask = (i++ & 1) ? &one : &two;
+		pid_t target = pid ? pid : RSEQ_READ_ONCE(sh->target);
+
+		if (!target)
+			continue;
+		if (!sched_setaffinity(target, sizeof(*mask), mask)) {
+			changes++;
+			continue;
+		}
+		/* The target can exit first: ESRCH is expected. */
+		if (errno != ESRCH) {
+			fprintf(stderr, "sched_setaffinity: %s\n", strerror(errno));
+			_exit(CHILD_ERROR);
+		}
+	}
+	sh->changes[index] = changes;
+	_exit(CHILD_PASS);
+}
+
+/* Count this instance of the process under test. Then call execve(2) again. */
+static int exec_loop_main(char **argv)
+{
+	long long end = atoll(argv[3]);
+	uint64_t one = 1;
+	int fd = atoi(argv[2]);
+
+	if (write(fd, &one, sizeof(one)) != sizeof(one))
+		return CHILD_ERROR;
+	if (now_us() >= end)
+		return CHILD_PASS;
+	execv("/proc/self/exe", argv);
+	fprintf(stderr, "execv: %s\n", strerror(errno));
+	return CHILD_ERROR;
+}
+
+/* Add the counts pending on @fd to @total. */
+static int drain(int fd, unsigned long long *total)
+{
+	for (;;) {
+		uint64_t value;
+		ssize_t ret = read(fd, &value, sizeof(value));
+
+		if (ret == sizeof(value)) {
+			*total += value;
+			continue;
+		}
+		if (ret < 0 && errno == EINTR)
+			continue;
+		if (ret < 0 && errno == EAGAIN)
+			return 0;
+		return -1;
+	}
+}
+
+/*
+ * Wait for the first instance of the process under test. Return -1 if it
+ * does not come.
+ */
+static int wait_first_exec(int fd, unsigned long long *execs)
+{
+	struct pollfd pfd = { .fd = fd, .events = POLLIN };
+	int ret;
+
+	do {
+		ret = poll(&pfd, 1, FIRST_EXEC_MS);
+	} while (ret < 0 && errno == EINTR);
+	if (ret != 1)
+		return -1;
+	if (drain(fd, execs) || !*execs)
+		return -1;
+	return 0;
+}
+
+/*
+ * Start the process under test on the CPU @cpu. It counts its instances
+ * through @fd and leaves its loop at @end. Return its pid.
+ */
+static pid_t start_exec_loop(char *name, int cpu, int fd, long long end)
+{
+	char mode[] = "exec-loop", fd_arg[16], end_arg[32];
+	char *const av[] = { name, mode, fd_arg, end_arg, NULL };
+	pid_t pid;
+
+	snprintf(fd_arg, sizeof(fd_arg), "%d", fd);
+	snprintf(end_arg, sizeof(end_arg), "%lld", end);
+	pid = fork();
+	if (pid)
+		return pid;
+	if (prctl(PR_SET_PDEATHSIG, SIGKILL, 0, 0, 0)) {
+		fprintf(stderr, "prctl(PR_SET_PDEATHSIG): %s\n", strerror(errno));
+		_exit(CHILD_ERROR);
+	}
+	if (pin(cpu)) {
+		fprintf(stderr, "sched_setaffinity: %s\n", strerror(errno));
+		_exit(CHILD_ERROR);
+	}
+	execv("/proc/self/exe", av);
+	fprintf(stderr, "execv: %s\n", strerror(errno));
+	_exit(CHILD_ERROR);
+}
+
+/* Wait for the process @pid. Return its exit status, or -1. */
+static int wait_status(pid_t pid)
+{
+	int status;
+	pid_t ret;
+
+	do {
+		ret = waitpid(pid, &status, 0);
+	} while (ret < 0 && errno == EINTR);
+	if (ret != pid || !WIFEXITED(status))
+		return -1;
+	return WEXITSTATUS(status);
+}
+
+/* Case 1. Change the affinity of a process that calls execve(2) in a loop. */
+static void run_exec_case(char *name, int runtime_s, struct shared *sh)
+{
+	unsigned long long execs = 0, changes = 0;
+	int a = cpus[nr_cpus - 2], b = cpus[nr_cpus - 1];
+	pid_t victim, setters[MAX_SETTERS];
+	int fd, i, nset, status = 0;
+	long long deadline;
+	pid_t ret;
+
+	nset = nr_cpus - 2 < MAX_SETTERS ? nr_cpus - 2 : MAX_SETTERS;
+	fd = eventfd(0, EFD_NONBLOCK);
+	if (fd < 0)
+		ksft_exit_fail_msg("eventfd: %s\n", strerror(errno));
+	/* The process under test outlives those that change its affinity. */
+	victim = start_exec_loop(name, a, fd,
+				 now_us() + (runtime_s + 1) * 1000000LL);
+	if (victim < 0)
+		ksft_exit_fail_msg("fork: %s\n", strerror(errno));
+	if (wait_first_exec(fd, &execs))
+		ksft_exit_fail_msg("the process under test did not call execve\n");
+
+	deadline = now_us() + runtime_s * 1000000LL;
+	for (i = 0; i < nset; i++) {
+		setters[i] = fork();
+		if (!setters[i])
+			setter_main(sh, i, victim, cpus[i], a, b, deadline);
+		if (setters[i] < 0)
+			ksft_exit_fail_msg("fork: %s\n", strerror(errno));
+	}
+	for (i = 0; i < nset; i++) {
+		if (wait_status(setters[i]))
+			ksft_exit_fail_msg("a process that changes the affinity failed\n");
+		changes += sh->changes[i];
+	}
+	if (drain(fd, &execs))
+		ksft_exit_fail_msg("read the count of the execve calls: %s\n",
+				   strerror(errno));
+
+	/* The process under test must still be in its loop. */
+	ret = waitpid(victim, &status, WNOHANG);
+	if (ret < 0)
+		ksft_exit_fail_msg("waitpid: %s\n", strerror(errno));
+	if (ret) {
+		ksft_print_msg("the process under test left its loop: 0x%x\n",
+			       status);
+		ksft_test_result_fail("%s\n", exec_case);
+		if (close(fd))
+			ksft_exit_fail_msg("close: %s\n", strerror(errno));
+		return;
+	}
+	if (kill(victim, SIGKILL))
+		ksft_exit_fail_msg("kill: %s\n", strerror(errno));
+	if (waitpid(victim, &status, 0) != victim)
+		ksft_exit_fail_msg("waitpid: %s\n", strerror(errno));
+	if (close(fd))
+		ksft_exit_fail_msg("close: %s\n", strerror(errno));
+
+	ksft_print_msg("%llu execve calls, %llu affinity changes\n",
+		       execs, changes);
+	ksft_test_result(execs && changes, "%s\n", exec_case);
+}
+
+/* Case 2. Change the affinity of processes while they exit. */
+static void run_exit_case(int runtime_s, const cpu_set_t *all,
+			  struct shared *sh)
+{
+	unsigned long long children = 0;
+	int a = cpus[0], b = cpus[1];
+	int delay = 0, bad = 0;
+	long long deadline;
+	pid_t setter, pid;
+
+	deadline = now_us() + runtime_s * 1000000LL;
+	if (pin(a))
+		ksft_exit_fail_msg("sched_setaffinity: %s\n", strerror(errno));
+	setter = fork();
+	if (!setter)
+		setter_main(sh, 0, 0, b, a, b, deadline);
+	if (setter < 0)
+		ksft_exit_fail_msg("fork: %s\n", strerror(errno));
+
+	while (now_us() < deadline) {
+		pid = fork();
+		if (!pid) {
+			/* Exit while the affinity of this process changes. */
+			busy_wait_us(delay);
+			_exit(CHILD_PASS);
+		}
+		if (pid < 0)
+			ksft_exit_fail_msg("fork: %s\n", strerror(errno));
+		RSEQ_WRITE_ONCE(sh->target, pid);
+		if (wait_status(pid))
+			bad++;
+		RSEQ_WRITE_ONCE(sh->target, 0);
+		children++;
+		/* The next process exits later, relative to the changes. */
+		delay = (delay + EXIT_DELAY_STEP_US) % MAX_EXIT_DELAY_US;
+	}
+	if (wait_status(setter))
+		ksft_exit_fail_msg("the process that changes the affinity failed\n");
+	if (sched_setaffinity(0, sizeof(*all), all))
+		ksft_exit_fail_msg("sched_setaffinity: %s\n", strerror(errno));
+
+	if (bad)
+		ksft_print_msg("%d processes out of %llu did not exit cleanly\n",
+			       bad, children);
+	ksft_print_msg("%llu processes exited, %llu affinity changes\n",
+		       children, sh->changes[0]);
+	ksft_test_result(!bad && children && sh->changes[0], "%s\n", exit_case);
+}
+
+int main(int argc, char **argv)
+{
+	int runtime_s = RUNTIME_S;
+	struct shared *sh;
+	cpu_set_t all;
+	int cpu;
+
+	if (argc == 4 && !strcmp(argv[1], "exec-loop"))
+		return exec_loop_main(argv);
+	if (argc > 1 && atoi(argv[1]) > 0)
+		runtime_s = atoi(argv[1]);
+
+	ksft_print_header();
+	ksft_set_plan(2);
+
+	if (!rseq_available())
+		ksft_exit_skip("rseq is unavailable\n");
+	if (rseq_register_current_thread())
+		ksft_exit_fail_msg("rseq_register_current_thread: %s\n",
+				   strerror(errno));
+	if (!rseq_mm_cid_available())
+		ksft_exit_skip("the rseq mm_cid field is unavailable\n");
+	if (sched_getaffinity(0, sizeof(all), &all))
+		ksft_exit_fail_msg("sched_getaffinity: %s\n", strerror(errno));
+	for (cpu = 0; cpu < CPU_SETSIZE && nr_cpus < MAX_CPUS; cpu++) {
+		if (CPU_ISSET(cpu, &all))
+			cpus[nr_cpus++] = cpu;
+	}
+	sh = mmap(NULL, sizeof(*sh), PROT_READ | PROT_WRITE,
+		  MAP_SHARED | MAP_ANONYMOUS, -1, 0);
+	if (sh == MAP_FAILED)
+		ksft_exit_fail_msg("mmap: %s\n", strerror(errno));
+
+	/* One CPU for the tested process, one to move it to, one to run on. */
+	if (nr_cpus < 3)
+		ksft_test_result_skip("%s: fewer than 3 CPUs\n", exec_case);
+	else
+		run_exec_case(argv[0], runtime_s, sh);
+
+	if (nr_cpus < 2)
+		ksft_test_result_skip("%s: fewer than 2 CPUs\n", exit_case);
+	else
+		run_exit_case(runtime_s, &all, sh);
+
+	ksft_finished();
+}
-- 
2.47.3


  parent reply	other threads:[~2026-11-17 16:17 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-17 16:17 [PATCH 00/17] sched/mmcid: fixes for execve, mode switches and affinity Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 01/17] sched/mmcid: Give a CID to a task that joins its mm while running Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 02/17] sched/mmcid: Give no concurrency ID to user workers Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 03/17] sched/mmcid: Reset stale state when the first user joins an mm Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 04/17] sched/mmcid: Let a running joiner take its CID after the fixups Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 05/17] sched/mmcid: Take the allowed CPUs of the first user when it joins Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 06/17] sched/mmcid: Synchronize MM teardown with affinity updates Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 07/17] selftests/rseq: Test the concurrency ID after a failed execve Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 08/17] selftests/rseq: Test the concurrency ID after the last user fails execve Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 09/17] selftests/rseq: Test the concurrency IDs across ownership mode switches Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 10/17] selftests/rseq: Test the upper bound of the concurrency IDs Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 11/17] selftests/rseq: Test affinity changes made by another process Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 12/17] selftests/rseq: Test an affinity change by another process during execve Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 13/17] selftests/rseq: Test concurrent thread creation and forced migration Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 14/17] selftests/rseq: Test the concurrency IDs after the affinity is widened Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 15/17] selftests/rseq: Test two race windows of the concurrency ID mode switches Mathieu Desnoyers
2026-11-17 16:17 ` Mathieu Desnoyers [this message]
2026-11-17 16:17 ` [PATCH 17/17] selftests/rseq: Test the last user after an affinity change during execve Mathieu Desnoyers

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ccc2a6c50429b1ad9c8d5cc45ec8e58c12df7fde.1794932121.git.mathieu.desnoyers@efficios.com \
    --to=mathieu.desnoyers@efficios.com \
    --cc=bestswngs@gmail.com \
    --cc=boqun@kernel.org \
    --cc=brauner@kernel.org \
    --cc=bsegall@google.com \
    --cc=dietmar.eggemann@arm.com \
    --cc=jack@suse.cz \
    --cc=juri.lelli@redhat.com \
    --cc=kees@kernel.org \
    --cc=kprateek.nayak@amd.com \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mgorman@suse.de \
    --cc=mingo@redhat.com \
    --cc=oleg@redhat.com \
    --cc=paulmck@kernel.org \
    --cc=peterz@infradead.org \
    --cc=rostedt@goodmis.org \
    --cc=shuah@kernel.org \
    --cc=tglx@kernel.org \
    --cc=vincent.guittot@linaro.org \
    --cc=viro@zeniv.linux.org.uk \
    --cc=vschneid@redhat.com \
    --cc=xmei5@asu.edu \
    --cc=xujiakai24@mails.ucas.ac.cn \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®