mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
To: tglx@kernel.org, peterz@infradead.org
Cc: linux-kernel@vger.kernel.org,
	Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
	mingo@redhat.com, juri.lelli@redhat.com,
	vincent.guittot@linaro.org, dietmar.eggemann@arm.com,
	rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de,
	vschneid@redhat.com, kprateek.nayak@amd.com, paulmck@kernel.org,
	boqun@kernel.org, shuah@kernel.org, kees@kernel.org,
	viro@zeniv.linux.org.uk, brauner@kernel.org, jack@suse.cz,
	oleg@redhat.com, bestswngs@gmail.com, xmei5@asu.edu,
	xujiakai24@mails.ucas.ac.cn, linux-mm@kvack.org,
	linux-fsdevel@vger.kernel.org, linux-kselftest@vger.kernel.org,
	stable@vger.kernel.org
Subject: [PATCH 04/17] sched/mmcid: Let a running joiner take its CID after the fixups
Date: Tue, 17 Nov 2026 11:17:04 -0500	[thread overview]
Message-ID: <f1b450d79e9ae67134ed3451b4af33e71cc5dd74.1794932121.git.mathieu.desnoyers@efficios.com> (raw)
In-Reply-To: <cover.1794932121.git.mathieu.desnoyers@efficios.com>

The switch to per CPU mode relies on the new user not running yet. The
comment above the code says so: the new user is not visible in the
system, and the task that forks it runs the fixup.

A failed execve() is missed. It rejoins its mm in
sched_mm_cid_after_execve(): the new user is the running task, and it
runs the fixup itself. If it is preempted before the fixup is done,
mm_get_cid() can loop forever at its schedule in, with the runqueue
lock held.

In the diagram below, the machine has two possible CPUs. A and B are
threads of one process, which is in per task mode with two CIDs. C is
the vfork() child of B. A holds mm_cid.mutex from sched_mm_cid_fork(A)
on.

  CPU 0                                     CPU 1

  A: execve()
     sched_mm_cid_before_execve()
       A leaves, users = 1
                                            B: vfork()
                                               sched_mm_cid_fork(C)
                                                 users = 2
                                                 C takes the last CID
  A: execve() fails
     sched_mm_cid_after_execve()
       sched_mm_cid_fork(A)
         users = 3, switch to per CPU mode
         A is active and has no CID
  A is preempted
  A is scheduled in
     mm_cid_schedin(A)
       mm_get_cid() loops:
       B and C hold both CIDs
                                            C: exit()
                                               sched_mm_cid_exit()
                                                 waits for mm_cid.mutex,
                                                 which A holds

Nothing ends the loop. Only the fixup could free a CID, and A is the
task that runs it.

A guest with two CPUs hangs on this sequence when a SCHED_FIFO task
wakes every 50us on CPU 0 to preempt A. CPU 0 stays in __schedule() and
CPU 1 in queued_spin_lock_slowpath(). With this change the same guest
completes 20000 attempts.

Fix this by keeping a running joiner inactive until the fixups are done.
A schedule in gives it no CID before that. sched_mm_cid_fork() then
marks it active, and it takes its CID under the runqueue lock.

Fixes: 653fda7ae73d ("sched/mmcid: Switch over to the new mechanism")
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: stable@vger.kernel.org
---
 kernel/sched/core.c | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index ab68e8605eed..510cd775723b 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -11054,7 +11054,16 @@ static void sched_mm_cid_fork(struct task_struct *t)
 			return;
 		}
 
-		if (!sched_mm_cid_add_user(t, mm)) {
+		flip = sched_mm_cid_add_user(t, mm);
+		/*
+		 * A running joiner takes no CID at a schedule in before the
+		 * fixups are done. The users it joins can hold every CID
+		 * until then, and the fixups are its own work.
+		 */
+		if (t == current)
+			t->mm_cid.active = 0;
+
+		if (!flip) {
 			if (!cid_on_cpu(mm->mm_cid.mode))
 				t->mm_cid.cid = mm_get_cid(mm);
 			break;
@@ -11064,7 +11073,6 @@ static void sched_mm_cid_fork(struct task_struct *t)
 		 * Handle the mode change and transfer current's CID. A
 		 * running joiner has no CID yet. It takes one below.
 		 */
-		flip = true;
 		percpu = cid_on_cpu(mm->mm_cid.mode);
 		if (!percpu)
 			mm_cid_transit_to_task(current, pcp);
@@ -11077,18 +11085,16 @@ static void sched_mm_cid_fork(struct task_struct *t)
 			mm_cid_fixup_tasks_to_cpus();
 		} else {
 			mm_cid_fixup_cpus_to_tasks(mm);
-			/*
-			 * A running joiner may have taken a CID at a
-			 * schedule in. If not, it takes one below.
-			 */
+			/* A running joiner takes its CID below. */
 			if (t != current)
 				t->mm_cid.cid = mm_get_cid(mm);
 		}
 	}
 
-	/* A running joiner with no CID takes one now, see above. */
+	/* The running joiner takes part from here on, see above. */
 	if (t == current) {
 		guard(task_rq_lock)(t);
+		t->mm_cid.active = 1;
 		if (t->mm_cid.cid == MM_CID_UNSET)
 			mm_cid_schedin(t);
 	}
-- 
2.47.3


  parent reply	other threads:[~2026-11-17 16:17 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-17 16:17 [PATCH 00/17] sched/mmcid: fixes for execve, mode switches and affinity Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 01/17] sched/mmcid: Give a CID to a task that joins its mm while running Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 02/17] sched/mmcid: Give no concurrency ID to user workers Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 03/17] sched/mmcid: Reset stale state when the first user joins an mm Mathieu Desnoyers
2026-11-17 16:17 ` Mathieu Desnoyers [this message]
2026-11-17 16:17 ` [PATCH 05/17] sched/mmcid: Take the allowed CPUs of the first user when it joins Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 06/17] sched/mmcid: Synchronize MM teardown with affinity updates Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 07/17] selftests/rseq: Test the concurrency ID after a failed execve Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 08/17] selftests/rseq: Test the concurrency ID after the last user fails execve Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 09/17] selftests/rseq: Test the concurrency IDs across ownership mode switches Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 10/17] selftests/rseq: Test the upper bound of the concurrency IDs Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 11/17] selftests/rseq: Test affinity changes made by another process Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 12/17] selftests/rseq: Test an affinity change by another process during execve Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 13/17] selftests/rseq: Test concurrent thread creation and forced migration Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 14/17] selftests/rseq: Test the concurrency IDs after the affinity is widened Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 15/17] selftests/rseq: Test two race windows of the concurrency ID mode switches Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 16/17] selftests/rseq: Test affinity changes while a task replaces or drops its mm Mathieu Desnoyers
2026-11-17 16:17 ` [PATCH 17/17] selftests/rseq: Test the last user after an affinity change during execve Mathieu Desnoyers

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=f1b450d79e9ae67134ed3451b4af33e71cc5dd74.1794932121.git.mathieu.desnoyers@efficios.com \
    --to=mathieu.desnoyers@efficios.com \
    --cc=bestswngs@gmail.com \
    --cc=boqun@kernel.org \
    --cc=brauner@kernel.org \
    --cc=bsegall@google.com \
    --cc=dietmar.eggemann@arm.com \
    --cc=jack@suse.cz \
    --cc=juri.lelli@redhat.com \
    --cc=kees@kernel.org \
    --cc=kprateek.nayak@amd.com \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mgorman@suse.de \
    --cc=mingo@redhat.com \
    --cc=oleg@redhat.com \
    --cc=paulmck@kernel.org \
    --cc=peterz@infradead.org \
    --cc=rostedt@goodmis.org \
    --cc=shuah@kernel.org \
    --cc=stable@vger.kernel.org \
    --cc=tglx@kernel.org \
    --cc=vincent.guittot@linaro.org \
    --cc=viro@zeniv.linux.org.uk \
    --cc=vschneid@redhat.com \
    --cc=xmei5@asu.edu \
    --cc=xujiakai24@mails.ucas.ac.cn \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®