Re: [PATCH] sched/mmcid: Bound the CID allocation busy wait

[Thomas Gleixner <tglx@linutronix.de>]

On Thu, Oct 15 2026 at 22:37, Thomas Gleixner wrote:
> On Fri, Sep 18 2026 at 12:43, Peter Zijlstra wrote:
> The stall makes no sense because mm_get_cid() is invoked from:
>
>   10967                 /* First user ? */
>   10968                 if (!mm->mm_cid.users) {
>   10969                         sched_mm_cid_add_user(t, mm);
>   10970                         t->mm_cid.cid = mm_get_cid(mm);
>
> which means this is the first CID user on the MM. So how can the bitmap
> be exhausted?
>
> I have a suspicion but it does not add up (yet).
>
> This is a two CPU VM. Assume that the process which invokes execve() has
> more than two threads. That means the MMCID of the process is in per CPU
> mode. sched_mm_cid_exit() ignores a mode change when the last user goes
> away and with two CPUs the threshold to switch back is when the user
> count goes below _ONE_. So there is probably stale state there, but the
> last outgoing task should have reclaimed the per CPU CID and dropped it,
> but the other CPU still has it assigned to it. So that first allocation
> should succeed...
>
> I'll try to write a trivial test program tomorrow which creates enough
> threads and does an execve().

Hmm. There is stale state, but there is always _ONE_ CID available
because the last exiting thread drops the per CPU CID.

As the mode is TRANSIT the whole scenario resolves when the new exec
spawns threads or schedules around. That just works by chance, but I
couldn't find a way to exhaust the bitmap for the initial allocation.

So I really need to see the reproducer for this splat.

The patch below cleans up the stale data and fixes a consistency issue
vs. the allowed CPUs accounting.

Thanks,

        tglx
---
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -10880,6 +10880,10 @@ static inline void mm_update_cpus_allowe
 	 */
 	mc = &mm->mm_cid;
 	guard(raw_spinlock)(&mc->lock);
+	/* Might have raced with exit() */
+	if (!mm->mm_cid.users)
+		return;
+
 	mm_allowed = mm_cpus_allowed(mm);
 	weight = cpumask_weighted_or(mm_allowed, mm_allowed, affmsk);
 	if (weight == mc->nr_cpus_allowed)
@@ -11019,6 +11023,33 @@ static bool sched_mm_cid_add_user(struct
 	return mm_update_max_cids(mm);
 }
 
+/*
+ * sched_mm_cid_exit() does not handle the mode switch for the last user
+ * because for normal exit() that's a pointless exercise. But if the
+ * process which issued execve() was multithreaded and the last exiting
+ * thread switched back from per CPU mode then this leaves stale state
+ * around. The indicator for that is mm_cid::mode having MM_CID_TRANSIT
+ * set.
+ */
+static void sched_mm_cid_cleanup_execve(struct task_struct *t, struct mm_struct *mm)
+{
+	if (likely(!mm->mm_cid.mode))
+		return;
+
+	WARN_ON_ONCE(mm->mm_cid.mode & MM_CID_ONCPU);
+	mm->mm_cid.mode = 0;
+
+	unsigned int cpu;
+
+	for_each_possible_cpu(cpu)
+		per_cpu(mm->mm_cid.pcpu->cid, cpu) = 0;
+
+	bitmap_zero(mm_cidmask(mm), num_possible_cpus());
+
+	mm->mm_cid.nr_cpus_allowed = t->nr_cpus_allowed;
+	cpumask_copy(mm_cpus_allowed(mm), &t->cpus_mask);
+}
+
 static void sched_mm_cid_fork(struct task_struct *t)
 {
 	struct mm_struct *mm = t->mm;
@@ -11035,6 +11066,7 @@ static void sched_mm_cid_fork(struct tas
 
 		/* First user ? */
 		if (!mm->mm_cid.users) {
+			sched_mm_cid_cleanup_execve(t, mm);
 			sched_mm_cid_add_user(t, mm);
 			t->mm_cid.cid = mm_get_cid(mm);
 			/* Required for execve() */