Re: [RFC PATCH 3/3] sched/core: Swap to lock owner for core-wide pick on core-cookie mismatch

[K Prateek Nayak <kprateek.nayak@amd.com>]

Hello folks,

On 7/17/2026 4:46 PM, K Prateek Nayak wrote:
> @@ -6728,6 +6735,24 @@ pick_next_task(struct rq *rq, struct rq_flags *rf)
>  	 */
>  	rq->core->core_task_seq++;
>  
> +	/* Last core_pick resolved to a blocked_donor! */
> +	if (sched_core_proxy_pick(rq)) {
> +		WARN_ON_ONCE(!donor->is_blocked);
> +
> +		donor->blocked_donor = NULL;
> +		owner = find_proxy_task(rq, donor, rf);
> +		if (owner && owner != rq->idle)
> +			goto restart_multi;
> +		/*
> +		 * Something changed in the proxy chain!
> +		 * Retry pick like normal. Increment core_task_seq
> +		 * since the core-wide lock might have been dropped
> +		 * during proxy-migration in find_proxy_task().
> +		 */
> +		rq->core_pick_blocked_donor = false;
> +		rq->core->core_task_seq++;
> +	}
> +
>  	/*
>  	 * Optimize for common case where this CPU has no cookies
>  	 * and there are no cookied tasks running on siblings.
> @@ -6786,6 +6811,21 @@ pick_next_task(struct rq *rq, struct rq_flags *rf)
>  	}
>  
>  	rq_max->core_pick_leader = true;
> +
> +	if (sched_core_proxy_pick(rq)) {
> +		WARN_ON_ONCE(!owner);
> +
> +		/*
> +		 * Core-wide pick resolved to the same state as last time.
> +		 * Swap the donor with the lock owner and continue the
> +		 * rest of the pick sequence.
> +		 */
> +		if (rq->core_pick_leader && rq->core_pick == donor) {
> +			rq_max->core_pick = owner;
> +			max = owner;
> +		}
> +	}

And I missed:

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index ab3b3104cd868..f12064c8222e6 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -6823,6 +6823,13 @@ pick_next_task(struct rq *rq, struct rq_flags *rf)
 		if (rq->core_pick_leader && rq->core_pick == donor) {
 			rq_max->core_pick = owner;
 			max = owner;
+		} else {
+			/*
+			 * Pick resolved to a different leader / donor task.
+			 * Clear the "core_pick_blocked_donor" indicator and
+			 * proceed like normal.
+			 */
+			rq->core_pick_blocked_donor = false;
 		}
 	}
 
---

I seemed to have managed without it because, once the proxy
chain has stabilized, we never drop the core-wide lock, and
pick always resolves to the same state for most part but a
->balance() on the re-pick can bring tasks in and that can
alter how the pick resolves.

priority-inversion-demo with the fix: https://imgur.com/ioppJWC

> +
>  	cookie = rq->core->core_cookie = max->core_cookie;
>  
>  	/*

-- 
Thanks and Regards,
Prateek