mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Tejun Heo <tj@kernel.org>
To: Andrea Righi <arighi@nvidia.com>
Cc: David Vernet <void@manifault.com>,
	Changwoo Min <changwoo@igalia.com>,
	John Stultz <jstultz@google.com>, Ingo Molnar <mingo@redhat.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Juri Lelli <juri.lelli@redhat.com>,
	Vincent Guittot <vincent.guittot@linaro.org>,
	Dietmar Eggemann <dietmar.eggemann@arm.com>,
	Steven Rostedt <rostedt@goodmis.org>,
	Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
	Valentin Schneider <vschneid@redhat.com>,
	K Prateek Nayak <kprateek.nayak@amd.com>,
	Christian Loehle <christian.loehle@arm.com>,
	David Dai <david.dai@linux.dev>, Koba Ko <kobak@nvidia.com>,
	Aiqun Yu <aiqun.yu@oss.qualcomm.com>,
	sched-ext@lists.linux.dev, linux-kernel@vger.kernel.org
Subject: Re: [PATCH 14/17] sched_ext: Delegate proxy donor admission to BPF schedulers
Date: Sun, 16 Aug 2026 17:02:03 -1000	[thread overview]
Message-ID: <aoJ5q7HbSUOxlC1H@slm.duckdns.org> (raw)
In-Reply-To: <20260816173732.17162-15-arighi@nvidia.com>

On Sun, Aug 16, 2026 at 07:35:12PM +0200, Andrea Righi wrote:
>  bool scx_allow_proxy_exec(const struct task_struct *p)
>  {
> -	return p->sched_class != &ext_sched_class;
> +	struct scx_sched *sch;
> +
> +	if (p->sched_class != &ext_sched_class)
> +		return true;
> +
> +	/*
> +	 * scx_enabled() may change while __schedule() holds only @p's rq lock.
> +	 * Once @p is associated with a scheduler, use that scheduler's policy
> +	 * even while the global enable state is transitioning.
> +	 */

I'm not sure this comment is necessary. scx_task_sched() is stable while
holding the task's rq lock.

> @@ -2059,19 +2086,25 @@ void scx_do_enqueue_task(struct rq *rq, struct task_struct *p, u64 enq_flags,
>  	if (p->scx.ddsp_dsq_id != SCX_DSQ_INVALID)
>  		goto direct;
>  
> +	enq_blocked = (sch->ops.flags & SCX_OPS_ENQ_BLOCKED) &&
> +		      p->is_blocked && !(enq_flags & SCX_ENQ_WAKEUP);

Why not just test directly in the if statement? It's not like the test
result is used anywhere else. Is the intention giving the test result an
intuitive name? I guess is_blocked && WAKEUP is the condition is the donor
gaining execution back? Might be worthwhile to add a comment.

> +	if (enq_blocked) {
> +		enq_flags |= SCX_ENQ_BLOCKED;
> +	} else {
> +		/* see %SCX_OPS_ENQ_EXITING */
> +		if (!(sch->ops.flags & SCX_OPS_ENQ_EXITING) &&
> +		    unlikely(p->flags & PF_EXITING)) {

While at it, can you swap the order? This is ordered this way because OPS
testing used to be static_key but now that these are regular tests, it makes
more sense to test the unlikely one first, or maybe that belongs in a
separate patch.

> @@ -2183,8 +2216,17 @@ static void enqueue_task_scx(struct rq *rq, struct task_struct *p, int core_enq_
>  	int sticky_cpu = p->scx.sticky_cpu;
>  	u64 enq_flags = core_enq_flags | rq->scx.remote_activate_enq_flags;
>  
> -	if (enq_flags & ENQUEUE_WAKEUP)
> +	/*
> +	 * p->is_blocked is cleared after wakeup_preempt(), so remember whether
> +	 * this is a full wakeup activation. If wakeup_preempt_scx() isn't called,
> +	 * set_next_task_scx() or a subsequent non-wakeup enqueue clears the flag.
> +	 */

I can't make heads or tails of this comment. This doesn't seem to explain
what TASK_ENQ_WAKEUP is used for but just goes into how it's managed.

> +	if (enq_flags & ENQUEUE_WAKEUP) {
>  		rq->scx.flags |= SCX_RQ_IN_WAKEUP;
> +		p->scx.flags |= SCX_TASK_ENQ_WAKEUP;
> +	} else {
> +		p->scx.flags &= ~SCX_TASK_ENQ_WAKEUP;
> +	}
>  
>  	/*
>  	 * Restoring the current scheduling context will be immediately followed
> @@ -2399,10 +2441,30 @@ static void wakeup_preempt_scx(struct rq *rq, struct task_struct *p, int wake_fl
>  	/*
>  	 * Preemption between SCX tasks is implemented by resetting the victim
>  	 * task's slice to 0 and triggering reschedule on the target CPU.
> -	 * Nothing to do.
> +	 *
> +	 * A mutex waiter can remain on-rq as a proxy donor while logically
> +	 * blocked. If it wakes without having been proxy-migrated,
> +	 * ttwu_runnable() calls here without another enqueue_task_scx(). Request
> +	 * rescheduling so that ops.dispatch() can reconsider the task after
> +	 * ttwu_runnable() clears is_blocked.
> +	 *
> +	 * A proxy-migrated donor instead returns through the full activation
> +	 * path, which calls enqueue_task_scx() before arriving here.
> +	 * SCX_TASK_ENQ_WAKEUP records that the enqueue already happened and an
> +	 * additional reschedule isn't needed.
>  	 */
> -	if (p->sched_class == &ext_sched_class)
> +	if (p->sched_class == &ext_sched_class) {
> +		bool enq_wakeup = p->scx.flags & SCX_TASK_ENQ_WAKEUP;

Ditto with bouncing test result.

> +
> +		p->scx.flags &= ~SCX_TASK_ENQ_WAKEUP;

I'm not a big fan of SCX_TASK_ENQ_WAKEUP. This is a roundabout way to detect
owner -> donor case, right? Doesn't the caller already know? If so, can't it
just pass in that as a wake_flag?

> +		if (!enq_wakeup && p->is_blocked) {
> +			struct scx_sched *sch = scx_task_sched(p);
> +
> +			if (sch && (sch->ops.flags & SCX_OPS_ENQ_BLOCKED))
> +				resched_curr(rq);
> +		}

It'd nice if we can gate the above behind proxy enabled.

> @@ -2517,6 +2579,20 @@ static bool task_can_run_on_remote_rq(struct scx_sched *sch,
>  
>  	WARN_ON_ONCE(task_cpu(p) == cpu);
>  
> +	/*
> +	 * A blocked donor may be moved normally to select a new callback rq.

What's "callback" rq?

> +	 * set_task_cpu() updates wake_cpu and makes the destination rq its new
> +	 * callback home.
> +	 *
> +	 * proxy_set_task_cpu() instead preserves wake_cpu when moving a donor to
> +	 * its lock owner's CPU. Keep such a donor on the proxy rq until it wakes;
> +	 * otherwise normal BPF placement may repeatedly pull it back to its
> +	 * callback rq only for proxy execution to move it to the owner again.
> +	 */
> +	if (sched_proxy_exec() && p->is_blocked &&
> +	    task_cpu(p) != p->wake_cpu)

No need for line break. Can you elaborate the scenario this scenario is
needed for? Is this the exact condition? Let's say a donor is running the
owner on the same CPU, so task_cpu(p) == p->wake_cpu. Wouldn't you still
want to block scx from moving it to another CPU? What am I missing?

Thanks.

-- 
tejun

  reply	other threads:[~2026-08-17  3:02 UTC|newest]

Thread overview: 35+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-08-16 17:34 [PATCHSET v12 sched_ext/for-7.3] sched: Make proxy execution compatible with sched_ext Andrea Righi
2026-08-16 17:34 ` [PATCH 01/17] sched/core: Drop mutex locks before proxy rescheduling Andrea Righi
2026-08-16 17:35 ` [PATCH 02/17] sched/core: Dequeue waking proxy donors before reset Andrea Righi
2026-08-16 20:20   ` Tejun Heo
2026-08-16 21:56     ` [PATCH v2] " Andrea Righi
2026-08-16 17:35 ` [PATCH 03/17] sched: Make NOHZ CFS bandwidth checks follow proxy donor Andrea Righi
2026-08-16 17:35 ` [PATCH 04/17] sched/core: Avoid false migration warning for proxy donors Andrea Righi
2026-08-16 17:35 ` [PATCH 05/17] sched: Pass next class to sched_change_begin() Andrea Righi
2026-08-16 17:35 ` [PATCH 06/17] sched: Add helper to block retained proxy donors Andrea Righi
2026-08-16 17:35 ` [PATCH 07/17] sched: Add sched_ext hooks for proxy execution Andrea Righi
2026-08-16 17:35 ` [PATCH 08/17] sched_ext: Block proxy donors across scheduler transitions Andrea Righi
2026-08-16 21:32   ` Tejun Heo
2026-08-16 22:06     ` Andrea Righi
2026-08-16 17:35 ` [PATCH 09/17] sched_ext: Fix ops.running/stopping() pairing for proxy-exec donors Andrea Righi
2026-08-16 22:10   ` Tejun Heo
2026-08-16 22:21     ` Andrea Righi
2026-08-16 22:29     ` [PATCH v2] " Andrea Righi
2026-08-16 17:35 ` [PATCH 10/17] sched_ext: Move reject DSQ draining into core Andrea Righi
2026-08-16 17:35 ` [PATCH 11/17] sched_ext: Generalize the reject DSQ reenqueue path Andrea Righi
2026-08-16 22:45   ` Tejun Heo
2026-08-17  6:29     ` Andrea Righi
2026-08-16 17:35 ` [PATCH 12/17] sched_ext: Handle proxy-exec races in remote DSQ transfers Andrea Righi
2026-08-16 23:54   ` Tejun Heo
2026-08-17  7:15     ` Andrea Righi
2026-08-17 17:17       ` Tejun Heo
2026-08-17 19:32         ` Andrea Righi
2026-08-17 19:49           ` Tejun Heo
2026-08-16 17:35 ` [PATCH 13/17] sched_ext: Split curr|donor references properly Andrea Righi
2026-08-16 17:35 ` [PATCH 14/17] sched_ext: Delegate proxy donor admission to BPF schedulers Andrea Righi
2026-08-17  3:02   ` Tejun Heo [this message]
2026-08-17 19:02     ` Andrea Righi
2026-08-17 20:24       ` Tejun Heo
2026-08-16 17:35 ` [PATCH 15/17] sched_ext: Add selftest for blocked donor admission Andrea Righi
2026-08-16 17:35 ` [PATCH 16/17] sched_ext: scx_qmap: Add proxy execution support Andrea Righi
2026-08-16 17:35 ` [PATCH 17/17] sched: Allow enabling proxy exec with sched_ext Andrea Righi

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=aoJ5q7HbSUOxlC1H@slm.duckdns.org \
    --to=tj@kernel.org \
    --cc=aiqun.yu@oss.qualcomm.com \
    --cc=arighi@nvidia.com \
    --cc=bsegall@google.com \
    --cc=changwoo@igalia.com \
    --cc=christian.loehle@arm.com \
    --cc=david.dai@linux.dev \
    --cc=dietmar.eggemann@arm.com \
    --cc=jstultz@google.com \
    --cc=juri.lelli@redhat.com \
    --cc=kobak@nvidia.com \
    --cc=kprateek.nayak@amd.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mgorman@suse.de \
    --cc=mingo@redhat.com \
    --cc=peterz@infradead.org \
    --cc=rostedt@goodmis.org \
    --cc=sched-ext@lists.linux.dev \
    --cc=vincent.guittot@linaro.org \
    --cc=void@manifault.com \
    --cc=vschneid@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®