From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ed1-f48.google.com (mail-ed1-f48.google.com [209.85.208.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6F2FF480DE8 for ; Wed, 14 Oct 2026 07:19:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=209.85.208.48 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791962377; cv=pass; b=G4ITWH4vCge6Uot0RgvkWc/TsIAkz57kQ5OsDDpNtzW9hGYkDnXu2xsyotF6ll8Gw8E7855yPg5kqn1/l09Jb7RO73Fr9y1CPANh3m5UdTnVfhVUhm2JN24KwOdeUy/bw/bxV3UETSHJxyYDzIhNXAJ3W2u59NJ6Cz1iHyePLS0= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791962377; c=relaxed/simple; bh=+nKE0aerXq5zcerPMktiueb2dpi7N/wqcFUGs1sFppk=; h=MIME-Version:References:In-Reply-To:From:Date:Message-ID:Subject: To:Cc:Content-Type; b=ujTXoYl4e2NblSChM616R4+s6OX3OVtT+hMxqNwfpjBMJ9eLSVleDYFQW1XPKTFnFSNLdEa2t1+tn9mjw3ntabmCaJB6QFkMhkq8dJ2ACRbpCuYRWAa796FAnegKTDoOXGOn7OmawYxHjE/pikZdXqbviOADQK1NkgrwYwPMv9Y= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=m7Sciar/; arc=pass smtp.client-ip=209.85.208.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="m7Sciar/" Received: by mail-ed1-f48.google.com with SMTP id 4fb4d7f45d1cf-6b1ab2ad719so4599450a12.3 for ; Wed, 14 Oct 2026 00:19:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1791962374; cv=none; d=google.com; s=arc-20260327; b=hLTjrTfNjS9qPOMpizkU5EUulqoFVW++V/IjttcTSsz4vktdyA0F2w9+HUDkq3odfI HcwWB9FzzSabNPlGS7VZlJJlPX1kjvKVFTQr7lGV6WiZakUKqx7Ipny9Ulchz+ve1kpD eIkWY7VnGuHfettEy1H4f/4V+VxNE4bBy8fW9zFe2clBkgNgpvnB1NU2rj7N5+Lr9A3r 7B3lVqJOibsRaDkGcwAS588ETLOPOjXTiz4m4hFMUiFo5jzVvn3ZYy+VgqiU6TcpFZ8P VWpeRvJ/CldQgIsaOlfbcwCOG8iw3JBXAcFd3q8xri684nvZbm4JbEu3CeevX/oqUoYz wBeQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:dkim-signature; bh=qbDfcVSYXH9BTPxOSJl5CrroWP8L6dvonpMNak5vQvA=; fh=PsyFn35V2TTyoc3itB7f+2epOn5diKkkpsV/IJDtiFY=; b=j8AvYCe2PHcAMMVVGHRBWIcquGYXHjQx5S3CG7THeS+3osgcIfq8zn8TXF5EJspHTi OXPMoFLwalgxbHGOLOKg4SWsD/pvEf81c/w1mv1dSsBBQR8CL5WqBPKPq6PP15KpJBa6 +vcaHWa+jDFeNDZd+0f/Y2XbY96NFqKiHWbui0awa72O4VtrkE38XZMVon8dJtnACXc5 7UE0uDDsrMiKz5C6URtuHC7I+bW4R8s2m/rjxoS2NsDWYDSvtzTRBN7Ly1CeUYobv/cR eU+HiUrMWmuQSB2CF8U0Y36gJnsUbe/3HgxbLTtLt7cbfalG327RmPCuz+V9fzIFgcyA hBmg==; darn=vger.kernel.org ARC-Authentication-Results: i=1; mx.google.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1791962374; x=1792567174; darn=vger.kernel.org; h=content-type:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version:from:to:cc:subject:date:message-id:reply-to :content-type; bh=qbDfcVSYXH9BTPxOSJl5CrroWP8L6dvonpMNak5vQvA=; b=m7Sciar/cWyTeS/n1S+loqRYRbKqCfFETScXUv3KhjhShHT1NtFVmkvV0x8oso9ysT M+npVKiKLO5KkRXITmdQ/MMZxAAzc5cEuPlqeKMcCwOzJ+NNlAViMCjYPZZFrOJjR5KO XS1dSK4yqOrjXy0LDoL1el0Xg9d56Z8OETmi9nbTRDWZ0vaaC/tSp+klfqmV27guQglx SRHyHlrCrsZBwtt0BIEXoFDt3hvbLW4UT8XynB0gI6CyTHprqSwsWXLnDj3ElnJN4cdb vrUSigBDcs9pKePEF7hgXV0rvKh2MeJmV+2QJrcz1mhjBN8qjb6n8eRyWvA8Yppw3gY2 3mJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1791962374; x=1792567174; h=content-type:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to:content-type; bh=qbDfcVSYXH9BTPxOSJl5CrroWP8L6dvonpMNak5vQvA=; b=j9Ijvmptmi+LJceijeObvS/2PasjQNhdIswWWqwKIgDTU0EuNPF2f/jzU4c4438G7l JQRIbB1iC3TB34kbr7o9VlMHidkhpgN9krycZo5FQYm+9VAAzmaTXg2g6mV+qNh/XNob 6CV3Fxu3I5U0w//QLJdG+La8ILilBXtktdX6ftbOVPC/pIeAwcJBeG1ceWqGu2tao63G LJZ/Ri8JAaXfQhVHmu0St8X896W+fM+sVlKGDJ3PE49sno/HejokqI2WYUzVo//jWtOj p5LbJ9Uv3Mkde+8ygJodOIRjHX4k6iApp/l5iyDfVs74FYMqHvtNq0HyOrs0p8CkQhDh TLzg== X-Forwarded-Encrypted: i=1; AKwUvBxo3QsIphuD4+qsZp1ufXX0RTyk3tZu9lT7HoHEK+Hz7lSHEZOMEIHb4rxzLjNi/UJmdJJV2PDNOo12zC4=@vger.kernel.org X-Gm-Message-State: AFq9FYI9wUGi3fWZu/wXayjP5L8Cbj9jLg9XLYzBVOPYKlPgsYjGc4Gf InV11cYG+79YJQ3ZtGFf14FuRbw+T7beDQ0qGa7IXVS5cgS28M2z1R5HzTd81HBSbDs88jYHs4e K/dcuwsuZ4rOsZBMH7f5IZu/Aw9rsp7/3XgqW5eDe0g== X-Gm-Gg: AYBFou1EGvyOy6IDo8yH9xENmachkodP+8caELMoUeb+vaCT87XpPMpCTg/oef/Nk8E nXIHAgbYiWAD2rWR5Uu4OppObbnqBGe2CZZ5zHMx73ATOlVSQETyxVVUDQqhtxHZpPG95Ce1apT +VB9S+fzTC68UoJZ5SxQ/QNn8hfZLWGXwqWo//wdnZxnshlC+c0PeXhCqEdJZxoc8LTeC3iIvPl s91PHYEk3kjXpJ4Xf0xPUQfDf23i2H0vUfpdO0O/VHZWLLfQcMj4VgeplMuuhdz5eMNh7T51JVS +AnWBXAoul80w6kgg9RmX6R/kspSbfxHuM80Hoqq9gPTCXPIo3V9Hgs+b1vwzXgbuKs9d9IAN09 awtmKoAf6CJo= X-Received: by 2002:a17:907:6095:b0:c34:c5e8:1dc5 with SMTP id a640c23a62f3a-c34c5e82123mr828929666b.20.1791962373480; Wed, 14 Oct 2026 00:19:33 -0700 (PDT) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 References: <20261002154415.2270586-1-vincent.guittot@linaro.org> <20261002154415.2270586-8-vincent.guittot@linaro.org> In-Reply-To: From: Vincent Guittot Date: Wed, 14 Oct 2026 09:19:21 +0200 X-Gm-Features: AclHuK8o97-tByDZxo3ZbCUEwmJUGSdqAopCL1P-qHwY8E-185Z3bC_zZwQQAnI Message-ID: Subject: Re: [PATCH 07/18] sched/fair: Add push task mechanism for fair To: Shrikanth Hegde Cc: qyousef@layalina.io, christian.loehle@arm.com, pierre.gondois@arm.com, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, kprateek.nayak@amd.com, linux-kernel@vger.kernel.org, lukasz.luba@arm.com, rafael@kernel.org, linux-pm@vger.kernel.org, tj@kernel.org, void@manifault.com, arighi@nvidia.com, changwoo@igalia.com, sched-ext@lists.linux.dev Content-Type: text/plain; charset="UTF-8" On Mon, 12 Oct 2026 at 16:07, Shrikanth Hegde wrote: > > Hi Vincent, > > Disclaimer: > I haven't read the full series yet. I will try to do. Sorry in case i read > it wrong. > Thought of quickly checking the push path since i had recently worked on it. > > On 10/2/26 9:14 PM, Vincent Guittot wrote: > > EAS is based on wakeup events to efficiently place tasks on the system, but > > there are cases where a task doesn't have wakeup events anymore or at a far > > too low pace. For such situation, we can take advantage of the task being > > put back in the enqueued list to check if it should be pushed on another > > CPU. > > > > Add a push task mechanism that enables fair scheduler to push runnable > > tasks. EAS will be one user but other feature like filling idle CPUs or > > short slice tasks can also take advantage of it. > > > > Signed-off-by: Vincent Guittot > > --- > > kernel/sched/core.c | 4 +- > > kernel/sched/fair.c | 163 +++++++++++++++++++++++++++++++++++++++++++ > > kernel/sched/sched.h | 7 ++ > > 3 files changed, 172 insertions(+), 2 deletions(-) > > > > diff --git a/kernel/sched/core.c b/kernel/sched/core.c > > index 837dc74c9a8d..18692b752814 100644 > > --- a/kernel/sched/core.c > > +++ b/kernel/sched/core.c > > @@ -2553,8 +2553,8 @@ static inline bool is_cpu_allowed(struct task_struct *p, int cpu) > > * > > * Returns (locked) new rq. Old rq's lock is released. > > */ > > -static struct rq *move_queued_task(struct rq *rq, struct rq_flags *rf, > > - struct task_struct *p, int new_cpu) > > +struct rq *move_queued_task(struct rq *rq, struct rq_flags *rf, > > + struct task_struct *p, int new_cpu) > > __must_hold(__rq_lockp(rq)) > > { > > lockdep_assert_rq_held(rq); > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > > index f12678850ce2..00078ac7fada 100644 > > --- a/kernel/sched/fair.c > > +++ b/kernel/sched/fair.c > > @@ -8179,6 +8179,8 @@ static void dequeue_hierarchy(struct task_struct *p, int flags) > > } > > } > > > > +static void fair_remove_pushable_task(struct rq *rq, struct task_struct *p); > > + > > /* > > * The part of dequeue_task_fair() that is needed to dequeue delayed tasks. > > * > > @@ -8195,6 +8197,7 @@ static bool __dequeue_task(struct rq *rq, struct task_struct *p, int flags) > > bool task_delayed = flags & DEQUEUE_DELAYED; > > > > clear_buddies(cfs_rq, se); > > + fair_remove_pushable_task(rq, p); > > > > update_curr_eevdf(cfs_rq); > > update_entity_lag(cfs_rq, se); > > @@ -9799,6 +9802,157 @@ static int find_energy_efficient_cpu(struct task_struct *p, int prev_cpu) > > return target; > > } > > > > +DEFINE_STATIC_KEY_FALSE(sched_push_task); > > + > > +static inline bool sched_push_task_enabled(void) > > +{ > > + return static_branch_unlikely(&sched_push_task); > > +} > > + > > +static bool __check_pushable_fair_task(struct rq *rq, struct task_struct *p) > > +{ > > + if (!task_on_rq_queued(p)) > > + return false; > > + > > + if (p->se.sched_delayed) > > + return false; > > + > > + if (p->nr_cpus_allowed <= 1) > > + return false; > > + > > + return true; > > +} > > + > > +static bool fair_check_pushable_task(struct rq *rq, struct task_struct *p, struct task_struct *next) > > +{ > > + if (!__check_pushable_fair_task(rq, p)) > > + return false; > > + > > + return false; > > +} > > + > > +static inline int has_pushable_tasks(struct rq *rq) > > +{ > > + return !plist_head_empty(&rq->cfs.pushable_tasks); > > +} > > + > > Should this use static key? Not needed, teh statis key has been test prior calling it > > > +static struct task_struct *pick_next_pushable_fair_task(struct rq *rq) > > +{ > > + struct task_struct *p; > > + > > + if (!has_pushable_tasks(rq)) > > + return NULL; > > + > > + p = plist_first_entry(&rq->cfs.pushable_tasks, > > + struct task_struct, pushable_tasks); > > + > > + WARN_ON_ONCE(rq->cpu != task_cpu(p)); > > + WARN_ON_ONCE(task_current(rq, p)); > > + WARN_ON_ONCE(p->nr_cpus_allowed <= 1); > > + WARN_ON_ONCE(!task_on_rq_queued(p)); > > + > > + /* > > + * Remove task from the pushable list as we try only once after that > > + * the task has been put back in enqueued list. > > + */ > > + plist_del(&p->pushable_tasks, &rq->cfs.pushable_tasks); > > + > > + return p; > > +} > > + > > +static int > > +select_task_rq_fair(struct task_struct *p, int prev_cpu, int wake_flags); > > + > > +/* > > + * See if the non running fair tasks on this rq can be sent on other CPUs > > + * that fits better with their profile. > > + */ > > +static bool fair_push_task(struct rq *rq) > > +{ > > + struct task_struct *next_task; > > + int prev_cpu, new_cpu; > > + struct rq_flags rf; > > + struct rq *cur_rq; > > You likely need below to keep clang's context tracking to be happy. > > context_unsafe_alias(rq); > > > + > > + next_task = pick_next_pushable_fair_task(rq); > > + if (!next_task) > > + return false; > > + > > + if (is_migration_disabled(next_task)) > > + return true; > > If migration is disabled, then it shouldn't endup on pushable_tasks? This can happen after it has been added to the list > > > + > > + /* We might release rq lock */ > > + get_task_struct(next_task); > > + > > + prev_cpu = rq->cpu; > > + > > + /* > > + * We need to release rq lock and take both task and rq w/o > > + * triggering a deadlock. > > + */ > > + raw_spin_rq_unlock(rq); > > + > > + cur_rq = task_rq_lock(next_task, &rf); > > + > > + /* Task already migrated */ > > + if (cur_rq->cpu != prev_cpu) > > + goto skip_push; > > + > > + new_cpu = select_task_rq_fair(next_task, prev_cpu, 0); > > + > > + /* Task doesn't need to migrate */ > > + if (new_cpu == prev_cpu) > > + goto skip_push; > > + > > + update_rq_clock(cur_rq); > > + cur_rq = move_queued_task(cur_rq, &rf, next_task, new_cpu); > > + > > +skip_push: > > + task_rq_unlock(cur_rq, next_task, &rf); > > + > > + /* Restore rq state */ > > + raw_spin_rq_lock(rq); > > + put_task_struct(next_task); > > + > > + return true; > > +} > > + > > +static void fair_push_tasks(struct rq *rq) > > +{ > > + /* fair_push_task() will return true if it moved a fair task */ > > + while (fair_push_task(rq)) > > + ; > > +} > > Would it better to iterate through the list? This is bit difficult > to read at first. I followed what is done in RT and deadline > > In failure cases also, it can return true. No? for example, > is_migration_disabled(), It is needed since you need to iterate the > rq to see if there is a task to move. Yes, we skip the current task and continue witht the next one > > > + > > +static DEFINE_PER_CPU(struct balance_callback, fair_push_head); > > + > > +static inline void fair_queue_push_tasks(struct rq *rq) > > +{ > > + if (!sched_push_task_enabled() || !has_pushable_tasks(rq)) > > + return; > > + > > + queue_balance_callback(rq, &per_cpu(fair_push_head, rq->cpu), fair_push_tasks); > > +} > > + > > +static void fair_remove_pushable_task(struct rq *rq, struct task_struct *p) > > +{ > > + if (sched_push_task_enabled()) > > + plist_del(&p->pushable_tasks, &rq->cfs.pushable_tasks); > > +} > > + > > +static void __fair_add_pushable_task(struct rq *rq, struct task_struct *p) > > +{ > > + plist_del(&p->pushable_tasks, &rq->cfs.pushable_tasks); > > + plist_node_init(&p->pushable_tasks, p->prio); > > + plist_add(&p->pushable_tasks, &rq->cfs.pushable_tasks); > > +} > > + > > +static void fair_add_pushable_prev(struct rq *rq, struct task_struct *prev, struct task_struct *next) > > +{ > > + if (sched_push_task_enabled() && fair_check_pushable_task(rq, prev, next)) > > + __fair_add_pushable_task(rq, prev); > > +} > > + > > /* > > * select_task_rq_fair: Select a target runqueue for the task. > > * There are 2 ways to select the target runqueue: > > @@ -10272,6 +10426,12 @@ static void put_prev_task_fair(struct rq *rq, struct task_struct *prev, struct t > > cfs_rq->curr = NULL; > > if (se->on_rq) > > __enqueue_entity(cfs_rq, se); > > + > > + /* > > + * The previous task might be eligible for being pushed on another cpu > > + * if it is still active. > > + */ > > + fair_add_pushable_prev(rq, prev, next); > > } > > > > /* > > @@ -15379,6 +15539,7 @@ static void set_next_task_fair(struct rq *rq, struct task_struct *p, enum snt_e > > goto repick; > > > > clear_buddies(cfs_rq, se); > > + fair_remove_pushable_task(rq, p); > > > > if (on_rq) > > __dequeue_entity(cfs_rq, se); > > @@ -15423,6 +15584,7 @@ static void set_next_task_fair(struct rq *rq, struct task_struct *p, enum snt_e > > > > update_misfit_status(p, rq); > > sched_fair_update_stop_tick(rq, p); > > + fair_queue_push_tasks(rq); > > > > repick: > > /* > > @@ -15439,6 +15601,7 @@ void init_cfs_rq(struct cfs_rq *cfs_rq) > > { > > cfs_rq->tasks_timeline = RB_ROOT_CACHED; > > cfs_rq->zero_vruntime = (u64)(-(1LL << 20)); > > + plist_head_init(&cfs_rq->pushable_tasks); > > raw_spin_lock_init(&cfs_rq->removed.lock); > > } > > > > diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h > > index e025d2a6c302..74130bd2a2c8 100644 > > --- a/kernel/sched/sched.h > > +++ b/kernel/sched/sched.h > > @@ -721,6 +721,8 @@ struct cfs_rq { > > unsigned long runnable_avg; > > } removed; > > > > + struct plist_head pushable_tasks; > > + > > #ifdef CONFIG_FAIR_GROUP_SCHED > > u64 last_update_tg_load_avg; > > unsigned long tg_load_avg_contrib; > > @@ -3877,6 +3879,8 @@ static inline bool sched_energy_enabled(void) { return false; } > > > > #endif /* !(CONFIG_ENERGY_MODEL && CONFIG_CPU_FREQ_GOV_SCHEDUTIL) */ > > > > +DECLARE_STATIC_KEY_FALSE(sched_push_task); > > + > > #ifdef CONFIG_MEMBARRIER > > > > /* > > @@ -4209,6 +4213,9 @@ void move_queued_task_locked(struct rq *src_rq, struct rq *dst_rq, struct task_s > > wakeup_preempt(dst_rq, task, 0); > > } > > > > +extern struct rq *move_queued_task(struct rq *rq, struct rq_flags *rf, > > + struct task_struct *p, int new_cpu); > > + > > static inline > > bool task_is_pushable(struct rq *rq, struct task_struct *p, int cpu) > > { >