From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wr1-f49.google.com (mail-wr1-f49.google.com [209.85.221.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DBE342D7DF2 for ; Fri, 6 Feb 2026 18:21:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.49 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770402089; cv=none; b=tU6f3K4JawA3vYhgtmFHphYQZ0fxhB9gEJZe8fPjRpzfwIZRgDPvHU0x+96oqo10vNsQugimXQIeTi6VjcJjPsiktSttDXWwqJJmpRDEAICYuZ99Mivg7ZQi7A8bwyPlMgClpODSmf2lv+lGaSxWd4LeG6nY+G8e1Z9tyA6lap0= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770402089; c=relaxed/simple; bh=UO5POCig5UZDlNtgaMSeJM2Q+BiV14JFcAEkZTqSucc=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Ls6F0jMTuo2BKM6zvwKLAfKEJDMUOYdWy/SCwywJAyaD7LyD8MknB7FndrTbpzD8Ka4ppWLrLbbLVaPSa3xbfLk8KJNvy556kvn3dzthdm7lYgM7ShXaw6QJL2m0+O3D6yekWJtA5s3oXIdifFjLl/O1/U5VODCJZH+SwUthorI= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io; spf=pass smtp.mailfrom=layalina.io; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b=DqPHPGyZ; arc=none smtp.client-ip=209.85.221.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=layalina.io Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b="DqPHPGyZ" Received: by mail-wr1-f49.google.com with SMTP id ffacd0b85a97d-43626796202so1622458f8f.3 for ; Fri, 06 Feb 2026 10:21:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=layalina-io.20230601.gappssmtp.com; s=20230601; t=1770402087; x=1771006887; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=ApPQBnY498dlSUV/bbVcy+lNxrNf/C+4xtxX9OOH6M0=; b=DqPHPGyZB8syyTQvlKcIIKiR5hAEzmgzE43v8W2dTKhuTryDjCrTNhUXOrMq40IFP5 KoM/elJhmFF1BkznamCdMl7Zn2P175tAVkwOMr1tcDt/kXRZAiw90m6vBkMm+gDSSwj/ E1lPuh2504oE5mX791gnF0m/ZXnST6ChP0zqLF4bCc6mMkAHEmpY1VvGu+KmeAF7vMCR sVwDpOcGAv/5n044yaFXlUk2WYxeSAEVZx6Af2LnBDBo5c1xsXL1sm5TMyHe8nKKrNA9 SBO0vpLSduDwv+ET6AxSmHVMCZmI9UxfRydEMTOmIMlANlbeqiY9e8ykul88qnKdqmyh z4rA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1770402087; x=1771006887; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ApPQBnY498dlSUV/bbVcy+lNxrNf/C+4xtxX9OOH6M0=; b=QFI7JN4UKrFeH2/197H3TIX0zKg3zsOleIcBxU3XK29GE4Z2YvtqabsFHfoNmsXCqB Dhbb2MDYkduM8KsBEE059CVtMyod0lZVyiMAXi81VGaFnB/EaO0nnc6Vz2wq5HeUfyw7 CodXDLAtme0ijxXWih47cUw4FCypBeTUwVCvv6JtoNIlp/4aKFwNIQA4yDtNVi/I/Das d26ngjwvqwhj9OYsYFKjuEEE2cB3P4Gb5VkwBtfTffbugkleeAIQs05ICfdObSjrrfod oJ82tOKRlexfvlFCjEZNPPh+0P9ZaqbvCqvQlLsYDdE73i6qzcmv5mvaGua1abL6MgrS O1Jg== X-Forwarded-Encrypted: i=1; AJvYcCU0O9TOFGTPl2AIxg7n2OJg1PJg9C6wxGuuNhMU3kE1aaEbiUb+86P7bkptW5+/YvR4J56TX+jlLBNvs1M=@vger.kernel.org X-Gm-Message-State: AOJu0YzN0vXPHkMzvn0AYs/8QV9c9YowU4lvbcNpEpNkZhwx8tdTwkbP z717nEaYr9mxd4HYR808f7BJ2+3k/ZkfxZz2fRyPdJp+le918mNrsKlowbUlwaAj1YCC0riaZrg aPuNi X-Gm-Gg: AZuq6aJtOG2W/iGXmHHvhAtBit0BeXj62K1+y44FZrCy8CPMZNp3o/XMjbiYeDwoeEY V7S8afim0LJMTife9TbgodkvuvmdL7GE8LSkyGd6s/jArEd4+UFAe7i4noe7cipDChBKYU7POZT SrVhJ5e0l72DoDpK7agOLwN+AeNsNaFZMurDpegIskvpNFMN8oEL3CmCwsGKC00N5IA6Zk+WK95 y2jio6KKSOuCNusrIbwG6qAKoTpbRyjeoN36brKMdRp12xwC2VCAQFi6mL2SQ2kmJ945xc+J86U Q1Sipw8JzqKjXrWdV1iRCJjPM8F3Jdjx9QIckemAWSt6jzra+ku8FGL2bTgRqStjkq1qQOBLmgy tBcyJUBqMtcqXIkYhfCK2BAU6GnyyTdw5azimSX0A0mySGpkrC9Kpzx3+2hz6An1fpL4DEdhGh/ vofPDzw657BYYH41A6aVai8BBx5WNudrVWUlApwFJcxI8ItkG/ud0j6NViDzNDPq8= X-Received: by 2002:a05:6000:250f:b0:432:dc23:368 with SMTP id ffacd0b85a97d-4362937bd17mr5374137f8f.49.1770402086964; Fri, 06 Feb 2026 10:21:26 -0800 (PST) Received: from airbuntu (host109-153-232-30.range109-153.btcentralplus.com. [109.153.232.30]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-436296b2ed9sm7498242f8f.5.2026.02.06.10.21.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Feb 2026 10:21:26 -0800 (PST) Date: Fri, 6 Feb 2026 18:21:24 +0000 From: Qais Yousef To: Vincent Guittot Cc: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, linux-kernel@vger.kernel.org, pierre.gondois@arm.com, kprateek.nayak@amd.com, hongyan.xia2@arm.com, christian.loehle@arm.com, luis.machado@arm.com Subject: Re: [PATCH 4/6 v8] sched/fair: Add push task mechanism for fair Message-ID: <20260206182124.kz7abi7nvdnvl64q@airbuntu> References: <20251202181242.1536213-1-vincent.guittot@linaro.org> <20251202181242.1536213-5-vincent.guittot@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20251202181242.1536213-5-vincent.guittot@linaro.org> On 12/02/25 19:12, Vincent Guittot wrote: > EAS is based on wakeup events to efficiently place tasks on the system, but > there are cases where a task doesn't have wakeup events anymore or at a far > too low pace. For such situation, we can take advantage of the task being > put back in the enqueued list to check if it should be pushed on another > CPU. > When the task is alone on the CPU, it's never put back in the enqueued > list; In this special case, we use the tick to run the check. > > Add a push task mechanism that enables fair scheduler to push runnable > tasks. EAS will be one user but other feature like filling idle CPUs > can also take advantage of it. I think worth adding that we are improving responsiveness of lb, this is a critical side effect. Currently pull mechanism is too slow - and takes wrong decisions for systems that relies on feec() as you pointed out. It also prepares for a unified decision between wake up and lb for more coherent task placement decisions. > > Signed-off-by: Vincent Guittot > --- > kernel/sched/fair.c | 212 ++++++++++++++++++++++++++++++++++++++++++- > kernel/sched/sched.h | 4 + > 2 files changed, 214 insertions(+), 2 deletions(-) > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > index 80c4131fb35b..252254168c92 100644 > --- a/kernel/sched/fair.c > +++ b/kernel/sched/fair.c > @@ -6989,6 +6989,8 @@ enqueue_task_fair(struct rq *rq, struct task_struct *p, int flags) > hrtick_update(rq); > } > > +static void fair_remove_pushable_task(struct rq *rq, struct task_struct *p); > + > /* > * Basically dequeue_task_fair(), except it can deal with dequeue_entity() > * failing half-way through and resume the dequeue later. > @@ -7017,6 +7019,8 @@ static int dequeue_entities(struct rq *rq, struct sched_entity *se, int flags) > h_nr_idle = task_has_idle_policy(p); > if (task_sleep || task_delayed || !se->sched_delayed) > h_nr_runnable = 1; > + > + fair_remove_pushable_task(rq, p); > } > > for_each_sched_entity(se) { > @@ -8504,6 +8508,187 @@ static int find_energy_efficient_cpu(struct task_struct *p, int prev_cpu) > return target; > } > > +DEFINE_STATIC_KEY_FALSE(sched_push_task); > + > +static inline bool sched_push_task_enabled(void) > +{ > + return static_branch_unlikely(&sched_push_task); > +} > + > +static bool fair_push_task(struct rq *rq, struct task_struct *p) I expected this to be named is_pushable_task()? > +{ > + return false; > +} > + > +static inline int has_pushable_tasks(struct rq *rq) > +{ > + return !plist_head_empty(&rq->cfs.pushable_tasks); > +} > + > +static struct task_struct *pick_next_pushable_fair_task(struct rq *rq) > +{ > + struct task_struct *p; > + > + if (!has_pushable_tasks(rq)) > + return NULL; > + > + p = plist_first_entry(&rq->cfs.pushable_tasks, > + struct task_struct, pushable_tasks); > + > + WARN_ON_ONCE(rq->cpu != task_cpu(p)); > + WARN_ON_ONCE(task_current(rq, p)); > + WARN_ON_ONCE(p->nr_cpus_allowed <= 1); > + WARN_ON_ONCE(!task_on_rq_queued(p)); > + > + /* > + * Remove task from the pushable list as we try only once after that > + * the task has been put back in enqueued list. > + */ > + plist_del(&p->pushable_tasks, &rq->cfs.pushable_tasks); > + > + return p; > +} > + > +static int > +select_task_rq_fair(struct task_struct *p, int prev_cpu, int wake_flags); > + > +/* > + * See if the non running fair tasks on this rq can be sent on other CPUs > + * that fits better with their profile. > + */ > +static bool push_fair_task(struct rq *rq) > +{ > + struct task_struct *next_task; > + int prev_cpu, new_cpu; > + struct rq *new_rq; > + > + next_task = pick_next_pushable_fair_task(rq); > + if (!next_task) > + return false; > + > + if (is_migration_disabled(next_task)) > + return true; When we loop to push tasks, the task might become unpushable say after pushing another task. Should we add a late check to verify the task is still pushable? if (!fair_push_task(next_task, rq)) return true; > + > + /* We might release rq lock */ > + get_task_struct(next_task); > + > + prev_cpu = rq->cpu; > + > + new_cpu = select_task_rq_fair(next_task, prev_cpu, 0); > + > + if (new_cpu == prev_cpu) > + goto out; > + > + new_rq = cpu_rq(new_cpu); > + > + if (double_lock_balance(rq, new_rq)) { > + /* The task has already migrated in between */ > + if (task_cpu(next_task) != rq->cpu) { > + double_unlock_balance(rq, new_rq); > + goto out; > + } > + > + deactivate_task(rq, next_task, 0); > + set_task_cpu(next_task, new_cpu); > + activate_task(new_rq, next_task, 0); > + > + resched_curr(new_rq); > + > + double_unlock_balance(rq, new_rq); > + } > + > +out: > + put_task_struct(next_task); > + > + return true; > +} > + > +static void push_fair_tasks(struct rq *rq) > +{ > + /* push_fair_task() will return true if it moved a fair task */ > + while (push_fair_task(rq)) > + ; > +} > + > +static DEFINE_PER_CPU(struct balance_callback, fair_push_head); > + > +static inline void fair_queue_pushable_tasks(struct rq *rq) > +{ > + if (!sched_push_task_enabled() || !has_pushable_tasks(rq)) > + return; > + > + queue_balance_callback(rq, &per_cpu(fair_push_head, rq->cpu), push_fair_tasks); > +} > + > +static void fair_remove_pushable_task(struct rq *rq, struct task_struct *p) > +{ > + if (sched_push_task_enabled()) > + plist_del(&p->pushable_tasks, &rq->cfs.pushable_tasks); > +} > + > +static void fair_add_pushable_task(struct rq *rq, struct task_struct *p) > +{ > + if (sched_push_task_enabled() && fair_push_task(rq, p)) { > + plist_del(&p->pushable_tasks, &rq->cfs.pushable_tasks); > + plist_node_init(&p->pushable_tasks, p->prio); > + plist_add(&p->pushable_tasks, &rq->cfs.pushable_tasks); > + } > +} > + > +static int active_load_balance_cpu_stop(void *data); > + > +/* > + * See if the alone task running on the CPU should migrate on a better than > + * the local one. > + */ > +static inline bool check_pushable_task(struct task_struct *p, struct rq *rq) > +{ > + int new_cpu, cpu = cpu_of(rq); > + > + if (!sched_push_task_enabled()) > + return false; > + > + if (WARN_ON(!p)) > + return false; > + > + if (WARN_ON(!task_current(rq, p))) > + return false; > + > + if (is_migration_disabled(p)) > + return false; > + > + /* If there are several task, wait for being put back */ > + if (rq->nr_running > 1) > + return false; > + > + if (!fair_push_task(rq, p)) > + return false; > + > + new_cpu = select_task_rq_fair(p, cpu, 0); > + > + if (new_cpu == cpu) > + return false; > + > + /* > + * ->active_balance synchronizes accesses to > + * ->active_balance_work. Once set, it's cleared > + * only after active load balance is finished. > + */ > + if (!rq->active_balance) { > + rq->active_balance = 1; > + rq->push_cpu = new_cpu; > + } else > + return false; > + > + raw_spin_rq_unlock(rq); > + stop_one_cpu_nowait(cpu, > + active_load_balance_cpu_stop, rq, > + &rq->active_balance_work); > + raw_spin_rq_lock(rq); > + > + return true; > +} > + > /* > * select_task_rq_fair: Select target runqueue for the waking task in domains > * that have the relevant SD flag set. In practice, this is SD_BALANCE_WAKE, > @@ -8973,6 +9158,12 @@ pick_next_task_fair(struct rq *rq, struct task_struct *prev, struct rq_flags *rf > put_prev_entity(cfs_rq, pse); > set_next_entity(cfs_rq, se); > > + /* > + * The previous task might be eligible for being pushed on > + * another cpu if it is still active. > + */ > + fair_add_pushable_task(rq, prev); > + > __set_next_task_fair(rq, p, true); > } > > @@ -9036,6 +9227,13 @@ static void put_prev_task_fair(struct rq *rq, struct task_struct *prev, struct t > cfs_rq = cfs_rq_of(se); > put_prev_entity(cfs_rq, se); > } > + > + /* > + * The previous task might be eligible for being pushed on another cpu > + * if it is still active. > + */ > + fair_add_pushable_task(rq, prev); > + > } > > /* > @@ -13390,8 +13588,10 @@ static void task_tick_fair(struct rq *rq, struct task_struct *curr, int queued) > if (static_branch_unlikely(&sched_numa_balancing)) > task_tick_numa(rq, curr); > > - update_misfit_status(curr, rq); > - check_update_overutilized_status(task_rq(curr)); > + if (!check_pushable_task(curr, rq)) { > + update_misfit_status(curr, rq); > + check_update_overutilized_status(task_rq(curr)); > + } > > task_tick_core(rq, curr); > } > @@ -13552,6 +13752,8 @@ static void __set_next_task_fair(struct rq *rq, struct task_struct *p, bool firs > { > struct sched_entity *se = &p->se; > > + fair_remove_pushable_task(rq, p); > + > if (task_on_rq_queued(p)) { > /* > * Move the next running task to the front of the list, so our > @@ -13567,6 +13769,11 @@ static void __set_next_task_fair(struct rq *rq, struct task_struct *p, bool firs > if (hrtick_enabled_fair(rq)) > hrtick_start_fair(rq, p); > > + /* > + * Try to push prev task before checking misfit for next task as > + * the migration of prev can make next fitting the CPU > + */ > + fair_queue_pushable_tasks(rq); > update_misfit_status(p, rq); > sched_fair_update_stop_tick(rq, p); > } > @@ -13596,6 +13803,7 @@ void init_cfs_rq(struct cfs_rq *cfs_rq) > { > cfs_rq->tasks_timeline = RB_ROOT_CACHED; > cfs_rq->zero_vruntime = (u64)(-(1LL << 20)); > + plist_head_init(&cfs_rq->pushable_tasks); > raw_spin_lock_init(&cfs_rq->removed.lock); > } > > diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h > index b419a4d98461..697bd654298a 100644 > --- a/kernel/sched/sched.h > +++ b/kernel/sched/sched.h > @@ -711,6 +711,8 @@ struct cfs_rq { > unsigned long runnable_avg; > } removed; > > + struct plist_head pushable_tasks; > + > #ifdef CONFIG_FAIR_GROUP_SCHED > u64 last_update_tg_load_avg; > unsigned long tg_load_avg_contrib; > @@ -3620,6 +3622,8 @@ static inline bool sched_energy_enabled(void) { return false; } > > #endif /* !(CONFIG_ENERGY_MODEL && CONFIG_CPU_FREQ_GOV_SCHEDUTIL) */ > > +DECLARE_STATIC_KEY_FALSE(sched_push_task); > + > #ifdef CONFIG_MEMBARRIER > > /* > -- > 2.43.0 >