From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0155F3E5EF3; Tue, 30 Jun 2026 09:03:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782810242; cv=none; b=t1+GDdFd5Rmw1k5CmgC7rH/5BfngEJynB18Z5ML9HdX/fomtpyyZ1JBCTWnfDF4uDRi6zX+82a6yhe4KLuRxvYxS2bUJ8rcORSLhJXI5QgIC1W3CGQZmAmxAclHDSq143rynMDY4K1jR6sMFE4V/WSIC7JR2upTWhZ/cpdaOCIo= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782810242; c=relaxed/simple; bh=WKXr12e+nQuGDnuybvMpPmRwr3q3srCTTMXD7XMB/sw=; h=Date:From:To:Subject:Cc:In-Reply-To:References:MIME-Version: Message-ID:Content-Type; b=PuhWy+PNka8ciIHFctj3pPf7Zfmudf7rowiSlsTCcbTpTkWS8oHvM7m+i0ajNqOWlfCD4suc0lISNfe3hvHG2X/JzmWCAVN0a7ngdURgxekNPTOQJdo+j1ls1Z+0kIcxG3EZoimBePUzzeeT3bVzwwG03ZNYuFVkwRAYKdNKKmQ= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=bJYvmucd; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=LQivQb1x; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="bJYvmucd"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="LQivQb1x" Date: Tue, 30 Jun 2026 09:03:54 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1782810235; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=W9/IisQus6xpOni87hezr5EcQ9yF6+JIxn9q6xoZ4Z8=; b=bJYvmucdbaf41WvQr82z7dKenNPzUcfYQh91QQ4JxAo8jfG20uj3GkKWlNEZKkEVggt4On pTl6ls9Vv7fgIeYpN5EZoqTLZwfCclbF9PFrZYa0r2yHM+igFPrltuJGRDh1EOk2aQaqYQ KU+ncuR3OL+jiTVaccGf7lsk+ULkx62VnW0OYNBoriw/WPfFfV/liLRtG5Y/lK0NporvqJ vjnVkG0wXEjNkS8PYun0zKkLHlBGAykhSEqX/h5MgowO5bP1DgZapC7LCqAh8b2f8zXWcq 4PSmGEwjobxV3sq/1E3OuhZALvQK515tJvOLkiTdlgxrUgQLwRO+KGo/oD3iqg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1782810235; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=W9/IisQus6xpOni87hezr5EcQ9yF6+JIxn9q6xoZ4Z8=; b=LQivQb1xHGCYF2BhkG0GrvF/ZRZiCzbKOrBqgFygu3QcfrhiFTJpk0LO5DILHjCa4Bis0i McgAC56OUlMN+MAg== From: "tip-bot2 for Peter Zijlstra" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: sched/core] sched/fair: Add cgroup_mode: max Cc: "Peter Zijlstra (Intel)" , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20260605124051.589618504@infradead.org> References: <20260605124051.589618504@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <178281023407.3843924.389063920114430889.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Precedence: bulk Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The following commit has been merged into the sched/core branch of tip: Commit-ID: 90ac22ffef48dbc6e7561434b6e01753a859bb51 Gitweb: https://git.kernel.org/tip/90ac22ffef48dbc6e7561434b6e01753a85= 9bb51 Author: Peter Zijlstra AuthorDate: Thu, 12 Mar 2026 15:42:35 +01:00 Committer: Peter Zijlstra CommitterDate: Tue, 30 Jun 2026 10:56:52 +02:00 sched/fair: Add cgroup_mode: max In order to avoid the average CPU fraction avg(F_g_n) becoming tiny '1/N', assume each cgroup is maximally concurrent and distrubute 'N*weight', such that: F_g_n' =3D N * F_g_n Giving: avg(F_g_n') =3D N*avg(F_g_n) ~ N * 1/N =3D 1 And while this sounds like it solves things, remember what that ~ meant. There is the corner case when a cgroup is minimally loaded, eg a single runnable task, therefore limit the CPU fraction to that of a nice -20 task to avoid getting too much load. This last bit is what makes it different from a previous proposal to allow raising cpu.weight to '100 * N', that would not limit the mininal concurrency case and results in a very large F_g_n. And just like F_g_n << 1 is problematic, so is F_g_n >> 1 for the exact same reasons (it would drown the kthreads, but it also risks overflowing the load values). So while this might appear to be a better scheme than the current default scheme, it doesn't really handle less than maximal concurrency nicely -- it clips and introduces artificially large weights. So where the traditional SMP mode works well when nr_tasks << nr_cpus, MAX doesn't work well in that regime and vice-versa. The meaning of "cpu.weight" would be: weight per allowed CPU. Included for completeness (and infrastructure). Signed-off-by: Peter Zijlstra (Intel) Link: https://patch.msgid.link/20260605124051.589618504%40infradead.org --- include/linux/cpuset.h | 6 +++++- kernel/cgroup/cpuset.c | 22 +++++++++++++++++- kernel/sched/debug.c | 1 +- kernel/sched/fair.c | 52 +++++++++++++++++++++++++++++++++++++---- 4 files changed, 76 insertions(+), 5 deletions(-) diff --git a/include/linux/cpuset.h b/include/linux/cpuset.h index 65d76a3..9db2d4f 100644 --- a/include/linux/cpuset.h +++ b/include/linux/cpuset.h @@ -80,6 +80,7 @@ extern void lockdep_assert_cpuset_lock_held(void); extern void cpuset_cpus_allowed_locked(struct task_struct *p, struct cpumask= *mask); extern void cpuset_cpus_allowed(struct task_struct *p, struct cpumask *mask); extern bool cpuset_cpus_allowed_fallback(struct task_struct *p); +extern int cpuset_num_cpus(struct cgroup *cgroup); extern nodemask_t cpuset_mems_allowed(struct task_struct *p); #define cpuset_current_mems_allowed (current->mems_allowed) void cpuset_init_current_mems_allowed(void); @@ -216,6 +217,11 @@ static inline bool cpuset_cpus_allowed_fallback(struct t= ask_struct *p) return false; } =20 +static inline int cpuset_num_cpus(struct cgroup *cgroup) +{ + return num_online_cpus(); +} + static inline nodemask_t cpuset_mems_allowed(struct task_struct *p) { return node_possible_map; diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index 591e3aa..e2f3da7 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -4124,6 +4124,28 @@ bool cpuset_cpus_allowed_fallback(struct task_struct *= tsk) return changed; } =20 +/* + * Returns the number of CPUs available for this cgroup. + * + * This only really works for cgroup-v2 where all the controllers are mounted + * in the same hierarchy. If not cgroup-v2 or no cpuset controller is + * configured it reverts to num_online_cpus(). + */ +int cpuset_num_cpus(struct cgroup *cgrp) +{ + int nr =3D num_online_cpus(); + struct cpuset *cs; + + if (is_in_v2_mode()) { + guard(rcu)(); + cs =3D css_cs(cgroup_e_css(cgrp, &cpuset_cgrp_subsys)); + if (cs) + nr =3D cpumask_weight(cs->effective_cpus); + } + + return nr; +} + void __init cpuset_init_current_mems_allowed(void) { nodes_setall(current->mems_allowed); diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c index 53b9e82..84e0ac4 100644 --- a/kernel/sched/debug.c +++ b/kernel/sched/debug.c @@ -640,6 +640,7 @@ static int cgroup_mode =3D 1; static const char *cgroup_mode_str[] =3D { "up", "smp", + "max", }; =20 static int sched_cgroup_mode(const char *str) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 3f8a280..b556d53 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -4801,12 +4801,10 @@ static inline int throttled_hierarchy(struct cfs_rq *= cfs_rq); * * hence icky! */ -static long calc_smp_shares(struct cfs_rq *cfs_rq) +static long __calc_smp_shares(struct cfs_rq *cfs_rq, long tg_shares, long sh= ares_max) { - long tg_weight, tg_shares, load, shares; struct task_group *tg =3D cfs_rq->tg; - - tg_shares =3D READ_ONCE(tg->shares); + long tg_weight, load, shares; =20 load =3D max(scale_load_down(cfs_rq->load.weight), cfs_rq->avg.load_avg); =20 @@ -4832,7 +4830,48 @@ static long calc_smp_shares(struct cfs_rq *cfs_rq) * case no task is runnable on a CPU MIN_SHARES=3D2 should be returned * instead of 0. */ - return clamp_t(long, shares, MIN_SHARES, tg_shares); + return clamp_t(long, shares, MIN_SHARES, shares_max); +} + +static int tg_cpus(struct task_group *tg) +{ + int nr =3D num_online_cpus(); + + if (cpusets_enabled()) { + struct cgroup *cgrp =3D tg->css.cgroup; + if (cgrp) + nr =3D cpuset_num_cpus(cgrp); + } + + return nr; +} + +/* + * Func: min(fraction(nr_cpus * tg->shares), nice -20) + * + * Scale tg->shares by the maximal number of CPUs; but clip the max shares at + * nice -20, otherwise a single spinner on a 512 CPU machine would result in + * 512*NICE_0_LOAD, which is also crazy. + */ +static long calc_max_shares(struct cfs_rq *cfs_rq) +{ + struct task_group *tg =3D cfs_rq->tg; + int nr =3D tg_cpus(tg); + long tg_shares =3D READ_ONCE(tg->shares); + long max_shares =3D scale_load(sched_prio_to_weight[0]); + return __calc_smp_shares(cfs_rq, tg_shares * nr, max_shares); +} + +/* + * Func: fraction(tg->shares) + * + * This infamously results in tiny shares when you have many CPUs. + */ +static long calc_smp_shares(struct cfs_rq *cfs_rq) +{ + struct task_group *tg =3D cfs_rq->tg; + long tg_shares =3D READ_ONCE(tg->shares); + return __calc_smp_shares(cfs_rq, tg_shares, tg_shares); } =20 /* @@ -4857,6 +4896,9 @@ void __sched_cgroup_mode_update(int mode) default: func =3D &calc_smp_shares; break; + case 2: + func =3D &calc_max_shares; + break; } static_call_update(calc_group_shares, func); }