From: luca abeni <luca.abeni@santannapisa.it>
To: Tejun Heo <tj@kernel.org>
Cc: Yuri Andriaccio <yurand2000@gmail.com>,
bsegall@google.com, cgroups@vger.kernel.org,
dietmar.eggemann@arm.com, hannes@cmpxchg.org,
juri.lelli@redhat.com, kprateek.nayak@amd.com,
linux-kernel@vger.kernel.org, mgorman@suse.de, mingo@redhat.com,
mkoutny@suse.com, peterz@infradead.org, rostedt@goodmis.org,
vincent.guittot@linaro.org, vschneid@redhat.com,
yuri.andriaccio@santannapisa.it
Subject: Re: [RFC PATCH v7 00/29] Hierarchical Constant Bandwidth Server
Date: Fri, 20 Nov 2026 12:06:59 +0100 [thread overview]
Message-ID: <20261120120659.2e902463@linux.fritz.box> (raw)
In-Reply-To: <471bf8d446becabfca00041fb2a78067@kernel.org>
Hi Tejun,
On Thu, 19 Nov 2026 20:46:18 -1000
Tejun Heo <tj@kernel.org> wrote:
> Hello, Luca.
>
> I'm only thinking about this from the interface POV, so please read
> the following with that in mind.
I understand (I hope :) the interface issue, and I'd like to find a
compromise that works from both an interface and a "performance
evaluation" perspective.
So below I'm trying to clarify the performance evaluation issues with an
example (maybe I am just repeating myself; if so, I am sorry... but I
hope discussing this example leads to a solution that works from every
perspective).
[...]
> > So, what about a different idea: a cgroup can be "max" or have
> > (runtime,period); when we set (runtime,period), this is set for all
> > the CPU cores, but then we can override (with runtime 0, or
> > different runtime/period) some cores (but we cannot override with
> > "max").
>
> If the interface is per-CPU, I think it should be per-CPU all the way.
> What a cgroup has on one CPU shouldn't say anything about what it has
> on another. That seems the most consistent in itself and with the
> other controllers.
Well, my "mixing" of per-CPU and per-cgroup interfaces is mainly due to
the fact that the tasks in the cgroup can be "global" (allowed to
migrate on all the cgroup's cores) or pinned to a CPU core... I
try to explain with an example, based on a cgroup with 2 cores,
core0 and core1:
- If both core0 and core1 are configured with runtime 10ms and period
100ms, then the real-time tasks running in the cgroup are guaranteed to
execute for 20ms every 100ms. Based on this, some theoretical analysis
can be used to check if the tasks will respect their deadlines, etc
- If both core0 and core1 are configured as "max", then the real-time
tasks contained in the cgroup will always execute with the parent
cgroup's deadline entities. Knowing the parameters of such deadline
entities, and knowing the tasks running in both cgroups, it is again
possible to provide some performance guarantees
- if core0 is configured with runtime 10ms and period 100ms and core1
is configured as "max", then real-time tasks migrating to core1 will
execute with the parent's deadline entity... But that deadline entity
is also shared with other real-time tasks. So, it is unclear how much
CPU time the cgroup's real-time tasks are expected to receive. Also,
the tasks migrating from core0 to core1 will consume the parent's
runtime on core1 (but not on core0), making it difficult to provide
performance guarantees for the parent cgroup too.
In other words, if tasks can migrate between cores, performance
guarantees should be "per-cgroup", not per-CPU, which makes it
difficult to mix "(runtime, period)" and "max" in the same cgroup.
If, on the other hand, tasks were pinned to cores or migrations were
disabled, the mix wouldn't be an issue.
> Take Juri's example of a top-level group which has an entry only for
> CPU0. Its tasks are limited on CPU0 and not on CPU1, which is what the
> config says.
I understand this if tasks are pinned, but I still fail to reason about
performance when tasks can freely migrate between CPU0 and CPU1 (I can
only think about a very pessimistic evaluation: assuming that all the
tasks execute on CPU0 when we evaluate this cgroup's performance, and
assuming that they all execute on CPU1 when evaluate the interference
on the parent's workload).
> io.max works the same way. A limit on one device says
> nothing about the other devices, and a device without an entry is at
> "max". If the group shouldn't run RT tasks on CPU1, it can be given
> runtime=0 there.
I do not know much about IO device and io.max, sorry... But maybe the
difference is that tasks do not "migrate" between different devices?
> Tasks moving across CPUs doesn't change how the resources are
> allocated. A task consumes from whatever serves its cgroup on the CPU
> it's running on.
>
> I'm not too familiar with the schedulability analysis side, so please
> correct me if I'm missing something. Wouldn't it work if the setups
> which need the analysis configure the group on all CPUs? That seems
> like something which can be a convention in how groups are configured
> rather than forced through the interface, which looks like it'd get
> clunky whichever shape we pick.
Something like leaving the "all-configured or all-max" as a system
administrator responsibility, instead of enforcing it, right?
I guess this can work, but I had no time to discuss this with Yuri, so
I do not know how difficult this can be from the implementation's
perspective.
I also fear this kind of solution can increase the probability of
misconfiguring the system (ending up with the feature not being used).
> > We think 0 as a default would be problematic, as it means "no
> > real-time tasks", and 0 in the root means "no real-time tasks in
> > all the cgroups"... Or am I missing your point?
>
> Yeah, zero doesn't work there. How about defaulting the root to "max"
> like the other cgroups? "max" on the root would mean that nothing is
> set aside for cgroups. RT tasks which aren't under any entry run
> without a server as they do now, and the budgets of top-level cgroups
> are admitted against the DL bandwidth directly. An entry on the root
> would be the total that cgroup servers can reserve on that CPU, and
> writing one fails if what's already reserved doesn't fit. RT tasks in
> the root itself keep running without a server either way.
Yes, I think this is OK. It means that hierarchical scheduling is
disabled by default, and the administrator has to write something in
the root cgroup to enable it.
Thanks,
Luca
next prev parent reply other threads:[~2026-11-20 11:07 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-09 16:35 Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 01/29] sched/deadline: Introduce dl_freq_invariant flag for fair/ext servers Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 02/29] sched/deadline: Do not access dl_se->rq directly Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 03/29] sched/deadline: Distinguish between dl_rq and my_q Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 04/29] sched/rt: Pass an rt_rq instead of an rq where needed Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 05/29] sched/rt: Move functions from rt.c to sched.h Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 06/29] sched/rt: Disable RT_GROUP_SCHED Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 07/29] sched/rt: Remove unnecessary runqueue pointer in struct rt_rq Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 08/29] sched/rt: Introduce HCBS specific structs in task_group Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 09/29] sched/core: Initialize HCBS specific structures Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 10/29] sched/deadline: Add dl_init_tg Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 11/29] sched/rt: Add {alloc/unregister/free}_rt_sched_group Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 12/29] sched/deadline: Account rt-cgroups bandwidth in deadline tasks schedulability tests Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 13/29] sched/rt: Implement dl-server operations for rt-cgroups Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 14/29] sched/rt: Update task event callbacks for HCBS scheduling Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 15/29] sched/rt: Remove support for cgroups-v1 Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 16/29] sched/rt: Update rt-cgroup schedulability checks Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 17/29] sched/rt: Update task's RT runqueue when switching scheduling class Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 18/29] sched/rt: Remove old RT_GROUP_SCHED data structures Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 19/29] sched/rt: Add HCBS migration code to related functions Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 20/29] sched/rt: Hook HCBS migration functions Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 21/29] sched/core: Execute enqueued balance callbacks when changing allowed CPUs Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 22/29] sched/rt: Try pull task on empty server pick Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 23/29] sched/core: Execute enqueued balance callbacks after migrate_disable_switch Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 24/29] sched/syscalls: Allow unpriviledged tasks to set RT priority inside cgroups Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 25/29] sched/rt: Enable HRTICK_DL for HCBS servers Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 26/29] Documentation: Update documentation for real-time cgroups Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 27/29] sched/rt: Integrate cpuset and cpu controllers Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 28/29] sched/rt: Prevent CPU offline operation on active rt-cgroups Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 29/29] sched/rt: Introduce multi-cpu support Yuri Andriaccio
2026-11-10 21:43 ` [RFC PATCH v7 00/29] Hierarchical Constant Bandwidth Server Tejun Heo
2026-11-11 17:56 ` Yuri Andriaccio
2026-11-17 15:49 ` Juri Lelli
2026-11-18 8:07 ` Tejun Heo
2026-11-19 11:23 ` Juri Lelli
2026-11-19 14:27 ` luca abeni
2026-11-19 14:36 ` Juri Lelli
2026-11-19 14:42 ` luca abeni
2026-11-19 14:14 ` luca abeni
2026-11-20 6:46 ` Tejun Heo
2026-11-20 11:06 ` luca abeni [this message]
2026-11-20 11:55 ` Tejun Heo
2026-11-16 14:25 ` Juri Lelli
2026-11-17 13:00 ` Yuri Andriaccio
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261120120659.2e902463@linux.fritz.box \
--to=luca.abeni@santannapisa.it \
--cc=bsegall@google.com \
--cc=cgroups@vger.kernel.org \
--cc=dietmar.eggemann@arm.com \
--cc=hannes@cmpxchg.org \
--cc=juri.lelli@redhat.com \
--cc=kprateek.nayak@amd.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=mkoutny@suse.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tj@kernel.org \
--cc=vincent.guittot@linaro.org \
--cc=vschneid@redhat.com \
--cc=yurand2000@gmail.com \
--cc=yuri.andriaccio@santannapisa.it \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®