Re: [RFC] RT throttling, DL/ fair server, RT_GROUP_SCHED and everything

[Juri Lelli <juri.lelli@redhat.com>]

On 05/11/26 13:09, Sebastian Andrzej Siewior wrote:
> On 2026-10-26 09:05:42 [+0100], Juri Lelli wrote:
> > Hi Sebastian,
> Hi Juri,
> 
> > On 23/10/26 18:13, Sebastian Andrzej Siewior wrote:
> > > Hi,
> > > 
> > > this is about the old RT throttling mechanism, the DL/fair server around
> > > the RT tasks, the /proc/sys/kernel/sched_rt_* mechanism to control the
> > > former and CONFIG_RT_GROUP_SCHED which never worked as it supposed to.
> > > The current status looks like a mess and I'm not sure if there is a bug
> > > somewhere leading to my headaches or if this is just not meant to be
> > > working together. If so out with it.
> > > 
> > > tl;dr /proc/…/sched_rt_*, sched_rt_runtime_exceeded() probably
> > > CONFIG_RT_GROUP_SCHED should be removed. A bit of docs should be added
> > > for the fair server (or the right place).
> > 
> > Thanks for putting this recap together!
> > 
> > > Now. Details.
> > 
> > ...
> > 
> > > Without enabled RT_GROUP_SCHED the files can be set. Are they useful?
> > > According to Documentation/scheduler/sched-deadline.rst it used for
> > > "admission control" whatever that means. Anyway the fair server still
> > > kicks if it is disabled. So I *think* this is just something else and
> > > not part for the docs there.
> > 
> > Right, confusing, but for a root_domain comprising M CPUs, -deadline tasks can
> > be created while the sum of their bandwidths stays below
> > 
> >   M * (sched_rt_runtime_us / sched_rt_period_us)
> 
> My understanding is that this is the runtime available to a DL task and
> the length of a DL period (after the accounted runtime is reset). The
> left over runtime (of runtime is less than period) is either buffer
> (idle) or SCHED_OTHER runtime.
> This defines how much runtime a DL task gets from the system, correct?

How much bandwidth (runtime/period) DL tasks get from the system
(comprised of M CPUs). And a corresponding cap (no more than that will
be allowed to enter the system) - admission control.

> > and writing -1 disables the check altogether. That's the "admission control"
> > the docs hand-wave about, and it's also why the knobs can't simply be deleted.
> > As you noticed already, the servers are completely independent of these knobs:
> > writing -1 does not disable the fair server, it only lifts the DL admission
> > cap. In principle we could move this to a different interface that reads closer
> > to the underlying purpose, but I'm not sure how much breakage it could cause.
> 
> That is my point. The rt_ runtime/ period is about DL properties and not
> RT as in SCHED_FIFO/ RR as it once was. Both are set to one second as of
> today and can not be changed with enabled CONFIG_RT_GROUP_SCHED.
> 
> Could we please move it to /sys/kernel/sched/ and rename it as dl
> something? Is there any reason to keep as-is? There is
> 	Documentation/scheduler/sched-deadline.rst
> 	Documentation/arch/arm64/asymmetric-32bit.rst
> 
> referring to it in DL context. There is "ftracetest" writing
> unconditionally -1 as a workaround, see commit 57c4cfd4a2eef
> ("ftrace/selftests: workaround cgroup RT scheduling issues").
> 
> If the runtime/ period arguments are fine then we could keep
> it and make them only apply for the DL period.
> 
> Two files have been repurposed and if your intention as to disable the
> RT throttler then you will be surprised.

Right, it's indeed confusing. I believe the intent for keeping as it is
today was that in the future, if we get to 'fix' RT_GROUP_SCHED with
HCBS, the existing knobs will be control both group and simple DL tasks
bandwidth (they share the same bandwidth). I'm not against making things
clearer. Maybe let's hear from others what they think as well?

...

> > With 820725b0eb59f ("sched: Add deadline tracepoints") we already have
> > some of that: sched_dl_{throttle,replenish,update,server_start,server_stop},
> > which fire for fair/ext servers too (there's a type field, DL_SERVER_FAIR
> > and friends). They of course require something like this [1] to be
> > activated. We could add more if they are still not enough to get a clear
> > picture about what's going on.
> 
> Hmm. You may want to merge Qais's current master to get it compiled
> again ;) But then nothing hooks it up, so I just replaced with with
> trace_printk() just to see what happens and got:

Ack, yeah, I didn't use them recently and they rotted. :/

I will try to find some time soon to refresh and check what's missing
from the information they provide (from a debugging point of view).

Thanks,
Juri