Re: [RFC] RT throttling, DL/ fair server, RT_GROUP_SCHED and everything

[Juri Lelli <juri.lelli@redhat.com>]

Hi Sebastian,

On 23/10/26 18:13, Sebastian Andrzej Siewior wrote:
> Hi,
> 
> this is about the old RT throttling mechanism, the DL/fair server around
> the RT tasks, the /proc/sys/kernel/sched_rt_* mechanism to control the
> former and CONFIG_RT_GROUP_SCHED which never worked as it supposed to.
> The current status looks like a mess and I'm not sure if there is a bug
> somewhere leading to my headaches or if this is just not meant to be
> working together. If so out with it.
> 
> tl;dr /proc/…/sched_rt_*, sched_rt_runtime_exceeded() probably
> CONFIG_RT_GROUP_SCHED should be removed. A bit of docs should be added
> for the fair server (or the right place).

Thanks for putting this recap together!

> Now. Details.

...

> Without enabled RT_GROUP_SCHED the files can be set. Are they useful?
> According to Documentation/scheduler/sched-deadline.rst it used for
> "admission control" whatever that means. Anyway the fair server still
> kicks if it is disabled. So I *think* this is just something else and
> not part for the docs there.

Right, confusing, but for a root_domain comprising M CPUs, -deadline tasks can
be created while the sum of their bandwidths stays below

  M * (sched_rt_runtime_us / sched_rt_period_us)

and writing -1 disables the check altogether. That's the "admission control"
the docs hand-wave about, and it's also why the knobs can't simply be deleted.
As you noticed already, the servers are completely independent of these knobs:
writing -1 does not disable the fair server, it only lifts the DL admission
cap. In principle we could move this to a different interface that reads closer
to the underlying purpose, but I'm not sure how much breakage it could cause.

...

> Anyway. Could this interface be moved to /sys/kernel/sched/fair_server
> away from the debugfs part? debugfs is not for stable kernels/ ABI. Is
> there a reason why this might be considered debugging and not final or
> restricted from productive?

I think this was raised already in the past, but at the time maybe it
was still to early to say if dl-servers will stick around. They now have
got more users (sched-ext), so it might be a time we discuss a stable
(non-debug) interface for it. Your suggestion doesn't look bad to me.

> Just to summarize:
> - Remove /proc/sys/kernel/sched_rt_*
>   [ ] Yes [ ] No, because
>               _____________________

Yes and no :). We need an interface to tweak admission control, but we
could find a better/different one.

> - Remove CONFIG_RT_GROUP_SCHED
>   [ ] Yes [ ] No, because
>               _____________________

Luca already commented on this. I believe the plan is still to come up
with a DEADLINE backed RT_GROUP_SCHED for Cgroups v2. If cleaner, an
approach could be to first merge the removal of RT_GROUP_SCHED and then
reimplement the whole lot as Yuri and Luca are proposing to do. The
series is already somewhat doing that, but it could be made more
explicit/clean I guess.

> - An additional trace sched event for fair server preemption
>   [ ] Yes   [ ] No, it is silly to have

With 820725b0eb59f ("sched: Add deadline tracepoints") we already have
some of that: sched_dl_{throttle,replenish,update,server_start,server_stop},
which fire for fair/ext servers too (there's a type field, DL_SERVER_FAIR
and friends). They of course require something like this [1] to be
activated. We could add more if they are still not enough to get a clear
picture about what's going on.

> - Fair server documentation needs to go to
>   [ ] It is fine where it is
>   [ ] To ____________________

Not fine where it is: the servers (fair & ext) work independently of
RT_GROUP_SCHED and deserve their own description (defaults 50ms/1s per
CPU, what they guarantee for fair tasks under RT pressure, how to tune
or disable them). Either a new section in sched-deadline.rst (they are
DL servers after all) or a small standalone file works for me.

Thanks!
Juri

1 - https://github.com/jlelli/sched_tp/commits/deadline-tp/
    from Qais' https://github.com/qais-yousef/sched_tp