[PATCH 0/1] sched/deadline: Log Fair Server re-enablement

[PATCH 0/1] sched/deadline: Log Fair Server re-enablement

[Aaron Tomlin]

Hi Ingo, Peter, Juri, Vincent,

During a recent audit of the Fair Server related code, I observed that
whilst disabling the server via the debugfs interface emits a console
notification, re-enabling it remains silent. This patch rectifies that
asymmetry by ensuring a corresponding message is logged when runtime is
restored.

This omission was identified during the investigation and code review
following the recent discussion regarding
SCHED_FEAT(RT_SUPPRESS_FAIR_SERVER) [1]. Please note that this patch is
standalone and does not depend upon the aforementioned RFC patch.

[1]: https://lore.kernel.org/lkml/20260106034209.2703289-2-atomlin@atomlin.com/

Aaron Tomlin (1):
  sched/deadline: Log Fair Server re-enablement for symmetry with
    debugfs

 kernel/sched/deadline.c | 6 ++++++
 1 file changed, 6 insertions(+)

-- 
2.51.0

[PATCH 1/1] sched/deadline: Log Fair Server re-enablement for symmetry with debugfs

[Aaron Tomlin]

Currently, the scheduler's debug interface emits a notification to the
console when the Fair Server is explicitly disabled via the fair_server
sysfs attribute. However, no corresponding log entry is generated when
the server is subsequently re-enabled.

This omission results in an asymmetry within the kernel logs,
potentially obscuring the true operational state of the scheduler during
debugging or performance analysis.

This patch amends dl_server_apply_params() to introduce the requisite
logging. By detecting the transition from zero to non-zero
bandwidth - strictly for the Fair Server entity and excluding
initialisation - we ensure that a "Fair server re-enabled" message is
emitted. This restores logging symmetry and provides administrators with
a clear audit trail of manual runtime adjustments.

Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
---
 kernel/sched/deadline.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index 319439fe1870..e64fb988e957 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -1867,6 +1867,7 @@ int dl_server_apply_params(struct sched_dl_entity *dl_se, u64 runtime, u64 perio
 	u64 old_bw = init ? 0 : to_ratio(dl_se->dl_period, dl_se->dl_runtime);
 	u64 new_bw = to_ratio(period, runtime);
 	struct rq *rq = dl_se->rq;
+	bool fair_server = dl_se == &rq->fair_server;
 	int cpu = cpu_of(rq);
 	struct dl_bw *dl_b;
 	unsigned long cap;
@@ -1876,6 +1877,11 @@ int dl_server_apply_params(struct sched_dl_entity *dl_se, u64 runtime, u64 perio
 	dl_b = dl_bw_of(cpu);
 	guard(raw_spinlock)(&dl_b->lock);
 
+	/* Symmetric to disable message in sched_fair_server_write() */
+	if (!init && fair_server && !old_bw && new_bw)
+		printk_deferred("Fair server re-enabled on CPU %d.\n",
+				cpu);
+
 	cpus = dl_bw_cpus(cpu);
 	cap = dl_bw_capacity(cpu);
 
-- 
2.51.0

Re: [PATCH 1/1] sched/deadline: Log Fair Server re-enablement for symmetry with debugfs

[K Prateek Nayak]

On 1/9/2026 8:49 AM, Aaron Tomlin wrote:
> Currently, the scheduler's debug interface emits a notification to the
> console when the Fair Server is explicitly disabled via the fair_server
> sysfs attribute. However, no corresponding log entry is generated when
> the server is subsequently re-enabled.
> 
> This omission results in an asymmetry within the kernel logs,
> potentially obscuring the true operational state of the scheduler during
> debugging or performance analysis.

Well, if you are disabling the fair_server, you're opening the doors to
bigger problems and that printk mainly serves as an indicator to dismiss
user induced starvation issues during debugs.

Why do you care about the symmetry of this log when you shouldn't be
setting the runtime to 0 in the first place?

> 
> This patch amends dl_server_apply_params() to introduce the requisite
> logging. By detecting the transition from zero to non-zero
> bandwidth - strictly for the Fair Server entity and excluding
> initialisation - we ensure that a "Fair server re-enabled" message is
> emitted. This restores logging symmetry and provides administrators with
> a clear audit trail of manual runtime adjustments.
> 
> Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
> ---
>  kernel/sched/deadline.c | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
> index 319439fe1870..e64fb988e957 100644
> --- a/kernel/sched/deadline.c
> +++ b/kernel/sched/deadline.c
> @@ -1867,6 +1867,7 @@ int dl_server_apply_params(struct sched_dl_entity *dl_se, u64 runtime, u64 perio
>  	u64 old_bw = init ? 0 : to_ratio(dl_se->dl_period, dl_se->dl_runtime);
>  	u64 new_bw = to_ratio(period, runtime);
>  	struct rq *rq = dl_se->rq;
> +	bool fair_server = dl_se == &rq->fair_server;
>  	int cpu = cpu_of(rq);
>  	struct dl_bw *dl_b;
>  	unsigned long cap;
> @@ -1876,6 +1877,11 @@ int dl_server_apply_params(struct sched_dl_entity *dl_se, u64 runtime, u64 perio
>  	dl_b = dl_bw_of(cpu);
>  	guard(raw_spinlock)(&dl_b->lock);
>  
> +	/* Symmetric to disable message in sched_fair_server_write() */
> +	if (!init && fair_server && !old_bw && new_bw)
> +		printk_deferred("Fair server re-enabled on CPU %d.\n",
> +				cpu);

That is an absolutely terrible place to put it. Why can't we have it in
sched_fair_server_write() for DL_RUNTIME when the
"rq->fair_server.dl_runtime" is 0 initially and is modified to a
non-zero value similar to the "Fair server disabled" message?

I still think once the fair server is disabled, the pieces are for the
user to keep. I wouldn't want us debugging:

    Fair server disabled in CPU X ...
    Fair server re-enabled in CPU X ...
    INFO: rcu_tasks detected stalls ...

only to realise the stalls were a result of starving the fair threads
and the fair server didn't run in time / didn't have enough B/W to
prevent that stall.

> +
>  	cpus = dl_bw_cpus(cpu);
>  	cap = dl_bw_capacity(cpu);
>  

-- 
Thanks and Regards,
Prateek

Re: [PATCH 1/1] sched/deadline: Log Fair Server re-enablement for symmetry with debugfs

[Aaron Tomlin]

[-- Attachment #1: Type: text/plain, Size: 4822 bytes --]

On Fri, Jan 09, 2026 at 11:47:10AM +0530, K Prateek Nayak wrote:
> Well, if you are disabling the fair_server, you're opening the doors to
> bigger problems and that printk mainly serves as an indicator to dismiss
> user induced starvation issues during debugs.
> 
> Why do you care about the symmetry of this log when you shouldn't be
> setting the runtime to 0 in the first place?

Hi Prateek,

Whilst I fully appreciate that indefinitely disabling the Fair Server
invites systemic peril, I would respectfully submit that there are
legitimate, transient scenarios where such intervention is warranted.

Consider a strictly partitioned environment utilising isolcpus=domain,5-8
alongside nohz_full=5-8. A latency-critical SCHED_FIFO task executing on
CPU 5 that never enters the kernel requires absolute isolation. If a
SCHED_NORMAL (CFS) task is enqueued - perhaps a CPU-specific kthread or
some other user-specific task - the current architecture wakes the Deadline
Server, which in turn restarts the clock-tick - see sched_can_stop_tick().
By temporarily disabling the Fair Server via the debug interface, an
administrator can preclude this interruption during a specific, sensitive
window of execution, before restoring standard operation once the critical
phase has concluded.

> > This patch amends dl_server_apply_params() to introduce the requisite
> > logging. By detecting the transition from zero to non-zero
> > bandwidth - strictly for the Fair Server entity and excluding
> > initialisation - we ensure that a "Fair server re-enabled" message is
> > emitted. This restores logging symmetry and provides administrators with
> > a clear audit trail of manual runtime adjustments.
> > 
> > Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
> > ---
> >  kernel/sched/deadline.c | 6 ++++++
> >  1 file changed, 6 insertions(+)
> > 
> > diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
> > index 319439fe1870..e64fb988e957 100644
> > --- a/kernel/sched/deadline.c
> > +++ b/kernel/sched/deadline.c
> > @@ -1867,6 +1867,7 @@ int dl_server_apply_params(struct sched_dl_entity *dl_se, u64 runtime, u64 perio
> >  	u64 old_bw = init ? 0 : to_ratio(dl_se->dl_period, dl_se->dl_runtime);
> >  	u64 new_bw = to_ratio(period, runtime);
> >  	struct rq *rq = dl_se->rq;
> > +	bool fair_server = dl_se == &rq->fair_server;
> >  	int cpu = cpu_of(rq);
> >  	struct dl_bw *dl_b;
> >  	unsigned long cap;
> > @@ -1876,6 +1877,11 @@ int dl_server_apply_params(struct sched_dl_entity *dl_se, u64 runtime, u64 perio
> >  	dl_b = dl_bw_of(cpu);
> >  	guard(raw_spinlock)(&dl_b->lock);
> >  
> > +	/* Symmetric to disable message in sched_fair_server_write() */
> > +	if (!init && fair_server && !old_bw && new_bw)
> > +		printk_deferred("Fair server re-enabled on CPU %d.\n",
> > +				cpu);
> 
> That is an absolutely terrible place to put it. Why can't we have it in
> sched_fair_server_write() for DL_RUNTIME when the
> "rq->fair_server.dl_runtime" is 0 initially and is modified to a
> non-zero value similar to the "Fair server disabled" message?
> 
> I still think once the fair server is disabled, the pieces are for the
> user to keep. I wouldn't want us debugging:
> 
>     Fair server disabled in CPU X ...
>     Fair server re-enabled in CPU X ...
>     INFO: rcu_tasks detected stalls ...
> 
> only to realise the stalls were a result of starving the fair threads
> and the fair server didn't run in time / didn't have enough B/W to
> prevent that stall.

Regarding the implementation, I concede that placing the logging logic
within dl_server_apply_params() was suboptimal. You are quite right;
sched_fair_server_write() is the appropriate location for this mechanism,
as it aligns the logging directly with the user-space interaction.

Regarding your concern about debugging RCU stalls and the "keep the pieces"
philosophy: I would argue that this is precisely why the symmetry in
logging is essential.

Without the "re-enabled" marker, the audit trail is incomplete. If a system
stalls, seeing only a "Fair server disabled" message leaves the duration of
the starvation event ambiguous. By explicitly logging the re-enablement, we
establish a definitive timeline. If an RCU stall occurs shortly after the
server is re-enabled, the timestamp provides the necessary evidence to
correlate the crash directly with the preceding starvation
period — confirming that the user's intervention was indeed the root cause.
Transparency, in this case, expedites the diagnosis of "user-induced"
failure.

I shall prepare a revised patch that moves the logic to
sched_fair_server_write() and ensures the message is emitted only upon the
transition from zero to a non-zero runtime.


Kind regards,
-- 
Aaron Tomlin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 1/1] sched/deadline: Log Fair Server re-enablement for symmetry with debugfs

[K Prateek Nayak]

Hello Aaron,

On 1/9/2026 8:00 PM, Aaron Tomlin wrote:
> Consider a strictly partitioned environment utilising isolcpus=domain,5-8
> alongside nohz_full=5-8. A latency-critical SCHED_FIFO task executing on
> CPU 5 that never enters the kernel requires absolute isolation. If a
> SCHED_NORMAL (CFS) task is enqueued - perhaps a CPU-specific kthread or
> some other user-specific task - the current architecture wakes the Deadline
> Server, which in turn restarts the clock-tick - see sched_can_stop_tick().
> By temporarily disabling the Fair Server via the debug interface, an
> administrator can preclude this interruption during a specific, sensitive
> window of execution, before restoring standard operation once the critical
> phase has concluded.

I believe the suggested solution to that was to trace the reason for the
kthread/fair task waking up on isolated CPUs and prevent the wakeup if
it is for some unnecessary operation as opposed to disabling the fair
server.

We have tools like https://docs.kernel.org/trace/osnoise-tracer.html to
capture these noise. Trace the noise, bring up the case where isolation
is broken on the current *upstream* kernel to the mailing list, and we
can solve it for everyone instead of disabling fair server as a duct
tape.

[..snip..]

>> I still think once the fair server is disabled, the pieces are for the
>> user to keep. I wouldn't want us debugging:
>>
>>     Fair server disabled in CPU X ...
>>     Fair server re-enabled in CPU X ...
>>     INFO: rcu_tasks detected stalls ...
>>
>> only to realise the stalls were a result of starving the fair threads
>> and the fair server didn't run in time / didn't have enough B/W to
>> prevent that stall.

[..snip..]
> Regarding your concern about debugging RCU stalls and the "keep the pieces"
> philosophy: I would argue that this is precisely why the symmetry in
> logging is essential.

I would argue that fiddling with the fair server is a terrible idea and
once the user disables it, all bets are off. It becomes their headache
to solve.

> 
> Without the "re-enabled" marker, the audit trail is incomplete. If a system
> stalls, seeing only a "Fair server disabled" message leaves the duration of
> the starvation event ambiguous. By explicitly logging the re-enablement, we
> establish a definitive timeline. If an RCU stall occurs shortly after the
> server is re-enabled, the timestamp provides the necessary evidence to
> correlate the crash directly with the preceding starvation
> period — confirming that the user's intervention was indeed the root cause.
> Transparency, in this case, expedites the diagnosis of "user-induced"
> failure.

Juri, Peter, is changing the fair server's bandwidth frequently very
common scenario is the field?

If not, can we add a pr_warn() for when the fair server's parameters
are changed by the userspace just to catch any absurd values that
reduce the bandwidth to a minimum without disabling the server?

I can do something absolutely stupid like this without dmesg logging
anything that would indicate I'm being stupid:

    # echo 4000000000 > /sys/kernel/debug/sched/fair_server/cpu0/period
    # echo 1 > /sys/kernel/debug/sched/fair_server/cpu0/runtime
    # sudo taskset -c 0 chrt -r 99 ~/scripts/loop&
    # taskset -c 0 bash -c 'mkdir /sys/fs/cgroup/cg0; echo $$ > /sys/fs/cgroup/cg0/cgroup.procs;'

    ... wait for a while

     INFO: task bash:4272 blocked for more than 120 seconds.
           Not tainted 6.19.0-rc1-tip+ #162
     "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
     task:bash            state:D stack:0     pid:4272  tgid:4272  ppid:4271   task_flags:0x400100 flags:0x00080000


A taint might be too far but a log should be acceptable?

-- 
Thanks and Regards,
Prateek

Re: [PATCH 1/1] sched/deadline: Log Fair Server re-enablement for symmetry with debugfs

[Aaron Tomlin]

[-- Attachment #1: Type: text/plain, Size: 3605 bytes --]

On Mon, Jan 12, 2026 at 10:44:03AM +0530, K Prateek Nayak wrote:
> I believe the suggested solution to that was to trace the reason for the
> kthread/fair task waking up on isolated CPUs and prevent the wakeup if
> it is for some unnecessary operation as opposed to disabling the fair
> server.

Hi Prateek,

> We have tools like https://docs.kernel.org/trace/osnoise-tracer.html to
> capture these noise. Trace the noise, bring up the case where isolation
> is broken on the current *upstream* kernel to the mailing list, and we
> can solve it for everyone instead of disabling fair server as a duct
> tape.

Thank you for your insights.

I fully concur that, in an ideal world, the "correct" solution is
invariably to identify and eliminate the root cause of any spurious
SCHED_NORMAL wakeups on isolated CPUs. Tools such as the osnoise tracer are
indeed invaluable for this pursuit.

However, I would respectfully submit that there remains a distinction
between the theoretical purity of the kernel and the pragmatic reality of
managing highly specialised, latency-critical partitions.

It is pertinent to note that the kernel currently affords users the
capability to manually modify the Fair Server's parameters via
/sys/kernel/debug/sched/fair_server/. As this resides within debugfs, it
is, by definition, a debug-only interface and not strictly considered
"production safe" or guaranteed to be free from side effects. The capacity
for a user to destabilise their system via this interface - effectively
"shooting themselves in the foot" - already exists. This existing interface
is useful for educated users who are willing to accept full accountability
for system stability in exchange for absolute determinism for a defined
period of time.

> Juri, Peter, is changing the fair server's bandwidth frequently very
> common scenario is the field?
> 
> If not, can we add a pr_warn() for when the fair server's parameters
> are changed by the userspace just to catch any absurd values that
> reduce the bandwidth to a minimum without disabling the server?
> 
> I can do something absolutely stupid like this without dmesg logging
> anything that would indicate I'm being stupid:
> 
>     # echo 4000000000 > /sys/kernel/debug/sched/fair_server/cpu0/period
>     # echo 1 > /sys/kernel/debug/sched/fair_server/cpu0/runtime
>     # sudo taskset -c 0 chrt -r 99 ~/scripts/loop&
>     # taskset -c 0 bash -c 'mkdir /sys/fs/cgroup/cg0; echo $$ > /sys/fs/cgroup/cg0/cgroup.procs;'
> 
>     ... wait for a while
> 
>      INFO: task bash:4272 blocked for more than 120 seconds.
>            Not tainted 6.19.0-rc1-tip+ #162
>      "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
>      task:bash            state:D stack:0     pid:4272  tgid:4272  ppid:4271   task_flags:0x400100 flags:0x00080000
> 
> 
> A taint might be too far but a log should be acceptable?

Regarding your valid concern about visibility and safety: I am agreeable to
hardening the observability of such changes. In the next iteration, I
propose to introduce a pr_warn() that triggers whenever the Fair Server's
runtime or period is modified from its default value (50 * NSEC_PER_MSEC
and 1000 * NSEC_PER_MSEC). This will ensure that any deviation - whether it
be a complete disablement or a reduction to unsafe levels - is clearly
logged, rightfully alerting administrators to the non-standard
configuration without removing the latitude required by those who
explicitly need to make that trade-off.


Kind regards,
-- 
Aaron Tomlin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 1/1] sched/deadline: Log Fair Server re-enablement for symmetry with debugfs

[Shrikanth Hegde]

On 1/12/26 8:02 PM, Aaron Tomlin wrote:
> On Mon, Jan 12, 2026 at 10:44:03AM +0530, K Prateek Nayak wrote:
>> I believe the suggested solution to that was to trace the reason for the
>> kthread/fair task waking up on isolated CPUs and prevent the wakeup if
>> it is for some unnecessary operation as opposed to disabling the fair
>> server.
> 
> Hi Prateek,
> 
>> We have tools like https://docs.kernel.org/trace/osnoise-tracer.html to
>> capture these noise. Trace the noise, bring up the case where isolation
>> is broken on the current *upstream* kernel to the mailing list, and we
>> can solve it for everyone instead of disabling fair server as a duct
>> tape.
> 
> Thank you for your insights.
> 
> I fully concur that, in an ideal world, the "correct" solution is
> invariably to identify and eliminate the root cause of any spurious
> SCHED_NORMAL wakeups on isolated CPUs. Tools such as the osnoise tracer are
> indeed invaluable for this pursuit.
> 
> However, I would respectfully submit that there remains a distinction
> between the theoretical purity of the kernel and the pragmatic reality of
> managing highly specialised, latency-critical partitions.
> 
> It is pertinent to note that the kernel currently affords users the
> capability to manually modify the Fair Server's parameters via
> /sys/kernel/debug/sched/fair_server/. As this resides within debugfs, it
> is, by definition, a debug-only interface and not strictly considered
> "production safe" or guaranteed to be free from side effects. The capacity
> for a user to destabilise their system via this interface - effectively
> "shooting themselves in the foot" - already exists. This existing interface
> is useful for educated users who are willing to accept full accountability
> for system stability in exchange for absolute determinism for a defined
> period of time.
> 
>> Juri, Peter, is changing the fair server's bandwidth frequently very
>> common scenario is the field?
>>
>> If not, can we add a pr_warn() for when the fair server's parameters
>> are changed by the userspace just to catch any absurd values that
>> reduce the bandwidth to a minimum without disabling the server?
>>
>> I can do something absolutely stupid like this without dmesg logging
>> anything that would indicate I'm being stupid:
>>
>>      # echo 4000000000 > /sys/kernel/debug/sched/fair_server/cpu0/period
>>      # echo 1 > /sys/kernel/debug/sched/fair_server/cpu0/runtime
>>      # sudo taskset -c 0 chrt -r 99 ~/scripts/loop&
>>      # taskset -c 0 bash -c 'mkdir /sys/fs/cgroup/cg0; echo $$ > /sys/fs/cgroup/cg0/cgroup.procs;'
>>
>>      ... wait for a while
>>
>>       INFO: task bash:4272 blocked for more than 120 seconds.
>>             Not tainted 6.19.0-rc1-tip+ #162
>>       "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
>>       task:bash            state:D stack:0     pid:4272  tgid:4272  ppid:4271   task_flags:0x400100 flags:0x00080000
>>
>>
>> A taint might be too far but a log should be acceptable?
> 
> Regarding your valid concern about visibility and safety: I am agreeable to
> hardening the observability of such changes. In the next iteration, I
> propose to introduce a pr_warn() that triggers whenever the Fair Server's
> runtime or period is modified from its default value (50 * NSEC_PER_MSEC
> and 1000 * NSEC_PER_MSEC). This will ensure that any deviation - whether it
> be a complete disablement or a reduction to unsafe levels - is clearly
> logged, rightfully alerting administrators to the non-standard
> configuration without removing the latitude required by those who
> explicitly need to make that trade-off.
> 

Currently it is 5%. It is going to be tricky to define unsafe levels.

Looks like Either one wants it or don't want interference from it. Are there any
users changing the default value?