mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
To: linux-kernel@vger.kernel.org
Subject: Re: [PATCH v4 00/17] sched/deadline: first bandwidth accounting fixes and selftests
Date: Thu, 19 Nov 2026 11:22:18 -0500	[thread overview]
Message-ID: <85b926c7-4f88-497d-8026-8cf1f1dce6a0@efficios.com> (raw)
In-Reply-To: <cover.1795103339.git.mathieu.desnoyers@efficios.com>

Issues with my patch send script. Will resend.


On 2026-11-19 11:21, Mathieu Desnoyers wrote:
> Hi Ingo, Juri,
> 
> This series fixes 14 bugs of SCHED_DEADLINE. Then it shows the deadline
> bandwidth of each runqueue in sched_debug, and adds two selftests. The
> fixes come first, so that stable can take patches 1 to 14.
> 
> Two of the bugs take the kernel down: a division by zero (patch 4), and
> an owner's CPU that locks up (patch 8). One leaves cpuset_mutex locked
> (patch 6). One leaves a task that never runs again (patch 7). Most of
> the others leave total_bw of a root domain, or this_bw of a runqueue,
> wrong. The root domain then admits deadline tasks beyond its capacity,
> or refuses every one.
> 
> These are the first 14 patches of "sched/deadline: bandwidth accounting
> fixes and selftests" v7 [1], as Juri proposed [2]. Their v7 numbers are
> 1-8, 10, 13, 14, 16, 20 and 37. The rest of v7 will follow as v8 of that
> series, on top of this one. Patch 10 was posted alone [3]. Juri
> asked to carry it here. It comes before patch 11, which adds a caller
> of the computation it fixes. v4 changes it, as described below.
> 
> Patch 12 is Juri's, patch 14 is Andrea Parri's. I carry them unchanged.
> Sebastian reviewed patch 14. Juri, three of your four fixes that Ingo
> dropped from sched/urgent are here:
> - 2/4 is patch 11.
> - 3/4 is patch 1.
> - 4/4 is patch 12.
> 1/4 stays in the second series.
> 
> The fixes
> =========
> 
>  1  Refuse SCHED_FLAG_KEEP_PARAMS requests that would change the policy
>     or the class.
>  2  Skip admission for parameters that SCHED_FLAG_KEEP_PARAMS keeps.
>  3  Let a task boosted by a deadline waiter change its policy.
>  4  Do not divide by zero active CPUs of a root domain.
>  5  Refuse a dl_server debugfs write while the runqueue is offline.
>  6  Take cpuset_mutex for a deadline policy change only with pi, so
>     that sysrq-n does not leave it locked.
>  7  Replenish a throttled task switched out and back, so that it runs
>     again.
>  8  Do not refill a boost from cleared parameters, so that the owner's
>     CPU no longer locks up.
>  9  Reset the deadline and the runtime with the parameters.
>  10 Compute the 0-lag time on 128 bits, so that a long period does not
>     release the bandwidth seconds late. Check first that the runtime
>     left fits the parameters until the deadline, so that the quotient
>     fits in 64 bits.
>  11 Release the bandwidth of a task that leaves the class asleep.
>  12 Reject SCHED_DEADLINE on exited tasks (Juri Lelli).
>  13 Keep this_bw right when a task changes during a remote wakeup.
>  14 Check the affinity of a deadline task under the runqueue lock, so
>     that sched_setaffinity() and sched_setattr() cannot both succeed
>     (Andrea Parri).
> 
> Patch 15 shows the deadline bandwidth of each runqueue in sched_debug.
> Patch 16 adds deadline_switch_test. Patch 17 adds deadline_bw_test, with
> the 47 of its 115 cases that pass with these fixes. The other cases
> come with their fixes, in the second series.
> 
> Testing
> =======
> 
> The 52 cases pass wherever they run, with lockdep and no kernel
> warning, in 25 QEMU guests:
> - 9 guests of 8 vCPUs, one with proxy execution off, and 1 of 4 vCPUs.
> - 5 guests with PREEMPT_RT, 4 of 8 vCPUs and 1 of 4.
> - 3 guests without CONFIG_CPUSETS, of 4, 4 and 5 vCPUs.
> - 4 guests of 5 vCPUs booted with isolcpus=domain,1,2 or
>   isolcpus=domain,0,1, 2 of them with PREEMPT_RT.
> - 1 guest of 8 vCPUs booted with isolcpus=domain,6,7.
> - 2 guests with CONFIG_RT_GROUP_SCHED, of 8 and 4 vCPUs.
> move_cancelled runs only in the last group. Without CONFIG_CPUSETS, the
> cases that need a cpuset skip. With 3 CPUs that are not isolated,
> params_waking skips: it needs 4.
> 
> On v7.4-rc2, six cases of deadline_bw_test fail in each of 24 guests of
> 8 vCPUs, in three runs. Patch 17 lists them. params_switched_asleep, new
> in v4, failed in the 8 guests of the third run. overrun_caught fails in
> 5 guests. In one, hotplug_server crashed the kernel with the division
> by zero that patch 4 fixes. throttled_round_trip and asleep_round_trip
> of deadline_switch_test fail too.
> 
> The third run also had the sysrq cases. sysrq_normalize failed in every
> guest. sysrq-n logged "BUG: sleeping function called from invalid
> context", from a mutex taken in __sched_setscheduler(). Patch 6 fixes
> this.
> 
> Each of the 15 kernel patches builds alone. The whole series also
> builds without CONFIG_CPUSETS, with PREEMPT_RT and with
> CONFIG_RT_GROUP_SCHED. The selftests build with no warning.
> 
> Juri ran v2 with cgroup v1 cpusets, and with the scx_simple sched_ext
> scheduler [4]. With sched_ext, the server cases of deadline_bw_test
> failed: they set the fair server, which sched_ext detaches. A change
> below makes them use the ext server. It was not tested with sched_ext.
> Not tested either: CPUs of unequal capacity.
> 
> How this was developed
> ======================
> 
> An LLM coding assistant wrote these patches, their messages and the
> tests, under my direction and review. It did not write patches 12 and
> 14. I know the concepts behind SCHED_DEADLINE, but not its
> implementation well. So I ask those who know this code for a careful
> reading of the fixes. A reproducer that stops failing shows that the
> symptom goes away. It does not show that the fix is the one you would
> have written.
> 
> The bugs came from:
> - flaky cgroup selftests of a feature I am developing;
> - the review of Juri's four fixes, and his review of v5;
> - building deadline_bw_test, and reading the code;
> - the Sashiko AI reviews of v1 to v7 of the original series, and of v1
>   to v3 of this one. Each finding was checked against the code first.
> 
> Changes since v3
> ================
> 
> - 10, from the Sashiko AI review of v3. On x86-64,
>   mul_u64_u64_div_u64() traps with a divide error when the quotient
>   does not fit in 64 bits. After a change of parameters, the runtime
>   left can be far above dl_runtime. With the period limit raised, v3
>   oopsed in task_non_contending() in an x86-64 guest. Before the
>   division, task_non_contending() now compares the runtime left with
>   what the parameters give until the deadline. A comment explains the
>   0-lag time after a change of parameters. Juri preferred this to a
>   change of the formula. This is the separate change promised in reply
>   to the Sashiko review of v2 [5].
> - 17: two new cases, params_switched_asleep and
>   params_shrunk_long_period, test patch 10. The modes and variables
>   that only the cases of the second series use are removed, from the
>   Sashiko AI review of v3.
> 
> Changes since v2
> ================
> 
> - 10 is new: "sched/deadline: Fix an overflow of the 0-lag time with
>   long periods", posted alone before [3].
> - 11: the known limit of the v2 cover moves into its message, with the
>   impact Juri gave.
> - 17, from the Sashiko AI review of v2 and Juri's testing:
>   - deboost_new_params_reboost moves to the second series. On this
>     series, it passes without the deboost it tests.
>   - The flag of the lock shared by the death cases is in shared memory.
>     Each case runs in its own child process.
>   - server_states skips when P cannot run SCHED_FIFO tasks. This is the
>     case with CONFIG_RT_GROUP_SCHED and the cpu controller on.
>   - While a sched_ext scheduler runs all fair tasks, the server cases
>     use the ext server, and hotplug_server skips.
> 
> Changes since v1
> ================
> 
> From the Sashiko AI review of v1, in deadline_bw_test:
> - rebuild_record_taken moves to the second series. It tests a fix of
>   that series, and fails without it when CPU 0 is isolated.
> - move_cancelled runs from the root cgroup. It turns the cpu controller
>   off at the end only if it turned it on.
> - Its setter thread is joined after a timeout too, and its result is
>   atomic.
> - The shared lock of the death cases is made consistent and destroyed
>   before it is set up again.
> - A failed probe fails the case, in hotplug_server, move_refused and
>   move_zero_bw.
> 
> Based on tip sched/urgent, ec6f6731538e (v7.4-rc2).
> 
> [1] https://lore.kernel.org/all/cover.1794922146.git.mathieu.desnoyers@efficios.com/
> [2] https://lore.kernel.org/all/av3J3zMfkRplg-N4@jlelli-thinkpadt14gen4.remote.csb/
> [3] https://lore.kernel.org/all/cover.1794926626.git.mathieu.desnoyers@efficios.com/
> [4] https://lore.kernel.org/all/av7NchMg0wnyzjcd@jlelli-thinkpadt14gen4.remote.csb/
> [5] https://lore.kernel.org/all/8282556f-a7bc-4950-b7a7-b5b78557412e@efficios.com/
> 
> v1: https://lore.kernel.org/all/cover.1795031605.git.mathieu.desnoyers@efficios.com/
> v2: https://lore.kernel.org/all/cover.1795037078.git.mathieu.desnoyers@efficios.com/
> v3: https://lore.kernel.org/all/cover.1795092090.git.mathieu.desnoyers@efficios.com/
> 
> Andrea Parri (1):
>   sched/deadline: Check SCHED_DEADLINE affinity under the rq lock
> 
> Juri Lelli (1):
>   sched/deadline: Reject SCHED_DEADLINE on exited tasks
> 
> Mathieu Desnoyers (15):
>   sched: Refuse SCHED_FLAG_KEEP_PARAMS that would change the policy or
>     the class
>   sched/deadline: Do not admit bandwidth for parameters that are kept
>   sched: Let a task boosted by a deadline waiter change its policy
>   sched/deadline: Do not divide by zero active CPUs of a root domain
>   sched/deadline: Refuse dl_server debugfs writes while the rq is
>     offline
>   sched: Take cpuset_mutex for a deadline policy change only with pi
>   sched/deadline: Do not lose a throttled task switched out and back
>   sched/deadline: Do not replenish a boost from cleared parameters
>   sched/deadline: Reset the deadline and the runtime with the parameters
>   sched/deadline: Fix an overflow of the 0-lag time with long periods
>   sched/deadline: Release the bandwidth of a task that leaves the class
>     asleep
>   sched/deadline: Fix this_bw when a task is changed during a remote
>     wakeup
>   sched/debug: Show the deadline bandwidth of each runqueue
>   selftests/sched: Add a test of SCHED_DEADLINE policy round trips
>   selftests/sched: Add a test of SCHED_DEADLINE bandwidth accounting
> 
>  include/linux/sched.h                         |    5 +
>  kernel/sched/core.c                           |   11 +
>  kernel/sched/deadline.c                       |  166 +-
>  kernel/sched/debug.c                          |   13 +-
>  kernel/sched/sched.h                          |    1 +
>  kernel/sched/syscalls.c                       |  103 +-
>  tools/testing/selftests/Makefile              |    1 +
>  .../selftests/sched/deadline/.gitignore       |    2 +
>  .../testing/selftests/sched/deadline/Makefile |    8 +
>  tools/testing/selftests/sched/deadline/config |    5 +
>  .../sched/deadline/deadline_bw_test.c         | 4960 +++++++++++++++++
>  .../sched/deadline/deadline_switch_test.c     | 1115 ++++
>  .../testing/selftests/sched/deadline/settings |    1 +
>  13 files changed, 6354 insertions(+), 37 deletions(-)
>  create mode 100644 tools/testing/selftests/sched/deadline/.gitignore
>  create mode 100644 tools/testing/selftests/sched/deadline/Makefile
>  create mode 100644 tools/testing/selftests/sched/deadline/config
>  create mode 100644 tools/testing/selftests/sched/deadline/deadline_bw_test.c
>  create mode 100644 tools/testing/selftests/sched/deadline/deadline_switch_test.c
>  create mode 100644 tools/testing/selftests/sched/deadline/settings
> 
> 
> base-commit: ec6f6731538e0d45872eb279fdeb477bed152d6f


-- 
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com

      parent reply	other threads:[~2026-11-19 16:22 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-19 16:21 Mathieu Desnoyers
2026-11-19 16:21 ` [PATCH v4 01/17] sched: Refuse SCHED_FLAG_KEEP_PARAMS that would change the policy or the class Mathieu Desnoyers
2026-11-19 16:21 ` [PATCH v4 02/17] sched/deadline: Do not admit bandwidth for parameters that are kept Mathieu Desnoyers
2026-11-19 16:21 ` [PATCH v4 03/17] sched: Let a task boosted by a deadline waiter change its policy Mathieu Desnoyers
2026-11-19 16:21 ` [PATCH v4 04/17] sched/deadline: Do not divide by zero active CPUs of a root domain Mathieu Desnoyers
2026-11-19 16:21 ` [PATCH v4 05/17] sched/deadline: Refuse dl_server debugfs writes while the rq is offline Mathieu Desnoyers
2026-11-19 16:21 ` [PATCH v4 06/17] sched: Take cpuset_mutex for a deadline policy change only with pi Mathieu Desnoyers
2026-11-19 16:21 ` [PATCH v4 07/17] sched/deadline: Do not lose a throttled task switched out and back Mathieu Desnoyers
2026-11-19 16:21 ` [PATCH v4 08/17] sched/deadline: Do not replenish a boost from cleared parameters Mathieu Desnoyers
2026-11-19 16:21 ` [PATCH v4 09/17] sched/deadline: Reset the deadline and the runtime with the parameters Mathieu Desnoyers
2026-11-19 16:21 ` [PATCH v4 10/17] sched/deadline: Fix an overflow of the 0-lag time with long periods Mathieu Desnoyers
2026-11-19 16:21 ` [PATCH v4 11/17] sched/deadline: Release the bandwidth of a task that leaves the class asleep Mathieu Desnoyers
2026-11-19 16:21 ` [PATCH v4 12/17] sched/deadline: Reject SCHED_DEADLINE on exited tasks Mathieu Desnoyers
2026-11-19 16:21 ` [PATCH v4 13/17] sched/deadline: Fix this_bw when a task is changed during a remote wakeup Mathieu Desnoyers
2026-11-19 16:21 ` [PATCH v4 14/17] sched/deadline: Check SCHED_DEADLINE affinity under the rq lock Mathieu Desnoyers
2026-11-19 16:21 ` [PATCH v4 15/17] sched/debug: Show the deadline bandwidth of each runqueue Mathieu Desnoyers
2026-11-19 16:21 ` [PATCH v4 16/17] selftests/sched: Add a test of SCHED_DEADLINE policy round trips Mathieu Desnoyers
2026-11-19 16:21 ` [PATCH v4 17/17] selftests/sched: Add a test of SCHED_DEADLINE bandwidth accounting Mathieu Desnoyers
2026-11-19 16:22 ` Mathieu Desnoyers [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=85b926c7-4f88-497d-8026-8cf1f1dce6a0@efficios.com \
    --to=mathieu.desnoyers@efficios.com \
    --cc=linux-kernel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®