[PATCH] sched: Restore the normalize_rt_tasks() cpuset_mutex exemption

[PATCH] sched: Restore the normalize_rt_tasks() cpuset_mutex exemption

[Donggeun Yoo]

sysrq-n runs normalize_rt_tasks(), which walks the task list under
read_lock(&tasklist_lock) and hands every user RT or deadline task to
__sched_setscheduler() with pi == false.  __sched_setscheduler() takes
cpuset_mutex whenever the old or the new policy is deadline, and
cpuset_lock() is a plain mutex_lock(), so a single SCHED_DEADLINE user
task makes the sysrq handler sleep in atomic context:

  sysrq: Nice All RT Tasks
  BUG: sleeping function called from invalid context at kernel/locking/mutex.c:623
  in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 1, name: init
  preempt_count: 1, expected: 0
  RCU nest depth: 1, expected: 0
  locks held by init/1: 3, last CPU#1:
   #0: ffff9fa2c34e1470 (sb_writers#3){.+.+}-{0:0}, at: ksys_write+0x74/0xf0
   #1: ffffffffb8f66540 (rcu_read_lock){....}-{1:3}, at: __handle_sysrq+0x3a/0x110
   #2: ffffffffb8e060d8 (tasklist_lock){.+.+}-{3:3}, at: normalize_rt_tasks+0x34/0x150
  Call Trace:
   __might_resched.cold+0xe3/0xf5
   __mutex_lock+0x8a/0x11c0
   __sched_setscheduler+0x35f/0x9c0
   normalize_rt_tasks+0xe6/0x150
   __handle_sysrq.cold+0x9b/0xde
   write_sysrq_trigger+0x65/0x90

Test pi along with the policy. The only pi == false caller is
normalize_rt_tasks(), where the lock is given up since the sysrq
emergency already voids deadline guarantees.

Fixes: 111cd11bbc54 ("sched/cpuset: Bring back cpuset_mutex")
Signed-off-by: Donggeun Yoo <donggeunyoo.kernel@gmail.com>
Cc: stable@vger.kernel.org
---
Reproduced under qemu-system-x86_64, -smp 2, on tip/sched/core
e81ee0630837 with x86_64_defconfig plus x86_debug.config less
GCOV_KERNEL, plus CPUSETS, MAGIC_SYSRQ, DEBUG_ATOMIC_SLEEP,
DEBUG_MUTEXES and PROVE_RAW_LOCK_NESTING.

  deadline tasks      base                      patched
  0, one RT task      no splat, RT normalized   same
  1 blocked           splat, normalized         no splat, normalized
  2 blocked, one RT   splat, sysrq-n hangs      no splat, all normalized
  1 blocked, x3       splat                     no splat
  1 spinning          splat, normalized         no splat, normalized
  2 spinning, one RT  splat, sysrq-n hangs      no splat, all normalized

 kernel/sched/syscalls.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c
index b215b0ead9a60..1d931ecfaad88 100644
--- a/kernel/sched/syscalls.c
+++ b/kernel/sched/syscalls.c
@@ -553,10 +553,10 @@ int __sched_setscheduler(struct task_struct *p,
 	}
 
 	/*
-	 * SCHED_DEADLINE bandwidth accounting relies on stable cpusets
-	 * information.
+	 * SCHED_DEADLINE needs stable cpusets.  However, the pi == false
+	 * caller normalize_rt_tasks() must skip the lock to avoid sleeping.
 	 */
-	if (dl_policy(policy) || dl_policy(p->policy)) {
+	if (pi && (dl_policy(policy) || dl_policy(p->policy))) {
 		cpuset_locked = true;
 		cpuset_lock();
 	}
-- 
2.53.0

Re: [PATCH] sched: Restore the normalize_rt_tasks() cpuset_mutex exemption

[Juri Lelli]

Hello,

On 22/09/26 12:07, Donggeun Yoo wrote:
> sysrq-n runs normalize_rt_tasks(), which walks the task list under
> read_lock(&tasklist_lock) and hands every user RT or deadline task to
> __sched_setscheduler() with pi == false.  __sched_setscheduler() takes
> cpuset_mutex whenever the old or the new policy is deadline, and
> cpuset_lock() is a plain mutex_lock(), so a single SCHED_DEADLINE user
> task makes the sysrq handler sleep in atomic context:
> 
>   sysrq: Nice All RT Tasks
>   BUG: sleeping function called from invalid context at kernel/locking/mutex.c:623
>   in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 1, name: init
>   preempt_count: 1, expected: 0
>   RCU nest depth: 1, expected: 0
>   locks held by init/1: 3, last CPU#1:
>    #0: ffff9fa2c34e1470 (sb_writers#3){.+.+}-{0:0}, at: ksys_write+0x74/0xf0
>    #1: ffffffffb8f66540 (rcu_read_lock){....}-{1:3}, at: __handle_sysrq+0x3a/0x110
>    #2: ffffffffb8e060d8 (tasklist_lock){.+.+}-{3:3}, at: normalize_rt_tasks+0x34/0x150
>   Call Trace:
>    __might_resched.cold+0xe3/0xf5
>    __mutex_lock+0x8a/0x11c0
>    __sched_setscheduler+0x35f/0x9c0
>    normalize_rt_tasks+0xe6/0x150
>    __handle_sysrq.cold+0x9b/0xde
>    write_sysrq_trigger+0x65/0x90
> 
> Test pi along with the policy. The only pi == false caller is
> normalize_rt_tasks(), where the lock is given up since the sysrq
> emergency already voids deadline guarantees.
> 
> Fixes: 111cd11bbc54 ("sched/cpuset: Bring back cpuset_mutex")
> Signed-off-by: Donggeun Yoo <donggeunyoo.kernel@gmail.com>
> Cc: stable@vger.kernel.org
> ---

Ah, yes indeed.

Acked-by: Juri Lelli <juri.lelli@redhat.com>

Thanks!
Juri