Re: [PATCH] sched/deadline: Fix DL server divide-by-zero for inactive CPUs

[Mikhail Zaslonko <zaslonko@linux.ibm.com>]

On 12-Aug-26 14:32, Hui Su wrote:
> Commit 4043f5498416 ("sched/deadline: Reject debugfs dl_server writes
> for offline CPUs") rejects per-CPU DL server parameter updates once the
> target CPU is offline. However, during CPU hot-unplug, the CPU is cleared
> from cpu_active_mask before it is marked offline.
> 
> This leaves a window where cpu_online() is still true while
> cpu_active() is already false. A debugfs write during this window passes
> the cpu_online() check in sched_server_write_common() and reaches
> dl_server_apply_params() with init=false.
> 
> dl_bw_cpus() counts the active CPUs in the root domain. For an isolated
> CPU whose root-domain span contains only that CPU, it returns zero once
> the CPU becomes inactive. If the server bandwidth is attached,
> dl_server_apply_params() then passes this zero CPU count to __dl_sub()
> and __dl_add(), both of which divide by the CPU count.

Hello Hui, Juri

We hit the same divide-by-zero on s390x, but from the sched_setscheduler()
syscall path rather than debugfs:

[  836.069103] fixpoint divide exception: 0009 ilc:2 [#1]SMP
[  836.069114] Modules linked in: algif_hash af_alg ctcm fsm zfcp scsi_transport_fc mlx5_ib ib_uverbs_support ib_core mlx5_vdpa vdpa vringh vhost_iotlb nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nf_tables mlx5_core s390_trng ism eadm_sch vfio_ccw mdev vfio_iommu_type1 vfio sch_fq_codel drm i2c_core drm_panel_orientation_quirks diag288_wdt hmac_s390 prng aes_s390 dm_mirror dm_region_hash dm_log pkey_ep11 pkey_cca zcrypt phmac_s390 paes_s390 rng_core pkey_pckmo pkey crypto_engine uvdevice autofs4 ecdsa_generic ecc sha512
[  836.069173] CPU: 19 UID: 0 PID: 1005719 Comm: stress-ng-cpu-s Kdump: loaded Tainted: G        W           7.3.0-20260917.rc3.git3.a10f6da4ba31.300.fc44.s390x #1 PREEMPTLAZY
[  836.069178] Tainted: [W]=WARN
[  836.069180] Hardware name: IBM 9175 ME1 705 (LPAR)
[  836.069182] Krnl PSW : 0404c00180000000 001925217a23136e (task_non_contending+0x19e/0x370)
[  836.069191]            R:0 T:1 IO:0 EX:0 Key:0 M:1 W:0 P:0 AS:3 CC:0 PM:0 RI:0 EA:3
[  836.069194] Krnl GPRS: 0000000000000000 0000000000000000 0000000000000000 00000000ffffffff
[  836.069197]            ffffffffffffffff 0000000000006666 001925217be2d038 001925217a237950
[  836.069199]            0000000000000014 0000000000006666 001925217c3785e8 000003f15af10330
[  836.069201]            0000000000000000 001925217b284f40 001925217a231366 0019249f1e61fbc0
[  836.069210] Krnl Code: 001925217a231360: c0e5ffffdff8        brasl   %r14,001925217a22d350
           001925217a231366: b9140059           lgfr    %r5,%r9
          *001925217a23136a: b90d0042           dsgr    %r4,%r2
          >001925217a23136e: e330f0a80004       lg      %r3,168(%r15)
           001925217a231374: c0a000dfe7da       larl    %r10,001925217be2e328
           001925217a23137a: e32030f80004       lg      %r2,248(%r3)
           001925217a231380: b9090029           sgr     %r2,%r9
           001925217a231384: 41903018           la      %r9,24(%r3)
[  836.069243] Call Trace:
[  836.069245]  [<001925217a23136e>] task_non_contending+0x19e/0x370
[  836.069249] ([<001925217a231366>] task_non_contending+0x196/0x370)
[  836.069252]  [<001925217a231678>] switched_from_dl+0x138/0x190
[  836.069256]  [<001925217a207f56>] sched_change_begin+0x116/0x330
[  836.069260]  [<001925217a23b0c6>] __sched_setscheduler+0x1e6/0xad0
[  836.069263]  [<001925217a23baba>] sched_setscheduler+0x7a/0xb0
[  836.069266]  [<001925217a23bb66>] do_sched_setscheduler+0x76/0x130
[  836.069268]  [<001925217a23be7c>] __s390x_sys_sched_setscheduler+0x3c/0x60
[  836.069271]  [<001925217b125850>] __do_syscall+0x1a0/0x4c0
[  836.069274]  [<001925217b135b42>] system_call+0x72/0x90
[  836.069277] Last Breaking-Event-Address:
[  836.069278]  [<001925217a22d3ac>] dl_bw_cpus+0x5c/0x70
[  836.069283] Kernel panic - not syncing: Fatal exception: panic_on_oops

In this case dl_bw_cpus(task_cpu(p)) returned 0 and task_non_contending() passed it to
__dl_sub().

CPU offlining was running concurrently. 11 seconds before the panic:

  [  824.924635] select_fallback_rq: 11 callbacks suppressed
  [  824.924640] process 1005879 (stress-ng-cpu-s) no longer affine to cpu118
  [  824.944307] process 1005883 (stress-ng-cpu-s) no longer affine to cpu122

There was also a preceding WARN_ON_ONCE(!cpu_online(new_cpu)) in set_task_cpu(),
reached from dl_task_timer(). That looks like a separate issue.

I have no exact reproducer at the moment, this is a CI run with several workloads in
parallel. But we can test a candidate patch in the same environment.


As for the patch, the same division is unguarded in several other callers
in deadline.c:

  task_non_contending()     (this trace)
  inactive_task_timer()
  set_cpus_allowed_dl()
  sched_dl_overflow()
  dl_bw_manage()
  
so rejecting inactive CPUs in dl_server_apply_params() closes only one of
them.

Would it make sense to guard inside __dl_sub() and __dl_add() instead?

	void __dl_sub(struct dl_bw *dl_b, u64 tsk_bw, int cpus)
	{
		dl_b->total_bw -= tsk_bw;
		if (cpus)
			__dl_update(dl_b, (s32)tsk_bw / cpus);
	}
	
	void __dl_add(struct dl_bw *dl_b, u64 tsk_bw, int cpus)
	{
		dl_b->total_bw += tsk_bw;
		if (cpus)
			__dl_update(dl_b, -((s32)tsk_bw / cpus));
	}

__dl_update() walks for_each_cpu_and(i, rd->span, cpu_active_mask) over the
same root domain, so when cpus == 0 the loop body never runs anyway.
Skipping the division changes nothing else. Yes, total_bw is still updated
on a root domain with no active CPUs, but it turns a kernel panic into an
accounting inaccuracy that already exists today.

What do you think?

Thanks,
Mikhail