Re: [PATCH] sched/fair: Change mode of period_timer to HRTIMER_MODE_ABS_PINNED_HARD

Re: [PATCH] sched/fair: Change mode of period_timer to HRTIMER_MODE_ABS_PINNED_HARD

[Peter Zijlstra]

On Thu, Oct 20, 2022 at 05:17:46PM +0800, Ruipeng Qi wrote:
> Previous mode of period_timer is HRTIMER_MODE_ABS_PINNED,which means
> timer callback function will be executed in soft irq context.
> 
> When one task group runs out of quota in a period,but holding lock of
> softirq_ctrl,the task will be rescheduled later,and then a system stall
> occurs:
>  - next period comes,
>  - __do_softirq can't acquire lock of softirq_ctrl,
>  - __do_softirq will not be invoked,
>  - callback of period_timer will not be invoked,
>  - task group will not get any quota in any new period.
>  - a system stall occurs.
> 
> Changing mode of period_timer to HRTIMER_MODE_ABS_PINNED_HARD,which mean
> timer callback function executed in hard irq context fix this problem.

What I'm missing here is a statement about how the runtime complexity of
that timer function is sufficiently bounded to run in hardirq context on
RT kernels.

Re: [PATCH] sched/fair: Change mode of period_timer to HRTIMER_MODE_ABS_PINNED_HARD

[Valentin Schneider]

On 20/10/22 17:17, Ruipeng Qi wrote:
> Previous mode of period_timer is HRTIMER_MODE_ABS_PINNED,which means
> timer callback function will be executed in soft irq context.
>
> When one task group runs out of quota in a period,but holding lock of
> softirq_ctrl,the task will be rescheduled later,and then a system stall
> occurs:
>  - next period comes,
>  - __do_softirq can't acquire lock of softirq_ctrl,
>  - __do_softirq will not be invoked,
>  - callback of period_timer will not be invoked,
>  - task group will not get any quota in any new period.
>  - a system stall occurs.
>
> Changing mode of period_timer to HRTIMER_MODE_ABS_PINNED_HARD,which mean
> timer callback function executed in hard irq context fix this problem.
>

What you want here is some form of PI between the task doing the timer
soft expiry and the one holding the softirq_ctrl.lock. This doesn't happen
ATM because of should_wake_ksoftirqd() - if the lock is held then we won't
wake ksoftirqd, so we won't get a PI.

If you look at the RT stack, this commit gives you that:
c1d68f0da449 ("softirq: Use a dedicated thread for timer wakeups.")
(from linux-6.1.y-rt-rebase)