Re: [RFC PATCH v2 0/7] Defer throttle when task exits to user

[Aaron Lu <ziqianlu@bytedance.com>]

Hi Prateek,

On Mon, Apr 14, 2025 at 10:04:02PM +0530, K Prateek Nayak wrote:
> Hello Aaron,
> 
> On 4/9/2025 5:37 PM, Aaron Lu wrote:
> > This is a continuous work based on Valentin Schneider's posting here:
> > Subject: [RFC PATCH v3 00/10] sched/fair: Defer CFS throttle to user entry
> > https://lore.kernel.org/lkml/20240711130004.2157737-1-vschneid@redhat.com/
> > 
> > Valentin has described the problem very well in the above link. We also
> > have task hung problem from time to time in our environment due to cfs quota.
> > It is mostly visible with rwsem: when a reader is throttled, writer comes in
> > and has to wait, the writer also makes all subsequent readers wait,
> > causing problems of priority inversion or even whole system hung.
> > 
> > To improve this situation, change the throttle model to task based, i.e.
> > when a cfs_rq is throttled, mark its throttled status but do not
> > remove it from cpu's rq. Instead, for tasks that belong to this cfs_rq,
> > when they get picked, add a task work to them so that when they return
> > to user, they can be dequeued. In this way, tasks throttled will not
> > hold any kernel resources. When cfs_rq gets unthrottled, enqueue back
> > those throttled tasks.
> 
> I tried to reproduce the scenario that Valentin describes in the
> parallel thread [1] and I haven't run into a stall yet with this
> series applied on top of v6.15-rc1 [2].

Great to hear this.

> So for Patch 1-6, feel free to add:
> 
> Tested-by: K Prateek Nayak <kprateek.nayak@amd.com>

Thanks!

> I'm slowly crawling through the series and haven't gotten to the Patch 7
> yet but so far I haven't seen anything unexpected in my initial testing
> and it seems to solve possible circular dependency on PREEMPT_RT with
> bandwidth replenishment (Sebastian has some doubts if my reproducer is
> correct but that discussion is for the other thread)
> 
> Thank you for working on this and a big thanks to Valentin for the solid
> groundwork.

Thank you a ton for your review and test.

> [1] https://lore.kernel.org/linux-rt-users/f2e2c74c-b15d-4185-a6ea-4a19eee02417@amd.com/
> [2] https://lore.kernel.org/linux-rt-users/534df953-3cfb-4b3d-8953-5ed9ef24eabc@amd.com/

Best regards,
Aaron