[PATCH 2/2] sched_ext: scx_qmap: Queue re-enqueued lowpri tasks on their FIFO

[Tejun Heo <tj@kernel.org>]

scx_qmap holds nice 19 tasks on LOWPRI_DSQ, and lowpri_timerfn() re-enqueues
them every 10ms. The re-enqueued tasks take qmap_enqueue()'s SCX_ENQ_REENQ
path to SHARED_DSQ, which qmap_dispatch() consumes before popping the FIFOs,
so enough nice 19 tasks starve every other task:

1. A nice 19 task's slice expires and qmap_enqueue() puts it on LOWPRI_DSQ.
2. Within 10ms, lowpri_timerfn() re-enqueues it onto SHARED_DSQ.
3. qmap_dispatch() finds SHARED_DSQ non-empty and doesn't pop the FIFOs.

With a dozen nice 19 CPU hogs, no FIFO was popped for over 5 seconds, and
the root was ejected as the watchdog work, whose kworker waited on a FIFO,
failed to check in.

Queue a re-enqueued lowpri task on its FIFO instead, where it is dispatched
in its FIFO's turn after its LOWPRI_DSQ wait.

Fixes: 84b1a0ea0b7c ("sched_ext: Implement scx_bpf_dsq_reenq() for user DSQs")
Signed-off-by: Tejun Heo <tj@kernel.org>
---
 tools/sched_ext/scx_qmap.bpf.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/tools/sched_ext/scx_qmap.bpf.c b/tools/sched_ext/scx_qmap.bpf.c
index 77f02fb70a86..3951f1f6e3c6 100644
--- a/tools/sched_ext/scx_qmap.bpf.c
+++ b/tools/sched_ext/scx_qmap.bpf.c
@@ -427,6 +427,7 @@ void BPF_STRUCT_OPS(qmap_enqueue, struct task_struct *p, u64 enq_flags)
 	static u32 user_cnt, kernel_cnt;
 	task_ctx_t *taskc;
 	int idx = weight_to_idx(p->scx.weight);
+	bool lowpri = p->scx.weight < 2 && !(p->flags & PF_KTHREAD);
 	s32 cid;
 
 	if (enq_flags & SCX_ENQ_REENQ) {
@@ -587,7 +588,7 @@ void BPF_STRUCT_OPS(qmap_enqueue, struct task_struct *p, u64 enq_flags)
 	}
 
 	/* see lowpri_timerfn() */
-	if (p->scx.weight < 2 && !(p->flags & PF_KTHREAD) && !(enq_flags & SCX_ENQ_REENQ)) {
+	if (lowpri && !(enq_flags & SCX_ENQ_REENQ)) {
 		scx_bpf_dsq_insert(p, LOWPRI_DSQ, slice_ns, enq_flags);
 		return;
 	}
@@ -606,8 +607,12 @@ void BPF_STRUCT_OPS(qmap_enqueue, struct task_struct *p, u64 enq_flags)
 	 * higher priority scheduling class, just re-enqueue the task directly
 	 * on the global DSQ. As we want another CPU to pick it up, find and
 	 * kick an idle cid.
+	 *
+	 * A lowpri task goes to its FIFO instead. qmap_dispatch() consumes
+	 * SHARED_DSQ ahead of the FIFOs, and enough lowpri tasks coming back
+	 * from LOWPRI_DSQ would keep it non-empty and starve every other task.
 	 */
-	if (enq_flags & SCX_ENQ_REENQ) {
+	if ((enq_flags & SCX_ENQ_REENQ) && !lowpri) {
 		s32 cid;
 
 		scx_bpf_dsq_insert(p, SHARED_DSQ, 0, enq_flags);
@@ -1357,7 +1362,7 @@ struct {
 
 /*
  * Nice 19 tasks are put into the lowpri DSQ. Every 10ms, reenq is triggered and
- * the tasks are transferred to SHARED_DSQ.
+ * qmap_enqueue() queues the tasks on their FIFO.
  */
 static int lowpri_timerfn(void *map, int *key, struct bpf_timer *timer)
 {
-- 
2.56.0