[RFC] sched: update rto_mask for RT runqueue

[RFC] sched: update rto_mask for RT runqueue

[Hillf Danton]

Hi Steven

Since {de, en}queue_pushable_task() is not always accompanied with
{dec, inc}_rt_migration(), when the last pushable RT task on RQ is picked in
pick_next_task_rt(), we have to update rto_mask for the RQ in question, which
helps puller that is guided by rto_mask.

Thanks,

Hillf
---
 kernel/sched_rt.c |   21 ++++++++++++++++++---
 1 files changed, 18 insertions(+), 3 deletions(-)

diff --git a/kernel/sched_rt.c b/kernel/sched_rt.c
index b03cd89..4daf0ae 100644
--- a/kernel/sched_rt.c
+++ b/kernel/sched_rt.c
@@ -96,6 +96,15 @@ static void update_rt_migration(struct rt_rq *rt_rq)
 	}
 }

+static void update_rt_migratory(struct rt_rq *rt_rq, int inc)
+{
+	if (inc)
+		rt_rq->rt_nr_migratory++;
+	else
+		rt_rq->rt_nr_migratory--;
+	update_rt_migration(rt_rq);
+}
+
 static void inc_rt_migration(struct sched_rt_entity *rt_se, struct
rt_rq *rt_rq)
 {
 	if (!rt_entity_is_task(rt_se))
@@ -1148,11 +1157,13 @@ static struct task_struct
*pick_next_task_rt(struct rq *rq)
 {
 	struct task_struct *p = _pick_next_task_rt(rq);

+#ifdef CONFIG_SMP
 	/* The running task is never eligible for pushing */
-	if (p)
+	if (p && p->rt.nr_cpus_allowed > 1) {
 		dequeue_pushable_task(rq, p);
+		update_rt_migratory(&rq->rt, 0);
+	}

-#ifdef CONFIG_SMP
 	/*
 	 * We detect this state here so that we can avoid taking the RQ
 	 * lock again later if there is no need to push
@@ -1168,12 +1179,16 @@ static void put_prev_task_rt(struct rq *rq,
struct task_struct *p)
 	update_curr_rt(rq);
 	p->se.exec_start = 0;

+#ifdef CONFIG_SMP
 	/*
 	 * The previous task needs to be made eligible for pushing
 	 * if it is still active
 	 */
-	if (on_rt_rq(&p->rt) && p->rt.nr_cpus_allowed > 1)
+	if (on_rt_rq(&p->rt) && p->rt.nr_cpus_allowed > 1) {
 		enqueue_pushable_task(rq, p);
+		update_rt_migratory(&rq->rt, 1);
+	}
+#endif
 }

 #ifdef CONFIG_SMP

Re: [RFC] sched: update rto_mask for RT runqueue

[Steven Rostedt]

On Thu, 2011-06-23 at 21:43 +0800, Hillf Danton wrote:
> Hi Steven
> 
> Since {de, en}queue_pushable_task() is not always accompanied with
> {dec, inc}_rt_migration(), when the last pushable RT task on RQ is picked in
> pick_next_task_rt(), we have to update rto_mask for the RQ in question, which
> helps puller that is guided by rto_mask.

Hmm, I don't care much for this patch, but you did bring up something
that is interesting. That there may be a disconnect between the rto_mask
and the pushable tasks. A pushable task is the same as a pullable task.
So I'm thinking about revamping this to merge the two and make the code
simpler.

This may take a lot of testing to make sure that we did not create any
regressions.

-- Steve