[Question] sched: migrate RT task on throttled leaf RT runqueue?

[Question] sched: migrate RT task on throttled leaf RT runqueue?

[Hillf Danton]

Hi Steven

When a non-boosted RT task is enqueued, if it is attached to task group, as
shown in the following diff format for clarity, and even if the RT runqueue
owned by the task group is throttled, the rt_rq is loaded up as leaf and the
task is added to pushable list if it is pushable.

Since RT tasks on throttled rt_runqueue will not be picked to be the next task
when scheduling, could they be migrated by pushing and pulling?

Thanks,

Hillf
---
 kernel/sched_rt.c |    9 +++++++++
 1 files changed, 9 insertions(+), 0 deletions(-)

diff --git a/kernel/sched_rt.c b/kernel/sched_rt.c
index 97540f0..8489953 100644
--- a/kernel/sched_rt.c
+++ b/kernel/sched_rt.c
@@ -882,6 +882,9 @@ static void __enqueue_rt_entity(struct
sched_rt_entity *rt_se, bool head)
 	if (group_rq && (rt_rq_throttled(group_rq) || !group_rq->rt_nr_running))
 		return;

+	/* if (rt_rq_throttled(rt_rq) && !rt_se_boosted(rt_se))
+		return; */
+
 	if (!rt_rq->rt_nr_running)
 		list_add_leaf_rt_rq(rt_rq);

@@ -1418,6 +1421,9 @@ retry:
 		return 0;
 	}

+	if (rt_rq_throttled(rt_rq_of_se(&next_task->rt)))
+		return 0;
+
 	/* We might release rq lock */
 	get_task_struct(next_task);

@@ -1519,6 +1525,9 @@ static int pull_rt_task(struct rq *this_rq)

 		p = pick_next_highest_task_rt(src_rq, this_cpu);

+		if (p && rt_rq_throttled(rt_rq_of_se(&p->rt)))
+			goto skip;
+
 		/*
 		 * Do we have an RT task that preempts
 		 * the to-be-scheduled task?

Re: [Question] sched: migrate RT task on throttled leaf RT runqueue?

[Steven Rostedt]

On Thu, 2011-07-14 at 21:43 +0800, Hillf Danton wrote:
> Hi Steven
> 
> When a non-boosted RT task is enqueued, if it is attached to task group, as
> shown in the following diff format for clarity, and even if the RT runqueue
> owned by the task group is throttled, the rt_rq is loaded up as leaf and the
> task is added to pushable list if it is pushable.
> 
> Since RT tasks on throttled rt_runqueue will not be picked to be the next task
> when scheduling, could they be migrated by pushing and pulling?

Note, run queues are not throttled, groups are. And tasks on a throttled
group should not be touched by the push/pull algorithm.

-- Steve

Re: [Question] sched: migrate RT task on throttled leaf RT runqueue?

[Hillf Danton]

On Thu, Jul 14, 2011 at 10:09 PM, Steven Rostedt <rostedt@goodmis.org> wrote:
> On Thu, 2011-07-14 at 21:43 +0800, Hillf Danton wrote:
>> Hi Steven
>>
>> When a non-boosted RT task is enqueued, if it is attached to task group, as
>> shown in the following diff format for clarity, and even if the RT runqueue
>> owned by the task group is throttled, the rt_rq is loaded up as leaf and the
>> task is added to pushable list if it is pushable.
>>
>> Since RT tasks on throttled rt_runqueue will not be picked to be the next task
>> when scheduling, could they be migrated by pushing and pulling?
>
> Note, run queues are not throttled, groups are. And tasks on a throttled
> group should not be touched by the push/pull algorithm.
>

Task group is one of the hard to understand in RT scheduler, according to the
following two functions, if task group is throttled, could say its ->rt_rq[cpu]
also throttled, where cpu could be anyone of the active_cpu_mask?

And how to relate rt_rq_throttled(p->rt.rt_rq) to throttled task group?

Thanks,
	Hillf

/* Change a task's cfs_rq and parent entity if it moves across CPUs/groups */
static inline void set_task_rq(struct task_struct *p, unsigned int cpu)
{
#ifdef CONFIG_FAIR_GROUP_SCHED
	p->se.cfs_rq = task_group(p)->cfs_rq[cpu];
	p->se.parent = task_group(p)->se[cpu];
#endif

#ifdef CONFIG_RT_GROUP_SCHED
	p->rt.rt_rq  = task_group(p)->rt_rq[cpu];
	p->rt.parent = task_group(p)->rt_se[cpu];
#endif
}

static inline struct rt_rq *rt_rq_of_se(struct sched_rt_entity *rt_se)
{
	return rt_se->rt_rq;
}

Re: [Question] sched: migrate RT task on throttled leaf RT runqueue?

[Steven Rostedt]

On Thu, 2011-07-14 at 22:58 +0800, Hillf Danton wrote:
> On Thu, Jul 14, 2011 at 10:09 PM, Steven Rostedt <rostedt@goodmis.org> wrote:
> > On Thu, 2011-07-14 at 21:43 +0800, Hillf Danton wrote:
> >> Hi Steven
> >>
> >> When a non-boosted RT task is enqueued, if it is attached to task group, as
> >> shown in the following diff format for clarity, and even if the RT runqueue
> >> owned by the task group is throttled, the rt_rq is loaded up as leaf and the
> >> task is added to pushable list if it is pushable.
> >>
> >> Since RT tasks on throttled rt_runqueue will not be picked to be the next task
> >> when scheduling, could they be migrated by pushing and pulling?
> >
> > Note, run queues are not throttled, groups are. And tasks on a throttled
> > group should not be touched by the push/pull algorithm.
> >
> 
> Task group is one of the hard to understand in RT scheduler, according to the
> following two functions, if task group is throttled, could say its ->rt_rq[cpu]
> also throttled, where cpu could be anyone of the active_cpu_mask?
> 
> And how to relate rt_rq_throttled(p->rt.rt_rq) to throttled task group?

Best to ask Peter, as that is his work.

-- Steve

> 
> Thanks,
> 	Hillf
> 
> /* Change a task's cfs_rq and parent entity if it moves across CPUs/groups */
> static inline void set_task_rq(struct task_struct *p, unsigned int cpu)
> {
> #ifdef CONFIG_FAIR_GROUP_SCHED
> 	p->se.cfs_rq = task_group(p)->cfs_rq[cpu];
> 	p->se.parent = task_group(p)->se[cpu];
> #endif
> 
> #ifdef CONFIG_RT_GROUP_SCHED
> 	p->rt.rt_rq  = task_group(p)->rt_rq[cpu];
> 	p->rt.parent = task_group(p)->rt_se[cpu];
> #endif
> }
> 
> static inline struct rt_rq *rt_rq_of_se(struct sched_rt_entity *rt_se)
> {
> 	return rt_se->rt_rq;
> }