* [PATCH] Fix to excess pre-schedule migrating during Real Time overload on multiple CPUs.
@ 2011-07-29 0:56 uberj
2011-07-29 12:01 ` Steven Rostedt
0 siblings, 1 reply; 2+ messages in thread
From: uberj @ 2011-07-29 0:56 UTC (permalink / raw)
To: Ingo Molnar, Peter Zijlstra; +Cc: linux-kernel
From cf87d70969357f46e5c28804ac39c2139652af8f Mon Sep 17 00:00:00 2001
From: Jacques Uber <uberj@onid.orst.edu>
Date: Thu, 28 Jul 2011 17:49:38 -0700
Subject: [PATCH] Fix to excess pre-schedule migrating during Real Time
overload on multiple CPUs.
If multiple processors are under real time overload there are situations where
a processor will pull multiple tasks from multiple CPU's while trying to
schedule the highest priority task within it's root domain. As an example, if
there are four CPU's all under overload and the highest priority task on CPU3
yields its time slice, CPU3 will try to pull the next highest priority
task onto
its the runqueue. Without the patch, if tasks were ordered in ascending order
on CPU[0-2], CPU3 would pull all the tasks, ruining any possibility of using
cached data.
The simple solution to this scenario is to maintain a "best choice"
for CPU3 to
pull to it's run queue. Only after checking all the queued tasks and choosing
which one should be ran, will it deactivate, set_task, and reactivate the task
onto it's runqueue.
This patch was inspired by a Linux Journal article about the Real Time
Scheduler.
http://www.linuxjournal.com/magazine/real-time-linux-kernel-scheduler?page=0,3
Signed-off-by: Jacques Uber <uberj@onid.orst.edu>
Signed-off-by: Kevin Strasser <strassek@onid.orst.edu>
---
kernel/sched_rt.c | 11 ++++++++---
1 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/kernel/sched_rt.c b/kernel/sched_rt.c
index 97540f0..067f159 100644
--- a/kernel/sched_rt.c
+++ b/kernel/sched_rt.c
@@ -1482,6 +1482,7 @@ static int pull_rt_task(struct rq *this_rq)
{
int this_cpu = this_rq->cpu, ret = 0, cpu;
struct task_struct *p;
+ struct task_struct *best = NULL;
struct rq *src_rq;
if (likely(!rt_overloaded(this_rq)))
@@ -1540,9 +1541,10 @@ static int pull_rt_task(struct rq *this_rq)
ret = 1;
- deactivate_task(src_rq, p, 0);
- set_task_cpu(p, this_cpu);
- activate_task(this_rq, p, 0);
+ if(!best)
+ best = p;
+ else if(p->prio > best->prio)
+ best = p;
/*
* We continue with the search, just in
* case there's an even higher prio task
@@ -1550,6 +1552,9 @@ static int pull_rt_task(struct rq *this_rq)
* but possible)
*/
}
+ deactivate_task(src_rq, p, 0);
+ set_task_cpu(p, this_cpu);
+ activate_task(this_rq, p, 0);
skip:
double_unlock_balance(this_rq, src_rq);
}
--
1.7.5.4
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: [PATCH] Fix to excess pre-schedule migrating during Real Time overload on multiple CPUs.
2011-07-29 0:56 [PATCH] Fix to excess pre-schedule migrating during Real Time overload on multiple CPUs uberj
@ 2011-07-29 12:01 ` Steven Rostedt
0 siblings, 0 replies; 2+ messages in thread
From: Steven Rostedt @ 2011-07-29 12:01 UTC (permalink / raw)
To: uberj; +Cc: Ingo Molnar, Peter Zijlstra, linux-kernel
On Fri, 2011-07-29 at 07:30 +0200, uberj@onid.orst.edu wrote:
> From cf87d70969357f46e5c28804ac39c2139652af8f Mon Sep 17 00:00:00 2001
> From: Jacques Uber <uberj@onid.orst.edu>
> Date: Thu, 28 Jul 2011 17:49:38 -0700
> Subject: [PATCH] Fix to excess pre-schedule migrating during Real Time
> overload on multiple CPUs.
>
> If multiple processors are under real time overload there are situations where
> a processor will pull multiple tasks from multiple CPU's while trying to
> schedule the highest priority task within it's root domain. As an example, if
> there are four CPU's all under overload and the highest priority task on CPU3
> yields its time slice, CPU3 will try to pull the next highest priority
> task onto
> its the runqueue. Without the patch, if tasks were ordered in ascending order
> on CPU[0-2], CPU3 would pull all the tasks, ruining any possibility of using
> cached data.
As I stated on IRC, this is a theoretical situation, as real RT systems
would bind important RT tasks to CPUs, and those non-RT systems that
happen to run a bunch of RT tasks, usually have those RT tasks set up at
the same priority. If this situation were to happen, it would seldom
happen. It is likely that a bunch of overloaded RT tasks, would bounce
anyway regardless how you do this.
>
> The simple solution to this scenario is to maintain a "best choice"
> for CPU3 to
> pull to it's run queue. Only after checking all the queued tasks and choosing
> which one should be ran, will it deactivate, set_task, and reactivate the task
> onto it's runqueue.
There's a major bug in this solution. You have to release the rq locks,
which means that once you found your "best" task, it may no longer be
available to pull.
Also, your patch is broken because it tests for the pull inside the if
statement, and it looks like you just pulled all tasks.
>
> This patch was inspired by a Linux Journal article about the Real Time
> Scheduler.
> http://www.linuxjournal.com/magazine/real-time-linux-kernel-scheduler?page=0,3
>
> Signed-off-by: Jacques Uber <uberj@onid.orst.edu>
> Signed-off-by: Kevin Strasser <strassek@onid.orst.edu>
> ---
> kernel/sched_rt.c | 11 ++++++++---
> 1 files changed, 8 insertions(+), 3 deletions(-)
>
> diff --git a/kernel/sched_rt.c b/kernel/sched_rt.c
> index 97540f0..067f159 100644
> --- a/kernel/sched_rt.c
> +++ b/kernel/sched_rt.c
> @@ -1482,6 +1482,7 @@ static int pull_rt_task(struct rq *this_rq)
> {
> int this_cpu = this_rq->cpu, ret = 0, cpu;
> struct task_struct *p;
> + struct task_struct *best = NULL;
broken whitespace throughout.
> struct rq *src_rq;
>
> if (likely(!rt_overloaded(this_rq)))
> @@ -1540,9 +1541,10 @@ static int pull_rt_task(struct rq *this_rq)
>
> ret = 1;
>
> - deactivate_task(src_rq, p, 0);
> - set_task_cpu(p, this_cpu);
> - activate_task(this_rq, p, 0);
> + if(!best)
> + best = p;
> + else if(p->prio > best->prio)
> + best = p;
> /*
> * We continue with the search, just in
> * case there's an even higher prio task
> @@ -1550,6 +1552,9 @@ static int pull_rt_task(struct rq *this_rq)
> * but possible)
> */
> }
> + deactivate_task(src_rq, p, 0);
> + set_task_cpu(p, this_cpu);
> + activate_task(this_rq, p, 0);
This is still inside the loop. You just pulled p without checking
anything. 'best' is not used at all. You just made things worse.
Before touching the scheduler, always have a test case that shows the
problem, and can show that a patch solves that problem. The scheduler is
too critical to the entire system to just apply random patches.
-- Steve
> skip:
> double_unlock_balance(this_rq, src_rq);
> }
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2011-07-29 12:01 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-07-29 0:56 [PATCH] Fix to excess pre-schedule migrating during Real Time overload on multiple CPUs uberj
2011-07-29 12:01 ` Steven Rostedt
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®