mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* [PATCH 0/4] RT: RT-Overload/Sched v3
@ 2007-10-16 16:08 Gregory Haskins
  2007-10-16 16:08 ` [PATCH 1/4] RT: Wrap the RQ notion of priority to make it conditional Gregory Haskins
                   ` (3 more replies)
  0 siblings, 4 replies; 5+ messages in thread
From: Gregory Haskins @ 2007-10-16 16:08 UTC (permalink / raw)
  To: Steven Rostedt, Peter Zijlstra; +Cc: RT, Ingo Molnar, LKML, Gregory Haskins

v3 contains the following changes since v2:

This is still based on 23-rt1 + Steve's last public patch (I think Steve has a
newer version available, but we have not rebased yet).

1) No longer includes the per-cpu-rtoverload patch, since Steve has already
   ACKed it
2) Dropped the affinity patch because it was buggy.  Will revisit later
3) Dropped the pull-patch, since it was RFC only anyway.  Will revisit later
4) Addressed review comments/feedback (added patch descriptions across the
   board, moved semi-colon in while loop, removed superfluous
   smp_processor_id() etc.
5) Got rid of the second TASK_PREEMPTS_CURR() check, since my anaylsys
   determined it was not needed.

With this patch series applied, my RT test now passes and we get an average of
23us wake-up latency under load. You can find the test here: 

 ftp://ftp.novell.com/dev/ghaskins/preempt.cc

Note: Do not use the "affinity" option, as the validator has a bug related to
it and may yield a false negative.

Regards,
-Greg


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH 1/4] RT: Wrap the RQ notion of priority to make it conditional
  2007-10-16 16:08 [PATCH 0/4] RT: RT-Overload/Sched v3 Gregory Haskins
@ 2007-10-16 16:08 ` Gregory Haskins
  2007-10-16 16:09 ` [PATCH 2/4] RT: Initialize the priority value Gregory Haskins
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 5+ messages in thread
From: Gregory Haskins @ 2007-10-16 16:08 UTC (permalink / raw)
  To: Steven Rostedt, Peter Zijlstra; +Cc: RT, Ingo Molnar, LKML, Gregory Haskins

A little cleanup to avoid #ifdef proliferation later in the series

Signed-off-by: Gregory Haskins <ghaskins@novell.com>
---

 kernel/sched.c |   16 +++++++++++++---
 1 files changed, 13 insertions(+), 3 deletions(-)

diff --git a/kernel/sched.c b/kernel/sched.c
index 0a1ad0e..1866a6e 100644
--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -365,6 +365,16 @@ struct rq {
 static DEFINE_PER_CPU_SHARED_ALIGNED(struct rq, runqueues);
 static DEFINE_MUTEX(sched_hotcpu_mutex);
 
+#if defined(CONFIG_PREEMPT_RT) && defined(CONFIG_SMP)
+static inline void set_rq_prio(struct rq *rq, int prio)
+{
+	rq->curr_prio = prio;
+}
+
+#else
+#define set_rq_prio(rq, prio) do { } while(0)
+#endif
+
 static inline void check_preempt_curr(struct rq *rq, struct task_struct *p)
 {
 	rq->curr->sched_class->check_preempt_curr(rq, p);
@@ -2331,9 +2341,9 @@ static inline void finish_task_switch(struct rq *rq, struct task_struct *prev)
 	 */
 	prev_state = prev->state;
 	_finish_arch_switch(prev);
-#if defined(CONFIG_PREEMPT_RT) && defined(CONFIG_SMP)
-	rq->curr_prio = current->prio;
-#endif
+
+	set_rq_prio(rq, current->prio);
+
 	finish_lock_switch(rq, prev);
 #if defined(CONFIG_PREEMPT_RT) && defined(CONFIG_SMP)
 	/*


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH 2/4] RT: Initialize the priority value
  2007-10-16 16:08 [PATCH 0/4] RT: RT-Overload/Sched v3 Gregory Haskins
  2007-10-16 16:08 ` [PATCH 1/4] RT: Wrap the RQ notion of priority to make it conditional Gregory Haskins
@ 2007-10-16 16:09 ` Gregory Haskins
  2007-10-16 16:09 ` [PATCH 3/4] RT: Add support for updating push-rt priority under PI boost Gregory Haskins
  2007-10-16 16:09 ` [PATCH 4/4] RT: Add support for low-priority wake-up to push_rt feature Gregory Haskins
  3 siblings, 0 replies; 5+ messages in thread
From: Gregory Haskins @ 2007-10-16 16:09 UTC (permalink / raw)
  To: Steven Rostedt, Peter Zijlstra; +Cc: RT, Ingo Molnar, LKML, Gregory Haskins

We should init the base value of the current RQ priority to "IDLE"

Signed-off-by: Gregory Haskins <ghaskins@novell.com>
---

 kernel/sched.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/kernel/sched.c b/kernel/sched.c
index 1866a6e..b69e49b 100644
--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -7388,6 +7388,8 @@ void __init sched_init(void)
 		highest_cpu = i;
 		/* delimiter for bitsearch: */
 		__set_bit(MAX_RT_PRIO, array->bitmap);
+
+		set_rq_prio(rq, MAX_PRIO);
 	}
 
 	set_load_weight(&init_task);


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH 3/4] RT: Add support for updating push-rt priority under PI boost
  2007-10-16 16:08 [PATCH 0/4] RT: RT-Overload/Sched v3 Gregory Haskins
  2007-10-16 16:08 ` [PATCH 1/4] RT: Wrap the RQ notion of priority to make it conditional Gregory Haskins
  2007-10-16 16:09 ` [PATCH 2/4] RT: Initialize the priority value Gregory Haskins
@ 2007-10-16 16:09 ` Gregory Haskins
  2007-10-16 16:09 ` [PATCH 4/4] RT: Add support for low-priority wake-up to push_rt feature Gregory Haskins
  3 siblings, 0 replies; 5+ messages in thread
From: Gregory Haskins @ 2007-10-16 16:09 UTC (permalink / raw)
  To: Steven Rostedt, Peter Zijlstra; +Cc: RT, Ingo Molnar, LKML, Gregory Haskins

Priority of the running task can change at run-time due to
PI boosting / nice, so be sure to update the RQ version of
the priority as well.

Signed-off-by: Gregory Haskins <ghaskins@novell.com>
---

 kernel/sched.c |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/kernel/sched.c b/kernel/sched.c
index b69e49b..bc9f4a6 100644
--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -4656,6 +4656,7 @@ void rt_mutex_setprio(struct task_struct *p, int prio)
 		 * this runqueue and our priority is higher than the current's
 		 */
 		if (task_running(rq, p)) {
+			set_rq_prio(rq, p->prio);
 			if (p->prio > oldprio)
 				resched_task(rq->curr);
 		} else {
@@ -4715,6 +4716,9 @@ void set_user_nice(struct task_struct *p, long nice)
 		 */
 		if (delta < 0 || (delta > 0 && task_running(rq, p)))
 			resched_task(rq->curr);
+
+		if (delta && task_running(rq, p))
+			set_rq_prio(rq, p->prio);
 	}
 out_unlock:
 	task_rq_unlock(rq, &flags);


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH 4/4] RT: Add support for low-priority wake-up to push_rt feature
  2007-10-16 16:08 [PATCH 0/4] RT: RT-Overload/Sched v3 Gregory Haskins
                   ` (2 preceding siblings ...)
  2007-10-16 16:09 ` [PATCH 3/4] RT: Add support for updating push-rt priority under PI boost Gregory Haskins
@ 2007-10-16 16:09 ` Gregory Haskins
  3 siblings, 0 replies; 5+ messages in thread
From: Gregory Haskins @ 2007-10-16 16:09 UTC (permalink / raw)
  To: Steven Rostedt, Peter Zijlstra; +Cc: RT, Ingo Molnar, LKML, Gregory Haskins

There are three events that require consideration for redistributing RT
tasks:

1) When one or more higher-priority tasks preempts a lower-one from a
   RQ
2) When a lower-priority task is woken up on a RQ
3) When a RQ downgrades its current priority

Steve Rostedt's push_rt patch addresses (1).  It hooks in right after
a new task has been switched-in.  If this was the result of an RT
preemption, or if more than one task was awoken at the same time, we
can try to push some of those other tasks away.

This patch addresses (2).  When we wake up a task, we check to see
if it would preempt the current task on the queue.  If it will not, we
attempt to find a better suited CPU (e.g. one running something lower
priority than the task being woken) and try to activate the task there.

Finally, we have (3).  In theory, we only need to balance_rt_tasks() if
the following conditions are met:
   1) One or more CPUs are in overload, AND
   2) We are about to switch to a task that lowers our priority.

(3) will be addressed in a later patch.

Signed-off-by: Gregory Haskins <ghaskins@novell.com>
---

 kernel/sched.c |   96 ++++++++++++++++++++++++++++----------------------------
 1 files changed, 48 insertions(+), 48 deletions(-)

diff --git a/kernel/sched.c b/kernel/sched.c
index bc9f4a6..567b7ab 100644
--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -1621,6 +1621,13 @@ out:
 	return ret;
 }
 
+/* Push all tasks that we can to other CPUs */
+static void push_rt_tasks(struct rq *this_rq)
+{
+	while (push_rt_task(this_rq))
+		;
+}
+
 /*
  * Pull RT tasks from other CPUs in the RT-overload
  * case. Interrupts are disabled, local rq is locked.
@@ -1981,7 +1988,32 @@ out_set_cpu:
 		this_cpu = smp_processor_id();
 		cpu = task_cpu(p);
 	}
-
+	
+	/*
+	 * If a newly woken up RT task cannot preempt the
+	 * current (RT) task (on a target runqueue) then try
+	 * to find another CPU it can preempt:
+	 */
+	if (rt_task(p) && !TASK_PREEMPTS_CURR(p, rq)) {
+		cpumask_t cpu_mask;
+		cpus_and(cpu_mask, cpu_online_map, p->cpus_allowed);
+
+		new_cpu = find_lowest_cpu(&cpu_mask, p, rq);
+		if ((new_cpu != -1) && (new_cpu != cpu)) {
+			set_task_cpu(p, new_cpu);
+			spin_unlock(&rq->lock);
+
+			/* The new lock was already acquired in find_lowest */ 
+			rq = cpu_rq(new_cpu);
+			old_state = p->state;
+			if (!(old_state & state))
+				goto out;
+			if (p->se.on_rq)
+				goto out_running;
+			
+			cpu = task_cpu(p);
+		}
+	}
 out_activate:
 #endif /* CONFIG_SMP */
 	update_rq_clock(rq);
@@ -1990,51 +2022,20 @@ out_activate:
 	trace_start_sched_wakeup(p, rq);
 
 	/*
-	 * If a newly woken up RT task cannot preempt the
-	 * current (RT) task (on a target runqueue) then try
-	 * to find another CPU it can preempt:
+	 * Sync wakeups (i.e. those types of wakeups where the waker
+	 * has indicated that it will leave the CPU in short order)
+	 * don't trigger a preemption, if the woken up task will run on
+	 * this cpu. (in this case the 'I will reschedule' promise of
+	 * the waker guarantees that the freshly woken up task is going
+	 * to be considered on this CPU.)
 	 */
-	if (rt_task(p) && !TASK_PREEMPTS_CURR(p, rq)) {
-		struct rq *this_rq = cpu_rq(this_cpu);
-		/*
-		 * Special-case: the task on this CPU can be
-		 * preempted. In that case there's no need to
-		 * trigger reschedules on other CPUs, we can
-		 * mark the current task for reschedule.
-		 *
-		 * (Note that it's safe to access this_rq without
-		 * extra locking in this particular case, because
-		 * we are on the current CPU.)
-		 */
-		if (TASK_PREEMPTS_CURR(p, this_rq))
-			set_tsk_need_resched(this_rq->curr);
-		else
-			/*
-			 * Neither the intended target runqueue
-			 * nor the current CPU can take this task.
-			 * Trigger a reschedule on all other CPUs
-			 * nevertheless, maybe one of them can take
-			 * this task:
-			 */
-			smp_send_reschedule_allbutself_cpumask(p->cpus_allowed);
-
-		schedstat_inc(this_rq, rto_wakeup);
-	} else {
-		/*
-		 * Sync wakeups (i.e. those types of wakeups where the waker
-		 * has indicated that it will leave the CPU in short order)
-		 * don't trigger a preemption, if the woken up task will run on
-		 * this cpu. (in this case the 'I will reschedule' promise of
-		 * the waker guarantees that the freshly woken up task is going
-		 * to be considered on this CPU.)
-		 */
-		if (!sync || cpu != this_cpu)
-			check_preempt_curr(rq, p);
-		else {
-			if (TASK_PREEMPTS_CURR(p, rq))
-				set_tsk_need_resched_delayed(rq->curr);
-		}
+	if (!sync || cpu != this_cpu)
+		check_preempt_curr(rq, p);
+	else {
+		if (TASK_PREEMPTS_CURR(p, rq))
+			set_tsk_need_resched_delayed(rq->curr);
 	}
+
 	if (rq->curr && p && rq && _need_resched())
 		trace_special_pid(p->pid, PRIO(p), PRIO(rq->curr));
 
@@ -2353,13 +2354,12 @@ static inline void finish_task_switch(struct rq *rq, struct task_struct *prev)
 	 * the lock was owned by prev, we need to release it
 	 * first via finish_lock_switch and then reaquire it.
 	 */
-	if (unlikely(rt_task(current))) {
+	if (unlikely(rq->rt_nr_running > 1)) {
 		spin_lock(&rq->lock);
-		/* push_rt_task will return true if it moved an RT */
-		while (push_rt_task(rq))
-			;
+		push_rt_tasks(rq);
 		spin_unlock(&rq->lock);
 	}
+
 #endif
 	fire_sched_in_preempt_notifiers(current);
 	trace_stop_sched_switched(current);


^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2007-10-16 16:24 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-10-16 16:08 [PATCH 0/4] RT: RT-Overload/Sched v3 Gregory Haskins
2007-10-16 16:08 ` [PATCH 1/4] RT: Wrap the RQ notion of priority to make it conditional Gregory Haskins
2007-10-16 16:09 ` [PATCH 2/4] RT: Initialize the priority value Gregory Haskins
2007-10-16 16:09 ` [PATCH 3/4] RT: Add support for updating push-rt priority under PI boost Gregory Haskins
2007-10-16 16:09 ` [PATCH 4/4] RT: Add support for low-priority wake-up to push_rt feature Gregory Haskins

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®