mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* [PATCH -tip/core/rcu 1/6] Cleanups and fixes for RCU in face of heavy CPU-hotplug stress
@ 2009-08-15 16:51 Paul E. McKenney
  2009-08-15 16:53 ` [PATCH -tip/core/rcu 1/6] Split hierarchical RCU initialization into boot-time and CPU-online pieces Paul E. McKenney
                   ` (6 more replies)
  0 siblings, 7 replies; 39+ messages in thread
From: Paul E. McKenney @ 2009-08-15 16:51 UTC (permalink / raw)
  To: linux-kernel
  Cc: mingo, laijs, dipankar, josht, akpm, mathieu.desnoyers, dvhltc,
	niv, tglx, peterz, rostedt, hugh.dickins, benh

This patch set provides the following cleanups and fixes for RCU in
workloads that involve lots of CPU hotplugging.  Ingo, please reset
core/rcu to 240ebbf before applying these patches.

o	Split Hierarchical RCU initialization into boot-time and
	CPU-hotplug pieces, so that other CPUs never see the data
	structures in the act of being stitched together.

o	Introduce a cpu_notifier() that works as does the current
	hotcpu_notifier(), but also notifies during boot time even
	if !HOTPLUG_CPU.

o	Make RCU use the new cpu_notifier() to simplify hotplug code
	and reduce the number of notifiers in use.  (This also fixes
	the problem Ingo and Hugh found.)

o	Preemptable RCU had a bug due to its overly clever attempt
	to move counts from CPUs going offline.  This is "unsafe
	at any speed", so remove the cleverness.

o	Since offline CPUs now can have non-zero counts, they must
	now be included in the tracing.

o	Fix typo in preemptable RCU's rcu_irq_enter() comment
	submitted by Josh Triplett.

These pass tests combining rcutorture and continuous CPU-hotplug
operations, in other words, no sleeping between successive CPU-hotplug
operations.  This set does not fix the bug involving deadlocks through
rcutorture, kthread_stop(), and migration threads.  However, this bug
does not affect production systems, so should not stand in the way of
applying these patches.

							Thanx, Paul

 b/include/linux/cpu.h       |   18 +++++++++-----
 b/kernel/rcupdate.c         |   16 ++++++++++++-
 b/kernel/rcupreempt.c       |   25 +-------------------
 b/kernel/rcupreempt_trace.c |    6 ++--
 b/kernel/rcutree.c          |   54 ++++++++++++++++++++++++++------------------
 kernel/rcupreempt.c         |   12 +--------
 kernel/rcutree.c            |   18 ++++----------
 7 files changed, 72 insertions(+), 77 deletions(-)

^ permalink raw reply	[flat|nested] 39+ messages in thread
* [PATCH -tip] Simplify RCU CPU-hotplug notification
@ 2009-07-10  0:48 Paul E. McKenney
  2009-07-14  6:21 ` [tip:core/rcu] rcu: " tip-bot for Paul E. McKenney
  0 siblings, 1 reply; 39+ messages in thread
From: Paul E. McKenney @ 2009-07-10  0:48 UTC (permalink / raw)
  To: linux-kernel; +Cc: mingo, laijs, dipankar

Simplify RCU's CPU-hotplug notifiers so that there is only one notifier.
This makes it trivial to provide the ordering guarantee that rcu_barrier()
depends on.  This change also allows rcu_barrier() to rely on locking
wholly contained in _cpu_down(), rather than that of its callers, making
the locking more straightforward.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---

 rcupdate.c   |    8 ++++++--
 rcupreempt.c |   22 +++++-----------------
 rcutree.c    |   18 +++++++++---------
 3 files changed, 20 insertions(+), 28 deletions(-)

diff --git a/kernel/rcupdate.c b/kernel/rcupdate.c
index eae29c2..3fea910 100644
--- a/kernel/rcupdate.c
+++ b/kernel/rcupdate.c
@@ -217,9 +217,13 @@ static void rcu_migrate_callback(struct rcu_head *notused)
 		wake_up(&rcu_migrate_wq);
 }
 
+extern int rcu_cpu_notify(struct notifier_block *self,
+			  unsigned long action, void *hcpu);
+
 static int __cpuinit rcu_barrier_cpu_hotplug(struct notifier_block *self,
 		unsigned long action, void *hcpu)
 {
+	rcu_cpu_notify(self, action, hcpu);
 	if (action == CPU_DYING) {
 		/*
 		 * preempt_disable() in on_each_cpu() prevents stop_machine(),
@@ -234,7 +238,7 @@ static int __cpuinit rcu_barrier_cpu_hotplug(struct notifier_block *self,
 		call_rcu_bh(rcu_migrate_head, rcu_migrate_callback);
 		call_rcu_sched(rcu_migrate_head + 1, rcu_migrate_callback);
 		call_rcu(rcu_migrate_head + 2, rcu_migrate_callback);
-	} else if (action == CPU_POST_DEAD) {
+	} else if (action == CPU_DEAD) {
 		/* rcu_migrate_head is protected by cpu_add_remove_lock */
 		wait_migrated_callbacks();
 	}
@@ -244,8 +248,8 @@ static int __cpuinit rcu_barrier_cpu_hotplug(struct notifier_block *self,
 
 void __init rcu_init(void)
 {
-	__rcu_init();
 	hotcpu_notifier(rcu_barrier_cpu_hotplug, 0);
+	__rcu_init();
 }
 
 void rcu_scheduler_starting(void)
diff --git a/kernel/rcupreempt.c b/kernel/rcupreempt.c
index beb0e65..4300212 100644
--- a/kernel/rcupreempt.c
+++ b/kernel/rcupreempt.c
@@ -1417,7 +1417,7 @@ int rcu_pending(int cpu)
 	return 0;
 }
 
-static int __cpuinit rcu_cpu_notify(struct notifier_block *self,
+int __cpuinit rcu_cpu_notify(struct notifier_block *self,
 				unsigned long action, void *hcpu)
 {
 	long cpu = (long)hcpu;
@@ -1439,10 +1439,6 @@ static int __cpuinit rcu_cpu_notify(struct notifier_block *self,
 	return NOTIFY_OK;
 }
 
-static struct notifier_block __cpuinitdata rcu_nb = {
-	.notifier_call = rcu_cpu_notify,
-};
-
 void __init __rcu_init(void)
 {
 	int cpu;
@@ -1471,22 +1467,14 @@ void __init __rcu_init(void)
 		rdp->waitschedtail = &rdp->waitschedlist;
 		rdp->rcu_sched_sleeping = 0;
 	}
-	register_cpu_notifier(&rcu_nb);
 
 	/*
-	 * We don't need protection against CPU-Hotplug here
-	 * since
-	 * a) If a CPU comes online while we are iterating over the
-	 *    cpu_online_mask below, we would only end up making a
-	 *    duplicate call to rcu_online_cpu() which sets the corresponding
-	 *    CPU's mask in the rcu_cpu_online_map.
-	 *
-	 * b) A CPU cannot go offline at this point in time since the user
-	 *    does not have access to the sysfs interface, nor do we
-	 *    suspend the system.
+	 * We don't need protection against CPU-hotplug here because
+	 * this is called early in boot, before either interrupts
+	 * or the scheduler are operational.
 	 */
 	for_each_online_cpu(cpu)
-		rcu_cpu_notify(&rcu_nb, CPU_UP_PREPARE,	(void *)(long) cpu);
+		rcu_cpu_notify(NULL, CPU_UP_PREPARE, (void *)(long) cpu);
 
 	open_softirq(RCU_SOFTIRQ, rcu_process_callbacks);
 }
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index 7717b95..e8e9e93 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -1407,13 +1407,12 @@ static void __cpuinit rcu_online_cpu(int cpu)
 {
 	rcu_init_percpu_data(cpu, &rcu_state);
 	rcu_init_percpu_data(cpu, &rcu_bh_state);
-	open_softirq(RCU_SOFTIRQ, rcu_process_callbacks);
 }
 
 /*
  * Handle CPU online/offline notifcation events.
  */
-static int __cpuinit rcu_cpu_notify(struct notifier_block *self,
+int __cpuinit rcu_cpu_notify(struct notifier_block *self,
 				unsigned long action, void *hcpu)
 {
 	long cpu = (long)hcpu;
@@ -1523,10 +1522,6 @@ do { \
 	} \
 } while (0)
 
-static struct notifier_block __cpuinitdata rcu_nb = {
-	.notifier_call	= rcu_cpu_notify,
-};
-
 void __init __rcu_init(void)
 {
 	int i;			/* All used by RCU_DATA_PTR_INIT(). */
@@ -1542,10 +1537,15 @@ void __init __rcu_init(void)
 	rcu_init_one(&rcu_bh_state);
 	RCU_DATA_PTR_INIT(&rcu_bh_state, rcu_bh_data);
 
+	/*
+	 * We don't need protection against CPU-hotplug here because
+	 * this is called early in boot, before either interrupts
+	 * or the scheduler are operational.
+	 */
 	for_each_online_cpu(i)
-		rcu_cpu_notify(&rcu_nb, CPU_UP_PREPARE, (void *)(long)i);
-	/* Register notifier for non-boot CPUs */
-	register_cpu_notifier(&rcu_nb);
+		rcu_cpu_notify(NULL, CPU_UP_PREPARE, (void *)(long)i);
+
+	open_softirq(RCU_SOFTIRQ, rcu_process_callbacks);
 }
 
 module_param(blimit, int, 0);

^ permalink raw reply	[flat|nested] 39+ messages in thread
end of thread, other threads:[~2009-08-21 15:37 UTC | newest]

Thread overview: 39+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2009-08-15 16:51 [PATCH -tip/core/rcu 1/6] Cleanups and fixes for RCU in face of heavy CPU-hotplug stress Paul E. McKenney
2009-08-15 16:53 ` [PATCH -tip/core/rcu 1/6] Split hierarchical RCU initialization into boot-time and CPU-online pieces Paul E. McKenney
2009-08-15 17:07   ` [tip:core/rcu] rcu: " tip-bot for Paul E. McKenney
2009-08-15 16:53 ` [PATCH -tip/core/rcu 2/6] Introduce cpu_notifier() to handle !HOTPLUG_CPU case Paul E. McKenney
2009-08-15 17:07   ` [tip:core/rcu] cpu hotplug: " tip-bot for Paul E. McKenney
2009-08-17 17:21   ` [PATCH -tip/core/rcu 2/6] " Josh Triplett
2009-08-17 18:28     ` Paul E. McKenney
2009-08-15 16:53 ` [PATCH -tip/core/rcu 3/6] Simplify RCU CPU-hotplug notification Paul E. McKenney
2009-08-15 17:07   ` [tip:core/rcu] rcu: " tip-bot for Paul E. McKenney
2009-08-20  4:02   ` [PATCH -tip/core/rcu 3/6] " Lai Jiangshan
2009-08-20  4:21     ` Paul E. McKenney
2009-08-15 16:53 ` [PATCH -tip/core/rcu 4/6] Make preemptable RCU scan all CPUs when summing RCU counters Paul E. McKenney
2009-08-15 17:07   ` [tip:core/rcu] rcu: " tip-bot for Paul E. McKenney
2009-08-15 16:53 ` [PATCH -tip/core/rcu 5/6] Make rcupreempt_trace.c look at offline CPUs Paul E. McKenney
2009-08-15 17:07   ` [tip:core/rcu] rcu: " tip-bot for Paul E. McKenney
2009-08-15 16:53 ` [PATCH -tip/core/rcu 6/6] Fix typo in rcu_irq_exit() comment header Paul E. McKenney
2009-08-15 17:00   ` Ingo Molnar
2009-08-15 17:10     ` Paul E. McKenney
2009-08-15 17:11       ` Ingo Molnar
2009-08-15 17:08   ` [tip:core/rcu] rcu: " tip-bot for Josh Triplett
2009-08-17 18:24 ` [PATCH -tip/core/rcu 1/6] Cleanups and fixes for RCU in face of heavy CPU-hotplug stress Josh Triplett
2009-08-17 19:20   ` Paul E. McKenney
2009-08-18 15:26     ` Ingo Molnar
2009-08-18 20:07       ` Paul E. McKenney
2009-08-19  6:06         ` Paul E. McKenney
2009-08-19 11:59           ` Ingo Molnar
2009-08-19 12:09           ` [tip:core/rcu] rcu: Delay rcu_barrier() wait until beginning of next CPU-hotunplug operation tip-bot for Paul E. McKenney
2009-08-19 15:24           ` [PATCH -tip/core/rcu 1/6] Cleanups and fixes for RCU in face of heavy CPU-hotplug stress Mathieu Desnoyers
2009-08-19 16:38             ` Paul E. McKenney
2009-08-19 18:10               ` Mathieu Desnoyers
2009-08-19 18:31                 ` Paul E. McKenney
2009-08-20 14:03       ` Mathieu Desnoyers
2009-08-21 14:17         ` Ingo Molnar
2009-08-21 14:29           ` Steven Rostedt
2009-08-21 14:44             ` Ingo Molnar
2009-08-21 15:00               ` Mathieu Desnoyers
2009-08-21 15:37               ` Paul E. McKenney
2009-08-21 14:58           ` Mathieu Desnoyers
  -- strict thread matches above, loose matches on Subject: below --
2009-07-10  0:48 [PATCH -tip] Simplify RCU CPU-hotplug notification Paul E. McKenney
2009-07-14  6:21 ` [tip:core/rcu] rcu: " tip-bot for Paul E. McKenney

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®