[PATCH] cpu/hotplug: Provide dynamic range for prepare stage

[PATCH] cpu/hotplug: Provide dynamic range for prepare stage

[Thomas Gleixner]

Mathieu reported that the LTTNG modules are broken as of 4.10-rc1 due to
the removal of the cpu hotplug notifiers.

Usually I don't care much about out of tree modules, but LTTNG is widely
used in distros. There are two ways to solve that:

1) Reserve a hotplug state for LTTNG

2) Add a dynamic range for the prepare states.

While #1 is the simplest solution, #2 is the proper one as we can convert
in tree users, which do not care about ordering, to the dynamic range as
well.

Add a dynamic range which allows LTTNG to request states in the prepare
stage.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>

---

diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
index 20bfefbe7594..d936a0021839 100644
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -74,6 +74,8 @@ enum cpuhp_state {
 	CPUHP_ZCOMP_PREPARE,
 	CPUHP_TIMERS_DEAD,
 	CPUHP_MIPS_SOC_PREPARE,
+	CPUHP_BP_PREPARE_DYN,
+	CPUHP_BP_PREPARE_DYN_END		= CPUHP_BP_PREPARE_DYN + 20,
 	CPUHP_BRINGUP_CPU,
 	CPUHP_AP_IDLE_DEAD,
 	CPUHP_AP_OFFLINE,
diff --git a/kernel/cpu.c b/kernel/cpu.c
index f75c4d031eeb..c47506357519 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -1302,10 +1302,24 @@ static int cpuhp_cb_check(enum cpuhp_state state)
  */
 static int cpuhp_reserve_state(enum cpuhp_state state)
 {
-	enum cpuhp_state i;
+	enum cpuhp_state i, end;
+	struct cpuhp_step *step;
 
-	for (i = CPUHP_AP_ONLINE_DYN; i <= CPUHP_AP_ONLINE_DYN_END; i++) {
-		if (!cpuhp_ap_states[i].name)
+	switch (state) {
+	case CPUHP_AP_ONLINE_DYN:
+		step = cpuhp_ap_states + CPUHP_AP_ONLINE_DYN;
+		end = CPUHP_AP_ONLINE_DYN_END;
+		break;
+	case CPUHP_BP_PREPARE_DYN:
+		step = cpuhp_bp_states + CPUHP_BP_PREPARE_DYN;
+		end = CPUHP_BP_PREPARE_DYN_END;
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	for (i = state; i <= end; i++, step++) {
+		if (!step->name)
 			return i;
 	}
 	WARN(1, "No more dynamic states available for CPU hotplug\n");
@@ -1323,7 +1337,7 @@ static int cpuhp_store_callbacks(enum cpuhp_state state, const char *name,
 
 	mutex_lock(&cpuhp_state_mutex);
 
-	if (state == CPUHP_AP_ONLINE_DYN) {
+	if (state == CPUHP_AP_ONLINE_DYN || state == CPUHP_BP_PREPARE_DYN) {
 		ret = cpuhp_reserve_state(state);
 		if (ret < 0)
 			goto out;

Re: [PATCH] cpu/hotplug: Provide dynamic range for prepare stage

[Mathieu Desnoyers]

----- On Jan 10, 2017, at 8:01 AM, Thomas Gleixner tglx@linutronix.de wrote:

> Mathieu reported that the LTTNG modules are broken as of 4.10-rc1 due to
> the removal of the cpu hotplug notifiers.
> 
> Usually I don't care much about out of tree modules, but LTTNG is widely
> used in distros. There are two ways to solve that:
> 
> 1) Reserve a hotplug state for LTTNG
> 
> 2) Add a dynamic range for the prepare states.
> 
> While #1 is the simplest solution, #2 is the proper one as we can convert
> in tree users, which do not care about ordering, to the dynamic range as
> well.
> 
> Add a dynamic range which allows LTTNG to request states in the prepare
> stage.

I prepared a lttng-modules commit [1] that uses CPUHP_BP_PREPARE_DYN on 4.10
kernels, and it works fine through my cpu hotplug stress-testing. It's a very
good cleanup! Thanks Thomas for this nice API.

I'll be able to merge my commit into lttng-modules as soon as your patch
lands into Linux mainline.

Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Tested-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Thanks,

Mathieu

[1] https://github.com/compudj/lttng-modules-dev/commit/f02214ac2e9a0f3d8d0270c0ac702c3143cdf120

> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> 
> ---
> 
> diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
> index 20bfefbe7594..d936a0021839 100644
> --- a/include/linux/cpuhotplug.h
> +++ b/include/linux/cpuhotplug.h
> @@ -74,6 +74,8 @@ enum cpuhp_state {
> 	CPUHP_ZCOMP_PREPARE,
> 	CPUHP_TIMERS_DEAD,
> 	CPUHP_MIPS_SOC_PREPARE,
> +	CPUHP_BP_PREPARE_DYN,
> +	CPUHP_BP_PREPARE_DYN_END		= CPUHP_BP_PREPARE_DYN + 20,
> 	CPUHP_BRINGUP_CPU,
> 	CPUHP_AP_IDLE_DEAD,
> 	CPUHP_AP_OFFLINE,
> diff --git a/kernel/cpu.c b/kernel/cpu.c
> index f75c4d031eeb..c47506357519 100644
> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -1302,10 +1302,24 @@ static int cpuhp_cb_check(enum cpuhp_state state)
>  */
> static int cpuhp_reserve_state(enum cpuhp_state state)
> {
> -	enum cpuhp_state i;
> +	enum cpuhp_state i, end;
> +	struct cpuhp_step *step;
> 
> -	for (i = CPUHP_AP_ONLINE_DYN; i <= CPUHP_AP_ONLINE_DYN_END; i++) {
> -		if (!cpuhp_ap_states[i].name)
> +	switch (state) {
> +	case CPUHP_AP_ONLINE_DYN:
> +		step = cpuhp_ap_states + CPUHP_AP_ONLINE_DYN;
> +		end = CPUHP_AP_ONLINE_DYN_END;
> +		break;
> +	case CPUHP_BP_PREPARE_DYN:
> +		step = cpuhp_bp_states + CPUHP_BP_PREPARE_DYN;
> +		end = CPUHP_BP_PREPARE_DYN_END;
> +		break;
> +	default:
> +		return -EINVAL;
> +	}
> +
> +	for (i = state; i <= end; i++, step++) {
> +		if (!step->name)
> 			return i;
> 	}
> 	WARN(1, "No more dynamic states available for CPU hotplug\n");
> @@ -1323,7 +1337,7 @@ static int cpuhp_store_callbacks(enum cpuhp_state state,
> const char *name,
> 
> 	mutex_lock(&cpuhp_state_mutex);
> 
> -	if (state == CPUHP_AP_ONLINE_DYN) {
> +	if (state == CPUHP_AP_ONLINE_DYN || state == CPUHP_BP_PREPARE_DYN) {
> 		ret = cpuhp_reserve_state(state);
> 		if (ret < 0)
>  			goto out;

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

[tip:smp/urgent] cpu/hotplug: Provide dynamic range for prepare stage

[tip-bot for Thomas Gleixner]

Commit-ID:  4205e4786d0b9fc3b4fec7b1910cf645a0468307
Gitweb:     http://git.kernel.org/tip/4205e4786d0b9fc3b4fec7b1910cf645a0468307
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Tue, 10 Jan 2017 14:01:05 +0100
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Mon, 16 Jan 2017 13:20:05 +0100

cpu/hotplug: Provide dynamic range for prepare stage

Mathieu reported that the LTTNG modules are broken as of 4.10-rc1 due to
the removal of the cpu hotplug notifiers.

Usually I don't care much about out of tree modules, but LTTNG is widely
used in distros. There are two ways to solve that:

1) Reserve a hotplug state for LTTNG

2) Add a dynamic range for the prepare states.

While #1 is the simplest solution, #2 is the proper one as we can convert
in tree users, which do not care about ordering, to the dynamic range as
well.

Add a dynamic range which allows LTTNG to request states in the prepare
stage.

Reported-and-tested-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sebastian Sewior <bigeasy@linutronix.de>
Cc: Steven Rostedt <rostedt@goodmis.org>
Link: http://lkml.kernel.org/r/alpine.DEB.2.20.1701101353010.3401@nanos
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>

---
 include/linux/cpuhotplug.h |  2 ++
 kernel/cpu.c               | 22 ++++++++++++++++++----
 2 files changed, 20 insertions(+), 4 deletions(-)

diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
index 20bfefb..d936a00 100644
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -74,6 +74,8 @@ enum cpuhp_state {
 	CPUHP_ZCOMP_PREPARE,
 	CPUHP_TIMERS_DEAD,
 	CPUHP_MIPS_SOC_PREPARE,
+	CPUHP_BP_PREPARE_DYN,
+	CPUHP_BP_PREPARE_DYN_END		= CPUHP_BP_PREPARE_DYN + 20,
 	CPUHP_BRINGUP_CPU,
 	CPUHP_AP_IDLE_DEAD,
 	CPUHP_AP_OFFLINE,
diff --git a/kernel/cpu.c b/kernel/cpu.c
index f75c4d0..c475063 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -1302,10 +1302,24 @@ static int cpuhp_cb_check(enum cpuhp_state state)
  */
 static int cpuhp_reserve_state(enum cpuhp_state state)
 {
-	enum cpuhp_state i;
+	enum cpuhp_state i, end;
+	struct cpuhp_step *step;
 
-	for (i = CPUHP_AP_ONLINE_DYN; i <= CPUHP_AP_ONLINE_DYN_END; i++) {
-		if (!cpuhp_ap_states[i].name)
+	switch (state) {
+	case CPUHP_AP_ONLINE_DYN:
+		step = cpuhp_ap_states + CPUHP_AP_ONLINE_DYN;
+		end = CPUHP_AP_ONLINE_DYN_END;
+		break;
+	case CPUHP_BP_PREPARE_DYN:
+		step = cpuhp_bp_states + CPUHP_BP_PREPARE_DYN;
+		end = CPUHP_BP_PREPARE_DYN_END;
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	for (i = state; i <= end; i++, step++) {
+		if (!step->name)
 			return i;
 	}
 	WARN(1, "No more dynamic states available for CPU hotplug\n");
@@ -1323,7 +1337,7 @@ static int cpuhp_store_callbacks(enum cpuhp_state state, const char *name,
 
 	mutex_lock(&cpuhp_state_mutex);
 
-	if (state == CPUHP_AP_ONLINE_DYN) {
+	if (state == CPUHP_AP_ONLINE_DYN || state == CPUHP_BP_PREPARE_DYN) {
 		ret = cpuhp_reserve_state(state);
 		if (ret < 0)
 			goto out;