[PATCH 01/13] thermal: intel: Keep directed package interrupts off after syscore shutdown

[Ricardo Neri <ricardo.neri-calderon@linux.intel.com>]

On kexec, the syscore shutdown callback disables the directed package-level
thermal interrupt (DPTI) on every package. kernel_kexec() runs the callback
with CPU hotplug disabled, but re-enables CPU hotplug before
machine_shutdown() stops the other CPUs. A CPU hotplug operation in that
window directs the interrupt again: offlining the CPU that handles it
migrates the interrupt to another CPU, and a CPU coming online claims a
package that has no handler. The next kernel inherits the enable bit on
that CPU, never records the CPU as a handler, and never clears it.

Set a flag in the shutdown callback and make the CPU hotplug callbacks
leave DPTI alone once it is set. No lock is needed. The callback runs with
CPU hotplug disabled. Every later CPU hotplug operation takes
cpu_add_remove_lock, which cpu_hotplug_enable() releases after the flag is
set.

Fixes: 1e768cc92bc7 ("thermal: intel: Add a syscore shutdown callback for kexec reboot")
Signed-off-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
---
Each package's handler is the first of its CPUs to come online, which for
package 0 is CPU0. On x86, CPU0 cannot be offlined. Its interrupt never
migrates: a single-package system never opens the window. The other
packages of a multi-package system can.

The sysfs interface added later in this series makes the window real on any
system: it can move the interrupt off CPU0, and each write programs a CPU
without a CPU hotplug operation.
---
 drivers/thermal/intel/therm_throt.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/drivers/thermal/intel/therm_throt.c b/drivers/thermal/intel/therm_throt.c
index 59520cac6cc8..1ed604bd70e0 100644
--- a/drivers/thermal/intel/therm_throt.c
+++ b/drivers/thermal/intel/therm_throt.c
@@ -578,6 +578,13 @@ static void config_directed_thermal_pkg_intr(void *info)
  */
 static unsigned int *directed_intr_handler_cpus;
 
+/*
+ * Set by the syscore shutdown callback before it disables the directed
+ * interrupt, so that no CPU hotplug operation enables it again on the way to
+ * kexec.
+ */
+static bool directed_pkg_intr_shut_down;
+
 static bool directed_thermal_pkg_intr_supported(void)
 {
 	if (!boot_cpu_has(X86_FEATURE_DPTI))
@@ -603,6 +610,10 @@ static void disable_directed_thermal_pkg_intr_all(void)
 	if (!directed_thermal_pkg_intr_supported())
 		return;
 
+	/*
+	 * No need to reset the handlers: callers free directed_intr_handler_cpus
+	 * or have shut DPTI down.
+	 */
 	for (i = 0; i < topology_max_packages(); i++) {
 		if (directed_intr_handler_cpus[i] == nr_cpu_ids)
 			continue;
@@ -625,6 +636,9 @@ static int enable_directed_thermal_pkg_intr(unsigned int cpu)
 	if (pkg_id >= topology_max_packages())
 		return -EINVAL;
 
+	if (directed_pkg_intr_shut_down)
+		return 0;
+
 	/* Another CPU in this package already handles the directed interrupt. */
 	if (directed_intr_handler_cpus[pkg_id] != nr_cpu_ids)
 		return 0;
@@ -662,6 +676,9 @@ static void disable_directed_thermal_pkg_intr(unsigned int cpu)
 	if (pkg_id >= topology_max_packages())
 		return;
 
+	if (directed_pkg_intr_shut_down)
+		return;
+
 	/* Not the CPU handling the directed interrupt. */
 	if (directed_intr_handler_cpus[pkg_id] != cpu)
 		return;
@@ -738,6 +755,8 @@ static int directed_pkg_intr_syscore_suspend(void *data)
 
 static void directed_pkg_intr_syscore_shutdown(void *data)
 {
+	directed_pkg_intr_shut_down = true;
+
 	disable_directed_thermal_pkg_intr_all();
 }
 

-- 
2.43.0