cpuidle: psci: cluster PM domain left off when idle-CPU wakeup races with dpm_suspend_late()

cpuidle: psci: cluster PM domain left off when idle-CPU wakeup races with dpm_suspend_late()

[朱恺乾]

Hi, all

This is reporting a race observed on v6.18.21 Android Kernel during the
ARM CPU s2idle.

In PSCI OSI mode with the hierarchical CPU topology, a CPU can come out of
idle after dpm_suspend_late() has already disabled runtime PM for its
device. The idle exit then restores the cluster PM domain through the
runtime PM path, however, since the runtime PM is disabled, it becomes a
no-op and the domain stays powered off in software while the CPU is running.
The CPU then silently stops requesting the cluster/system power level in
CPU_SUSPEND.

The race
--------
dpm_suspend_late() wakes all idle CPUs first and only then walks the device
list, disabling runtime PM per device as it goes:

  dpm_suspend_late()
    wake_up_all_idle_cpus()               <-- asynchronous cpu wakeup
      device_suspend_late(dev, ...)
        __pm_runtime_disable(dev,false)    <-- disable_depth++

wake_up_all_idle_cpus() only requests that the idle CPUs leave idle; it
does not wait for them. A CPU parked in a deep PSCI state can take a long time
to come back that its idle exit lands after the runtime disable:

  CPU0: dpm_suspend_late()             CPUx: parked in idle
  ------------------------------------             ----------------------------
  wake_up_all_idle_cpus()
                                     powering ON
  device_suspend_late(pd_dev)
    __pm_runtime_disable(pd_dev)
      disable_depth++
                                     powered ON
                                     __psci_enter_domain_idle_state()
                                        pm_runtime_get_sync(pd_dev)
                                          no-op: disable_depth > 0

So genpd_runtime_resume() never runs and the domain is never powered back on.
Later, as the CPU and its domain remains OFF, PSCI CPU s2idle silently skips the genpd
suspend. When the CPU powers off with the runtime PD disabled, it can't wakeup to
ON next time even when the S2IDLE_STATE_ENTER is set.

A fix could be within the dpm_suspend_late, wait until all the CPUs back
before the runtime PM is disabled. Or, in the PSCI driver, when the CPUs wake up and
S2IDLE_STATE_ENTER is set, forcibly restore its domain to the ON state.

What do you think?


Thanks,
Kaiqian

#/******本邮件及其附件含有小米公司的保密信息，仅限于发送给上面地址中列出的个人或群组。禁止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、或散发）本邮件中的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本邮件！ This e-mail and its attachments contain confidential information from XIAOMI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!******/#