[RFD] CPU hotplug and suspend

["Rafael J. Wysocki" <rjw@sisk.pl>]

Hi,

Currently, we use the CPU hotplug to disable nonboot CPUs in the suspend code
paths, but with the recent change of code ordering (ie. nonboot CPUs are
disabled after freezing tasks _and_ devices) it has become quite troublesome.
The reason of this is that there are some CPU hotplug notifiers registered and
called on each run of cpu_up()/cpu_down() that assume the system to be fully
functional, which is not the case during the suspend.  Moreover, at least some
of them do things that are not really necessary for disabling or enabling the
nonboot CPUs.

For example, it doesn't seem to be necessary to stop worker threads bound to
the nonboot CPUs when they are disabled, because these CPUs most likely
reappear during the resume.  This particular problem has caused us to make all
workqueus nonfreezable, although at least some of them should be freezable, as
far as the suspend is concerned.

The advantage of using the CPU hotplug (in its current form) for suspending is
that if some CPUs don't reappear during the resume, we are safe.  Still, I
think it would be more appropriate, and simpler in the long run, to notify the
interested subsystems _only_ if one (or more) CPUs are not functional after the
resume.  In fact, with the current code ordering the subsystems don't even need
to know that we have disabled and enabled the nonboot CPUs, unless something
goes wrong.

For this reason, I'd like to change the suspend code to use a simplified CPU
management, sharing some low-level code with the current CPU hotplug, that
won't call all of the CPU hotplug notifiers at all, but will be able to call
some other special notifiers in case one (or more) of the nonboot CPUs cannot
be enabled.  Of course, that would require the subsystems to register separate
CPU notifiers for the resume, but I think they may share some code with the
current CPU hotplug notifiers.

It seems to me that we should separate the special case of suspend from the
"run-time" CPU hotplug, or things will get more and more complicated over time.
Still, that would be quite radical redesign, so I'm not sure if it's generally
acceptable.

Please advise.

Greetings,
Rafael