Re: [RFC PATCH 0/6] CPU hotplug: Reverse invocation of notifiers during CPU hotplug

[Thomas Gleixner <tglx@linutronix.de>]

On Wed, 25 Jul 2012, Srivatsa S. Bhat wrote:
> On 07/25/2012 08:27 PM, Alan Stern wrote:
> One of the other ideas to improve the hotplug notifier stuff that came up during some
> of the discussions was to implement explicit dependency tracking between the notifiers
> and perhaps get rid of the priority numbers that are currently being used to provide
> some sort of ordering between the callbacks. Links to some of the related discussions
> are provided below.

The current code which brings up/down a CPU (mostly architecture
specific) code is comnpletely asymetric.

We really want a fully symetric state machine here, which also gives
us the proper invocation points for the other subsystems callbacks.

While I thought about having a full dependency tracking system, I'm
quite convinced by now, that hotplug is a rather linear sequence which
does not provide much room for paralell setup/teardown.

At least we should start with a simple linear chain.

The problem with the current notifiers is, that we only have ordering
for a few specific callbacks, but we don't have the faintest idea in
which order all other random stuff is brought up and torn down.

So I started experimenting with the following:

struct hotplug_event {
       int (*bring_up)(unsigned int cpu);
       int (*tear_down)(unsigned int cpu);
};

enum hotplug_events {
     CPU_HOTPLUG_START,
     CPU_HOTPLUG_CREATE_THREADS,
     CPU_HOTPLUG_INIT_TIMERS,
     ...
     CPU_HOTPLUG_KICK_CPU,
     ...
     CPU_HOTPLUG_START_THREADS,
     ...
     CPU_HOTPLUG_SET_ONLINE,
     ...
     CPU_HOTPLUG_MAX_EVENTS,
};

Now I have two arrays:

struct hotplug_event hotplug_events_bp[CPU_HOTPLUG_MAX_EVENTS];
struct hotplug_event hotplug_events_ap[CPU_HOTPLUG_MAX_EVENTS];
   
The _bp one is the list of events which are executed on the active cpu
and the _ap ones are those executed on the hotplugged cpu.

The core code advances the events in sync steps, so both BP and AP can
issue a stop on the process and cause a rollback.

Most of the callbacks can be added to the arrays at compile time, just
the stuff which is in modules requires an register/unregister
interface.

Though in any case the enum gives us a very explicit ordering of
setup/teardown, so rollback or partial online/offline should be simple
to achieve.

The only drawback is that it will prevent out of tree modules to use
the hotplug infrastructure, but I really couldn't care less.

Thoughts?

	tglx