Re: [Xen-devel] [PATCH 2/2] xen/events: Convert to hotplug state machine

[Boris Ostrovsky <boris.ostrovsky@oracle.com>]

On 08/15/2016 11:06 AM, David Vrabel wrote:
> On 15/08/16 15:46, Boris Ostrovsky wrote:
>> From: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
>>
>> Install the callbacks via the state machine.
> [...]
>> +static int xen_evtchn_cpu_dead(unsigned int cpu)
>> +{
>> +	__evtchn_fifo_handle_events(cpu, true);
>> +	return 0;
>> +}
> I'm not familiar with the new state machine.  When this is called, what
> state is the CPU in?
>
> In particular, local interrupts must be disabled and all non-percpu irqs
> must have been migrated to other CPUs.

This (xen_evtchn_cpu_dead()) is called immediately after notify_dead()
on the way down.

The state machine is walking cpuhp_state list and for each member of the
list it calls the callback that has been registered for that member.

So when we bring a CPU up first we call xen_evtchn_cpu_prepare() and
then in the next iteration of the state machine loop notify_prepare()
(because CPUHP_XEN_EVTCHN_PREPARE is immediately before
CPUHP_NOTIFY_PREPARE). On the way down it's done in reverse: first
notify_dead() and then xen_evtchn_cpu_dead().

In other words, the old notification scheme is part of new state machine:

	CPUHP_RCUTREE_PREP,
 	CPUHP_XEN_PREPARE,
+	CPUHP_XEN_EVTCHN_PREPARE,
 	CPUHP_NOTIFY_PREPARE,  <=== CPU notifiers callback
	CPUHP_TIMERS_DEAD,
	CPUHP_BRINGUP_CPU,


-boris


>
>
>>  int __init xen_evtchn_fifo_init(void)
>>  {
>> @@ -456,7 +444,9 @@ int __init xen_evtchn_fifo_init(void)
>>  
>>  	evtchn_ops = &evtchn_ops_fifo;
>>  
>> -	register_cpu_notifier(&evtchn_fifo_cpu_notifier);
>> +	cpuhp_setup_state_nocalls(CPUHP_XEN_EVTCHN_PREPARE,
>> +				  "CPUHP_XEN_EVTCHN_PREPARE",
>> +				  xen_evtchn_cpu_prepare, xen_evtchn_cpu_dead);
>>  out:
>>  	put_cpu();
>>  	return ret;
>> diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
>> index d6beeb9..c60a17c 100644
>> --- a/include/linux/cpuhotplug.h
>> +++ b/include/linux/cpuhotplug.h
>> @@ -22,6 +22,7 @@ enum cpuhp_state {
>>  	CPUHP_SMPCFD_PREPARE,
>>  	CPUHP_RCUTREE_PREP,
>>  	CPUHP_XEN_PREPARE,
>> +	CPUHP_XEN_EVTCHN_PREPARE,
>>  	CPUHP_NOTIFY_PREPARE,
>>  	CPUHP_TIMERS_DEAD,
>>  	CPUHP_BRINGUP_CPU,
>>