Re: smp_send_reschedule vs. smp_migrate_task

[James Bottomley <James.Bottomley@SteelEye.com>]

> In that post I was really asking the following questions:
> * how often does smp_send_reschedule get called? * how often does
> smp_migrate_task get called? * if smp_send_reschedule and
> smp_migrate_task were mutually exclusive,
>   i.e. both used the same spinlock, could that lead to deadlock?

> James Bottomley answered the first two for me but not the third.

I think the answer to the third is yes.

The potential deadlock is inherent in smp_migrate_task().  Any code which 
takes a spinlock on one CPU and unlocks it on another via an IPI is asking for 
a deadlock.

Here's the scenario:

CPU 1 does a smp_migrate_task() to CPU 2 at the same time CPU 2 does the same 
thing to CPU 1.  They both contend for the migration lock.  CPU 1 wins, 
acquires the migration lock and sends the IPI to CPU 2.  If CPU 2 is spinning 
on the migration lock *with interrupts disabled* then you have a deadlock (it 
can never accept the IPI and release the lock).

The way out is to make sure interrupts are always enabled when taking the 
migration lock (which is true for all the task migration code paths).  This, 
in turn imposes a condition:  the lock may never be taken from an interrupt 
(otherwise it may deadlock itself).

Since smp_send_reschedule() is called down many process wake up code paths, 
I'm not sure you can comply with the no calling from interrupt requirement if 
you make it share a lock with smp_migrate_task().

James