Re: 2.6.14-git3: scheduling while atomic from cpufreq on Athlon64

[Andrew Morton <akpm@osdl.org>]

Ashok Raj <ashok.raj@intel.com> wrote:
>
> On Sat, Nov 05, 2005 at 03:33:04PM -0800, Andrew Morton wrote:
> > Ashok Raj <ashok.raj@intel.com> wrote:
> > >
> > > Now we leave a trace in current->flags indicating current thread already 
> > >  is under cpucontrol lock held, so we dont attempt to do this another time.
> > > 
> > > ..
> > > +#define PF_HOTPLUG_CPU	0x01000000	/* Currently performing CPU hotplug */
> > >
> > 
> > It's still hacky - I mean, we could use this trick to avoid recursion onto
> > any lock in the kernel whenever we get ourselves into a mess.  We'd gain an
> > awful lot of PF_* flags.
> > 
> > So we should still view this as a temporary fix.
> > 
> > I don't think I've seen an analysis of the actual deadlock yet.  Are you
> > able to provide a stack trace of the offending callpath?
> 
> Hi Andrew,
> 
> we call the exact same functions in cpufreq during startup and in
> response to cpu hotplug events, to create or destroy
> sysfs entries /sys/devices/system/cpu/cpuX/cpufreq/*. cpufreq_add_dev().
> 
> problem is cpufreq_set_policy()  eventually ends up calling
> __cpufreq_driver_target() during the CPU_ONLINE, and CPU_DOWN_PREPARE
> that takes cpucontrol lock.
> 
> Since when we already in the cpu notifier callbacks, cpucontrol is already 
> held by the cpu_up() or the cpu_down() that caused the double lock.

I appreciate that, but was hoping to see the call graph for the deadlock,
with sysrq-t.

See, one fix might be to take the lock_cpu_hotplug() call out of
__cpufreq_driver_target() and move it higher up the cpufreq call tree. 
Into the lowest-level caller which is _above_ the function which the
hotplug code calls.

Like this:

	hotplug -> down() -> ... ->
	                            __cpufreq_driver_target()
	cpufreq -> down() -> ... -> 

and not like this:

	hotplug -> down() -> ... ->
	                            __cpufreq_driver_target() -> down()
	cpufreq -> ...           ->