Re: [RFC PATCH 2/4] x86: use dmi check to treat disabled cpus as hotplug cpus.

[Suresh Siddha <suresh.b.siddha@intel.com>]

On Tue, 2010-01-12 at 07:19 -0800, Linus Torvalds wrote:
> On Tue, 12 Jan 2010, Yinghai Lu wrote:
> > > And if it's _really_ important:
> > >
> > >  - if flat mode is so important that you want to enable it whenever
> > >   possible, what about enabling/disabling it dynamically at CPU hotplug
> > >   time? That does sound _very_ painful, but it's still better than having
> > >   to maintain some list of all systems that can ever hot-plug.
> > 
> > interesting, could be done.
> > init_apic_ldr is called even for physical flat on 64 bit.
> > could change apic on fly.
> 
> Quite frankly, while I suggested it as an option, I really suspect it's 
> too much complexity for very little real gain.

I agree.

> Say that you have only four cores, but the kernel decided that it can't 
> use logical flat APIC mode because it sees three disabled sockets and 
> thinks "ok, we may end up with a total of 16 cores if those sockets are 
> hotplugged". Is that such a disaster?
> 
> Realistically, do we really care? Do you have performance numbers that say 
> that logical flat mode is so important that we really _really_ want to use 
> it, 

We had some customers in the past who wanted to use logical flat mode
when there are only 8 logical cpus mainly because they can use chipset
based interrupt routing. I think in one case, they wanted to use HW's
round-robin algorithm so that the interrupt load was uniformly
distributed to all the logical cpu's etc. This is probably fine if all
the logical cpu's are in the same socket (/under same memory
controller). But this might be a bad idea if those 8 logical cpu's are
spread across different sockets etc.

Also, sending IPI's becomes easier as we can target multiple logical
cpu's in the logical IPI destination mask etc.

> even at the cost of nasty run-time complexity with having to 
> re-program the APIC setup entirely when going from 8->9 CPU's?

No. I don't think it is worth it. As we have more and more cores, flat
mode usage will reduce and perhaps will remain mainly for
netbooks/laptops (before we have > 8 logical cpus in that space
aswell...). Also future generations will start supporting x2apic, where
we can use x2apic cluster mode.

For now, I think we should just make sure that for smaller HW configs
like laptops/desktops we should use flat mode when we have no more than
8 logical cpu's and use physical mode when there is a potential of
supporting more than 8 cpu's.

thanks,
suresh