Re: [patch 07/11] x86/perf/uncore: Track packages not per cpu data

[Thomas Gleixner <tglx@linutronix.de>]

On Thu, 18 Feb 2016, Thomas Gleixner wrote:
> On Thu, 18 Feb 2016, Peter Zijlstra wrote:
> > On Wed, Feb 17, 2016 at 11:16:40PM +0100, Andi Kleen wrote:
> > > > Do you have any data to back that up or is that just "believe" ?
> > > 
> > > I've seen systems with discontiguous apic ids before.
> > > 
> > > It is obvious if you consider setups with node hotplug.
> > 
> > Those systems will also have a num_possible_cpus() that's inflated to
> > account for the possible hotplug of new sockets, right?
> > 
> > So while the phys_pkg_id of the present sockets might be 2 and 3
> > (leaving 0 and 1 available for hotplug), the num_possible_cpus() should
> > be big enough to actually allow for that hotplug to happen.
> > 
> > Because if num_possible_cpus() is too small, we could not accommodate
> > the hotplug operation.
> > 
> > And if num_possible_cpus() is of the right size, then the computed
> > max_packages() should be of the right size too.
> > 
> > Now clearly, BIOS can completely wreck things and indeed report too
> > small an apic_id range or whatever, and in this case we're up a creek
> > without a paddle.
> > 
> > But I think you can check for that at boot and report errors/warns
> > whatever, because if you trigger this, your system is not really
> > 'correct' anyway.
> 
> Correct. Furthermore, we have a limitation of apic ids anyway. So there is a
> limitation of creative apic id assignements already.
> 
> Now if the system implementer can assign apic ids randomly in that space, the
> assumption of 
> 
> 	   maxpackages = num_possible_cpus / cpus_per_package
> 
> is not longer true. But that's not a blocker for the approach of tracking per
> package data. We simply can create logical package ids and use those.
> 
> So there is another possible issue. The above is also not true if we get
> heterogenous systems, i.e. cpus_per_package is non constant. But if we create
> logical package ids when building the system topoplogy the we can account for
> that and adjust maxpackages accordingly.

Thinking more about it. Even on heterogeneous systems the implementer needs to
be sane with the apic ids. The APIC id has 4 levels

   [Cluster ID][Package ID][Core ID][SMT ID]

So for any given system, the number of [Core ID + SMT ID] bits must be the
same for every package independent of the actual number of cores in the
package. If you violate this, then your APIC ID space is not longer
consistent. So we need to check for this anyway.

The SDM agrees with that:

Intel supports multi-threading systems where all physical processors report
identical values in CPUID leaf 0BH, CPUID.1:EBX[23:16]), CPUID.4 10
:EAX[31:26], and CPUID.4 11 :EAX[25:14].

So there is a limitation to BIOS creativity and all we need is to maintain a
logical package id.

Thanks,

	tglx