perf, how to support multiple x86 hw pmus?

perf, how to support multiple x86 hw pmus?

[Lin Ming]

Hi, all

Here multiple x86 hw pmus means, for example, Intel "core" and "uncore"
pmu. "core" pmu is to collect per cpu data, cpu-cycles, branch-misses,
etc. "uncore" pmu is to collect per package data, L3 cache, Intel QPI,
integrated memory controller, etc.

I am going to add Intel uncore pmu support to perf. To reduce code
duplicate, "uncore" pmu should reuse most of the "core" pmu code. But
currently, the x86 core pmu code(arch/x86/kernel/cpu/perf_event.c) only
supports one pmu, with a definition as below.

static struct x86_pmu x86_pmu __read_mostly;

Many functions use above global definition "x86_pmu". It seems to me
that we need to re-structure x86 pmu code to support multiple hw pmus.

Any idea?

Thanks,
Lin Ming

Re: perf, how to support multiple x86 hw pmus?

[Peter Zijlstra]

On Mon, 2010-08-16 at 16:09 +0800, Lin Ming wrote:
> Hi, all
> 
> Here multiple x86 hw pmus means, for example, Intel "core" and "uncore"
> pmu. "core" pmu is to collect per cpu data, cpu-cycles, branch-misses,
> etc. "uncore" pmu is to collect per package data, L3 cache, Intel QPI,
> integrated memory controller, etc.
> 
> I am going to add Intel uncore pmu support to perf. To reduce code
> duplicate, "uncore" pmu should reuse most of the "core" pmu code. But
> currently, the x86 core pmu code(arch/x86/kernel/cpu/perf_event.c) only
> supports one pmu, with a definition as below.
> 
> static struct x86_pmu x86_pmu __read_mostly;
> 
> Many functions use above global definition "x86_pmu". It seems to me
> that we need to re-structure x86 pmu code to support multiple hw pmus.
> 
> Any idea?

Yes, see my patch series http://lkml.org/lkml/2010/7/9/96 reworking the
pmu interface.

After that and some patches adding per pmu contexts adding multiple
hardware pmus should be simple.

uncore should not share any code with the regular pmu, since they're
mostly unrelated.

Re: perf, how to support multiple x86 hw pmus?

[Lin Ming]

On Mon, 2010-08-16 at 16:25 +0800, Peter Zijlstra wrote:
> On Mon, 2010-08-16 at 16:09 +0800, Lin Ming wrote:
> > Hi, all
> > 
> > Here multiple x86 hw pmus means, for example, Intel "core" and "uncore"
> > pmu. "core" pmu is to collect per cpu data, cpu-cycles, branch-misses,
> > etc. "uncore" pmu is to collect per package data, L3 cache, Intel QPI,
> > integrated memory controller, etc.
> > 
> > I am going to add Intel uncore pmu support to perf. To reduce code
> > duplicate, "uncore" pmu should reuse most of the "core" pmu code. But
> > currently, the x86 core pmu code(arch/x86/kernel/cpu/perf_event.c) only
> > supports one pmu, with a definition as below.
> > 
> > static struct x86_pmu x86_pmu __read_mostly;
> > 
> > Many functions use above global definition "x86_pmu". It seems to me
> > that we need to re-structure x86 pmu code to support multiple hw pmus.
> > 
> > Any idea?
> 
> Yes, see my patch series http://lkml.org/lkml/2010/7/9/96 reworking the
> pmu interface.

Yes, I know that series.

> 
> After that and some patches adding per pmu contexts adding multiple
> hardware pmus should be simple.

I didn't see the per pmu contexts patches, are you still working on
them?

> 
> uncore should not share any code with the regular pmu, since they're
> mostly unrelated.

But should they share code like collect_events, schedule_events,
x86_perf_event_set_period(with some modification) etc...?

Re: perf, how to support multiple x86 hw pmus?

[Peter Zijlstra]

On Mon, 2010-08-16 at 16:39 +0800, Lin Ming wrote:
> > After that and some patches adding per pmu contexts adding multiple
> > hardware pmus should be simple.
> 
> I didn't see the per pmu contexts patches, are you still working on
> them?
> 
Yes, although 3 weeks of holidays and a week of LinuxCon didn't help to
finish that work ;-)

I hope to be back to that soon ;-)

> > 
> > uncore should not share any code with the regular pmu, since they're
> > mostly unrelated.
> 
> But should they share code like collect_events, schedule_events,
> x86_perf_event_set_period(with some modification) etc...?
> 
Maybe, I'd have to look at the uncore stuff again, I can't remember if
it has much in the way of event scheduling constraints and if the
counter programming is close enough to the regular pmu to make it worth
sharing code over.

Re: perf, how to support multiple x86 hw pmus?

[Lin Ming]

On Mon, 2010-08-16 at 17:12 +0800, Peter Zijlstra wrote:
> On Mon, 2010-08-16 at 16:39 +0800, Lin Ming wrote:
> > > After that and some patches adding per pmu contexts adding multiple
> > > hardware pmus should be simple.
> > 
> > I didn't see the per pmu contexts patches, are you still working on
> > them?
> > 
> Yes, although 3 weeks of holidays and a week of LinuxCon didn't help to
> finish that work ;-)
> 
> I hope to be back to that soon ;-)

Really thanks for your work, that makes uncore support much easier.

> 
> > > 
> > > uncore should not share any code with the regular pmu, since they're
> > > mostly unrelated.
> > 
> > But should they share code like collect_events, schedule_events,
> > x86_perf_event_set_period(with some modification) etc...?
> > 
> Maybe, I'd have to look at the uncore stuff again, I can't remember if
> it has much in the way of event scheduling constraints and if the
> counter programming is close enough to the regular pmu to make it worth
> sharing code over.

I looked at the Nehalem uncore stuff and it does not have event
constraints. Each event can be monitored with any uncore counter.

The counter programming is close to regular pmu. It has
PERFEVTSELx/GLOBAL_STATUS/GLOBAL_OVF_CTRL/FIXED_CTR_CTRL like regular
pmu.