Re: Help understanding slow down

[Ingo Molnar <mingo@elte.hu>]

* Andrew Morton <akpm@osdl.org> wrote:

> > it makes it a bit more plausible, but kernel profiling based on ticks in
> > a HT environment is still quite unreliable, even with idle=poll. The HT
> > cores will yield to each other on various occasions - like spinlock
> > loops. This disproportionatly increases the hits of various looping
> > functions, creating false impressions of lock contention where there's
> > only little contention. Plus idle=poll is a constant ~20% performance
> > drain on the non-idle HT core, further distorting the profile. HT makes
> > profiling really hard, no matter what.
> 
> But often one is looking for relativities rather than real absolute
> numbers.  (In which case the absent idle time doesn't matter, but it
> freaks me out...)

but it's the relativities that get skewed by HT, fundamentally - both
with and without idle=poll. A function that does not generate alot of
HT-yield activity will fare much less on the profiler histogram than a
function that happens to hit some contention point every now and then.
Also, cachemisses get far more prominent due to HT (they are a yield
point too) - suppressing other, possibly more important functions in the
list. So a kernel profile done under HT gives a rough idea but a
function could easily have half the overhead that the profiler credits
it to have. Profiling under HT magnifies the effect of lock-contention
and cache-misses, and shrinks the effect of algorithmic overhead. Plus
this skewing is not deterministic, it depends on the actual system
activity.

unfortunately i see no good way to do reliable time-based profiling
under HT, given the interaction of logical CPUs. The virtual CPUs break
the single-EIP notion of 'what is this physical CPU doing now'. Perhaps
it would be more reliable to not use time but use some of the other
triggers: e.g. # of uops retired?

	Ingo