Re: Scheduler benchmarks - a follow-up

["Rob Hussey" <robjhussey@gmail.com>]

On 9/18/07, Ingo Molnar <mingo@elte.hu> wrote:
>
> * Rob Hussey <robjhussey@gmail.com> wrote:
>
> > A cursory glance suggests that performance wrt lat_ctx and hackbench
> > has increased (lower numbers), but degraded quite a lot for pipe-test.
> > The numbers for pipe-test are extremely stable though, while the
> > numbers for hackbench are more erratic (which isn't saying much since
> > the original numbers gave nearly a straight line). I'm still willing
> > to try out any more ideas.
>
> pipe-test is a very stable workload, and is thus quite sensitive to the
> associativity of the CPU cache. Even killing the task and repeating the
> same test isnt enough to get rid of the systematic skew that this can
> cause. I've seen divergence of up to 10% in pipe-test. One way to test
> it is to run pipe-test, then to stop it, then to "ssh localhost" (this
> in itself uses up a couple of pipe objects and file objects and changes
> the cache layout picture), then run pipe-test again, then again "ssh
> localhost", etc. Via this trick one can often see cache-layout
> artifacts. How much 'skew' does pipe-test have on your system if you try
> this manually?
>

I did 7 data sets of 5 runs each using this method. With pipe-test
bound to one sibling, there were 10 unique values in these 7 sets. The
lowest value was 9.22, the highest value was 9.62, and the median of
the unique values was 9.47. So the deviation from mean for the lowest
and highest values was {-0.25, 0.15} The numbers were even tighter for
pipe-test not bound to a single sibling: {-0.07, 0.12}