Odd interaction between tasket and isolated cores (3.16.x)

[Edd Barrett <edd@theunixzoo.co.uk>]

Hi,
 
I'm experiencing an unexpected behaviour regarding thread scheduling on
3.16.x kernels.

What I am trying to do is: run multithreaded benchmarks on a set of
isolated cores and have threads balance over the isolated cores.

My approach is to boot with an isolcpus kernel argument, then use
taskset with an affinity mask so as to force benchmarks onto the
isolated set of cores. However, I have found that all threads get bound
to one core. Since this is not the case if you don't use taskset, or use
a realtime scheduling policy, I tend to think this is a bug, but I am
not sure.

I've raised a bug here:
https://bugzilla.kernel.org/show_bug.cgi?id=116701

Bug description contains a test case and detailed instructions to
reproduce. I've also used systemtap to hone in on the troublesome
section of code. I think it's to do with affine wake, but I'm not sure.

I'd be interested to hear what people think about this. If this is
expected behaviour, is there a way to disable affine wakes?

Thanks

-- 
Best Regards
Edd Barrett

http://www.theunixzoo.co.uk