Re: HT (Hyper Threading) aware process scheduling doesn't work as it should

[Henrique de Moraes Holschuh <hmh@hmh.eng.br>]

On Sun, 30 Oct 2011, Artem S. Tashkinov wrote:
> I've found out that even on Linux 3.0.8 the process scheduler doesn't correctly distributes
> the load amongst virtual CPUs. E.g. on a 4-core system (8 total virtual CPUs) the process
> scheduler often run some instances of four different tasks on the same physical CPU.

Please check how your sched_mc_power_savings and sched_smt_power_savings
tunables.   Here's the doc from lesswats.org:

'sched_mc_power_savings' tunable under /sys/devices/system/cpu/ controls
the Multi-core related tunable. By default, this is set to '0' (for
optimal performance). By setting this to '1', under light load
scenarios, the process load is distributed such that all the cores in a
processor package are busy before distributing the process load to other
processor packages.

[...]

'sched_smt_power_savings' tunable under /sys/devices/system/cpu/
controls the multi-threading related tunable. By default, this is set to
'0' (for optimal performance). By setting this to '1', under light load
scenarios, the process load is distributed such that all the threads in
a core and all the cores in a processor package are busy before
distributing the process load to threads and cores, in other processor
packages. 

Please make sure both are set to 0.  If they were not 0 at the time you
ran your tests, please retest and report back.

You also want to make sure you _do_ have the SMT scheduler compiled in
whatever kernel you're using, just in case.

It is certainly possible that there is a bug in the scheduler, but it is
best to make sure it is not something else, first.

You may also want to refer to: http://oss.intel.com/pdfs/mclinux.pdf and
to the irqbalance and hwloc[1] utilities, since you're apparently
interested in SMP/SMT/NUMA scheduler performance.

[1] http://www.open-mpi.org/projects/hwloc/

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh