CPU scheduler question/problem

[Pawel Dziekonski <dzieko@gmail.com>]

Hello,

all my compute intensive processes stick to one core,
whatever I put in /proc/sys/kernel/sched_min_granularity_ns - big number,
small number.

yes, I read /usr/src/linux/Documentation/scheduler/sched-design-CFS.txt
and yes, I did google about CFS tuning. no, I'm not a kernel developer. ;-)

also creation of cpu.shares does not help.

kernels tested:
2.6.27.7-9-default from openSUSE 11.1 distro
and vanilla 2.6.18.1.

this is a very simple test showing openssl performance:

dd if=/dev/zero bs=1M count=10000 | \
openssl enc -k qqqq -aes-128-cbc | \
pv > /dev/null

(you can skip pv if you like or not have it installed)

when I start top or pidstat I see that all of them run on the same core
and are competing for CPU time (pv 2%, dd 15%, openssl 83%).
to workaround this problem I use taskset to place processes on different cores:

taskset -c 0 dd if=/dev/zero bs=1M count=10000 | \
taskset -c 1 ./openssl enc -k qqqq -aes-128-cbc | \
taskset -c 2 pv > /dev/null

now the benchmark runs on 3 different cores and gives waaay better results
(115 MB/s compared to 185 MB/s).

another example: I start a computing application (quantum chemistry) that spawns
four processes that share workload. it happens very often that 2 processes share
same core for a moment and then jump to another core. taskset -p helps but it is
very inconvenient.

I am not using cpufreq.

question: is there a way to better balance processes over different
cores and do it
automagically? automating is necessary because I plan do build a compute cluster
with a batch system.

thanks in advance!

ps. my system is openSUSE 11.1 on Intel Core i7 CPU 940, HT enabled.
pps. I am not subscribed to LKML - please CC me - thanks!