Re: [PATCH v4 00/10] steal tasks to improve CPU utilization

[Valentin Schneider <valentin.schneider@arm.com>]

Hi,

On 10/12/2018 16:29, Steven Sistare wrote:
[...]
>> I have run some hackbench tests on my hikey arm64 octo cores with your
>> patchset. My original intent was to send a tested-by but I have some
>> performances regressions.
>> This hikey is the smp one and not the asymetric hikey960 that Valentin
>> used for his tests
>> The sched domain topology is
>> domain-0: span=0-3 level=MC  and domain-0: span=4-7 level=MC
>> domain-1: span=0-7 level=DIE
>>
>> I have run 12 times hackbench -g $j -P -l 2000 with j equals to 1 2 3 4 8
>>
>> grps  time
>> 1      1.396
>> 2      2.699
>> 3      3.617
>> 4      4.498
>> 8      7.721
>>
>> Then after disabling STEAL in sched_feature with echo NO_STEAL >
>> /sys/kernel/debug/sched_features , the results become:
>> grps  time
>> 1      1.217
>> 2      1.973
>> 3      2.855
>> 4      3.932
>> 8      7.674
>>
>> I haven't looked in details about some possible reasons of such
>> difference yet and haven't collected the stats that you added with
>> patch 10.
>> Have you got a script to collect and post process them ?
>>

I used the script that Steve sent just before LPC [1] - I probably should
have given that a try sooner...


Running the base "hackseries" on my H960 gave me this:

          --- base --    --- new ---
groups    time %stdev    time %stdev  %speedup
     1   1.021    9.1   1.214    9.1     -15.9
     2   1.066    4.3   1.232    7.1     -13.5
     3   1.140    9.3   1.247    3.0      -8.6
     4   1.207    5.4   1.246    6.2      -3.2


Now that board struggles with thermal, so I swapped the order of testing
(STEAL enabled first, then NO_STEAL) out of curiosity:

          --- base --    --- new ---
groups    time %stdev    time %stdev  %speedup
     1   0.986    8.6   1.218    4.9     -19.1
     2   1.096    5.5   1.290    6.2     -15.1
     3   1.124    5.5   1.237    8.1      -9.2
     4   1.181    8.7   1.238    5.9      -4.7


And actually running the same test twice with NO_STEAL gives me:

          --- base --    --- new ---
groups    time %stdev    time %stdev  %speedup
     1   1.005    8.3   1.225    5.5     -18.0
     2   1.126    6.4   1.220    7.1      -7.8
     3   1.199    5.8   1.264    3.2      -5.2
     4   1.167    4.6   1.314    8.5     -11.2


We might need some other benchmark to test this. Or a much bigger fan...

[1]: https://lore.kernel.org/lkml/0eaa3ee9-64d6-4739-eec9-e28befa0e97f@oracle.com/

>> Regards,
>> Vincent
> 
> Thanks Vincent.  What is the value of /proc/sys/kernel/sched_wakeup_granularity_ns?
> Try 15000000.  Your 8-core system is heavily overloaded with 40 * groups tasks,
> and I suspect preemptions are killing performance.
> 

While hackbench is not super representative of "real life", I wonder if we
shouldn't do something about the default if using stealing suggests it
(wild speculation here).

> I have a python script to post-process schedstat files, but it does many things
> and is large and I am not ready to share it.  I can write a short bash script if
> that would help.
> 
> - Steve
>