FYI, we noticed a -32.9% improvement of reaim.child_utime due to commit: commit ab522e33f91799661aad47bebb691f241a9f6bb8 ("sched/fair: Fix fixed point arithmetic width for shares and effective load") https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master in testcase: reaim on test machine: 28 threads Intel(R) Xeon(R) CPU E5-2695 v3 @ 2.30GHz with 256G memory with following parameters: runtime: 300s nr_task: 600 test: creat_clo cpufreq_governor: performance REAIM is an updated and improved version of AIM 7 benchmark. Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone git://git.kernel.org/pub/scm/linux/kernel/git/wfg/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/kconfig/nr_task/rootfs/runtime/tbox_group/test/testcase: gcc-6/performance/x86_64-rhel-7.2/600/debian-x86_64-2016-08-31.cgz/300s/lkp-hsw-ep5/creat_clo/reaim commit: 8f37961cf2 ("sched/core, x86/topology: Fix NUMA in package topology bug") ab522e33f9 ("sched/fair: Fix fixed point arithmetic width for shares and effective load") 8f37961cf22304fb ab522e33f91799661aad47bebb ---------------- -------------------------- fail:runs %reproduction fail:runs | | | %stddev %change %stddev \ | \ 7.16 ± 3% -32.9% 4.80 ± 0% reaim.child_utime 61691 ± 2% +20.8% 74498 ± 8% reaim.jobs_per_min 102.82 ± 2% +20.8% 124.16 ± 8% reaim.jobs_per_min_child 63961 ± 3% +17.9% 75402 ± 8% reaim.max_jobs_per_min 58.44 ± 2% -16.7% 48.65 ± 8% reaim.parent_time 1.11 ± 1% -14.3% 0.95 ± 10% reaim.std_dev_time 56976 ± 9% -27.7% 41216 ± 11% reaim.time.involuntary_context_switches 2231 ± 0% +24.8% 2784 ± 1% reaim.time.percent_of_cpu_this_job_got 7371 ± 5% +23.6% 9108 ± 4% reaim.time.system_time 39.58 ± 12% -20.9% 31.30 ± 7% reaim.time.user_time 2574189 ± 1% +90.4% 4900794 ± 18% reaim.time.voluntary_context_switches 20246 ± 1% +24.4% 25194 ± 11% meminfo.Shmem 1651242 ± 9% +16.3% 1920509 ± 8% numa-numastat.node0.local_node 1651257 ± 9% +16.3% 1920520 ± 8% numa-numastat.node0.numa_hit 973623 ± 7% +15.2% 1121509 ± 7% numa-vmstat.node0.numa_hit 973612 ± 7% +15.2% 1121498 ± 7% numa-vmstat.node0.numa_local 669559 ± 9% +42.5% 953890 ± 4% softirqs.RCU 3989816 ± 6% +21.2% 4835284 ± 4% softirqs.TIMER 5060 ± 1% +24.5% 6298 ± 11% proc-vmstat.nr_shmem 235142 ± 8% +18.4% 278460 ± 7% proc-vmstat.numa_pte_updates 6554 ± 3% +28.5% 8419 ± 11% proc-vmstat.pgactivate 22.75 ± 1% +23.1% 28.00 ± 0% vmstat.procs.r 18537 ± 4% +76.1% 32653 ± 13% vmstat.system.cs 59046 ± 0% +1.6% 60013 ± 0% vmstat.system.in 76839 ± 1% +21.4% 93317 ± 2% slabinfo.kmalloc-256.active_objs 1235 ± 1% +22.3% 1511 ± 2% slabinfo.kmalloc-256.active_slabs 79114 ± 1% +22.3% 96764 ± 2% slabinfo.kmalloc-256.num_objs 1235 ± 1% +22.3% 1511 ± 2% slabinfo.kmalloc-256.num_slabs 1.486e+08 ± 8% +170.2% 4.015e+08 ± 21% cpuidle.C1-HSW.time 732700 ± 9% +228.1% 2404155 ± 30% cpuidle.C1-HSW.usage 293274 ± 5% +23.8% 363153 ± 6% cpuidle.C1E-HSW.usage 2.803e+08 ± 6% +27.3% 3.567e+08 ± 6% cpuidle.C3-HSW.time 1177259 ± 4% +37.6% 1620250 ± 7% cpuidle.C3-HSW.usage 1.04e+10 ± 6% -20.8% 8.238e+09 ± 6% cpuidle.C6-HSW.time 11658113 ± 7% -19.9% 9342841 ± 6% cpuidle.C6-HSW.usage 24794280 ± 7% +96.2% 48658115 ± 20% cpuidle.POLL.time 7138 ± 5% +390.7% 35025 ± 32% cpuidle.POLL.usage 42.14 ± 0% +21.8% 51.31 ± 1% turbostat.%Busy 600.50 ± 4% +89.9% 1140 ± 1% turbostat.Avg_MHz 1424 ± 4% +56.0% 2222 ± 0% turbostat.Bzy_MHz 0.91 ± 3% +30.1% 1.19 ± 6% turbostat.CPU%c3 22.96 ± 2% -42.6% 13.18 ± 1% turbostat.CPU%c6 54.50 ± 0% +19.7% 65.25 ± 1% turbostat.CoreTmp 5.81 ± 6% -58.8% 2.40 ± 12% turbostat.Pkg%pc2 60.00 ± 2% +17.1% 70.25 ± 0% turbostat.PkgTmp 88.35 ± 4% +47.2% 130.02 ± 0% turbostat.PkgWatt 77.97 ± 0% +2.7% 80.03 ± 0% turbostat.RAMWatt 1.432e+12 ± 3% +76.7% 2.531e+12 ± 3% perf-stat.branch-instructions 0.23 ± 5% -38.4% 0.14 ± 2% perf-stat.branch-miss-rate% 17.47 ± 3% +51.4% 26.45 ± 3% perf-stat.cache-miss-rate% 3.028e+09 ± 3% +66.8% 5.051e+09 ± 13% perf-stat.cache-misses 6164715 ± 2% +75.4% 10813552 ± 16% perf-stat.context-switches 1.15e+13 ± 2% +87.1% 2.151e+13 ± 4% perf-stat.cpu-cycles 799519 ± 4% -38.3% 493538 ± 5% perf-stat.cpu-migrations 0.05 ± 3% -46.5% 0.02 ± 5% perf-stat.dTLB-load-miss-rate% 1.527e+12 ± 3% +71.2% 2.614e+12 ± 3% perf-stat.dTLB-loads 0.02 ± 3% -30.6% 0.01 ± 8% perf-stat.dTLB-store-miss-rate% 73733750 ± 8% -21.1% 58179599 ± 7% perf-stat.dTLB-store-misses 33.64 ± 3% -22.1% 26.20 ± 5% perf-stat.iTLB-load-miss-rate% 93248101 ± 10% -21.1% 73553819 ± 5% perf-stat.iTLB-load-misses 1.833e+08 ± 5% +13.3% 2.076e+08 ± 7% perf-stat.iTLB-loads 5.949e+12 ± 4% +67.3% 9.954e+12 ± 4% perf-stat.instructions 64245 ± 7% +110.9% 135465 ± 2% perf-stat.instructions-per-iTLB-miss 0.52 ± 1% -10.5% 0.46 ± 1% perf-stat.ipc 1.311e+09 ± 6% +66.6% 2.183e+09 ± 17% perf-stat.node-load-misses 59529399 ± 13% +63.8% 97490648 ± 23% perf-stat.node-loads 1.115e+09 ± 4% +70.9% 1.906e+09 ± 10% perf-stat.node-store-misses 3.569e+08 ± 3% +75.8% 6.274e+08 ± 13% perf-stat.node-stores 60166 ± 0% +23.7% 74404 ± 2% sched_debug.cfs_rq:/.exec_clock.avg 82931 ± 1% +19.6% 99175 ± 1% sched_debug.cfs_rq:/.exec_clock.max 36938 ± 0% +29.5% 47845 ± 3% sched_debug.cfs_rq:/.exec_clock.min 1528331 ± 1% +50.2% 2295237 ± 4% sched_debug.cfs_rq:/.min_vruntime.avg 2131382 ± 2% +43.9% 3067242 ± 3% sched_debug.cfs_rq:/.min_vruntime.max 921219 ± 1% +59.1% 1465415 ± 5% sched_debug.cfs_rq:/.min_vruntime.min 561511 ± 3% +26.6% 710779 ± 1% sched_debug.cfs_rq:/.min_vruntime.stddev 0.37 ± 17% +31.3% 0.49 ± 9% sched_debug.cfs_rq:/.nr_running.avg 7.19 ± 15% +29.4% 9.30 ± 9% sched_debug.cfs_rq:/.runnable_load_avg.avg -551097 ± -3% +19.9% -660673 ± -8% sched_debug.cfs_rq:/.spread0.avg -1158313 ± -3% +28.7% -1490600 ± -5% sched_debug.cfs_rq:/.spread0.min 561585 ± 3% +26.6% 710858 ± 1% sched_debug.cfs_rq:/.spread0.stddev 421.48 ± 13% +29.0% 543.61 ± 4% sched_debug.cfs_rq:/.util_avg.avg 168.61 ± 17% +26.6% 213.52 ± 6% sched_debug.cfs_rq:/.util_avg.stddev 771938 ± 5% -24.8% 580157 ± 4% sched_debug.cpu.avg_idle.avg 1404008 ± 8% -25.4% 1046878 ± 5% sched_debug.cpu.avg_idle.max 102116 ± 28% -83.2% 17179 ± 59% sched_debug.cpu.avg_idle.min 8.12 ± 35% -47.5% 4.26 ± 11% sched_debug.cpu.clock.stddev 8.12 ± 35% -47.5% 4.26 ± 11% sched_debug.cpu.clock_task.stddev 6.89 ± 14% +28.7% 8.87 ± 9% sched_debug.cpu.cpu_load[0].avg 8.23 ± 13% +26.1% 10.38 ± 4% sched_debug.cpu.cpu_load[1].avg 8.36 ± 13% +27.0% 10.62 ± 4% sched_debug.cpu.cpu_load[2].avg 8.56 ± 13% +28.4% 10.99 ± 3% sched_debug.cpu.cpu_load[3].avg 9.05 ± 12% +28.6% 11.64 ± 2% sched_debug.cpu.cpu_load[4].avg 2481 ± 17% +45.2% 3604 ± 12% sched_debug.cpu.curr->pid.avg 2816 ± 10% +20.0% 3379 ± 2% sched_debug.cpu.curr->pid.stddev 776584 ± 9% -25.6% 577905 ± 10% sched_debug.cpu.max_idle_balance_cost.max 46709 ± 26% -75.5% 11429 ± 72% sched_debug.cpu.max_idle_balance_cost.stddev 0.00 ± 7% +10.2% 0.00 ± 3% sched_debug.cpu.next_balance.stddev 92406 ± 0% +11.6% 103109 ± 1% sched_debug.cpu.nr_load_updates.avg 112923 ± 0% +10.5% 124787 ± 0% sched_debug.cpu.nr_load_updates.max 72587 ± 1% +11.5% 80935 ± 1% sched_debug.cpu.nr_load_updates.min 0.38 ± 16% +31.9% 0.50 ± 8% sched_debug.cpu.nr_running.avg 51528 ± 4% +72.6% 88953 ± 13% sched_debug.cpu.nr_switches.avg 70435 ± 4% +74.7% 123077 ± 14% sched_debug.cpu.nr_switches.max 35395 ± 6% +70.1% 60224 ± 10% sched_debug.cpu.nr_switches.min 11963 ± 5% +78.3% 21328 ± 20% sched_debug.cpu.nr_switches.stddev 58157 ± 5% +77.3% 103088 ± 12% sched_debug.cpu.sched_count.avg 77799 ± 5% +80.3% 140261 ± 13% sched_debug.cpu.sched_count.max 39855 ± 6% +75.2% 69834 ± 10% sched_debug.cpu.sched_count.min 13678 ± 6% +79.5% 24558 ± 19% sched_debug.cpu.sched_count.stddev 23236 ± 5% +80.6% 41960 ± 14% sched_debug.cpu.sched_goidle.avg 31734 ± 5% +83.3% 58158 ± 14% sched_debug.cpu.sched_goidle.max 15443 ± 6% +81.4% 28017 ± 12% sched_debug.cpu.sched_goidle.min 5899 ± 5% +79.0% 10557 ± 20% sched_debug.cpu.sched_goidle.stddev 36829 ± 4% +74.1% 64121 ± 10% sched_debug.cpu.ttwu_count.avg 53547 ± 4% +71.0% 91589 ± 11% sched_debug.cpu.ttwu_count.max 22504 ± 4% +78.3% 40131 ± 9% sched_debug.cpu.ttwu_count.min 11824 ± 7% +56.4% 18492 ± 17% sched_debug.cpu.ttwu_count.stddev 0.95 ± 4% -29.9% 0.67 ± 11% perf-profile.calltrace.cycles-pp.____fput.task_work_run.exit_to_usermode_loop.syscall_return_slowpath.entry_SYSCALL_64_fastpath 2.00 ± 3% -17.2% 1.66 ± 8% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.try_to_wake_up.wake_up_q.rwsem_wake.call_rwsem_wake 28.63 ± 4% -24.8% 21.53 ± 9% perf-profile.calltrace.cycles-pp.call_cpuidle.cpu_startup_entry.start_secondary 50.45 ± 3% +22.2% 61.64 ± 4% perf-profile.calltrace.cycles-pp.call_rwsem_down_write_failed.down_write.path_openat.do_filp_open.do_sys_open 3.56 ± 4% -12.7% 3.11 ± 10% perf-profile.calltrace.cycles-pp.call_rwsem_wake.up_write.path_openat.do_filp_open.do_sys_open 28.86 ± 4% -24.7% 21.74 ± 9% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary 28.63 ± 4% -24.8% 21.53 ± 9% perf-profile.calltrace.cycles-pp.cpuidle_enter.call_cpuidle.cpu_startup_entry.start_secondary 28.26 ± 4% -24.8% 21.26 ± 9% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.call_cpuidle.cpu_startup_entry.start_secondary 1.86 ± 7% -21.5% 1.46 ± 8% perf-profile.calltrace.cycles-pp.do_truncate.path_openat.do_filp_open.do_sys_open.sys_creat 51.67 ± 3% +21.2% 62.64 ± 4% perf-profile.calltrace.cycles-pp.down_write.path_openat.do_filp_open.do_sys_open.sys_creat 1.25 ± 4% -48.1% 0.65 ± 59% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.syscall_return_slowpath.entry_SYSCALL_64_fastpath 1.02 ± 4% -21.2% 0.81 ± 10% perf-profile.calltrace.cycles-pp.get_empty_filp.path_openat.do_filp_open.do_sys_open.sys_creat 28.04 ± 4% -25.6% 20.87 ± 9% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.call_cpuidle.cpu_startup_entry 1.75 ± 4% -29.5% 1.23 ± 12% perf-profile.calltrace.cycles-pp.link_path_walk.path_openat.do_filp_open.do_sys_open.sys_creat 1.52 ± 5% -19.2% 1.23 ± 14% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.try_to_wake_up.wake_up_q.rwsem_wake 47.16 ± 4% +24.5% 58.73 ± 4% perf-profile.calltrace.cycles-pp.osq_lock.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write.path_openat 66.74 ± 2% +12.4% 75.03 ± 3% perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_open.sys_creat.entry_SYSCALL_64_fastpath 49.86 ± 3% +22.7% 61.20 ± 4% perf-profile.calltrace.cycles-pp.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write.path_openat.do_filp_open 3.53 ± 4% -12.5% 3.09 ± 10% perf-profile.calltrace.cycles-pp.rwsem_wake.call_rwsem_wake.up_write.path_openat.do_filp_open 28.88 ± 4% -24.7% 21.75 ± 9% perf-profile.calltrace.cycles-pp.start_secondary 1.28 ± 4% -48.0% 0.67 ± 59% perf-profile.calltrace.cycles-pp.syscall_return_slowpath.entry_SYSCALL_64_fastpath 1.13 ± 4% -48.5% 0.58 ± 59% perf-profile.calltrace.cycles-pp.task_work_run.exit_to_usermode_loop.syscall_return_slowpath.entry_SYSCALL_64_fastpath 2.29 ± 3% -17.2% 1.89 ± 8% perf-profile.calltrace.cycles-pp.try_to_wake_up.wake_up_q.rwsem_wake.call_rwsem_wake.up_write 4.17 ± 3% -10.9% 3.71 ± 9% perf-profile.calltrace.cycles-pp.up_write.path_openat.do_filp_open.do_sys_open.sys_creat 0.94 ± 6% -27.2% 0.68 ± 11% perf-profile.calltrace.cycles-pp.vfs_open.path_openat.do_filp_open.do_sys_open.sys_creat 2.39 ± 3% -15.9% 2.01 ± 8% perf-profile.calltrace.cycles-pp.wake_up_q.rwsem_wake.call_rwsem_wake.up_write.path_openat 0.95 ± 4% -29.1% 0.68 ± 11% perf-profile.children.cycles-pp.____fput 1.19 ± 8% -24.9% 0.90 ± 21% perf-profile.children.cycles-pp.__inode_permission 2.05 ± 4% -16.2% 1.72 ± 6% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 28.89 ± 4% -24.8% 21.73 ± 9% perf-profile.children.cycles-pp.call_cpuidle 50.49 ± 3% +22.2% 61.69 ± 4% perf-profile.children.cycles-pp.call_rwsem_down_write_failed 3.57 ± 4% -11.3% 3.17 ± 8% perf-profile.children.cycles-pp.call_rwsem_wake 29.12 ± 4% -24.6% 21.95 ± 9% perf-profile.children.cycles-pp.cpu_startup_entry 28.88 ± 4% -24.8% 21.73 ± 9% perf-profile.children.cycles-pp.cpuidle_enter 28.47 ± 4% -24.7% 21.44 ± 9% perf-profile.children.cycles-pp.cpuidle_enter_state 66.91 ± 2% +12.3% 75.16 ± 3% perf-profile.children.cycles-pp.do_filp_open 67.91 ± 2% +11.7% 75.87 ± 2% perf-profile.children.cycles-pp.do_sys_open 1.87 ± 7% -20.5% 1.48 ± 8% perf-profile.children.cycles-pp.do_truncate 51.83 ± 3% +21.1% 62.78 ± 4% perf-profile.children.cycles-pp.down_write 70.10 ± 1% +10.5% 77.47 ± 2% perf-profile.children.cycles-pp.entry_SYSCALL_64_fastpath 1.25 ± 4% -29.5% 0.88 ± 11% perf-profile.children.cycles-pp.exit_to_usermode_loop 1.02 ± 4% -19.8% 0.82 ± 10% perf-profile.children.cycles-pp.get_empty_filp 1.25 ± 8% -25.5% 0.93 ± 21% perf-profile.children.cycles-pp.inode_permission 28.25 ± 4% -25.5% 21.05 ± 9% perf-profile.children.cycles-pp.intel_idle 1.76 ± 4% -28.9% 1.25 ± 11% perf-profile.children.cycles-pp.link_path_walk 47.22 ± 4% +24.5% 58.80 ± 4% perf-profile.children.cycles-pp.osq_lock 66.75 ± 2% +12.4% 75.04 ± 3% perf-profile.children.cycles-pp.path_openat 49.91 ± 3% +22.7% 61.25 ± 4% perf-profile.children.cycles-pp.rwsem_down_write_failed 1.31 ± 5% -33.6% 0.87 ± 21% perf-profile.children.cycles-pp.rwsem_spin_on_owner 3.54 ± 4% -11.2% 3.14 ± 8% perf-profile.children.cycles-pp.rwsem_wake 0.91 ± 10% -29.8% 0.64 ± 12% perf-profile.children.cycles-pp.security_inode_permission 28.88 ± 4% -24.7% 21.75 ± 9% perf-profile.children.cycles-pp.start_secondary 67.99 ± 2% +11.7% 75.93 ± 2% perf-profile.children.cycles-pp.sys_creat 1.29 ± 4% -29.3% 0.91 ± 11% perf-profile.children.cycles-pp.syscall_return_slowpath 1.13 ± 4% -29.4% 0.80 ± 11% perf-profile.children.cycles-pp.task_work_run 2.33 ± 3% -16.3% 1.94 ± 7% perf-profile.children.cycles-pp.try_to_wake_up 4.25 ± 4% -9.7% 3.84 ± 7% perf-profile.children.cycles-pp.up_write 0.95 ± 6% -26.6% 0.70 ± 12% perf-profile.children.cycles-pp.vfs_open 2.40 ± 3% -14.4% 2.06 ± 7% perf-profile.children.cycles-pp.wake_up_q 28.25 ± 4% -25.5% 21.05 ± 9% perf-profile.self.cycles-pp.intel_idle 47.22 ± 4% +24.5% 58.80 ± 4% perf-profile.self.cycles-pp.osq_lock 1.31 ± 5% -33.6% 0.87 ± 21% perf-profile.self.cycles-pp.rwsem_spin_on_owner Thanks, Xiaolong