Greeting, FYI, we noticed a -4.9% regression of will-it-scale.per_thread_ops due to commit: commit: 12aceb89b0bce19eb89735f9de7a9983e4f0adae ("eventfd: convert to f_op->read_iter()") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: will-it-scale on test machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory with following parameters: nr_task: 16 mode: thread test: eventfd1 cpufreq_governor: performance ucode: 0x5002f01 test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two. test-url: https://github.com/antonblanchard/will-it-scale In addition to that, the commit also has significant impact on the following tests: +------------------+---------------------------------------------------------------------------+ | testcase: change | will-it-scale: will-it-scale.per_process_ops -3.1% regression | | test machine | 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory | | test parameters | cpufreq_governor=performance | | | mode=process | | | nr_task=100% | | | test=eventfd1 | | | ucode=0x2006906 | +------------------+---------------------------------------------------------------------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode: gcc-9/performance/x86_64-rhel-8.3/thread/16/debian-10.4-x86_64-20200603.cgz/lkp-csl-2ap2/eventfd1/will-it-scale/0x5002f01 commit: v5.7-rc1 12aceb89b0 ("eventfd: convert to f_op->read_iter()") v5.7-rc1 12aceb89b0bce19eb89735f9de7 ---------------- --------------------------- %stddev %change %stddev \ | \ 2643543 -4.9% 2513964 will-it-scale.per_thread_ops 42296699 -4.9% 40223447 will-it-scale.workload 3.50 ± 14% +58250.0% 2042 meminfo.Active(file) 3.878e+08 ±136% -99.8% 691264 ± 25% cpuidle.C1.time 3295968 ±150% -99.3% 21481 ± 7% cpuidle.C1.usage 0.50 ±100% +1e+05% 510.25 proc-vmstat.nr_active_file 286480 +1.1% 289671 proc-vmstat.nr_file_pages 67.00 +229.1% 220.50 proc-vmstat.nr_inactive_file 25892 +1.2% 26192 proc-vmstat.nr_kernel_stack 0.50 ±100% +1e+05% 510.25 proc-vmstat.nr_zone_active_file 67.00 +229.1% 220.50 proc-vmstat.nr_zone_inactive_file 12833 ± 7% +27.6% 16380 ± 12% sched_debug.cfs_rq:/.load.avg 20081 ± 2% +26.4% 25384 ± 21% sched_debug.cfs_rq:/.load.stddev 14.69 ± 7% +26.1% 18.52 ± 13% sched_debug.cfs_rq:/.load_avg.avg 0.22 ± 7% +24.9% 0.28 ± 12% sched_debug.cfs_rq:/.nr_running.avg 0.36 ± 3% +9.1% 0.39 ± 5% sched_debug.cfs_rq:/.nr_running.stddev 264.38 ± 6% +20.3% 317.93 ± 9% sched_debug.cfs_rq:/.runnable_avg.avg 226.43 ± 6% +23.5% 279.72 ± 11% sched_debug.cfs_rq:/.util_avg.avg 355.64 ± 2% +9.1% 388.01 ± 5% sched_debug.cfs_rq:/.util_avg.stddev 3153 ± 9% -9.8% 2845 ± 3% slabinfo.PING.active_objs 3153 ± 9% -9.8% 2845 ± 3% slabinfo.PING.num_objs 803.25 ± 5% +25.4% 1007 ± 9% slabinfo.mnt_cache.active_objs 803.25 ± 5% +25.4% 1007 ± 9% slabinfo.mnt_cache.num_objs 248.00 +137.5% 589.00 ± 9% slabinfo.numa_policy.active_objs 248.00 +137.5% 589.00 ± 9% slabinfo.numa_policy.num_objs 5774 ± 7% -7.5% 5339 ± 3% slabinfo.sock_inode_cache.active_objs 5774 ± 7% -7.5% 5339 ± 3% slabinfo.sock_inode_cache.num_objs 1293 ± 3% -12.6% 1130 ± 4% slabinfo.task_group.active_objs 1293 ± 3% -12.6% 1130 ± 4% slabinfo.task_group.num_objs 12747 ± 34% +127.4% 28983 ± 19% numa-vmstat.node0.nr_active_anon 12609 ± 34% +129.8% 28971 ± 19% numa-vmstat.node0.nr_anon_pages 0.00 +1.1e+104% 108.75 ± 71% numa-vmstat.node0.nr_inactive_file 138.25 ± 33% +90.2% 263.00 ± 7% numa-vmstat.node0.nr_page_table_pages 6730 ± 10% +37.4% 9248 ± 24% numa-vmstat.node0.nr_slab_reclaimable 21494 ± 10% +23.2% 26480 ± 6% numa-vmstat.node0.nr_slab_unreclaimable 12747 ± 34% +127.4% 28983 ± 19% numa-vmstat.node0.nr_zone_active_anon 0.00 +1.1e+104% 108.75 ± 71% numa-vmstat.node0.nr_zone_inactive_file 29815 ± 40% -76.5% 6993 ± 92% numa-vmstat.node2.nr_active_anon 29587 ± 40% -77.0% 6796 ± 95% numa-vmstat.node2.nr_anon_pages 295.50 ± 18% -47.6% 154.75 ± 57% numa-vmstat.node2.nr_page_table_pages 23534 ± 11% -18.5% 19171 ± 9% numa-vmstat.node2.nr_slab_unreclaimable 29815 ± 40% -76.5% 6993 ± 92% numa-vmstat.node2.nr_zone_active_anon 50993 ± 34% +129.3% 116952 ± 19% numa-meminfo.node0.Active 50993 ± 34% +127.3% 115930 ± 19% numa-meminfo.node0.Active(anon) 30272 ± 57% +178.1% 84174 ± 22% numa-meminfo.node0.AnonHugePages 50443 ± 35% +129.7% 115883 ± 19% numa-meminfo.node0.AnonPages 26924 ± 10% +37.4% 36996 ± 24% numa-meminfo.node0.KReclaimable 679911 ± 5% +16.6% 792914 ± 6% numa-meminfo.node0.MemUsed 552.50 ± 33% +90.7% 1053 ± 7% numa-meminfo.node0.PageTables 26924 ± 10% +37.4% 36996 ± 24% numa-meminfo.node0.SReclaimable 85979 ± 10% +23.2% 105924 ± 6% numa-meminfo.node0.SUnreclaim 112904 ± 10% +26.6% 142921 ± 10% numa-meminfo.node0.Slab 119254 ± 40% -76.1% 28463 ± 90% numa-meminfo.node2.Active 119250 ± 40% -76.6% 27958 ± 92% numa-meminfo.node2.Active(anon) 82482 ± 44% -84.3% 12981 ±141% numa-meminfo.node2.AnonHugePages 118340 ± 40% -77.0% 27170 ± 95% numa-meminfo.node2.AnonPages 804692 ± 8% -18.5% 655591 ± 5% numa-meminfo.node2.MemUsed 1186 ± 18% -48.3% 613.50 ± 56% numa-meminfo.node2.PageTables 94139 ± 11% -18.5% 76685 ± 9% numa-meminfo.node2.SUnreclaim 16346 ±108% -95.9% 666.00 ± 63% interrupts.31:PCI-MSI.524289-edge.eth0-TxRx-0 1.00 ± 70% +13250.0% 133.50 ±170% interrupts.CPU1.TLB:TLB_shootdowns 16346 ±108% -95.9% 666.00 ± 63% interrupts.CPU10.31:PCI-MSI.524289-edge.eth0-TxRx-0 1011 ± 16% -18.4% 825.00 ± 3% interrupts.CPU106.CAL:Function_call_interrupts 2177 ± 34% +129.6% 4999 ± 24% interrupts.CPU11.NMI:Non-maskable_interrupts 2177 ± 34% +129.6% 4999 ± 24% interrupts.CPU11.PMI:Performance_monitoring_interrupts 57.25 ± 73% -96.1% 2.25 ± 36% interrupts.CPU148.RES:Rescheduling_interrupts 1882 ±168% -99.6% 8.00 ± 66% interrupts.CPU149.RES:Rescheduling_interrupts 115.00 ± 8% +23.7% 142.25 ± 17% interrupts.CPU154.NMI:Non-maskable_interrupts 115.00 ± 8% +23.7% 142.25 ± 17% interrupts.CPU154.PMI:Performance_monitoring_interrupts 918.50 ± 8% -7.6% 848.50 ± 3% interrupts.CPU186.CAL:Function_call_interrupts 1526 ± 4% -22.1% 1188 ± 19% interrupts.CPU2.CAL:Function_call_interrupts 894.75 ± 9% -8.1% 822.25 ± 3% interrupts.CPU23.CAL:Function_call_interrupts 2856 +80.4% 5153 ± 26% interrupts.CPU5.NMI:Non-maskable_interrupts 2856 +80.4% 5153 ± 26% interrupts.CPU5.PMI:Performance_monitoring_interrupts 100.50 ± 24% +24.1% 124.75 ± 7% interrupts.CPU52.NMI:Non-maskable_interrupts 100.50 ± 24% +24.1% 124.75 ± 7% interrupts.CPU52.PMI:Performance_monitoring_interrupts 2500 ± 24% +71.4% 4285 ± 33% interrupts.CPU6.NMI:Non-maskable_interrupts 2500 ± 24% +71.4% 4285 ± 33% interrupts.CPU6.PMI:Performance_monitoring_interrupts 2143 ± 33% +133.3% 4999 ± 24% interrupts.CPU9.NMI:Non-maskable_interrupts 2143 ± 33% +133.3% 4999 ± 24% interrupts.CPU9.PMI:Performance_monitoring_interrupts 1574 ± 4% -14.0% 1355 ± 16% interrupts.CPU97.CAL:Function_call_interrupts 2499 ± 24% +219.7% 7992 ± 16% interrupts.CPU98.NMI:Non-maskable_interrupts 2499 ± 24% +219.7% 7992 ± 16% interrupts.CPU98.PMI:Performance_monitoring_interrupts 1595 ± 4% -7.6% 1474 ± 5% interrupts.CPU99.CAL:Function_call_interrupts 1.375e+10 +7.4% 1.477e+10 perf-stat.i.branch-instructions 1.27 ± 5% -0.2 1.05 perf-stat.i.branch-miss-rate% 1.745e+08 ± 5% -11.3% 1.547e+08 perf-stat.i.branch-misses 13.19 ± 70% +15.2 28.43 ± 13% perf-stat.i.cache-miss-rate% 4337067 ± 3% +46.7% 6363040 ± 20% perf-stat.i.cache-misses 0.79 -6.0% 0.74 perf-stat.i.cpi 5.591e+10 +1.6% 5.683e+10 perf-stat.i.cpu-cycles 13010 ± 3% -28.1% 9353 ± 19% perf-stat.i.cycles-between-cache-misses 2.092e+10 +6.9% 2.237e+10 perf-stat.i.dTLB-loads 1.355e+10 +7.9% 1.461e+10 perf-stat.i.dTLB-stores 98.43 -0.9 97.49 perf-stat.i.iTLB-load-miss-rate% 1.616e+08 -6.6% 1.508e+08 perf-stat.i.iTLB-load-misses 7.059e+10 +8.2% 7.638e+10 perf-stat.i.instructions 437.94 +15.8% 507.28 perf-stat.i.instructions-per-iTLB-miss 1.26 +6.4% 1.34 perf-stat.i.ipc 0.29 +1.6% 0.30 perf-stat.i.metric.GHz 251.50 +7.2% 269.67 perf-stat.i.metric.M/sec 334380 ± 35% +89.2% 632805 ± 2% perf-stat.i.node-load-misses 28946 ± 39% +155.0% 73824 ± 37% perf-stat.i.node-loads 1.27 ± 5% -0.2 1.05 perf-stat.overall.branch-miss-rate% 13.09 ± 70% +15.2 28.32 ± 13% perf-stat.overall.cache-miss-rate% 0.79 -6.1% 0.74 perf-stat.overall.cpi 12909 ± 3% -28.0% 9296 ± 19% perf-stat.overall.cycles-between-cache-misses 98.46 -0.9 97.53 perf-stat.overall.iTLB-load-miss-rate% 436.98 +15.9% 506.42 perf-stat.overall.instructions-per-iTLB-miss 1.26 +6.4% 1.34 perf-stat.overall.ipc 502424 +13.9% 572023 perf-stat.overall.path-length 1.37e+10 +7.4% 1.472e+10 perf-stat.ps.branch-instructions 1.739e+08 ± 5% -11.3% 1.542e+08 perf-stat.ps.branch-misses 4322393 ± 3% +46.7% 6341797 ± 20% perf-stat.ps.cache-misses 5.572e+10 +1.6% 5.664e+10 perf-stat.ps.cpu-cycles 2.085e+10 +6.9% 2.229e+10 perf-stat.ps.dTLB-loads 1.35e+10 +7.9% 1.456e+10 perf-stat.ps.dTLB-stores 1.61e+08 -6.6% 1.503e+08 perf-stat.ps.iTLB-load-misses 7.036e+10 +8.2% 7.612e+10 perf-stat.ps.instructions 333343 ± 35% +89.2% 630738 ± 2% perf-stat.ps.node-load-misses 28896 ± 39% +154.7% 73601 ± 37% perf-stat.ps.node-loads 2.125e+13 +8.3% 2.301e+13 perf-stat.total.instructions 9060 ±112% -95.5% 403.25 ± 48% softirqs.CPU10.NET_RX 23552 ± 16% -17.9% 19348 ± 4% softirqs.CPU10.RCU 18458 ± 12% -14.7% 15740 ± 3% softirqs.CPU100.RCU 24546 ± 9% -20.0% 19634 ± 8% softirqs.CPU101.RCU 24720 ± 9% -18.5% 20145 softirqs.CPU102.RCU 18180 ± 10% -13.4% 15744 ± 3% softirqs.CPU104.RCU 24427 ± 7% -14.8% 20806 ± 2% softirqs.CPU105.RCU 19491 ± 12% -19.8% 15638 ± 2% softirqs.CPU106.RCU 24007 ± 7% -14.4% 20545 ± 3% softirqs.CPU107.RCU 18194 ± 12% -14.2% 15616 ± 3% softirqs.CPU108.RCU 23298 ± 5% -15.4% 19717 ± 10% softirqs.CPU109.RCU 19120 ± 6% -11.1% 17006 softirqs.CPU11.RCU 19182 ± 14% -18.0% 15723 softirqs.CPU110.RCU 24531 ± 9% -15.9% 20631 softirqs.CPU111.RCU 17894 ± 10% -15.7% 15090 ± 2% softirqs.CPU112.RCU 17855 ± 9% -14.7% 15225 ± 3% softirqs.CPU113.RCU 18217 ± 9% -18.0% 14937 ± 3% softirqs.CPU114.RCU 17620 ± 8% -14.8% 15005 ± 3% softirqs.CPU115.RCU 17715 ± 9% -17.9% 14551 softirqs.CPU116.RCU 18005 ± 10% -19.0% 14576 softirqs.CPU117.RCU 18090 ± 10% -16.2% 15152 ± 2% softirqs.CPU118.RCU 17980 ± 11% -18.6% 14638 ± 4% softirqs.CPU119.RCU 16931 ± 6% -11.0% 15064 ± 3% softirqs.CPU120.RCU 17142 ± 8% -11.3% 15212 ± 7% softirqs.CPU123.RCU 114880 ± 2% -8.9% 104633 ± 5% softirqs.CPU123.TIMER 113579 ± 2% -8.6% 103763 ± 5% softirqs.CPU124.TIMER 17329 ± 10% -10.9% 15444 ± 4% softirqs.CPU126.RCU 20601 ± 10% -15.4% 17424 ± 4% softirqs.CPU129.RCU 20559 ± 9% -15.3% 17412 ± 4% softirqs.CPU130.RCU 20261 ± 9% -15.0% 17220 ± 4% softirqs.CPU134.RCU 23453 ± 14% -16.0% 19709 ± 2% softirqs.CPU14.RCU 114208 ± 2% -8.5% 104503 ± 5% softirqs.CPU140.TIMER 114437 -8.6% 104605 ± 5% softirqs.CPU142.TIMER 120886 ± 10% -12.5% 105763 ± 6% softirqs.CPU147.TIMER 19964 ± 8% -11.0% 17775 ± 2% softirqs.CPU148.RCU 19678 ± 8% -14.8% 16772 softirqs.CPU15.RCU 20092 ± 8% -12.2% 17639 ± 2% softirqs.CPU150.RCU 19591 ± 7% -11.1% 17425 ± 2% softirqs.CPU151.RCU 19463 ± 7% -11.5% 17232 ± 2% softirqs.CPU152.RCU 19682 ± 7% -9.9% 17732 ± 3% softirqs.CPU153.RCU 18837 ± 6% -8.3% 17268 ± 4% softirqs.CPU158.RCU 17991 ± 8% -11.6% 15911 ± 2% softirqs.CPU16.RCU 120462 ± 9% -12.9% 104932 ± 5% softirqs.CPU164.TIMER 20223 ± 5% -13.7% 17457 ± 2% softirqs.CPU167.RCU 18928 ± 8% -12.2% 16611 ± 2% softirqs.CPU172.RCU 19627 ± 11% -14.5% 16788 ± 3% softirqs.CPU176.RCU 19488 ± 10% -14.6% 16637 ± 4% softirqs.CPU177.RCU 19536 ± 10% -15.4% 16532 ± 2% softirqs.CPU179.RCU 18402 ± 8% -11.8% 16231 ± 3% softirqs.CPU18.RCU 19186 ± 9% -13.3% 16630 ± 3% softirqs.CPU181.RCU 19542 ± 11% -14.4% 16728 ± 3% softirqs.CPU183.RCU 19089 ± 6% -13.0% 16614 ± 3% softirqs.CPU186.RCU 19192 ± 8% -13.7% 16568 ± 3% softirqs.CPU189.RCU 18925 ± 6% -12.3% 16596 ± 2% softirqs.CPU190.RCU 18723 ± 5% -10.5% 16750 softirqs.CPU191.RCU 24020 ± 13% -25.2% 17965 ± 7% softirqs.CPU2.RCU 3669 ± 2% +662.4% 27973 ± 50% softirqs.CPU2.SCHED 18222 ± 8% -12.6% 15922 ± 2% softirqs.CPU20.RCU 18326 ± 8% -14.0% 15756 softirqs.CPU21.RCU 18417 ± 9% -10.3% 16526 ± 2% softirqs.CPU22.RCU 18339 ± 9% -11.6% 16215 ± 4% softirqs.CPU23.RCU 17242 ± 5% -14.1% 14818 ± 8% softirqs.CPU24.RCU 18057 ± 11% -15.7% 15215 softirqs.CPU26.RCU 106145 ± 4% -7.9% 97809 softirqs.CPU26.TIMER 17835 ± 5% -14.3% 15277 ± 2% softirqs.CPU27.RCU 19771 ± 9% -13.5% 17094 ± 2% softirqs.CPU3.RCU 17828 ± 11% -14.2% 15296 softirqs.CPU30.RCU 20422 ± 10% -13.3% 17700 ± 3% softirqs.CPU34.RCU 124749 ± 17% -18.3% 101976 ± 2% softirqs.CPU35.TIMER 20348 ± 8% -13.9% 17514 ± 2% softirqs.CPU38.RCU 21945 ± 19% -22.0% 17112 ± 4% softirqs.CPU39.RCU 23685 ± 14% -19.5% 19057 ± 4% softirqs.CPU4.RCU 21144 ± 9% -24.0% 16072 ± 16% softirqs.CPU40.RCU 20768 ± 8% -14.9% 17675 ± 3% softirqs.CPU42.RCU 19967 ± 4% -11.7% 17627 ± 3% softirqs.CPU44.RCU 112341 ± 6% -9.5% 101623 ± 2% softirqs.CPU44.TIMER 20051 ± 10% -11.9% 17657 ± 3% softirqs.CPU46.RCU 19623 ± 10% -14.7% 16734 ± 7% softirqs.CPU47.RCU 40047 ± 4% -15.2% 33972 ± 19% softirqs.CPU49.SCHED 20279 ± 4% -14.9% 17250 ± 4% softirqs.CPU52.RCU 19933 ± 7% -12.0% 17532 ± 2% softirqs.CPU53.RCU 20262 ± 8% -15.8% 17065 ± 12% softirqs.CPU54.RCU 20031 ± 8% -13.6% 17308 ± 6% softirqs.CPU55.RCU 20272 ± 7% -11.2% 17998 softirqs.CPU56.RCU 19967 ± 6% -12.4% 17489 ± 2% softirqs.CPU57.RCU 20424 ± 7% -12.3% 17919 ± 3% softirqs.CPU58.RCU 20110 ± 9% -14.5% 17200 ± 4% softirqs.CPU6.RCU 19747 ± 5% -14.3% 16924 ± 8% softirqs.CPU68.RCU 119401 ± 10% -14.4% 102217 ± 3% softirqs.CPU68.TIMER 20986 ± 9% -18.2% 17165 ± 2% softirqs.CPU7.RCU 19087 ± 8% -11.5% 16888 softirqs.CPU74.RCU 19585 ± 8% -14.0% 16846 ± 3% softirqs.CPU76.RCU 19485 ± 9% -12.8% 16982 softirqs.CPU77.RCU 19068 ± 2% -10.4% 17078 softirqs.CPU78.RCU 23660 ± 13% -17.2% 19600 ± 3% softirqs.CPU8.RCU 20245 ± 10% -14.9% 17224 softirqs.CPU80.RCU 20171 ± 10% -15.7% 17003 ± 2% softirqs.CPU81.RCU 110104 ± 5% -8.3% 100998 ± 3% softirqs.CPU81.TIMER 20158 ± 8% -13.7% 17391 ± 2% softirqs.CPU82.RCU 19872 ± 9% -13.1% 17277 ± 2% softirqs.CPU85.RCU 20317 ± 9% -15.5% 17165 ± 2% softirqs.CPU89.RCU 20099 ± 11% -17.3% 16614 ± 3% softirqs.CPU9.RCU 19696 ± 7% -14.0% 16941 softirqs.CPU90.RCU 20096 ± 8% -14.9% 17096 softirqs.CPU91.RCU 20002 ± 8% -15.0% 17011 ± 2% softirqs.CPU93.RCU 19552 ± 6% -13.3% 16942 softirqs.CPU94.RCU 109666 ± 5% -8.5% 100332 ± 3% softirqs.CPU94.TIMER 25091 ± 9% -20.0% 20077 ± 9% softirqs.CPU97.RCU 24912 ± 10% -15.8% 20976 softirqs.CPU99.RCU 15316 ± 68% -80.9% 2933 ± 57% softirqs.NET_RX 3763395 ± 8% -12.4% 3298303 ± 2% softirqs.RCU 7.23 -7.2 0.00 perf-profile.calltrace.cycles-pp.eventfd_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 32.55 -5.7 26.82 ± 8% perf-profile.calltrace.cycles-pp.__libc_write 21.11 -3.7 17.42 ± 8% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_write 20.01 -3.5 16.55 ± 8% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write 17.96 -2.9 15.09 ± 8% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write 13.27 -2.1 11.18 ± 8% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write 7.63 -1.4 6.25 ± 9% perf-profile.calltrace.cycles-pp.eventfd_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 7.08 -1.3 5.73 ± 7% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64.__libc_read 7.04 -1.3 5.71 ± 7% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64.__libc_write 3.27 ± 19% -0.9 2.32 ± 21% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 4.08 -0.7 3.34 ± 8% perf-profile.calltrace.cycles-pp._copy_from_user.eventfd_write.vfs_write.ksys_write.do_syscall_64 2.40 ± 8% -0.7 1.68 ± 11% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle 3.48 -0.6 2.84 ± 8% perf-profile.calltrace.cycles-pp.__pthread_enable_asynccancel 1.02 ± 28% -0.6 0.41 ± 57% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state 3.51 -0.5 2.96 ± 8% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.79 -0.5 2.27 ± 7% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read 2.58 -0.5 2.11 ± 8% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.49 ± 8% -0.5 1.04 ± 9% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 2.81 -0.5 2.36 ± 7% perf-profile.calltrace.cycles-pp.__pthread_disable_asynccancel 2.50 -0.4 2.10 ± 10% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write 2.20 -0.4 1.81 ± 9% perf-profile.calltrace.cycles-pp.fsnotify.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.15 -0.4 1.77 ± 8% perf-profile.calltrace.cycles-pp.__fget_files.__fget_light.__fdget_pos.ksys_read.do_syscall_64 2.31 -0.4 1.94 ± 10% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.00 -0.4 1.64 ± 10% perf-profile.calltrace.cycles-pp.__fget_files.__fget_light.__fdget_pos.ksys_write.do_syscall_64 2.62 -0.3 2.27 ± 6% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.60 -0.3 0.27 ±100% perf-profile.calltrace.cycles-pp.fsnotify_parent.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.68 ± 2% -0.3 1.37 ± 10% perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.eventfd_write.vfs_write.ksys_write.do_syscall_64 1.04 -0.3 0.74 ± 7% perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.__libc_read 1.26 -0.3 0.97 ± 7% perf-profile.calltrace.cycles-pp.copy_user_generic_unrolled._copy_from_user.eventfd_write.vfs_write.ksys_write 1.23 -0.3 0.97 ± 8% perf-profile.calltrace.cycles-pp.fput_many.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write 1.20 ± 2% -0.2 0.99 ± 8% perf-profile.calltrace.cycles-pp.fput_many.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read 0.91 -0.2 0.74 ± 8% perf-profile.calltrace.cycles-pp.fsnotify.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.84 ± 3% -0.1 0.70 ± 8% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string._copy_from_user.eventfd_write.vfs_write.ksys_write 0.64 ± 3% -0.1 0.53 ± 6% perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.__libc_write 0.00 +0.7 0.70 ± 6% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout._copy_to_iter.eventfd_read.new_sync_read 0.00 +0.7 0.73 ± 8% perf-profile.calltrace.cycles-pp.__might_fault._copy_to_iter.eventfd_read.new_sync_read.vfs_read 0.00 +1.0 0.97 ± 7% perf-profile.calltrace.cycles-pp.copy_user_generic_unrolled.copyout._copy_to_iter.eventfd_read.new_sync_read 1.13 ± 25% +1.0 2.14 ± 18% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_kernel.secondary_startup_64 1.13 ± 25% +1.0 2.14 ± 18% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_kernel.secondary_startup_64 1.13 ± 25% +1.0 2.14 ± 18% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_kernel.secondary_startup_64 1.13 ± 25% +1.0 2.14 ± 18% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_kernel 1.13 ± 25% +1.0 2.14 ± 18% perf-profile.calltrace.cycles-pp.start_kernel.secondary_startup_64 0.00 +1.4 1.41 ± 9% perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.eventfd_read.new_sync_read.vfs_read.ksys_read 0.00 +2.2 2.18 ± 7% perf-profile.calltrace.cycles-pp.copyout._copy_to_iter.eventfd_read.new_sync_read.vfs_read 0.00 +4.2 4.20 ± 7% perf-profile.calltrace.cycles-pp._copy_to_iter.eventfd_read.new_sync_read.vfs_read.ksys_read 0.00 +7.2 7.19 ± 8% perf-profile.calltrace.cycles-pp.eventfd_read.new_sync_read.vfs_read.ksys_read.do_syscall_64 0.00 +8.5 8.55 ± 8% perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 29.26 +9.0 38.28 ± 12% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64 29.26 +9.0 38.28 ± 12% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64 29.25 +9.0 38.27 ± 12% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 25.36 ± 4% +9.8 35.17 ± 15% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 30.39 +10.0 40.42 ± 12% perf-profile.calltrace.cycles-pp.secondary_startup_64 32.47 -5.8 26.70 ± 8% perf-profile.children.cycles-pp.__libc_write 18.08 -2.9 15.18 ± 8% perf-profile.children.cycles-pp.ksys_write 13.39 -2.1 11.28 ± 8% perf-profile.children.cycles-pp.vfs_write 9.51 -1.8 7.69 ± 8% perf-profile.children.cycles-pp.entry_SYSCALL_64 7.73 -1.4 6.36 ± 9% perf-profile.children.cycles-pp.eventfd_write 6.55 -1.3 5.27 ± 8% perf-profile.children.cycles-pp.syscall_return_via_sysret 6.31 -1.0 5.34 ± 7% perf-profile.children.cycles-pp.security_file_permission 5.51 -1.0 4.55 ± 9% perf-profile.children.cycles-pp.__fdget_pos 5.13 -0.9 4.24 ± 9% perf-profile.children.cycles-pp.__fget_light 3.15 ± 9% -0.8 2.37 ± 11% perf-profile.children.cycles-pp.apic_timer_interrupt 4.27 -0.8 3.50 ± 9% perf-profile.children.cycles-pp.__fget_files 4.17 -0.8 3.41 ± 8% perf-profile.children.cycles-pp._copy_from_user 2.62 ± 7% -0.8 1.86 ± 11% perf-profile.children.cycles-pp.smp_apic_timer_interrupt 2.54 -0.7 1.84 ± 9% perf-profile.children.cycles-pp.__might_fault 3.48 -0.6 2.84 ± 8% perf-profile.children.cycles-pp.__pthread_enable_asynccancel 3.13 -0.5 2.59 ± 8% perf-profile.children.cycles-pp.fsnotify 1.15 ± 24% -0.5 0.62 ± 10% perf-profile.children.cycles-pp.__hrtimer_run_queues 1.69 ± 7% -0.5 1.19 ± 10% perf-profile.children.cycles-pp.hrtimer_interrupt 2.43 -0.5 1.96 ± 8% perf-profile.children.cycles-pp.fput_many 2.81 -0.5 2.36 ± 7% perf-profile.children.cycles-pp.__pthread_disable_asynccancel 1.29 -0.4 0.86 ± 11% perf-profile.children.cycles-pp.___might_sleep 1.67 ± 2% -0.4 1.28 ± 8% perf-profile.children.cycles-pp.fsnotify_parent 0.79 ± 22% -0.4 0.41 ± 11% perf-profile.children.cycles-pp.tick_sched_timer 0.66 ± 29% -0.3 0.33 ± 13% perf-profile.children.cycles-pp.tick_sched_handle 0.64 ± 28% -0.3 0.33 ± 12% perf-profile.children.cycles-pp.update_process_times 1.17 -0.2 0.99 ± 10% perf-profile.children.cycles-pp.aa_file_perm 0.58 -0.1 0.45 ± 8% perf-profile.children.cycles-pp.__indirect_thunk_start 0.28 ± 13% -0.1 0.16 ± 12% perf-profile.children.cycles-pp.irq_enter 0.40 ± 12% -0.1 0.28 ± 16% perf-profile.children.cycles-pp.__softirqentry_text_start 0.69 ± 2% -0.1 0.58 ± 7% perf-profile.children.cycles-pp.__might_sleep 0.28 ± 15% -0.1 0.18 ± 13% perf-profile.children.cycles-pp.scheduler_tick 0.30 ± 12% -0.1 0.21 ± 14% perf-profile.children.cycles-pp.apparmor_file_permission 0.19 ± 12% -0.1 0.10 ± 14% perf-profile.children.cycles-pp.tick_irq_enter 0.10 ± 18% -0.1 0.04 ± 57% perf-profile.children.cycles-pp.load_balance 0.08 ± 21% -0.1 0.03 ±100% perf-profile.children.cycles-pp.update_rq_clock 0.17 ± 8% -0.1 0.12 ± 12% perf-profile.children.cycles-pp.rebalance_domains 0.26 ± 3% -0.0 0.21 ± 7% perf-profile.children.cycles-pp.testcase 0.15 ± 14% -0.0 0.11 ± 10% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler 0.07 ± 12% -0.0 0.03 ±100% perf-profile.children.cycles-pp.update_sd_lb_stats 0.22 ± 6% -0.0 0.18 ± 2% perf-profile.children.cycles-pp.fput 0.15 ± 14% -0.0 0.11 ± 10% perf-profile.children.cycles-pp.sched_clock_cpu 0.08 ± 14% -0.0 0.04 ± 57% perf-profile.children.cycles-pp.find_busiest_group 0.11 ± 11% -0.0 0.08 ± 5% perf-profile.children.cycles-pp.irqtime_account_irq 0.13 ± 3% -0.0 0.10 ± 15% perf-profile.children.cycles-pp.read@plt 0.08 ± 6% -0.0 0.06 ± 9% perf-profile.children.cycles-pp.__vfs_write 0.47 ± 4% +0.1 0.59 ± 9% perf-profile.children.cycles-pp.rw_verify_area 0.00 +0.2 0.19 ± 8% perf-profile.children.cycles-pp.iov_iter_init 0.91 ± 2% +0.6 1.49 ± 7% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 1.57 +0.9 2.45 ± 7% perf-profile.children.cycles-pp.copy_user_generic_unrolled 1.13 ± 25% +1.0 2.14 ± 18% perf-profile.children.cycles-pp.start_kernel 0.00 +2.2 2.18 ± 7% perf-profile.children.cycles-pp.copyout 0.00 +4.3 4.27 ± 7% perf-profile.children.cycles-pp._copy_to_iter 0.00 +8.7 8.69 ± 8% perf-profile.children.cycles-pp.new_sync_read 29.26 +9.0 38.28 ± 12% perf-profile.children.cycles-pp.start_secondary 28.83 ± 2% +9.3 38.15 ± 13% perf-profile.children.cycles-pp.cpuidle_enter 28.83 ± 2% +9.3 38.14 ± 13% perf-profile.children.cycles-pp.cpuidle_enter_state 25.36 ± 4% +9.8 35.17 ± 15% perf-profile.children.cycles-pp.intel_idle 30.39 +10.0 40.42 ± 12% perf-profile.children.cycles-pp.secondary_startup_64 30.39 +10.0 40.42 ± 12% perf-profile.children.cycles-pp.cpu_startup_entry 30.39 +10.0 40.42 ± 12% perf-profile.children.cycles-pp.do_idle 6.55 -1.3 5.26 ± 8% perf-profile.self.cycles-pp.syscall_return_via_sysret 4.85 -0.9 3.91 ± 8% perf-profile.self.cycles-pp.entry_SYSCALL_64 3.35 -0.9 2.50 ± 7% perf-profile.self.cycles-pp.do_syscall_64 4.13 -0.7 3.40 ± 9% perf-profile.self.cycles-pp.__fget_files 2.30 -0.7 1.61 ± 9% perf-profile.self.cycles-pp.eventfd_read 3.50 -0.7 2.83 ± 7% perf-profile.self.cycles-pp.__libc_write 3.49 -0.6 2.85 ± 7% perf-profile.self.cycles-pp.__libc_read 3.47 -0.6 2.83 ± 8% perf-profile.self.cycles-pp.__pthread_enable_asynccancel 3.00 -0.5 2.49 ± 8% perf-profile.self.cycles-pp.fsnotify 1.57 -0.5 1.09 ± 8% perf-profile.self.cycles-pp.vfs_read 2.79 -0.5 2.34 ± 6% perf-profile.self.cycles-pp.__pthread_disable_asynccancel 2.31 -0.4 1.86 ± 8% perf-profile.self.cycles-pp.fput_many 1.29 -0.4 0.86 ± 11% perf-profile.self.cycles-pp.___might_sleep 2.00 ± 2% -0.4 1.61 ± 8% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 1.57 ± 3% -0.4 1.22 ± 7% perf-profile.self.cycles-pp.fsnotify_parent 1.43 ± 2% -0.3 1.17 ± 10% perf-profile.self.cycles-pp.vfs_write 1.84 -0.3 1.59 ± 10% perf-profile.self.cycles-pp.eventfd_write 1.28 ± 2% -0.2 1.04 ± 8% perf-profile.self.cycles-pp.security_file_permission 1.09 -0.2 0.92 ± 9% perf-profile.self.cycles-pp.aa_file_perm 0.91 ± 4% -0.2 0.76 ± 10% perf-profile.self.cycles-pp.ksys_read 0.63 -0.1 0.48 ± 7% perf-profile.self.cycles-pp.__might_sleep 0.88 ± 2% -0.1 0.74 ± 8% perf-profile.self.cycles-pp.__fget_light 0.54 ± 3% -0.1 0.43 ± 7% perf-profile.self.cycles-pp._copy_from_user 0.46 ± 3% -0.1 0.37 ± 8% perf-profile.self.cycles-pp.__fdget_pos 0.37 ± 3% -0.1 0.30 ± 9% perf-profile.self.cycles-pp.__indirect_thunk_start 0.17 ± 19% -0.1 0.11 ± 17% perf-profile.self.cycles-pp.apparmor_file_permission 0.26 ± 3% -0.0 0.21 ± 7% perf-profile.self.cycles-pp.testcase 0.12 ± 7% -0.0 0.09 ± 4% perf-profile.self.cycles-pp.fput 0.08 ± 6% -0.0 0.06 ± 9% perf-profile.self.cycles-pp.__vfs_write 0.06 ± 6% +0.0 0.10 ± 11% perf-profile.self.cycles-pp.__vfs_read 0.47 ± 4% +0.1 0.57 ± 9% perf-profile.self.cycles-pp.rw_verify_area 0.00 +0.1 0.15 ± 8% perf-profile.self.cycles-pp.iov_iter_init 0.00 +0.2 0.19 ± 11% perf-profile.self.cycles-pp.copyout 0.91 ± 2% +0.6 1.49 ± 7% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 1.50 +0.8 2.35 ± 7% perf-profile.self.cycles-pp.copy_user_generic_unrolled 0.78 ± 73% +0.9 1.65 ± 10% perf-profile.self.cycles-pp.menu_select 0.00 +1.2 1.18 ± 12% perf-profile.self.cycles-pp.new_sync_read 0.00 +1.3 1.26 ± 7% perf-profile.self.cycles-pp._copy_to_iter 25.35 ± 4% +9.8 35.17 ± 15% perf-profile.self.cycles-pp.intel_idle will-it-scale.per_thread_ops 2.7e+06 +----------------------------------------------------------------+ | +..+. | 2.65e+06 |-+ .. +..+.. .+..+.+..+..+..+.+..| |.. .+..+..+..+.+.. .+..+ +.+. | 2.6e+06 |-++ +..+ | | | 2.55e+06 |-+ | | O O O O O O O O | 2.5e+06 |-+ O O O O | | | 2.45e+06 |-+ | | O O O O O O | 2.4e+06 |-+ | | O O | 2.35e+06 +----------------------------------------------------------------+ will-it-scale.workload 4.3e+07 +----------------------------------------------------------------+ 4.25e+07 |-+ .. +.+.. | | .+ +..+. .+..+.+..+..+..+.+..| 4.2e+07 |..+.+..+..+..+.+..+..+.+. +. | 4.15e+07 |-+ | 4.1e+07 |-+ | 4.05e+07 |-+ | | O O O O O O O O O O | 4e+07 |-+ O O | 3.95e+07 |-+ | 3.9e+07 |-+ | 3.85e+07 |-+O O O O O O | | | 3.8e+07 |-+ O O | 3.75e+07 +----------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample *************************************************************************************************** lkp-csl-2ap2: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen