Greeting, FYI, we noticed a 10.7% improvement of will-it-scale.per_process_ops due to commit: commit: 8aebfffacfa379ba400da573a5bf9e49634e38cb ("configfs: fix config_item refcnt leak in configfs_rmdir()") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: will-it-scale on test machine: 288 threads Intel(R) Xeon Phi(TM) CPU 7295 @ 1.50GHz with 80G memory with following parameters: nr_task: 100% mode: process test: mmap1 cpufreq_governor: performance ucode: 0x11 test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two. test-url: https://github.com/antonblanchard/will-it-scale In addition to that, the commit also has significant impact on the following tests: +------------------+----------------------------------------------------------------------+ | testcase: change | will-it-scale: will-it-scale.per_thread_ops 11.9% improvement | | test machine | 288 threads Intel(R) Xeon Phi(TM) CPU 7295 @ 1.50GHz with 80G memory | | test parameters | cpufreq_governor=performance | | | mode=thread | | | nr_task=100% | | | test=open1 | | | ucode=0x11 | +------------------+----------------------------------------------------------------------+ | testcase: change | will-it-scale: will-it-scale.per_thread_ops 17.8% improvement | | test machine | 288 threads Intel(R) Xeon Phi(TM) CPU 7295 @ 1.50GHz with 80G memory | | test parameters | cpufreq_governor=performance | | | mode=thread | | | nr_task=100% | | | test=sched_yield | | | ucode=0x11 | +------------------+----------------------------------------------------------------------+ | testcase: change | will-it-scale: will-it-scale.per_process_ops -14.1% regression | | test machine | 288 threads Intel(R) Xeon Phi(TM) CPU 7295 @ 1.50GHz with 80G memory | | test parameters | cpufreq_governor=performance | | | mode=process | | | nr_task=100% | | | test=page_fault2 | | | ucode=0x11 | +------------------+----------------------------------------------------------------------+ Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode: gcc-7/performance/x86_64-rhel-7.6/process/100%/debian-x86_64-20191114.cgz/lkp-knm01/mmap1/will-it-scale/0x11 commit: v5.7-rc3 8aebfffacf ("configfs: fix config_item refcnt leak in configfs_rmdir()") v5.7-rc3 8aebfffacfa379ba400da573a5b ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 5:6 -83% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x :6 17% 1:4 dmesg.WARNING:at_ip_fsnotify_parent/0x 2:6 -33% :4 dmesg.WARNING:at_ip_perf_event_mmap_output/0x %stddev %change %stddev \ | \ 1471 +10.7% 1628 will-it-scale.per_process_ops 158244 -93.8% 9886 will-it-scale.time.maximum_resident_set_size 87613 -93.0% 6106 will-it-scale.time.minor_page_faults 1.17 ± 31% -100.0% 0.00 will-it-scale.time.percent_of_cpu_this_job_got 423766 +10.7% 469037 will-it-scale.workload 45.02 +4.4% 47.01 ± 2% boot-time.boot 0.54 ± 5% +0.2 0.76 ± 8% mpstat.cpu.all.usr% 1381971 +11.0% 1534587 vmstat.memory.cache 360.06 -100.0% 0.00 uptime.boot 14088 ± 3% -100.0% 0.00 uptime.idle 1261069 +11.3% 1403860 meminfo.Cached 48276 ± 22% +48.4% 71641 ± 22% meminfo.Mapped 1096270 +12.8% 1236320 meminfo.Unevictable 315278 +11.3% 350955 proc-vmstat.nr_file_pages 12192 ± 22% +47.3% 17953 ± 22% proc-vmstat.nr_mapped 30130 +8.1% 32577 proc-vmstat.nr_slab_reclaimable 274067 +12.8% 309080 proc-vmstat.nr_unevictable 274067 +12.8% 309080 proc-vmstat.nr_zone_unevictable 1074698 ± 11% -17.1% 890766 ± 5% proc-vmstat.pgfree 565456 +12.4% 635335 numa-meminfo.node0.Unevictable 530814 +13.2% 600985 numa-meminfo.node1.FilePages 17836 ± 6% +27.3% 22702 numa-meminfo.node1.KReclaimable 10865 +14.1% 12399 numa-meminfo.node1.Mapped 623752 +11.9% 697704 numa-meminfo.node1.MemUsed 17836 ± 6% +27.3% 22702 numa-meminfo.node1.SReclaimable 530814 +13.2% 600985 numa-meminfo.node1.Unevictable 141363 +12.4% 158833 numa-vmstat.node0.nr_unevictable 141363 +12.4% 158833 numa-vmstat.node0.nr_zone_unevictable 154650 +11.3% 172062 numa-vmstat.node0.numa_interleave 132703 +13.2% 150246 numa-vmstat.node1.nr_file_pages 2715 +14.1% 3099 numa-vmstat.node1.nr_mapped 4459 ± 6% +27.3% 5675 numa-vmstat.node1.nr_slab_reclaimable 132703 +13.2% 150246 numa-vmstat.node1.nr_unevictable 132703 +13.2% 150246 numa-vmstat.node1.nr_zone_unevictable 155463 +11.0% 172640 numa-vmstat.node1.numa_hit 154878 +11.1% 172055 numa-vmstat.node1.numa_interleave 154891 +11.1% 172068 numa-vmstat.node1.numa_other 108714 ± 3% -8.1% 99959 ± 5% sched_debug.cfs_rq:/.exec_clock.min 35857851 ± 3% -7.8% 33046430 ± 5% sched_debug.cfs_rq:/.min_vruntime.min 566.50 ± 7% -11.5% 501.41 ± 3% sched_debug.cfs_rq:/.util_est_enqueued.avg 2057901 ± 11% +24.7% 2565766 ± 10% sched_debug.cpu.avg_idle.avg 1139755 ± 8% +48.0% 1686683 ± 11% sched_debug.cpu.avg_idle.stddev 608.73 ± 7% -14.8% 518.47 ± 3% sched_debug.cpu.clock.stddev 608.73 ± 7% -14.8% 518.47 ± 3% sched_debug.cpu.clock_task.stddev 2502457 ± 25% +52.1% 3805340 ± 21% sched_debug.cpu.max_idle_balance_cost.max 227444 ± 12% +57.7% 358618 ± 12% sched_debug.cpu.max_idle_balance_cost.stddev 1059 ± 22% -47.5% 556.70 ± 2% sched_debug.cpu.nr_switches.min 418.53 ± 4% -12.7% 365.55 ± 3% sched_debug.cpu.sched_count.min 0.03 ± 82% +1207.4% 0.45 ± 41% sched_debug.rt_rq:/.rt_time.avg 4.65 ±211% +2609.5% 125.93 ± 46% sched_debug.rt_rq:/.rt_time.max 0.30 ±185% +2337.7% 7.41 ± 46% sched_debug.rt_rq:/.rt_time.stddev 65.33 ± 31% +157.1% 168.00 slabinfo.btrfs_extent_map.active_objs 65.33 ± 31% +157.1% 168.00 slabinfo.btrfs_extent_map.num_objs 8668 ± 8% +13.9% 9873 ± 2% slabinfo.eventpoll_pwq.active_objs 8668 ± 8% +13.9% 9873 ± 2% slabinfo.eventpoll_pwq.num_objs 84377 +10.6% 93315 slabinfo.inode_cache.active_objs 1567 +10.6% 1733 slabinfo.inode_cache.active_slabs 84674 +10.6% 93652 slabinfo.inode_cache.num_objs 1567 +10.6% 1733 slabinfo.inode_cache.num_slabs 597.33 ± 14% -27.7% 432.00 ± 19% slabinfo.kmalloc-rcl-128.active_objs 597.33 ± 14% -27.7% 432.00 ± 19% slabinfo.kmalloc-rcl-128.num_objs 6319 ± 6% +23.9% 7831 ± 9% slabinfo.kmalloc-rcl-64.active_objs 6319 ± 6% +23.9% 7831 ± 9% slabinfo.kmalloc-rcl-64.num_objs 28831 +15.7% 33359 slabinfo.radix_tree_node.active_objs 28831 +15.7% 33359 slabinfo.radix_tree_node.num_objs 986.50 ± 5% -100.0% 0.00 slabinfo.skbuff_ext_cache.active_objs 986.50 ± 5% -100.0% 0.00 slabinfo.skbuff_ext_cache.num_objs 5697 ± 33% -34.2% 3746 interrupts.CPU106.NMI:Non-maskable_interrupts 5697 ± 33% -34.2% 3746 interrupts.CPU106.PMI:Performance_monitoring_interrupts 6325 ± 28% -25.8% 4690 ± 35% interrupts.CPU112.NMI:Non-maskable_interrupts 6325 ± 28% -25.8% 4690 ± 35% interrupts.CPU112.PMI:Performance_monitoring_interrupts 5692 ± 33% -17.6% 4691 ± 35% interrupts.CPU124.NMI:Non-maskable_interrupts 5692 ± 33% -17.6% 4691 ± 35% interrupts.CPU124.PMI:Performance_monitoring_interrupts 9.33 ± 23% +408.9% 47.50 ±111% interrupts.CPU147.RES:Rescheduling_interrupts 83.00 ± 32% +54.2% 128.00 ± 15% interrupts.CPU150.RES:Rescheduling_interrupts 170.50 ± 26% -38.7% 104.50 ± 22% interrupts.CPU160.RES:Rescheduling_interrupts 184.83 ± 75% -52.4% 88.00 ± 51% interrupts.CPU170.RES:Rescheduling_interrupts 83.17 ± 15% +149.8% 207.75 ± 16% interrupts.CPU188.RES:Rescheduling_interrupts 97.67 ± 37% +68.7% 164.75 ± 30% interrupts.CPU204.RES:Rescheduling_interrupts 147.50 ± 55% -52.2% 70.50 ± 81% interrupts.CPU208.RES:Rescheduling_interrupts 6327 ± 28% -26.1% 4675 ± 35% interrupts.CPU220.NMI:Non-maskable_interrupts 6327 ± 28% -26.1% 4675 ± 35% interrupts.CPU220.PMI:Performance_monitoring_interrupts 6941 ± 20% -32.6% 4676 ± 35% interrupts.CPU227.NMI:Non-maskable_interrupts 6941 ± 20% -32.6% 4676 ± 35% interrupts.CPU227.PMI:Performance_monitoring_interrupts 5678 ± 33% -18.5% 4628 ± 31% interrupts.CPU238.NMI:Non-maskable_interrupts 5678 ± 33% -18.5% 4628 ± 31% interrupts.CPU238.PMI:Performance_monitoring_interrupts 5669 ± 33% -34.0% 3744 interrupts.CPU248.NMI:Non-maskable_interrupts 5669 ± 33% -34.0% 3744 interrupts.CPU248.PMI:Performance_monitoring_interrupts 29.83 ± 50% +202.5% 90.25 ± 59% interrupts.CPU264.RES:Rescheduling_interrupts 6324 ± 28% -40.7% 3752 interrupts.CPU281.NMI:Non-maskable_interrupts 6324 ± 28% -40.7% 3752 interrupts.CPU281.PMI:Performance_monitoring_interrupts 93.67 ± 20% -56.8% 40.50 ± 33% interrupts.CPU29.RES:Rescheduling_interrupts 80.17 ± 22% +231.8% 266.00 ± 64% interrupts.CPU70.RES:Rescheduling_interrupts 5699 ± 33% -18.6% 4641 ± 32% interrupts.CPU90.NMI:Non-maskable_interrupts 5699 ± 33% -18.6% 4641 ± 32% interrupts.CPU90.PMI:Performance_monitoring_interrupts 35.57 ± 8% -6.3 29.27 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.munmap 35.56 ± 8% -6.3 29.26 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap 35.69 ± 8% -6.3 29.40 perf-profile.calltrace.cycles-pp.munmap 35.34 ± 8% -6.3 29.05 perf-profile.calltrace.cycles-pp.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap 35.35 ± 8% -6.3 29.07 perf-profile.calltrace.cycles-pp.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap 34.52 ± 8% -5.9 28.63 perf-profile.calltrace.cycles-pp.mmap64 34.37 ± 8% -5.9 28.48 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.mmap64 34.38 ± 8% -5.9 28.49 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.mmap64 34.17 ± 8% -5.9 28.31 perf-profile.calltrace.cycles-pp.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe.mmap64 34.16 ± 8% -5.9 28.30 perf-profile.calltrace.cycles-pp.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe.mmap64 0.73 ± 15% -0.5 0.27 ±100% perf-profile.calltrace.cycles-pp._raw_spin_unlock_irqrestore.percpu_counter_add_batch.__do_munmap.__vm_munmap.__x64_sys_munmap 0.72 ± 13% -0.5 0.26 ±100% perf-profile.calltrace.cycles-pp._raw_spin_unlock_irqrestore.percpu_counter_add_batch.__vm_enough_memory.mmap_region.do_mmap 0.91 ± 7% -0.2 0.68 ± 4% perf-profile.calltrace.cycles-pp.unmap_vmas.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap 0.85 ± 7% -0.2 0.64 ± 4% perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.unmap_region.__do_munmap.__vm_munmap 49.96 -0.2 49.76 perf-profile.calltrace.cycles-pp.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe 48.00 -0.2 47.81 perf-profile.calltrace.cycles-pp.percpu_counter_add_batch.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64 46.09 +0.4 46.48 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.percpu_counter_add_batch.__vm_enough_memory.mmap_region 46.13 +0.4 46.53 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.percpu_counter_add_batch.__vm_enough_memory.mmap_region.do_mmap 14.25 ± 19% +6.0 20.24 perf-profile.calltrace.cycles-pp.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe 14.26 ± 19% +6.0 20.25 perf-profile.calltrace.cycles-pp.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe 14.75 ± 19% +6.0 20.79 perf-profile.calltrace.cycles-pp.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe 14.76 ± 19% +6.0 20.80 perf-profile.calltrace.cycles-pp.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe 29.34 ± 19% +12.2 41.52 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 29.35 ± 19% +12.2 41.54 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 35.71 ± 8% -6.3 29.41 perf-profile.children.cycles-pp.munmap 34.53 ± 8% -5.9 28.64 perf-profile.children.cycles-pp.mmap64 1.92 ± 9% -0.4 1.57 ± 11% perf-profile.children.cycles-pp.smp_apic_timer_interrupt 1.66 ± 9% -0.3 1.33 ± 11% perf-profile.children.cycles-pp.hrtimer_interrupt 1.38 ± 6% -0.3 1.11 ± 10% perf-profile.children.cycles-pp.__hrtimer_run_queues 50.09 -0.2 49.85 perf-profile.children.cycles-pp.__vm_munmap 50.11 -0.2 49.88 perf-profile.children.cycles-pp.__x64_sys_munmap 49.97 -0.2 49.77 perf-profile.children.cycles-pp.__do_munmap 1.30 -0.1 1.17 ± 3% perf-profile.children.cycles-pp.unmap_vmas 1.25 -0.1 1.12 ± 3% perf-profile.children.cycles-pp.unmap_page_range 0.94 ± 3% -0.1 0.83 ± 9% perf-profile.children.cycles-pp.tick_sched_timer 0.90 ± 3% -0.1 0.79 ± 10% perf-profile.children.cycles-pp.tick_sched_handle 0.88 ± 3% -0.1 0.78 ± 10% perf-profile.children.cycles-pp.update_process_times 0.70 ± 4% -0.1 0.60 ± 10% perf-profile.children.cycles-pp.scheduler_tick 0.49 ± 2% -0.1 0.42 ± 10% perf-profile.children.cycles-pp.task_tick_fair 0.12 ± 19% -0.1 0.06 ± 20% perf-profile.children.cycles-pp.__remove_hrtimer 0.59 -0.0 0.56 ± 3% perf-profile.children.cycles-pp.___might_sleep 0.15 ± 5% -0.0 0.12 ± 12% perf-profile.children.cycles-pp.irq_exit 0.14 ± 5% -0.0 0.11 ± 9% perf-profile.children.cycles-pp.update_curr 0.11 ± 6% -0.0 0.09 ± 7% perf-profile.children.cycles-pp.entry_SYSCALL_64 0.07 ± 5% +0.0 0.09 ± 7% perf-profile.children.cycles-pp.remove_vma 0.10 ± 9% +0.0 0.13 ± 5% perf-profile.children.cycles-pp.kmem_cache_alloc 0.13 ± 7% +0.0 0.16 ± 6% perf-profile.children.cycles-pp.vm_area_alloc 0.14 ± 5% +0.0 0.18 ± 8% perf-profile.children.cycles-pp.syscall_return_via_sysret 0.00 +0.1 0.05 ± 9% perf-profile.children.cycles-pp.kmem_cache_free 0.01 ±223% +0.1 0.07 ± 7% perf-profile.children.cycles-pp.__might_sleep 0.00 +0.1 0.07 ± 5% perf-profile.children.cycles-pp.fpregs_assert_state_consistent 0.05 +0.1 0.14 ± 7% perf-profile.children.cycles-pp.lru_add_drain 0.02 ±141% +0.1 0.13 ± 6% perf-profile.children.cycles-pp.lru_add_drain_cpu 92.90 +0.5 93.36 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 93.01 +0.5 93.48 perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.20 ± 6% -0.1 0.13 ± 9% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore 0.51 ± 2% -0.1 0.43 ± 3% perf-profile.self.cycles-pp.unmap_page_range 0.09 ± 27% -0.1 0.03 ±102% perf-profile.self.cycles-pp.hrtimer_interrupt 0.10 ± 14% -0.0 0.06 ± 6% perf-profile.self.cycles-pp.perf_event_mmap 0.56 -0.0 0.53 ± 3% perf-profile.self.cycles-pp.___might_sleep 0.24 ± 3% -0.0 0.22 ± 5% perf-profile.self.cycles-pp.percpu_counter_add_batch 0.09 ± 5% -0.0 0.08 ± 8% perf-profile.self.cycles-pp.entry_SYSCALL_64 0.12 ± 8% +0.0 0.15 ± 10% perf-profile.self.cycles-pp.perf_iterate_sb 0.14 ± 5% +0.0 0.18 ± 8% perf-profile.self.cycles-pp.syscall_return_via_sysret 0.00 +0.1 0.05 ± 9% perf-profile.self.cycles-pp.kmem_cache_free 0.00 +0.1 0.06 ± 6% perf-profile.self.cycles-pp.__might_sleep 0.00 +0.1 0.07 ± 5% perf-profile.self.cycles-pp.fpregs_assert_state_consistent 0.01 ±223% +0.1 0.13 ± 6% perf-profile.self.cycles-pp.lru_add_drain_cpu 92.90 +0.5 93.36 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 134459 ± 2% -8.7% 122798 softirqs.CPU0.TIMER 133411 ± 2% -8.0% 122678 softirqs.CPU100.TIMER 133333 ± 2% -8.0% 122724 softirqs.CPU101.TIMER 133598 ± 2% -8.4% 122435 softirqs.CPU110.TIMER 134909 -9.1% 122634 softirqs.CPU113.TIMER 133179 ± 2% -7.7% 122926 ± 2% softirqs.CPU114.TIMER 133436 ± 2% -8.3% 122320 softirqs.CPU119.TIMER 134596 ± 2% -9.3% 122103 softirqs.CPU12.TIMER 135545 ± 4% -9.8% 122253 softirqs.CPU122.TIMER 134335 ± 2% -8.6% 122754 softirqs.CPU125.TIMER 135177 ± 4% -9.2% 122750 softirqs.CPU128.TIMER 134010 ± 3% -8.4% 122696 softirqs.CPU129.TIMER 133968 ± 2% -8.5% 122525 softirqs.CPU13.TIMER 132997 ± 2% -8.0% 122308 softirqs.CPU134.TIMER 132866 ± 2% -8.1% 122135 softirqs.CPU136.TIMER 133212 ± 2% -8.4% 122064 softirqs.CPU137.TIMER 133362 ± 2% -8.2% 122381 softirqs.CPU138.TIMER 134902 ± 2% -8.7% 123132 softirqs.CPU14.TIMER 133290 ± 2% -8.2% 122366 softirqs.CPU140.TIMER 133372 ± 3% -8.2% 122468 softirqs.CPU141.TIMER 132826 ± 2% -8.0% 122159 softirqs.CPU142.TIMER 132860 ± 2% -8.6% 121443 softirqs.CPU144.TIMER 133202 ± 2% -8.2% 122321 softirqs.CPU146.TIMER 133355 ± 2% -8.7% 121769 softirqs.CPU148.TIMER 133645 ± 2% -8.3% 122597 softirqs.CPU149.TIMER 133171 ± 3% -8.2% 122221 softirqs.CPU156.TIMER 135643 ± 4% -9.6% 122677 softirqs.CPU16.TIMER 133388 ± 2% -8.1% 122526 softirqs.CPU163.TIMER 133749 ± 2% -8.5% 122375 softirqs.CPU17.TIMER 132919 ± 2% -8.5% 121670 softirqs.CPU171.TIMER 133345 ± 2% -8.3% 122262 softirqs.CPU173.TIMER 132748 ± 3% -8.0% 122131 softirqs.CPU176.TIMER 133989 ± 2% -8.9% 122054 softirqs.CPU18.TIMER 133046 ± 2% -8.1% 122223 softirqs.CPU180.TIMER 132979 ± 2% -8.1% 122264 softirqs.CPU181.TIMER 133378 ± 2% -8.4% 122150 softirqs.CPU182.TIMER 133506 ± 3% -8.7% 121872 softirqs.CPU188.TIMER 133634 ± 2% -8.4% 122410 softirqs.CPU19.TIMER 133179 ± 2% -8.5% 121804 softirqs.CPU195.TIMER 132997 ± 3% -8.6% 121596 softirqs.CPU197.TIMER 132919 ± 2% -8.5% 121655 softirqs.CPU198.TIMER 133502 ± 2% -8.6% 121974 softirqs.CPU20.TIMER 132247 ± 2% -7.7% 122080 softirqs.CPU200.TIMER 132682 ± 2% -8.3% 121707 softirqs.CPU201.TIMER 132349 ± 3% -7.7% 122114 softirqs.CPU202.TIMER 132434 ± 3% -7.7% 122283 softirqs.CPU203.TIMER 132590 ± 2% -8.4% 121393 softirqs.CPU208.TIMER 132398 ± 2% -8.2% 121482 softirqs.CPU209.TIMER 133514 ± 2% -8.7% 121956 softirqs.CPU21.TIMER 133008 ± 3% -8.3% 121918 softirqs.CPU210.TIMER 133510 ± 3% -8.9% 121656 softirqs.CPU211.TIMER 132399 ± 2% -8.2% 121476 softirqs.CPU212.TIMER 132399 ± 2% -8.2% 121476 softirqs.CPU213.TIMER 132523 ± 2% -8.0% 121977 softirqs.CPU214.TIMER 132473 ± 2% -8.5% 121240 softirqs.CPU215.TIMER 137798 ± 7% -11.5% 121900 softirqs.CPU22.TIMER 133101 ± 2% -8.6% 121611 softirqs.CPU220.TIMER 134009 ± 3% -9.2% 121636 softirqs.CPU221.TIMER 132697 ± 2% -8.4% 121490 softirqs.CPU222.TIMER 133397 ± 2% -8.4% 122135 softirqs.CPU228.TIMER 133531 ± 2% -8.7% 121970 softirqs.CPU23.TIMER 132213 ± 2% -7.9% 121829 softirqs.CPU230.TIMER 132975 ± 3% -8.3% 121978 softirqs.CPU238.TIMER 133584 ± 3% -8.7% 121978 softirqs.CPU24.TIMER 132432 ± 2% -8.0% 121787 softirqs.CPU242.TIMER 132561 ± 2% -8.4% 121491 softirqs.CPU243.TIMER 132407 ± 2% -7.8% 122027 ± 2% softirqs.CPU244.TIMER 132955 ± 2% -8.6% 121534 softirqs.CPU245.TIMER 133660 ± 2% -8.8% 121922 softirqs.CPU25.TIMER 132138 ± 2% -7.8% 121774 softirqs.CPU250.TIMER 132361 ± 3% -8.2% 121514 softirqs.CPU252.TIMER 132422 ± 3% -8.2% 121586 softirqs.CPU253.TIMER 137025 ± 7% -11.3% 121487 softirqs.CPU254.TIMER 132352 ± 2% -8.4% 121197 softirqs.CPU255.TIMER 131979 ± 2% -8.2% 121176 softirqs.CPU256.TIMER 132151 ± 2% -8.4% 121098 softirqs.CPU257.TIMER 132837 ± 3% -8.5% 121489 softirqs.CPU260.TIMER 132653 -8.8% 120999 ± 2% softirqs.CPU266.TIMER 132231 ± 2% -8.9% 120478 softirqs.CPU269.TIMER 133698 ± 2% -8.7% 122032 softirqs.CPU27.TIMER 132110 ± 2% -8.9% 120375 softirqs.CPU270.TIMER 132329 ± 2% -9.0% 120457 softirqs.CPU272.TIMER 132508 ± 2% -8.8% 120911 softirqs.CPU273.TIMER 131921 ± 2% -8.6% 120544 softirqs.CPU274.TIMER 132230 ± 2% -8.7% 120729 softirqs.CPU275.TIMER 132181 ± 2% -8.8% 120573 softirqs.CPU276.TIMER 134398 ± 5% -10.1% 120833 softirqs.CPU277.TIMER 131843 ± 2% -8.5% 120602 softirqs.CPU278.TIMER 133421 ± 2% -8.5% 122025 softirqs.CPU28.TIMER 131856 ± 2% -8.7% 120359 softirqs.CPU280.TIMER 132034 ± 2% -8.7% 120592 softirqs.CPU281.TIMER 132557 ± 2% -9.0% 120614 softirqs.CPU282.TIMER 133019 ± 2% -9.7% 120142 softirqs.CPU283.TIMER 131831 ± 2% -8.5% 120600 softirqs.CPU284.TIMER 132329 ± 2% -8.6% 120971 softirqs.CPU285.TIMER 131836 ± 2% -8.9% 120073 softirqs.CPU286.TIMER 126308 ± 3% -14.3% 108210 ± 7% softirqs.CPU287.TIMER 133749 ± 2% -8.7% 122054 softirqs.CPU29.TIMER 133821 ± 2% -9.0% 121831 softirqs.CPU3.TIMER 133931 -8.9% 122056 softirqs.CPU30.TIMER 133730 ± 2% -8.8% 121904 softirqs.CPU31.TIMER 135936 ± 3% -10.5% 121705 softirqs.CPU32.TIMER 133364 ± 2% -8.7% 121789 softirqs.CPU33.TIMER 133499 -8.8% 121705 softirqs.CPU34.TIMER 134336 ± 2% -9.2% 121940 softirqs.CPU35.TIMER 133118 ± 2% -8.3% 122113 softirqs.CPU36.TIMER 133775 ± 2% -9.0% 121748 softirqs.CPU37.TIMER 133045 ± 2% -8.3% 121998 softirqs.CPU38.TIMER 132874 ± 2% -8.0% 122287 softirqs.CPU39.TIMER 134208 ± 2% -7.4% 124329 ± 4% softirqs.CPU4.TIMER 133599 ± 2% -8.6% 122060 softirqs.CPU41.TIMER 133991 ± 2% -8.9% 122058 softirqs.CPU42.TIMER 133783 ± 2% -8.7% 122132 softirqs.CPU45.TIMER 133698 ± 2% -8.4% 122501 softirqs.CPU47.TIMER 133904 ± 2% -8.9% 121925 softirqs.CPU5.TIMER 134059 ± 2% -8.6% 122491 softirqs.CPU53.TIMER 140548 ± 11% -13.0% 122285 softirqs.CPU55.TIMER 134013 ± 2% -8.8% 122274 softirqs.CPU56.TIMER 135542 ± 4% -9.3% 122942 softirqs.CPU57.TIMER 133683 ± 2% -8.7% 122070 softirqs.CPU58.TIMER 133734 ± 2% -8.2% 122801 softirqs.CPU6.TIMER 133469 ± 2% -8.4% 122305 softirqs.CPU60.TIMER 136154 ± 4% -10.1% 122353 softirqs.CPU61.TIMER 133819 ± 2% -8.9% 121944 softirqs.CPU62.TIMER 133824 ± 2% -9.3% 121414 softirqs.CPU63.TIMER 133697 ± 2% -8.5% 122288 softirqs.CPU64.TIMER 133719 ± 2% -8.8% 122016 softirqs.CPU66.TIMER 133634 ± 2% -8.9% 121724 softirqs.CPU67.TIMER 133851 ± 2% -8.5% 122495 softirqs.CPU68.TIMER 133618 ± 2% -8.5% 122325 softirqs.CPU69.TIMER 133816 ± 2% -8.7% 122110 softirqs.CPU70.TIMER 133338 ± 2% -8.2% 122340 softirqs.CPU71.TIMER 8074 ± 32% -43.6% 4557 softirqs.CPU73.RCU 139691 ± 5% -12.2% 122619 softirqs.CPU73.TIMER 134414 ± 2% -8.7% 122695 softirqs.CPU76.TIMER 134661 ± 2% -8.9% 122674 softirqs.CPU77.TIMER 133670 ± 2% -8.7% 122040 softirqs.CPU8.TIMER 134093 ± 2% -8.4% 122854 softirqs.CPU80.TIMER 133919 ± 2% -8.6% 122458 softirqs.CPU83.TIMER 133776 ± 2% -8.2% 122797 softirqs.CPU86.TIMER 135212 ± 4% -9.4% 122496 softirqs.CPU89.TIMER 133802 ± 2% -8.9% 121958 softirqs.CPU9.TIMER 134030 ± 2% -8.6% 122545 softirqs.CPU91.TIMER 134236 ± 3% -8.4% 123012 softirqs.CPU94.TIMER 138956 ± 7% -11.4% 123126 softirqs.CPU96.TIMER 1565 ± 5% +381.2% 7535 ± 49% softirqs.NET_RX 38399366 ± 2% -8.2% 35256509 softirqs.TIMER will-it-scale.per_process_ops 1800 +--------------------------------------------------------------------+ | O O O O O O O O O O O O O O O O O | 1600 |..+ ..+.. .+..+...+..+... ..+..+...+..| 1400 |-+: +...+..+. +...+. +..+. | | : : | 1200 |-+ : : | 1000 |-+ : : | | : : | 800 |-+ : : | 600 |-+ : : | | : : | 400 |-+ : : | 200 |-+ : : | | : : | 0 +--------------------------------------------------------------------+ will-it-scale.workload 500000 +------------------------------------------------------------------+ 450000 |-+O O O O O O O O O O O O O O O O O | |..+ +..+..+...+.. ..+..+..+...+..+.. ..+..+..+...+..| 400000 |-+: : +. +. | 350000 |-+ : : | | : : | 300000 |-+ : : | 250000 |-+ : : | 200000 |-+ : : | | : : | 150000 |-+ : : | 100000 |-+ : : | | : : | 50000 |-+ : : | 0 +------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample *************************************************************************************************** lkp-knm01: 288 threads Intel(R) Xeon Phi(TM) CPU 7295 @ 1.50GHz with 80G memory ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode: gcc-7/performance/x86_64-rhel-7.6/thread/100%/debian-x86_64-20191114.cgz/lkp-knm01/open1/will-it-scale/0x11 commit: v5.7-rc3 8aebfffacf ("configfs: fix config_item refcnt leak in configfs_rmdir()") v5.7-rc3 8aebfffacfa379ba400da573a5b ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 5:5 -100% :1 dmesg.WARNING:at#for_ip_interrupt_entry/0x :5 20% 1:1 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x 4:5 -80% :1 dmesg.WARNING:at_ip_fsnotify_parent/0x %stddev %change %stddev \ | \ 888.20 +11.9% 994.00 will-it-scale.per_thread_ops 542327 -100.0% 6.00 will-it-scale.time.involuntary_context_switches 244880 -96.0% 9772 will-it-scale.time.maximum_resident_set_size 52934 -88.5% 6070 will-it-scale.time.minor_page_faults 28447 -100.0% 0.00 will-it-scale.time.percent_of_cpu_this_job_got 87059 -100.0% 0.19 will-it-scale.time.system_time 313.04 ± 3% -99.9% 0.24 will-it-scale.time.user_time 2065 ± 21% -96.8% 67.00 will-it-scale.time.voluntary_context_switches 255979 +11.9% 286557 will-it-scale.workload 45.76 ± 2% +3.8% 47.50 boot-time.boot 11391 ± 2% +3.9% 11836 boot-time.idle 0.06 ± 5% +0.1 0.11 mpstat.cpu.all.soft% 0.50 ± 2% +0.2 0.73 mpstat.cpu.all.usr% 360.53 -100.0% 0.00 uptime.boot 14281 -100.0% 0.00 uptime.idle 1366998 +10.9% 1515912 vmstat.memory.cache 4322 +40.8% 6087 vmstat.system.cs 1247353 +11.1% 1386310 meminfo.Cached 681.20 ±122% +150.6% 1707 meminfo.Mlocked 1095965 +12.9% 1237288 meminfo.Unevictable 565128 +12.6% 636336 numa-meminfo.node0.Unevictable 530836 +13.2% 600952 numa-meminfo.node1.FilePages 17615 ± 9% +32.5% 23348 numa-meminfo.node1.KReclaimable 10925 +13.8% 12429 numa-meminfo.node1.Mapped 622609 +12.2% 698516 numa-meminfo.node1.MemUsed 17615 ± 9% +32.5% 23348 numa-meminfo.node1.SReclaimable 31688 ± 8% +18.7% 37628 numa-meminfo.node1.Slab 530836 +13.2% 600952 numa-meminfo.node1.Unevictable 141281 +12.6% 159083 numa-vmstat.node0.nr_unevictable 141281 +12.6% 159083 numa-vmstat.node0.nr_zone_unevictable 154743 +11.2% 172123 numa-vmstat.node0.numa_interleave 132709 +13.2% 150238 numa-vmstat.node1.nr_file_pages 2731 +13.7% 3105 numa-vmstat.node1.nr_mapped 4403 ± 9% +32.5% 5837 numa-vmstat.node1.nr_slab_reclaimable 132709 +13.2% 150238 numa-vmstat.node1.nr_unevictable 132709 +13.2% 150238 numa-vmstat.node1.nr_zone_unevictable 155367 +11.1% 172552 numa-vmstat.node1.numa_hit 154782 +11.1% 171964 numa-vmstat.node1.numa_interleave 154795 +11.1% 171976 numa-vmstat.node1.numa_other 125539 +1.4% 127359 proc-vmstat.nr_anon_pages 311850 +11.1% 346545 proc-vmstat.nr_file_pages 11616 +5.4% 12242 proc-vmstat.nr_mapped 29822 +8.5% 32345 proc-vmstat.nr_slab_reclaimable 87439 +1.4% 88664 proc-vmstat.nr_slab_unreclaimable 273990 +12.9% 309322 proc-vmstat.nr_unevictable 273990 +12.9% 309322 proc-vmstat.nr_zone_unevictable 6518 ± 3% -9.2% 5921 proc-vmstat.numa_hint_faults 6518 ± 3% -9.2% 5921 proc-vmstat.numa_hint_faults_local 894126 -1.5% 880458 proc-vmstat.numa_hit 894126 -1.5% 880458 proc-vmstat.numa_local 356019 +7.0% 380894 proc-vmstat.numa_pte_updates 846963 -1.9% 830953 proc-vmstat.pgfault 962825 -7.8% 887792 proc-vmstat.pgfree 1511 ± 6% +27.9% 1933 slabinfo.file_lock_cache.active_objs 1511 ± 6% +27.9% 1933 slabinfo.file_lock_cache.num_objs 84372 +10.6% 93300 slabinfo.inode_cache.active_objs 1567 +10.6% 1733 slabinfo.inode_cache.active_slabs 84657 +10.6% 93607 slabinfo.inode_cache.num_objs 1567 +10.6% 1733 slabinfo.inode_cache.num_slabs 2465 ± 10% +27.2% 3137 slabinfo.khugepaged_mm_slot.active_objs 2465 ± 10% +27.2% 3137 slabinfo.khugepaged_mm_slot.num_objs 6099 ± 8% +16.9% 7129 slabinfo.kmalloc-rcl-64.active_objs 6099 ± 8% +16.9% 7129 slabinfo.kmalloc-rcl-64.num_objs 657.20 ± 7% +22.6% 806.00 slabinfo.numa_policy.active_objs 657.20 ± 7% +22.6% 806.00 slabinfo.numa_policy.num_objs 28780 +15.5% 33230 slabinfo.radix_tree_node.active_objs 28780 +15.5% 33230 slabinfo.radix_tree_node.num_objs 1136 ± 4% -100.0% 0.00 slabinfo.skbuff_ext_cache.active_objs 1136 ± 4% -100.0% 0.00 slabinfo.skbuff_ext_cache.num_objs 1026 ± 4% -15.2% 870.00 slabinfo.skbuff_fclone_cache.active_objs 1026 ± 4% -15.2% 870.00 slabinfo.skbuff_fclone_cache.num_objs 1255 ± 3% +15.5% 1450 slabinfo.task_group.active_objs 1255 ± 3% +15.5% 1450 slabinfo.task_group.num_objs 1472 -18.5% 1200 sched_debug.cfs_rq:/.exec_clock.stddev 5.09 ± 5% +24.2% 6.32 sched_debug.cfs_rq:/.load_avg.avg 147.32 ± 36% -38.2% 91.00 sched_debug.cfs_rq:/.load_avg.max 644505 -15.9% 541941 sched_debug.cfs_rq:/.min_vruntime.stddev 128.50 ± 40% +80.8% 232.36 sched_debug.cfs_rq:/.nr_spread_over.avg 254.64 ± 21% +46.5% 373.00 sched_debug.cfs_rq:/.nr_spread_over.max 19.72 ± 8% +84.7% 36.42 sched_debug.cfs_rq:/.nr_spread_over.stddev 47.08 ± 7% +13.0% 53.19 sched_debug.cfs_rq:/.util_avg.stddev 8576310 ± 11% +23.4% 10584065 sched_debug.cpu.avg_idle.max 1270 -33.7% 842.03 sched_debug.cpu.clock.stddev 1270 -33.7% 842.03 sched_debug.cpu.clock_task.stddev 1969070 ± 10% +36.3% 2683386 sched_debug.cpu.max_idle_balance_cost.max 207363 ± 10% +17.2% 243056 sched_debug.cpu.max_idle_balance_cost.stddev 0.00 -33.5% 0.00 sched_debug.cpu.next_balance.stddev 4788 +15.0% 5504 sched_debug.cpu.nr_switches.avg 22523 ± 12% +34.6% 30319 sched_debug.cpu.nr_switches.max 1789 ± 19% +53.3% 2743 sched_debug.cpu.nr_switches.min 1922 ± 5% +15.5% 2220 sched_debug.cpu.nr_switches.stddev 0.00 ± 54% -100.0% 0.00 sched_debug.cpu.nr_uninterruptible.avg 2066 +33.0% 2748 sched_debug.cpu.sched_count.avg 18124 ± 17% +43.4% 25987 sched_debug.cpu.sched_count.max 1336 +45.4% 1943 sched_debug.cpu.sched_count.min 1489 ± 9% +21.2% 1805 sched_debug.cpu.sched_count.stddev 906.60 +38.5% 1255 sched_debug.cpu.ttwu_count.avg 8824 ± 16% +42.9% 12610 sched_debug.cpu.ttwu_count.max 600.40 +51.8% 911.40 sched_debug.cpu.ttwu_count.min 771.87 ± 8% +19.8% 924.84 sched_debug.cpu.ttwu_count.stddev 838.79 +41.8% 1189 sched_debug.cpu.ttwu_local.avg 8251 ± 17% +47.4% 12161 sched_debug.cpu.ttwu_local.max 592.04 +50.8% 893.00 sched_debug.cpu.ttwu_local.min 671.44 ± 11% +24.5% 836.15 sched_debug.cpu.ttwu_local.stddev 0.02 ± 13% +2158.1% 0.49 sched_debug.rt_rq:/.rt_time.avg 0.23 ± 23% +61069.8% 140.99 sched_debug.rt_rq:/.rt_time.max 0.05 ± 8% +15455.0% 8.29 sched_debug.rt_rq:/.rt_time.stddev 17897 +10.0% 19690 softirqs.CPU102.RCU 17584 ± 3% +8.6% 19097 softirqs.CPU104.RCU 118027 ± 13% +17.3% 138420 softirqs.CPU105.TIMER 1567 ±146% +1449.6% 24286 softirqs.CPU11.NET_RX 20056 ± 8% +47.9% 29665 softirqs.CPU11.RCU 105076 ± 9% +14.9% 120696 softirqs.CPU11.TIMER 125179 ± 10% +19.2% 149205 softirqs.CPU112.TIMER 122851 ± 12% +19.7% 147077 softirqs.CPU122.TIMER 295.00 ± 70% +3917.3% 11851 softirqs.CPU13.NET_RX 19060 ± 3% +48.6% 28327 softirqs.CPU13.RCU 118005 ± 12% +20.9% 142700 softirqs.CPU134.TIMER 115236 ± 9% +17.7% 135682 softirqs.CPU135.TIMER 124737 ± 12% +17.5% 146568 softirqs.CPU138.TIMER 118041 ± 11% +16.2% 137150 softirqs.CPU146.TIMER 117207 ± 13% +22.4% 143500 softirqs.CPU147.TIMER 123578 ± 12% +15.2% 142420 softirqs.CPU152.TIMER 119024 ± 12% +18.6% 141137 softirqs.CPU153.TIMER 114261 ± 13% +23.4% 141010 softirqs.CPU157.TIMER 127481 ± 11% +14.9% 146417 softirqs.CPU159.TIMER 19230 ± 4% +9.6% 21072 softirqs.CPU16.RCU 124958 ± 11% +14.9% 143591 softirqs.CPU161.TIMER 124030 ± 13% +16.2% 144177 softirqs.CPU165.TIMER 115582 ± 14% +19.0% 137497 softirqs.CPU173.TIMER 17444 ± 2% +17.3% 20465 softirqs.CPU182.RCU 120903 ± 13% +19.7% 144709 softirqs.CPU187.TIMER 116780 ± 12% +17.5% 137227 softirqs.CPU195.TIMER 17548 ± 3% +17.4% 20609 softirqs.CPU196.RCU 116381 ± 13% +19.5% 139093 softirqs.CPU199.TIMER 17195 ± 2% +9.3% 18790 softirqs.CPU200.RCU 118401 ± 11% +16.4% 137831 softirqs.CPU200.TIMER 124742 ± 11% +15.4% 143893 softirqs.CPU202.TIMER 117531 ± 11% +18.9% 139749 softirqs.CPU215.TIMER 116289 ± 11% +19.6% 139030 softirqs.CPU216.TIMER 116587 ± 11% +23.7% 144193 softirqs.CPU220.TIMER 17695 ± 3% +8.1% 19135 softirqs.CPU221.RCU 117205 ± 11% +19.4% 139956 softirqs.CPU221.TIMER 119762 ± 11% +20.7% 144564 softirqs.CPU225.TIMER 17214 ± 2% +9.1% 18776 softirqs.CPU229.RCU 122398 ± 11% +17.4% 143709 softirqs.CPU233.TIMER 112338 ± 13% +17.5% 131952 softirqs.CPU238.TIMER 17326 +15.2% 19952 softirqs.CPU239.RCU 119806 ± 11% +19.6% 143316 softirqs.CPU246.TIMER 118281 ± 13% +17.1% 138534 softirqs.CPU247.TIMER 110402 ± 13% +20.4% 132888 softirqs.CPU256.TIMER 17218 ± 2% +40.6% 24209 softirqs.CPU265.RCU 103283 ± 8% +71.8% 177475 softirqs.CPU265.TIMER 18905 ± 2% +11.8% 21138 softirqs.CPU28.RCU 17230 ± 3% +10.3% 18999 softirqs.CPU282.RCU 19392 ± 4% +21.9% 23632 softirqs.CPU30.RCU 18951 ± 3% +18.4% 22433 softirqs.CPU31.RCU 105516 ± 6% -11.3% 93580 softirqs.CPU32.TIMER 18749 ± 2% +13.1% 21201 softirqs.CPU34.RCU 108972 ± 10% +22.1% 133022 softirqs.CPU43.TIMER 18723 ± 3% +27.8% 23926 softirqs.CPU53.RCU 109283 ± 10% +22.0% 133326 softirqs.CPU64.TIMER 18586 ± 9% +45.0% 26960 softirqs.CPU72.RCU 117359 ± 13% +30.5% 153156 softirqs.CPU72.TIMER 17989 +24.6% 22407 softirqs.CPU75.RCU 112037 ± 12% +19.8% 134168 softirqs.CPU75.TIMER 115165 ± 12% +21.8% 140265 softirqs.CPU80.TIMER 110504 ± 12% +17.9% 130255 softirqs.CPU81.TIMER 122916 ± 12% +16.8% 143533 softirqs.CPU82.TIMER 116896 ± 12% +17.2% 137045 softirqs.CPU84.TIMER 118538 ± 10% +13.6% 134673 softirqs.CPU85.TIMER 116481 ± 12% +19.4% 139126 softirqs.CPU93.TIMER 3411 ± 68% +1005.0% 37695 softirqs.NET_RX 48.48 -0.3 48.13 perf-profile.calltrace.cycles-pp.__alloc_fd.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 48.22 -0.3 47.94 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__alloc_fd.do_sys_openat2.do_sys_open 49.17 -0.3 48.91 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_close 49.15 -0.3 48.90 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_close 49.95 -0.2 49.72 perf-profile.calltrace.cycles-pp.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_open 49.94 -0.2 49.71 perf-profile.calltrace.cycles-pp.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_open 48.20 -0.2 47.98 perf-profile.calltrace.cycles-pp._raw_spin_lock.__alloc_fd.do_sys_openat2.do_sys_open.do_syscall_64 48.62 -0.2 48.41 perf-profile.calltrace.cycles-pp.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_close 50.02 -0.2 49.81 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_open 50.03 -0.2 49.82 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_open 48.47 -0.2 48.27 perf-profile.calltrace.cycles-pp.__close_fd.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_close 49.34 -0.2 49.15 perf-profile.calltrace.cycles-pp.__GI___libc_close 48.28 -0.2 48.10 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__close_fd.__x64_sys_close.do_syscall_64 50.22 -0.2 50.06 perf-profile.calltrace.cycles-pp.__GI___libc_open 48.27 -0.1 48.14 perf-profile.calltrace.cycles-pp._raw_spin_lock.__close_fd.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.89 ± 3% +0.2 3.11 perf-profile.calltrace.cycles-pp.apic_timer_interrupt.native_queued_spin_lock_slowpath._raw_spin_lock.__alloc_fd.do_sys_openat2 2.77 ± 3% +0.2 3.01 perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.native_queued_spin_lock_slowpath._raw_spin_lock.__alloc_fd 3.55 ± 3% +0.2 3.80 perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.native_queued_spin_lock_slowpath 2.91 ± 4% +0.3 3.18 perf-profile.calltrace.cycles-pp.apic_timer_interrupt.native_queued_spin_lock_slowpath._raw_spin_lock.__close_fd.__x64_sys_close 2.79 ± 4% +0.3 3.07 perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.native_queued_spin_lock_slowpath._raw_spin_lock.__close_fd 4.56 ± 3% +0.3 4.91 perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.native_queued_spin_lock_slowpath._raw_spin_lock 96.64 -0.5 96.17 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 96.69 -0.5 96.23 perf-profile.children.cycles-pp._raw_spin_lock 48.51 -0.3 48.16 perf-profile.children.cycles-pp.__alloc_fd 99.46 -0.3 99.16 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 99.43 -0.3 99.13 perf-profile.children.cycles-pp.do_syscall_64 49.95 -0.2 49.72 perf-profile.children.cycles-pp.do_sys_openat2 49.96 -0.2 49.73 perf-profile.children.cycles-pp.do_sys_open 48.62 -0.2 48.41 perf-profile.children.cycles-pp.__x64_sys_close 48.47 -0.2 48.27 perf-profile.children.cycles-pp.__close_fd 49.35 -0.2 49.16 perf-profile.children.cycles-pp.__GI___libc_close 50.22 -0.2 50.07 perf-profile.children.cycles-pp.__GI___libc_open 0.17 ± 13% -0.1 0.08 perf-profile.children.cycles-pp.update_rq_clock 0.49 -0.1 0.41 perf-profile.children.cycles-pp.exit_to_usermode_loop 0.45 -0.1 0.37 perf-profile.children.cycles-pp.task_work_run 0.48 ± 2% -0.1 0.41 perf-profile.children.cycles-pp.alloc_empty_file 0.34 -0.1 0.28 perf-profile.children.cycles-pp.kmem_cache_alloc 0.16 ± 10% -0.0 0.11 perf-profile.children.cycles-pp._raw_spin_lock_irq 0.14 ± 22% -0.0 0.09 perf-profile.children.cycles-pp.rcu_sched_clock_irq 0.09 ± 10% -0.0 0.05 perf-profile.children.cycles-pp.expand_files 0.30 -0.0 0.26 perf-profile.children.cycles-pp.__fput 0.42 ± 2% -0.0 0.38 perf-profile.children.cycles-pp.__alloc_file 0.14 ± 5% -0.0 0.11 perf-profile.children.cycles-pp.security_file_alloc 0.11 ± 4% -0.0 0.10 perf-profile.children.cycles-pp.strncpy_from_user 0.06 -0.0 0.05 perf-profile.children.cycles-pp.security_file_open 0.16 ± 2% +0.0 0.17 perf-profile.children.cycles-pp.link_path_walk 0.06 ± 6% +0.0 0.08 perf-profile.children.cycles-pp.tick_program_event 0.15 ± 4% +0.0 0.17 perf-profile.children.cycles-pp.do_dentry_open 0.04 ± 50% +0.0 0.06 perf-profile.children.cycles-pp.lapic_next_deadline 0.05 +0.0 0.07 perf-profile.children.cycles-pp.dput 0.19 ± 2% +0.0 0.21 perf-profile.children.cycles-pp.getname_flags 0.05 ± 50% +0.0 0.07 perf-profile.children.cycles-pp.native_write_msr 0.13 ± 6% +0.0 0.15 perf-profile.children.cycles-pp.enqueue_hrtimer 0.11 ± 7% +0.0 0.13 perf-profile.children.cycles-pp.timerqueue_add 0.06 ± 7% +0.0 0.09 perf-profile.children.cycles-pp.___might_sleep 0.16 ± 9% +0.0 0.19 perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.03 ± 81% +0.0 0.06 perf-profile.children.cycles-pp.fsnotify 0.03 ± 81% +0.0 0.06 perf-profile.children.cycles-pp._cond_resched 0.19 ± 29% +0.0 0.24 perf-profile.children.cycles-pp.rb_next 0.09 ± 23% +0.1 0.14 perf-profile.children.cycles-pp.rcu_irq_enter 0.01 ±200% +0.1 0.07 perf-profile.children.cycles-pp.shmem_getpage_gfp 0.03 ± 81% +0.1 0.09 perf-profile.children.cycles-pp.__slab_free 0.01 ±200% +0.1 0.07 perf-profile.children.cycles-pp.fput_many 0.24 ± 25% +0.1 0.31 perf-profile.children.cycles-pp.timerqueue_del 0.13 ± 3% +0.1 0.20 perf-profile.children.cycles-pp.syscall_return_via_sysret 0.14 ± 2% +0.1 0.21 perf-profile.children.cycles-pp.kmem_cache_free 0.48 ± 7% +0.1 0.55 perf-profile.children.cycles-pp.irq_exit 0.00 +0.1 0.08 perf-profile.children.cycles-pp.account_process_tick 0.00 +0.1 0.10 perf-profile.children.cycles-pp.fpregs_assert_state_consistent 0.14 ± 34% +0.1 0.24 perf-profile.children.cycles-pp.generic_perform_write 0.15 ± 32% +0.1 0.25 perf-profile.children.cycles-pp.__generic_file_write_iter 0.19 ± 2% +0.1 0.30 perf-profile.children.cycles-pp.file_free_rcu 0.18 ± 21% +0.1 0.29 perf-profile.children.cycles-pp.irq_enter 0.50 ± 28% +0.1 0.62 perf-profile.children.cycles-pp.__remove_hrtimer 0.15 ± 34% +0.1 0.27 perf-profile.children.cycles-pp.generic_file_write_iter 0.16 ± 33% +0.1 0.28 perf-profile.children.cycles-pp.new_sync_write 0.18 ± 32% +0.1 0.32 perf-profile.children.cycles-pp.vfs_write 0.19 ± 32% +0.1 0.34 perf-profile.children.cycles-pp.ksys_write 0.24 ± 22% +0.1 0.39 perf-profile.children.cycles-pp.update_curr 0.06 +0.2 0.22 perf-profile.children.cycles-pp.ret_from_fork 0.06 ± 6% +0.2 0.22 perf-profile.children.cycles-pp.kthread 0.29 ± 2% +0.2 0.45 perf-profile.children.cycles-pp.rcu_core 0.27 +0.2 0.44 perf-profile.children.cycles-pp.rcu_do_batch 0.25 ± 32% +0.2 0.42 perf-profile.children.cycles-pp.main 0.25 ± 32% +0.2 0.42 perf-profile.children.cycles-pp.cmd_record 0.00 +0.2 0.19 perf-profile.children.cycles-pp.run_ksoftirqd 0.00 +0.2 0.20 perf-profile.children.cycles-pp.smpboot_thread_fn 0.42 ± 4% +0.2 0.62 perf-profile.children.cycles-pp.__softirqentry_text_start 3.69 ± 2% +0.3 3.97 perf-profile.children.cycles-pp.__hrtimer_run_queues 4.75 ± 2% +0.4 5.15 perf-profile.children.cycles-pp.hrtimer_interrupt 6.02 ± 4% +0.6 6.59 perf-profile.children.cycles-pp.apic_timer_interrupt 5.76 ± 3% +0.6 6.35 perf-profile.children.cycles-pp.smp_apic_timer_interrupt 90.71 -1.0 89.67 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 0.27 ± 3% -0.1 0.16 perf-profile.self.cycles-pp._raw_spin_lock 0.23 -0.1 0.14 perf-profile.self.cycles-pp.kmem_cache_alloc 0.15 ± 7% -0.1 0.10 perf-profile.self.cycles-pp._raw_spin_lock_irq 0.10 ± 23% -0.1 0.05 perf-profile.self.cycles-pp.rcu_sched_clock_irq 0.08 ± 4% -0.0 0.05 perf-profile.self.cycles-pp.do_syscall_64 0.09 ± 4% -0.0 0.07 perf-profile.self.cycles-pp.__close_fd 0.10 ± 5% -0.0 0.08 perf-profile.self.cycles-pp.__alloc_fd 0.08 ± 5% +0.0 0.10 perf-profile.self.cycles-pp.timerqueue_add 0.09 ± 4% +0.0 0.11 perf-profile.self.cycles-pp.kmem_cache_free 0.06 ± 6% +0.0 0.08 perf-profile.self.cycles-pp.tick_program_event 0.05 ± 50% +0.0 0.07 perf-profile.self.cycles-pp.native_write_msr 0.06 ± 8% +0.0 0.08 perf-profile.self.cycles-pp.___might_sleep 0.16 ± 9% +0.0 0.19 perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.02 ±122% +0.0 0.06 perf-profile.self.cycles-pp.__softirqentry_text_start 0.08 ± 6% +0.0 0.12 perf-profile.self.cycles-pp.__alloc_file 0.18 ± 28% +0.0 0.23 perf-profile.self.cycles-pp.rb_next 0.03 ± 81% +0.0 0.08 perf-profile.self.cycles-pp.__slab_free 0.01 ±200% +0.0 0.06 perf-profile.self.cycles-pp.run_timer_softirq 0.00 +0.1 0.05 perf-profile.self.cycles-pp.do_filp_open 0.09 ± 19% +0.1 0.15 perf-profile.self.cycles-pp.irq_enter 0.08 ± 22% +0.1 0.14 perf-profile.self.cycles-pp.rcu_irq_enter 0.13 ± 3% +0.1 0.20 perf-profile.self.cycles-pp.syscall_return_via_sysret 0.00 +0.1 0.08 perf-profile.self.cycles-pp.account_process_tick 0.18 ± 4% +0.1 0.28 perf-profile.self.cycles-pp.file_free_rcu 0.00 +0.1 0.10 perf-profile.self.cycles-pp.fpregs_assert_state_consistent 0.14 ± 29% +0.1 0.25 perf-profile.self.cycles-pp.update_curr 2513 ±155% +1249.9% 33932 interrupts.31:IR-PCI-MSI.2097154-edge.eth0-TxRx-1 446.60 ± 74% +3332.4% 15329 interrupts.33:IR-PCI-MSI.2097156-edge.eth0-TxRx-3 875.40 ± 71% -68.5% 276.00 interrupts.CPU1.RES:Rescheduling_interrupts 6595 ± 21% -44.2% 3683 interrupts.CPU10.NMI:Non-maskable_interrupts 6595 ± 21% -44.2% 3683 interrupts.CPU10.PMI:Performance_monitoring_interrupts 4331 ± 33% +69.1% 7322 interrupts.CPU101.NMI:Non-maskable_interrupts 4331 ± 33% +69.1% 7322 interrupts.CPU101.PMI:Performance_monitoring_interrupts 115.20 ± 11% -17.5% 95.00 interrupts.CPU103.RES:Rescheduling_interrupts 570.60 ± 89% -83.4% 95.00 interrupts.CPU104.RES:Rescheduling_interrupts 104.80 ± 8% +88.0% 197.00 interrupts.CPU105.RES:Rescheduling_interrupts 3644 +97.8% 7209 interrupts.CPU106.NMI:Non-maskable_interrupts 3644 +97.8% 7209 interrupts.CPU106.PMI:Performance_monitoring_interrupts 3619 +101.2% 7282 interrupts.CPU108.NMI:Non-maskable_interrupts 3619 +101.2% 7282 interrupts.CPU108.PMI:Performance_monitoring_interrupts 169.00 ± 25% -55.6% 75.00 interrupts.CPU108.RES:Rescheduling_interrupts 2513 ±155% +1249.9% 33932 interrupts.CPU11.31:IR-PCI-MSI.2097154-edge.eth0-TxRx-1 337.60 ± 11% -13.5% 292.00 interrupts.CPU11.RES:Rescheduling_interrupts 241.20 ± 65% -62.3% 91.00 interrupts.CPU110.RES:Rescheduling_interrupts 121.60 ± 28% -56.4% 53.00 interrupts.CPU111.RES:Rescheduling_interrupts 228.40 ± 53% -64.1% 82.00 interrupts.CPU112.RES:Rescheduling_interrupts 5826 ± 30% -38.1% 3609 interrupts.CPU113.NMI:Non-maskable_interrupts 5826 ± 30% -38.1% 3609 interrupts.CPU113.PMI:Performance_monitoring_interrupts 125.00 ± 14% -29.6% 88.00 interrupts.CPU115.RES:Rescheduling_interrupts 199.60 ± 18% -48.4% 103.00 interrupts.CPU116.RES:Rescheduling_interrupts 144.80 ± 41% -62.7% 54.00 interrupts.CPU117.RES:Rescheduling_interrupts 168.20 ± 10% -57.8% 71.00 interrupts.CPU119.RES:Rescheduling_interrupts 5899 ± 30% -37.7% 3675 interrupts.CPU12.NMI:Non-maskable_interrupts 5899 ± 30% -37.7% 3675 interrupts.CPU12.PMI:Performance_monitoring_interrupts 195.80 ± 44% -62.7% 73.00 interrupts.CPU120.RES:Rescheduling_interrupts 4313 ± 33% +67.8% 7240 interrupts.CPU121.NMI:Non-maskable_interrupts 4313 ± 33% +67.8% 7240 interrupts.CPU121.PMI:Performance_monitoring_interrupts 138.20 ± 17% -37.8% 86.00 interrupts.CPU121.RES:Rescheduling_interrupts 6509 ± 21% -43.9% 3651 interrupts.CPU123.NMI:Non-maskable_interrupts 6509 ± 21% -43.9% 3651 interrupts.CPU123.PMI:Performance_monitoring_interrupts 4330 ± 33% +67.1% 7235 interrupts.CPU127.NMI:Non-maskable_interrupts 4330 ± 33% +67.1% 7235 interrupts.CPU127.PMI:Performance_monitoring_interrupts 154.20 ± 33% -31.3% 106.00 interrupts.CPU129.RES:Rescheduling_interrupts 446.60 ± 74% +3332.4% 15329 interrupts.CPU13.33:IR-PCI-MSI.2097156-edge.eth0-TxRx-3 162.00 ± 55% -52.5% 77.00 interrupts.CPU131.RES:Rescheduling_interrupts 142.60 ± 19% -32.7% 96.00 interrupts.CPU132.RES:Rescheduling_interrupts 162.60 ± 8% +58.1% 257.00 interrupts.CPU134.RES:Rescheduling_interrupts 160.60 ± 23% -34.0% 106.00 interrupts.CPU138.RES:Rescheduling_interrupts 5820 ± 30% -38.6% 3572 interrupts.CPU139.NMI:Non-maskable_interrupts 5820 ± 30% -38.6% 3572 interrupts.CPU139.PMI:Performance_monitoring_interrupts 324.40 ± 7% -12.8% 283.00 interrupts.CPU14.RES:Rescheduling_interrupts 167.80 ± 32% -70.8% 49.00 interrupts.CPU140.RES:Rescheduling_interrupts 179.40 ± 29% -57.1% 77.00 interrupts.CPU142.RES:Rescheduling_interrupts 7243 -50.2% 3605 interrupts.CPU150.NMI:Non-maskable_interrupts 7243 -50.2% 3605 interrupts.CPU150.PMI:Performance_monitoring_interrupts 3644 +98.8% 7246 interrupts.CPU151.NMI:Non-maskable_interrupts 3644 +98.8% 7246 interrupts.CPU151.PMI:Performance_monitoring_interrupts 58.20 ± 21% +92.4% 112.00 interrupts.CPU154.RES:Rescheduling_interrupts 3598 +101.1% 7237 interrupts.CPU160.NMI:Non-maskable_interrupts 3598 +101.1% 7237 interrupts.CPU160.PMI:Performance_monitoring_interrupts 3618 +101.5% 7291 interrupts.CPU165.NMI:Non-maskable_interrupts 3618 +101.5% 7291 interrupts.CPU165.PMI:Performance_monitoring_interrupts 63.80 ± 86% +459.6% 357.00 interrupts.CPU167.RES:Rescheduling_interrupts 5818 ± 30% -38.1% 3599 interrupts.CPU170.NMI:Non-maskable_interrupts 5818 ± 30% -38.1% 3599 interrupts.CPU170.PMI:Performance_monitoring_interrupts 63.00 ± 42% +96.8% 124.00 interrupts.CPU176.RES:Rescheduling_interrupts 39.60 ± 27% +185.4% 113.00 interrupts.CPU179.RES:Rescheduling_interrupts 56.00 ± 28% +166.1% 149.00 interrupts.CPU180.RES:Rescheduling_interrupts 35.80 ± 20% +441.9% 194.00 interrupts.CPU187.RES:Rescheduling_interrupts 46.60 ± 16% +116.7% 101.00 interrupts.CPU191.RES:Rescheduling_interrupts 4318 ± 33% +68.8% 7291 interrupts.CPU198.NMI:Non-maskable_interrupts 4318 ± 33% +68.8% 7291 interrupts.CPU198.PMI:Performance_monitoring_interrupts 4310 ± 33% +68.4% 7258 interrupts.CPU201.NMI:Non-maskable_interrupts 4310 ± 33% +68.4% 7258 interrupts.CPU201.PMI:Performance_monitoring_interrupts 37.80 ± 31% +243.9% 130.00 interrupts.CPU201.RES:Rescheduling_interrupts 5055 ± 35% +44.1% 7285 interrupts.CPU202.NMI:Non-maskable_interrupts 5055 ± 35% +44.1% 7285 interrupts.CPU202.PMI:Performance_monitoring_interrupts 75.80 ± 28% +911.9% 767.00 interrupts.CPU202.RES:Rescheduling_interrupts 4311 ± 33% +68.2% 7250 interrupts.CPU203.NMI:Non-maskable_interrupts 4311 ± 33% +68.2% 7250 interrupts.CPU203.PMI:Performance_monitoring_interrupts 3593 +100.4% 7199 interrupts.CPU205.NMI:Non-maskable_interrupts 3593 +100.4% 7199 interrupts.CPU205.PMI:Performance_monitoring_interrupts 129.60 ± 11% +240.3% 441.00 interrupts.CPU21.RES:Rescheduling_interrupts 74.00 ± 17% +456.8% 412.00 interrupts.CPU210.RES:Rescheduling_interrupts 5033 ± 34% +44.1% 7254 interrupts.CPU213.NMI:Non-maskable_interrupts 5033 ± 34% +44.1% 7254 interrupts.CPU213.PMI:Performance_monitoring_interrupts 43.60 ± 39% +2166.1% 988.00 interrupts.CPU213.RES:Rescheduling_interrupts 67.80 ± 88% -72.0% 19.00 interrupts.CPU216.RES:Rescheduling_interrupts 27.80 ± 45% +310.1% 114.00 interrupts.CPU223.RES:Rescheduling_interrupts 3586 +101.4% 7224 interrupts.CPU224.NMI:Non-maskable_interrupts 3586 +101.4% 7224 interrupts.CPU224.PMI:Performance_monitoring_interrupts 22.20 ± 28% +404.5% 112.00 interrupts.CPU239.RES:Rescheduling_interrupts 3602 +100.8% 7234 interrupts.CPU240.NMI:Non-maskable_interrupts 3602 +100.8% 7234 interrupts.CPU240.PMI:Performance_monitoring_interrupts 3594 +101.4% 7239 interrupts.CPU241.NMI:Non-maskable_interrupts 3594 +101.4% 7239 interrupts.CPU241.PMI:Performance_monitoring_interrupts 33.60 ± 51% +370.2% 158.00 interrupts.CPU241.RES:Rescheduling_interrupts 4765 -11.6% 4212 interrupts.CPU244.CAL:Function_call_interrupts 4316 ± 32% +67.2% 7218 interrupts.CPU248.NMI:Non-maskable_interrupts 4316 ± 32% +67.2% 7218 interrupts.CPU248.PMI:Performance_monitoring_interrupts 4409 ± 32% +68.4% 7425 interrupts.CPU25.NMI:Non-maskable_interrupts 4409 ± 32% +68.4% 7425 interrupts.CPU25.PMI:Performance_monitoring_interrupts 118.80 ± 17% -31.0% 82.00 interrupts.CPU25.RES:Rescheduling_interrupts 3602 +99.0% 7167 interrupts.CPU250.NMI:Non-maskable_interrupts 3602 +99.0% 7167 interrupts.CPU250.PMI:Performance_monitoring_interrupts 31.60 ± 66% +263.9% 115.00 interrupts.CPU251.RES:Rescheduling_interrupts 38.60 ± 28% +648.7% 289.00 interrupts.CPU258.RES:Rescheduling_interrupts 3611 +102.2% 7303 interrupts.CPU259.NMI:Non-maskable_interrupts 3611 +102.2% 7303 interrupts.CPU259.PMI:Performance_monitoring_interrupts 4332 ± 33% +66.8% 7228 interrupts.CPU267.NMI:Non-maskable_interrupts 4332 ± 33% +66.8% 7228 interrupts.CPU267.PMI:Performance_monitoring_interrupts 5778 ± 31% -39.0% 3526 interrupts.CPU268.NMI:Non-maskable_interrupts 5778 ± 31% -39.0% 3526 interrupts.CPU268.PMI:Performance_monitoring_interrupts 3697 +99.5% 7377 interrupts.CPU27.NMI:Non-maskable_interrupts 3697 +99.5% 7377 interrupts.CPU27.PMI:Performance_monitoring_interrupts 5037 ± 35% +44.5% 7278 interrupts.CPU275.NMI:Non-maskable_interrupts 5037 ± 35% +44.5% 7278 interrupts.CPU275.PMI:Performance_monitoring_interrupts 23.00 ± 22% +1034.8% 261.00 interrupts.CPU278.RES:Rescheduling_interrupts 4322 ± 32% +66.5% 7197 interrupts.CPU280.NMI:Non-maskable_interrupts 4322 ± 32% +66.5% 7197 interrupts.CPU280.PMI:Performance_monitoring_interrupts 3585 +101.5% 7225 interrupts.CPU285.NMI:Non-maskable_interrupts 3585 +101.5% 7225 interrupts.CPU285.PMI:Performance_monitoring_interrupts 359.80 ± 7% +60.6% 578.00 interrupts.CPU3.RES:Rescheduling_interrupts 131.80 ± 20% -37.0% 83.00 interrupts.CPU30.RES:Rescheduling_interrupts 125.40 ± 14% -30.6% 87.00 interrupts.CPU32.RES:Rescheduling_interrupts 5948 ± 30% -38.8% 3639 interrupts.CPU33.NMI:Non-maskable_interrupts 5948 ± 30% -38.8% 3639 interrupts.CPU33.PMI:Performance_monitoring_interrupts 4408 ± 34% +68.8% 7443 interrupts.CPU34.NMI:Non-maskable_interrupts 4408 ± 34% +68.8% 7443 interrupts.CPU34.PMI:Performance_monitoring_interrupts 131.80 ± 23% -40.8% 78.00 interrupts.CPU34.RES:Rescheduling_interrupts 5152 ± 34% +43.5% 7395 interrupts.CPU35.NMI:Non-maskable_interrupts 5152 ± 34% +43.5% 7395 interrupts.CPU35.PMI:Performance_monitoring_interrupts 127.80 ± 24% -34.3% 84.00 interrupts.CPU36.RES:Rescheduling_interrupts 141.00 ± 16% +26.2% 178.00 interrupts.CPU38.RES:Rescheduling_interrupts 135.60 ± 36% -59.4% 55.00 interrupts.CPU39.RES:Rescheduling_interrupts 175.80 ± 44% -58.5% 73.00 interrupts.CPU40.RES:Rescheduling_interrupts 128.00 ± 4% -32.8% 86.00 interrupts.CPU41.RES:Rescheduling_interrupts 94.40 ± 10% -33.3% 63.00 interrupts.CPU43.RES:Rescheduling_interrupts 5834 ± 30% -38.5% 3587 interrupts.CPU44.NMI:Non-maskable_interrupts 5834 ± 30% -38.5% 3587 interrupts.CPU44.PMI:Performance_monitoring_interrupts 131.60 ± 15% -23.3% 101.00 interrupts.CPU44.RES:Rescheduling_interrupts 170.00 ± 22% -47.1% 90.00 interrupts.CPU46.RES:Rescheduling_interrupts 159.20 ± 35% -49.7% 80.00 interrupts.CPU47.RES:Rescheduling_interrupts 127.60 ± 34% -43.6% 72.00 interrupts.CPU49.RES:Rescheduling_interrupts 179.20 ± 41% -51.5% 87.00 interrupts.CPU50.RES:Rescheduling_interrupts 122.80 ± 17% -39.7% 74.00 interrupts.CPU51.RES:Rescheduling_interrupts 123.60 ± 18% -35.3% 80.00 interrupts.CPU53.RES:Rescheduling_interrupts 141.20 ± 19% +76.3% 249.00 interrupts.CPU57.RES:Rescheduling_interrupts 5820 ± 30% -38.3% 3590 interrupts.CPU58.NMI:Non-maskable_interrupts 5820 ± 30% -38.3% 3590 interrupts.CPU58.PMI:Performance_monitoring_interrupts 148.20 ± 33% -69.0% 46.00 interrupts.CPU59.RES:Rescheduling_interrupts 339.40 ± 14% +158.7% 878.00 interrupts.CPU6.RES:Rescheduling_interrupts 3594 +100.6% 7211 interrupts.CPU60.NMI:Non-maskable_interrupts 3594 +100.6% 7211 interrupts.CPU60.PMI:Performance_monitoring_interrupts 165.40 ± 16% -46.8% 88.00 interrupts.CPU60.RES:Rescheduling_interrupts 119.60 ± 13% -32.3% 81.00 interrupts.CPU65.RES:Rescheduling_interrupts 6569 ± 22% -45.8% 3559 interrupts.CPU68.NMI:Non-maskable_interrupts 6569 ± 22% -45.8% 3559 interrupts.CPU68.PMI:Performance_monitoring_interrupts 149.60 ± 18% -53.9% 69.00 interrupts.CPU68.RES:Rescheduling_interrupts 124.60 ± 23% -55.1% 56.00 interrupts.CPU69.RES:Rescheduling_interrupts 351.40 ± 21% -19.7% 282.00 interrupts.CPU7.RES:Rescheduling_interrupts 5826 ± 31% -38.5% 3584 interrupts.CPU76.NMI:Non-maskable_interrupts 5826 ± 31% -38.5% 3584 interrupts.CPU76.PMI:Performance_monitoring_interrupts 3615 +100.9% 7263 interrupts.CPU80.NMI:Non-maskable_interrupts 3615 +100.9% 7263 interrupts.CPU80.PMI:Performance_monitoring_interrupts 4325 ± 32% +66.8% 7216 interrupts.CPU81.NMI:Non-maskable_interrupts 4325 ± 32% +66.8% 7216 interrupts.CPU81.PMI:Performance_monitoring_interrupts 114.20 ± 22% -39.6% 69.00 interrupts.CPU81.RES:Rescheduling_interrupts 120.00 ± 32% -62.5% 45.00 interrupts.CPU83.RES:Rescheduling_interrupts 293.40 ± 63% -73.4% 78.00 interrupts.CPU86.RES:Rescheduling_interrupts 126.20 ± 24% -45.3% 69.00 interrupts.CPU89.RES:Rescheduling_interrupts 311.00 ± 8% +187.8% 895.00 interrupts.CPU9.RES:Rescheduling_interrupts 175.60 ± 31% -55.6% 78.00 interrupts.CPU90.RES:Rescheduling_interrupts 100.40 ± 29% -29.3% 71.00 interrupts.CPU91.RES:Rescheduling_interrupts 177.40 ± 16% -58.9% 73.00 interrupts.CPU95.RES:Rescheduling_interrupts 185.20 ± 38% -50.9% 91.00 interrupts.CPU96.RES:Rescheduling_interrupts 113.40 ± 12% +20.8% 137.00 interrupts.CPU99.RES:Rescheduling_interrupts *************************************************************************************************** lkp-knm01: 288 threads Intel(R) Xeon Phi(TM) CPU 7295 @ 1.50GHz with 80G memory ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode: gcc-7/performance/x86_64-rhel-7.6/thread/100%/debian-x86_64-20191114.cgz/lkp-knm01/sched_yield/will-it-scale/0x11 commit: v5.7-rc3 8aebfffacf ("configfs: fix config_item refcnt leak in configfs_rmdir()") v5.7-rc3 8aebfffacfa379ba400da573a5b ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 1:5 -20% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x %stddev %change %stddev \ | \ 129619 +17.8% 152634 will-it-scale.per_thread_ops 12586764 ± 6% -100.0% 2.25 ± 72% will-it-scale.time.involuntary_context_switches 245608 -96.0% 9887 will-it-scale.time.maximum_resident_set_size 57640 ± 4% -89.4% 6105 will-it-scale.time.minor_page_faults 28478 -100.0% 0.00 will-it-scale.time.percent_of_cpu_this_job_got 74208 -100.0% 0.19 ± 7% will-it-scale.time.system_time 14260 -100.0% 0.25 ± 3% will-it-scale.time.user_time 37330584 +17.8% 43958990 will-it-scale.workload 1348004 +11.0% 1496355 vmstat.memory.cache 1228847 +11.3% 1368042 meminfo.Cached 1096118 +12.8% 1236404 meminfo.Unevictable 363.61 -100.0% 0.00 uptime.boot 14062 ± 3% -100.0% 0.00 uptime.idle 45.55 ± 2% +4.3% 47.52 ± 2% boot-time.boot 27.68 ± 4% +7.5% 29.77 ± 3% boot-time.dhcp 11319 ± 3% +5.0% 11886 ± 2% boot-time.idle 126266 +1.4% 127983 proc-vmstat.nr_anon_pages 307237 +11.3% 341994 proc-vmstat.nr_file_pages 29707 +7.7% 31999 proc-vmstat.nr_slab_reclaimable 274029 +12.8% 309100 proc-vmstat.nr_unevictable 274029 +12.8% 309100 proc-vmstat.nr_zone_unevictable 991130 +1.2% 1003460 proc-vmstat.pgalloc_normal 818717 +1.3% 829269 proc-vmstat.pgfault 698116 +9.9% 767014 numa-meminfo.node0.FilePages 565294 +12.4% 635266 numa-meminfo.node0.Unevictable 530824 +13.2% 601139 numa-meminfo.node1.FilePages 18760 ± 6% +19.6% 22432 ± 5% numa-meminfo.node1.KReclaimable 10936 +14.3% 12504 numa-meminfo.node1.Mapped 625411 +11.6% 698082 numa-meminfo.node1.MemUsed 18760 ± 6% +19.6% 22432 ± 5% numa-meminfo.node1.SReclaimable 530824 +13.2% 601139 numa-meminfo.node1.Unevictable 174502 +9.9% 191744 numa-vmstat.node0.nr_file_pages 141323 +12.4% 158816 numa-vmstat.node0.nr_unevictable 141323 +12.4% 158816 numa-vmstat.node0.nr_zone_unevictable 154664 +11.3% 172075 numa-vmstat.node0.numa_interleave 132706 +13.2% 150284 numa-vmstat.node1.nr_file_pages 2735 +14.3% 3124 numa-vmstat.node1.nr_mapped 4690 ± 6% +19.6% 5608 ± 5% numa-vmstat.node1.nr_slab_reclaimable 132706 +13.2% 150284 numa-vmstat.node1.nr_unevictable 132706 +13.2% 150284 numa-vmstat.node1.nr_zone_unevictable 155440 +11.1% 172714 numa-vmstat.node1.numa_hit 154855 +11.2% 172128 numa-vmstat.node1.numa_interleave 154868 +11.2% 172141 numa-vmstat.node1.numa_other 56.00 +150.0% 140.00 ± 20% slabinfo.btrfs_extent_map.active_objs 56.00 +150.0% 140.00 ± 20% slabinfo.btrfs_extent_map.num_objs 1568 +9.7% 1720 slabinfo.inode_cache.active_slabs 84716 +9.7% 92925 slabinfo.inode_cache.num_objs 1568 +9.7% 1720 slabinfo.inode_cache.num_slabs 524.80 ± 14% -29.9% 368.00 ± 7% slabinfo.kmalloc-rcl-128.active_objs 524.80 ± 14% -29.9% 368.00 ± 7% slabinfo.kmalloc-rcl-128.num_objs 6284 ± 4% +28.9% 8098 ± 7% slabinfo.kmalloc-rcl-64.active_objs 6284 ± 4% +28.9% 8098 ± 7% slabinfo.kmalloc-rcl-64.num_objs 28669 +16.0% 33247 slabinfo.radix_tree_node.active_objs 28669 +16.0% 33247 slabinfo.radix_tree_node.num_objs 1066 ± 10% -100.0% 0.00 slabinfo.skbuff_ext_cache.active_objs 1066 ± 10% -100.0% 0.00 slabinfo.skbuff_ext_cache.num_objs 945.80 ± 2% +10.2% 1042 ± 6% slabinfo.skbuff_fclone_cache.active_objs 945.80 ± 2% +10.2% 1042 ± 6% slabinfo.skbuff_fclone_cache.num_objs 2289 ± 2% -14.9% 1947 sched_debug.cfs_rq:/.exec_clock.stddev 4558 ± 8% -10.1% 4098 ± 3% sched_debug.cfs_rq:/.load.avg 21597 ± 25% -25.8% 16031 ± 4% sched_debug.cfs_rq:/.load.stddev 938759 ± 2% -13.0% 816467 ± 3% sched_debug.cfs_rq:/.min_vruntime.stddev 35.32 ± 32% +45.2% 51.30 ± 17% sched_debug.cfs_rq:/.nr_spread_over.max 2.90 ± 19% +35.1% 3.92 ± 3% sched_debug.cfs_rq:/.nr_spread_over.stddev -8467500 -16.3% -7085934 sched_debug.cfs_rq:/.spread0.min 641459 ± 5% -12.2% 563287 ± 5% sched_debug.cfs_rq:/.spread0.stddev 1937 ± 3% -14.2% 1662 sched_debug.cpu.clock.stddev 1937 ± 3% -14.2% 1662 sched_debug.cpu.clock_task.stddev 229162 ± 14% +15.5% 264772 ± 6% sched_debug.cpu.max_idle_balance_cost.stddev 0.00 ± 2% -14.6% 0.00 sched_debug.cpu.next_balance.stddev 17032335 +15.7% 19705462 sched_debug.cpu.sched_count.avg 41820934 +10.5% 46197041 sched_debug.cpu.sched_count.max 9425674 ± 2% +20.7% 11374378 sched_debug.cpu.sched_count.stddev 17031308 +15.7% 19704439 sched_debug.cpu.yld_count.avg 41819326 +10.5% 46194610 sched_debug.cpu.yld_count.max 9425595 ± 2% +20.7% 11374229 sched_debug.cpu.yld_count.stddev 1645 ± 17% -44.1% 919.00 ± 45% interrupts.CPU1.RES:Rescheduling_interrupts 126.40 ± 55% +337.5% 553.00 ± 97% interrupts.CPU101.RES:Rescheduling_interrupts 1103 ± 95% -77.4% 249.50 ± 29% interrupts.CPU104.RES:Rescheduling_interrupts 5548 ± 29% -37.4% 3473 interrupts.CPU139.NMI:Non-maskable_interrupts 5548 ± 29% -37.4% 3473 interrupts.CPU139.PMI:Performance_monitoring_interrupts 5486 ± 29% -37.5% 3427 interrupts.CPU147.NMI:Non-maskable_interrupts 5486 ± 29% -37.5% 3427 interrupts.CPU147.PMI:Performance_monitoring_interrupts 5554 ± 29% -37.6% 3465 interrupts.CPU148.NMI:Non-maskable_interrupts 5554 ± 29% -37.6% 3465 interrupts.CPU148.PMI:Performance_monitoring_interrupts 132.60 ± 33% -59.5% 53.75 ± 25% interrupts.CPU148.RES:Rescheduling_interrupts 454.80 ± 28% +27.7% 581.00 ± 17% interrupts.CPU15.RES:Rescheduling_interrupts 62.00 ± 85% +127.0% 140.75 ± 69% interrupts.CPU150.RES:Rescheduling_interrupts 21.60 ±100% +841.0% 203.25 ± 73% interrupts.CPU161.RES:Rescheduling_interrupts 307.00 ±137% -95.5% 13.75 ± 48% interrupts.CPU167.RES:Rescheduling_interrupts 5547 ± 30% -38.6% 3404 interrupts.CPU171.NMI:Non-maskable_interrupts 5547 ± 30% -38.6% 3404 interrupts.CPU171.PMI:Performance_monitoring_interrupts 102.20 ± 35% -90.5% 9.75 ± 18% interrupts.CPU193.RES:Rescheduling_interrupts 183.40 ± 70% -79.4% 37.75 ± 46% interrupts.CPU196.RES:Rescheduling_interrupts 5638 ± 29% -37.7% 3515 interrupts.CPU201.NMI:Non-maskable_interrupts 5638 ± 29% -37.7% 3515 interrupts.CPU201.PMI:Performance_monitoring_interrupts 268.00 ± 45% +140.2% 643.75 ± 51% interrupts.CPU21.RES:Rescheduling_interrupts 5463 ± 29% -36.5% 3468 interrupts.CPU216.NMI:Non-maskable_interrupts 5463 ± 29% -36.5% 3468 interrupts.CPU216.PMI:Performance_monitoring_interrupts 304.20 ± 34% +109.8% 638.25 ± 24% interrupts.CPU22.RES:Rescheduling_interrupts 131.60 ± 88% -66.2% 44.50 ± 49% interrupts.CPU220.RES:Rescheduling_interrupts 79.00 ± 25% -50.0% 39.50 ± 64% interrupts.CPU228.RES:Rescheduling_interrupts 423.20 ± 41% +97.7% 836.50 ± 27% interrupts.CPU24.RES:Rescheduling_interrupts 136.00 ± 38% -78.1% 29.75 ± 84% interrupts.CPU240.RES:Rescheduling_interrupts 76.80 ± 61% -74.0% 20.00 ± 68% interrupts.CPU266.RES:Rescheduling_interrupts 480.80 ± 54% +81.8% 874.00 ± 14% interrupts.CPU28.RES:Rescheduling_interrupts 229.20 ± 43% +218.2% 729.25 ± 90% interrupts.CPU29.RES:Rescheduling_interrupts 434.40 ± 95% +105.7% 893.75 ± 47% interrupts.CPU30.RES:Rescheduling_interrupts 6433 ± 21% -44.8% 3552 interrupts.CPU66.NMI:Non-maskable_interrupts 6433 ± 21% -44.8% 3552 interrupts.CPU66.PMI:Performance_monitoring_interrupts 415.00 ± 55% -79.0% 87.25 ± 16% interrupts.CPU72.RES:Rescheduling_interrupts 312.00 ± 34% -54.6% 141.50 ± 37% interrupts.CPU75.RES:Rescheduling_interrupts 841.60 ± 29% -43.8% 473.00 ± 24% interrupts.CPU8.RES:Rescheduling_interrupts 4127 ± 31% +64.4% 6787 interrupts.CPU89.NMI:Non-maskable_interrupts 4127 ± 31% +64.4% 6787 interrupts.CPU89.PMI:Performance_monitoring_interrupts 217.80 ± 31% -59.5% 88.25 ± 58% interrupts.CPU89.RES:Rescheduling_interrupts 285.00 ± 66% -79.6% 58.00 ± 63% interrupts.CPU93.RES:Rescheduling_interrupts 96.20 ± 53% +81.1% 174.25 ± 47% interrupts.CPU95.RES:Rescheduling_interrupts 471.80 ± 40% +210.9% 1466 ± 66% interrupts.CPU98.RES:Rescheduling_interrupts 10.39 -4.7 5.72 ± 2% perf-profile.calltrace.cycles-pp._raw_spin_lock.do_sched_yield.__x64_sys_sched_yield.do_syscall_64.entry_SYSCALL_64_after_hwframe 11.81 -4.3 7.56 ± 2% perf-profile.calltrace.cycles-pp.update_rq_clock.__sched_text_start.schedule.__x64_sys_sched_yield.do_syscall_64 5.25 ± 2% -2.4 2.82 ± 3% perf-profile.calltrace.cycles-pp.sched_clock.sched_clock_cpu.update_rq_clock.__sched_text_start.schedule 6.08 ± 2% -2.4 3.66 ± 3% perf-profile.calltrace.cycles-pp.sched_clock_cpu.update_rq_clock.__sched_text_start.schedule.__x64_sys_sched_yield 5.47 ± 2% -2.4 3.05 ± 3% perf-profile.calltrace.cycles-pp.native_sched_clock.sched_clock.sched_clock_cpu.update_rq_clock.__sched_text_start 3.46 -2.1 1.41 ± 2% perf-profile.calltrace.cycles-pp._raw_spin_lock.__sched_text_start.schedule.__x64_sys_sched_yield.do_syscall_64 14.72 -2.0 12.71 ± 2% perf-profile.calltrace.cycles-pp.do_sched_yield.__x64_sys_sched_yield.do_syscall_64.entry_SYSCALL_64_after_hwframe.__sched_yield 1.94 ± 3% -1.3 0.64 ± 4% perf-profile.calltrace.cycles-pp.rcu_qs.rcu_note_context_switch.__sched_text_start.schedule.__x64_sys_sched_yield 4.86 ± 12% -1.2 3.62 ± 13% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.__raw_spin_unlock_irq.__sched_text_start.schedule.__x64_sys_sched_yield 4.70 ± 11% -1.2 3.48 ± 13% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.__raw_spin_unlock_irq.__sched_text_start.schedule 5.60 ± 10% -1.1 4.54 ± 10% perf-profile.calltrace.cycles-pp.__raw_spin_unlock_irq.__sched_text_start.schedule.__x64_sys_sched_yield.do_syscall_64 3.89 ± 11% -1.0 2.87 ± 15% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.__raw_spin_unlock_irq.__sched_text_start 76.22 -0.9 75.35 perf-profile.calltrace.cycles-pp.__x64_sys_sched_yield.do_syscall_64.entry_SYSCALL_64_after_hwframe.__sched_yield 2.75 ± 10% -0.7 2.07 ± 15% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.__raw_spin_unlock_irq 1.48 ± 16% -0.7 0.81 ± 65% perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt 81.87 -0.7 81.21 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__sched_yield 1.35 ± 18% -0.6 0.71 ± 65% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt 1.28 ± 18% -0.6 0.67 ± 65% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt 0.93 ± 19% -0.6 0.36 ±101% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues 83.76 -0.5 83.26 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__sched_yield 0.71 ± 13% -0.4 0.30 ±100% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.__sched_yield 2.69 ± 2% -0.4 2.33 ± 3% perf-profile.calltrace.cycles-pp.rcu_note_context_switch.__sched_text_start.schedule.__x64_sys_sched_yield.do_syscall_64 1.28 ± 2% -0.2 1.04 ± 2% perf-profile.calltrace.cycles-pp.__list_del_entry_valid.pick_next_task_fair.__sched_text_start.schedule.__x64_sys_sched_yield 5.32 -0.2 5.09 perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.__sched_yield 0.89 ± 14% -0.2 0.67 ± 15% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.__sched_yield 0.85 ± 13% -0.2 0.64 ± 15% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.__sched_yield 0.42 ± 50% +0.1 0.55 ± 3% perf-profile.calltrace.cycles-pp.check_cfs_rq_runtime.pick_next_task_fair.__sched_text_start.schedule.__x64_sys_sched_yield 0.86 ± 2% +0.2 1.02 ± 3% perf-profile.calltrace.cycles-pp.fpregs_assert_state_consistent.do_syscall_64.entry_SYSCALL_64_after_hwframe.__sched_yield 1.36 ± 2% +0.2 1.53 ± 3% perf-profile.calltrace.cycles-pp.pick_next_entity.pick_next_task_fair.__sched_text_start.schedule.__x64_sys_sched_yield 0.64 ± 2% +0.2 0.82 perf-profile.calltrace.cycles-pp.update_min_vruntime.update_curr.pick_next_task_fair.__sched_text_start.schedule 2.44 +0.4 2.80 ± 3% perf-profile.calltrace.cycles-pp.__calc_delta.update_curr.pick_next_task_fair.__sched_text_start.schedule 58.36 +0.5 58.86 perf-profile.calltrace.cycles-pp.__sched_text_start.schedule.__x64_sys_sched_yield.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.61 +0.7 6.34 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64.__sched_yield 60.37 +1.1 61.43 perf-profile.calltrace.cycles-pp.schedule.__x64_sys_sched_yield.do_syscall_64.entry_SYSCALL_64_after_hwframe.__sched_yield 6.36 ± 2% +1.8 8.11 ± 3% perf-profile.calltrace.cycles-pp.cpuacct_charge.update_curr.pick_next_task_fair.__sched_text_start.schedule 19.28 +3.6 22.89 ± 2% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__sched_text_start.schedule.__x64_sys_sched_yield.do_syscall_64 12.50 +4.2 16.73 ± 2% perf-profile.calltrace.cycles-pp.update_curr.pick_next_task_fair.__sched_text_start.schedule.__x64_sys_sched_yield 14.19 -6.9 7.32 ± 2% perf-profile.children.cycles-pp._raw_spin_lock 12.08 -4.4 7.71 ± 2% perf-profile.children.cycles-pp.update_rq_clock 5.67 ± 2% -2.5 3.19 ± 2% perf-profile.children.cycles-pp.sched_clock 6.25 ± 2% -2.5 3.77 ± 2% perf-profile.children.cycles-pp.sched_clock_cpu 5.59 ± 2% -2.5 3.11 ± 2% perf-profile.children.cycles-pp.native_sched_clock 14.75 -2.0 12.75 ± 2% perf-profile.children.cycles-pp.do_sched_yield 1.99 ± 3% -1.3 0.67 ± 4% perf-profile.children.cycles-pp.rcu_qs 5.74 ± 10% -1.1 4.66 ± 10% perf-profile.children.cycles-pp.__raw_spin_unlock_irq 76.26 -0.9 75.39 perf-profile.children.cycles-pp.__x64_sys_sched_yield 82.34 -0.7 81.65 perf-profile.children.cycles-pp.do_syscall_64 84.21 -0.5 83.67 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 2.75 ± 2% -0.4 2.39 ± 3% perf-profile.children.cycles-pp.rcu_note_context_switch 1.32 ± 2% -0.2 1.08 ± 2% perf-profile.children.cycles-pp.__list_del_entry_valid 5.94 -0.2 5.76 perf-profile.children.cycles-pp.syscall_return_via_sysret 0.66 ± 9% -0.1 0.54 ± 15% perf-profile.children.cycles-pp.irq_exit 0.28 ± 8% -0.1 0.18 ± 16% perf-profile.children.cycles-pp._raw_spin_lock_irq 0.17 ± 19% -0.1 0.07 ± 72% perf-profile.children.cycles-pp.rcu_sched_clock_irq 0.16 ± 13% -0.1 0.07 ± 64% perf-profile.children.cycles-pp.perf_event_task_tick 0.20 ± 11% -0.1 0.13 ± 10% perf-profile.children.cycles-pp.native_irq_return_iret 0.10 ± 13% -0.1 0.04 ±107% perf-profile.children.cycles-pp.rcu_core 0.05 ± 51% +0.0 0.08 ± 6% perf-profile.children.cycles-pp.ret_from_intr 0.61 ± 2% +0.0 0.65 ± 2% perf-profile.children.cycles-pp.check_cfs_rq_runtime 0.06 ± 57% +0.0 0.10 ± 25% perf-profile.children.cycles-pp.irq_enter 0.00 +0.1 0.10 ± 11% perf-profile.children.cycles-pp.idle_cpu 0.89 +0.2 1.05 ± 3% perf-profile.children.cycles-pp.fpregs_assert_state_consistent 1.48 ± 2% +0.2 1.66 ± 3% perf-profile.children.cycles-pp.pick_next_entity 0.70 ± 2% +0.2 0.89 perf-profile.children.cycles-pp.update_min_vruntime 2.49 +0.4 2.85 ± 3% perf-profile.children.cycles-pp.__calc_delta 58.57 +0.5 59.08 perf-profile.children.cycles-pp.__sched_text_start 4.87 +0.6 5.50 perf-profile.children.cycles-pp.entry_SYSCALL_64 60.44 +1.1 61.50 perf-profile.children.cycles-pp.schedule 6.49 ± 3% +1.7 8.22 ± 2% perf-profile.children.cycles-pp.cpuacct_charge 19.54 +3.6 23.16 ± 2% perf-profile.children.cycles-pp.pick_next_task_fair 12.90 ± 2% +4.2 17.12 ± 2% perf-profile.children.cycles-pp.update_curr 14.15 -6.9 7.28 ± 2% perf-profile.self.cycles-pp._raw_spin_lock 5.30 ± 2% -2.5 2.81 ± 2% perf-profile.self.cycles-pp.native_sched_clock 5.86 -1.9 3.96 perf-profile.self.cycles-pp.update_rq_clock 1.95 ± 3% -1.3 0.62 ± 5% perf-profile.self.cycles-pp.rcu_qs 3.39 -0.6 2.79 ± 2% perf-profile.self.cycles-pp.pick_next_task_fair 0.97 ± 26% -0.4 0.56 ± 21% perf-profile.self.cycles-pp.hrtimer_interrupt 1.29 ± 2% -0.2 1.05 ± 2% perf-profile.self.cycles-pp.__list_del_entry_valid 5.92 -0.2 5.74 perf-profile.self.cycles-pp.syscall_return_via_sysret 0.27 ± 9% -0.1 0.18 ± 16% perf-profile.self.cycles-pp._raw_spin_lock_irq 0.16 ± 13% -0.1 0.07 ± 64% perf-profile.self.cycles-pp.perf_event_task_tick 0.19 ± 18% -0.1 0.11 ± 40% perf-profile.self.cycles-pp.task_tick_fair 0.10 ± 19% -0.1 0.03 ±100% perf-profile.self.cycles-pp.rcu_sched_clock_irq 0.20 ± 11% -0.1 0.13 ± 10% perf-profile.self.cycles-pp.native_irq_return_iret 0.20 ± 2% +0.0 0.22 ± 3% perf-profile.self.cycles-pp.__list_add_valid 0.55 +0.0 0.58 ± 2% perf-profile.self.cycles-pp.check_cfs_rq_runtime 0.02 ±122% +0.0 0.06 ± 6% perf-profile.self.cycles-pp.ret_from_intr 0.97 +0.1 1.03 ± 2% perf-profile.self.cycles-pp.yield_task_fair 1.02 +0.1 1.11 ± 2% perf-profile.self.cycles-pp.__x64_sys_sched_yield 0.00 +0.1 0.09 ± 11% perf-profile.self.cycles-pp.idle_cpu 0.99 ± 2% +0.2 1.15 ± 3% perf-profile.self.cycles-pp.pick_next_entity 0.83 ± 2% +0.2 0.99 ± 3% perf-profile.self.cycles-pp.fpregs_assert_state_consistent 1.84 +0.2 2.00 ± 2% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 0.73 ± 6% +0.2 0.90 perf-profile.self.cycles-pp.__raw_spin_unlock_irq 0.65 ± 2% +0.2 0.84 perf-profile.self.cycles-pp.update_min_vruntime 2.44 +0.4 2.79 ± 2% perf-profile.self.cycles-pp.__calc_delta 3.20 +0.4 3.61 perf-profile.self.cycles-pp.__sched_yield 4.05 +0.5 4.58 perf-profile.self.cycles-pp.entry_SYSCALL_64 1.79 ± 3% +0.5 2.32 ± 2% perf-profile.self.cycles-pp.schedule 0.77 +1.0 1.73 ± 3% perf-profile.self.cycles-pp.rcu_note_context_switch 6.44 ± 3% +1.7 8.17 ± 2% perf-profile.self.cycles-pp.cpuacct_charge 3.33 ± 2% +2.0 5.29 ± 2% perf-profile.self.cycles-pp.update_curr 3.11 ± 2% +2.7 5.79 perf-profile.self.cycles-pp.do_sched_yield 14.98 ± 4% +4.6 19.54 ± 2% perf-profile.self.cycles-pp.__sched_text_start 113119 ± 7% +16.3% 131545 ± 4% softirqs.CPU0.TIMER 114026 ± 6% -13.7% 98362 ± 3% softirqs.CPU10.TIMER 124842 ± 11% +15.5% 144145 ± 4% softirqs.CPU100.TIMER 120351 ± 11% +16.0% 139650 ± 4% softirqs.CPU101.TIMER 114377 ± 9% +23.7% 141538 softirqs.CPU103.TIMER 119810 ± 9% +20.0% 143803 softirqs.CPU104.TIMER 124302 ± 10% +19.0% 147917 ± 2% softirqs.CPU105.TIMER 128843 ± 6% +16.0% 149410 ± 2% softirqs.CPU106.TIMER 133894 ± 6% +14.4% 153196 softirqs.CPU107.TIMER 121089 ± 8% +19.4% 144552 ± 2% softirqs.CPU108.TIMER 116435 ± 9% +18.9% 138420 ± 6% softirqs.CPU109.TIMER 122605 ± 9% +20.6% 147851 ± 2% softirqs.CPU110.TIMER 125118 ± 11% +18.0% 147647 ± 3% softirqs.CPU111.TIMER 120414 ± 11% +24.0% 149357 ± 4% softirqs.CPU112.TIMER 118061 ± 9% +24.5% 147002 ± 2% softirqs.CPU113.TIMER 113262 ± 12% +21.7% 137886 ± 4% softirqs.CPU114.TIMER 115072 ± 8% +15.8% 133253 ± 2% softirqs.CPU115.TIMER 126037 ± 7% +17.3% 147845 ± 3% softirqs.CPU116.TIMER 125201 ± 9% +27.9% 160135 ± 5% softirqs.CPU117.TIMER 126815 ± 7% +15.3% 146183 ± 3% softirqs.CPU119.TIMER 113033 ± 11% +22.8% 138784 ± 5% softirqs.CPU120.TIMER 124716 ± 8% +20.6% 150419 ± 9% softirqs.CPU124.TIMER 119654 ± 9% +23.3% 147567 ± 2% softirqs.CPU125.TIMER 121689 ± 11% +17.8% 143395 ± 3% softirqs.CPU126.TIMER 124001 ± 7% +18.0% 146288 softirqs.CPU128.TIMER 129333 ± 8% +15.5% 149415 ± 2% softirqs.CPU129.TIMER 109263 ± 3% +24.0% 135462 ± 4% softirqs.CPU13.TIMER 120280 ± 8% +18.5% 142504 ± 2% softirqs.CPU130.TIMER 127568 ± 9% +15.9% 147805 ± 2% softirqs.CPU131.TIMER 121492 ± 10% +18.9% 144478 ± 3% softirqs.CPU132.TIMER 121701 ± 12% +20.0% 146072 ± 4% softirqs.CPU133.TIMER 118584 ± 12% +19.4% 141554 ± 2% softirqs.CPU134.TIMER 118139 ± 12% +19.3% 140941 ± 8% softirqs.CPU135.TIMER 111559 ± 10% +23.7% 137951 ± 8% softirqs.CPU136.TIMER 116550 ± 6% +21.1% 141123 ± 4% softirqs.CPU138.TIMER 119925 ± 11% +20.9% 145018 ± 3% softirqs.CPU139.TIMER 110559 ± 8% +19.2% 131773 ± 4% softirqs.CPU14.TIMER 118476 ± 9% +15.1% 136359 ± 2% softirqs.CPU140.TIMER 123705 ± 9% +17.5% 145309 ± 2% softirqs.CPU141.TIMER 119453 ± 9% +17.7% 140644 ± 2% softirqs.CPU142.TIMER 130404 ± 10% +16.3% 151657 ± 2% softirqs.CPU143.TIMER 127286 ± 8% +17.1% 149108 ± 2% softirqs.CPU144.TIMER 125136 ± 10% +16.3% 145511 softirqs.CPU145.TIMER 126142 ± 6% +15.4% 145582 ± 2% softirqs.CPU146.TIMER 125707 ± 8% +14.3% 143729 ± 2% softirqs.CPU147.TIMER 120985 ± 10% +18.4% 143290 ± 3% softirqs.CPU148.TIMER 123289 ± 11% +15.8% 142709 ± 3% softirqs.CPU149.TIMER 110288 ± 3% +23.7% 136419 ± 2% softirqs.CPU15.TIMER 118787 ± 14% +21.0% 143741 ± 3% softirqs.CPU150.TIMER 118711 ± 10% +11.9% 132818 ± 6% softirqs.CPU151.TIMER 124926 ± 7% +18.0% 147353 ± 2% softirqs.CPU152.TIMER 123757 ± 7% +15.6% 143106 ± 2% softirqs.CPU153.TIMER 128552 ± 8% +14.5% 147151 ± 2% softirqs.CPU154.TIMER 124377 ± 8% +17.8% 146546 softirqs.CPU155.TIMER 125536 ± 10% +15.9% 145472 ± 2% softirqs.CPU156.TIMER 116239 ± 9% +15.8% 134554 ± 3% softirqs.CPU157.TIMER 123726 ± 10% +16.4% 143992 ± 4% softirqs.CPU158.TIMER 123210 ± 9% +16.6% 143679 ± 2% softirqs.CPU159.TIMER 109481 ± 2% +33.6% 146300 ± 4% softirqs.CPU16.TIMER 119827 ± 7% +23.5% 148008 softirqs.CPU160.TIMER 123247 ± 9% +18.8% 146472 ± 2% softirqs.CPU161.TIMER 112096 ± 9% +19.4% 133805 ± 3% softirqs.CPU162.TIMER 115002 ± 8% +24.0% 142655 ± 3% softirqs.CPU164.TIMER 124289 ± 6% +26.0% 156586 softirqs.CPU165.TIMER 124167 ± 7% +13.2% 140537 softirqs.CPU166.TIMER 120607 ± 7% +25.4% 151242 softirqs.CPU167.TIMER 124851 ± 7% +15.4% 144078 softirqs.CPU168.TIMER 126304 ± 8% +16.0% 146520 softirqs.CPU169.TIMER 110273 ± 6% +22.0% 134573 ± 3% softirqs.CPU17.TIMER 125873 ± 8% +17.0% 147232 softirqs.CPU170.TIMER 128114 ± 6% +18.9% 152291 softirqs.CPU171.TIMER 124202 ± 11% +13.2% 140567 ± 3% softirqs.CPU172.TIMER 124056 ± 12% +18.0% 146348 ± 3% softirqs.CPU174.TIMER 121501 ± 10% +20.5% 146393 ± 2% softirqs.CPU175.TIMER 118121 ± 10% +19.3% 140906 ± 3% softirqs.CPU176.TIMER 117434 ± 11% +21.4% 142608 ± 3% softirqs.CPU177.TIMER 114173 ± 10% +15.6% 131982 ± 3% softirqs.CPU178.TIMER 116001 ± 9% +17.5% 136283 ± 2% softirqs.CPU179.TIMER 113960 ± 5% +30.4% 148616 ± 2% softirqs.CPU18.TIMER 124274 ± 7% +18.2% 146928 ± 2% softirqs.CPU180.TIMER 124996 ± 5% +17.7% 147104 ± 2% softirqs.CPU181.TIMER 111019 ± 6% +28.1% 142231 ± 3% softirqs.CPU182.TIMER 111383 ± 9% +18.5% 131989 ± 2% softirqs.CPU183.TIMER 118735 ± 8% +21.4% 144187 ± 2% softirqs.CPU185.TIMER 119430 ± 10% +20.0% 143357 ± 2% softirqs.CPU186.TIMER 119779 ± 13% +18.5% 141916 ± 2% softirqs.CPU187.TIMER 114844 ± 10% +19.3% 137044 ± 5% softirqs.CPU188.TIMER 112217 ± 10% +16.6% 130821 ± 4% softirqs.CPU189.TIMER 108078 ± 3% +18.7% 128293 ± 2% softirqs.CPU19.TIMER 128086 ± 7% +18.9% 152236 ± 2% softirqs.CPU190.TIMER 127315 ± 9% +14.8% 146133 softirqs.CPU192.TIMER 127602 ± 7% +17.6% 150031 ± 2% softirqs.CPU193.TIMER 126385 ± 7% +14.4% 144530 softirqs.CPU195.TIMER 119831 ± 7% +21.2% 145237 ± 2% softirqs.CPU196.TIMER 121831 ± 9% +17.1% 142687 ± 3% softirqs.CPU197.TIMER 116601 ± 11% +15.6% 134796 ± 5% softirqs.CPU198.TIMER 113355 ± 7% +24.9% 141568 softirqs.CPU2.TIMER 110416 ± 8% +17.0% 129173 ± 4% softirqs.CPU200.TIMER 113086 ± 11% +19.5% 135112 ± 5% softirqs.CPU201.TIMER 119543 ± 12% +18.6% 141771 ± 2% softirqs.CPU202.TIMER 118165 ± 10% +21.5% 143606 ± 2% softirqs.CPU203.TIMER 116981 ± 8% +19.0% 139231 ± 2% softirqs.CPU204.TIMER 118396 ± 10% +21.8% 144211 ± 2% softirqs.CPU206.TIMER 133254 ± 6% +12.7% 150153 ± 2% softirqs.CPU207.TIMER 129492 ± 7% +16.5% 150874 ± 3% softirqs.CPU208.TIMER 126629 ± 8% +16.3% 147271 ± 2% softirqs.CPU209.TIMER 120427 ± 7% +20.1% 144661 ± 2% softirqs.CPU210.TIMER 115844 ± 5% +23.4% 142908 ± 2% softirqs.CPU211.TIMER 123594 ± 8% +20.2% 148590 ± 3% softirqs.CPU212.TIMER 124179 ± 11% +16.0% 144109 ± 3% softirqs.CPU213.TIMER 117713 ± 10% +20.0% 141292 ± 3% softirqs.CPU214.TIMER 121682 ± 11% +13.7% 138390 softirqs.CPU215.TIMER 125855 ± 9% +14.6% 144201 ± 2% softirqs.CPU216.TIMER 119457 ± 7% +18.1% 141135 ± 2% softirqs.CPU217.TIMER 129515 ± 7% +15.2% 149225 ± 2% softirqs.CPU218.TIMER 126641 ± 8% +16.2% 147164 softirqs.CPU219.TIMER 118461 ± 12% +16.9% 138460 ± 4% softirqs.CPU220.TIMER 118142 ± 12% +17.8% 139186 ± 4% softirqs.CPU221.TIMER 123028 ± 11% +17.9% 145077 ± 4% softirqs.CPU223.TIMER 116760 ± 11% +21.0% 141309 ± 4% softirqs.CPU224.TIMER 117769 ± 13% +22.9% 144787 ± 2% softirqs.CPU225.TIMER 115921 ± 9% +21.0% 140316 ± 3% softirqs.CPU226.TIMER 117756 ± 12% +23.8% 145835 ± 4% softirqs.CPU227.TIMER 127787 ± 7% +13.1% 144484 ± 3% softirqs.CPU228.TIMER 129622 ± 8% +15.0% 149116 softirqs.CPU229.TIMER 112443 ± 9% +17.6% 132259 ± 6% softirqs.CPU230.TIMER 111976 ± 8% +17.5% 131521 ± 4% softirqs.CPU231.TIMER 120500 ± 9% +16.7% 140660 ± 3% softirqs.CPU233.TIMER 115604 ± 10% +15.7% 133702 ± 3% softirqs.CPU234.TIMER 125051 ± 11% +14.2% 142772 ± 2% softirqs.CPU235.TIMER 119846 ± 9% +22.5% 146843 ± 2% softirqs.CPU236.TIMER 125549 ± 10% +18.4% 148672 ± 3% softirqs.CPU237.TIMER 121442 ± 10% +16.6% 141632 ± 2% softirqs.CPU238.TIMER 110362 ± 3% +26.7% 139858 ± 2% softirqs.CPU24.TIMER 120794 ± 9% +17.6% 141997 softirqs.CPU240.TIMER 113824 ± 9% +17.1% 133278 ± 3% softirqs.CPU241.TIMER 119687 ± 11% +21.7% 145654 softirqs.CPU242.TIMER 115912 ± 12% +19.2% 138121 softirqs.CPU243.TIMER 110237 ± 8% +14.0% 125635 ± 7% softirqs.CPU244.TIMER 111771 ± 8% +16.2% 129872 ± 6% softirqs.CPU245.TIMER 118060 ± 11% +18.0% 139342 ± 2% softirqs.CPU246.TIMER 116693 ± 13% +20.5% 140666 softirqs.CPU247.TIMER 118072 ± 7% +28.4% 151653 softirqs.CPU248.TIMER 108763 ± 9% +26.2% 137221 softirqs.CPU249.TIMER 122016 ± 10% +19.8% 146178 softirqs.CPU250.TIMER 117815 ± 10% +17.3% 138149 softirqs.CPU251.TIMER 121858 ± 10% +19.6% 145741 ± 3% softirqs.CPU253.TIMER 107240 ± 6% +12.1% 120268 ± 4% softirqs.CPU254.TIMER 108103 ± 6% +17.9% 127505 ± 5% softirqs.CPU255.TIMER 116381 ± 12% +15.6% 134529 ± 2% softirqs.CPU256.TIMER 122438 ± 9% +17.1% 143331 ± 3% softirqs.CPU259.TIMER 120442 ± 10% +17.3% 141291 ± 3% softirqs.CPU260.TIMER 122688 ± 10% +18.3% 145140 ± 2% softirqs.CPU261.TIMER 120694 ± 8% +20.5% 145449 ± 3% softirqs.CPU262.TIMER 114187 ± 9% +11.8% 127621 ± 3% softirqs.CPU266.TIMER 107478 ± 6% +10.0% 118190 ± 6% softirqs.CPU269.TIMER 107966 ± 5% +11.4% 120301 ± 7% softirqs.CPU273.TIMER 109304 ± 6% +13.9% 124478 ± 4% softirqs.CPU276.TIMER 110399 ± 8% +16.2% 128241 ± 5% softirqs.CPU277.TIMER 106029 ± 6% +9.9% 116510 ± 7% softirqs.CPU281.TIMER 106824 ± 4% +19.7% 127862 ± 9% softirqs.CPU283.TIMER 105328 ± 5% +12.4% 118346 ± 9% softirqs.CPU284.TIMER 98787 ± 5% +11.0% 109671 ± 4% softirqs.CPU287.TIMER 109815 ± 6% +22.1% 134129 ± 4% softirqs.CPU36.TIMER 109122 ± 2% +29.2% 140981 ± 4% softirqs.CPU37.TIMER 108522 ± 2% +23.9% 134419 ± 4% softirqs.CPU38.TIMER 109707 ± 3% +26.6% 138933 ± 4% softirqs.CPU39.TIMER 107644 ± 4% +15.6% 124484 ± 5% softirqs.CPU42.TIMER 108155 ± 4% +10.8% 119800 ± 7% softirqs.CPU45.TIMER 112088 ± 7% +16.9% 130979 ± 6% softirqs.CPU47.TIMER 112818 ± 8% +14.3% 128922 ± 6% softirqs.CPU49.TIMER 110167 ± 3% +17.6% 129599 ± 4% softirqs.CPU59.TIMER 107521 ± 5% +11.1% 119499 ± 3% softirqs.CPU62.TIMER 105396 ± 4% +11.2% 117248 ± 7% softirqs.CPU66.TIMER 109136 ± 5% +9.1% 119076 ± 3% softirqs.CPU68.TIMER 107512 ± 4% +11.6% 120020 ± 5% softirqs.CPU69.TIMER 109104 ± 5% +20.5% 131425 ± 6% softirqs.CPU7.TIMER 109245 ± 6% +8.5% 118577 ± 3% softirqs.CPU70.TIMER 116561 ± 10% +24.6% 145283 ± 3% softirqs.CPU72.TIMER 42353 ± 7% -12.2% 37180 ± 9% softirqs.CPU73.RCU 119948 ± 12% +15.9% 139063 ± 3% softirqs.CPU73.TIMER 110624 ± 7% +20.1% 132912 ± 3% softirqs.CPU74.TIMER 115257 ± 12% +19.8% 138101 ± 3% softirqs.CPU75.TIMER 119711 ± 12% +16.5% 139480 ± 3% softirqs.CPU76.TIMER 117787 ± 12% +19.3% 140577 ± 4% softirqs.CPU77.TIMER 117562 ± 9% +19.8% 140828 ± 4% softirqs.CPU80.TIMER 116478 ± 8% +18.3% 137819 ± 4% softirqs.CPU81.TIMER 124014 ± 11% +13.9% 141248 ± 3% softirqs.CPU82.TIMER 120670 ± 10% +17.8% 142089 ± 2% softirqs.CPU83.TIMER 117102 ± 7% +23.9% 145050 softirqs.CPU85.TIMER 125172 ± 8% +18.8% 148643 ± 2% softirqs.CPU86.TIMER 121015 ± 8% +21.2% 146694 ± 2% softirqs.CPU87.TIMER 127523 ± 10% +14.4% 145924 ± 2% softirqs.CPU88.TIMER 119199 ± 11% +17.0% 139463 ± 3% softirqs.CPU90.TIMER 125653 ± 9% +18.7% 149146 ± 2% softirqs.CPU92.TIMER 121635 ± 10% +21.5% 147752 softirqs.CPU93.TIMER 121228 ± 8% +22.1% 147960 softirqs.CPU95.TIMER 111706 ± 8% +16.4% 130048 softirqs.CPU96.TIMER 115196 ± 11% +16.1% 133731 ± 3% softirqs.CPU97.TIMER 121196 ± 13% +20.3% 145771 softirqs.CPU98.TIMER 126502 ± 11% +18.8% 150267 softirqs.CPU99.TIMER 33761845 ± 7% +15.1% 38854076 ± 2% softirqs.TIMER *************************************************************************************************** lkp-knm01: 288 threads Intel(R) Xeon Phi(TM) CPU 7295 @ 1.50GHz with 80G memory ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode: gcc-7/performance/x86_64-rhel-7.6/process/100%/debian-x86_64-20191114.cgz/lkp-knm01/page_fault2/will-it-scale/0x11 commit: v5.7-rc3 8aebfffacf ("configfs: fix config_item refcnt leak in configfs_rmdir()") v5.7-rc3 8aebfffacfa379ba400da573a5b ---------------- --------------------------- %stddev %change %stddev \ | \ 1945 -14.1% 1671 will-it-scale.per_process_ops 978.60 ± 7% -99.8% 2.00 ± 50% will-it-scale.time.involuntary_context_switches 158244 -93.7% 9896 will-it-scale.time.maximum_resident_set_size 87626 -93.0% 6109 will-it-scale.time.minor_page_faults 6.80 ± 5% -100.0% 0.00 will-it-scale.time.percent_of_cpu_this_job_got 22.04 -99.2% 0.18 will-it-scale.time.system_time 560392 -14.1% 481535 will-it-scale.workload 90.00 -1.1% 89.00 vmstat.cpu.sy 45.31 ± 4% +5.6% 47.83 ± 2% boot-time.boot 11253 ± 4% +6.3% 11962 ± 3% boot-time.idle 294444 ± 14% -26.3% 217051 ± 4% meminfo.Mapped 1096009 +12.9% 1236982 meminfo.Unevictable 7.71 +0.9 8.60 mpstat.cpu.all.idle% 1.23 ± 4% +0.3 1.55 ± 4% mpstat.cpu.all.usr% 2.043e+08 ± 4% -15.5% 1.726e+08 numa-numastat.node0.local_node 2.043e+08 ± 4% -15.5% 1.726e+08 numa-numastat.node0.numa_hit 430.09 ± 4% -100.0% 0.00 uptime.boot 16131 ± 3% -100.0% 0.00 uptime.idle 6498 ± 10% +16.8% 7587 ± 5% slabinfo.kmalloc-rcl-64.active_objs 6498 ± 10% +16.8% 7587 ± 5% slabinfo.kmalloc-rcl-64.num_objs 706.80 ± 4% +9.6% 775.00 ± 4% slabinfo.numa_policy.active_objs 706.80 ± 4% +9.6% 775.00 ± 4% slabinfo.numa_policy.num_objs 1237 ± 13% -100.0% 0.00 slabinfo.skbuff_ext_cache.active_objs 1237 ± 13% -100.0% 0.00 slabinfo.skbuff_ext_cache.num_objs 282427 ± 14% -27.3% 205355 ± 5% numa-meminfo.node0.Mapped 565175 +12.5% 635871 numa-meminfo.node0.Unevictable 530833 +13.2% 601112 numa-meminfo.node1.FilePages 17652 ± 6% +20.6% 21280 ± 7% numa-meminfo.node1.KReclaimable 11323 +13.6% 12867 numa-meminfo.node1.Mapped 623495 +11.8% 696907 numa-meminfo.node1.MemUsed 17652 ± 6% +20.6% 21280 ± 7% numa-meminfo.node1.SReclaimable 32701 ± 5% +9.5% 35810 ± 5% numa-meminfo.node1.Slab 530833 +13.2% 601111 numa-meminfo.node1.Unevictable 107.40 ± 45% +31.3% 141.00 proc-vmstat.nr_isolated_anon 74270 ± 15% -26.0% 54987 ± 4% proc-vmstat.nr_mapped 49908 +3.7% 51731 proc-vmstat.nr_slab_reclaimable 274002 +12.9% 309245 proc-vmstat.nr_unevictable 274002 +12.9% 309245 proc-vmstat.nr_zone_unevictable 109.00 ± 40% +1585.8% 1837 ± 89% proc-vmstat.numa_hint_faults 109.00 ± 40% +1585.8% 1837 ± 89% proc-vmstat.numa_hint_faults_local 2.043e+08 ± 4% -15.5% 1.726e+08 proc-vmstat.numa_hit 2.043e+08 ± 4% -15.5% 1.726e+08 proc-vmstat.numa_local 57344 ± 38% +89.4% 108606 ± 3% proc-vmstat.numa_pte_updates 59052 ±121% -99.5% 311.50 proc-vmstat.pgactivate 2.002e+08 ± 4% -15.3% 1.695e+08 proc-vmstat.pgalloc_normal 1.942e+08 ± 5% -16.3% 1.626e+08 proc-vmstat.pgfault 2.037e+08 ± 4% -15.4% 1.722e+08 proc-vmstat.pgfree 113.20 ± 46% +30.7% 148.00 ± 6% numa-vmstat.node0.nr_isolated_anon 70785 ± 14% -29.7% 49758 ± 3% numa-vmstat.node0.nr_mapped 141293 +12.5% 158966 numa-vmstat.node0.nr_unevictable 141293 +12.5% 158966 numa-vmstat.node0.nr_zone_unevictable 1.107e+08 ± 4% -15.3% 93735997 numa-vmstat.node0.numa_hit 154693 +11.3% 172115 numa-vmstat.node0.numa_interleave 1.107e+08 ± 4% -15.3% 93737665 numa-vmstat.node0.numa_local 132708 +13.2% 150278 numa-vmstat.node1.nr_file_pages 2836 +13.6% 3222 numa-vmstat.node1.nr_mapped 4413 ± 6% +20.6% 5320 ± 7% numa-vmstat.node1.nr_slab_reclaimable 132708 +13.2% 150277 numa-vmstat.node1.nr_unevictable 132708 +13.2% 150277 numa-vmstat.node1.nr_zone_unevictable 155429 +11.1% 172634 numa-vmstat.node1.numa_hit 154844 +11.1% 172048 numa-vmstat.node1.numa_interleave 154857 +11.1% 172060 numa-vmstat.node1.numa_other 64.47 ± 9% -26.9% 47.16 ± 12% sched_debug.cfs_rq:/.nr_spread_over.avg 39.33 ± 8% -25.7% 29.23 ± 15% sched_debug.cfs_rq:/.nr_spread_over.min -10184167 -37.1% -6401034 sched_debug.cfs_rq:/.spread0.min 1352 ± 6% +14.2% 1544 sched_debug.cfs_rq:/.util_avg.max 140.69 ± 9% +16.3% 163.69 ± 5% sched_debug.cfs_rq:/.util_avg.stddev 5737899 ± 8% +22.0% 7002653 ± 2% sched_debug.cpu.avg_idle.avg 25342220 ± 7% +23.9% 31404672 sched_debug.cpu.avg_idle.max 4341642 ± 5% +26.7% 5499420 sched_debug.cpu.avg_idle.stddev 614.61 ± 5% +11.8% 687.14 ± 2% sched_debug.cpu.curr->pid.stddev 2726049 ± 8% +24.7% 3400632 sched_debug.cpu.max_idle_balance_cost.avg 11766110 ± 4% +21.6% 14305126 sched_debug.cpu.max_idle_balance_cost.max 2097272 ± 6% +28.6% 2698037 sched_debug.cpu.max_idle_balance_cost.stddev 0.00 ± 12% -46.1% 0.00 ± 4% sched_debug.cpu.next_balance.stddev 17846 ± 11% -17.3% 14763 ± 3% sched_debug.cpu.nr_switches.max 0.00 ± 55% +134.9% 0.00 ± 2% sched_debug.cpu.nr_uninterruptible.avg 237.17 ± 8% -20.9% 187.48 ± 7% sched_debug.cpu.ttwu_count.min 11.28 ± 4% -8.8 2.53 ± 32% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 11.28 ± 4% -8.8 2.53 ± 32% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 10.98 ± 4% -8.7 2.26 ± 35% perf-profile.calltrace.cycles-pp.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe 10.98 ± 4% -8.7 2.26 ± 35% perf-profile.calltrace.cycles-pp.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.44 ± 2% -0.3 3.12 ± 3% perf-profile.calltrace.cycles-pp.tlb_flush_mmu.tlb_finish_mmu.unmap_region.__do_munmap.__vm_munmap 3.44 ± 2% -0.3 3.12 ± 3% perf-profile.calltrace.cycles-pp.tlb_finish_mmu.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap 3.42 ± 2% -0.3 3.12 ± 3% perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu.tlb_finish_mmu.unmap_region.__do_munmap 3.40 ± 2% -0.3 3.10 ± 3% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.release_pages.tlb_flush_mmu.tlb_finish_mmu 3.40 ± 2% -0.3 3.10 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.release_pages.tlb_flush_mmu.tlb_finish_mmu.unmap_region 19.79 ± 2% +8.4 28.19 ± 4% perf-profile.calltrace.cycles-pp.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap 19.79 ± 2% +8.4 28.19 ± 4% perf-profile.calltrace.cycles-pp.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap 19.82 ± 2% +8.4 28.23 ± 4% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.munmap 19.82 ± 2% +8.4 28.23 ± 4% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap 19.82 ± 2% +8.4 28.23 ± 4% perf-profile.calltrace.cycles-pp.munmap 0.19 ± 34% -0.1 0.08 ± 5% perf-profile.children.cycles-pp.free_unref_page_list 0.15 ± 35% -0.1 0.06 perf-profile.children.cycles-pp.free_pcppages_bulk 19.82 ± 2% +8.4 28.23 ± 4% perf-profile.children.cycles-pp.munmap 30744 ± 22% +20.4% 37021 ± 19% softirqs.CPU0.SCHED 40150 ± 6% -14.2% 34450 ± 3% softirqs.CPU10.TIMER 40980 ± 5% -12.6% 35798 ± 4% softirqs.CPU102.TIMER 41457 ± 9% -14.0% 35671 ± 3% softirqs.CPU11.TIMER 40837 ± 6% -13.0% 35533 ± 3% softirqs.CPU116.TIMER 2991 ± 7% +137.4% 7100 ± 56% softirqs.CPU118.SCHED 40675 ± 4% -13.3% 35251 ± 3% softirqs.CPU125.TIMER 2916 ± 11% +218.9% 9298 ± 66% softirqs.CPU153.SCHED 43448 ± 9% -15.7% 36639 ± 3% softirqs.CPU16.TIMER 2948 ± 12% +131.9% 6838 ± 56% softirqs.CPU165.SCHED 39548 ± 5% -12.3% 34675 softirqs.CPU187.TIMER 43043 ± 13% -18.2% 35210 ± 3% softirqs.CPU193.TIMER 44340 ± 8% -18.1% 36293 ± 2% softirqs.CPU2.TIMER 42478 ± 14% -16.6% 35430 softirqs.CPU200.TIMER 40184 ± 4% -11.6% 35535 ± 3% softirqs.CPU213.TIMER 2891 ± 15% +125.4% 6517 ± 57% softirqs.CPU221.SCHED 4677 ± 69% -46.1% 2522 ± 2% softirqs.CPU239.SCHED 39582 ± 4% -12.0% 34848 ± 2% softirqs.CPU264.TIMER 39242 ± 3% -12.1% 34500 ± 2% softirqs.CPU273.TIMER 39720 ± 5% -11.0% 35370 softirqs.CPU274.TIMER 39463 ± 5% -14.5% 33747 ± 3% softirqs.CPU275.TIMER 39046 ± 5% -11.1% 34704 ± 2% softirqs.CPU282.TIMER 39107 ± 5% -12.9% 34052 ± 3% softirqs.CPU284.TIMER 39427 ± 4% -12.2% 34614 softirqs.CPU285.TIMER 38657 ± 5% -14.5% 33049 softirqs.CPU286.TIMER 40552 ± 4% -11.7% 35817 ± 3% softirqs.CPU32.TIMER 2980 ± 8% +292.0% 11681 ± 72% softirqs.CPU47.SCHED 41953 ± 9% -13.9% 36109 ± 4% softirqs.CPU50.TIMER 40401 ± 4% -11.1% 35926 ± 2% softirqs.CPU56.TIMER 42468 ± 8% -14.4% 36338 ± 4% softirqs.CPU58.TIMER 43418 ± 10% -17.0% 36044 softirqs.CPU60.TIMER 2199 ± 4% +185.4% 6278 ± 63% softirqs.CPU71.RCU 40825 ± 3% -12.2% 35851 ± 3% softirqs.CPU72.TIMER 2814 ± 12% +154.9% 7174 ± 56% softirqs.CPU78.SCHED 44413 ± 31% +20.6% 53565 ± 24% interrupts.CPU0.RES:Rescheduling_interrupts 1311 ± 24% -89.3% 140.00 ± 71% interrupts.CPU1.RES:Rescheduling_interrupts 25362 ± 49% +47.1% 37320 ± 36% interrupts.CPU10.NMI:Non-maskable_interrupts 25362 ± 49% +47.1% 37320 ± 36% interrupts.CPU10.PMI:Performance_monitoring_interrupts 23833 ± 44% +50.0% 35755 ± 38% interrupts.CPU100.NMI:Non-maskable_interrupts 23833 ± 44% +50.0% 35755 ± 38% interrupts.CPU100.PMI:Performance_monitoring_interrupts 21886 ± 55% +57.9% 34568 ± 26% interrupts.CPU101.NMI:Non-maskable_interrupts 21886 ± 55% +57.9% 34568 ± 26% interrupts.CPU101.PMI:Performance_monitoring_interrupts 115.60 ± 67% -68.9% 36.00 ± 13% interrupts.CPU102.RES:Rescheduling_interrupts 17917 ± 40% +90.8% 34188 ± 25% interrupts.CPU107.NMI:Non-maskable_interrupts 17917 ± 40% +90.8% 34188 ± 25% interrupts.CPU107.PMI:Performance_monitoring_interrupts 116731 ± 6% -9.9% 105211 interrupts.CPU108.LOC:Local_timer_interrupts 113992 ± 3% -10.3% 102241 interrupts.CPU109.LOC:Local_timer_interrupts 117808 -10.7% 105251 interrupts.CPU113.LOC:Local_timer_interrupts 23944 ± 43% +51.1% 36178 ± 39% interrupts.CPU113.NMI:Non-maskable_interrupts 23944 ± 43% +51.1% 36178 ± 39% interrupts.CPU113.PMI:Performance_monitoring_interrupts 112586 ± 2% -9.6% 101751 interrupts.CPU115.LOC:Local_timer_interrupts 17938 ± 39% +92.1% 34458 ± 25% interrupts.CPU115.NMI:Non-maskable_interrupts 17938 ± 39% +92.1% 34458 ± 25% interrupts.CPU115.PMI:Performance_monitoring_interrupts 119013 ± 3% -12.4% 104204 interrupts.CPU116.LOC:Local_timer_interrupts 23941 ± 46% +44.6% 34625 ± 24% interrupts.CPU116.NMI:Non-maskable_interrupts 23941 ± 46% +44.6% 34625 ± 24% interrupts.CPU116.PMI:Performance_monitoring_interrupts 62.60 ± 46% -55.3% 28.00 ± 7% interrupts.CPU117.RES:Rescheduling_interrupts 22842 ± 42% +52.1% 34744 ± 27% interrupts.CPU118.NMI:Non-maskable_interrupts 22842 ± 42% +52.1% 34744 ± 27% interrupts.CPU118.PMI:Performance_monitoring_interrupts 55.40 ± 60% +14820.6% 8266 ± 98% interrupts.CPU118.RES:Rescheduling_interrupts 119354 ± 7% -14.2% 102437 interrupts.CPU119.LOC:Local_timer_interrupts 21602 ± 17% +69.2% 36544 ± 39% interrupts.CPU119.NMI:Non-maskable_interrupts 21602 ± 17% +69.2% 36544 ± 39% interrupts.CPU119.PMI:Performance_monitoring_interrupts 19795 ± 31% +85.9% 36795 ± 40% interrupts.CPU122.NMI:Non-maskable_interrupts 19795 ± 31% +85.9% 36795 ± 40% interrupts.CPU122.PMI:Performance_monitoring_interrupts 2230 ±182% -98.3% 37.00 interrupts.CPU122.RES:Rescheduling_interrupts 23701 ± 46% +52.3% 36100 ± 39% interrupts.CPU123.NMI:Non-maskable_interrupts 23701 ± 46% +52.3% 36100 ± 39% interrupts.CPU123.PMI:Performance_monitoring_interrupts 3061 ±175% -99.0% 31.50 ± 11% interrupts.CPU124.RES:Rescheduling_interrupts 120574 ± 5% -13.6% 104157 ± 2% interrupts.CPU126.LOC:Local_timer_interrupts 23625 ± 46% +45.5% 34373 ± 25% interrupts.CPU126.NMI:Non-maskable_interrupts 23625 ± 46% +45.5% 34373 ± 25% interrupts.CPU126.PMI:Performance_monitoring_interrupts 117690 ± 4% -14.2% 100992 interrupts.CPU127.LOC:Local_timer_interrupts 26834 ± 31% +38.1% 37049 ± 30% interrupts.CPU13.NMI:Non-maskable_interrupts 26834 ± 31% +38.1% 37049 ± 30% interrupts.CPU13.PMI:Performance_monitoring_interrupts 23710 ± 46% +46.9% 34829 ± 26% interrupts.CPU131.NMI:Non-maskable_interrupts 23710 ± 46% +46.9% 34829 ± 26% interrupts.CPU131.PMI:Performance_monitoring_interrupts 121993 ± 6% -19.3% 98444 interrupts.CPU132.LOC:Local_timer_interrupts 118641 ± 5% -13.3% 102859 interrupts.CPU133.LOC:Local_timer_interrupts 116855 ± 5% -15.2% 99116 interrupts.CPU135.LOC:Local_timer_interrupts 21994 ± 56% +65.0% 36281 ± 39% interrupts.CPU136.NMI:Non-maskable_interrupts 21994 ± 56% +65.0% 36281 ± 39% interrupts.CPU136.PMI:Performance_monitoring_interrupts 24176 ± 46% +50.0% 36269 ± 39% interrupts.CPU137.NMI:Non-maskable_interrupts 24176 ± 46% +50.0% 36269 ± 39% interrupts.CPU137.PMI:Performance_monitoring_interrupts 19777 ± 31% +85.4% 36666 ± 40% interrupts.CPU139.NMI:Non-maskable_interrupts 19777 ± 31% +85.4% 36666 ± 40% interrupts.CPU139.PMI:Performance_monitoring_interrupts 27476 ± 37% +74.8% 48017 ± 7% interrupts.CPU14.NMI:Non-maskable_interrupts 27476 ± 37% +74.8% 48017 ± 7% interrupts.CPU14.PMI:Performance_monitoring_interrupts 17615 ± 39% +95.8% 34493 ± 28% interrupts.CPU141.NMI:Non-maskable_interrupts 17615 ± 39% +95.8% 34493 ± 28% interrupts.CPU141.PMI:Performance_monitoring_interrupts 19759 ± 31% +82.6% 36089 ± 38% interrupts.CPU142.NMI:Non-maskable_interrupts 19759 ± 31% +82.6% 36089 ± 38% interrupts.CPU142.PMI:Performance_monitoring_interrupts 47.60 ± 22% +15915.8% 7623 ± 98% interrupts.CPU143.RES:Rescheduling_interrupts 21504 ± 56% +59.1% 34204 ± 28% interrupts.CPU145.NMI:Non-maskable_interrupts 21504 ± 56% +59.1% 34204 ± 28% interrupts.CPU145.PMI:Performance_monitoring_interrupts 121982 ± 9% -14.0% 104947 ± 2% interrupts.CPU146.LOC:Local_timer_interrupts 121216 ± 4% -12.6% 105990 interrupts.CPU148.LOC:Local_timer_interrupts 66.60 ± 57% +795.6% 596.50 ± 82% interrupts.CPU148.RES:Rescheduling_interrupts 119881 ± 3% -13.4% 103848 interrupts.CPU15.LOC:Local_timer_interrupts 58.60 ± 73% +9627.0% 5700 ± 56% interrupts.CPU152.RES:Rescheduling_interrupts 62.60 ± 20% +19400.8% 12207 ± 99% interrupts.CPU153.RES:Rescheduling_interrupts 62.80 ± 44% +117.4% 136.50 ± 4% interrupts.CPU154.RES:Rescheduling_interrupts 81.40 ± 30% +65.2% 134.50 ± 13% interrupts.CPU157.RES:Rescheduling_interrupts 122848 ± 5% -14.8% 104642 interrupts.CPU158.LOC:Local_timer_interrupts 18722 ± 29% +78.7% 33462 ± 28% interrupts.CPU160.NMI:Non-maskable_interrupts 18722 ± 29% +78.7% 33462 ± 28% interrupts.CPU160.PMI:Performance_monitoring_interrupts 54.80 ± 81% +1773.2% 1026 ± 95% interrupts.CPU164.RES:Rescheduling_interrupts 51.80 ± 41% +14703.1% 7668 ± 99% interrupts.CPU165.RES:Rescheduling_interrupts 119687 ± 3% -14.5% 102376 interrupts.CPU167.LOC:Local_timer_interrupts 19851 ± 18% +66.0% 32947 ± 27% interrupts.CPU167.NMI:Non-maskable_interrupts 19851 ± 18% +66.0% 32947 ± 27% interrupts.CPU167.PMI:Performance_monitoring_interrupts 20526 ± 53% +62.9% 33435 ± 37% interrupts.CPU168.NMI:Non-maskable_interrupts 20526 ± 53% +62.9% 33435 ± 37% interrupts.CPU168.PMI:Performance_monitoring_interrupts 22011 ± 42% +55.3% 34182 ± 39% interrupts.CPU169.NMI:Non-maskable_interrupts 22011 ± 42% +55.3% 34182 ± 39% interrupts.CPU169.PMI:Performance_monitoring_interrupts 16490 ± 43% +105.5% 33884 ± 38% interrupts.CPU175.NMI:Non-maskable_interrupts 16490 ± 43% +105.5% 33884 ± 38% interrupts.CPU175.PMI:Performance_monitoring_interrupts 98.80 ± 49% -72.7% 27.00 ± 3% interrupts.CPU175.RES:Rescheduling_interrupts 119432 ± 6% -15.3% 101195 interrupts.CPU177.LOC:Local_timer_interrupts 21582 ± 42% +97.4% 42614 ± 7% interrupts.CPU180.NMI:Non-maskable_interrupts 21582 ± 42% +97.4% 42614 ± 7% interrupts.CPU180.PMI:Performance_monitoring_interrupts 119772 ± 3% -13.7% 103342 interrupts.CPU183.LOC:Local_timer_interrupts 114335 ± 5% -10.8% 101938 interrupts.CPU184.LOC:Local_timer_interrupts 22474 ± 43% +91.1% 42954 ± 6% interrupts.CPU184.NMI:Non-maskable_interrupts 22474 ± 43% +91.1% 42954 ± 6% interrupts.CPU184.PMI:Performance_monitoring_interrupts 67.00 ± 60% -61.2% 26.00 ± 3% interrupts.CPU184.RES:Rescheduling_interrupts 117913 ± 5% -10.6% 105418 interrupts.CPU185.LOC:Local_timer_interrupts 20531 ± 54% +51.4% 31081 ± 26% interrupts.CPU185.NMI:Non-maskable_interrupts 20531 ± 54% +51.4% 31081 ± 26% interrupts.CPU185.PMI:Performance_monitoring_interrupts 114480 ± 2% -13.8% 98655 interrupts.CPU187.LOC:Local_timer_interrupts 117946 ± 7% -13.1% 102502 ± 2% interrupts.CPU188.LOC:Local_timer_interrupts 117037 ± 6% -12.1% 102841 interrupts.CPU189.LOC:Local_timer_interrupts 20504 ± 54% +53.6% 31491 ± 25% interrupts.CPU189.NMI:Non-maskable_interrupts 20504 ± 54% +53.6% 31491 ± 25% interrupts.CPU189.PMI:Performance_monitoring_interrupts 119458 ± 5% -16.3% 99979 ± 2% interrupts.CPU191.LOC:Local_timer_interrupts 17934 ± 35% +84.9% 33158 ± 38% interrupts.CPU191.NMI:Non-maskable_interrupts 17934 ± 35% +84.9% 33158 ± 38% interrupts.CPU191.PMI:Performance_monitoring_interrupts 117361 ± 4% -11.9% 103355 interrupts.CPU193.LOC:Local_timer_interrupts 19054 ± 37% +65.3% 31490 ± 26% interrupts.CPU193.NMI:Non-maskable_interrupts 19054 ± 37% +65.3% 31490 ± 26% interrupts.CPU193.PMI:Performance_monitoring_interrupts 744.20 ±175% -94.4% 41.50 ± 18% interrupts.CPU193.RES:Rescheduling_interrupts 120732 ± 6% -15.1% 102480 interrupts.CPU194.LOC:Local_timer_interrupts 16696 ± 45% +155.6% 42678 ± 7% interrupts.CPU194.NMI:Non-maskable_interrupts 16696 ± 45% +155.6% 42678 ± 7% interrupts.CPU194.PMI:Performance_monitoring_interrupts 20990 ± 59% +51.7% 31840 ± 41% interrupts.CPU196.NMI:Non-maskable_interrupts 20990 ± 59% +51.7% 31840 ± 41% interrupts.CPU196.PMI:Performance_monitoring_interrupts 119369 ± 4% -14.2% 102421 ± 3% interrupts.CPU199.LOC:Local_timer_interrupts 16633 ± 45% +83.9% 30586 ± 27% interrupts.CPU199.NMI:Non-maskable_interrupts 16633 ± 45% +83.9% 30586 ± 27% interrupts.CPU199.PMI:Performance_monitoring_interrupts 118932 ± 7% -15.5% 100519 ± 3% interrupts.CPU2.LOC:Local_timer_interrupts 22407 ± 43% +34.9% 30220 ± 29% interrupts.CPU200.NMI:Non-maskable_interrupts 22407 ± 43% +34.9% 30220 ± 29% interrupts.CPU200.PMI:Performance_monitoring_interrupts 123037 ± 4% -13.5% 106382 ± 3% interrupts.CPU202.LOC:Local_timer_interrupts 116719 ± 6% -15.4% 98794 interrupts.CPU203.LOC:Local_timer_interrupts 117259 ± 4% -12.8% 102265 interrupts.CPU204.LOC:Local_timer_interrupts 17009 ± 47% +78.4% 30342 ± 24% interrupts.CPU205.NMI:Non-maskable_interrupts 17009 ± 47% +78.4% 30342 ± 24% interrupts.CPU205.PMI:Performance_monitoring_interrupts 3714 ±196% -99.3% 27.50 ± 20% interrupts.CPU205.RES:Rescheduling_interrupts 16910 ± 46% +91.8% 32437 ± 42% interrupts.CPU206.NMI:Non-maskable_interrupts 16910 ± 46% +91.8% 32437 ± 42% interrupts.CPU206.PMI:Performance_monitoring_interrupts 120037 ± 5% -14.5% 102637 ± 2% interrupts.CPU21.LOC:Local_timer_interrupts 117963 ± 4% -12.1% 103659 interrupts.CPU211.LOC:Local_timer_interrupts 17939 ± 35% +65.9% 29764 ± 23% interrupts.CPU212.NMI:Non-maskable_interrupts 17939 ± 35% +65.9% 29764 ± 23% interrupts.CPU212.PMI:Performance_monitoring_interrupts 115452 ± 3% -12.9% 100528 interrupts.CPU214.LOC:Local_timer_interrupts 466.80 ±178% -93.3% 31.50 ± 4% interrupts.CPU217.RES:Rescheduling_interrupts 121828 ± 5% -18.3% 99577 interrupts.CPU218.LOC:Local_timer_interrupts 1660 ±192% -98.5% 24.50 ± 30% interrupts.CPU218.RES:Rescheduling_interrupts 122819 ± 5% -13.2% 106581 interrupts.CPU22.LOC:Local_timer_interrupts 50.40 ± 22% +13769.0% 6990 ± 95% interrupts.CPU221.RES:Rescheduling_interrupts 17704 ± 35% +64.8% 29176 ± 27% interrupts.CPU222.NMI:Non-maskable_interrupts 17704 ± 35% +64.8% 29176 ± 27% interrupts.CPU222.PMI:Performance_monitoring_interrupts 61.00 ± 47% +4351.6% 2715 ± 97% interrupts.CPU222.RES:Rescheduling_interrupts 41.80 ± 32% +15075.8% 6343 ± 87% interrupts.CPU224.RES:Rescheduling_interrupts 117616 ± 7% -13.1% 102187 interrupts.CPU225.LOC:Local_timer_interrupts 56.40 ± 54% +640.2% 417.50 interrupts.CPU225.RES:Rescheduling_interrupts 25163 ± 43% +57.1% 39525 ± 11% interrupts.CPU227.NMI:Non-maskable_interrupts 25163 ± 43% +57.1% 39525 ± 11% interrupts.CPU227.PMI:Performance_monitoring_interrupts 76.60 ± 27% -56.9% 33.00 interrupts.CPU228.RES:Rescheduling_interrupts 35.20 ± 31% +1846.0% 685.00 ± 78% interrupts.CPU229.RES:Rescheduling_interrupts 17795 ± 35% +123.2% 39716 ± 8% interrupts.CPU231.NMI:Non-maskable_interrupts 17795 ± 35% +123.2% 39716 ± 8% interrupts.CPU231.PMI:Performance_monitoring_interrupts 119991 ± 4% -13.3% 104031 interrupts.CPU235.LOC:Local_timer_interrupts 66.20 ± 97% +659.1% 502.50 ± 90% interrupts.CPU237.RES:Rescheduling_interrupts 120331 ± 3% -16.0% 101114 interrupts.CPU238.LOC:Local_timer_interrupts 4018 ±122% -99.3% 27.50 interrupts.CPU238.RES:Rescheduling_interrupts 121416 ± 3% -18.1% 99384 ± 3% interrupts.CPU239.LOC:Local_timer_interrupts 134.80 ± 74% -69.6% 41.00 ± 41% interrupts.CPU24.RES:Rescheduling_interrupts 119296 ± 6% -14.1% 102465 interrupts.CPU240.LOC:Local_timer_interrupts 2321 ±192% -99.2% 19.50 ± 7% interrupts.CPU240.RES:Rescheduling_interrupts 39.00 ± 62% +2380.8% 967.50 ± 96% interrupts.CPU243.RES:Rescheduling_interrupts 16832 ± 48% +133.0% 39225 ± 9% interrupts.CPU244.NMI:Non-maskable_interrupts 16832 ± 48% +133.0% 39225 ± 9% interrupts.CPU244.PMI:Performance_monitoring_interrupts 114446 ± 5% -12.4% 100302 interrupts.CPU247.LOC:Local_timer_interrupts 45.80 ± 51% +183.8% 130.00 ± 43% interrupts.CPU247.RES:Rescheduling_interrupts 116970 ± 8% -15.8% 98458 interrupts.CPU248.LOC:Local_timer_interrupts 117471 ± 7% -13.7% 101346 interrupts.CPU249.LOC:Local_timer_interrupts 2762 ±196% -99.0% 28.50 ± 5% interrupts.CPU249.RES:Rescheduling_interrupts 122057 ± 6% -12.0% 107449 interrupts.CPU25.LOC:Local_timer_interrupts 115489 ± 5% -15.0% 98109 interrupts.CPU253.LOC:Local_timer_interrupts 24369 ± 41% +63.0% 39726 ± 7% interrupts.CPU253.NMI:Non-maskable_interrupts 24369 ± 41% +63.0% 39726 ± 7% interrupts.CPU253.PMI:Performance_monitoring_interrupts 119795 ± 3% -13.9% 103127 interrupts.CPU255.LOC:Local_timer_interrupts 114991 ± 3% -11.3% 102052 interrupts.CPU257.LOC:Local_timer_interrupts 19912 ± 49% +45.4% 28955 ± 25% interrupts.CPU259.NMI:Non-maskable_interrupts 19912 ± 49% +45.4% 28955 ± 25% interrupts.CPU259.PMI:Performance_monitoring_interrupts 126.80 ±122% +917.4% 1290 ± 94% interrupts.CPU259.RES:Rescheduling_interrupts 17936 ± 37% +59.6% 28626 ± 27% interrupts.CPU260.NMI:Non-maskable_interrupts 17936 ± 37% +59.6% 28626 ± 27% interrupts.CPU260.PMI:Performance_monitoring_interrupts 21974 ± 51% +78.4% 39200 ± 8% interrupts.CPU261.NMI:Non-maskable_interrupts 21974 ± 51% +78.4% 39200 ± 8% interrupts.CPU261.PMI:Performance_monitoring_interrupts 37.80 ± 59% +22356.3% 8488 ± 99% interrupts.CPU261.RES:Rescheduling_interrupts 115566 ± 4% -12.6% 101038 ± 4% interrupts.CPU263.LOC:Local_timer_interrupts 117096 ± 4% -13.8% 100936 ± 3% interrupts.CPU264.LOC:Local_timer_interrupts 115215 ± 5% -11.6% 101842 ± 2% interrupts.CPU265.LOC:Local_timer_interrupts 16524 ± 46% +72.2% 28453 ± 24% interrupts.CPU265.NMI:Non-maskable_interrupts 16524 ± 46% +72.2% 28453 ± 24% interrupts.CPU265.PMI:Performance_monitoring_interrupts 16588 ± 46% +75.8% 29154 ± 27% interrupts.CPU269.NMI:Non-maskable_interrupts 16588 ± 46% +75.8% 29154 ± 27% interrupts.CPU269.PMI:Performance_monitoring_interrupts 119181 ± 3% -12.3% 104485 interrupts.CPU270.LOC:Local_timer_interrupts 126083 ± 3% -18.6% 102582 interrupts.CPU271.LOC:Local_timer_interrupts 120889 ± 4% -10.4% 108294 interrupts.CPU274.LOC:Local_timer_interrupts 20501 ± 61% +86.6% 38253 ± 8% interrupts.CPU274.NMI:Non-maskable_interrupts 20501 ± 61% +86.6% 38253 ± 8% interrupts.CPU274.PMI:Performance_monitoring_interrupts 43.60 ± 38% +311.7% 179.50 ± 79% interrupts.CPU274.RES:Rescheduling_interrupts 20162 ± 27% +90.5% 38419 ± 11% interrupts.CPU276.NMI:Non-maskable_interrupts 20162 ± 27% +90.5% 38419 ± 11% interrupts.CPU276.PMI:Performance_monitoring_interrupts 469.60 ±180% -94.5% 26.00 ± 23% interrupts.CPU276.RES:Rescheduling_interrupts 119850 ± 2% -16.4% 100136 ± 2% interrupts.CPU277.LOC:Local_timer_interrupts 19782 ± 50% +39.4% 27572 ± 22% interrupts.CPU281.NMI:Non-maskable_interrupts 19782 ± 50% +39.4% 27572 ± 22% interrupts.CPU281.PMI:Performance_monitoring_interrupts 19467 ± 49% +98.5% 38647 ± 11% interrupts.CPU283.NMI:Non-maskable_interrupts 19467 ± 49% +98.5% 38647 ± 11% interrupts.CPU283.PMI:Performance_monitoring_interrupts 118186 ± 4% -12.5% 103417 interrupts.CPU284.LOC:Local_timer_interrupts 20890 ± 41% +31.7% 27517 ± 22% interrupts.CPU284.NMI:Non-maskable_interrupts 20890 ± 41% +31.7% 27517 ± 22% interrupts.CPU284.PMI:Performance_monitoring_interrupts 16703 ± 47% +79.1% 29908 ± 42% interrupts.CPU285.NMI:Non-maskable_interrupts 16703 ± 47% +79.1% 29908 ± 42% interrupts.CPU285.PMI:Performance_monitoring_interrupts 120367 ± 3% -14.3% 103207 ± 3% interrupts.CPU286.LOC:Local_timer_interrupts 203105 ± 16% -21.3% 159888 interrupts.CPU287.LOC:Local_timer_interrupts 116843 -12.2% 102631 interrupts.CPU29.LOC:Local_timer_interrupts 19918 ± 30% +82.9% 36430 ± 26% interrupts.CPU29.NMI:Non-maskable_interrupts 19918 ± 30% +82.9% 36430 ± 26% interrupts.CPU29.PMI:Performance_monitoring_interrupts 115537 ± 5% -12.8% 100799 interrupts.CPU31.LOC:Local_timer_interrupts 24936 ± 14% +42.8% 35617 ± 28% interrupts.CPU31.NMI:Non-maskable_interrupts 24936 ± 14% +42.8% 35617 ± 28% interrupts.CPU31.PMI:Performance_monitoring_interrupts 117133 ± 4% -13.5% 101372 interrupts.CPU33.LOC:Local_timer_interrupts 24316 ± 49% +99.0% 48379 ± 4% interrupts.CPU33.NMI:Non-maskable_interrupts 24316 ± 49% +99.0% 48379 ± 4% interrupts.CPU33.PMI:Performance_monitoring_interrupts 26337 ± 39% +86.4% 49100 ± 5% interrupts.CPU35.NMI:Non-maskable_interrupts 26337 ± 39% +86.4% 49100 ± 5% interrupts.CPU35.PMI:Performance_monitoring_interrupts 26228 ± 39% +43.4% 37614 ± 39% interrupts.CPU36.NMI:Non-maskable_interrupts 26228 ± 39% +43.4% 37614 ± 39% interrupts.CPU36.PMI:Performance_monitoring_interrupts 116348 ± 5% -11.5% 102962 interrupts.CPU37.LOC:Local_timer_interrupts 19410 ± 30% +85.7% 36045 ± 28% interrupts.CPU37.NMI:Non-maskable_interrupts 19410 ± 30% +85.7% 36045 ± 28% interrupts.CPU37.PMI:Performance_monitoring_interrupts 22299 ± 28% +115.7% 48099 interrupts.CPU39.NMI:Non-maskable_interrupts 22299 ± 28% +115.7% 48099 interrupts.CPU39.PMI:Performance_monitoring_interrupts 20744 ± 27% +72.7% 35832 ± 28% interrupts.CPU4.NMI:Non-maskable_interrupts 20744 ± 27% +72.7% 35832 ± 28% interrupts.CPU4.PMI:Performance_monitoring_interrupts 378.60 ± 83% +694.8% 3009 ± 34% interrupts.CPU4.RES:Rescheduling_interrupts 118387 ± 4% -10.7% 105718 interrupts.CPU40.LOC:Local_timer_interrupts 25688 ± 35% +47.2% 37822 ± 37% interrupts.CPU40.NMI:Non-maskable_interrupts 25688 ± 35% +47.2% 37822 ± 37% interrupts.CPU40.PMI:Performance_monitoring_interrupts 118362 ± 4% -12.7% 103323 ± 2% interrupts.CPU41.LOC:Local_timer_interrupts 92.20 ± 94% -74.0% 24.00 ± 4% interrupts.CPU44.RES:Rescheduling_interrupts 121354 ± 7% -15.5% 102554 ± 2% interrupts.CPU46.LOC:Local_timer_interrupts 119439 ± 4% -12.3% 104763 ± 3% interrupts.CPU47.LOC:Local_timer_interrupts 118786 ± 7% -14.9% 101047 interrupts.CPU49.LOC:Local_timer_interrupts 123.60 ± 51% +3720.4% 4722 ± 96% interrupts.CPU5.RES:Rescheduling_interrupts 119935 ± 6% -15.3% 101599 ± 2% interrupts.CPU50.LOC:Local_timer_interrupts 25639 ± 42% +48.5% 38066 ± 38% interrupts.CPU53.NMI:Non-maskable_interrupts 25639 ± 42% +48.5% 38066 ± 38% interrupts.CPU53.PMI:Performance_monitoring_interrupts 123762 ± 6% -17.3% 102351 interrupts.CPU55.LOC:Local_timer_interrupts 120248 ± 6% -15.6% 101459 interrupts.CPU61.LOC:Local_timer_interrupts 27467 ± 31% +33.8% 36765 ± 27% interrupts.CPU61.NMI:Non-maskable_interrupts 27467 ± 31% +33.8% 36765 ± 27% interrupts.CPU61.PMI:Performance_monitoring_interrupts 63.20 ± 38% +2386.6% 1571 ± 90% interrupts.CPU62.RES:Rescheduling_interrupts 120083 ± 7% -13.0% 104487 interrupts.CPU64.LOC:Local_timer_interrupts 22757 ± 50% +67.5% 38115 ± 39% interrupts.CPU67.NMI:Non-maskable_interrupts 22757 ± 50% +67.5% 38115 ± 39% interrupts.CPU67.PMI:Performance_monitoring_interrupts 25527 ± 54% +90.2% 48549 ± 9% interrupts.CPU73.NMI:Non-maskable_interrupts 25527 ± 54% +90.2% 48549 ± 9% interrupts.CPU73.PMI:Performance_monitoring_interrupts 1079 ±187% -96.2% 41.00 ± 7% interrupts.CPU73.RES:Rescheduling_interrupts 116963 ± 5% -11.9% 103068 interrupts.CPU74.LOC:Local_timer_interrupts 119414 ± 5% -12.3% 104777 interrupts.CPU76.LOC:Local_timer_interrupts 96.40 ± 93% +8929.0% 8704 ± 99% interrupts.CPU78.RES:Rescheduling_interrupts 119473 ± 4% -10.1% 107439 interrupts.CPU8.LOC:Local_timer_interrupts 101.40 ± 40% +1138.2% 1255 ± 90% interrupts.CPU8.RES:Rescheduling_interrupts 100.40 ± 35% +2870.6% 2982 ± 8% interrupts.CPU80.RES:Rescheduling_interrupts 63.80 ± 18% +1049.7% 733.50 ± 85% interrupts.CPU81.RES:Rescheduling_interrupts 121318 ± 6% -12.7% 105910 interrupts.CPU82.LOC:Local_timer_interrupts 20126 ± 26% +67.5% 33715 ± 24% interrupts.CPU83.NMI:Non-maskable_interrupts 20126 ± 26% +67.5% 33715 ± 24% interrupts.CPU83.PMI:Performance_monitoring_interrupts 67.20 ± 46% +8900.7% 6048 ± 98% interrupts.CPU84.RES:Rescheduling_interrupts 20042 ± 27% +79.5% 35969 ± 40% interrupts.CPU86.NMI:Non-maskable_interrupts 20042 ± 27% +79.5% 35969 ± 40% interrupts.CPU86.PMI:Performance_monitoring_interrupts 50.80 ± 31% +566.3% 338.50 ± 85% interrupts.CPU88.RES:Rescheduling_interrupts 22713 ± 17% +66.5% 37828 ± 36% interrupts.CPU9.NMI:Non-maskable_interrupts 22713 ± 17% +66.5% 37828 ± 36% interrupts.CPU9.PMI:Performance_monitoring_interrupts 250.80 ±119% +735.3% 2095 ± 70% interrupts.CPU9.RES:Rescheduling_interrupts 21922 ± 53% +53.3% 33598 ± 25% interrupts.CPU90.NMI:Non-maskable_interrupts 21922 ± 53% +53.3% 33598 ± 25% interrupts.CPU90.PMI:Performance_monitoring_interrupts 47.20 ± 39% +280.3% 179.50 ± 70% interrupts.CPU92.RES:Rescheduling_interrupts 23428 ± 43% +53.5% 35964 ± 40% interrupts.CPU93.NMI:Non-maskable_interrupts 23428 ± 43% +53.5% 35964 ± 40% interrupts.CPU93.PMI:Performance_monitoring_interrupts 121016 ± 7% -12.7% 105693 ± 2% interrupts.CPU94.LOC:Local_timer_interrupts 17635 ± 39% +99.2% 35125 ± 40% interrupts.CPU95.NMI:Non-maskable_interrupts 17635 ± 39% +99.2% 35125 ± 40% interrupts.CPU95.PMI:Performance_monitoring_interrupts 25621 ± 32% +33.1% 34090 ± 27% interrupts.CPU96.NMI:Non-maskable_interrupts 25621 ± 32% +33.1% 34090 ± 27% interrupts.CPU96.PMI:Performance_monitoring_interrupts 118451 ± 6% -12.9% 103182 interrupts.CPU97.LOC:Local_timer_interrupts Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen