Greeting, FYI, we noticed a 262.8% improvement of stress-ng.msg.ops_per_sec due to commit: commit: 72d1e611082eda18689106a0c192f2827072713c ("ipc/msg: mitigate the lock contention with percpu counter") https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git mm-everything in testcase: stress-ng on test machine: 96 threads 2 sockets Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz (Cascade Lake) with 512G memory with following parameters: nr_threads: 100% testtime: 60s sc_pid_max: 4194304 class: scheduler test: msg cpufreq_governor: performance Details are as below: ========================================================================================= class/compiler/cpufreq_governor/kconfig/nr_threads/rootfs/sc_pid_max/tbox_group/test/testcase/testtime: scheduler/gcc-11/performance/x86_64-rhel-8.3/100%/debian-11.1-x86_64-20220510.cgz/4194304/lkp-csl-2sp7/msg/stress-ng/60s commit: 5d0ce3595a ("percpu: add percpu_counter_add_local and percpu_counter_sub_local") 72d1e61108 ("ipc/msg: mitigate the lock contention with percpu counter") 5d0ce3595ab75330 72d1e611082eda18689106a0c19 ---------------- --------------------------- %stddev %change %stddev \ | \ 3.141e+08 +263.2% 1.141e+09 stress-ng.msg.ops 5192613 +262.8% 18837790 stress-ng.msg.ops_per_sec 260380 +42.8% 371816 ± 3% stress-ng.time.involuntary_context_switches 22498 -10.5% 20124 stress-ng.time.minor_page_faults 6813 -31.5% 4669 stress-ng.time.percent_of_cpu_this_job_got 4222 -33.8% 2794 stress-ng.time.system_time 52.03 +162.4% 136.53 ± 3% stress-ng.time.user_time 47960019 -35.2% 31093209 stress-ng.time.voluntary_context_switches 20.33 -0.8% 20.17 boot-time.dhcp 4844 +27.3% 6167 uptime.idle 1.75e+09 +77.6% 3.109e+09 cpuidle..time 41428140 -36.2% 26440706 cpuidle..usage 26987 ± 5% -12.6% 23598 ± 5% meminfo.Active 26350 ± 6% -12.9% 22961 ± 6% meminfo.Active(anon) 36104 ± 6% -14.9% 30713 ± 5% meminfo.Shmem 28.42 +22.3 50.70 mpstat.cpu.all.idle% 1.14 ± 2% -0.1 1.00 ± 2% mpstat.cpu.all.irq% 69.22 -23.5 45.71 mpstat.cpu.all.sys% 1.12 +1.4 2.49 ± 3% mpstat.cpu.all.usr% 2225561 ± 41% -58.6% 920422 ±132% numa-meminfo.node0.Unevictable 25879 ± 6% -15.6% 21834 ± 7% numa-meminfo.node1.Active 25243 ± 6% -14.3% 21622 ± 7% numa-meminfo.node1.Active(anon) 33938 ± 7% -19.4% 27346 ± 10% numa-meminfo.node1.Shmem 30.00 +72.2% 51.67 vmstat.cpu.id 72.33 ± 2% -40.3% 43.17 ± 7% vmstat.procs.r 1408523 -35.3% 910912 vmstat.system.cs 363813 -15.3% 308282 vmstat.system.in 556389 ± 41% -58.6% 230105 ±132% numa-vmstat.node0.nr_unevictable 556389 ± 41% -58.6% 230105 ±132% numa-vmstat.node0.nr_zone_unevictable 6310 ± 6% -14.3% 5405 ± 7% numa-vmstat.node1.nr_active_anon 8484 ± 7% -19.4% 6836 ± 10% numa-vmstat.node1.nr_shmem 6310 ± 6% -14.3% 5405 ± 7% numa-vmstat.node1.nr_zone_active_anon 6587 ± 6% -12.9% 5740 ± 6% proc-vmstat.nr_active_anon 10635 ± 2% -6.1% 9987 proc-vmstat.nr_mapped 9025 ± 6% -14.9% 7677 ± 5% proc-vmstat.nr_shmem 6587 ± 6% -12.9% 5740 ± 6% proc-vmstat.nr_zone_active_anon 17181 ± 6% -16.6% 14323 ± 6% proc-vmstat.pgactivate 2066 -30.4% 1437 turbostat.Avg_MHz 74.06 -22.5 51.58 turbostat.Busy% 23899322 -74.2% 6171939 ± 21% turbostat.C1 5624190 ± 3% +232.6% 18703687 ± 7% turbostat.C1E 16.26 ± 15% +22.3 38.57 ± 8% turbostat.C1E% 25.52 +86.3% 47.55 turbostat.CPU%c1 0.04 +225.0% 0.13 turbostat.IPC 24328141 -15.3% 20609928 turbostat.IRQ 11586443 -88.5% 1326977 ± 8% turbostat.POLL 0.82 -0.5 0.28 ± 9% turbostat.POLL% 56.33 +3.8% 58.50 turbostat.PkgTmp 239.06 +5.8% 252.82 turbostat.PkgWatt 38.95 +6.6% 41.53 turbostat.RAMWatt 762.67 ± 29% -29.3% 539.17 sched_debug.cfs_rq:/.load_avg.max 1449885 -51.6% 701690 sched_debug.cfs_rq:/.min_vruntime.avg 1555986 -52.4% 740207 sched_debug.cfs_rq:/.min_vruntime.max 1350339 -49.0% 689030 sched_debug.cfs_rq:/.min_vruntime.min 69465 ± 12% -87.8% 8445 ± 12% sched_debug.cfs_rq:/.min_vruntime.stddev 535.56 ± 4% -25.4% 399.38 ± 2% sched_debug.cfs_rq:/.runnable_avg.avg 1895 ± 12% -33.0% 1270 ± 12% sched_debug.cfs_rq:/.runnable_avg.max 357.02 ± 7% -23.8% 272.01 ± 5% sched_debug.cfs_rq:/.runnable_avg.stddev 53257 ± 18% -109.8% -5230 sched_debug.cfs_rq:/.spread0.avg 159409 ± 16% -79.1% 33308 ± 39% sched_debug.cfs_rq:/.spread0.max -46234 -61.2% -17921 sched_debug.cfs_rq:/.spread0.min 69434 ± 12% -87.8% 8441 ± 12% sched_debug.cfs_rq:/.spread0.stddev 495.72 ± 4% -20.1% 396.07 ± 2% sched_debug.cfs_rq:/.util_avg.avg 1796 ± 12% -29.7% 1261 ± 12% sched_debug.cfs_rq:/.util_avg.max 338.68 ± 7% -20.1% 270.73 ± 5% sched_debug.cfs_rq:/.util_avg.stddev 129.36 ± 26% -45.5% 70.54 ± 6% sched_debug.cfs_rq:/.util_est_enqueued.avg 897.17 ± 9% -23.5% 686.00 ± 12% sched_debug.cfs_rq:/.util_est_enqueued.max 165.67 ± 5% -21.9% 129.32 ± 3% sched_debug.cfs_rq:/.util_est_enqueued.stddev 3063 ± 12% +128.6% 7003 ± 38% sched_debug.cpu.avg_idle.min 203936 ± 5% -17.7% 167885 ± 3% sched_debug.cpu.avg_idle.stddev 0.00 ± 10% -25.3% 0.00 ± 11% sched_debug.cpu.next_balance.stddev 468755 -35.1% 304073 sched_debug.cpu.nr_switches.avg 694515 ± 3% -45.5% 378328 sched_debug.cpu.nr_switches.max 328982 ± 3% -16.6% 274253 sched_debug.cpu.nr_switches.min 112645 ± 11% -83.0% 19135 ± 9% sched_debug.cpu.nr_switches.stddev 2.185e+09 ± 4% -23.0% 1.681e+09 ± 8% sched_debug.cpu.nr_uninterruptible.avg 5.392e+09 +138.5% 1.286e+10 perf-stat.i.branch-instructions 1.17 -0.3 0.85 perf-stat.i.branch-miss-rate% 58802448 +58.7% 93338979 perf-stat.i.branch-misses 22.60 -2.2 20.40 perf-stat.i.cache-miss-rate% 92976185 +122.5% 2.069e+08 perf-stat.i.cache-misses 4.09e+08 +147.2% 1.011e+09 perf-stat.i.cache-references 1487973 -35.3% 962275 perf-stat.i.context-switches 7.73 -71.8% 2.18 perf-stat.i.cpi 2.044e+11 -30.8% 1.415e+11 perf-stat.i.cpu-cycles 55597 +101.4% 111973 perf-stat.i.cpu-migrations 2209 -66.7% 735.14 ± 2% perf-stat.i.cycles-between-cache-misses 0.02 ± 10% -0.0 0.01 ± 6% perf-stat.i.dTLB-load-miss-rate% 1268585 ± 7% +81.9% 2308078 ± 8% perf-stat.i.dTLB-load-misses 6.963e+09 +146.5% 1.716e+10 perf-stat.i.dTLB-loads 0.00 ± 8% -0.0 0.00 ± 8% perf-stat.i.dTLB-store-miss-rate% 146770 ± 3% +117.8% 319631 ± 4% perf-stat.i.dTLB-store-misses 3.623e+09 +177.1% 1.004e+10 perf-stat.i.dTLB-stores 54.35 +16.9 71.22 perf-stat.i.iTLB-load-miss-rate% 9731875 +119.0% 21313415 ± 4% perf-stat.i.iTLB-load-misses 2.59e+10 +147.2% 6.403e+10 perf-stat.i.instructions 2979 +10.8% 3301 ± 3% perf-stat.i.instructions-per-iTLB-miss 0.16 +199.8% 0.47 perf-stat.i.ipc 2.13 -30.8% 1.47 perf-stat.i.metric.GHz 460.63 +126.4% 1042 perf-stat.i.metric.K/sec 170.65 +150.7% 427.80 perf-stat.i.metric.M/sec 95.23 +1.1 96.30 perf-stat.i.node-load-miss-rate% 21792545 +192.0% 63637628 perf-stat.i.node-load-misses 855255 ± 2% +122.2% 1900158 perf-stat.i.node-loads 10944429 +120.3% 24108332 perf-stat.i.node-store-misses 1.09 -0.4 0.73 perf-stat.overall.branch-miss-rate% 22.72 -2.3 20.46 perf-stat.overall.cache-miss-rate% 7.89 -72.0% 2.21 perf-stat.overall.cpi 2198 -68.9% 684.17 perf-stat.overall.cycles-between-cache-misses 0.02 ± 7% -0.0 0.01 ± 7% perf-stat.overall.dTLB-load-miss-rate% 0.00 ± 3% -0.0 0.00 ± 5% perf-stat.overall.dTLB-store-miss-rate% 55.25 +17.5 72.73 perf-stat.overall.iTLB-load-miss-rate% 2661 +13.1% 3009 ± 4% perf-stat.overall.instructions-per-iTLB-miss 0.13 +257.0% 0.45 perf-stat.overall.ipc 5.309e+09 +138.4% 1.266e+10 perf-stat.ps.branch-instructions 57807111 +58.9% 91875333 perf-stat.ps.branch-misses 91543136 +122.4% 2.036e+08 perf-stat.ps.cache-misses 4.03e+08 +147.0% 9.953e+08 perf-stat.ps.cache-references 1466557 -35.4% 947266 perf-stat.ps.context-switches 2.013e+11 -30.8% 1.393e+11 perf-stat.ps.cpu-cycles 54749 +101.3% 110229 perf-stat.ps.cpu-migrations 1251114 ± 7% +81.5% 2271328 ± 8% perf-stat.ps.dTLB-load-misses 6.857e+09 +146.4% 1.69e+10 perf-stat.ps.dTLB-loads 144598 ± 3% +117.6% 314612 ± 4% perf-stat.ps.dTLB-store-misses 3.569e+09 +176.9% 9.883e+09 perf-stat.ps.dTLB-stores 9587082 +118.8% 20981028 ± 4% perf-stat.ps.iTLB-load-misses 2.551e+10 +147.1% 6.303e+10 perf-stat.ps.instructions 21457539 +192.0% 62646387 perf-stat.ps.node-load-misses 843598 ± 2% +121.8% 1870813 perf-stat.ps.node-loads 10776219 +120.2% 23732746 perf-stat.ps.node-store-misses 1.641e+12 +147.4% 4.059e+12 perf-stat.total.instructions 83.06 -24.3 58.74 perf-profile.calltrace.cycles-pp.stress_run 45.22 -21.0 24.23 perf-profile.calltrace.cycles-pp.do_msgsnd.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_msgsnd.stress_run 45.66 -19.5 26.12 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_msgsnd.stress_run 45.72 -19.2 26.54 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_msgsnd.stress_run 46.10 -18.1 27.97 perf-profile.calltrace.cycles-pp.__libc_msgsnd.stress_run 12.26 -8.9 3.34 perf-profile.calltrace.cycles-pp._raw_spin_lock.do_msgrcv.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_msgrcv 35.66 -8.6 27.03 perf-profile.calltrace.cycles-pp.do_msgrcv.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_msgrcv.stress_run 35.96 -8.5 27.48 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_msgrcv.stress_run 36.02 -8.3 27.70 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_msgrcv.stress_run 9.60 -7.8 1.79 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.do_msgrcv.do_syscall_64.entry_SYSCALL_64_after_hwframe 36.48 -7.5 29.01 perf-profile.calltrace.cycles-pp.__libc_msgrcv.stress_run 10.97 -7.0 3.93 perf-profile.calltrace.cycles-pp._raw_spin_lock.do_msgsnd.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_msgsnd 8.64 ± 2% -6.4 2.29 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.do_msgsnd.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.79 ± 8% -1.8 3.02 ± 7% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.mwait_idle_with_hints.intel_idle_irq.cpuidle_enter_state.cpuidle_enter 2.12 -0.2 1.95 ± 4% perf-profile.calltrace.cycles-pp.schedule.do_msgrcv.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_msgrcv 2.10 -0.2 1.93 ± 4% perf-profile.calltrace.cycles-pp.__schedule.schedule.do_msgrcv.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.66 -0.1 0.56 ± 2% perf-profile.calltrace.cycles-pp.flush_smp_call_function_queue.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 1.01 +0.2 1.22 ± 4% perf-profile.calltrace.cycles-pp.try_to_wake_up.wake_up_q.do_msgsnd.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.58 ± 4% +0.4 1.00 ± 7% perf-profile.calltrace.cycles-pp.update_sg_lb_stats.update_sd_lb_stats.find_busiest_group.load_balance.newidle_balance 1.12 +0.4 1.54 ± 3% perf-profile.calltrace.cycles-pp.wake_up_q.do_msgsnd.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_msgsnd 0.63 ± 5% +0.4 1.08 ± 6% perf-profile.calltrace.cycles-pp.update_sd_lb_stats.find_busiest_group.load_balance.newidle_balance.pick_next_task_fair 0.64 ± 5% +0.4 1.09 ± 6% perf-profile.calltrace.cycles-pp.find_busiest_group.load_balance.newidle_balance.pick_next_task_fair.__schedule 0.93 ± 4% +0.5 1.40 ± 6% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.do_msgrcv.do_syscall_64 0.91 ± 4% +0.5 1.39 ± 6% perf-profile.calltrace.cycles-pp.newidle_balance.pick_next_task_fair.__schedule.schedule.do_msgrcv 0.76 ± 5% +0.5 1.26 ± 6% perf-profile.calltrace.cycles-pp.load_balance.newidle_balance.pick_next_task_fair.__schedule.schedule 0.00 +0.5 0.53 ± 2% perf-profile.calltrace.cycles-pp.__check_heap_object.__check_object_size.store_msg.do_msg_fill.do_msgrcv 0.00 +0.5 0.55 ± 4% perf-profile.calltrace.cycles-pp.get_obj_cgroup_from_current.__kmalloc.alloc_msg.load_msg.do_msgsnd 0.00 +0.6 0.56 perf-profile.calltrace.cycles-pp.__schedule.schedule.rwsem_down_write_slowpath.msgctl_down.ksys_msgctl 0.00 +0.6 0.56 perf-profile.calltrace.cycles-pp.schedule.rwsem_down_write_slowpath.msgctl_down.ksys_msgctl.do_syscall_64 0.00 +0.6 0.57 ± 3% perf-profile.calltrace.cycles-pp.__radix_tree_lookup.ipc_obtain_object_check.do_msgsnd.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +0.6 0.58 ± 3% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irq.rwsem_down_read_slowpath.sysvipc_proc_start.seq_read_iter 0.00 +0.6 0.60 perf-profile.calltrace.cycles-pp.__schedule.schedule.rwsem_down_read_slowpath.msgctl_info.ksys_msgctl 0.00 +0.6 0.60 ± 3% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irq.rwsem_down_write_slowpath.msgctl_down.ksys_msgctl 0.00 +0.6 0.60 ± 2% perf-profile.calltrace.cycles-pp.schedule.rwsem_down_read_slowpath.msgctl_info.ksys_msgctl.do_syscall_64 0.00 +0.6 0.62 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.rwsem_down_read_slowpath.sysvipc_proc_start.seq_read_iter.seq_read 0.00 +0.7 0.66 perf-profile.calltrace.cycles-pp.rwsem_wake.msgctl_down.ksys_msgctl.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +0.7 0.67 ± 2% perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.rwsem_down_write_slowpath.msgctl_down.ksys_msgctl.do_syscall_64 0.00 +0.7 0.71 ± 2% perf-profile.calltrace.cycles-pp.__entry_text_start.__libc_msgsnd.stress_run 0.00 +0.7 0.72 perf-profile.calltrace.cycles-pp.__entry_text_start.__libc_msgrcv.stress_run 0.00 +0.8 0.81 perf-profile.calltrace.cycles-pp.ss_wakeup.do_msgrcv.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_msgrcv 0.00 +0.9 0.86 perf-profile.calltrace.cycles-pp._copy_to_user.store_msg.do_msg_fill.do_msgrcv.do_syscall_64 0.00 +0.9 0.90 perf-profile.calltrace.cycles-pp.stress_msg.stress_run 0.00 +0.9 0.92 perf-profile.calltrace.cycles-pp.__x64_sys_msgsnd.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_msgsnd.stress_run 0.00 +0.9 0.94 ± 3% perf-profile.calltrace.cycles-pp.osq_lock.rwsem_optimistic_spin.rwsem_down_write_slowpath.msgctl_down.ksys_msgctl 0.00 +1.0 1.04 ± 2% perf-profile.calltrace.cycles-pp.__check_object_size.load_msg.do_msgsnd.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +1.1 1.07 ± 3% perf-profile.calltrace.cycles-pp.rwsem_down_read_slowpath.sysvipc_proc_start.seq_read_iter.seq_read.vfs_read 0.00 +1.1 1.09 perf-profile.calltrace.cycles-pp.__list_del_entry_valid.do_msgrcv.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_msgrcv 0.00 +1.1 1.10 perf-profile.calltrace.cycles-pp.check_heap_object.__check_object_size.store_msg.do_msg_fill.do_msgrcv 0.00 +1.1 1.11 ± 2% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irq.rwsem_down_read_slowpath.msgctl_info.ksys_msgctl 0.00 +1.1 1.14 ± 2% perf-profile.calltrace.cycles-pp.sysvipc_proc_start.seq_read_iter.seq_read.vfs_read.ksys_read 0.00 +1.2 1.21 perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.rwsem_down_read_slowpath.msgctl_info.ksys_msgctl.do_syscall_64 0.00 +1.3 1.28 ± 3% perf-profile.calltrace.cycles-pp.rwsem_optimistic_spin.rwsem_down_write_slowpath.msgctl_down.ksys_msgctl.do_syscall_64 0.00 +1.3 1.29 perf-profile.calltrace.cycles-pp._copy_from_user.load_msg.do_msgsnd.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.08 ± 2% +1.4 2.50 perf-profile.calltrace.cycles-pp.__slab_free.free_msg.do_msgrcv.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +1.6 1.64 perf-profile.calltrace.cycles-pp.percpu_counter_add_batch.do_msgrcv.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_msgrcv 0.00 +1.7 1.73 perf-profile.calltrace.cycles-pp.percpu_counter_add_batch.do_msgsnd.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_msgsnd 0.60 +1.7 2.34 perf-profile.calltrace.cycles-pp.kfree.free_msg.do_msgrcv.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +1.8 1.83 perf-profile.calltrace.cycles-pp.__check_object_size.store_msg.do_msg_fill.do_msgrcv.do_syscall_64 0.00 +1.9 1.88 ± 2% perf-profile.calltrace.cycles-pp.seq_read_iter.seq_read.vfs_read.ksys_read.do_syscall_64 0.00 +1.9 1.88 ± 2% perf-profile.calltrace.cycles-pp.seq_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +1.9 1.90 ± 2% perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 0.00 +1.9 1.90 ± 2% perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 0.00 +1.9 1.91 ± 2% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read 0.00 +1.9 1.91 ± 2% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 0.00 +1.9 1.92 ± 2% perf-profile.calltrace.cycles-pp.read 0.51 ± 2% +2.0 2.49 perf-profile.calltrace.cycles-pp.memcg_slab_post_alloc_hook.__kmalloc.alloc_msg.load_msg.do_msgsnd 0.00 +2.1 2.13 perf-profile.calltrace.cycles-pp.rwsem_down_read_slowpath.msgctl_info.ksys_msgctl.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +2.7 2.69 perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.msgctl_down.ksys_msgctl.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +2.7 2.70 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__percpu_counter_sum.msgctl_info.ksys_msgctl 0.82 +2.7 3.56 perf-profile.calltrace.cycles-pp.store_msg.do_msg_fill.do_msgrcv.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +2.8 2.76 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__percpu_counter_sum.msgctl_info.ksys_msgctl.do_syscall_64 0.53 ± 2% +3.1 3.65 perf-profile.calltrace.cycles-pp.msgctl_down.ksys_msgctl.do_syscall_64.entry_SYSCALL_64_after_hwframe.msgctl 1.77 +3.3 5.05 perf-profile.calltrace.cycles-pp.free_msg.do_msgrcv.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_msgrcv 0.96 +3.3 4.26 perf-profile.calltrace.cycles-pp.__kmalloc.alloc_msg.load_msg.do_msgsnd.do_syscall_64 1.09 +3.4 4.46 perf-profile.calltrace.cycles-pp.do_msg_fill.do_msgrcv.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_msgrcv 0.99 +3.4 4.40 perf-profile.calltrace.cycles-pp.alloc_msg.load_msg.do_msgsnd.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +3.6 3.55 perf-profile.calltrace.cycles-pp.__percpu_counter_sum.msgctl_info.ksys_msgctl.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +3.9 3.91 perf-profile.calltrace.cycles-pp.idr_find.ipc_obtain_object_check.do_msgsnd.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +4.0 4.00 perf-profile.calltrace.cycles-pp.idr_find.ipc_obtain_object_check.do_msgrcv.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.66 +5.3 6.96 perf-profile.calltrace.cycles-pp.load_msg.do_msgsnd.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_msgsnd 0.78 +5.5 6.28 perf-profile.calltrace.cycles-pp.ipc_obtain_object_check.do_msgrcv.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_msgrcv 0.84 +5.5 6.37 perf-profile.calltrace.cycles-pp.ipc_obtain_object_check.do_msgsnd.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_msgsnd 0.00 +6.2 6.19 perf-profile.calltrace.cycles-pp.msgctl_info.ksys_msgctl.do_syscall_64.entry_SYSCALL_64_after_hwframe.msgctl 1.04 ± 2% +9.0 10.05 perf-profile.calltrace.cycles-pp.ksys_msgctl.do_syscall_64.entry_SYSCALL_64_after_hwframe.msgctl 1.06 ± 2% +9.0 10.10 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.msgctl 1.06 ± 2% +9.1 10.12 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.msgctl 1.09 ± 2% +9.1 10.21 perf-profile.calltrace.cycles-pp.msgctl 15.18 ± 4% +13.4 28.63 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 15.18 ± 4% +13.4 28.63 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify 15.17 ± 4% +13.5 28.62 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 15.37 ± 4% +13.5 28.92 perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify 13.06 ± 5% +14.0 27.01 perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry 13.45 ± 5% +14.0 27.41 perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 13.07 ± 5% +14.0 27.04 perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary 6.33 ± 8% +15.3 21.61 ± 2% perf-profile.calltrace.cycles-pp.mwait_idle_with_hints.intel_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call 6.33 ± 8% +15.3 21.62 ± 2% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle 83.06 -24.3 58.74 perf-profile.children.cycles-pp.stress_run 45.26 -21.0 24.29 perf-profile.children.cycles-pp.do_msgsnd 46.20 -17.8 28.40 perf-profile.children.cycles-pp.__libc_msgsnd 83.01 -17.2 65.78 perf-profile.children.cycles-pp.do_syscall_64 83.14 -16.8 66.36 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 23.90 -16.1 7.80 perf-profile.children.cycles-pp._raw_spin_lock 18.40 -9.0 9.43 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 35.69 -8.6 27.10 perf-profile.children.cycles-pp.do_msgrcv 36.59 -7.1 29.47 perf-profile.children.cycles-pp.__libc_msgrcv 1.17 -1.1 0.07 ± 5% perf-profile.children.cycles-pp.wake_q_add 3.08 ± 7% -0.8 2.26 ± 5% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt 0.98 ± 4% -0.5 0.45 ± 9% perf-profile.children.cycles-pp.poll_idle 0.83 ± 2% -0.3 0.50 perf-profile.children.cycles-pp.schedule_idle 0.41 ± 3% -0.2 0.21 ± 3% perf-profile.children.cycles-pp.ttwu_queue_wakelist 0.24 ± 3% -0.1 0.10 ± 4% perf-profile.children.cycles-pp.__switch_to 0.31 ± 3% -0.1 0.19 ± 4% perf-profile.children.cycles-pp.asm_sysvec_call_function_single 0.63 -0.1 0.52 ± 2% perf-profile.children.cycles-pp.sched_ttwu_pending 0.20 ± 2% -0.1 0.08 ± 5% perf-profile.children.cycles-pp.switch_fpu_return 0.31 ± 3% -0.1 0.20 ± 3% perf-profile.children.cycles-pp.finish_task_switch 0.26 ± 2% -0.1 0.15 ± 4% perf-profile.children.cycles-pp.sysvec_call_function_single 0.23 ± 2% -0.1 0.13 ± 3% perf-profile.children.cycles-pp.__sysvec_call_function_single 0.66 -0.1 0.57 perf-profile.children.cycles-pp.flush_smp_call_function_queue 0.19 ± 3% -0.1 0.12 ± 4% perf-profile.children.cycles-pp.__flush_smp_call_function_queue 0.17 ± 4% -0.1 0.10 ± 6% perf-profile.children.cycles-pp.llist_add_batch 0.14 ± 4% -0.1 0.08 ± 6% perf-profile.children.cycles-pp.switch_mm_irqs_off 0.17 ± 4% -0.1 0.10 ± 4% perf-profile.children.cycles-pp.__smp_call_single_queue 0.17 ± 19% -0.1 0.12 ± 8% perf-profile.children.cycles-pp.task_tick_fair 0.16 ± 4% -0.0 0.11 ± 9% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length 0.16 ± 5% -0.0 0.12 ± 4% perf-profile.children.cycles-pp.prepare_task_switch 0.10 ± 4% -0.0 0.07 ± 14% perf-profile.children.cycles-pp.tick_nohz_next_event 0.07 ± 5% -0.0 0.06 ± 8% perf-profile.children.cycles-pp.llist_reverse_order 0.11 ± 3% -0.0 0.10 ± 3% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template 0.12 -0.0 0.11 ± 4% perf-profile.children.cycles-pp.__switch_to_asm 0.06 ± 9% +0.0 0.07 ± 6% perf-profile.children.cycles-pp.rebalance_domains 0.13 ± 2% +0.0 0.15 ± 3% perf-profile.children.cycles-pp.update_rq_clock 0.45 +0.0 0.47 perf-profile.children.cycles-pp.dequeue_task_fair 0.03 ± 70% +0.0 0.06 perf-profile.children.cycles-pp.detach_tasks 0.38 +0.0 0.40 perf-profile.children.cycles-pp.dequeue_entity 0.08 ± 4% +0.0 0.11 perf-profile.children.cycles-pp.update_blocked_averages 0.31 ± 4% +0.0 0.35 perf-profile.children.cycles-pp.select_task_rq 0.10 ± 8% +0.0 0.14 ± 3% perf-profile.children.cycles-pp.__softirqentry_text_start 0.05 ± 7% +0.1 0.10 ± 6% perf-profile.children.cycles-pp.select_idle_core 0.10 ± 10% +0.1 0.15 ± 5% perf-profile.children.cycles-pp.ktime_get 0.00 +0.1 0.05 ± 7% perf-profile.children.cycles-pp.restore_fpregs_from_fpstate 0.00 +0.1 0.05 ± 7% perf-profile.children.cycles-pp.task_h_load 0.00 +0.1 0.05 ± 7% perf-profile.children.cycles-pp.__update_blocked_fair 0.08 ± 12% +0.1 0.13 ± 6% perf-profile.children.cycles-pp.clockevents_program_event 0.12 ± 7% +0.1 0.18 ± 2% perf-profile.children.cycles-pp.__irq_exit_rcu 0.23 ± 3% +0.1 0.29 ± 3% perf-profile.children.cycles-pp.exit_to_user_mode_prepare 0.34 ± 2% +0.1 0.40 ± 3% perf-profile.children.cycles-pp.update_load_avg 0.00 +0.1 0.06 ± 9% perf-profile.children.cycles-pp.is_vmalloc_addr 0.10 ± 7% +0.1 0.16 ± 4% perf-profile.children.cycles-pp.select_idle_cpu 0.26 ± 3% +0.1 0.32 ± 2% perf-profile.children.cycles-pp.select_task_rq_fair 0.00 +0.1 0.06 perf-profile.children.cycles-pp.set_task_cpu 0.00 +0.1 0.06 ± 6% perf-profile.children.cycles-pp.security_msg_msg_alloc 0.12 ± 6% +0.1 0.19 perf-profile.children.cycles-pp.select_idle_sibling 0.00 +0.1 0.07 ± 11% perf-profile.children.cycles-pp.security_msg_queue_msgrcv 0.07 ± 8% +0.1 0.14 ± 5% perf-profile.children.cycles-pp.msgctl_stat 0.00 +0.1 0.07 perf-profile.children.cycles-pp.__x64_sys_msgrcv 0.00 +0.1 0.07 ± 5% perf-profile.children.cycles-pp.memcpy_erms 0.00 +0.1 0.07 ± 5% perf-profile.children.cycles-pp.kmalloc_slab 0.06 ± 6% +0.1 0.14 ± 5% perf-profile.children.cycles-pp.___slab_alloc 0.00 +0.1 0.08 ± 6% perf-profile.children.cycles-pp.security_ipc_permission 0.00 +0.1 0.08 perf-profile.children.cycles-pp.__cond_resched 0.00 +0.1 0.08 ± 5% perf-profile.children.cycles-pp.down_write 0.00 +0.1 0.09 ± 4% perf-profile.children.cycles-pp.syscall_exit_to_user_mode_prepare 0.00 +0.1 0.09 ± 4% perf-profile.children.cycles-pp.rwsem_mark_wake 0.01 ±223% +0.1 0.11 ± 6% perf-profile.children.cycles-pp.find_busiest_queue 0.00 +0.1 0.10 ± 6% perf-profile.children.cycles-pp.check_stack_object 0.00 +0.1 0.11 ± 7% perf-profile.children.cycles-pp.idr_get_next_ul 0.00 +0.1 0.11 ± 6% perf-profile.children.cycles-pp.idr_get_next 0.04 ± 44% +0.1 0.16 ± 3% perf-profile.children.cycles-pp.security_msg_msg_free 0.00 +0.1 0.12 ± 4% perf-profile.children.cycles-pp.up_read 0.60 ± 9% +0.1 0.73 ± 2% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt 0.00 +0.1 0.13 ± 3% perf-profile.children.cycles-pp.number 0.07 ± 5% +0.1 0.22 ± 6% perf-profile.children.cycles-pp._find_next_bit 0.00 +0.1 0.15 ± 2% perf-profile.children.cycles-pp.refill_obj_stock 0.03 ± 70% +0.2 0.18 ± 6% perf-profile.children.cycles-pp.obj_cgroup_charge 0.05 +0.2 0.20 ± 3% perf-profile.children.cycles-pp.syscall_enter_from_user_mode 0.04 ± 44% +0.2 0.20 ± 2% perf-profile.children.cycles-pp.sysvipc_proc_next 0.00 +0.2 0.16 ± 2% perf-profile.children.cycles-pp.down_read 0.00 +0.2 0.16 ± 3% perf-profile.children.cycles-pp.format_decode 0.08 ± 8% +0.2 0.30 ± 5% perf-profile.children.cycles-pp.idle_cpu 0.00 +0.2 0.22 ± 2% perf-profile.children.cycles-pp.rwsem_spin_on_owner 0.09 ± 8% +0.3 0.35 perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 0.06 ± 21% +0.3 0.32 ± 7% perf-profile.children.cycles-pp.__get_obj_cgroup_from_memcg 0.14 ± 5% +0.3 0.41 ± 2% perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack 0.09 ± 6% +0.3 0.38 ± 3% perf-profile.children.cycles-pp.__get_user_8 0.09 +0.3 0.38 ± 3% perf-profile.children.cycles-pp.__put_user_nocheck_8 0.11 ± 6% +0.3 0.40 ± 5% perf-profile.children.cycles-pp.syscall_return_via_sysret 0.14 ± 3% +0.3 0.44 perf-profile.children.cycles-pp.ipcperms 0.09 ± 9% +0.3 0.39 ± 2% perf-profile.children.cycles-pp.__list_add_valid 0.10 ± 4% +0.3 0.41 ± 2% perf-profile.children.cycles-pp.__might_sleep 0.27 +0.3 0.60 perf-profile.children.cycles-pp.syscall_exit_to_user_mode 0.13 ± 2% +0.4 0.49 ± 3% perf-profile.children.cycles-pp.__virt_addr_valid 0.10 ± 5% +0.4 0.46 ± 2% perf-profile.children.cycles-pp.vsnprintf 0.10 ± 4% +0.4 0.46 perf-profile.children.cycles-pp.seq_printf 0.10 ± 7% +0.4 0.48 perf-profile.children.cycles-pp.sysvipc_msg_proc_show 0.14 ± 3% +0.4 0.56 ± 2% perf-profile.children.cycles-pp.mod_objcg_state 0.10 ± 16% +0.4 0.55 ± 4% perf-profile.children.cycles-pp.get_obj_cgroup_from_current 0.23 ± 2% +0.5 0.70 perf-profile.children.cycles-pp.__might_resched 1.18 +0.6 1.79 ± 2% perf-profile.children.cycles-pp.try_to_wake_up 3.34 ± 2% +0.6 3.95 ± 2% perf-profile.children.cycles-pp.__schedule 0.29 ± 4% +0.6 0.92 perf-profile.children.cycles-pp.stress_msg 0.18 ± 2% +0.6 0.82 perf-profile.children.cycles-pp.copy_user_short_string 0.18 ± 5% +0.6 0.82 perf-profile.children.cycles-pp.rwsem_wake 0.21 ± 2% +0.7 0.92 perf-profile.children.cycles-pp.entry_SYSRETQ_unsafe_stack 0.09 ± 10% +0.7 0.82 perf-profile.children.cycles-pp.ss_wakeup 0.21 ± 3% +0.7 0.94 perf-profile.children.cycles-pp._copy_to_user 0.23 ± 2% +0.7 0.96 ± 2% perf-profile.children.cycles-pp.__check_heap_object 0.21 ± 2% +0.7 0.95 perf-profile.children.cycles-pp.__x64_sys_msgsnd 0.22 ± 5% +0.8 0.99 ± 2% perf-profile.children.cycles-pp.__radix_tree_lookup 0.29 ± 2% +0.8 1.10 perf-profile.children.cycles-pp.__might_fault 1.31 +0.9 2.22 ± 2% perf-profile.children.cycles-pp.wake_up_q 2.54 +0.9 3.47 ± 2% perf-profile.children.cycles-pp.schedule 0.38 ± 2% +0.9 1.31 perf-profile.children.cycles-pp._copy_from_user 0.20 ± 3% +0.9 1.14 perf-profile.children.cycles-pp.__list_del_entry_valid 0.00 +0.9 0.94 ± 3% perf-profile.children.cycles-pp.osq_lock 0.10 ± 4% +1.0 1.14 ± 2% perf-profile.children.cycles-pp.sysvipc_proc_start 0.80 ± 5% +1.1 1.86 ± 4% perf-profile.children.cycles-pp.update_sg_lb_stats 0.87 ± 4% +1.1 1.99 ± 4% perf-profile.children.cycles-pp.update_sd_lb_stats 0.34 ± 2% +1.1 1.46 perf-profile.children.cycles-pp.__entry_text_start 0.88 ± 4% +1.1 2.01 ± 4% perf-profile.children.cycles-pp.find_busiest_group 0.37 ± 2% +1.2 1.56 perf-profile.children.cycles-pp.check_heap_object 0.05 ± 7% +1.2 1.29 ± 3% perf-profile.children.cycles-pp.rwsem_optimistic_spin 1.05 ± 5% +1.3 2.31 ± 4% perf-profile.children.cycles-pp.load_balance 1.34 ± 4% +1.3 2.63 ± 4% perf-profile.children.cycles-pp.pick_next_task_fair 1.20 ± 4% +1.3 2.50 ± 4% perf-profile.children.cycles-pp.newidle_balance 1.09 ± 2% +1.4 2.50 perf-profile.children.cycles-pp.__slab_free 0.27 ± 4% +1.6 1.88 ± 2% perf-profile.children.cycles-pp.seq_read_iter 0.27 ± 3% +1.6 1.88 ± 2% perf-profile.children.cycles-pp.seq_read 0.28 ± 4% +1.6 1.90 ± 2% perf-profile.children.cycles-pp.vfs_read 0.28 ± 4% +1.6 1.90 ± 2% perf-profile.children.cycles-pp.ksys_read 0.28 ± 4% +1.6 1.92 ± 2% perf-profile.children.cycles-pp.read 0.63 +1.8 2.42 perf-profile.children.cycles-pp.kfree 0.51 ± 2% +2.0 2.50 perf-profile.children.cycles-pp.memcg_slab_post_alloc_hook 0.70 +2.2 2.94 perf-profile.children.cycles-pp.__check_object_size 0.27 ± 3% +2.4 2.69 perf-profile.children.cycles-pp.rwsem_down_write_slowpath 0.07 ± 8% +2.4 2.51 perf-profile.children.cycles-pp._raw_spin_lock_irq 0.83 +2.8 3.59 perf-profile.children.cycles-pp.store_msg 0.31 ± 3% +2.9 3.20 perf-profile.children.cycles-pp.rwsem_down_read_slowpath 0.26 +3.0 3.28 perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.53 ± 2% +3.1 3.65 perf-profile.children.cycles-pp.msgctl_down 1.78 +3.3 5.07 perf-profile.children.cycles-pp.free_msg 0.97 +3.4 4.34 perf-profile.children.cycles-pp.__kmalloc 0.00 +3.4 3.38 perf-profile.children.cycles-pp.percpu_counter_add_batch 0.99 +3.4 4.41 perf-profile.children.cycles-pp.alloc_msg 1.10 +3.5 4.61 perf-profile.children.cycles-pp.do_msg_fill 0.00 +3.6 3.56 perf-profile.children.cycles-pp.__percpu_counter_sum 1.68 +5.3 7.01 perf-profile.children.cycles-pp.load_msg 0.43 ± 2% +5.8 6.20 perf-profile.children.cycles-pp.msgctl_info 0.60 ± 2% +7.4 7.97 perf-profile.children.cycles-pp.idr_find 1.04 ± 2% +9.0 10.05 perf-profile.children.cycles-pp.ksys_msgctl 1.10 ± 2% +9.1 10.23 perf-profile.children.cycles-pp.msgctl 1.63 +11.1 12.72 perf-profile.children.cycles-pp.ipc_obtain_object_check 15.18 ± 4% +13.4 28.63 perf-profile.children.cycles-pp.start_secondary 15.36 ± 4% +13.5 28.91 perf-profile.children.cycles-pp.do_idle 15.37 ± 4% +13.5 28.92 perf-profile.children.cycles-pp.secondary_startup_64_no_verify 15.37 ± 4% +13.5 28.92 perf-profile.children.cycles-pp.cpu_startup_entry 13.61 ± 4% +14.1 27.69 perf-profile.children.cycles-pp.cpuidle_idle_call 13.23 ± 5% +14.1 27.31 perf-profile.children.cycles-pp.cpuidle_enter 13.23 ± 5% +14.1 27.31 perf-profile.children.cycles-pp.cpuidle_enter_state 12.02 ± 5% +14.5 26.52 perf-profile.children.cycles-pp.mwait_idle_with_hints 6.40 ± 8% +15.4 21.83 ± 2% perf-profile.children.cycles-pp.intel_idle 29.09 -26.2 2.86 perf-profile.self.cycles-pp.do_msgsnd 17.19 -15.2 1.98 perf-profile.self.cycles-pp.do_msgrcv 18.27 -8.9 9.35 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 5.53 -1.9 3.62 perf-profile.self.cycles-pp._raw_spin_lock 1.17 -1.1 0.07 ± 5% perf-profile.self.cycles-pp.wake_q_add 0.94 ± 3% -0.5 0.43 ± 8% perf-profile.self.cycles-pp.poll_idle 0.41 -0.2 0.18 ± 2% perf-profile.self.cycles-pp.__schedule 0.24 ± 3% -0.1 0.10 ± 3% perf-profile.self.cycles-pp.__switch_to 0.17 ± 4% -0.1 0.10 ± 6% perf-profile.self.cycles-pp.llist_add_batch 0.14 ± 4% -0.1 0.08 ± 6% perf-profile.self.cycles-pp.switch_mm_irqs_off 0.12 ± 4% -0.1 0.06 ± 7% perf-profile.self.cycles-pp.__flush_smp_call_function_queue 0.07 ± 6% -0.0 0.03 ± 70% perf-profile.self.cycles-pp.sched_ttwu_pending 0.09 ± 4% -0.0 0.05 ± 8% perf-profile.self.cycles-pp.ttwu_queue_wakelist 0.11 ± 6% -0.0 0.08 ± 6% perf-profile.self.cycles-pp.finish_task_switch 0.14 ± 4% -0.0 0.12 ± 3% perf-profile.self.cycles-pp.msgctl_info 0.11 ± 6% -0.0 0.09 ± 5% perf-profile.self.cycles-pp.prepare_task_switch 0.12 -0.0 0.11 ± 4% perf-profile.self.cycles-pp.__switch_to_asm 0.08 ± 4% -0.0 0.07 ± 5% perf-profile.self.cycles-pp.enqueue_entity 0.09 ± 7% +0.0 0.11 ± 6% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq 0.17 ± 4% +0.0 0.20 ± 4% perf-profile.self.cycles-pp.update_load_avg 0.08 +0.0 0.11 ± 5% perf-profile.self.cycles-pp.update_rq_clock 0.04 ± 44% +0.0 0.09 ± 4% perf-profile.self.cycles-pp.update_sd_lb_stats 0.07 ± 15% +0.1 0.12 ± 6% perf-profile.self.cycles-pp.ktime_get 0.00 +0.1 0.05 ± 7% perf-profile.self.cycles-pp.restore_fpregs_from_fpstate 0.00 +0.1 0.05 ± 7% perf-profile.self.cycles-pp.security_msg_msg_alloc 0.00 +0.1 0.05 ± 7% perf-profile.self.cycles-pp.task_h_load 0.00 +0.1 0.06 ± 9% perf-profile.self.cycles-pp.load_balance 0.00 +0.1 0.06 ± 8% perf-profile.self.cycles-pp.rwsem_mark_wake 0.11 ± 6% +0.1 0.17 ± 3% perf-profile.self.cycles-pp.menu_select 0.00 +0.1 0.06 perf-profile.self.cycles-pp.__x64_sys_msgrcv 0.00 +0.1 0.06 ± 7% perf-profile.self.cycles-pp.security_msg_queue_msgrcv 0.00 +0.1 0.06 ± 7% perf-profile.self.cycles-pp.kmalloc_slab 0.00 +0.1 0.07 ± 8% perf-profile.self.cycles-pp.memcpy_erms 0.00 +0.1 0.07 perf-profile.self.cycles-pp.free_msg 0.00 +0.1 0.07 ± 5% perf-profile.self.cycles-pp.security_ipc_permission 0.00 +0.1 0.07 ± 6% perf-profile.self.cycles-pp.syscall_exit_to_user_mode_prepare 0.00 +0.1 0.08 perf-profile.self.cycles-pp.security_msg_msg_free 0.00 +0.1 0.08 perf-profile.self.cycles-pp.down_write 0.00 +0.1 0.08 ± 7% perf-profile.self.cycles-pp.alloc_msg 0.04 ± 44% +0.1 0.13 ± 6% perf-profile.self.cycles-pp.___slab_alloc 0.00 +0.1 0.09 ± 7% perf-profile.self.cycles-pp.check_stack_object 0.00 +0.1 0.10 ± 7% perf-profile.self.cycles-pp.idr_get_next_ul 0.00 +0.1 0.10 ± 4% perf-profile.self.cycles-pp.vsnprintf 0.00 +0.1 0.10 ± 4% perf-profile.self.cycles-pp.find_busiest_queue 0.00 +0.1 0.10 ± 4% perf-profile.self.cycles-pp.rwsem_optimistic_spin 0.24 ± 2% +0.1 0.36 perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.00 +0.1 0.12 ± 3% perf-profile.self.cycles-pp._copy_to_user 0.00 +0.1 0.12 ± 4% perf-profile.self.cycles-pp.number 0.00 +0.1 0.12 ± 3% perf-profile.self.cycles-pp.format_decode 0.00 +0.1 0.12 ± 3% perf-profile.self.cycles-pp.up_read 0.06 ± 7% +0.1 0.20 ± 5% perf-profile.self.cycles-pp._find_next_bit 0.00 +0.1 0.14 ± 2% perf-profile.self.cycles-pp.rwsem_down_write_slowpath 0.00 +0.1 0.14 ± 6% perf-profile.self.cycles-pp.obj_cgroup_charge 0.00 +0.1 0.15 ± 3% perf-profile.self.cycles-pp.exit_to_user_mode_prepare 0.00 +0.1 0.15 ± 3% perf-profile.self.cycles-pp.refill_obj_stock 0.00 +0.2 0.16 ± 4% perf-profile.self.cycles-pp.syscall_enter_from_user_mode 0.00 +0.2 0.16 ± 2% perf-profile.self.cycles-pp.down_read 0.03 ± 70% +0.2 0.21 ± 3% perf-profile.self.cycles-pp.load_msg 0.04 ± 71% +0.2 0.22 ± 3% perf-profile.self.cycles-pp.__might_fault 0.01 ±223% +0.2 0.21 ± 2% perf-profile.self.cycles-pp._raw_spin_lock_irq 0.00 +0.2 0.21 ± 8% perf-profile.self.cycles-pp.get_obj_cgroup_from_current 0.00 +0.2 0.21 ± 2% perf-profile.self.cycles-pp.syscall_exit_to_user_mode 0.00 +0.2 0.21 ± 2% perf-profile.self.cycles-pp.rwsem_spin_on_owner 0.08 ± 8% +0.2 0.30 ± 4% perf-profile.self.cycles-pp.idle_cpu 0.07 +0.2 0.29 ± 2% perf-profile.self.cycles-pp.__check_object_size 0.08 ± 5% +0.2 0.33 perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 0.06 ± 6% +0.2 0.31 ± 2% perf-profile.self.cycles-pp.__x64_sys_msgsnd 0.06 ± 21% +0.3 0.32 ± 8% perf-profile.self.cycles-pp.__get_obj_cgroup_from_memcg 0.14 ± 5% +0.3 0.40 ± 2% perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack 0.09 ± 5% +0.3 0.36 ± 2% perf-profile.self.cycles-pp.__might_sleep 0.11 ± 6% +0.3 0.38 ± 5% perf-profile.self.cycles-pp.syscall_return_via_sysret 0.08 ± 5% +0.3 0.37 ± 3% perf-profile.self.cycles-pp.__get_user_8 0.14 ± 3% +0.3 0.42 perf-profile.self.cycles-pp.ipcperms 0.09 +0.3 0.38 ± 3% perf-profile.self.cycles-pp.__put_user_nocheck_8 0.13 ± 3% +0.3 0.42 perf-profile.self.cycles-pp.wake_up_q 0.08 ± 5% +0.3 0.38 ± 3% perf-profile.self.cycles-pp.__list_add_valid 0.00 +0.3 0.30 ± 3% perf-profile.self.cycles-pp.rwsem_down_read_slowpath 0.12 ± 3% +0.3 0.46 ± 4% perf-profile.self.cycles-pp.__virt_addr_valid 0.06 ± 6% +0.3 0.41 ± 3% perf-profile.self.cycles-pp.do_msg_fill 0.19 ± 3% +0.4 0.55 perf-profile.self.cycles-pp._copy_from_user 0.13 ± 3% +0.4 0.54 perf-profile.self.cycles-pp.mod_objcg_state 0.14 ± 2% +0.4 0.58 perf-profile.self.cycles-pp.__libc_msgsnd 0.23 ± 2% +0.5 0.69 perf-profile.self.cycles-pp.__might_resched 0.12 ± 3% +0.5 0.59 perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 0.14 ± 5% +0.5 0.61 perf-profile.self.cycles-pp.__entry_text_start 0.15 ± 4% +0.5 0.63 ± 2% perf-profile.self.cycles-pp.__libc_msgrcv 0.14 ± 2% +0.5 0.64 ± 2% perf-profile.self.cycles-pp.__kmalloc 0.17 ± 4% +0.5 0.68 perf-profile.self.cycles-pp.do_syscall_64 0.20 ± 2% +0.5 0.71 ± 2% perf-profile.self.cycles-pp.store_msg 0.28 ± 4% +0.6 0.87 perf-profile.self.cycles-pp.stress_msg 0.17 ± 3% +0.6 0.79 perf-profile.self.cycles-pp.copy_user_short_string 0.20 +0.7 0.89 perf-profile.self.cycles-pp.entry_SYSRETQ_unsafe_stack 0.08 ± 8% +0.7 0.80 perf-profile.self.cycles-pp.ss_wakeup 0.13 ± 5% +0.7 0.85 ± 6% perf-profile.self.cycles-pp.try_to_wake_up 0.23 ± 2% +0.7 0.96 ± 2% perf-profile.self.cycles-pp.__check_heap_object 0.00 +0.7 0.74 perf-profile.self.cycles-pp.__percpu_counter_sum 0.21 ± 6% +0.8 0.96 perf-profile.self.cycles-pp.__radix_tree_lookup 0.67 ± 4% +0.8 1.44 ± 4% perf-profile.self.cycles-pp.update_sg_lb_stats 0.22 ± 4% +0.8 1.04 ± 2% perf-profile.self.cycles-pp.check_heap_object 0.00 +0.9 0.92 ± 3% perf-profile.self.cycles-pp.osq_lock 0.20 ± 4% +0.9 1.12 perf-profile.self.cycles-pp.__list_del_entry_valid 0.49 ± 2% +1.4 1.85 perf-profile.self.cycles-pp.kfree 1.08 ± 2% +1.4 2.49 perf-profile.self.cycles-pp.__slab_free 0.46 ± 3% +1.8 2.28 perf-profile.self.cycles-pp.memcg_slab_post_alloc_hook 0.81 +2.9 3.76 perf-profile.self.cycles-pp.ipc_obtain_object_check 0.00 +3.3 3.34 perf-profile.self.cycles-pp.percpu_counter_add_batch 0.59 +7.3 7.92 perf-profile.self.cycles-pp.idr_find 11.93 ± 5% +14.5 26.39 perf-profile.self.cycles-pp.mwait_idle_with_hints To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests sudo bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run sudo bin/lkp run generated-yaml-file # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. -- 0-DAY CI Kernel Test Service https://01.org/lkp