Greeting, FYI, we noticed a -61.6% regression of netperf.Throughput_total_tps due to commit: commit: 38f27d0787552bbd012ea533351435215e3cdfa6 ("vfs: Implement a filesystem superblock creation/configuration context") https://git.kernel.org/cgit/linux/kernel/git/dhowells/linux-fs.git mount-api-uapi in testcase: netperf on test machine: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory with following parameters: ip: ipv4 runtime: 300s nr_threads: 50% cluster: cs-localhost test: TCP_CRR cpufreq_governor: performance test-description: Netperf is a benchmark that can be use to measure various aspect of networking performance. test-url: http://www.netperf.org/netperf/ Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= cluster/compiler/cpufreq_governor/ip/kconfig/nr_threads/rootfs/runtime/tbox_group/test/testcase: cs-localhost/gcc-7/performance/ipv4/x86_64-rhel-7.2/50%/debian-x86_64-2018-04-03.cgz/300s/lkp-bdw-ep2/TCP_CRR/netperf commit: 936d8e7dfd ("vfs: Separate changing mount flags full remount") 38f27d0787 ("vfs: Implement a filesystem superblock creation/configuration context") 936d8e7dfd044dfa 38f27d0787552bbd012ea53335 ---------------- -------------------------- fail:runs %reproduction fail:runs | | | 1:4 -25% :4 dmesg.WARNING:at_ip_tcp_fastopen_defer_connect/0x 16:4 -262% 5:4 perf-profile.children.cycles-pp.schedule_timeout %stddev %change %stddev \ | \ 713415 ± 3% -61.6% 274105 netperf.Throughput_total_tps 16789 -62.9% 6229 netperf.Throughput_tps 1976574 ± 15% -28.5% 1412346 netperf.time.involuntary_context_switches 3717 ± 2% +11.3% 4136 netperf.time.percent_of_cpu_this_job_got 10750 ± 2% +15.5% 12419 netperf.time.system_time 590.70 ± 2% -65.8% 202.31 netperf.time.user_time 4.306e+08 -62.2% 1.63e+08 netperf.time.voluntary_context_switches 2.14e+08 ± 3% -61.6% 82231508 netperf.workload 576581 ± 10% -32.2% 390687 ± 5% numa-numastat.node0.local_node 583764 ± 10% -31.4% 400700 ± 6% numa-numastat.node0.numa_hit 483.67 -7.6% 446.76 pmeter.Average_Active_Power 34.71 -59.8% 13.95 pmeter.performance_per_watt 111199 ± 20% +34.6% 149662 ± 15% numa-meminfo.node0.AnonPages 111075 ± 19% -32.5% 75006 ± 31% numa-meminfo.node1.AnonHugePages 155532 ± 15% -25.5% 115837 ± 18% numa-meminfo.node1.AnonPages 6.623e+08 -62.5% 2.485e+08 softirqs.NET_RX 51641103 ± 13% -44.1% 28860122 softirqs.RCU 7781693 +38.8% 10802802 softirqs.SCHED 54.50 ± 2% +34.9% 73.50 ± 2% vmstat.procs.r 5037932 ± 2% -59.2% 2057894 vmstat.system.cs 196920 ± 2% +2.9% 202632 vmstat.system.in 38.31 ± 4% -23.2 15.16 ± 10% mpstat.cpu.idle% 21.82 -14.4 7.44 ± 2% mpstat.cpu.soft% 36.73 ± 3% +39.5 76.24 mpstat.cpu.sys% 3.13 ± 3% -2.0 1.17 ± 2% mpstat.cpu.usr% 7.018e+09 -63.7% 2.549e+09 cpuidle.C1.time 7.293e+08 -60.7% 2.869e+08 cpuidle.C1.usage 6.197e+08 ± 8% -81.0% 1.179e+08 ± 9% cpuidle.C1E.time 25585732 ± 4% -79.7% 5204543 ± 9% cpuidle.C1E.usage 1.241e+08 ± 3% -64.9% 43618992 ± 2% cpuidle.POLL.time 18052173 ± 10% -60.8% 7070699 ± 15% cpuidle.POLL.usage 15560 ± 13% +35.6% 21100 ± 19% numa-vmstat.node0 27807 ± 20% +34.6% 37421 ± 15% numa-vmstat.node0.nr_anon_pages 18107 ± 11% -30.6% 12568 ± 32% numa-vmstat.node1 38890 ± 15% -25.5% 28965 ± 18% numa-vmstat.node1.nr_anon_pages 677616 ± 7% -18.0% 555700 ± 7% numa-vmstat.node1.numa_hit 510099 ± 10% -23.4% 390637 ± 9% numa-vmstat.node1.numa_local 4899 +2.0% 4999 proc-vmstat.nr_inactive_anon 53982 -1.3% 53274 proc-vmstat.nr_slab_unreclaimable 226.75 ±133% -78.4% 49.00 proc-vmstat.nr_written 4899 +2.0% 4999 proc-vmstat.nr_zone_inactive_anon 1165190 ± 4% -30.2% 813783 proc-vmstat.numa_hit 1148022 ± 4% -30.6% 796626 proc-vmstat.numa_local 1415980 ± 5% -35.6% 911335 proc-vmstat.pgalloc_normal 1348585 ± 5% -37.3% 846011 proc-vmstat.pgfree 1881 ± 2% +29.0% 2426 turbostat.Avg_MHz 67.58 ± 2% +19.4 86.95 turbostat.Busy% 7.293e+08 -60.7% 2.869e+08 turbostat.C1 25.46 ± 2% -16.2 9.29 ± 2% turbostat.C1% 25585350 ± 4% -79.7% 5204256 ± 9% turbostat.C1E 2.25 ± 8% -1.8 0.43 ± 9% turbostat.C1E% 29.90 ± 4% -64.5% 10.62 ± 2% turbostat.CPU%c1 263.47 -9.6% 238.22 turbostat.PkgWatt 3822 ± 10% -23.2% 2936 ± 10% slabinfo.eventpoll_pwq.active_objs 3822 ± 10% -23.2% 2936 ± 10% slabinfo.eventpoll_pwq.num_objs 36117 ± 3% -29.0% 25656 slabinfo.filp.active_objs 1130 ± 3% -28.8% 804.75 slabinfo.filp.active_slabs 36198 ± 3% -28.8% 25767 slabinfo.filp.num_objs 1130 ± 3% -28.8% 804.75 slabinfo.filp.num_slabs 123324 -9.0% 112286 slabinfo.kmalloc-64.active_objs 1930 -9.0% 1756 slabinfo.kmalloc-64.active_slabs 123575 -9.0% 112467 slabinfo.kmalloc-64.num_objs 1930 -9.0% 1756 slabinfo.kmalloc-64.num_slabs 5.415e+12 -10.5% 4.849e+12 perf-stat.branch-instructions 2.60 -1.4 1.16 perf-stat.branch-miss-rate% 1.409e+11 -60.2% 5.611e+10 perf-stat.branch-misses 1.15 +0.7 1.83 perf-stat.cache-miss-rate% 8.556e+09 -34.3% 5.622e+09 perf-stat.cache-misses 7.472e+11 -58.9% 3.069e+11 perf-stat.cache-references 1.585e+09 ± 2% -59.3% 6.449e+08 perf-stat.context-switches 1.92 +60.0% 3.07 perf-stat.cpi 5.241e+13 ± 2% +26.8% 6.646e+13 perf-stat.cpu-cycles 1916531 ± 28% -70.9% 558026 ± 5% perf-stat.cpu-migrations 0.12 ± 2% -0.1 0.06 perf-stat.dTLB-load-miss-rate% 9.874e+09 ± 3% -61.5% 3.797e+09 perf-stat.dTLB-load-misses 8.274e+12 -27.9% 5.965e+12 perf-stat.dTLB-loads 0.02 ± 3% -0.0 0.02 ± 2% perf-stat.dTLB-store-miss-rate% 9.868e+08 ± 3% -67.9% 3.166e+08 ± 2% perf-stat.dTLB-store-misses 5.282e+12 -60.7% 2.077e+12 perf-stat.dTLB-stores 8.869e+09 ± 3% -59.8% 3.569e+09 ± 12% perf-stat.iTLB-load-misses 3.395e+10 ± 2% -62.9% 1.26e+10 perf-stat.iTLB-loads 2.732e+13 -20.8% 2.165e+13 perf-stat.instructions 3082 ± 3% +99.7% 6156 ± 11% perf-stat.instructions-per-iTLB-miss 0.52 -37.5% 0.33 perf-stat.ipc 90.17 +5.5 95.67 perf-stat.node-load-miss-rate% 3.261e+09 -18.2% 2.667e+09 perf-stat.node-load-misses 3.554e+08 ± 3% -66.0% 1.207e+08 ± 3% perf-stat.node-loads 82.19 -23.3 58.86 perf-stat.node-store-miss-rate% 2.71e+09 -55.1% 1.216e+09 perf-stat.node-store-misses 5.875e+08 ± 2% +44.6% 8.497e+08 perf-stat.node-stores 127712 +106.1% 263223 perf-stat.path-length 3402504 ± 3% +43.0% 4863968 ± 15% sched_debug.cfs_rq:/.MIN_vruntime.max 2.71 ± 56% +189.2% 7.83 ± 27% sched_debug.cfs_rq:/.load_avg.min 3402504 ± 3% +43.0% 4863968 ± 15% sched_debug.cfs_rq:/.max_vruntime.max 3323827 ± 3% +58.7% 5275872 sched_debug.cfs_rq:/.min_vruntime.avg 3473943 ± 2% +55.1% 5388654 sched_debug.cfs_rq:/.min_vruntime.max 3178116 ± 4% +62.6% 5167570 sched_debug.cfs_rq:/.min_vruntime.min 61982 ± 15% -35.2% 40166 ± 6% sched_debug.cfs_rq:/.min_vruntime.stddev 0.59 +29.7% 0.77 sched_debug.cfs_rq:/.nr_running.avg 0.44 -27.3% 0.32 ± 2% sched_debug.cfs_rq:/.nr_running.stddev 11.21 ± 6% +38.5% 15.52 sched_debug.cfs_rq:/.runnable_load_avg.avg 9.25 ± 6% -24.8% 6.96 ± 2% sched_debug.cfs_rq:/.runnable_load_avg.stddev 222942 ± 33% -45.6% 121355 ± 37% sched_debug.cfs_rq:/.spread0.max 61989 ± 15% -35.2% 40158 ± 6% sched_debug.cfs_rq:/.spread0.stddev 742.06 ± 7% +18.8% 881.73 sched_debug.cfs_rq:/.util_avg.avg 1566 ± 3% +12.7% 1764 ± 2% sched_debug.cfs_rq:/.util_avg.max 322.17 ± 7% -15.2% 273.25 ± 4% sched_debug.cfs_rq:/.util_avg.stddev 427.33 ± 3% +56.8% 670.16 sched_debug.cfs_rq:/.util_est_enqueued.avg 987.38 ± 10% +25.4% 1238 ± 11% sched_debug.cfs_rq:/.util_est_enqueued.max 341.79 -16.9% 284.18 sched_debug.cfs_rq:/.util_est_enqueued.stddev 11.30 ± 6% +33.5% 15.09 sched_debug.cpu.cpu_load[0].avg 8.43 ± 4% -19.0% 6.83 ± 5% sched_debug.cpu.cpu_load[0].stddev 11.34 ± 5% +34.8% 15.28 sched_debug.cpu.cpu_load[1].avg 0.08 ± 99% +2400.0% 2.08 ± 37% sched_debug.cpu.cpu_load[1].min 6.06 ± 4% -23.5% 4.64 ± 4% sched_debug.cpu.cpu_load[1].stddev 11.61 ± 4% +34.5% 15.61 sched_debug.cpu.cpu_load[2].avg 0.88 ± 20% +476.2% 5.04 ± 18% sched_debug.cpu.cpu_load[2].min 5.21 ± 9% -31.2% 3.59 ± 4% sched_debug.cpu.cpu_load[2].stddev 12.09 ± 5% +33.3% 16.12 sched_debug.cpu.cpu_load[3].avg 2.08 ± 25% +268.0% 7.67 ± 11% sched_debug.cpu.cpu_load[3].min 5.67 ± 29% -45.4% 3.10 ± 5% sched_debug.cpu.cpu_load[3].stddev 12.85 ± 4% +29.7% 16.66 sched_debug.cpu.cpu_load[4].avg 1.29 ± 36% +716.1% 10.54 ± 7% sched_debug.cpu.cpu_load[4].min 8.46 ± 33% -51.6% 4.09 ± 28% sched_debug.cpu.cpu_load[4].stddev 1130 +33.7% 1511 sched_debug.cpu.curr->pid.avg 999.01 -25.8% 741.70 ± 2% sched_debug.cpu.curr->pid.stddev 0.00 ± 6% +45.4% 0.00 ± 2% sched_debug.cpu.next_balance.stddev 0.59 +29.5% 0.77 sched_debug.cpu.nr_running.avg 0.45 -27.5% 0.32 ± 5% sched_debug.cpu.nr_running.stddev 8960269 -59.6% 3623716 sched_debug.cpu.nr_switches.avg 9249479 -60.2% 3678097 sched_debug.cpu.nr_switches.max 8415736 ± 2% -58.1% 3528976 sched_debug.cpu.nr_switches.min 161945 ± 8% -81.2% 30435 ± 5% sched_debug.cpu.nr_switches.stddev -6.96 +40.1% -9.75 sched_debug.cpu.nr_uninterruptible.min 31.20 ± 2% -19.5 11.68 perf-profile.calltrace.cycles-pp.secondary_startup_64 30.82 ± 2% -19.3 11.52 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64 30.81 ± 2% -19.3 11.52 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64 30.74 ± 2% -19.2 11.50 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 24.54 -16.8 7.73 perf-profile.calltrace.cycles-pp.__local_bh_enable_ip.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb 24.47 -16.8 7.71 perf-profile.calltrace.cycles-pp.do_softirq.__local_bh_enable_ip.ip_finish_output2.ip_output.__ip_queue_xmit 24.22 -16.6 7.64 perf-profile.calltrace.cycles-pp.do_softirq_own_stack.do_softirq.__local_bh_enable_ip.ip_finish_output2.ip_output 24.14 -16.5 7.62 perf-profile.calltrace.cycles-pp.__softirqentry_text_start.do_softirq_own_stack.do_softirq.__local_bh_enable_ip.ip_finish_output2 23.62 -16.2 7.45 perf-profile.calltrace.cycles-pp.net_rx_action.__softirqentry_text_start.do_softirq_own_stack.do_softirq.__local_bh_enable_ip 23.19 -15.9 7.30 perf-profile.calltrace.cycles-pp.process_backlog.net_rx_action.__softirqentry_text_start.do_softirq_own_stack.do_softirq 22.64 -15.5 7.12 perf-profile.calltrace.cycles-pp.__netif_receive_skb_one_core.process_backlog.net_rx_action.__softirqentry_text_start.do_softirq_own_stack 25.32 ± 3% -15.5 9.83 perf-profile.calltrace.cycles-pp.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 22.16 -15.2 6.93 perf-profile.calltrace.cycles-pp.ip_rcv.__netif_receive_skb_one_core.process_backlog.net_rx_action.__softirqentry_text_start 24.41 ± 3% -14.9 9.53 perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary 21.51 -14.8 6.74 perf-profile.calltrace.cycles-pp.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core.process_backlog.net_rx_action 21.30 -14.6 6.66 perf-profile.calltrace.cycles-pp.ip_local_deliver_finish.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core.process_backlog 21.04 -14.5 6.58 perf-profile.calltrace.cycles-pp.tcp_v4_rcv.ip_local_deliver_finish.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core 17.01 -11.9 5.06 perf-profile.calltrace.cycles-pp.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames 16.66 -11.7 4.95 perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit 15.16 -10.8 4.40 perf-profile.calltrace.cycles-pp.sock_close.__fput.task_work_run.exit_to_usermode_loop.do_syscall_64 15.13 -10.7 4.38 perf-profile.calltrace.cycles-pp.__sock_release.sock_close.__fput.task_work_run.exit_to_usermode_loop 14.97 -10.6 4.33 perf-profile.calltrace.cycles-pp.inet_release.__sock_release.sock_close.__fput.task_work_run 14.59 -10.4 4.21 perf-profile.calltrace.cycles-pp.tcp_close.inet_release.__sock_release.sock_close.__fput 14.86 -9.7 5.17 perf-profile.calltrace.cycles-pp.__x64_sys_connect.do_syscall_64.entry_SYSCALL_64_after_hwframe 14.82 -9.7 5.16 perf-profile.calltrace.cycles-pp.__sys_connect.__x64_sys_connect.do_syscall_64.entry_SYSCALL_64_after_hwframe 13.84 -9.4 4.41 ± 2% perf-profile.calltrace.cycles-pp.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe 13.79 -9.4 4.40 ± 2% perf-profile.calltrace.cycles-pp.__sys_sendto.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe 14.38 -9.4 5.00 perf-profile.calltrace.cycles-pp.inet_stream_connect.__sys_connect.__x64_sys_connect.do_syscall_64.entry_SYSCALL_64_after_hwframe 14.27 -9.3 4.97 perf-profile.calltrace.cycles-pp.__inet_stream_connect.inet_stream_connect.__sys_connect.__x64_sys_connect.do_syscall_64 13.60 -9.3 4.33 ± 2% perf-profile.calltrace.cycles-pp.sock_sendmsg.__sys_sendto.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe 13.40 -9.1 4.27 ± 2% perf-profile.calltrace.cycles-pp.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto.do_syscall_64 12.65 -8.8 3.87 perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.tcp_v4_rcv.ip_local_deliver_finish.ip_local_deliver.ip_rcv 11.85 -8.1 3.72 ± 2% perf-profile.calltrace.cycles-pp.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto 10.78 -7.5 3.25 perf-profile.calltrace.cycles-pp.__tcp_push_pending_frames.tcp_close.inet_release.__sock_release.sock_close 10.75 -7.5 3.25 perf-profile.calltrace.cycles-pp.tcp_write_xmit.__tcp_push_pending_frames.tcp_close.inet_release.__sock_release 10.54 -7.2 3.29 perf-profile.calltrace.cycles-pp.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto 10.50 -7.2 3.28 perf-profile.calltrace.cycles-pp.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg 9.57 -6.7 2.85 perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_close.inet_release 9.17 -6.5 2.71 perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_close 8.32 -5.8 2.52 perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg 8.07 -5.6 2.43 perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked 6.75 -4.8 1.94 ± 2% perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_local_deliver_finish.ip_local_deliver 6.83 -4.7 2.09 perf-profile.calltrace.cycles-pp.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe 6.76 -4.7 2.07 perf-profile.calltrace.cycles-pp.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe 7.02 -4.6 2.40 perf-profile.calltrace.cycles-pp.release_sock.__inet_stream_connect.inet_stream_connect.__sys_connect.__x64_sys_connect 6.98 -4.6 2.38 perf-profile.calltrace.cycles-pp.__release_sock.release_sock.__inet_stream_connect.inet_stream_connect.__sys_connect 6.88 -4.5 2.34 perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.__release_sock.release_sock.__inet_stream_connect.inet_stream_connect 6.98 -4.5 2.49 perf-profile.calltrace.cycles-pp.tcp_v4_connect.__inet_stream_connect.inet_stream_connect.__sys_connect.__x64_sys_connect 6.44 -4.5 1.95 perf-profile.calltrace.cycles-pp.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe 6.35 -4.4 1.91 perf-profile.calltrace.cycles-pp.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64 6.22 -4.1 2.12 perf-profile.calltrace.cycles-pp.tcp_rcv_state_process.tcp_v4_do_rcv.__release_sock.release_sock.__inet_stream_connect 5.82 -3.9 1.88 perf-profile.calltrace.cycles-pp.tcp_rcv_state_process.tcp_v4_do_rcv.tcp_v4_rcv.ip_local_deliver_finish.ip_local_deliver 5.87 -3.8 2.06 perf-profile.calltrace.cycles-pp.tcp_connect.tcp_v4_connect.__inet_stream_connect.inet_stream_connect.__sys_connect 5.34 -3.5 1.81 perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_rcv_state_process.tcp_v4_do_rcv.__release_sock.release_sock 5.19 -3.4 1.75 perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_rcv_state_process.tcp_v4_do_rcv.__release_sock 5.08 -3.4 1.71 perf-profile.calltrace.cycles-pp.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_rcv_state_process.tcp_v4_do_rcv 4.91 -3.3 1.65 perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_rcv_state_process 4.38 ± 2% -3.2 1.21 perf-profile.calltrace.cycles-pp.__x64_sys_accept.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.93 -3.2 1.76 perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_connect.tcp_v4_connect.__inet_stream_connect.inet_stream_connect 4.36 ± 2% -3.2 1.21 perf-profile.calltrace.cycles-pp.__sys_accept4.__x64_sys_accept.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.65 -3.0 1.66 perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_connect.tcp_v4_connect.__inet_stream_connect 4.08 -2.8 1.33 perf-profile.calltrace.cycles-pp.sk_wait_data.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom 4.11 -2.6 1.48 perf-profile.calltrace.cycles-pp.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_connect.tcp_v4_connect 4.04 -2.6 1.45 perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_connect 2.94 -2.1 0.84 perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 2.85 -2.0 0.82 perf-profile.calltrace.cycles-pp.__sched_text_start.schedule_idle.do_idle.cpu_startup_entry.start_secondary 3.06 -2.0 1.06 perf-profile.calltrace.cycles-pp.wait_woken.sk_wait_data.tcp_recvmsg.inet_recvmsg.__sys_recvfrom 2.81 -1.8 0.99 perf-profile.calltrace.cycles-pp.schedule_timeout.wait_woken.sk_wait_data.tcp_recvmsg.inet_recvmsg 2.79 -1.8 0.98 perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.wait_woken.sk_wait_data.tcp_recvmsg 2.74 -1.8 0.96 perf-profile.calltrace.cycles-pp.__sched_text_start.schedule.schedule_timeout.wait_woken.sk_wait_data 2.45 ± 2% -1.7 0.73 perf-profile.calltrace.cycles-pp.tcp_data_queue.tcp_rcv_state_process.tcp_v4_do_rcv.tcp_v4_rcv.ip_local_deliver_finish 2.38 ± 2% -1.7 0.70 perf-profile.calltrace.cycles-pp.tcp_fin.tcp_data_queue.tcp_rcv_state_process.tcp_v4_do_rcv.tcp_v4_rcv 2.14 ± 6% -1.6 0.52 perf-profile.calltrace.cycles-pp.dput.__fput.task_work_run.exit_to_usermode_loop.do_syscall_64 2.35 -1.6 0.75 ± 2% perf-profile.calltrace.cycles-pp.__x64_sys_socket.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.32 -1.6 0.74 perf-profile.calltrace.cycles-pp.__sys_socket.__x64_sys_socket.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.19 -1.5 0.65 perf-profile.calltrace.cycles-pp.inet_accept.__sys_accept4.__x64_sys_accept.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.11 -1.4 0.67 perf-profile.calltrace.cycles-pp.tcp_child_process.tcp_v4_rcv.ip_local_deliver_finish.ip_local_deliver.ip_rcv 2.13 -1.3 0.78 perf-profile.calltrace.cycles-pp.tcp_conn_request.tcp_rcv_state_process.tcp_v4_do_rcv.tcp_v4_rcv.ip_local_deliver_finish 1.67 -1.3 0.39 ± 57% perf-profile.calltrace.cycles-pp.__sock_create.__sys_socket.__x64_sys_socket.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.83 -1.3 0.56 ± 2% perf-profile.calltrace.cycles-pp.inet_csk_accept.inet_accept.__sys_accept4.__x64_sys_accept.do_syscall_64 1.62 ± 2% -1.2 0.38 ± 57% perf-profile.calltrace.cycles-pp.release_sock.tcp_close.inet_release.__sock_release.sock_close 1.84 -1.2 0.67 perf-profile.calltrace.cycles-pp.tcp_check_req.tcp_v4_rcv.ip_local_deliver_finish.ip_local_deliver.ip_rcv 1.42 ± 2% -1.2 0.26 ±100% perf-profile.calltrace.cycles-pp.release_sock.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto 1.68 -1.1 0.59 ± 2% perf-profile.calltrace.cycles-pp.__entry_SYSCALL_64_trampoline 1.55 -1.0 0.51 perf-profile.calltrace.cycles-pp.tcp_v4_syn_recv_sock.tcp_check_req.tcp_v4_rcv.ip_local_deliver_finish.ip_local_deliver 0.00 +0.6 0.60 ± 8% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.native_queued_spin_lock_slowpath._raw_spin_lock.mntput_no_expire 0.00 +0.6 0.65 ± 7% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.native_queued_spin_lock_slowpath._raw_spin_lock.mntput_no_expire.__fput 0.00 +0.9 0.87 perf-profile.calltrace.cycles-pp.mnt_get_count.mntput_no_expire.__fput.task_work_run.exit_to_usermode_loop 63.93 +22.7 86.64 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 63.62 +22.9 86.53 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 18.21 ± 2% +53.6 71.84 perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 18.05 ± 2% +53.7 71.80 perf-profile.calltrace.cycles-pp.task_work_run.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 17.79 ± 2% +53.9 71.65 perf-profile.calltrace.cycles-pp.__fput.task_work_run.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +65.4 65.45 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.mntput_no_expire.__fput.task_work_run 0.00 +65.6 65.56 perf-profile.calltrace.cycles-pp._raw_spin_lock.mntput_no_expire.__fput.task_work_run.exit_to_usermode_loop 0.00 +66.6 66.59 perf-profile.calltrace.cycles-pp.mntput_no_expire.__fput.task_work_run.exit_to_usermode_loop.do_syscall_64 30.38 -20.7 9.64 perf-profile.children.cycles-pp.__tcp_transmit_skb 28.87 -19.8 9.08 perf-profile.children.cycles-pp.__ip_queue_xmit 31.20 ± 2% -19.5 11.68 perf-profile.children.cycles-pp.secondary_startup_64 31.20 ± 2% -19.5 11.68 perf-profile.children.cycles-pp.cpu_startup_entry 31.17 ± 2% -19.5 11.67 perf-profile.children.cycles-pp.do_idle 30.82 ± 2% -19.3 11.52 perf-profile.children.cycles-pp.start_secondary 28.01 -19.2 8.79 perf-profile.children.cycles-pp.ip_output 26.80 -18.4 8.39 perf-profile.children.cycles-pp.ip_finish_output2 24.94 -17.1 7.85 perf-profile.children.cycles-pp.__local_bh_enable_ip 24.65 -16.9 7.75 perf-profile.children.cycles-pp.do_softirq 24.98 -16.9 8.08 perf-profile.children.cycles-pp.__softirqentry_text_start 24.38 -16.7 7.67 perf-profile.children.cycles-pp.do_softirq_own_stack 23.80 -16.3 7.47 perf-profile.children.cycles-pp.net_rx_action 23.35 -16.0 7.33 perf-profile.children.cycles-pp.process_backlog 25.66 ± 3% -15.7 9.97 perf-profile.children.cycles-pp.cpuidle_enter_state 22.81 -15.7 7.15 perf-profile.children.cycles-pp.__netif_receive_skb_one_core 22.56 -15.4 7.15 perf-profile.children.cycles-pp.tcp_v4_do_rcv 22.31 -15.4 6.96 perf-profile.children.cycles-pp.ip_rcv 24.72 ± 3% -15.1 9.67 perf-profile.children.cycles-pp.intel_idle 21.66 -14.9 6.76 perf-profile.children.cycles-pp.ip_local_deliver 21.33 -14.8 6.55 perf-profile.children.cycles-pp.__tcp_push_pending_frames 21.45 -14.8 6.69 perf-profile.children.cycles-pp.ip_local_deliver_finish 21.28 -14.7 6.53 perf-profile.children.cycles-pp.tcp_write_xmit 21.21 -14.6 6.62 perf-profile.children.cycles-pp.tcp_v4_rcv 15.17 -10.8 4.40 perf-profile.children.cycles-pp.sock_close 15.13 -10.7 4.39 perf-profile.children.cycles-pp.__sock_release 14.97 -10.6 4.33 perf-profile.children.cycles-pp.inet_release 14.60 -10.4 4.22 perf-profile.children.cycles-pp.tcp_close 14.87 -9.7 5.17 perf-profile.children.cycles-pp.__x64_sys_connect 14.82 -9.7 5.16 perf-profile.children.cycles-pp.__sys_connect 14.17 -9.5 4.67 perf-profile.children.cycles-pp.tcp_rcv_state_process 13.85 -9.4 4.42 ± 2% perf-profile.children.cycles-pp.__x64_sys_sendto 13.79 -9.4 4.40 ± 2% perf-profile.children.cycles-pp.__sys_sendto 14.39 -9.4 5.00 perf-profile.children.cycles-pp.inet_stream_connect 14.27 -9.3 4.97 perf-profile.children.cycles-pp.__inet_stream_connect 13.60 -9.3 4.33 ± 2% perf-profile.children.cycles-pp.sock_sendmsg 13.41 -9.1 4.28 ± 2% perf-profile.children.cycles-pp.tcp_sendmsg 11.85 -8.1 3.73 ± 2% perf-profile.children.cycles-pp.tcp_sendmsg_locked 10.84 -7.3 3.58 perf-profile.children.cycles-pp.release_sock 10.19 -6.8 3.39 perf-profile.children.cycles-pp.__release_sock 8.21 -5.8 2.42 ± 2% perf-profile.children.cycles-pp.tcp_rcv_established 6.84 -4.7 2.09 perf-profile.children.cycles-pp.__x64_sys_recvfrom 6.76 -4.7 2.07 perf-profile.children.cycles-pp.__sys_recvfrom 6.87 -4.7 2.21 perf-profile.children.cycles-pp.__sched_text_start 6.44 -4.5 1.95 perf-profile.children.cycles-pp.inet_recvmsg 6.98 -4.5 2.49 perf-profile.children.cycles-pp.tcp_v4_connect 6.36 -4.4 1.92 perf-profile.children.cycles-pp.tcp_recvmsg 5.62 -4.0 1.65 perf-profile.children.cycles-pp.tcp_data_queue 5.87 -3.8 2.06 perf-profile.children.cycles-pp.tcp_connect 5.00 -3.6 1.44 perf-profile.children.cycles-pp.__wake_up_common_lock 4.70 -3.4 1.35 ± 2% perf-profile.children.cycles-pp.__wake_up_common 4.46 -3.2 1.27 perf-profile.children.cycles-pp.try_to_wake_up 4.38 ± 2% -3.2 1.21 perf-profile.children.cycles-pp.__x64_sys_accept 4.36 ± 2% -3.2 1.21 perf-profile.children.cycles-pp.__sys_accept4 4.08 -2.8 1.33 perf-profile.children.cycles-pp.sk_wait_data 3.79 -2.7 1.09 perf-profile.children.cycles-pp.tcp_fin 3.83 -2.7 1.14 perf-profile.children.cycles-pp.sock_def_readable 3.97 -2.6 1.38 perf-profile.children.cycles-pp.schedule 2.97 -2.1 0.85 perf-profile.children.cycles-pp.schedule_idle 3.12 -2.0 1.08 perf-profile.children.cycles-pp.wait_woken 2.68 -1.8 0.89 perf-profile.children.cycles-pp.tcp_ack 2.52 ± 3% -1.8 0.77 ± 5% perf-profile.children.cycles-pp.__dev_queue_xmit 2.38 -1.7 0.68 perf-profile.children.cycles-pp.ttwu_do_activate 2.14 ± 6% -1.6 0.52 perf-profile.children.cycles-pp.dput 2.35 -1.6 0.75 ± 2% perf-profile.children.cycles-pp.__x64_sys_socket 2.22 -1.6 0.62 ± 2% perf-profile.children.cycles-pp.enqueue_task_fair 2.32 -1.6 0.74 perf-profile.children.cycles-pp.__sys_socket 2.19 -1.5 0.65 ± 2% perf-profile.children.cycles-pp.inet_accept 2.00 ± 7% -1.5 0.47 ± 2% perf-profile.children.cycles-pp.dentry_kill 1.94 ± 7% -1.5 0.45 ± 2% perf-profile.children.cycles-pp.__dentry_kill 2.25 -1.5 0.78 ± 2% perf-profile.children.cycles-pp.__entry_SYSCALL_64_trampoline 2.12 -1.4 0.67 perf-profile.children.cycles-pp.tcp_child_process 1.95 -1.4 0.55 ± 2% perf-profile.children.cycles-pp.enqueue_entity 1.81 ± 7% -1.4 0.43 perf-profile.children.cycles-pp.sock_alloc_file 1.79 ± 7% -1.4 0.43 perf-profile.children.cycles-pp.alloc_file_pseudo 2.14 -1.4 0.78 perf-profile.children.cycles-pp.tcp_conn_request 1.83 -1.3 0.56 ± 3% perf-profile.children.cycles-pp.inet_csk_accept 1.73 ± 4% -1.2 0.51 ± 9% perf-profile.children.cycles-pp.dev_hard_start_xmit 1.85 -1.2 0.67 perf-profile.children.cycles-pp.tcp_check_req 1.68 -1.2 0.51 perf-profile.children.cycles-pp.__sock_create 1.46 -1.2 0.30 ± 2% perf-profile.children.cycles-pp._raw_spin_lock_bh 1.60 ± 4% -1.1 0.47 ± 10% perf-profile.children.cycles-pp.loopback_xmit 1.62 -1.1 0.51 ± 2% perf-profile.children.cycles-pp.tcp_clean_rtx_queue 1.51 -1.1 0.40 ± 3% perf-profile.children.cycles-pp.sock_def_wakeup 1.29 ± 10% -1.1 0.23 ± 3% perf-profile.children.cycles-pp.destroy_inode 1.62 -1.1 0.56 perf-profile.children.cycles-pp.dequeue_task_fair 1.27 ± 10% -1.1 0.22 perf-profile.children.cycles-pp.__destroy_inode 1.51 -1.0 0.47 ± 2% perf-profile.children.cycles-pp.__x64_sys_bind 1.56 -1.0 0.52 perf-profile.children.cycles-pp.tcp_v4_syn_recv_sock 1.32 ± 5% -1.0 0.28 ± 2% perf-profile.children.cycles-pp.inet_csk_destroy_sock 1.48 -1.0 0.46 perf-profile.children.cycles-pp.__sys_bind 1.45 -1.0 0.45 ± 2% perf-profile.children.cycles-pp.tcp_filter 1.40 -1.0 0.42 ± 2% perf-profile.children.cycles-pp.update_load_avg 1.40 -1.0 0.43 ± 2% perf-profile.children.cycles-pp.sk_filter_trim_cap 1.11 ± 12% -1.0 0.16 ± 2% perf-profile.children.cycles-pp.security_inode_free 1.42 ± 3% -0.9 0.48 ± 2% perf-profile.children.cycles-pp.__alloc_skb 1.34 -0.9 0.41 ± 2% perf-profile.children.cycles-pp.pick_next_task_fair 1.39 -0.9 0.47 perf-profile.children.cycles-pp.syscall_return_via_sysret 1.09 ± 11% -0.9 0.17 ± 2% perf-profile.children.cycles-pp.d_instantiate 1.05 ± 6% -0.9 0.13 ± 3% perf-profile.children.cycles-pp.sk_forced_mem_schedule 1.31 -0.9 0.41 ± 2% perf-profile.children.cycles-pp.security_sock_rcv_skb 1.30 -0.9 0.40 perf-profile.children.cycles-pp.switch_mm_irqs_off 1.39 -0.9 0.49 perf-profile.children.cycles-pp.dequeue_entity 1.35 -0.9 0.45 perf-profile.children.cycles-pp.nf_hook_slow 1.24 -0.9 0.38 perf-profile.children.cycles-pp.autoremove_wake_function 1.17 -0.8 0.32 ± 2% perf-profile.children.cycles-pp.__kfree_skb 1.07 ± 4% -0.8 0.23 perf-profile.children.cycles-pp.tcp_send_fin 1.21 -0.8 0.38 ± 2% perf-profile.children.cycles-pp.selinux_socket_sock_rcv_skb 0.96 ± 8% -0.8 0.12 ± 4% perf-profile.children.cycles-pp.__sk_mem_reduce_allocated 1.06 -0.8 0.24 ± 3% perf-profile.children.cycles-pp.lock_sock_nested 0.92 ± 13% -0.8 0.11 ± 4% perf-profile.children.cycles-pp.security_d_instantiate 1.09 ± 2% -0.8 0.30 ± 2% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.88 ± 6% -0.8 0.12 ± 5% perf-profile.children.cycles-pp.percpu_counter_add_batch 0.82 ± 15% -0.7 0.10 ± 5% perf-profile.children.cycles-pp.selinux_inode_free_security 0.81 ± 14% -0.7 0.10 ± 5% perf-profile.children.cycles-pp.inode_doinit_with_dentry 0.96 -0.7 0.28 ± 2% perf-profile.children.cycles-pp.__inet_lookup_established 0.96 -0.6 0.32 ± 2% perf-profile.children.cycles-pp.selinux_ip_postroute 0.90 -0.6 0.26 ± 2% perf-profile.children.cycles-pp.tcp_create_openreq_child 0.87 -0.6 0.24 ± 2% perf-profile.children.cycles-pp.inet_create 0.84 ± 5% -0.6 0.25 ± 3% perf-profile.children.cycles-pp.sk_stream_alloc_skb 0.78 ± 7% -0.6 0.21 ± 22% perf-profile.children.cycles-pp.netif_rx_internal 0.82 -0.6 0.25 perf-profile.children.cycles-pp.security_socket_bind 0.90 -0.6 0.34 perf-profile.children.cycles-pp.sk_reset_timer 0.78 -0.5 0.25 perf-profile.children.cycles-pp.sel_netport_sid 0.77 -0.5 0.24 perf-profile.children.cycles-pp.selinux_socket_bind 0.72 -0.5 0.20 ± 2% perf-profile.children.cycles-pp.inet_csk_clone_lock 0.83 ± 2% -0.5 0.30 ± 3% perf-profile.children.cycles-pp.menu_select 0.72 ± 2% -0.5 0.21 ± 4% perf-profile.children.cycles-pp.set_next_entity 0.77 -0.5 0.26 perf-profile.children.cycles-pp.selinux_ip_postroute_compat 0.78 ± 3% -0.5 0.27 perf-profile.children.cycles-pp.tcp_v4_send_synack 0.76 -0.5 0.24 ± 2% perf-profile.children.cycles-pp.sched_clock_cpu 0.82 -0.5 0.31 perf-profile.children.cycles-pp.tcp_done 0.67 -0.5 0.17 ± 2% perf-profile.children.cycles-pp.sk_clone_lock 0.70 -0.5 0.22 perf-profile.children.cycles-pp.select_task_rq_fair 0.86 -0.5 0.38 perf-profile.children.cycles-pp.mod_timer 0.70 -0.5 0.22 ± 3% perf-profile.children.cycles-pp.kmem_cache_free 0.70 -0.5 0.23 ± 3% perf-profile.children.cycles-pp.selinux_sock_rcv_skb_compat 0.69 ± 3% -0.5 0.23 perf-profile.children.cycles-pp.ip_route_output_key_hash 0.67 -0.5 0.21 ± 2% perf-profile.children.cycles-pp.sched_clock 0.66 ± 3% -0.4 0.21 ± 2% perf-profile.children.cycles-pp.ip_route_output_key_hash_rcu 0.66 -0.4 0.21 ± 2% perf-profile.children.cycles-pp.update_curr 0.64 -0.4 0.21 ± 2% perf-profile.children.cycles-pp.native_sched_clock 0.69 -0.4 0.26 ± 2% perf-profile.children.cycles-pp.sock_alloc 0.58 ± 3% -0.4 0.15 ± 3% perf-profile.children.cycles-pp.skb_release_all 0.64 -0.4 0.21 ± 2% perf-profile.children.cycles-pp.switch_mm 0.64 -0.4 0.22 perf-profile.children.cycles-pp.tcp_init_transfer 0.60 ± 4% -0.4 0.17 ± 2% perf-profile.children.cycles-pp.tcp_v4_destroy_sock 0.65 -0.4 0.23 ± 2% perf-profile.children.cycles-pp.new_inode_pseudo 0.67 -0.4 0.26 ± 3% perf-profile.children.cycles-pp.tcp_event_new_data_sent 0.54 ± 3% -0.4 0.14 ± 3% perf-profile.children.cycles-pp.skb_release_head_state 0.54 ± 3% -0.4 0.14 ± 6% perf-profile.children.cycles-pp.__slab_free 0.57 ± 2% -0.4 0.16 ± 9% perf-profile.children.cycles-pp.__inet_bind 0.62 -0.4 0.22 ± 4% perf-profile.children.cycles-pp.ip_local_out 0.58 -0.4 0.19 ± 4% perf-profile.children.cycles-pp.__ip_local_out 0.61 ± 2% -0.4 0.22 perf-profile.children.cycles-pp.alloc_inode 0.59 ± 3% -0.4 0.21 perf-profile.children.cycles-pp.load_new_mm_cr3 0.57 ± 3% -0.4 0.19 ± 2% perf-profile.children.cycles-pp.avc_has_perm 0.60 ± 2% -0.4 0.22 perf-profile.children.cycles-pp.kmem_cache_alloc 0.50 ± 2% -0.4 0.14 ± 3% perf-profile.children.cycles-pp.ttwu_do_wakeup 0.58 ± 4% -0.4 0.23 ± 14% perf-profile.children.cycles-pp.rcu_process_callbacks 0.53 ± 2% -0.4 0.18 ± 7% perf-profile.children.cycles-pp.poll_idle 0.48 ± 3% -0.3 0.13 ± 3% perf-profile.children.cycles-pp.check_preempt_curr 0.38 ± 5% -0.3 0.04 ± 58% perf-profile.children.cycles-pp.smpboot_thread_fn 0.48 ± 3% -0.3 0.15 ± 4% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 0.40 ± 5% -0.3 0.07 ± 20% perf-profile.children.cycles-pp.ret_from_fork 0.40 ± 5% -0.3 0.07 ± 20% perf-profile.children.cycles-pp.kthread 0.48 -0.3 0.15 ± 2% perf-profile.children.cycles-pp.tcp_get_metrics 0.50 -0.3 0.18 ± 4% perf-profile.children.cycles-pp.tcp_finish_connect 0.49 -0.3 0.16 ± 2% perf-profile.children.cycles-pp.__x64_sys_getsockopt 0.47 -0.3 0.15 ± 4% perf-profile.children.cycles-pp.__sk_destruct 0.42 ± 2% -0.3 0.10 ± 4% perf-profile.children.cycles-pp.tcp_v4_init_sock 0.48 ± 2% -0.3 0.17 ± 2% perf-profile.children.cycles-pp.lock_timer_base 0.40 ± 3% -0.3 0.09 perf-profile.children.cycles-pp.dst_release 0.45 ± 4% -0.3 0.14 ± 5% perf-profile.children.cycles-pp.ip_route_output_flow 0.46 ± 3% -0.3 0.15 ± 4% perf-profile.children.cycles-pp.__tcp_send_ack 0.40 ± 2% -0.3 0.09 perf-profile.children.cycles-pp.tcp_init_sock 0.49 -0.3 0.19 ± 2% perf-profile.children.cycles-pp.tcp_set_state 0.46 -0.3 0.15 ± 4% perf-profile.children.cycles-pp.__sys_getsockopt 0.45 -0.3 0.14 ± 3% perf-profile.children.cycles-pp.alloc_file 0.45 -0.3 0.15 ± 3% perf-profile.children.cycles-pp.___perf_sw_event 0.37 ± 14% -0.3 0.08 ± 67% perf-profile.children.cycles-pp.ktime_get_with_offset 0.40 ± 2% -0.3 0.11 ± 7% perf-profile.children.cycles-pp.inet_csk_get_port 0.41 ± 2% -0.3 0.12 perf-profile.children.cycles-pp.__x64_sys_close 0.42 ± 2% -0.3 0.13 ± 5% perf-profile.children.cycles-pp.enqueue_to_backlog 0.40 ± 2% -0.3 0.11 ± 4% perf-profile.children.cycles-pp.__tcp_get_metrics 0.45 ± 3% -0.3 0.17 ± 2% perf-profile.children.cycles-pp.___might_sleep 0.45 ± 2% -0.3 0.17 ± 3% perf-profile.children.cycles-pp.__call_rcu 0.36 ± 3% -0.3 0.09 ± 5% perf-profile.children.cycles-pp.resched_curr 0.43 ± 5% -0.3 0.16 ± 2% perf-profile.children.cycles-pp.__kmalloc_reserve 0.40 ± 2% -0.3 0.13 ± 6% perf-profile.children.cycles-pp.__switch_to 0.40 ± 2% -0.3 0.13 ± 3% perf-profile.children.cycles-pp.ipv4_mtu 0.39 ± 8% -0.3 0.12 ± 24% perf-profile.children.cycles-pp.read_tsc 0.30 ± 4% -0.3 0.04 ± 57% perf-profile.children.cycles-pp.sk_stream_kill_queues 0.40 -0.3 0.13 ± 3% perf-profile.children.cycles-pp.alloc_empty_file 0.38 -0.3 0.12 ± 3% perf-profile.children.cycles-pp.finish_task_switch 0.40 -0.3 0.14 ± 3% perf-profile.children.cycles-pp.__skb_clone 0.39 ± 4% -0.3 0.13 perf-profile.children.cycles-pp.kmem_cache_alloc_node 0.39 ± 2% -0.3 0.14 ± 3% perf-profile.children.cycles-pp.selinux_parse_skb 0.40 -0.2 0.15 ± 2% perf-profile.children.cycles-pp.__switch_to_asm 0.33 ± 2% -0.2 0.09 perf-profile.children.cycles-pp.tick_nohz_idle_exit 0.36 -0.2 0.12 ± 3% perf-profile.children.cycles-pp.sockfd_lookup_light 0.31 ± 8% -0.2 0.07 perf-profile.children.cycles-pp.tcp_write_queue_purge 0.38 ± 5% -0.2 0.14 ± 3% perf-profile.children.cycles-pp.__kmalloc_node_track_caller 0.36 -0.2 0.12 ± 3% perf-profile.children.cycles-pp.__alloc_file 0.35 ± 2% -0.2 0.11 ± 4% perf-profile.children.cycles-pp.sk_alloc 0.32 ± 2% -0.2 0.08 ± 8% perf-profile.children.cycles-pp.skb_release_data 0.38 -0.2 0.15 ± 3% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore 0.36 -0.2 0.12 perf-profile.children.cycles-pp.sk_stop_timer 0.36 ± 2% -0.2 0.13 ± 6% perf-profile.children.cycles-pp.inode_init_always 0.30 ± 4% -0.2 0.06 ± 6% perf-profile.children.cycles-pp.sock_rfree 0.33 -0.2 0.10 ± 4% perf-profile.children.cycles-pp.tcp_update_metrics 0.39 ± 4% -0.2 0.16 ± 4% perf-profile.children.cycles-pp.__netif_receive_skb_core 0.32 -0.2 0.09 perf-profile.children.cycles-pp.tcp_time_wait 0.33 ± 2% -0.2 0.10 perf-profile.children.cycles-pp.tcp_init_metrics 0.35 ± 3% -0.2 0.12 ± 5% perf-profile.children.cycles-pp.security_socket_connect 0.35 ± 2% -0.2 0.12 ± 3% perf-profile.children.cycles-pp.kfree 0.34 ± 4% -0.2 0.11 ± 3% perf-profile.children.cycles-pp.__might_sleep 0.33 ± 4% -0.2 0.10 perf-profile.children.cycles-pp.ip_finish_output 0.31 ± 2% -0.2 0.09 ± 5% perf-profile.children.cycles-pp.__sk_dst_check 0.30 ± 3% -0.2 0.08 ± 8% perf-profile.children.cycles-pp.account_entity_enqueue 0.36 ± 3% -0.2 0.14 ± 8% perf-profile.children.cycles-pp.update_cfs_group 0.38 ± 6% -0.2 0.16 ± 14% perf-profile.children.cycles-pp.start_kernel 0.32 ± 8% -0.2 0.11 ± 4% perf-profile.children.cycles-pp.fib_table_lookup 0.30 -0.2 0.09 ± 4% perf-profile.children.cycles-pp.__list_del_entry_valid 0.30 ± 3% -0.2 0.08 ± 5% perf-profile.children.cycles-pp.ipv4_dst_check 0.32 -0.2 0.11 ± 7% perf-profile.children.cycles-pp.sk_prot_alloc 0.29 ± 3% -0.2 0.08 ± 5% perf-profile.children.cycles-pp.rcu_segcblist_enqueue 0.30 ± 7% -0.2 0.10 ± 5% perf-profile.children.cycles-pp.validate_xmit_skb 0.33 -0.2 0.13 ± 9% perf-profile.children.cycles-pp.ktime_get 0.28 -0.2 0.09 ± 9% perf-profile.children.cycles-pp.select_idle_sibling 0.33 ± 3% -0.2 0.13 ± 3% perf-profile.children.cycles-pp.inet_csk_clear_xmit_timers 0.29 ± 2% -0.2 0.10 ± 5% perf-profile.children.cycles-pp.sock_getsockopt 0.29 ± 2% -0.2 0.10 ± 7% perf-profile.children.cycles-pp.selinux_socket_connect 0.28 ± 2% -0.2 0.10 ± 5% perf-profile.children.cycles-pp.__inet_hash_connect 0.27 ± 4% -0.2 0.08 ± 5% perf-profile.children.cycles-pp.reweight_entity 0.24 ± 2% -0.2 0.06 perf-profile.children.cycles-pp.__list_add_valid 0.26 ± 3% -0.2 0.08 ± 6% perf-profile.children.cycles-pp.filp_close 0.25 ± 4% -0.2 0.07 ± 6% perf-profile.children.cycles-pp.inet_csk_bind_conflict 0.29 ± 2% -0.2 0.11 perf-profile.children.cycles-pp.__update_load_avg_se 0.26 -0.2 0.08 ± 6% perf-profile.children.cycles-pp.tcp_current_mss 0.27 -0.2 0.09 ± 7% perf-profile.children.cycles-pp.__fget_light 0.27 -0.2 0.09 perf-profile.children.cycles-pp.selinux_socket_connect_helper 0.25 -0.2 0.08 ± 10% perf-profile.children.cycles-pp.update_rq_clock 0.25 ± 4% -0.2 0.08 ± 5% perf-profile.children.cycles-pp.ip_rcv_core 0.24 ± 3% -0.2 0.07 ± 5% perf-profile.children.cycles-pp.security_socket_post_create 0.24 ± 4% -0.2 0.07 perf-profile.children.cycles-pp.cpuidle_governor_latency_req 0.25 -0.2 0.08 ± 5% perf-profile.children.cycles-pp.kmem_cache_alloc_trace 0.23 -0.2 0.06 perf-profile.children.cycles-pp.ip_send_check 0.26 ± 4% -0.2 0.10 perf-profile.children.cycles-pp.__x64_sys_setsockopt 0.23 ± 4% -0.2 0.07 perf-profile.children.cycles-pp.selinux_socket_post_create 0.24 ± 2% -0.2 0.08 ± 5% perf-profile.children.cycles-pp.__might_fault 0.22 -0.2 0.07 ± 7% perf-profile.children.cycles-pp.copy_user_generic_unrolled 0.24 ± 2% -0.2 0.08 ± 5% perf-profile.children.cycles-pp.__inet_check_established 0.24 -0.2 0.08 perf-profile.children.cycles-pp.skb_copy_datagram_iter 0.25 -0.2 0.10 ± 4% perf-profile.children.cycles-pp.tcp_make_synack 0.24 ± 3% -0.2 0.09 perf-profile.children.cycles-pp.__sys_setsockopt 0.22 -0.2 0.07 ± 7% perf-profile.children.cycles-pp.sock_has_perm 0.24 ± 2% -0.2 0.09 ± 4% perf-profile.children.cycles-pp.sock_alloc_inode 0.23 ± 3% -0.2 0.08 ± 5% perf-profile.children.cycles-pp.tick_nohz_idle_enter 0.24 ± 2% -0.1 0.09 perf-profile.children.cycles-pp.del_timer 0.24 -0.1 0.09 ± 4% perf-profile.children.cycles-pp._cond_resched 0.23 ± 3% -0.1 0.08 perf-profile.children.cycles-pp.inet_put_port 0.23 ± 2% -0.1 0.08 ± 5% perf-profile.children.cycles-pp.tcp_schedule_loss_probe 0.19 ± 9% -0.1 0.04 ± 58% perf-profile.children.cycles-pp.netif_skb_features 0.23 ± 2% -0.1 0.08 ± 5% perf-profile.children.cycles-pp.inet_csk_route_req 0.22 ± 3% -0.1 0.07 perf-profile.children.cycles-pp.inet_csk_route_child_sock 0.23 -0.1 0.09 perf-profile.children.cycles-pp.security_inet_conn_request 0.25 ± 6% -0.1 0.11 ± 4% perf-profile.children.cycles-pp.pick_next_task_idle 0.17 ± 3% -0.1 0.03 ±100% perf-profile.children.cycles-pp.pick_next_entity 0.21 ± 5% -0.1 0.07 ± 6% perf-profile.children.cycles-pp.security_socket_create 0.19 ± 2% -0.1 0.05 ± 9% perf-profile.children.cycles-pp.fput 0.21 ± 2% -0.1 0.07 perf-profile.children.cycles-pp._copy_to_iter 0.21 ± 3% -0.1 0.08 ± 6% perf-profile.children.cycles-pp.security_inode_alloc 0.21 ± 2% -0.1 0.07 ± 5% perf-profile.children.cycles-pp.inet_unhash 0.18 ± 2% -0.1 0.05 perf-profile.children.cycles-pp.remove_wait_queue 0.16 ± 4% -0.1 0.03 ±100% perf-profile.children.cycles-pp.tcp_send_mss 0.16 -0.1 0.03 ±100% perf-profile.children.cycles-pp.move_addr_to_user 0.18 ± 2% -0.1 0.05 ± 8% perf-profile.children.cycles-pp.ksize 0.24 ± 3% -0.1 0.11 ± 7% perf-profile.children.cycles-pp.inet_csk_reqsk_queue_hash_add 0.22 -0.1 0.09 perf-profile.children.cycles-pp.evict 0.20 ± 2% -0.1 0.07 ± 6% perf-profile.children.cycles-pp.selinux_inode_alloc_security 0.20 ± 2% -0.1 0.07 ± 5% perf-profile.children.cycles-pp.entry_SYSCALL_64_stage2 0.18 ± 2% -0.1 0.05 ± 8% perf-profile.children.cycles-pp.tcp_wfree 0.20 ± 2% -0.1 0.07 ± 5% perf-profile.children.cycles-pp.tcp_init_xmit_timers 0.21 -0.1 0.09 ± 4% perf-profile.children.cycles-pp._copy_from_iter_full 0.17 ± 2% -0.1 0.05 ± 8% perf-profile.children.cycles-pp.available_idle_cpu 0.18 ± 4% -0.1 0.06 ± 7% perf-profile.children.cycles-pp.selinux_socket_create 0.20 ± 4% -0.1 0.08 ± 5% perf-profile.children.cycles-pp.selinux_inet_conn_request 0.17 -0.1 0.05 perf-profile.children.cycles-pp.inet_ehash_insert 0.19 ± 2% -0.1 0.07 ± 6% perf-profile.children.cycles-pp.tcp_release_cb 0.21 ± 2% -0.1 0.10 ± 4% perf-profile.children.cycles-pp.native_write_msr 0.18 ± 6% -0.1 0.06 perf-profile.children.cycles-pp.__d_alloc 0.18 ± 2% -0.1 0.06 ± 6% perf-profile.children.cycles-pp.memcpy_erms 0.14 -0.1 0.03 ±100% perf-profile.children.cycles-pp.secure_tcp_seq 0.14 ± 6% -0.1 0.03 ±100% perf-profile.children.cycles-pp.selinux_netlbl_socket_post_create 0.16 ± 5% -0.1 0.05 perf-profile.children.cycles-pp.rcu_eqs_enter 0.22 ± 3% -0.1 0.11 ± 4% perf-profile.children.cycles-pp.tcp_ack_update_rtt 0.17 ± 2% -0.1 0.06 ± 6% perf-profile.children.cycles-pp.inet_ehashfn 0.15 -0.1 0.04 ± 57% perf-profile.children.cycles-pp.inet_sk_rebuild_header 0.15 ± 2% -0.1 0.04 ± 57% perf-profile.children.cycles-pp.inet_csk_complete_hashdance 0.16 ± 4% -0.1 0.05 ± 8% perf-profile.children.cycles-pp.security_sk_alloc 0.16 ± 4% -0.1 0.05 ± 9% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length 0.17 ± 3% -0.1 0.06 ± 6% perf-profile.children.cycles-pp.bictcp_acked 0.15 ± 5% -0.1 0.05 perf-profile.children.cycles-pp.memset_erms 0.12 ± 4% -0.1 0.03 ±100% perf-profile.children.cycles-pp.selinux_sk_alloc_security 0.17 ± 6% -0.1 0.08 ± 6% perf-profile.children.cycles-pp.ip_rcv_finish 0.15 ± 7% -0.1 0.06 ± 7% perf-profile.children.cycles-pp.__inet_lookup_listener 0.15 ± 2% -0.1 0.05 ± 9% perf-profile.children.cycles-pp.sock_setsockopt 0.15 ± 2% -0.1 0.05 perf-profile.children.cycles-pp.tcp_parse_options 0.15 ± 5% -0.1 0.05 ± 8% perf-profile.children.cycles-pp.selinux_ipv4_output 0.14 ± 3% -0.1 0.06 ± 7% perf-profile.children.cycles-pp.put_prev_task_fair 0.15 ± 2% -0.1 0.06 ± 6% perf-profile.children.cycles-pp.tcp_v4_inbound_md5_hash 0.15 ± 3% -0.1 0.06 ± 6% perf-profile.children.cycles-pp.__inet_inherit_port 0.12 ± 5% -0.1 0.04 ± 57% perf-profile.children.cycles-pp.kfree_skbmem 0.13 ± 3% -0.1 0.05 perf-profile.children.cycles-pp.__alloc_fd 0.13 ± 3% -0.1 0.05 perf-profile.children.cycles-pp.tcp_cleanup_rbuf 0.09 ± 7% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__internal_add_timer 0.11 ± 4% -0.1 0.05 perf-profile.children.cycles-pp.account_entity_dequeue 0.11 ± 4% -0.1 0.05 perf-profile.children.cycles-pp.tcp_init_buffer_space 0.11 ± 7% -0.1 0.06 ± 7% perf-profile.children.cycles-pp.ip_rcv_finish_core 0.12 ± 5% -0.0 0.09 ± 17% perf-profile.children.cycles-pp.clockevents_program_event 0.09 ± 4% +0.0 0.13 ± 8% perf-profile.children.cycles-pp._raw_spin_trylock 0.08 ± 5% +0.1 0.14 ± 12% perf-profile.children.cycles-pp.rebalance_domains 0.00 +0.1 0.13 ± 3% perf-profile.children.cycles-pp.find_next_bit 0.00 +0.2 0.22 perf-profile.children.cycles-pp.cpumask_next 0.00 +0.9 0.89 perf-profile.children.cycles-pp.mnt_get_count 63.97 +22.7 86.71 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 63.66 +22.9 86.59 perf-profile.children.cycles-pp.do_syscall_64 18.22 ± 2% +53.6 71.85 perf-profile.children.cycles-pp.exit_to_usermode_loop 18.07 ± 2% +53.7 71.81 perf-profile.children.cycles-pp.task_work_run 17.81 ± 2% +53.8 71.66 perf-profile.children.cycles-pp.__fput 3.11 ± 7% +63.2 66.28 perf-profile.children.cycles-pp._raw_spin_lock 1.59 ± 12% +64.1 65.65 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 0.00 +66.6 66.59 perf-profile.children.cycles-pp.mntput_no_expire 24.72 ± 3% -15.0 9.67 perf-profile.self.cycles-pp.intel_idle 2.14 -1.4 0.75 ± 2% perf-profile.self.cycles-pp.__entry_SYSCALL_64_trampoline 1.79 -1.1 0.65 ± 2% perf-profile.self.cycles-pp._raw_spin_lock 1.39 -0.9 0.47 perf-profile.self.cycles-pp.syscall_return_via_sysret 1.03 ± 7% -0.9 0.12 ± 3% perf-profile.self.cycles-pp.sk_forced_mem_schedule 1.13 -0.9 0.27 ± 2% perf-profile.self.cycles-pp._raw_spin_lock_bh 0.94 ± 8% -0.8 0.12 ± 6% perf-profile.self.cycles-pp.__sk_mem_reduce_allocated 1.07 ± 2% -0.8 0.29 perf-profile.self.cycles-pp._raw_spin_lock_irqsave 1.08 -0.8 0.31 ± 2% perf-profile.self.cycles-pp.__sched_text_start 0.87 ± 6% -0.8 0.11 ± 4% perf-profile.self.cycles-pp.percpu_counter_add_batch 0.83 -0.6 0.23 ± 2% perf-profile.self.cycles-pp.__inet_lookup_established 0.91 -0.5 0.36 perf-profile.self.cycles-pp.__tcp_transmit_skb 0.80 -0.5 0.27 perf-profile.self.cycles-pp.tcp_v4_rcv 0.70 ± 2% -0.5 0.19 ± 2% perf-profile.self.cycles-pp.switch_mm_irqs_off 0.68 ± 2% -0.5 0.19 ± 2% perf-profile.self.cycles-pp.update_load_avg 0.67 -0.5 0.19 ± 6% perf-profile.self.cycles-pp.tcp_recvmsg 0.66 -0.4 0.23 ± 3% perf-profile.self.cycles-pp.tcp_ack 0.61 -0.4 0.20 ± 2% perf-profile.self.cycles-pp.native_sched_clock 0.58 -0.4 0.17 ± 3% perf-profile.self.cycles-pp.tcp_clean_rtx_queue 0.54 ± 3% -0.4 0.14 ± 3% perf-profile.self.cycles-pp.__slab_free 0.59 ± 3% -0.4 0.21 perf-profile.self.cycles-pp.load_new_mm_cr3 0.56 ± 3% -0.4 0.18 ± 2% perf-profile.self.cycles-pp.avc_has_perm 0.52 ± 2% -0.4 0.16 ± 8% perf-profile.self.cycles-pp.ip_finish_output2 0.50 ± 2% -0.4 0.14 ± 3% perf-profile.self.cycles-pp.selinux_socket_sock_rcv_skb 0.54 ± 2% -0.4 0.18 ± 2% perf-profile.self.cycles-pp.sel_netport_sid 0.49 -0.3 0.15 perf-profile.self.cycles-pp.update_curr 0.48 ± 2% -0.3 0.14 ± 5% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq 0.39 ± 3% -0.3 0.08 ± 5% perf-profile.self.cycles-pp.dst_release 0.42 -0.3 0.13 ± 3% perf-profile.self.cycles-pp.tcp_write_xmit 0.40 ± 2% -0.3 0.11 ± 4% perf-profile.self.cycles-pp.__tcp_get_metrics 0.41 ± 3% -0.3 0.14 ± 3% perf-profile.self.cycles-pp.__dev_queue_xmit 0.41 -0.3 0.14 ± 3% perf-profile.self.cycles-pp.__alloc_skb 0.42 ± 5% -0.3 0.14 ± 7% perf-profile.self.cycles-pp.poll_idle 0.43 ± 2% -0.3 0.16 ± 2% perf-profile.self.cycles-pp.___might_sleep 0.41 ± 2% -0.3 0.14 ± 3% perf-profile.self.cycles-pp.___perf_sw_event 0.41 -0.3 0.14 ± 3% perf-profile.self.cycles-pp.tcp_rcv_state_process 0.40 -0.3 0.13 perf-profile.self.cycles-pp.__ip_queue_xmit 0.42 -0.3 0.15 ± 2% perf-profile.self.cycles-pp.do_syscall_64 0.38 ± 4% -0.3 0.11 perf-profile.self.cycles-pp.net_rx_action 0.39 ± 2% -0.3 0.13 ± 5% perf-profile.self.cycles-pp.__switch_to 0.38 -0.3 0.12 ± 3% perf-profile.self.cycles-pp.select_task_rq_fair 0.34 ± 3% -0.3 0.08 ± 5% perf-profile.self.cycles-pp.resched_curr 0.38 -0.3 0.12 ± 3% perf-profile.self.cycles-pp.do_idle 0.39 ± 2% -0.3 0.13 ± 3% perf-profile.self.cycles-pp.__softirqentry_text_start 0.36 ± 8% -0.3 0.11 ± 26% perf-profile.self.cycles-pp.read_tsc 0.40 -0.2 0.15 ± 2% perf-profile.self.cycles-pp.__switch_to_asm 0.39 ± 2% -0.2 0.15 ± 2% perf-profile.self.cycles-pp.kmem_cache_free 0.36 ± 2% -0.2 0.12 perf-profile.self.cycles-pp.ipv4_mtu 0.32 ± 2% -0.2 0.08 ± 10% perf-profile.self.cycles-pp.skb_release_data 0.36 -0.2 0.13 ± 3% perf-profile.self.cycles-pp.selinux_parse_skb 0.29 ± 3% -0.2 0.06 perf-profile.self.cycles-pp.sock_rfree 0.35 ± 3% -0.2 0.12 perf-profile.self.cycles-pp.__local_bh_enable_ip 0.39 ± 4% -0.2 0.16 ± 5% perf-profile.self.cycles-pp.__netif_receive_skb_core 0.34 ± 3% -0.2 0.11 ± 3% perf-profile.self.cycles-pp.process_backlog 0.39 ± 3% -0.2 0.17 ± 3% perf-profile.self.cycles-pp.menu_select 0.32 ± 2% -0.2 0.10 ± 4% perf-profile.self.cycles-pp.__skb_clone 0.30 ± 2% -0.2 0.08 ± 5% perf-profile.self.cycles-pp.set_next_entity 0.35 ± 2% -0.2 0.13 ± 3% perf-profile.self.cycles-pp.ip_output 0.34 ± 2% -0.2 0.12 ± 3% perf-profile.self.cycles-pp.kfree 0.35 ± 3% -0.2 0.13 ± 5% perf-profile.self.cycles-pp.update_cfs_group 0.31 ± 2% -0.2 0.09 perf-profile.self.cycles-pp.loopback_xmit 0.35 ± 2% -0.2 0.14 ± 3% perf-profile.self.cycles-pp.kmem_cache_alloc 0.30 ± 2% -0.2 0.09 ± 5% perf-profile.self.cycles-pp.wait_woken 0.34 -0.2 0.12 ± 3% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore 0.32 ± 7% -0.2 0.10 ± 4% perf-profile.self.cycles-pp.fib_table_lookup 0.30 ± 2% -0.2 0.09 ± 4% perf-profile.self.cycles-pp.enqueue_entity 0.29 -0.2 0.09 ± 4% perf-profile.self.cycles-pp.__list_del_entry_valid 0.28 ± 4% -0.2 0.08 ± 5% perf-profile.self.cycles-pp.rcu_segcblist_enqueue 0.22 -0.2 0.03 ±100% perf-profile.self.cycles-pp.ip_rcv 0.28 ± 4% -0.2 0.08 ± 5% perf-profile.self.cycles-pp.try_to_wake_up 0.28 -0.2 0.09 ± 5% perf-profile.self.cycles-pp.finish_task_switch 0.27 -0.2 0.07 ± 5% perf-profile.self.cycles-pp.ipv4_dst_check 0.29 ± 3% -0.2 0.10 perf-profile.self.cycles-pp.__might_sleep 0.27 ± 3% -0.2 0.08 ± 6% perf-profile.self.cycles-pp.do_softirq 0.30 -0.2 0.11 ± 4% perf-profile.self.cycles-pp.tcp_rcv_established 0.27 ± 2% -0.2 0.08 perf-profile.self.cycles-pp.enqueue_task_fair 0.31 -0.2 0.12 perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 0.27 ± 4% -0.2 0.08 perf-profile.self.cycles-pp.reweight_entity 0.28 -0.2 0.10 ± 11% perf-profile.self.cycles-pp.tcp_sendmsg_locked 0.28 ± 5% -0.2 0.10 ± 4% perf-profile.self.cycles-pp.kmem_cache_alloc_node 0.26 ± 3% -0.2 0.08 ± 5% perf-profile.self.cycles-pp.enqueue_to_backlog 0.25 ± 5% -0.2 0.07 ± 6% perf-profile.self.cycles-pp.inet_csk_bind_conflict 0.24 ± 3% -0.2 0.06 perf-profile.self.cycles-pp.__list_add_valid 0.29 ± 3% -0.2 0.11 ± 3% perf-profile.self.cycles-pp.pick_next_task_fair 0.26 ± 2% -0.2 0.09 ± 9% perf-profile.self.cycles-pp.__fget_light 0.25 ± 3% -0.2 0.08 ± 5% perf-profile.self.cycles-pp.ip_rcv_core 0.27 ± 6% -0.2 0.10 ± 4% perf-profile.self.cycles-pp.__kmalloc_node_track_caller 0.23 ± 3% -0.2 0.06 perf-profile.self.cycles-pp.ip_send_check 0.24 -0.2 0.08 ± 5% perf-profile.self.cycles-pp.ip_route_output_key_hash_rcu 0.25 ± 3% -0.2 0.09 ± 4% perf-profile.self.cycles-pp.selinux_ip_postroute_compat 0.23 ± 3% -0.2 0.08 ± 5% perf-profile.self.cycles-pp.selinux_sock_rcv_skb_compat 0.21 ± 2% -0.2 0.06 ± 6% perf-profile.self.cycles-pp.copy_user_generic_unrolled 0.20 ± 3% -0.2 0.05 perf-profile.self.cycles-pp.dequeue_entity 0.21 ± 3% -0.1 0.06 perf-profile.self.cycles-pp.sock_def_readable 0.24 -0.1 0.09 ± 4% perf-profile.self.cycles-pp.__update_load_avg_se 0.21 -0.1 0.07 ± 7% perf-profile.self.cycles-pp.sock_has_perm 0.21 ± 3% -0.1 0.07 ± 6% perf-profile.self.cycles-pp.dequeue_task_fair 0.21 ± 3% -0.1 0.07 ± 6% perf-profile.self.cycles-pp.ip_local_deliver 0.20 -0.1 0.06 ± 6% perf-profile.self.cycles-pp.account_entity_enqueue 0.20 ± 2% -0.1 0.07 ± 12% perf-profile.self.cycles-pp.entry_SYSCALL_64_stage2 0.20 ± 2% -0.1 0.07 ± 7% perf-profile.self.cycles-pp.selinux_ip_postroute 0.18 ± 2% -0.1 0.05 ± 8% perf-profile.self.cycles-pp.ksize 0.19 ± 4% -0.1 0.07 ± 7% perf-profile.self.cycles-pp.__sk_destruct 0.19 -0.1 0.06 ± 6% perf-profile.self.cycles-pp.tcp_close 0.17 ± 2% -0.1 0.05 ± 8% perf-profile.self.cycles-pp.available_idle_cpu 0.21 ± 2% -0.1 0.10 ± 4% perf-profile.self.cycles-pp.native_write_msr 0.17 ± 2% -0.1 0.05 ± 8% perf-profile.self.cycles-pp.tcp_wfree 0.24 -0.1 0.12 ± 4% perf-profile.self.cycles-pp.mod_timer 0.15 ± 7% -0.1 0.04 ± 58% perf-profile.self.cycles-pp.tcp_conn_request 0.17 ± 3% -0.1 0.05 ± 8% perf-profile.self.cycles-pp.tcp_data_queue 0.16 ± 5% -0.1 0.05 perf-profile.self.cycles-pp.rcu_eqs_enter 0.18 ± 2% -0.1 0.07 ± 7% perf-profile.self.cycles-pp.tcp_release_cb 0.16 ± 4% -0.1 0.05 perf-profile.self.cycles-pp.sk_clone_lock 0.17 ± 4% -0.1 0.06 perf-profile.self.cycles-pp.tcp_connect 0.17 ± 3% -0.1 0.06 ± 11% perf-profile.self.cycles-pp.inet_ehashfn 0.17 -0.1 0.07 ± 19% perf-profile.self.cycles-pp.ktime_get 0.16 ± 4% -0.1 0.06 perf-profile.self.cycles-pp.bictcp_acked 0.12 ± 4% -0.1 0.03 ±100% perf-profile.self.cycles-pp.dev_hard_start_xmit 0.15 -0.1 0.05 perf-profile.self.cycles-pp.ip_local_deliver_finish 0.15 ± 2% -0.1 0.05 ± 8% perf-profile.self.cycles-pp.kmem_cache_alloc_trace 0.14 ± 7% -0.1 0.04 ± 58% perf-profile.self.cycles-pp.nf_hook_slow 0.15 ± 7% -0.1 0.06 ± 7% perf-profile.self.cycles-pp.__inet_lookup_listener 0.15 ± 3% -0.1 0.05 perf-profile.self.cycles-pp.memset_erms 0.15 ± 3% -0.1 0.05 perf-profile.self.cycles-pp.tcp_parse_options 0.16 ± 2% -0.1 0.06 ± 6% perf-profile.self.cycles-pp.memcpy_erms 0.13 ± 6% -0.1 0.04 ± 57% perf-profile.self.cycles-pp.__release_sock 0.12 ± 4% -0.1 0.04 ± 57% perf-profile.self.cycles-pp.tcp_schedule_loss_probe 0.15 ± 3% -0.1 0.06 perf-profile.self.cycles-pp.tcp_ack_update_rtt 0.15 ± 3% -0.1 0.07 ± 5% perf-profile.self.cycles-pp.__call_rcu 0.11 ± 4% -0.1 0.04 ± 57% perf-profile.self.cycles-pp.account_entity_dequeue 0.11 ± 8% -0.1 0.05 ± 9% perf-profile.self.cycles-pp.ip_rcv_finish_core 0.10 ± 5% -0.0 0.06 ± 6% perf-profile.self.cycles-pp.lock_timer_base 0.09 ± 4% +0.0 0.13 ± 9% perf-profile.self.cycles-pp._raw_spin_trylock 0.00 +0.1 0.08 ± 6% perf-profile.self.cycles-pp.cpumask_next 0.00 +0.1 0.12 ± 4% perf-profile.self.cycles-pp.find_next_bit 0.00 +0.1 0.13 perf-profile.self.cycles-pp.mntput_no_expire 0.00 +0.7 0.69 perf-profile.self.cycles-pp.mnt_get_count 1.57 ± 12% +63.4 65.02 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath netperf.Throughput_tps 18000 +-+-----------------------------------------------------------------+ | .+. .+.+. .+.+..+.| 16000 +-+..+.+.+.+..+ +.+..+ +..+.+.+..+.+.+.+..+.+.+.+..+.+ | | | | | 14000 +-+ | | | 12000 +-+ | | | 10000 +-+ | | | | | 8000 +-+ | | | 6000 O-O--O-O-O-O--O-O-O-O--O-O-O-O--O-O-O--O-O-O-O--O-O-O-O--O-O--------+ netperf.Throughput_total_tps 750000 +-+----------------------------------------------------------------+ 700000 +-+.+..+.+.+.+. +.+. +.+..+.+.+.+.+..+.+.+.+..+ +. .| | + | 650000 +-+ | 600000 +-+ | | | 550000 +-+ | 500000 +-+ | 450000 +-+ | | | 400000 +-+ | 350000 +-+ | | | 300000 O-O O O O O O O O O O O O O O O O O O O O O O O O O O | 250000 +-+----------------------------------------------------------------+ netperf.workload 2.4e+08 +-+---------------------------------------------------------------+ | | 2.2e+08 +-+.+.. .+.+.+.+..+.+.+.+.+..+.+.+. .+..+.+.+. .+..+.+.+.+.+..+. .| 2e+08 +-+ + + + + | | | 1.8e+08 +-+ | | | 1.6e+08 +-+ | | | 1.4e+08 +-+ | 1.2e+08 +-+ | | | 1e+08 +-+ | | | 8e+07 O-O-O--O-O-O-O-O--O-O-O-O-O--O-O-O-O-O--O-O-O-O-O--O-O-O-O--------+ pmeter.Average_Active_Power 500 +-+-------------------------------------------------------------------+ 450 O-O O O O O O O O O O O O O O O O O O O +.| | O O O O O O | 400 +-+ | 350 +-+ | | | 300 +-+ | 250 +-+ | 200 +-+ | | | 150 +-+ | 100 +-+ | | | 50 +-+ | 0 +-+----O--------------------------------------------------------------+ netperf.time.user_time 650 +-+-------------------------------------------------------------------+ 600 +-+..+. .+..+. .+..+. .+.. .+. .+.+. .+. .+..+. .+.. .+.. .+. .| | + + + + +. +. + + +.+ + +..+ | 550 +-+ | 500 +-+ | | | 450 +-+ | 400 +-+ | 350 +-+ | | | 300 +-+ | 250 +-+ | O O O O O O O O O O O O O O O O O O O | 200 +-+ O O O O O O O O | 150 +-+-------------------------------------------------------------------+ netperf.time.system_time 12500 +-+-----------------------------O--------O--------O-O-O----O--------+ O O O O O O O O O O O O O O O O O O O O O | | | 12000 +-+ | | | | | 11500 +-+ | | | 11000 +-+ .+..+.+.+.+..+. | |.+..+.+.+ +.+.+..+.+.+..+.+.+.+..+.+.+.+..+.+.+. | | +.. :| 10500 +-+ :| | + | | | 10000 +-+-----------------------------------------------------------------+ netperf.time.percent_of_cpu_this_job_got 4200 +-+------------------------------------------------------------------+ | O O O O O O O O O O O O O O O O O O O | 4100 O-O O O O O O O | | | 4000 +-+ | | | 3900 +-+ | | | 3800 +-+..+. .+..+.+.+.+..+.+.+..+.+.+..+. .+.+..+.+. .+.+.+..+.+.+ | | + + +. + | 3700 +-+ +.. :| | :| 3600 +-+ + | | | 3500 +-+------------------------------------------------------------------+ netperf.time.voluntary_context_switches 4.5e+08 +-+---------------------------------------------------------------+ |.+.+..+.+.+.+.+..+.+.+.+.+..+.+.+.+.+..+.+.+.+.+..+.+.+.+.+..+.+.| 4e+08 +-+ | | | | | 3.5e+08 +-+ | | | 3e+08 +-+ | | | 2.5e+08 +-+ | | | | | 2e+08 +-+ | O O O O O O O O O O O O O O O O O O O O O O O O O O O | 1.5e+08 +-+---------------------------------------------------------------+ pmeter.performance_per_watt 40 +-+--------------------------------------------------------------------+ | | 35 +-+..+.+.+..+.+..+.+.+..+.+.+..+.+..+.+.+..+.+.+..+.+.+..+.+..+.+.+..+.| 30 +-+ | | | 25 +-+ | | | 20 +-+ | | | 15 O-O O O O O O O O O O O O O O O O O O O O O O O O O | 10 +-+ | | | 5 +-+ | | | 0 +-+----O---------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen