mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Ingo Molnar <mingo@elte.hu>
To: Michael Buesch <mb@bu3sch.de>
Cc: Con Kolivas <kernel@kolivas.org>,
	linux-kernel@vger.kernel.org,
	Peter Zijlstra <a.p.zijlstra@chello.nl>,
	Mike Galbraith <efault@gmx.de>, Felix Fietkau <nbd@openwrt.org>
Subject: Re: BFS vs. mainline scheduler benchmarks and measurements
Date: Mon, 7 Sep 2009 20:26:29 +0200	[thread overview]
Message-ID: <20090907182629.GA3484@elte.hu> (raw)
In-Reply-To: <200909071716.57722.mb@bu3sch.de>


* Michael Buesch <mb@bu3sch.de> wrote:

> Here's a very simple test setup on an embedded singlecore bcm47xx 
> machine (WL500GPv2) It uses iperf for performance testing. The 
> iperf server is run on the embedded device. The device is so slow 
> that the iperf test is completely CPU bound. The network 
> connection is a 100MBit on the device connected via patch cable to 
> a 1000MBit machine.
> 
> The kernel is openwrt-2.6.30.5.
> 
> Here are the results:
> 
> 
> 
> Mainline CFS scheduler:
> 
> mb@homer:~$ iperf -c 192.168.1.1
> ------------------------------------------------------------
> Client connecting to 192.168.1.1, TCP port 5001
> TCP window size: 16.0 KByte (default)
> ------------------------------------------------------------
> [  3] local 192.168.1.99 port 35793 connected with 192.168.1.1 port 5001
> [ ID] Interval       Transfer     Bandwidth
> [  3]  0.0-10.0 sec  27.4 MBytes  23.0 Mbits/sec
> mb@homer:~$ iperf -c 192.168.1.1
> ------------------------------------------------------------
> Client connecting to 192.168.1.1, TCP port 5001
> TCP window size: 16.0 KByte (default)
> ------------------------------------------------------------
> [  3] local 192.168.1.99 port 35794 connected with 192.168.1.1 port 5001
> [ ID] Interval       Transfer     Bandwidth
> [  3]  0.0-10.0 sec  27.3 MBytes  22.9 Mbits/sec
> mb@homer:~$ iperf -c 192.168.1.1
> ------------------------------------------------------------
> Client connecting to 192.168.1.1, TCP port 5001
> TCP window size: 16.0 KByte (default)
> ------------------------------------------------------------
> [  3] local 192.168.1.99 port 56147 connected with 192.168.1.1 port 5001
> [ ID] Interval       Transfer     Bandwidth
> [  3]  0.0-10.0 sec  27.3 MBytes  22.9 Mbits/sec
> 
> 
> BFS scheduler:
> 
> mb@homer:~$ iperf -c 192.168.1.1
> ------------------------------------------------------------
> Client connecting to 192.168.1.1, TCP port 5001
> TCP window size: 16.0 KByte (default)
> ------------------------------------------------------------
> [  3] local 192.168.1.99 port 52489 connected with 192.168.1.1 port 5001
> [ ID] Interval       Transfer     Bandwidth
> [  3]  0.0-10.0 sec  38.2 MBytes  32.0 Mbits/sec
> mb@homer:~$ iperf -c 192.168.1.1
> ------------------------------------------------------------
> Client connecting to 192.168.1.1, TCP port 5001
> TCP window size: 16.0 KByte (default)
> ------------------------------------------------------------
> [  3] local 192.168.1.99 port 52490 connected with 192.168.1.1 port 5001
> [ ID] Interval       Transfer     Bandwidth
> [  3]  0.0-10.0 sec  38.1 MBytes  31.9 Mbits/sec
> mb@homer:~$ iperf -c 192.168.1.1
> ------------------------------------------------------------
> Client connecting to 192.168.1.1, TCP port 5001
> TCP window size: 16.0 KByte (default)
> ------------------------------------------------------------
> [  3] local 192.168.1.99 port 52491 connected with 192.168.1.1 port 5001
> [ ID] Interval       Transfer     Bandwidth
> [  3]  0.0-10.0 sec  38.1 MBytes  31.9 Mbits/sec

That's interesting. I tried to reproduce it on x86, but the profile 
does not show any scheduler overhead at all on the server:

$ perf report

#
# Samples: 8369
#
# Overhead  Symbol
# ........  ......
#
     9.20%  [k] copy_user_generic_string
     3.80%  [k] e1000_clean
     3.58%  [k] ipt_do_table
     2.72%  [k] mwait_idle
     2.68%  [k] nf_iterate
     2.28%  [k] e1000_intr
     2.15%  [k] tcp_packet
     2.10%  [k] __hash_conntrack
     1.59%  [k] read_tsc
     1.52%  [k] _local_bh_enable_ip
     1.34%  [k] eth_type_trans
     1.29%  [k] __alloc_skb
     1.19%  [k] tcp_recvmsg
     1.19%  [k] ip_rcv
     1.17%  [k] e1000_clean_rx_irq
     1.12%  [k] apic_timer_interrupt
     0.99%  [k] vsnprintf
     0.96%  [k] nf_conntrack_in
     0.96%  [k] kmem_cache_free
     0.93%  [k] __kmalloc_track_caller


Could you profile it please? Also, what's the context-switch rate?

Below is the call-graph profile as well - all the overhead is in 
networking and SLAB.

	Ingo

 $ perf report --call-graph fractal,5

#
# Samples: 8947
#
# Overhead         Command                  Shared Object  Symbol
# ........  ..............  .............................  ......
#
     9.06%           iperf  [kernel]                       [k] copy_user_generic_string
                |          
                |--98.89%-- skb_copy_datagram_iovec
                |          |          
                |          |--77.18%-- tcp_recvmsg
                |          |          sock_common_recvmsg
                |          |          __sock_recvmsg
                |          |          sock_recvmsg
                |          |          sys_recvfrom
                |          |          system_call_fastpath
                |          |          __recv
                |          |          
                |           --22.82%-- tcp_rcv_established
                |                     tcp_v4_do_rcv
                |                     tcp_prequeue_process
                |                     tcp_recvmsg
                |                     sock_common_recvmsg
                |                     __sock_recvmsg
                |                     sock_recvmsg
                |                     sys_recvfrom
                |                     system_call_fastpath
                |                     __recv
                |          
                 --1.11%-- system_call_fastpath
                           __GI___libc_nanosleep

     3.62%          [init]  [kernel]                       [k] e1000_clean
     2.96%          [init]  [kernel]                       [k] ipt_do_table
     2.79%          [init]  [kernel]                       [k] mwait_idle
     2.22%          [init]  [kernel]                       [k] e1000_intr
     1.93%          [init]  [kernel]                       [k] nf_iterate
     1.65%          [init]  [kernel]                       [k] __hash_conntrack
     1.52%          [init]  [kernel]                       [k] tcp_packet
     1.29%          [init]  [kernel]                       [k] ip_rcv
     1.18%          [init]  [kernel]                       [k] __alloc_skb
     1.15%           iperf  [kernel]                       [k] tcp_recvmsg

     1.04%          [init]  [kernel]                       [k] _local_bh_enable_ip
     1.02%          [init]  [kernel]                       [k] apic_timer_interrupt
     1.02%          [init]  [kernel]                       [k] eth_type_trans
     1.01%          [init]  [kernel]                       [k] tcp_v4_rcv
     0.96%           iperf  [kernel]                       [k] kfree
                |          
                |--95.35%-- skb_release_data
                |          __kfree_skb
                |          |          
                |          |--79.27%-- tcp_recvmsg
                |          |          sock_common_recvmsg
                |          |          __sock_recvmsg
                |          |          sock_recvmsg
                |          |          sys_recvfrom
                |          |          system_call_fastpath
                |          |          __recv
                |          |          
                |           --20.73%-- tcp_rcv_established
                |                     tcp_v4_do_rcv
                |                     tcp_prequeue_process
                |                     tcp_recvmsg
                |                     sock_common_recvmsg
                |                     __sock_recvmsg
                |                     sock_recvmsg
                |                     sys_recvfrom
                |                     system_call_fastpath
                |                     __recv
                |          
                 --4.65%-- __kfree_skb
                           |          
                           |--75.00%-- tcp_rcv_established
                           |          tcp_v4_do_rcv
                           |          tcp_prequeue_process
                           |          tcp_recvmsg
                           |          sock_common_recvmsg
                           |          __sock_recvmsg
                           |          sock_recvmsg
                           |          sys_recvfrom
                           |          system_call_fastpath
                           |          __recv
                           |          
                            --25.00%-- tcp_recvmsg
                                      sock_common_recvmsg
                                      __sock_recvmsg
                                      sock_recvmsg
                                      sys_recvfrom
                                      system_call_fastpath
                                      __recv

     0.96%          [init]  [kernel]                       [k] read_tsc
     0.92%           iperf  [kernel]                       [k] tcp_v4_do_rcv
                |          
                |--95.12%-- tcp_prequeue_process
                |          tcp_recvmsg
                |          sock_common_recvmsg
                |          __sock_recvmsg
                |          sock_recvmsg
                |          sys_recvfrom
                |          system_call_fastpath
                |          __recv
                |          
                 --4.88%-- tcp_recvmsg
                           sock_common_recvmsg
                           __sock_recvmsg
                           sock_recvmsg
                           sys_recvfrom
                           system_call_fastpath
                           __recv

     0.92%          [init]  [kernel]                       [k] e1000_clean_rx_irq
     0.86%           iperf  [kernel]                       [k] tcp_rcv_established
                |          
                |--96.10%-- tcp_v4_do_rcv
                |          tcp_prequeue_process
                |          tcp_recvmsg
                |          sock_common_recvmsg
                |          __sock_recvmsg
                |          sock_recvmsg
                |          sys_recvfrom
                |          system_call_fastpath
                |          __recv
                |          
                 --3.90%-- tcp_prequeue_process
                           tcp_recvmsg
                           sock_common_recvmsg
                           __sock_recvmsg
                           sock_recvmsg
                           sys_recvfrom
                           system_call_fastpath
                           __recv

     0.84%           iperf  [kernel]                       [k] kmem_cache_free
                |          
                |--93.33%-- __kfree_skb
                |          |          
                |          |--71.43%-- tcp_recvmsg
                |          |          sock_common_recvmsg
                |          |          __sock_recvmsg
                |          |          sock_recvmsg
                |          |          sys_recvfrom
                |          |          system_call_fastpath
                |          |          __recv
                |          |          
                |           --28.57%-- tcp_rcv_established
                |                     tcp_v4_do_rcv
                |                     tcp_prequeue_process
                |                     tcp_recvmsg
                |                     sock_common_recvmsg
                |                     __sock_recvmsg
                |                     sock_recvmsg
                |                     sys_recvfrom
                |                     system_call_fastpath
                |                     __recv
                |          
                |--4.00%-- tcp_recvmsg
                |          sock_common_recvmsg
                |          __sock_recvmsg
                |          sock_recvmsg
                |          sys_recvfrom
                |          system_call_fastpath
                |          __recv
                |          
                 --2.67%-- tcp_rcv_established
                           tcp_v4_do_rcv
                           tcp_prequeue_process
                           tcp_recvmsg
                           sock_common_recvmsg
                           __sock_recvmsg
                           sock_recvmsg
                           sys_recvfrom
                           system_call_fastpath
                           __recv

     0.80%          [init]  [kernel]                       [k] netif_receive_skb
     0.79%           iperf  [kernel]                       [k] tcp_event_data_recv
                |          
                |--83.10%-- tcp_rcv_established
                |          tcp_v4_do_rcv
                |          tcp_prequeue_process
                |          tcp_recvmsg
                |          sock_common_recvmsg
                |          __sock_recvmsg
                |          sock_recvmsg
                |          sys_recvfrom
                |          system_call_fastpath
                |          __recv
                |          
                |--12.68%-- tcp_v4_do_rcv
                |          tcp_prequeue_process
                |          tcp_recvmsg
                |          sock_common_recvmsg
                |          __sock_recvmsg
                |          sock_recvmsg
                |          sys_recvfrom
                |          system_call_fastpath
                |          __recv
                |          
                 --4.23%-- tcp_data_queue
                           tcp_rcv_established
                           tcp_v4_do_rcv
                           tcp_prequeue_process
                           tcp_recvmsg
                           sock_common_recvmsg
                           __sock_recvmsg
                           sock_recvmsg
                           sys_recvfrom
                           system_call_fastpath
                           __recv

     0.67%            perf  [kernel]                       [k] format_decode
                |          
                |--91.67%-- vsnprintf
                |          seq_printf
                |          |          
                |          |--67.27%-- show_map_vma
                |          |          show_map
                |          |          seq_read
                |          |          vfs_read
                |          |          sys_read
                |          |          system_call_fastpath
                |          |          __GI_read
                |          |          
                |          |--23.64%-- render_sigset_t
                |          |          proc_pid_status
                |          |          proc_single_show
                |          |          seq_read
                |          |          vfs_read
                |          |          sys_read
                |          |          system_call_fastpath
                |          |          __GI_read
                |          |          
                |          |--7.27%-- proc_pid_status
                |          |          proc_single_show
                |          |          seq_read
                |          |          vfs_read
                |          |          sys_read
                |          |          system_call_fastpath
                |          |          __GI_read
                |          |          
                |           --1.82%-- cpuset_task_status_allowed
                |                     proc_pid_status
                |                     proc_single_show
                |                     seq_read
                |                     vfs_read
                |                     sys_read
                |                     system_call_fastpath
                |                     __GI_read
                |          
                 --8.33%-- seq_printf
                           |          
                           |--60.00%-- proc_pid_status
                           |          proc_single_show
                           |          seq_read
                           |          vfs_read
                           |          sys_read
                           |          system_call_fastpath
                           |          __GI_read
                           |          
                            --40.00%-- show_map_vma
                                      show_map
                                      seq_read
                                      vfs_read
                                      sys_read
                                      system_call_fastpath
                                      __GI_read

     0.65%          [init]  [kernel]                       [k] __kmalloc_track_caller
     0.63%          [init]  [kernel]                       [k] nf_conntrack_in
     0.63%          [init]  [kernel]                       [k] ip_route_input
     0.58%            perf  [kernel]                       [k] vsnprintf
                |          
                |--98.08%-- seq_printf
                |          |          
                |          |--60.78%-- show_map_vma
                |          |          show_map
                |          |          seq_read
                |          |          vfs_read
                |          |          sys_read
                |          |          system_call_fastpath
                |          |          __GI_read
                |          |          
                |          |--19.61%-- render_sigset_t
                |          |          proc_pid_status
                |          |          proc_single_show
                |          |          seq_read
                |          |          vfs_read
                |          |          sys_read
                |          |          system_call_fastpath
                |          |          __GI_read
                |          |          
                |          |--9.80%-- proc_pid_status
                |          |          proc_single_show
                |          |          seq_read
                |          |          vfs_read
                |          |          sys_read
                |          |          system_call_fastpath
                |          |          __GI_read
                |          |          
                |          |--3.92%-- task_mem
                |          |          proc_pid_status
                |          |          proc_single_show
                |          |          seq_read
                |          |          vfs_read
                |          |          sys_read
                |          |          system_call_fastpath
                |          |          __GI_read
                |          |          
                |          |--3.92%-- cpuset_task_status_allowed
                |          |          proc_pid_status
                |          |          proc_single_show
                |          |          seq_read
                |          |          vfs_read
                |          |          sys_read
                |          |          system_call_fastpath
                |          |          __GI_read
                |          |          
                |           --1.96%-- render_cap_t
                |                     proc_pid_status
                |                     proc_single_show
                |                     seq_read
                |                     vfs_read
                |                     sys_read
                |                     system_call_fastpath
                |                     __GI_read
                |          
                 --1.92%-- snprintf
                           proc_task_readdir
                           vfs_readdir
                           sys_getdents
                           system_call_fastpath
                           __getdents64
                           0x69706565000a3430

     0.57%          [init]  [kernel]                       [k] ktime_get
     0.57%          [init]  [kernel]                       [k] nf_nat_fn
     0.56%           iperf  [kernel]                       [k] tcp_packet
                |          
                |--68.00%-- __tcp_ack_snd_check
                |          tcp_rcv_established
                |          tcp_v4_do_rcv
                |          tcp_prequeue_process
                |          tcp_recvmsg
                |          sock_common_recvmsg
                |          __sock_recvmsg
                |          sock_recvmsg
                |          sys_recvfrom
                |          system_call_fastpath
                |          __recv
                |          
                 --32.00%-- tcp_cleanup_rbuf
                           tcp_recvmsg
                           sock_common_recvmsg
                           __sock_recvmsg
                           sock_recvmsg
                           sys_recvfrom
                           system_call_fastpath
                           __recv

     0.56%           iperf  /usr/bin/iperf                 [.] 0x000000000059f8
                |          
                |--8.00%-- 0x4059f8
                |          
                |--8.00%-- 0x405a16
                |          
                |--8.00%-- 0x4059fd
                |          
                |--4.00%-- 0x409d22
                |          
                |--4.00%-- 0x405871
                |          
                |--4.00%-- 0x406ee1
                |          
                |--4.00%-- 0x405726
                |          
                |--4.00%-- 0x4058db
                |          
                |--4.00%-- 0x406ee8
                |          
                |--2.00%-- 0x405b60
                |          
                |--2.00%-- 0x4058fd
                |          
                |--2.00%-- 0x4058d5
                |          
                |--2.00%-- 0x405490
                |          
                |--2.00%-- 0x4058bb
                |          
                |--2.00%-- 0x405b93
                |          
                |--2.00%-- 0x405b8e
                |          
                |--2.00%-- 0x405903
                |          
                |--2.00%-- 0x405ba8
                |          
                |--2.00%-- 0x406eae
                |          
                |--2.00%-- 0x405545
                |          
                |--2.00%-- 0x405870
                |          
                |--2.00%-- 0x405b67
                |          
                |--2.00%-- 0x4058ce
                |          
                |--2.00%-- 0x40570e
                |          
                |--2.00%-- 0x406ee4
                |          
                |--2.00%-- 0x405a02
                |          
                |--2.00%-- 0x406eec
                |          
                |--2.00%-- 0x405b82
                |          
                |--2.00%-- 0x40556a
                |          
                |--2.00%-- 0x405755
                |          
                |--2.00%-- 0x405a0a
                |          
                |--2.00%-- 0x405498
                |          
                |--2.00%-- 0x409d20
                |          
                |--2.00%-- 0x405b21
                |          
                 --2.00%-- 0x405a2c

     0.56%          [init]  [kernel]                       [k] kmem_cache_alloc
     0.56%          [init]  [kernel]                       [k] __inet_lookup_established
     0.55%            perf  [kernel]                       [k] number
                |          
                |--95.92%-- vsnprintf
                |          |          
                |          |--97.87%-- seq_printf
                |          |          |          
                |          |          |--56.52%-- show_map_vma
                |          |          |          show_map
                |          |          |          seq_read
                |          |          |          vfs_read
                |          |          |          sys_read
                |          |          |          system_call_fastpath
                |          |          |          __GI_read
                |          |          |          
                |          |          |--28.26%-- render_sigset_t
                |          |          |          proc_pid_status
                |          |          |          proc_single_show
                |          |          |          seq_read
                |          |          |          vfs_read
                |          |          |          sys_read
                |          |          |          system_call_fastpath
                |          |          |          __GI_read
                |          |          |          
                |          |          |--6.52%-- proc_pid_status
                |          |          |          proc_single_show
                |          |          |          seq_read
                |          |          |          vfs_read
                |          |          |          sys_read
                |          |          |          system_call_fastpath
                |          |          |          __GI_read
                |          |          |          
                |          |          |--4.35%-- render_cap_t
                |          |          |          proc_pid_status
                |          |          |          proc_single_show
                |          |          |          seq_read
                |          |          |          vfs_read
                |          |          |          sys_read
                |          |          |          system_call_fastpath
                |          |          |          __GI_read
                |          |          |          
                |          |           --4.35%-- task_mem
                |          |                     proc_pid_status
                |          |                     proc_single_show
                |          |                     seq_read
                |          |                     vfs_read
                |          |                     sys_read
                |          |                     system_call_fastpath
                |          |                     __GI_read
                |          |          
                |           --2.13%-- scnprintf
                |                     bitmap_scnlistprintf
                |                     seq_bitmap_list
                |                     cpuset_task_status_allowed
                |                     proc_pid_status
                |                     proc_single_show
                |                     seq_read
                |                     vfs_read
                |                     sys_read
                |                     system_call_fastpath
                |                     __GI_read
                |          
                 --4.08%-- seq_printf
                           |          
                           |--50.00%-- show_map_vma
                           |          show_map
                           |          seq_read
                           |          vfs_read
                           |          sys_read
                           |          system_call_fastpath
                           |          __GI_read
                           |          
                            --50.00%-- render_sigset_t
                                      proc_pid_status
                                      proc_single_show
                                      seq_read
                                      vfs_read
                                      sys_read
                                      system_call_fastpath
                                      __GI_read

     0.55%          [init]  [kernel]                       [k] native_sched_clock
     0.50%           iperf  [kernel]                       [k] e1000_xmit_frame
                |          
                |--71.11%-- __tcp_ack_snd_check
                |          tcp_rcv_established
                |          tcp_v4_do_rcv
                |          tcp_prequeue_process
                |          tcp_recvmsg
                |          sock_common_recvmsg
                |          __sock_recvmsg
                |          sock_recvmsg
                |          sys_recvfrom
                |          system_call_fastpath
                |          __recv
                |          
                 --28.89%-- tcp_cleanup_rbuf
                           tcp_recvmsg
                           sock_common_recvmsg
                           __sock_recvmsg
                           sock_recvmsg
                           sys_recvfrom
                           system_call_fastpath
                           __recv

     0.50%           iperf  [kernel]                       [k] ipt_do_table
                |          
                |--37.78%-- ipt_local_hook
                |          nf_iterate
                |          nf_hook_slow
                |          __ip_local_out
                |          ip_local_out
                |          ip_queue_xmit
                |          tcp_transmit_skb
                |          tcp_send_ack
                |          |          
                |          |--58.82%-- __tcp_ack_snd_check
                |          |          tcp_rcv_established
                |          |          tcp_v4_do_rcv
                |          |          tcp_prequeue_process
                |          |          tcp_recvmsg
                |          |          sock_common_recvmsg
                |          |          __sock_recvmsg
                |          |          sock_recvmsg
                |          |          sys_recvfrom
                |          |          system_call_fastpath
                |          |          __recv
                |          |          
                |           --41.18%-- tcp_cleanup_rbuf
                |                     tcp_recvmsg
                |                     sock_common_recvmsg
                |                     __sock_recvmsg
                |                     sock_recvmsg
                |                     sys_recvfrom
                |                     system_call_fastpath
                |                     __recv
                |          
                |--31.11%-- ipt_post_routing_hook
                |          nf_iterate
                |          nf_hook_slow
                |          ip_output
                |          ip_local_out
                |          ip_queue_xmit
                |          tcp_transmit_skb
                |          tcp_send_ack
                |          |          
                |          |--64.29%-- __tcp_ack_snd_check
                |          |          tcp_rcv_established
                |          |          tcp_v4_do_rcv
                |          |          tcp_prequeue_process
                |          |          tcp_recvmsg
                |          |          sock_common_recvmsg
                |          |          __sock_recvmsg
                |          |          sock_recvmsg
                |          |          sys_recvfrom
                |          |          system_call_fastpath
                |          |          __recv
                |          |          
                |           --35.71%-- tcp_cleanup_rbuf
                |                     tcp_recvmsg
                |                     sock_common_recvmsg
                |                     __sock_recvmsg
                |                     sock_recvmsg
                |                     sys_recvfrom
                |                     system_call_fastpath
                |                     __recv
                |          
                |--20.00%-- ipt_local_out_hook
                |          nf_iterate
                |          nf_hook_slow
                |          __ip_local_out
                |          ip_local_out
                |          ip_queue_xmit
                |          tcp_transmit_skb
                |          tcp_send_ack
                |          |          
                |          |--88.89%-- __tcp_ack_snd_check
                |          |          tcp_rcv_established
                |          |          tcp_v4_do_rcv
                |          |          tcp_prequeue_process
                |          |          tcp_recvmsg
                |          |          sock_common_recvmsg
                |          |          __sock_recvmsg
                |          |          sock_recvmsg
                |          |          sys_recvfrom
                |          |          system_call_fastpath
                |          |          __recv
                |          |          
                |           --11.11%-- tcp_cleanup_rbuf
                |                     tcp_recvmsg
                |                     sock_common_recvmsg
                |                     __sock_recvmsg
                |                     sock_recvmsg
                |                     sys_recvfrom
                |                     system_call_fastpath
                |                     __recv
                |          
                |--6.67%-- nf_iterate
                |          nf_hook_slow
                |          |          
                |          |--66.67%-- ip_output
                |          |          ip_local_out
                |          |          ip_queue_xmit
                |          |          tcp_transmit_skb
                |          |          tcp_send_ack
                |          |          tcp_cleanup_rbuf
                |          |          tcp_recvmsg
                |          |          sock_common_recvmsg
                |          |          __sock_recvmsg
                |          |          sock_recvmsg
                |          |          sys_recvfrom
                |          |          system_call_fastpath
                |          |          __recv
                |          |          
                |           --33.33%-- __ip_local_out
                |                     ip_local_out
                |                     ip_queue_xmit
                |                     tcp_transmit_skb
                |                     tcp_send_ack
                |                     __tcp_ack_snd_check
                |                     tcp_rcv_established
                |                     tcp_v4_do_rcv
                |                     tcp_prequeue_process
                |                     tcp_recvmsg
                |                     sock_common_recvmsg
                |                     __sock_recvmsg
                |                     sock_recvmsg
                |                     sys_recvfrom
                |                     system_call_fastpath
                |                     __recv
                |          
                |--2.22%-- ipt_local_in_hook
                |          nf_iterate
                |          nf_hook_slow
                |          ip_local_deliver
                |          ip_rcv_finish
                |          ip_rcv
                |          netif_receive_skb
                |          napi_skb_finish
                |          napi_gro_receive
                |          e1000_receive_skb
                |          e1000_clean_rx_irq
                |          e1000_clean
                |          net_rx_action
                |          __do_softirq
                |          call_softirq
                |          do_softirq
                |          irq_exit
                |          do_IRQ
                |          ret_from_intr
                |          vgettimeofday
                |          
                 --2.22%-- ipt_pre_routing_hook
                           nf_iterate
                           nf_hook_slow
                           ip_rcv
                           netif_receive_skb
                           napi_skb_finish
                           napi_gro_receive
                           e1000_receive_skb
                           e1000_clean_rx_irq
                           e1000_clean
                           net_rx_action
                           __do_softirq
                           call_softirq
                           do_softirq
                           irq_exit
                           do_IRQ
                           ret_from_intr
                           __GI___libc_nanosleep

     0.50%           iperf  [kernel]                       [k] schedule
                |          
                |--57.78%-- do_nanosleep
                |          hrtimer_nanosleep
                |          sys_nanosleep
                |          system_call_fastpath
                |          __GI___libc_nanosleep
                |          
                |--33.33%-- schedule_timeout
                |          sk_wait_data
                |          tcp_recvmsg
                |          sock_common_recvmsg
                |          __sock_recvmsg
                |          sock_recvmsg
                |          sys_recvfrom
                |          system_call_fastpath
                |          __recv
                |          
                |--6.67%-- hrtimer_nanosleep
                |          sys_nanosleep
                |          system_call_fastpath
                |          __GI___libc_nanosleep
                |          
                 --2.22%-- sk_wait_data
                           tcp_recvmsg
                           sock_common_recvmsg
                           __sock_recvmsg
                           sock_recvmsg
                           sys_recvfrom
                           system_call_fastpath
                           __recv

     0.49%           iperf  [kernel]                       [k] tcp_transmit_skb
                |          
                |--97.73%-- tcp_send_ack
                |          |          
                |          |--83.72%-- __tcp_ack_snd_check
                |          |          tcp_rcv_established
                |          |          tcp_v4_do_rcv
                |          |          |          
                |          |          |--97.22%-- tcp_prequeue_process
                |          |          |          tcp_recvmsg
                |          |          |          sock_common_recvmsg
                |          |          |          __sock_recvmsg
                |          |          |          sock_recvmsg
                |          |          |          sys_recvfrom
                |          |          |          system_call_fastpath
                |          |          |          __recv
                |          |          |          
                |          |           --2.78%-- release_sock
                |          |                     tcp_recvmsg
                |          |                     sock_common_recvmsg
                |          |                     __sock_recvmsg
                |          |                     sock_recvmsg
                |          |                     sys_recvfrom
                |          |                     system_call_fastpath
                |          |                     __recv
                |          |          
                |           --16.28%-- tcp_cleanup_rbuf
                |                     tcp_recvmsg
                |                     sock_common_recvmsg
                |                     __sock_recvmsg
                |                     sock_recvmsg
                |                     sys_recvfrom
                |                     system_call_fastpath
                |                     __recv
                |          
                 --2.27%-- __tcp_ack_snd_check
                           tcp_rcv_established
                           tcp_v4_do_rcv
                           tcp_prequeue_process
                           tcp_recvmsg
                           sock_common_recvmsg
                           __sock_recvmsg
                           sock_recvmsg
                           sys_recvfrom
                           system_call_fastpath
                           __recv

     0.49%          [init]  [kernel]                       [k] nf_hook_slow
     0.48%           iperf  [kernel]                       [k] virt_to_head_page
                |          
                |--53.49%-- kfree
                |          skb_release_data
                |          __kfree_skb
                |          |          
                |          |--65.22%-- tcp_recvmsg
                |          |          sock_common_recvmsg
                |          |          __sock_recvmsg
                |          |          sock_recvmsg
                |          |          sys_recvfrom
                |          |          system_call_fastpath
                |          |          __recv
                |          |          
                |           --34.78%-- tcp_rcv_established
                |                     tcp_v4_do_rcv
                |                     tcp_prequeue_process
                |                     tcp_recvmsg
                |                     sock_common_recvmsg
                |                     __sock_recvmsg
                |                     sock_recvmsg
                |                     sys_recvfrom
                |                     system_call_fastpath
                |                     __recv
                |          
                |--18.60%-- skb_release_data
                |          __kfree_skb
                |          |          
                |          |--62.50%-- tcp_rcv_established
                |          |          tcp_v4_do_rcv
                |          |          tcp_prequeue_process
                |          |          tcp_recvmsg
                |          |          sock_common_recvmsg
                |          |          __sock_recvmsg
                |          |          sock_recvmsg
                |          |          sys_recvfrom
                |          |          system_call_fastpath
                |          |          __recv
                |          |          
                |           --37.50%-- tcp_recvmsg
                |                     sock_common_recvmsg
                |                     __sock_recvmsg
                |                     sock_recvmsg
                |                     sys_recvfrom
                |                     system_call_fastpath
                |                     __recv
                |          
                |--18.60%-- kmem_cache_free
                |          __kfree_skb
                |          |          
                |          |--62.50%-- tcp_rcv_established
                |          |          tcp_v4_do_rcv
                |          |          tcp_prequeue_process
                |          |          tcp_recvmsg
                |          |          sock_common_recvmsg
                |          |          __sock_recvmsg
                |          |          sock_recvmsg
                |          |          sys_recvfrom
                |          |          system_call_fastpath
                |          |          __recv
                |          |          
                |           --37.50%-- tcp_recvmsg
                |                     sock_common_recvmsg
                |                     __sock_recvmsg
                |                     sock_recvmsg
                |                     sys_recvfrom
                |                     system_call_fastpath
                |                     __recv
                |          
                 --9.30%-- __kfree_skb
                           |          
                           |--75.00%-- tcp_rcv_established
                           |          tcp_v4_do_rcv
                           |          tcp_prequeue_process
                           |          tcp_recvmsg
                           |          sock_common_recvmsg
                           |          __sock_recvmsg
                           |          sock_recvmsg
                           |          sys_recvfrom
                           |          system_call_fastpath
                           |          __recv
                           |          
                            --25.00%-- tcp_recvmsg
                                      sock_common_recvmsg
                                      __sock_recvmsg
                                      sock_recvmsg
                                      sys_recvfrom
                                      system_call_fastpath
                                      __recv
 ...

  reply	other threads:[~2009-09-07 18:26 UTC|newest]

Thread overview: 224+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2009-09-06 20:59 Ingo Molnar
2009-09-07  2:05 ` Frans Pop
2009-09-07 12:16   ` [quad core results] " Ingo Molnar
2009-09-07 12:36     ` Stefan Richter
2009-09-07 13:41     ` Markus Tornqvist
2009-09-07 13:59       ` Ingo Molnar
2009-09-09  5:54         ` Markus Tornqvist
2009-09-07 14:45       ` Arjan van de Ven
2009-09-07 15:20         ` Frans Pop
2009-09-07 15:36           ` Arjan van de Ven
2009-09-07 15:47             ` Frans Pop
2009-09-07 15:24         ` Xavier Bestel
2009-09-07 15:37           ` Arjan van de Ven
2009-09-07 16:00           ` Diego Calleja
2009-09-07 15:34     ` Nikos Chantziaras
2009-09-07  3:38 ` Nikos Chantziaras
2009-09-07 11:01   ` Frederic Weisbecker
2009-09-08 18:15     ` Nikos Chantziaras
2009-09-10 20:25       ` Frederic Weisbecker
2009-09-07 14:40   ` Arjan van de Ven
2009-09-08  7:19     ` Nikos Chantziaras
2009-09-08  8:31       ` Arjan van de Ven
2009-09-08 20:22         ` Frans Pop
2009-09-08 21:10           ` Michal Schmidt
2009-09-08 21:11           ` Frans Pop
2009-09-08 21:40             ` GeunSik Lim
2009-09-08 22:36               ` Frans Pop
2009-09-09  9:53           ` Benjamin Herrenschmidt
2009-09-09 11:14             ` David Newall
2009-09-09 11:32               ` Benjamin Herrenschmidt
2009-09-09 11:55             ` Frans Pop
2009-09-11  1:36               ` Benjamin Herrenschmidt
2009-09-16 18:27                 ` Frans Pop
2009-09-17  1:29                   ` Benjamin Herrenschmidt
2009-10-01  9:36                     ` Frans Pop
2009-09-08  8:38       ` Arjan van de Ven
2009-09-08 10:13         ` Nikos Chantziaras
2009-09-08 11:32           ` Juergen Beisert
2009-09-08 22:00             ` Nikos Chantziaras
2009-09-08 23:20               ` Jiri Kosina
2009-09-08 23:38                 ` Nikos Chantziaras
2009-09-08 12:03           ` Theodore Tso
2009-09-08 21:28             ` Nikos Chantziaras
2009-09-08 14:20           ` Arjan van de Ven
2009-09-08 22:53             ` Nikos Chantziaras
2009-09-07 23:54   ` Thomas Fjellstrom
2009-09-08 11:30     ` Nikos Chantziaras
2009-09-07  3:50 ` Con Kolivas
2009-09-07 18:20   ` Jerome Glisse
2009-09-07  9:49 ` Jens Axboe
2009-09-07 10:12   ` Nikos Chantziaras
2009-09-07 10:41     ` Jens Axboe
2009-09-07 11:57   ` Jens Axboe
2009-09-07 14:14     ` Ingo Molnar
2009-09-07 17:38       ` Jens Axboe
2009-09-07 20:44         ` Jens Axboe
2009-09-08  9:13           ` Jens Axboe
2009-09-08 15:23             ` Peter Zijlstra
2009-09-08 20:34               ` Jens Axboe
2009-09-09  6:13                 ` Ingo Molnar
2009-09-09  8:34                   ` Nikos Chantziaras
2009-09-09  8:52                   ` Mike Galbraith
2009-09-09  9:02                     ` Peter Zijlstra
2009-09-09  9:18                       ` Mike Galbraith
2009-09-09  9:05                     ` Nikos Chantziaras
2009-09-09  9:17                       ` Peter Zijlstra
2009-09-09  9:40                         ` Nikos Chantziaras
2009-09-09 10:17                           ` Nikos Chantziaras
2009-09-10 19:45                         ` Martin Steigerwald
2009-09-10 20:06                           ` Ingo Molnar
2009-09-10 20:39                             ` Martin Steigerwald
2009-09-10 20:42                               ` Ingo Molnar
2009-09-10 21:19                                 ` Martin Steigerwald
2009-09-11  9:26                                   ` Mat
2009-09-12 11:26                                     ` Martin Steigerwald
2009-09-09  9:10                     ` Jens Axboe
2009-09-09 11:54                       ` Jens Axboe
2009-09-09 12:20                         ` Jens Axboe
2009-09-09 18:04                           ` Ingo Molnar
2009-09-09 20:12                             ` Nikos Chantziaras
2009-09-09 20:50                               ` Jens Axboe
2009-09-10  1:02                                 ` Con Kolivas
2009-09-10 11:03                                   ` Jens Axboe
2009-09-10  3:15                               ` Mike Galbraith
2009-09-10  6:08                               ` Ingo Molnar
2009-09-10  6:40                                 ` Ingo Molnar
2009-09-10  9:54                                   ` Jens Axboe
2009-09-10 10:03                                     ` Ingo Molnar
2009-09-10 10:11                                       ` Jens Axboe
2009-09-10 10:28                                         ` Jens Axboe
2009-09-10 10:57                                           ` Mike Galbraith
2009-09-10 11:09                                             ` Jens Axboe
2009-09-10 11:21                                               ` Mike Galbraith
2009-09-10 11:24                                                 ` Jens Axboe
2009-09-10 11:28                                                   ` Mike Galbraith
2009-09-10 11:35                                                     ` Jens Axboe
2009-09-10 11:42                                                       ` Mike Galbraith
2009-09-10 16:02                                 ` Bret Towe
2009-09-10 16:05                                   ` Peter Zijlstra
2009-09-10 16:12                                     ` Bret Towe
2009-09-10 16:26                                       ` Ingo Molnar
2009-09-10 16:33                                         ` Bret Towe
2009-09-10 17:03                                           ` Ingo Molnar
2009-09-10 17:53                                 ` Nikos Chantziaras
2009-09-10 18:46                                   ` Ingo Molnar
2009-09-10 18:51                                   ` [tip:sched/core] sched: Disable NEW_FAIR_SLEEPERS for now tip-bot for Ingo Molnar
2009-09-10 18:57                                   ` [tip:sched/core] sched: Fix sched::sched_stat_wait tracepoint field tip-bot for Ingo Molnar
2009-09-10  9:48                             ` BFS vs. mainline scheduler benchmarks and measurements Jens Axboe
2009-09-10  9:59                               ` Ingo Molnar
2009-09-10 10:01                                 ` Jens Axboe
2009-09-10  6:55                           ` Peter Zijlstra
2009-09-10  6:58                             ` Jens Axboe
2009-09-10  7:04                               ` Ingo Molnar
2009-09-10  9:44                                 ` Jens Axboe
2009-09-10  9:45                                   ` Jens Axboe
2009-09-10 13:53                                   ` Steven Rostedt
2009-09-10  7:33                               ` Jens Axboe
2009-09-10  7:49                                 ` Ingo Molnar
2009-09-10  7:53                                   ` Jens Axboe
2009-09-10 10:02                                     ` Ingo Molnar
2009-09-10 10:09                                       ` Jens Axboe
2009-09-10 18:00                                       ` [crash, bisected] Re: clocksource: Resolve cpu hotplug dead lock with TSC unstable Ingo Molnar
2009-09-11  7:37                                         ` Ingo Molnar
2009-09-11  7:48                                           ` Martin Schwidefsky
2009-09-11 13:33                                           ` Martin Schwidefsky
2009-09-11 18:22                                             ` [tip:timers/core] clocksource: Resolve cpu hotplug dead lock with TSC unstable, fix crash tip-bot for Martin Schwidefsky
2009-09-14 15:19                                             ` [crash, bisected] Re: clocksource: Resolve cpu hotplug dead lock with TSC unstable Ingo Molnar
2009-09-14 15:37                                               ` Martin Schwidefsky
2009-09-14 17:59                                               ` Martin Schwidefsky
2009-09-10  6:59                             ` BFS vs. mainline scheduler benchmarks and measurements Ingo Molnar
2009-09-09 12:48                         ` Mike Galbraith
2009-09-09 15:37                     ` [tip:sched/core] sched: Turn off child_runs_first tip-bot for Mike Galbraith
2009-09-09 17:57                       ` Theodore Tso
2009-09-09 18:08                         ` Ingo Molnar
2009-09-09 18:59                           ` Chris Friesen
2009-09-09 19:48                           ` Pavel Machek
2009-09-09 15:37                     ` [tip:sched/core] sched: Re-tune the scheduler latency defaults to decrease worst-case latencies tip-bot for Mike Galbraith
2009-09-12 11:45                       ` Martin Steigerwald
2009-09-09 15:37                     ` [tip:sched/core] sched: Keep kthreads at default priority tip-bot for Mike Galbraith
2009-09-09 16:55                       ` Dmitry Torokhov
2009-09-09 17:06                         ` Peter Zijlstra
2009-09-09 17:34                           ` Mike Galbraith
2009-09-12 11:48                             ` Martin Steigerwald
2009-09-12 12:19                               ` Mike Galbraith
2009-09-09 11:52               ` BFS vs. mainline scheduler benchmarks and measurements Nikos Chantziaras
2009-09-07 18:02   ` Avi Kivity
2009-09-07 18:46     ` Jens Axboe
2009-09-07 20:36       ` Ingo Molnar
2009-09-07 20:46         ` Jens Axboe
2009-09-07 21:03           ` Peter Zijlstra
2009-09-07 21:05             ` Jens Axboe
2009-09-07 22:18               ` Ingo Molnar
2009-09-09  7:38   ` Pavel Machek
2009-09-10 12:19     ` latt location (Was Re: BFS vs. mainline scheduler benchmarks and measurements) Jens Axboe
2009-09-07 15:16 ` BFS vs. mainline scheduler benchmarks and measurements Michael Buesch
2009-09-07 18:26   ` Ingo Molnar [this message]
2009-09-07 18:47     ` Daniel Walker
2009-09-07 18:51     ` Michael Buesch
2009-09-07 20:57       ` Ingo Molnar
2009-09-07 23:24         ` Pekka Pietikainen
2009-09-08  8:04           ` Ingo Molnar
2009-09-08  8:13             ` Nikos Chantziaras
2009-09-08 10:12               ` Ingo Molnar
2009-09-08 10:40                 ` Nikos Chantziaras
2009-09-08 11:35                   ` Ingo Molnar
2009-09-08 19:06                     ` Nikos Chantziaras
2009-09-08 12:00                 ` el_es
2009-09-08 15:45         ` Michael Buesch
2009-09-08  7:48     ` Ingo Molnar
2009-09-08  9:50       ` Benjamin Herrenschmidt
2009-09-08 13:09         ` Ralf Baechle
2009-09-09  1:36           ` Felix Fietkau
2009-09-08 13:09         ` Felix Fietkau
2009-09-09  0:28           ` Benjamin Herrenschmidt
2009-09-09  0:37             ` David Miller
2009-09-08 14:45       ` Michael Buesch
2009-09-18 11:24         ` Ingo Molnar
2009-09-18 14:46           ` Felix Fietkau
2009-09-19 18:01             ` Ingo Molnar
2009-09-19 18:43               ` Felix Fietkau
2009-09-19 19:39                 ` Ingo Molnar
2009-09-19 20:15                   ` Felix Fietkau
2009-09-19 20:22                     ` Ingo Molnar
2009-09-19 20:33                       ` Felix Fietkau
2009-09-20 18:10                         ` Ingo Molnar
2009-09-08 12:57 ` Epic regression in throughput since v2.6.23 Serge Belyshev
2009-09-08 17:47   ` Jesse Brandeburg
2009-09-08 18:20     ` Nikos Chantziaras
2009-09-08 19:00     ` Jeff Garzik
2009-09-08 19:20       ` Serge Belyshev
2009-09-08 19:26         ` Jeff Garzik
2009-09-08 18:37   ` Nikos Chantziaras
2009-09-08 22:15   ` Serge Belyshev
2009-09-09 15:52     ` Ingo Molnar
2009-09-09 20:49       ` Serge Belyshev
2009-09-09 21:23         ` Cory Fields
2009-09-10  6:53         ` Ingo Molnar
2009-09-10 23:23           ` Serge Belyshev
2009-09-11  6:10             ` Ingo Molnar
2009-09-11  8:55               ` Serge Belyshev
2009-09-13 15:27               ` Serge Belyshev
2009-09-13 15:47                 ` Ingo Molnar
2009-09-13 19:17                   ` Mike Galbraith
2009-09-14  6:15                     ` Mike Galbraith
2009-09-16 19:45                 ` Ingo Molnar
2009-09-16 23:18                   ` Serge Belyshev
2009-09-17  4:55                     ` [patchlet] " Mike Galbraith
2009-09-17  5:06                       ` Mike Galbraith
2009-09-17  7:21                         ` Ingo Molnar
2009-09-10  7:43 ` [updated] BFS vs. mainline scheduler benchmarks and measurements Ingo Molnar
2009-09-14  9:46 ` Phoronix CFS vs BFS bencharks Nikos Chantziaras
2009-09-14 11:35   ` Mike Galbraith
     [not found]     ` <f42384a10909140727k463ff460q3859892dcb79bcc5@mail.gmail.com>
2009-09-14 15:32       ` Mike Galbraith
2009-09-14 19:14         ` Marcin Letyns
2009-09-14 20:49           ` Willy Tarreau
2009-09-15  8:37             ` Mike Galbraith
2009-09-10 21:17 BFS vs. mainline scheduler benchmarks and measurements Martin Steigerwald
2009-09-11 10:10 Mat
2009-09-11 18:33 Volker Armin Hemmann
2009-09-12  7:37 ` Nikos Chantziaras
2009-09-12  7:51   ` Arjan van de Ven
2009-09-12  8:27   ` Volker Armin Hemmann
2009-09-12  9:03     ` Nikos Chantziaras
2009-09-12  9:34       ` Volker Armin Hemmann

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20090907182629.GA3484@elte.hu \
    --to=mingo@elte.hu \
    --cc=a.p.zijlstra@chello.nl \
    --cc=efault@gmx.de \
    --cc=kernel@kolivas.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mb@bu3sch.de \
    --cc=nbd@openwrt.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

Powered by JetHome