From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751213AbbDYShg (ORCPT ); Sat, 25 Apr 2015 14:37:36 -0400 Received: from mail-ie0-f181.google.com ([209.85.223.181]:35572 "EHLO mail-ie0-f181.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750712AbbDYShd (ORCPT ); Sat, 25 Apr 2015 14:37:33 -0400 Message-ID: <1429987050.22254.187.camel@edumazet-glaptop2.roam.corp.google.com> Subject: Re: [PATCH 1/2] timer: Avoid waking up an idle-core by migrate running timer From: Eric Dumazet To: Thomas Gleixner Cc: Peter Zijlstra , viresh kumar , Ingo Molnar , linaro-kernel@lists.linaro.org, LKML , Steven Miao , shashim@codeaurora.org, Frederic Weisbecker , "Paul E. McKenney" , cl@kernel.org Date: Sat, 25 Apr 2015 11:37:30 -0700 In-Reply-To: References: <80182e47a7103608d2ddab7f62c0c3dffc99fdcc.1427782893.git.viresh.kumar@linaro.org> <5530C086.2020700@linaro.org> <1429653295.18561.16.camel@edumazet-glaptop2.roam.corp.google.com> <20150422152940.GC3007@worktop.Skamania.guest> <1429718577.18561.103.camel@edumazet-glaptop2.roam.corp.google.com> <1429732743.18561.134.camel@edumazet-glaptop2.roam.corp.google.com> <1429772251.22254.35.camel@edumazet-glaptop2.roam.corp.google.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.10.4-0ubuntu2 Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, 2015-04-23 at 14:45 +0200, Thomas Gleixner wrote: > You definitely have a point from the high throughput networking > perspective. > > Though in a power optimizing scenario with minimal network traffic > this might be the wrong decision. We have to gather data from the > power maniacs whether this matters or not. The FULL_NO_HZ camp might > be pretty unhappy about the above. Sure, I understand. To make this clear, here the profile on a moderately loaded TCP server, pushing ~20Gbits of data. Most of TCP output is ACK clock driven (thus from softirq context). (using regular sendmsg() system calls, that why the get_nohz_timer_target() is 'only' second in the profile, but add the find_next_bit() to it and this is very close being at first position) PerfTop: 4712 irqs/sec kernel:96.7% exact: 0.0% [4000Hz cycles], (all, 72 CPUs) ----------------------------------------------------------------------------------------- 10.16% [kernel] [k] copy_user_enhanced_fast_string 5.66% [kernel] [k] get_nohz_timer_target 5.59% [kernel] [k] _raw_spin_lock 2.53% [kernel] [k] __netif_receive_skb_core 2.27% [kernel] [k] find_next_bit 1.90% [kernel] [k] tcp_ack Maybe a reasonable heuristic would be to change /proc/sys/kernel/timer_migration default to 0 on hosts with more than 32 cpus. profile with timer_migration = 0 PerfTop: 3656 irqs/sec kernel:94.3% exact: 0.0% [4000Hz cycles], (all, 72 CPUs) ----------------------------------------------------------------------------------------- 13.95% [kernel] [k] copy_user_enhanced_fast_string 4.65% [kernel] [k] _raw_spin_lock 2.57% [kernel] [k] __netif_receive_skb_core 2.33% [kernel] [k] tcp_ack Thanks.