From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754260AbbG2OII (ORCPT ); Wed, 29 Jul 2015 10:08:08 -0400 Received: from mail.skyhub.de ([78.46.96.112]:38329 "EHLO mail.skyhub.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754242AbbG2OID (ORCPT ); Wed, 29 Jul 2015 10:08:03 -0400 Date: Wed, 29 Jul 2015 16:07:59 +0200 From: Borislav Petkov To: Peter Zijlstra Cc: linux-kernel@vger.kernel.org, mingo@kernel.org, jasonbaron0@gmail.com, luto@amacapital.net, tglx@linutronix.de, rostedt@goodmis.org, will.deacon@arm.com, liuj97@gmail.com, rabin@rab.in, ralf@linux-mips.org, ddaney@caviumnetworks.com, benh@kernel.crashing.org, michael@ellerman.id.au, heiko.carstens@de.ibm.com, davem@davemloft.net, vbabka@suse.cz Subject: Re: [PATCH -v2 8/8] x86, tsc: Employ static_branch_likely() Message-ID: <20150729140759.GB17367@nazgul.tnic> References: <20150728132055.203176565@infradead.org> <20150728132313.388604668@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20150728132313.388604668@infradead.org> User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Jul 28, 2015 at 03:21:03PM +0200, Peter Zijlstra wrote: > Because of the static_key restrictions we had to take an unconditional > jump for the most likely case, causing $I bloat. > > Rewrite to use the new primitives. > > Signed-off-by: Peter Zijlstra (Intel) > --- > arch/x86/kernel/tsc.c | 22 ++++++++++------------ > 1 file changed, 10 insertions(+), 12 deletions(-) > > --- a/arch/x86/kernel/tsc.c > +++ b/arch/x86/kernel/tsc.c > @@ -38,7 +38,7 @@ static int __read_mostly tsc_unstable; > erroneous rdtsc usage on !cpu_has_tsc processors */ > static int __read_mostly tsc_disabled = -1; > > -static struct static_key __use_tsc = STATIC_KEY_INIT; > +static DEFINE_STATIC_KEY_FALSE(__use_tsc); > > int tsc_clocksource_reliable; > > @@ -274,7 +274,12 @@ static void set_cyc2ns_scale(unsigned lo > */ > u64 native_sched_clock(void) > { > - u64 tsc_now; > + if (static_branch_likely(&__use_tsc)) { > + u64 tsc_now = rdtsc(); > + > + /* return the value in ns */ > + return cycles_2_ns(tsc_now); > + } Hallelujah, this asm finally looks good: native_sched_clock: pushq %rbp # movq %rsp, %rbp #, andq $-16, %rsp #, #APP # 36 "./arch/x86/include/asm/jump_label.h" 1 1:.byte 0xe9 .long .L121 - 2f # 2: .pushsection __jump_table, "aw" .balign 8 .quad 1b, .L121, __use_tsc+1 #, .popsection # 0 "" 2 # 124 "./arch/x86/include/asm/msr.h" 1 rdtsc # 0 "" 2 #NO_APP ... leave ret .L121: imulq $1000000, jiffies_64(%rip), %rdx #, jiffies_64, D.28480 movabsq $-4294667296000000, %rax #, tmp135 leave addq %rdx, %rax # D.28480, D.28480 ret -- Regards/Gruss, Boris. ECO tip #101: Trim your mails when you reply. --