From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932741AbeARPJb (ORCPT ); Thu, 18 Jan 2018 10:09:31 -0500 Received: from bombadil.infradead.org ([65.50.211.133]:40645 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754799AbeARO56 (ORCPT ); Thu, 18 Jan 2018 09:57:58 -0500 Message-Id: <20180118140153.138069275@infradead.org> User-Agent: quilt/0.63-1 Date: Thu, 18 Jan 2018 14:48:28 +0100 From: Peter Zijlstra From: Peter Zijlstra To: David Woodhouse , Thomas Gleixner , Josh Poimboeuf Cc: linux-kernel@vger.kernel.org, Dave Hansen , Ashok Raj , Tim Chen , Andy Lutomirski , Linus Torvalds , Greg KH , Andrea Arcangeli , Andi Kleen , Arjan Van De Ven , Dan Williams , Paolo Bonzini , Jun Nakajima , Asit Mallick , Jason Baron , Peter Zijlstra Subject: [PATCH 28/35] x86/idle: Control Indirect Branch Speculation in idle References: <20180118134800.711245485@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Disposition: inline; filename=x86-idle--Control-Indirect-Branch-Speculation-in-idle.patch Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Thomas Gleixner Indirect Branch Speculation (IBS) is controlled per physical core. If one thread disables it then it's disabled for the core. If a thread enters idle it makes sense to reenable IBS so the sibling thread can run with full speculation enabled in user space. This makes only sense in mwait_idle_with_hints() because mwait_idle() can serve an interrupt immediately before speculation can be stopped again. SKL which requires IBRS should use mwait_idle_with_hints() so this is a non issue and in the worst case a missed optimization. [peterz: fixed stop_indirect_branch_speculation placement] Originally-by: Tim Chen Signed-off-by: Thomas Gleixner Signed-off-by: Peter Zijlstra (Intel) --- arch/x86/include/asm/mwait.h | 14 ++++++++++++++ arch/x86/kernel/process.c | 14 ++++++++++++++ 2 files changed, 28 insertions(+) --- a/arch/x86/include/asm/mwait.h +++ b/arch/x86/include/asm/mwait.h @@ -6,6 +6,7 @@ #include #include +#include #define MWAIT_SUBSTATE_MASK 0xf #define MWAIT_CSTATE_MASK 0xf @@ -106,9 +107,22 @@ static inline void mwait_idle_with_hints mb(); } + /* + * Indirect Branch Speculation (IBS) is controlled per + * physical core. If one thread disables it, then it's + * disabled on all threads of the core. The kernel disables + * it on entry from user space. Reenable it on the thread + * which goes idle so the other thread has a chance to run + * with full speculation enabled in userspace. + */ + restart_indirect_branch_speculation(); __monitor((void *)¤t_thread_info()->flags, 0, 0); if (!need_resched()) __mwait(eax, ecx); + /* + * Stop IBS again to protect kernel execution. + */ + stop_indirect_branch_speculation(); } current_clr_polling(); } --- a/arch/x86/kernel/process.c +++ b/arch/x86/kernel/process.c @@ -461,6 +461,20 @@ static __cpuidle void mwait_idle(void) mb(); /* quirk */ } + /* + * Indirect Branch Speculation (IBS) is controlled per + * physical core. If one thread disables it, then it's + * disabled on all threads of the core. The kernel disables + * it on entry from user space. For __sti_mwait() it's + * wrong to reenable it because an interrupt can be served + * before speculation can be stopped again. + * + * To plug that hole the interrupt entry code would need to + * save current state and restore. Not worth the trouble as + * SKL should not use mwait_idle(). It should use + * mwait_idle_with_hints() which can do speculation control + * safely. + */ __monitor((void *)¤t_thread_info()->flags, 0, 0); if (!need_resched()) __sti_mwait(0, 0);