From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S935367AbcCJLCv (ORCPT ); Thu, 10 Mar 2016 06:02:51 -0500 Received: from torg.zytor.com ([198.137.202.12]:37984 "EHLO terminus.zytor.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S932236AbcCJLCm (ORCPT ); Thu, 10 Mar 2016 06:02:42 -0500 Date: Thu, 10 Mar 2016 03:01:29 -0800 From: tip-bot for Andy Lutomirski Message-ID: Cc: andrew.cooper3@citrix.com, tglx@linutronix.de, brgerst@gmail.com, peterz@infradead.org, hpa@zytor.com, torvalds@linux-foundation.org, bp@alien8.de, luto@amacapital.net, linux-kernel@vger.kernel.org, dvlasenk@redhat.com, mingo@kernel.org, luto@kernel.org, oleg@redhat.com Reply-To: luto@amacapital.net, dvlasenk@redhat.com, linux-kernel@vger.kernel.org, luto@kernel.org, oleg@redhat.com, mingo@kernel.org, peterz@infradead.org, brgerst@gmail.com, hpa@zytor.com, andrew.cooper3@citrix.com, tglx@linutronix.de, torvalds@linux-foundation.org, bp@alien8.de In-Reply-To: <07daef3d44bd1ed62a2c866e143e8df64edb40ee.1457578375.git.luto@kernel.org> References: <07daef3d44bd1ed62a2c866e143e8df64edb40ee.1457578375.git.luto@kernel.org> To: linux-tip-commits@vger.kernel.org Subject: [tip:x86/asm] x86/entry/32: Filter NT and speed up AC filtering in SYSENTER Git-Commit-ID: 67f590e8d4d718d9bd377b39223f7f69678d6a10 X-Mailer: tip-git-log-daemon Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 Content-Disposition: inline Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Commit-ID: 67f590e8d4d718d9bd377b39223f7f69678d6a10 Gitweb: http://git.kernel.org/tip/67f590e8d4d718d9bd377b39223f7f69678d6a10 Author: Andy Lutomirski AuthorDate: Wed, 9 Mar 2016 19:00:26 -0800 Committer: Ingo Molnar CommitDate: Thu, 10 Mar 2016 09:48:12 +0100 x86/entry/32: Filter NT and speed up AC filtering in SYSENTER This makes the 32-bit code work just like the 64-bit code. It should speed up syscalls on 32-bit kernels on Skylake by something like 20 cycles (by analogy to the 64-bit compat case). It also cleans up NT just like we do for the 64-bit case. Signed-off-by: Andy Lutomirski Cc: Andrew Cooper Cc: Andy Lutomirski Cc: Borislav Petkov Cc: Brian Gerst Cc: Denys Vlasenko Cc: H. Peter Anvin Cc: Linus Torvalds Cc: Oleg Nesterov Cc: Peter Zijlstra Cc: Thomas Gleixner Link: http://lkml.kernel.org/r/07daef3d44bd1ed62a2c866e143e8df64edb40ee.1457578375.git.luto@kernel.org Signed-off-by: Ingo Molnar --- arch/x86/entry/entry_32.S | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/arch/x86/entry/entry_32.S b/arch/x86/entry/entry_32.S index e130272..8daa812 100644 --- a/arch/x86/entry/entry_32.S +++ b/arch/x86/entry/entry_32.S @@ -294,7 +294,6 @@ sysenter_past_esp: pushl $__USER_DS /* pt_regs->ss */ pushl %ebp /* pt_regs->sp (stashed in bp) */ pushfl /* pt_regs->flags (except IF = 0) */ - ASM_CLAC /* Clear AC after saving FLAGS */ orl $X86_EFLAGS_IF, (%esp) /* Fix IF */ pushl $__USER_CS /* pt_regs->cs */ pushl $0 /* pt_regs->ip = 0 (placeholder) */ @@ -302,6 +301,23 @@ sysenter_past_esp: SAVE_ALL pt_regs_ax=$-ENOSYS /* save rest */ /* + * SYSENTER doesn't filter flags, so we need to clear NT and AC + * ourselves. To save a few cycles, we can check whether + * either was set instead of doing an unconditional popfq. + * This needs to happen before enabling interrupts so that + * we don't get preempted with NT set. + * + * NB.: .Lsysenter_fix_flags is a label with the code under it moved + * out-of-line as an optimization: NT is unlikely to be set in the + * majority of the cases and instead of polluting the I$ unnecessarily, + * we're keeping that code behind a branch which will predict as + * not-taken and therefore its instructions won't be fetched. + */ + testl $X86_EFLAGS_NT|X86_EFLAGS_AC, PT_EFLAGS(%esp) + jnz .Lsysenter_fix_flags +.Lsysenter_flags_fixed: + + /* * User mode is traced as though IRQs are on, and SYSENTER * turned them off. */ @@ -339,6 +355,11 @@ sysenter_past_esp: .popsection _ASM_EXTABLE(1b, 2b) PTGS_TO_GS_EX + +.Lsysenter_fix_flags: + pushl $X86_EFLAGS_FIXED + popfl + jmp .Lsysenter_flags_fixed ENDPROC(entry_SYSENTER_32) # system call handler stub