From mboxrd@z Thu Jan 1 00:00:00 1970 Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754175AbeAJT3c (ORCPT + 1 other); Wed, 10 Jan 2018 14:29:32 -0500 Received: from wtarreau.pck.nerim.net ([62.212.114.60]:39484 "EHLO 1wt.eu" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754139AbeAJT30 (ORCPT ); Wed, 10 Jan 2018 14:29:26 -0500 From: Willy Tarreau To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Willy Tarreau , Andy Lutomirski , Borislav Petkov , Brian Gerst , Dave Hansen , Ingo Molnar , Linus Torvalds , Peter Zijlstra , Thomas Gleixner , Josh Poimboeuf , "H. Peter Anvin" , Kees Cook Subject: [RFC PATCH v3 8/8] x86/entry/pti: don't switch PGD when TIF_DISABLE_PTI_NOW is set Date: Wed, 10 Jan 2018 20:28:20 +0100 Message-Id: <1515612500-14505-9-git-send-email-w@1wt.eu> X-Mailer: git-send-email 2.8.0.rc2.1.gbe9624a In-Reply-To: <1515612500-14505-1-git-send-email-w@1wt.eu> References: <1515612500-14505-1-git-send-email-w@1wt.eu> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Return-Path: When a syscall returns to userspace with TIF_DISABLE_PTI_NOW set on the task, it means it is configured to disable page table isolation (PTI). In this case, returns from kernel to user will not switch the CR3, leaving it to the kernel one which already maps both user and kernel pages. This avoids a TLB flush, and saves another one on next entry. Thanks to these changes, haproxy running under KVM went back from 12700 conn/s (without PCID) or 19700 (with PCID) to 23100 once loaded after calling prctl(), indicating that PTI has no measurable impact on this workload. Signed-off-by: Willy Tarreau Cc: Andy Lutomirski Cc: Borislav Petkov Cc: Brian Gerst Cc: Dave Hansen Cc: Ingo Molnar Cc: Linus Torvalds Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Josh Poimboeuf Cc: "H. Peter Anvin" Cc: Kees Cook v3: - switched back to using a task flag v2: - use pti_disable instead of task flag --- arch/x86/entry/calling.h | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/arch/x86/entry/calling.h b/arch/x86/entry/calling.h index 19c6790..563478d 100644 --- a/arch/x86/entry/calling.h +++ b/arch/x86/entry/calling.h @@ -1,5 +1,6 @@ /* SPDX-License-Identifier: GPL-2.0 */ #include +#include #include #include #include @@ -229,6 +230,12 @@ .macro SWITCH_TO_USER_CR3_NOSTACK scratch_reg:req scratch_reg2:req ALTERNATIVE "jmp .Lend_\@", "", X86_FEATURE_PTI + + /* This task may be exempt from PTI */ + movq PER_CPU_VAR(current_task), \scratch_reg + btq $TIF_DISABLE_PTI_NOW, TASK_TI_flags(\scratch_reg) + jc .Lend_\@ + mov %cr3, \scratch_reg ALTERNATIVE "jmp .Lwrcr3_\@", "", X86_FEATURE_PCID -- 1.7.12.1