From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932285AbdK2KgB (ORCPT ); Wed, 29 Nov 2017 05:36:01 -0500 Received: from bombadil.infradead.org ([65.50.211.133]:54951 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753461AbdK2Kf6 (ORCPT ); Wed, 29 Nov 2017 05:35:58 -0500 Message-Id: <20171129103512.918991807@infradead.org> User-Agent: quilt/0.63-1 Date: Wed, 29 Nov 2017 11:33:07 +0100 From: Peter Zijlstra To: linux-kernel@vger.kernel.org, Thomas Gleixner Cc: Dave Hansen , Andy Lutomirski , Ingo Molnar , Borislav Petkov , Brian Gerst , Denys Vlasenko , "H. Peter Anvin" , Josh Poimboeuf , Linus Torvalds , Peter Zijlstra , Rik van Riel , daniel.gruss@iaik.tugraz.at, hughd@google.com, keescook@google.com, linux-mm@kvack.org, michael.schwarz@iaik.tugraz.at, moritz.lipp@iaik.tugraz.at, richard.fellner@student.tugraz.at Subject: [PATCH 6/6] x86/mm/kaiser: Optimize __native_flush_tlb References: <20171129103301.131535445@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Disposition: inline; filename=peterz-kaiser-opt-tlb.patch Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Now that we have lazy user asid flushing, use that even if we have INVPCID. Even if INVPCID would not be slower than a flushing CR3 write (it is) this allows folding multiple user flushes. Signed-off-by: Peter Zijlstra (Intel) --- arch/x86/include/asm/tlbflush.h | 38 ++++++++++++++------------------------ 1 file changed, 14 insertions(+), 24 deletions(-) --- a/arch/x86/include/asm/tlbflush.h +++ b/arch/x86/include/asm/tlbflush.h @@ -377,33 +377,23 @@ static inline void flush_user_asid(u16 a static inline void __native_flush_tlb(void) { - if (!cpu_feature_enabled(X86_FEATURE_INVPCID)) { - flush_user_asid(this_cpu_read(cpu_tlbstate.loaded_mm_asid)); + flush_user_asid(this_cpu_read(cpu_tlbstate.loaded_mm_asid)); - /* - * If current->mm == NULL then we borrow a mm - * which may change during a task switch and - * therefore we must not be preempted while we - * write CR3 back: - */ - preempt_disable(); - native_write_cr3(__native_read_cr3()); - preempt_enable(); - /* - * Does not need tlb_flush_shared_nonglobals() - * since the CR3 write without PCIDs flushes all - * non-globals. - */ - return; - } /* - * We are no longer using globals with KAISER, so a - * "nonglobals" flush would work too. But, this is more - * conservative. - * - * Note, this works with CR4.PCIDE=0 or 1. + * If current->mm == NULL then we borrow a mm + * which may change during a task switch and + * therefore we must not be preempted while we + * write CR3 back: */ - invpcid_flush_all(); + preempt_disable(); + native_write_cr3(__native_read_cr3()); + preempt_enable(); + /* + * Does not need tlb_flush_shared_nonglobals() + * since the CR3 write without PCIDs flushes all + * non-globals. + */ + return; } static inline void __native_flush_tlb_global_irq_disabled(void)