From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754765AbbCaNtm (ORCPT ); Tue, 31 Mar 2015 09:49:42 -0400 Received: from smtprelay0200.hostedemail.com ([216.40.44.200]:35015 "EHLO smtprelay.hostedemail.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752974AbbCaNtg (ORCPT ); Tue, 31 Mar 2015 09:49:36 -0400 X-Session-Marker: 726F737465647440676F6F646D69732E6F7267 X-Spam-Summary: 2,0,0,,d41d8cd98f00b204,rostedt@goodmis.org,:::::::::::::::::::::::::,RULES_HIT:41:355:379:541:599:800:960:968:973:988:989:1260:1277:1311:1313:1314:1345:1359:1437:1515:1516:1518:1534:1542:1593:1594:1711:1730:1747:1777:1792:2197:2198:2199:2200:2393:2553:2559:2562:2904:3138:3139:3140:3141:3142:3354:3622:3653:3865:3866:3867:3868:3870:3871:3872:3873:3874:4250:5007:6117:6119:6120:6261:6691:6742:7875:7903:10004:10400:10848:10967:11026:11232:11473:11658:11914:12295:12296:12438:12517:12519:12740:13161:13229:14096:14097:21080,0,RBL:none,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Custom_rules:0:0:0 X-HE-Tag: wall97_707561e97e72d X-Filterd-Recvd-Size: 3372 Date: Tue, 31 Mar 2015 09:49:31 -0400 From: Steven Rostedt To: Denys Vlasenko Cc: Andy Lutomirski , Linus Torvalds , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , Oleg Nesterov , Frederic Weisbecker , Alexei Starovoitov , Will Drewry , Kees Cook , x86@kernel.org, linux-kernel@vger.kernel.org Subject: Re: [RFC PATCH] x86: optimize IRET returns to kernel Message-ID: <20150331094931.257a6920@gandalf.local.home> In-Reply-To: <1427805981-15085-1-git-send-email-dvlasenk@redhat.com> References: <1427805981-15085-1-git-send-email-dvlasenk@redhat.com> X-Mailer: Claws Mail 3.11.1 (GTK+ 2.24.25; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 31 Mar 2015 14:46:21 +0200 Denys Vlasenko wrote: > @@ -750,6 +750,53 @@ retint_kernel: > * The iretq could re-enable interrupts: > */ > TRACE_IRQS_IRETQ > + > + /* > + * Since we return to kernel, CS and SS do not need changing. > + * Only RSP, RIP and RFLAGS do. > + * We can use POPF + near RET, which is much faster. > + * The below code may seem excessive, but IRET is _very_ slow. > + * Hundreds of cycles. > + * > + * However, there is a complication. Interrupts in 64-bit mode > + * align stack to 16 bytes. This changes location > + * where we need to store EFLAGS and RIP: > + */ > +#if 0 > + testb $8, RSP(%rsp) Shouldn't this be: testb $0xf, RSP(%rsp) ? The stack should be word (8 bytes) aligned, but I never like to assume anything. As an interrupt can come in anywhere, and %rsp can be modified to anything in assembly, I wouldn't want some "hack" that performs a non word aligned rsp manipulation to suddenly break. It would be rather hard to debug. > + jnz 1f > +#else > + /* There is a complication #2: 64-bit mode has IST stacks */ > + leaq SIZEOF_PTREGS+8(%rsp), %rax > + cmpq %rax, RSP(%rsp) > + je 1f > + subq $8, %rax > + cmpq %rax, RSP(%rsp) > + jne restore_args /* probably IST stack, can't optimize */ > +#endif > + /* there is no padding above iret frame */ > + movq EFLAGS(%rsp), %rax > + movq RIP(%rsp), %rcx > + movq %rax, (SIZEOF_PTREGS-2*8)(%rsp) > + movq %rcx, (SIZEOF_PTREGS-1*8)(%rsp) > + CFI_REMEMBER_STATE > + RESTORE_C_REGS > + REMOVE_PT_GPREGS_FROM_STACK 4*8 /* remove all except last two words */ > + popfq_cfi > + retq BTW, have you made sure that this path has been hit? -- Steve > + CFI_RESTORE_STATE > +1: /* there are 8 bytes of padding above iret frame */ > + movq EFLAGS(%rsp), %rax > + movq RIP(%rsp), %rcx > + movq %rax, (SIZEOF_PTREGS-2*8 + 8)(%rsp) > + movq %rcx, (SIZEOF_PTREGS-1*8 + 8)(%rsp) > + CFI_REMEMBER_STATE > + RESTORE_C_REGS > + REMOVE_PT_GPREGS_FROM_STACK 4*8 + 8 > + popfq_cfi > + retq > + CFI_RESTORE_STATE > + > restore_args: > RESTORE_C_REGS > REMOVE_PT_GPREGS_FROM_STACK 8