mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Steven Rostedt <rostedt@goodmis.org>
To: Andy Lutomirski <luto@amacapital.net>
Cc: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Ingo Molnar <mingo@kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Borislav Petkov <bp@alien8.de>, Andi Kleen <andi@firstfloor.org>
Subject: Re: [RFC] x86_64: A real proposal for iret-less return to kernel
Date: Tue, 20 May 2014 22:27:07 -0400	[thread overview]
Message-ID: <1400639227.9759.21.camel@pippen.local.home> (raw)
In-Reply-To: <CALCETrXudJ8BkNF_M-r4O40XLN+PnZ5TOZw0P7N4kqo3qngzyg@mail.gmail.com>

On Tue, 2014-05-20 at 17:53 -0700, Andy Lutomirski wrote:
> Here's a real proposal for iret-less return.  If this is correct, then
> NMIs will never nest, which will probably delete a lot more scariness
> than is added by the code I'm describing.

Perhaps we can add this for one window release before we rip out the NMI
nesting code. Perhaps we can add a BUG() if we detect a NMI nest?

> 
> The rest of this email is valid markdown :)  If I end up implementing
> this, this text will go straight into Documentation/x86/x86_64.
> 
> tl;dr: The only particularly tricky cases are exit from #DB, #BP, and
> #MC.  I think they're not so bad, though.
> 
> FWIW, if there's a way to read the NMI masking bit, this would be a
> lot simpler.  I don't know of any way to do that, though.

Is there such a thing on all x86?

> 
> `IRET`-less return
> ==================
> 
> There are at least two ways that we can return from a trap entry:
> `IRET` and `RET`.  They have a few important differences.
> 
>   * `IRET` is very slow on all current (2014) CPUs -- it seems to
>     take hundreds of cycles.  `RET` is fast.

s/fast/faster/ or /fast/much faster/

> 
>   * `IRET` unconditionally unmasks NMIs.  `RET` never unmasks NMIs.
> 
>   * `IRET` can change `CS`, `RSP`, `SS`, `RIP`, and `RFLAGS`
>     atomically.  `RET` can't; it requires a return address on the
>     stack, and it can't apply anything other than a small offset to
>     the stack pointer.  It can, in theory, change `CS`, but this
>     seems unlikely to be helpful.
> 
> Times when we must use `IRET`
> =============================
> 
>   * If we're returning to a different `CS` (i.e. if firmware is
>     doing something funny or if we're returning to userspace), then
>     `RET` won't help; we need to use `IRET` unless we're willing to
>     play fragile games with `SYSEXIT` or `SYSRET`.
> 
>   * If we are changing stacks, the we need to be extremely careful

s/the we/then we/

>     about using `RET`: using `RET` requires that we put the target
>     `RIP` on the target stack, so the target stack must be valid.
>     This means that we cannot use `RET` if, for example, a `SYSCALL`
>     just happened.
> 
>   * If we're returning from NMI, we `IRET` is mandatory: we need to

s/we/then/

>     unmask NMIs, and `IRET` is the only way to do that.
> 
> Note that, if `RFLAGS.IF` is set, then interrupts were enabled when
> we trapped, so `RET` is safe.

Is it? You mean if IF is set *and* we are in the kernel?

> 
> Times when we must use `RET`
> ============================
> 
> If there's an NMI on the stack, we must use `RET` until we're ready
> to re-enabled NMIs.

I'm a little confused by NMI on the stack. Do you mean NMI on the target
stack? If so, please state that.


> 
> Assumptions
> ===========
> 
>   * Neither the NMI, the MCE handler, nor anything that nests inside
>     them will ever change `CS` or run with an invalid stack.
> 
>   * Interrupts will never be enabled with an NMI on the stack

target stack?

> .
> 
>   * We explicitly do not assume that we can reliably determine
>     whether we were on user `GS` or kernel `GS` when a trap happens.
>     In current (3.15) kernels we can tell, but if we ever enable
>     `WRGSBASE` then we will lose that ability.
> 
>   * The IST interrupts are: #DB #BP #NM #DF #SS, and #MC.
> 
>   * We can add a per-cpu variable `nmi_mce_nest_count` that is nonzero
>     whenever an NMI or MCE is on the stack.  We'll increment it at the
>     very beginning of the NMI handler and clear it at the very end.
>     We will also increment it in `do_machine_check` before doing
>     anything that can cause an interrupt.  The result is that the only
>     interrupt that can happen with `nmi_mce_nest_count == 0` in NMI
>     context is an MCE at the beginning or end of the NMI handler.

Just note that this will probably be done in the C code, as NMI has
issues with gs being safe.

Also, should we call it "nmi" specifically. Perhaps
"ist_stack_nest_count", stating that the stack is ist to match
do_machine_check as well? Maybe that's not a good name either. Someone
else can come up with something that's a little more generic than NMI?

> 
> 
> The algorithm
> =============
> 
>   1. If the target `CS` is not the standard 64-bit kernel CPL0
>      selector, then never use `RET`.  This is safe: this will never
>      happen with an NMI on the stack.

target stack?

> 
>   2. If we are returning from a non-IST interrupt, then use `RET`.
>      Non-IST interrupts use the interrupted code's stack, so the
>      stack is always valid.
> 
>   3. If we are returning from #NM, then use `IRET`.
> 
>   4. If we are returning from #DF or #SS, then use `IRET`.  These
>      interrupts cannot occur inside an NMI, or, at the very least,
>      if they do happen, then they are not recoverable.
> 
>   5. If we are returning from #DB or #BP, then use `RET` if
>      `nmi_mce_nest_count != 0` and `IRET` otherwise.
> 
>   6. If we are returning from #MC, use `IRET`, unless the return address is
>      to the NMI entry or exit code, in which case we use `RET`.

Seems interesting.

-- Steve



  reply	other threads:[~2014-05-21  2:27 UTC|newest]

Thread overview: 68+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-05-21  0:53 Andy Lutomirski
2014-05-21  2:27 ` Steven Rostedt [this message]
2014-05-21  2:33   ` H. Peter Anvin
2014-05-21  2:39   ` Andy Lutomirski
2014-05-21  9:46     ` Borislav Petkov
2014-05-21 15:21       ` Andy Lutomirski
2014-05-21 16:30         ` Borislav Petkov
2014-05-21 17:52           ` Andy Lutomirski
2014-05-21 18:07             ` Borislav Petkov
2014-05-21 12:51     ` Jiri Kosina
2014-05-21 15:21       ` Andy Lutomirski
2014-05-21 16:33         ` Borislav Petkov
2014-05-21 21:25           ` Jiri Kosina
2014-05-21 21:35             ` Andy Lutomirski
2014-05-21 21:48               ` Borislav Petkov
2014-05-21 21:52                 ` Andy Lutomirski
2014-05-21 21:55                   ` Borislav Petkov
2014-05-21 21:59                     ` Jiri Kosina
2014-05-21 21:59                     ` Andy Lutomirski
2014-05-21 22:01                   ` Luck, Tony
2014-05-21 22:13                     ` Andy Lutomirski
2014-05-21 22:17                       ` Borislav Petkov
2014-05-21 22:20                         ` Andy Lutomirski
2014-05-21 22:36                           ` Borislav Petkov
2014-05-21 22:18                       ` Luck, Tony
2014-05-21 22:24                         ` Andy Lutomirski
2014-05-21 22:32                           ` Luck, Tony
2014-05-21 22:39                             ` Andy Lutomirski
2014-05-21 22:48                               ` Borislav Petkov
2014-05-21 22:52                                 ` Andy Lutomirski
2014-05-21 23:02                                   ` Borislav Petkov
2014-05-21 23:05                                 ` Luck, Tony
2014-05-21 23:07                                   ` Andy Lutomirski
2014-05-21 23:19                                     ` Luck, Tony
2014-05-21 23:30                                       ` Linus Torvalds
2014-05-21 23:40                                         ` Luck, Tony
2014-05-21 23:51                                         ` Borislav Petkov
2014-05-22  0:03                                           ` Linus Torvalds
2014-05-22  8:50                                             ` Borislav Petkov
2014-05-22  0:05                                           ` Andy Lutomirski
2014-05-21 21:37             ` Linus Torvalds
2014-05-21 21:43               ` Borislav Petkov
2014-05-21 21:45                 ` H. Peter Anvin
2014-05-21 21:47                   ` Andy Lutomirski
2014-05-21 21:54                     ` Borislav Petkov
2014-05-21 22:00                       ` H. Peter Anvin
2014-05-21 22:11                         ` Borislav Petkov
2014-05-21 22:13                           ` H. Peter Anvin
2014-05-21 22:21                             ` Borislav Petkov
2014-05-26 10:18                             ` [PATCH] x86, MCE: Flesh out when to panic comment Borislav Petkov
2014-05-26 10:51                               ` Jiri Kosina
2014-05-26 11:06                                 ` Borislav Petkov
2014-05-26 16:47                                   ` Andy Lutomirski
2014-05-26 17:51                                     ` Borislav Petkov
2014-05-26 17:59                                       ` Andy Lutomirski
2014-05-27 21:53                                   ` Luck, Tony
2014-05-27 22:24                                     ` Borislav Petkov
2014-05-27 22:33                                       ` Luck, Tony
2014-05-21 21:50                   ` [RFC] x86_64: A real proposal for iret-less return to kernel Jiri Kosina
2014-05-21 18:11 ` Andy Lutomirski
2014-05-21 22:36   ` H. Peter Anvin
2014-05-21 22:41     ` Andy Lutomirski
2014-05-21 23:03       ` H. Peter Anvin
2014-05-21 22:25 ` Andi Kleen
2014-05-21 22:32   ` Andy Lutomirski
2014-05-21 22:33   ` Linus Torvalds
2014-05-21 23:23     ` Andi Kleen
2014-05-21 23:34       ` Linus Torvalds

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1400639227.9759.21.camel@pippen.local.home \
    --to=rostedt@goodmis.org \
    --cc=andi@firstfloor.org \
    --cc=bp@alien8.de \
    --cc=hpa@zytor.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=luto@amacapital.net \
    --cc=mingo@kernel.org \
    --cc=tglx@linutronix.de \
    --cc=torvalds@linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®