Re: [PATCH] x86/entry: re-enable interrupts before exiting

[Tycho Andersen <tycho@tycho.ws>]

On Fri, Apr 05, 2019 at 10:58:33AM -0500, Josh Poimboeuf wrote:
> On Fri, Apr 05, 2019 at 09:35:24AM -0600, Tycho Andersen wrote:
> > If the kernel oopses in an interrupt, nothing re-enables interrupts:
> > 
> > Aug 23 19:30:27 xpfo kernel: [   38.302714] BUG: sleeping function called from invalid context at
> > ./include/linux/percpu-rwsem.h:33
> > Aug 23 19:30:27 xpfo kernel: [   38.303837] in_atomic(): 0, irqs_disabled(): 1, pid: 1970, name:
> > lkdtm_xpfo_test
> > Aug 23 19:30:27 xpfo kernel: [   38.304758] CPU: 3 PID: 1970 Comm: lkdtm_xpfo_test Tainted: G      D
> > 4.13.0-rc5+ #228
> > Aug 23 19:30:27 xpfo kernel: [   38.305813] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
> > 1.10.1-1ubuntu1 04/01/2014
> > Aug 23 19:30:27 xpfo kernel: [   38.306926] Call Trace:
> > Aug 23 19:30:27 xpfo kernel: [   38.307243]  dump_stack+0x63/0x8b
> > Aug 23 19:30:27 xpfo kernel: [   38.307665]  ___might_sleep+0xec/0x110
> > Aug 23 19:30:27 xpfo kernel: [   38.308139]  __might_sleep+0x45/0x80
> > Aug 23 19:30:27 xpfo kernel: [   38.308593]  exit_signals+0x21/0x1c0
> > Aug 23 19:30:27 xpfo kernel: [   38.309046]  ? blocking_notifier_call_chain+0x11/0x20
> > Aug 23 19:30:27 xpfo kernel: [   38.309677]  do_exit+0x98/0xbf0
> > Aug 23 19:30:27 xpfo kernel: [   38.310078]  ? smp_reader+0x27/0x40 [lkdtm]
> > Aug 23 19:30:27 xpfo kernel: [   38.310604]  ? kthread+0x10f/0x150
> > Aug 23 19:30:27 xpfo kernel: [   38.311045]  ? read_user_with_flags+0x60/0x60 [lkdtm]
> > Aug 23 19:30:27 xpfo kernel: [   38.311680]  rewind_stack_do_exit+0x17/0x20
> > 
> > do_exit() expects to be called in a well-defined environment, so let's
> > re-enable interrupts after unwinding the stack, in case they were disabled.
> > 
> > Note that if any spinlocks are held, etc. we'll also get the above warning,
> > so this isn't a silver bullet. So, let's add a C helper in case someone
> > wants to add fancier lock busting or if we've forgotten to unwind something
> > else.
> > 
> > I've had to add back in the hack that Josh removed in 8c1f75587a18
> > ("x86/entry/64: Add unwind hint annotations") with the loop after the call,
> > because for whatever reason without that I get a warning:
> > 
> >   AS      arch/x86/entry/entry_64.o
> > arch/x86/entry/entry_64.o: warning: objtool: .entry.text: unexpected end of section
> > 
> > It seems to actually work fine for me though, since the new helper is also
> > __noreturn. Perhaps there's a better way to do this?
> 
> Unfortunately, objtool doesn't have a way to detect noreturn functions
> in other objects, so they're hard-coded in a list.  You can add
> __finish_rewind_stack_do_exit to the global_noreturns array in
> tools/objtool/check.c.

Awesome, this is the magic I was missing. Thanks!

Tycho