mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Roman Pen <roman.penyaev@profitbricks.com>
To: unlisted-recipients:; (no To-header on input)
Cc: Roman Pen <roman.penyaev@profitbricks.com>,
	Andy Lutomirski <luto@kernel.org>,
	Josh Poimboeuf <jpoimboe@redhat.com>,
	Borislav Petkov <bp@alien8.de>, Brian Gerst <brgerst@gmail.com>,
	Denys Vlasenko <dvlasenk@redhat.com>,
	"H . Peter Anvin" <hpa@zytor.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Tejun Heo <tj@kernel.org>,
	linux-kernel@vger.kernel.org
Subject: [PATCH 2/2] sched: do not call workqueue sleep hook if task is already dead
Date: Wed, 21 Sep 2016 17:43:50 +0200	[thread overview]
Message-ID: <20160921154350.13128-2-roman.penyaev@profitbricks.com> (raw)
In-Reply-To: <20160921154350.13128-1-roman.penyaev@profitbricks.com>

If panic_on_oops is not set and oops happens inside workqueue kthread,
kernel kills this kthread.  Current patch fixes recursive GPF which
happens in that case with the following stack:

  [<ffffffff81397f75>] dump_stack+0x68/0x93
  [<ffffffff8106954b>] ? do_exit+0x7ab/0xc10
  [<ffffffff8108fd73>] __schedule_bug+0x83/0xe0
  [<ffffffff81716d5a>] __schedule+0x7ea/0xba0
  [<ffffffff810c864f>] ? vprintk_default+0x1f/0x30
  [<ffffffff8116a63c>] ? printk+0x48/0x50
  [<ffffffff81717150>] schedule+0x40/0x90
  [<ffffffff8106976a>] do_exit+0x9ca/0xc10
  [<ffffffff810c8e3d>] ? kmsg_dump+0x11d/0x190
  [<ffffffff810c8d37>] ? kmsg_dump+0x17/0x190
  [<ffffffff81021ee9>] oops_end+0x99/0xd0
  [<ffffffff81052da5>] no_context+0x185/0x3e0
  [<ffffffff81053083>] __bad_area_nosemaphore+0x83/0x1c0
  [<ffffffff810c820e>] ? vprintk_emit+0x25e/0x530
  [<ffffffff810531d4>] bad_area_nosemaphore+0x14/0x20
  [<ffffffff8105355c>] __do_page_fault+0xac/0x570
  [<ffffffff810c66fe>] ? console_trylock+0x1e/0xe0
  [<ffffffff81002036>] ? trace_hardirqs_off_thunk+0x1a/0x1c
  [<ffffffff81053a2c>] do_page_fault+0xc/0x10
  [<ffffffff8171f812>] page_fault+0x22/0x30
  [<ffffffff81089bc3>] ? kthread_data+0x33/0x40
  [<ffffffff8108427e>] ? wq_worker_sleeping+0xe/0x80
  [<ffffffff817169eb>] __schedule+0x47b/0xba0
  [<ffffffff81717150>] schedule+0x40/0x90
  [<ffffffff8106957d>] do_exit+0x7dd/0xc10
  [<ffffffff81021ee9>] oops_end+0x99/0xd0

The root cause is that zeroed task->vfork_done member is accessed from
wq_worker_sleeping() hook.  The zeroing out happens on the following
path:

   oops_end()
   do_exit()
   exit_mm()
   mm_release()
   complete_vfork_done()

In order to fix a bug dead tasks must be ignored.

Signed-off-by: Roman Pen <roman.penyaev@profitbricks.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: linux-kernel@vger.kernel.org
---
 kernel/sched/core.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 2c303e7..50772e5 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3380,8 +3380,22 @@ static void __sched notrace __schedule(bool preempt)
 			 * If a worker went to sleep, notify and ask workqueue
 			 * whether it wants to wake up a task to maintain
 			 * concurrency.
+			 *
+			 * Also the following stack is possible:
+			 *    oops_end()
+			 *    do_exit()
+			 *    schedule()
+			 *
+			 * If panic_on_oops is not set and oops happens on
+			 * a workqueue execution path, thread will be killed.
+			 * That is definitly sad, but not to make the situation
+			 * even worse we have to ignore dead tasks in order not
+			 * to step on zeroed out members (e.g. t->vfork_done is
+			 * already NULL on that path, since we were called by
+			 * do_exit()))
 			 */
-			if (prev->flags & PF_WQ_WORKER) {
+			if (prev->flags & PF_WQ_WORKER &&
+			    prev->state != TASK_DEAD) {
 				struct task_struct *to_wakeup;
 
 				to_wakeup = wq_worker_sleeping(prev);
-- 
2.9.3

  reply	other threads:[~2016-09-21 15:49 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-09-21 15:43 [PATCH 1/2] x86/dumpstack: on oops do not rewind stack for kthread Roman Pen
2016-09-21 15:43 ` Roman Pen [this message]
2016-10-20 23:08   ` [PATCH 2/2] sched: do not call workqueue sleep hook if task is already dead Andy Lutomirski
2016-10-21 15:47     ` Oleg Nesterov
2016-10-24 16:01       ` Roman Penyaev
2016-10-21  5:39   ` Peter Zijlstra
2016-10-20 23:07 ` [PATCH 1/2] x86/dumpstack: on oops do not rewind stack for kthread Andy Lutomirski
2016-10-21  5:56   ` Peter Zijlstra
2016-10-21  8:05     ` Thomas Gleixner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20160921154350.13128-2-roman.penyaev@profitbricks.com \
    --to=roman.penyaev@profitbricks.com \
    --cc=bp@alien8.de \
    --cc=brgerst@gmail.com \
    --cc=dvlasenk@redhat.com \
    --cc=hpa@zytor.com \
    --cc=jpoimboe@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=luto@kernel.org \
    --cc=mingo@redhat.com \
    --cc=peterz@infradead.org \
    --cc=tglx@linutronix.de \
    --cc=tj@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®