From: Hoeun Ryu <hoeun.ryu@lge.com.com>
To: Petr Mladek <pmladek@suse.com>,
Sergey Senozhatsky <sergey.senozhatsky@gmail.com>,
Steven Rostedt <rostedt@goodmis.org>
Cc: Hoeun Ryu <hoeun.ryu@lge.com>, linux-kernel@vger.kernel.org
Subject: [PATCH v2] printk: make printk_safe_flush safe in NMI context by skipping flushing
Date: Mon, 4 Jun 2018 07:37:34 +0900 [thread overview]
Message-ID: <1528065471-22830-1-git-send-email-hoeun.ryu@lge.com.com> (raw)
From: Hoeun Ryu <hoeun.ryu@lge.com>
Make printk_safe_flush() safe in NMI context.
nmi_trigger_cpumask_backtrace() can be called in NMI context. For example the
function is called in watchdog_overflow_callback() if the flag of hardlockup
backtrace (sysctl_hardlockup_all_cpu_backtrace) is true and
watchdog_overflow_callback() function is called in NMI context on some
architectures.
Calling printk_safe_flush() in nmi_trigger_cpumask_backtrace() eventually tries
to lock logbuf_lock in vprintk_emit() that might be already be part
of another non-nmi context on the same CPU or a soft- or hard-lockup on another
CPU. The example of deadlock can be
CPU0
local_irq_save();
for (;;)
req = blk_peek_request(q);
if (unlikely(!scsi_device_online(sdev)))
printk()
vprintk_emit()
console_unlock()
logbuf_lock_irqsave()
slow-serial-console-write() // close to watchdog threshold
watchdog_overflow_callback()
trigger_allbutself_cpu_backtrace()
printk_safe_flush()
vprintk_emit()
logbuf_lock_irqsave()
^^^^ deadlock
and some other cases.
This patch prevents a deadlock in printk_safe_flush() in NMI context. It makes
sure that we continue and eventually call printk_safe_flush_on_panic() from panic()
that has better chances to succeed.
There is a risk that logbuf_lock was not part of a soft- or dead-lockup and we
might just loose the messages. But then there is a high chance that irq_work will
get called and the messages will get flushed the normal way.
Signed-off-by: Hoeun Ryu <hoeun.ryu@lge.com>
Suggested-by: Petr Mladek <pmladek@suse.com>
Suggested-by: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>
---
v2: fix comments in commit message and code. no change in code itself.
kernel/printk/printk_safe.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/kernel/printk/printk_safe.c b/kernel/printk/printk_safe.c
index 3e3c200..3b5c660 100644
--- a/kernel/printk/printk_safe.c
+++ b/kernel/printk/printk_safe.c
@@ -254,6 +254,17 @@ void printk_safe_flush(void)
{
int cpu;
+ /*
+ * Just avoid a deadlock here.
+ * It makes sure that we continue and eventually call
+ * printk_safe_flush_on_panic() from panic() that has better chances to succeed.
+ * There is a risk that logbuf_lock was not part of a soft- or dead-lockup and
+ * we might just loose the messages. But then there is a high chance that
+ * irq_work will get called and the messages will get flushed the normal way.
+ */
+ if (this_cpu_read(printk_context) & PRINTK_NMI_CONTEXT_MASK)
+ return;
+
for_each_possible_cpu(cpu) {
#ifdef CONFIG_PRINTK_NMI
__printk_safe_flush(&per_cpu(nmi_print_seq, cpu).work);
--
2.1.4
reply other threads:[~2018-06-03 22:38 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1528065471-22830-1-git-send-email-hoeun.ryu@lge.com.com \
--to=hoeun.ryu@lge.com.com \
--cc=hoeun.ryu@lge.com \
--cc=linux-kernel@vger.kernel.org \
--cc=pmladek@suse.com \
--cc=rostedt@goodmis.org \
--cc=sergey.senozhatsky@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®