From: Vitaly Kuznetsov <vkuznets@redhat.com>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: HATAYAMA Daisuke <d.hatayama@jp.fujitsu.com>,
Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>,
Jiri Kosina <jkosina@suse.cz>, Baoquan He <bhe@redhat.com>,
Prarit Bhargava <prarit@redhat.com>,
Xie XiuQi <xiexiuqi@huawei.com>,
Seth Jennings <sjenning@redhat.com>,
linux-kernel@vger.kernel.org,
"K. Y. Srinivasan" <kys@microsoft.com>
Subject: [PATCH] panic: release stale console lock to always get the logbuf printed out
Date: Wed, 7 Oct 2015 19:02:22 +0200 [thread overview]
Message-ID: <1444237342-13282-1-git-send-email-vkuznets@redhat.com> (raw)
In some cases we may end up killing the CPU holding the console lock
while still having valuable data in logbuf. E.g. I'm observing the
following:
- A crash is happening on one CPU and console_unlock() is being called on
some other.
- console_unlock() tries to print out the buffer before releasing the lock
and on slow console it takes time.
- in the meanwhile crashing CPU does lots of printk()-s with valuable data
(which go to the logbuf) and sends IPIs to all other CPUs.
- console_unlock() finishes printing previous chunk and enables interrupts
before trying to print out the rest, the CPU catches the IPI and never
releases console lock.
This is not the only possible case: in VT/fb subsystems we have many other
console_lock()/console_unlock() users. Non-masked interrupts (or receiving
NMI in case of extreme slowness) will have the same result. Getting the
whole console buffer printed out on crash should be top priority.
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
kernel/panic.c | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/kernel/panic.c b/kernel/panic.c
index 04e91ff..f94525f 100644
--- a/kernel/panic.c
+++ b/kernel/panic.c
@@ -23,6 +23,7 @@
#include <linux/sysrq.h>
#include <linux/init.h>
#include <linux/nmi.h>
+#include <linux/console.h>
#define PANIC_TIMER_STEP 100
#define PANIC_BLINK_SPD 18
@@ -147,6 +148,15 @@ void panic(const char *fmt, ...)
bust_spinlocks(0);
+ /*
+ * We may have ended up killing the CPU holding the lock and still have
+ * some valuable data in console buffer. Try to acquire the lock and
+ * release it regardless of the result. The release will also print the
+ * buffers out.
+ */
+ console_trylock();
+ console_unlock();
+
if (!panic_blink)
panic_blink = no_blink;
--
2.4.3
next reply other threads:[~2015-10-07 17:02 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-10-07 17:02 Vitaly Kuznetsov [this message]
2015-10-07 22:34 ` Andrew Morton
2015-10-08 9:01 ` Jan Kara
2015-10-08 10:03 ` Vitaly Kuznetsov
2015-10-08 20:56 ` Andrew Morton
2015-10-09 10:10 ` Vitaly Kuznetsov
2015-10-09 12:44 ` Vitaly Kuznetsov
2015-10-12 3:02 ` kbuild test robot
2015-10-08 9:51 ` Vitaly Kuznetsov
2015-10-08 20:54 ` Andrew Morton
2015-10-09 10:09 ` Vitaly Kuznetsov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1444237342-13282-1-git-send-email-vkuznets@redhat.com \
--to=vkuznets@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=bhe@redhat.com \
--cc=d.hatayama@jp.fujitsu.com \
--cc=jkosina@suse.cz \
--cc=kys@microsoft.com \
--cc=linux-kernel@vger.kernel.org \
--cc=masami.hiramatsu.pt@hitachi.com \
--cc=prarit@redhat.com \
--cc=sjenning@redhat.com \
--cc=xiexiuqi@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Powered by JetHome