From: Petr Mladek <pmladek@suse.com>
To: Bradley Morgan <include@grrlz.net>
Cc: akpm@linux-foundation.org, baoquan.he@linux.dev,
feng.tang@linux.alibaba.com, gregkh@linuxfoundation.org,
arnd@arndb.de, corbet@lwn.net, rdunlap@infradead.org,
gpiccoli@igalia.com, kexec@lists.infradead.org,
linux-kernel@vger.kernel.org
Subject: Re: [RFC PATCH 0/4] panic: a pre kdump notifier list for hypervisor upcalls
Date: Fri, 17 Jul 2026 14:56:15 +0200 [thread overview]
Message-ID: <alomb3-GRqMOouAD@pathway.suse.cz> (raw)
In-Reply-To: <20260711002253.1115-1-include@grrlz.net>
On Sat 2026-07-11 00:22:49, Bradley Morgan wrote:
> When a crash kernel is loaded, panic() jumps to it before the panic
> notifiers run, unless crash_kexec_post_notifiers is set. So the
> hypervisor or firmware never finds out the guest panicked. Hyper-V
> doesn't get the crash registers, gsmi drops its firmware log entry,
> pvpanic stays silent, SEV-SNP skips its firmware and IOMMU shutdown.
> Whatever's watching the machine, the host, the BMC, fleet, sees a
> clean reboot instead of a crash.
>
> The only way around it today is crash_kexec_post_notifiers, but that
> runs the whole legacy notifier list before the kdump. That list has
> slow callbacks in it, IPMI being the obvious one since it talks to a
> BMC, so turning it on slows down every crash dump on the box. Hyper-V
> turns it on anyway and eats the cost for one callback. SNP did the
> same.
IMHO, it is not about speed at all. The main criteria are:
+ what has to be done at which stage
+ reliability
I see that there are 4 situations where crash_kexec_post_notifiers is
set to true:
$> git grep "crash_kexec_post_notifiers = true"
arch/powerpc/kernel/fadump.c: crash_kexec_post_notifiers = true;
arch/x86/hyperv/hv_crash.c: crash_kexec_post_notifiers = true;
arch/x86/virt/svm/sev.c: crash_kexec_post_notifiers = true;
drivers/hv/hv_common.c: crash_kexec_post_notifiers = true;
IMHO, these point to the notifiers have to be called before
kdump because otherwise something goes wrong.
The rest are users who do not care. They have happily worked
as post-kdump notifiers for years.
> This adds a separate list that runs before the crash kexec no matter
> what. Callbacks on it have to follow a contract: no locks, no
> allocation, no sleeping,
This is required by any code called in panic(). It is not special
to the pre-kdump notifiers.
> other CPUs may still be running
Good question. My upderstanding is that people prefer when kdump
catches the system when all CPUs are still running. But it also
complicates any lockless solution in the notifiers.
> and it has
> to tolerate being entered again if the panic path itself panics.
The panic-in-panic is a dark corner for me. I believe that it might
happen but I have never met it. And any panic() code should do
its best to avoid it in the first place.
By other words, panic-in-panic is a corner case. We should not
focus on it too much.
> This is the minimum: the list, the hook in the panic path, one driver
> converted so there's a real user, and the MAINTAINERS entry. pvpanic
> is the first one. Its callback is a self contained upcall that already
> takes its lock as a trylock, so it fits the contract as is.
> The rest
> (Hyper-V, Xen, gsmi, SNP) come in a follow on series. gsmi needs a
> rework to a trylock because its old deadlock guard assumed the other
> CPUs were stopped, which isn't true on this list. Once SNP's notifier
> is on the new list, the crash_kexec_post_notifiers forcing in
> snp_rmptable_init() goes away and SNP gets the early crash kexec back.
> Hyper-V keeps its forcing for now because its kmsg dump pass also needs
> to run before kdump. The register report just doesn't depend on it
> anymore.
>
> IPMI stays out. Its panic handling assumes the other CPUs have been
> stopped, and poking a BMC before the crash kexec would slow down every
> kdump on any box with a BMC. It stays on the legacy list, where kdump
> skips it like before.
I am not sure if I got it correctly.
IMHO, the ultimate plan should be to remove the
"crash_kexec_post_notifiers" option. It is a black magic. Someone
has to set it because otherwise crashdump does not work. Others
disable it because it breaks or might break crashdump.
Alternative solution would be to move the potentially dangerous
notifiers to some optional list, aka, panic_extra_debug_notifiers.
> The legacy list, its position in the panic path and
> crash_kexec_post_notifiers itself are untouched, so the remaining
> registrants see zero change and nothing is renamed.
>
> This is on purpose. Piccoli's 2022 series tried to classify every
> panic notifier and the list split didn't go in. This does the one piece
> that fixes the kdump versus hypervisor conflict and stops.
I am fine with moving one notifier by one. But we first need
to make sure that the split, ordering, and naming makes sense.
The good thing about Piccoli's patchset was that we saw the whole
picture. And I believe that we need the whole picture to make
a reasonable move here.
> One alternative is splitting the legacy list by priority and calling the
> high priority half before the crash kexec.
I personally prefer two or more notifier lists. Different lists
make it clear that they are proceed at different stages. The names
might even help to decide which list is the right one. The priority
is much harder to maintain.
Best Regards,
Petr
prev parent reply other threads:[~2026-07-17 12:56 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-07-11 0:22 Bradley Morgan
2026-07-11 0:22 ` [RFC PATCH 1/4] panic: add a pre kdump notifier list Bradley Morgan
2026-07-17 13:06 ` Petr Mladek
2026-07-11 0:22 ` [RFC PATCH 2/4] panic: run the pre kdump list before crash kexec Bradley Morgan
2026-07-11 0:22 ` [RFC PATCH 3/4] misc/pvpanic: notify the host on the pre kdump list Bradley Morgan
2026-07-11 0:22 ` [RFC PATCH 4/4] MAINTAINERS: add an entry for the pre kdump notifier list Bradley Morgan
2026-07-17 12:56 ` Petr Mladek [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=alomb3-GRqMOouAD@pathway.suse.cz \
--to=pmladek@suse.com \
--cc=akpm@linux-foundation.org \
--cc=arnd@arndb.de \
--cc=baoquan.he@linux.dev \
--cc=corbet@lwn.net \
--cc=feng.tang@linux.alibaba.com \
--cc=gpiccoli@igalia.com \
--cc=gregkh@linuxfoundation.org \
--cc=include@grrlz.net \
--cc=kexec@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=rdunlap@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox