[RFC PATCH 0/3] powerpc: initial support for Kexec HandOver (KHO)

[RFC PATCH 0/3] powerpc: initial support for Kexec HandOver (KHO)

[Sourabh Jain]

This series adds initial powerpc support for Kexec HandOver (KHO),
allowing state to be preserved across kexec on powerpc. Posting as RFC
because some of the patches are workarounds for issues I ran into, and
I'd like input on how best to address them.

Patch 1 is a small Kconfig fix needed to unblock patch 2. It won't be
needed once the ordering problem described under patch 2 is
addressed properly.

Patch 2 wires up ARCH_SUPPORTS_KEXEC_HANDOVER and adds the code to
retrieve the KHO FDT and scratch region from /chosen on boot. On
powerpc, crashkernel memory is reserved very early, from arch-specific
code, while KHO's scratch region is reserved later, from generic code
(kho_memory_init()). Crashkernel reservation always runs first, so by
the time KHO reserves its scratch region, not enough memory is left
and the reservation fails. Until scratch reservation can be moved
ahead of crashkernel reservation, this patch makes KHO and CRASH_DUMP
mutually exclusive on powerpc as a stopgap. The failure scenario and
the reasoning behind this restriction are described in the commit
message with an example.

Patch 3 excludes TRANSPARENT_HUGEPAGE and HUGETLB_PAGE from
ARCH_SUPPORTS_KEXEC_HANDOVER, since KHO's static_assert() on
SCRATCH_ALIGNMENT_BYTES >= CMA_MIN_ALIGNMENT_BYTES fails to build
when either is enabled -- pageblock_order isn't a compile-time
constant on powerpc in that configuration.

With all three patches applied, CONFIG_TEST_KEXEC_HANDOVER passes on
powerpc.

Open questions I'd appreciate feedback on:

 - How should KHO scratch reservation and powerpc's crashkernel
   reservation be ordered so both can be enabled together, given that
   scratch reservation currently happens in generic code
   (kho_memory_init())?

- Excluding THP/HUGETLB_PAGE is a workaround for the build failure.
  Looking for input on how to handle CMA_MIN_ALIGNMENT_BYTES when
  pageblock_order is non-constant.

Cc: Aditya Gupta <adityag@linux.ibm.com>
Cc: Alexander Graf <graf@amazon.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
Cc: Hari Bathini <hbathini@linux.ibm.com>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Mahesh Salgaonkar <mahesh@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Pasha Tatashin <pasha.tatashin@soleen.com>
Cc: Pratyush Yadav <pratyush@kernel.org>
Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Cc: Shivang Upadhyay <shivangu@linux.ibm.com>
Cc: Shrikanth Hegde <sshegde@linux.ibm.com>
Cc: Sourabh Jain <sourabhjain@linux.ibm.com>
Cc: kexec@lists.infradead.org
Cc: linux-kernel@vger.kernel.org

Sourabh Jain (3):
  kernel/liveupdate: make KEXEC_HANDOVER depend on KEXEC_FILE instead of
    selecting it
  powerpc: add support for Kexec HandOver (KHO)
  powerpc/kho: exclude THP and HUGETLB_PAGE

 arch/powerpc/Kconfig               |  6 ++++++
 arch/powerpc/kernel/setup-common.c | 33 ++++++++++++++++++++++++++++++
 kernel/liveupdate/Kconfig          |  2 +-
 3 files changed, 40 insertions(+), 1 deletion(-)

-- 
2.55.0

[RFC PATCH 1/3] kernel/liveupdate: make KEXEC_HANDOVER depend on KEXEC_FILE instead of selecting it

[Sourabh Jain]

An upcoming patch makes ARCH_SUPPORTS_KEXEC_HANDOVER depend on
CRASH_DUMP on powerpc:

    config ARCH_SUPPORTS_KEXEC_HANDOVER
    	def_bool y
    	depends on PPC64
    	depends on !CRASH_DUMP

That triggers a recursive dependency error:

    error: recursive dependency detected!
    	symbol KEXEC_HANDOVER depends on ARCH_SUPPORTS_KEXEC_HANDOVER
    	symbol ARCH_SUPPORTS_KEXEC_HANDOVER depends on CRASH_DUMP
    	symbol CRASH_DUMP depends on KEXEC_CORE
    	symbol KEXEC_CORE is selected by KEXEC_FILE
    	symbol KEXEC_FILE is selected by KEXEC_HANDOVER

KEXEC_HANDOVER and KEXEC_CORE end up depending on each other when
ARCH_SUPPORTS_KEXEC_HANDOVER depends on CRASH_DUMP. This creates a
recursive Kconfig dependency.

Change KEXEC_FILE from select to depends on to break this cycle.
This means KEXEC_FILE must be enabled explicitly instead of being
enabled automatically by KEXEC_HANDOVER.

Keep this change until the powerpc-specific KHO and CRASH_DUMP
dependency is resolved.

Cc: Aditya Gupta <adityag@linux.ibm.com>
Cc: Alexander Graf <graf@amazon.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
Cc: Hari Bathini <hbathini@linux.ibm.com>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Mahesh Salgaonkar <mahesh@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Pasha Tatashin <pasha.tatashin@soleen.com>
Cc: Pratyush Yadav <pratyush@kernel.org>
Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Cc: Shivang Upadhyay <shivangu@linux.ibm.com>
Cc: Shrikanth Hegde <sshegde@linux.ibm.com>
Cc: kexec@lists.infradead.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
---
 kernel/liveupdate/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/liveupdate/Kconfig b/kernel/liveupdate/Kconfig
index c13af38ba23a..22e317dc1e1a 100644
--- a/kernel/liveupdate/Kconfig
+++ b/kernel/liveupdate/Kconfig
@@ -6,7 +6,7 @@ config KEXEC_HANDOVER
 	bool "kexec handover"
 	depends on ARCH_SUPPORTS_KEXEC_HANDOVER && ARCH_SUPPORTS_KEXEC_FILE
 	select MEMBLOCK_KHO_SCRATCH
-	select KEXEC_FILE
+	depends on KEXEC_FILE
 	select LIBFDT
 	select CMA
 	help
-- 
2.55.0

[RFC PATCH 2/3] powerpc: add support for Kexec HandOver (KHO)

[Sourabh Jain]

Add the architecture bits needed to enable CONFIG_KEXEC_HANDOVER on
powerpc.

Set ARCH_SUPPORTS_KEXEC_HANDOVER for PPC64, following the existing
pattern used by ARCH_SUPPORTS_KEXEC and ARCH_SUPPORTS_KEXEC_FILE.

On the boot path, parse the "linux,kho-fdt" and "linux,kho-scratch"
properties from /chosen and pass them to kho_populate(). This lets a
kernel booted via KHO kexec recover the FDT and scratch region left
behind by the previous kernel. The call is placed early in
setup_arch(), before unflatten_device_tree().

Open issues:
============

This patch also adds "depends on !CRASH_DUMP" to
ARCH_SUPPORTS_KEXEC_HANDOVER. This is needed because of an ordering
conflict between crashkernel reservation and KHO scratch reservation
on powerpc.

Crashkernel memory is reserved very early in boot, from arch-specific
code: head.S -> early_setup() -> early_init_devtree() ->
arch_reserve_crashkernel() / fadump_reserve_mem(). KHO's scratch
region is reserved later, from generic code: start_kernel() ->
mm_core_init() -> kho_memory_init(). So on powerpc, crashkernel
memory is always reserved first.

This ordering causes a real failure. In the common case, crashkernel
reservation on powerpc starts at a 512M offset (the exact offset can
vary, but 512M is typical). So with crashkernel=3G, the reservation
occupies memory from 512M up to 3.5G -- roughly 75% of the entire low
4G area.

Since crashkernel reservation always happens first, that 3G is
already committed by the time kho_memory_init() runs. It then tries
to reserve a low scratch region sized at 200% of whatever is already
reserved below 4G. With ~75% of that 4G area already taken by
crashkernel memory, 200% of that easily exceeds the remaining space
-- and since the low scratch region is itself capped at 4G, there's
no room left to fit it. The reservation fails.

To work around this and get KHO working on powerpc, this patch:

1. Makes KHO usable on powerpc only when CRASH_DUMP is disabled.
2. Calls kho_populate() from setup_arch(), so it runs before
   kho_memory_init() reserves the scratch region.

The real fix would be to reserve the KHO scratch region before
crashkernel memory instead of after. But scratch reservation happens
in generic code (kho_memory_init(), called from mm_core_init()), so
this isn't something powerpc can address on its own -- it needs
discussion on how to influence the ordering between generic scratch
reservation and arch-specific crashkernel reservation. This patch
doesn't attempt that; it's meant as a starting point for that
discussion.

Together with patch 3, this gets CONFIG_TEST_KEXEC_HANDOVER passing
on powerpc.

Cc: Aditya Gupta <adityag@linux.ibm.com>
Cc: Alexander Graf <graf@amazon.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
Cc: Hari Bathini <hbathini@linux.ibm.com>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Mahesh Salgaonkar <mahesh@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Pasha Tatashin <pasha.tatashin@soleen.com>
Cc: Pratyush Yadav <pratyush@kernel.org>
Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Cc: Shivang Upadhyay <shivangu@linux.ibm.com>
Cc: Shrikanth Hegde <sshegde@linux.ibm.com>
Cc: kexec@lists.infradead.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
---
 arch/powerpc/Kconfig               |  5 +++++
 arch/powerpc/kernel/setup-common.c | 33 ++++++++++++++++++++++++++++++
 2 files changed, 38 insertions(+)

diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index 2580e27e4328..61350d3e7a19 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -716,6 +716,11 @@ config ARCH_SELECTS_CRASH_DUMP
 	depends on CRASH_DUMP
 	select RELOCATABLE if PPC64 || 44x || PPC_85xx
 
+config ARCH_SUPPORTS_KEXEC_HANDOVER
+	def_bool y
+	depends on PPC64
+	depends on !CRASH_DUMP
+
 config ARCH_SUPPORTS_CRASH_HOTPLUG
 	def_bool y
 	depends on PPC64
diff --git a/arch/powerpc/kernel/setup-common.c b/arch/powerpc/kernel/setup-common.c
index 4afaba19b586..1fee743abdf2 100644
--- a/arch/powerpc/kernel/setup-common.c
+++ b/arch/powerpc/kernel/setup-common.c
@@ -35,6 +35,8 @@
 #include <linux/of_irq.h>
 #include <linux/hugetlb.h>
 #include <linux/pgtable.h>
+#include <linux/libfdt.h>
+#include <linux/kexec_handover.h>
 #include <asm/io.h>
 #include <asm/paca.h>
 #include <asm/processor.h>
@@ -910,6 +912,33 @@ static void __init smp_setup_pacas(void)
 }
 #endif
 
+#ifdef CONFIG_PPC64
+static void __init init_kho(const void *fdt)
+{
+	unsigned long node;
+	u64 fdt_start, fdt_size, scratch_start, scratch_size;
+
+	if (!IS_ENABLED(CONFIG_KEXEC_HANDOVER))
+		return;
+
+	/* Find and verify the /chosen node, same as early_init_dt_scan_chosen() does */
+	node = fdt_path_offset(fdt, "/chosen");
+	if ((long)node < 0)
+		node = fdt_path_offset(fdt, "/chosen@0");
+	if ((long)node < 0)
+		return;
+
+	if (!of_flat_dt_get_addr_size(node, "linux,kho-fdt",
+				      &fdt_start, &fdt_size))
+		return;
+	if (!of_flat_dt_get_addr_size(node, "linux,kho-scratch",
+				      &scratch_start, &scratch_size))
+		return;
+
+	kho_populate(fdt_start, fdt_size, scratch_start, scratch_size);
+}
+#endif
+
 /*
  * Called into from start_kernel this initializes memblock, which is used
  * to manage page allocation until mem_init is called.
@@ -923,6 +952,10 @@ void __init setup_arch(char **cmdline_p)
 	/* Set a half-reasonable default so udelay does something sensible */
 	loops_per_jiffy = 500000000 / HZ;
 
+#ifdef CONFIG_PPC64
+	init_kho(initial_boot_params);
+#endif
+
 	/* Unflatten the device-tree passed by prom_init or kexec */
 	unflatten_device_tree();
 
-- 
2.55.0

[RFC PATCH 3/3] powerpc/kho: exclude THP and HUGETLB_PAGE

[Sourabh Jain]

Enabling CONFIG_KEXEC_HANDOVER together with CONFIG_TRANSPARENT_HUGEPAGE
or CONFIG_HUGETLB_PAGE on powerpc fails to build:

    kernel/liveupdate/kexec_handover.c:49:39: error: expression in static assertion is not constant
       49 | static_assert(SCRATCH_ALIGNMENT_BYTES >= CMA_MIN_ALIGNMENT_BYTES);
    ./include/linux/build_bug.h:80:56: note: in definition of macro '__static_assert'
       80 | #define __static_assert(expr, msg, ...) _Static_assert(expr, msg)
          |                                                        ^~~~
    kernel/liveupdate/kexec_handover.c:49:1: note: in expansion of macro 'static_assert'
       49 | static_assert(SCRATCH_ALIGNMENT_BYTES >= CMA_MIN_ALIGNMENT_BYTES);

CMA_MIN_ALIGNMENT_BYTES is derived from pageblock_order, which on
powerpc is not a compile-time constant when THP or HUGETLB_PAGE is
enabled -- powerpc can determine huge page size at runtime in that
configuration, so pageblock_order becomes a variable rather than a
constant expression. KHO's static_assert() on
SCRATCH_ALIGNMENT_BYTES >= CMA_MIN_ALIGNMENT_BYTES then fails to
compile, since it requires both sides to be constant expressions.

Until CMA_MIN_ALIGNMENT_BYTES (or KHO's use of it) is made to handle
the non-constant pageblock_order case on powerpc, exclude
TRANSPARENT_HUGEPAGE and HUGETLB_PAGE from ARCH_SUPPORTS_KEXEC_HANDOVER
so the two can't be enabled together and hit this build failure.

Cc: Aditya Gupta <adityag@linux.ibm.com>
Cc: Alexander Graf <graf@amazon.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
Cc: Hari Bathini <hbathini@linux.ibm.com>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Mahesh Salgaonkar <mahesh@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Pasha Tatashin <pasha.tatashin@soleen.com>
Cc: Pratyush Yadav <pratyush@kernel.org>
Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Cc: Shivang Upadhyay <shivangu@linux.ibm.com>
Cc: Shrikanth Hegde <sshegde@linux.ibm.com>
Cc: kexec@lists.infradead.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
---
 arch/powerpc/Kconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index 61350d3e7a19..f68b9fc883ba 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -720,6 +720,7 @@ config ARCH_SUPPORTS_KEXEC_HANDOVER
 	def_bool y
 	depends on PPC64
 	depends on !CRASH_DUMP
+	depends on !TRANSPARENT_HUGEPAGE && !HUGETLB_PAGE
 
 config ARCH_SUPPORTS_CRASH_HOTPLUG
 	def_bool y
-- 
2.55.0

Re: [RFC PATCH 3/3] powerpc/kho: exclude THP and HUGETLB_PAGE

[Pratyush Yadav]

On Fri, Aug 21 2026, Sourabh Jain wrote:

> Enabling CONFIG_KEXEC_HANDOVER together with CONFIG_TRANSPARENT_HUGEPAGE
> or CONFIG_HUGETLB_PAGE on powerpc fails to build:
>
>     kernel/liveupdate/kexec_handover.c:49:39: error: expression in static assertion is not constant
>        49 | static_assert(SCRATCH_ALIGNMENT_BYTES >= CMA_MIN_ALIGNMENT_BYTES);
>     ./include/linux/build_bug.h:80:56: note: in definition of macro '__static_assert'
>        80 | #define __static_assert(expr, msg, ...) _Static_assert(expr, msg)
>           |                                                        ^~~~
>     kernel/liveupdate/kexec_handover.c:49:1: note: in expansion of macro 'static_assert'
>        49 | static_assert(SCRATCH_ALIGNMENT_BYTES >= CMA_MIN_ALIGNMENT_BYTES);
>
> CMA_MIN_ALIGNMENT_BYTES is derived from pageblock_order, which on
> powerpc is not a compile-time constant when THP or HUGETLB_PAGE is
> enabled -- powerpc can determine huge page size at runtime in that
> configuration, so pageblock_order becomes a variable rather than a
> constant expression. KHO's static_assert() on
> SCRATCH_ALIGNMENT_BYTES >= CMA_MIN_ALIGNMENT_BYTES then fails to
> compile, since it requires both sides to be constant expressions.
>
> Until CMA_MIN_ALIGNMENT_BYTES (or KHO's use of it) is made to handle
> the non-constant pageblock_order case on powerpc, exclude
> TRANSPARENT_HUGEPAGE and HUGETLB_PAGE from ARCH_SUPPORTS_KEXEC_HANDOVER
> so the two can't be enabled together and hit this build failure.

This should be a part of patch 2 I think. It is not a good idea to break
the build in one patch and fix it in the next.

That said, this is a debug check of sorts since SCRATCH_ALIGNMENT_BYTES
is of MAX_ORDER_NR_PAGES, and pageblocks _should_ always be smaller of
equal to this.

Anyway, I don't think this is something you should gate by a config. I
think you should either remove the static_assert(), or turn it into a
runtime check.

+Cc Michal.

>
> Cc: Aditya Gupta <adityag@linux.ibm.com>
> Cc: Alexander Graf <graf@amazon.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Baoquan He <baoquan.he@linux.dev>
> Cc: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
> Cc: Hari Bathini <hbathini@linux.ibm.com>
> Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
> Cc: Mahesh Salgaonkar <mahesh@linux.ibm.com>
> Cc: Michael Ellerman <mpe@ellerman.id.au>
> Cc: Mike Rapoport <rppt@kernel.org>
> Cc: Nicholas Piggin <npiggin@gmail.com>
> Cc: Pasha Tatashin <pasha.tatashin@soleen.com>
> Cc: Pratyush Yadav <pratyush@kernel.org>
> Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
> Cc: Shivang Upadhyay <shivangu@linux.ibm.com>
> Cc: Shrikanth Hegde <sshegde@linux.ibm.com>
> Cc: kexec@lists.infradead.org
> Cc: linux-kernel@vger.kernel.org
> Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
> ---
>  arch/powerpc/Kconfig | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
> index 61350d3e7a19..f68b9fc883ba 100644
> --- a/arch/powerpc/Kconfig
> +++ b/arch/powerpc/Kconfig
> @@ -720,6 +720,7 @@ config ARCH_SUPPORTS_KEXEC_HANDOVER
>  	def_bool y
>  	depends on PPC64
>  	depends on !CRASH_DUMP
> +	depends on !TRANSPARENT_HUGEPAGE && !HUGETLB_PAGE
>  
>  config ARCH_SUPPORTS_CRASH_HOTPLUG
>  	def_bool y

-- 
Regards,
Pratyush Yadav

Re: [RFC PATCH 2/3] powerpc: add support for Kexec HandOver (KHO)

[Pratyush Yadav]

On Fri, Aug 21 2026, Sourabh Jain wrote:

> Add the architecture bits needed to enable CONFIG_KEXEC_HANDOVER on
> powerpc.
>
> Set ARCH_SUPPORTS_KEXEC_HANDOVER for PPC64, following the existing
> pattern used by ARCH_SUPPORTS_KEXEC and ARCH_SUPPORTS_KEXEC_FILE.
>
> On the boot path, parse the "linux,kho-fdt" and "linux,kho-scratch"
> properties from /chosen and pass them to kho_populate(). This lets a
> kernel booted via KHO kexec recover the FDT and scratch region left
> behind by the previous kernel. The call is placed early in
> setup_arch(), before unflatten_device_tree().
>
> Open issues:
> ============
>
> This patch also adds "depends on !CRASH_DUMP" to
> ARCH_SUPPORTS_KEXEC_HANDOVER. This is needed because of an ordering
> conflict between crashkernel reservation and KHO scratch reservation
> on powerpc.
>
> Crashkernel memory is reserved very early in boot, from arch-specific
> code: head.S -> early_setup() -> early_init_devtree() ->
> arch_reserve_crashkernel() / fadump_reserve_mem(). KHO's scratch
> region is reserved later, from generic code: start_kernel() ->
> mm_core_init() -> kho_memory_init(). So on powerpc, crashkernel
> memory is always reserved first.
>
> This ordering causes a real failure. In the common case, crashkernel
> reservation on powerpc starts at a 512M offset (the exact offset can
> vary, but 512M is typical). So with crashkernel=3G, the reservation
> occupies memory from 512M up to 3.5G -- roughly 75% of the entire low
> 4G area.
>
> Since crashkernel reservation always happens first, that 3G is
> already committed by the time kho_memory_init() runs. It then tries
> to reserve a low scratch region sized at 200% of whatever is already
> reserved below 4G. With ~75% of that 4G area already taken by
> crashkernel memory, 200% of that easily exceeds the remaining space
> -- and since the low scratch region is itself capped at 4G, there's
> no room left to fit it. The reservation fails.

crashkernel has the variant "crashkernel=size[KMG],high", which ensures
memory is allocated above 4G. Unless powerpc has some requirement for
strictly having the crashkernel below 4G, I think it will make a lot of
sense to enable support for this feature. So KHO users can specify this
to get crashkernel working with KHO.

Powerpc doesn't support this right now, but from a quick skim of the
code, I think it should be simple enough. From
arch_reserve_crashkernel() you just need to pass a bool * to
parse_crashkernel(), and then pass the result to
reserve_crashkernel_generic().

Solving the ordering of crash reservations and KHO is tricky and comes
with some difficult tradeoffs. Allocating crash from highmem should be a
lot simpler.

And on that note, I don't think you should do a depends on !CRASH_DUMP.
Even when CONFIG_KEXEC_HANDOVER is enabled, KHO isn't on by default
(well, unless KEXEC_HANDOVER_ENABLE_DEFAULT is set). You need to enable
it via cmdline. So it is entirely possible for people using KHO on PPC
to not use crash and vice versa. This decision can be made at deployment
time, not at compile time.

>
> To work around this and get KHO working on powerpc, this patch:
>
> 1. Makes KHO usable on powerpc only when CRASH_DUMP is disabled.
> 2. Calls kho_populate() from setup_arch(), so it runs before
>    kho_memory_init() reserves the scratch region.
>
> The real fix would be to reserve the KHO scratch region before
> crashkernel memory instead of after. But scratch reservation happens
> in generic code (kho_memory_init(), called from mm_core_init()), so
> this isn't something powerpc can address on its own -- it needs
> discussion on how to influence the ordering between generic scratch
> reservation and arch-specific crashkernel reservation. This patch
> doesn't attempt that; it's meant as a starting point for that
> discussion.
>
[...]
> diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
> index 2580e27e4328..61350d3e7a19 100644
> --- a/arch/powerpc/Kconfig
> +++ b/arch/powerpc/Kconfig
> @@ -716,6 +716,11 @@ config ARCH_SELECTS_CRASH_DUMP
>  	depends on CRASH_DUMP
>  	select RELOCATABLE if PPC64 || 44x || PPC_85xx
>  
> +config ARCH_SUPPORTS_KEXEC_HANDOVER
> +	def_bool y
> +	depends on PPC64
> +	depends on !CRASH_DUMP
> +
>  config ARCH_SUPPORTS_CRASH_HOTPLUG
>  	def_bool y
>  	depends on PPC64
> diff --git a/arch/powerpc/kernel/setup-common.c b/arch/powerpc/kernel/setup-common.c
> index 4afaba19b586..1fee743abdf2 100644
> --- a/arch/powerpc/kernel/setup-common.c
> +++ b/arch/powerpc/kernel/setup-common.c
[...]
>  }
>  #endif
>  
> +#ifdef CONFIG_PPC64
> +static void __init init_kho(const void *fdt)
> +{
> +	unsigned long node;
> +	u64 fdt_start, fdt_size, scratch_start, scratch_size;
> +
> +	if (!IS_ENABLED(CONFIG_KEXEC_HANDOVER))
> +		return;
> +
> +	/* Find and verify the /chosen node, same as early_init_dt_scan_chosen() does */
> +	node = fdt_path_offset(fdt, "/chosen");
> +	if ((long)node < 0)
> +		node = fdt_path_offset(fdt, "/chosen@0");
> +	if ((long)node < 0)
> +		return;
> +
> +	if (!of_flat_dt_get_addr_size(node, "linux,kho-fdt",
> +				      &fdt_start, &fdt_size))
> +		return;
> +	if (!of_flat_dt_get_addr_size(node, "linux,kho-scratch",
> +				      &scratch_start, &scratch_size))
> +		return;
> +
> +	kho_populate(fdt_start, fdt_size, scratch_start, scratch_size);
> +}

This looks pretty much a duplicate of early_init_dt_check_kho(). On
arm64 this is called via early_init_dt_scan(). But from a quick search I
don't see powerpc calling it.

Would it make sense to call this function (or
early_init_dt_scan_nodes()) for powerpc?

If not, I think it would be a better idea to expose
early_init_dt_check_kho() and call it from powerpc setup_arch() instead
of duplicating the logic.

> +#endif
> +
>  /*
>   * Called into from start_kernel this initializes memblock, which is used
>   * to manage page allocation until mem_init is called.
[...]

-- 
Regards,
Pratyush Yadav

Re: [RFC PATCH 3/3] powerpc/kho: exclude THP and HUGETLB_PAGE

[Sourabh Jain]

On 21/08/26 17:04, Pratyush Yadav wrote:
> On Fri, Aug 21 2026, Sourabh Jain wrote:
>
>> Enabling CONFIG_KEXEC_HANDOVER together with CONFIG_TRANSPARENT_HUGEPAGE
>> or CONFIG_HUGETLB_PAGE on powerpc fails to build:
>>
>>      kernel/liveupdate/kexec_handover.c:49:39: error: expression in static assertion is not constant
>>         49 | static_assert(SCRATCH_ALIGNMENT_BYTES >= CMA_MIN_ALIGNMENT_BYTES);
>>      ./include/linux/build_bug.h:80:56: note: in definition of macro '__static_assert'
>>         80 | #define __static_assert(expr, msg, ...) _Static_assert(expr, msg)
>>            |                                                        ^~~~
>>      kernel/liveupdate/kexec_handover.c:49:1: note: in expansion of macro 'static_assert'
>>         49 | static_assert(SCRATCH_ALIGNMENT_BYTES >= CMA_MIN_ALIGNMENT_BYTES);
>>
>> CMA_MIN_ALIGNMENT_BYTES is derived from pageblock_order, which on
>> powerpc is not a compile-time constant when THP or HUGETLB_PAGE is
>> enabled -- powerpc can determine huge page size at runtime in that
>> configuration, so pageblock_order becomes a variable rather than a
>> constant expression. KHO's static_assert() on
>> SCRATCH_ALIGNMENT_BYTES >= CMA_MIN_ALIGNMENT_BYTES then fails to
>> compile, since it requires both sides to be constant expressions.
>>
>> Until CMA_MIN_ALIGNMENT_BYTES (or KHO's use of it) is made to handle
>> the non-constant pageblock_order case on powerpc, exclude
>> TRANSPARENT_HUGEPAGE and HUGETLB_PAGE from ARCH_SUPPORTS_KEXEC_HANDOVER
>> so the two can't be enabled together and hit this build failure.
> This should be a part of patch 2 I think. It is not a good idea to break
> the build in one patch and fix it in the next.

Agree. Since this is an RFC patch series, I kept it separate so that 
this issue could
be reviewed independently from patch 2. Going forward, I will merge them.


>
> That said, this is a debug check of sorts since SCRATCH_ALIGNMENT_BYTES
> is of MAX_ORDER_NR_PAGES, and pageblocks _should_ always be smaller of
> equal to this.
>
> Anyway, I don't think this is something you should gate by a config. I
> think you should either remove the static_assert(), or turn it into a
> runtime check.

Agreed. I will evaluate both options and decide which approach makes 
more sense.

Thanks for the review Pratyush.

- Sourabh Jain


>
> +Cc Michal.
>
>> Cc: Aditya Gupta <adityag@linux.ibm.com>
>> Cc: Alexander Graf <graf@amazon.com>
>> Cc: Andrew Morton <akpm@linux-foundation.org>
>> Cc: Baoquan He <baoquan.he@linux.dev>
>> Cc: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
>> Cc: Hari Bathini <hbathini@linux.ibm.com>
>> Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
>> Cc: Mahesh Salgaonkar <mahesh@linux.ibm.com>
>> Cc: Michael Ellerman <mpe@ellerman.id.au>
>> Cc: Mike Rapoport <rppt@kernel.org>
>> Cc: Nicholas Piggin <npiggin@gmail.com>
>> Cc: Pasha Tatashin <pasha.tatashin@soleen.com>
>> Cc: Pratyush Yadav <pratyush@kernel.org>
>> Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
>> Cc: Shivang Upadhyay <shivangu@linux.ibm.com>
>> Cc: Shrikanth Hegde <sshegde@linux.ibm.com>
>> Cc: kexec@lists.infradead.org
>> Cc: linux-kernel@vger.kernel.org
>> Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
>> ---
>>   arch/powerpc/Kconfig | 1 +
>>   1 file changed, 1 insertion(+)
>>
>> diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
>> index 61350d3e7a19..f68b9fc883ba 100644
>> --- a/arch/powerpc/Kconfig
>> +++ b/arch/powerpc/Kconfig
>> @@ -720,6 +720,7 @@ config ARCH_SUPPORTS_KEXEC_HANDOVER
>>   	def_bool y
>>   	depends on PPC64
>>   	depends on !CRASH_DUMP
>> +	depends on !TRANSPARENT_HUGEPAGE && !HUGETLB_PAGE
>>   
>>   config ARCH_SUPPORTS_CRASH_HOTPLUG
>>   	def_bool y

Re: [RFC PATCH 2/3] powerpc: add support for Kexec HandOver (KHO)

[Sourabh Jain]

On 21/08/26 17:26, Pratyush Yadav wrote:
> On Fri, Aug 21 2026, Sourabh Jain wrote:
>
>> Add the architecture bits needed to enable CONFIG_KEXEC_HANDOVER on
>> powerpc.
>>
>> Set ARCH_SUPPORTS_KEXEC_HANDOVER for PPC64, following the existing
>> pattern used by ARCH_SUPPORTS_KEXEC and ARCH_SUPPORTS_KEXEC_FILE.
>>
>> On the boot path, parse the "linux,kho-fdt" and "linux,kho-scratch"
>> properties from /chosen and pass them to kho_populate(). This lets a
>> kernel booted via KHO kexec recover the FDT and scratch region left
>> behind by the previous kernel. The call is placed early in
>> setup_arch(), before unflatten_device_tree().
>>
>> Open issues:
>> ============
>>
>> This patch also adds "depends on !CRASH_DUMP" to
>> ARCH_SUPPORTS_KEXEC_HANDOVER. This is needed because of an ordering
>> conflict between crashkernel reservation and KHO scratch reservation
>> on powerpc.
>>
>> Crashkernel memory is reserved very early in boot, from arch-specific
>> code: head.S -> early_setup() -> early_init_devtree() ->
>> arch_reserve_crashkernel() / fadump_reserve_mem(). KHO's scratch
>> region is reserved later, from generic code: start_kernel() ->
>> mm_core_init() -> kho_memory_init(). So on powerpc, crashkernel
>> memory is always reserved first.
>>
>> This ordering causes a real failure. In the common case, crashkernel
>> reservation on powerpc starts at a 512M offset (the exact offset can
>> vary, but 512M is typical). So with crashkernel=3G, the reservation
>> occupies memory from 512M up to 3.5G -- roughly 75% of the entire low
>> 4G area.
>>
>> Since crashkernel reservation always happens first, that 3G is
>> already committed by the time kho_memory_init() runs. It then tries
>> to reserve a low scratch region sized at 200% of whatever is already
>> reserved below 4G. With ~75% of that 4G area already taken by
>> crashkernel memory, 200% of that easily exceeds the remaining space
>> -- and since the low scratch region is itself capped at 4G, there's
>> no room left to fit it. The reservation fails.
> crashkernel has the variant "crashkernel=size[KMG],high", which ensures
> memory is allocated above 4G. Unless powerpc has some requirement for
> strictly having the crashkernel below 4G, I think it will make a lot of
> sense to enable support for this feature. So KHO users can specify this
> to get crashkernel working with KHO.

I agree that this is one way to work around the low-memory reservation 
problem.
However, there are a few things that come into play here:

1. On powerpc, the crashkernel reservation can go up to 64 GB for kdump. 
With the
    current default scratch memory reservation policy, this could result 
in reserving
    up to 256 GB of scratch memory: 200% for the high-memory reservation 
and another
    200% for per-node memory.
    For fadump, which is the powerpc-specific memory dump capture 
mechanism, the crashkernel
    reservation can go up to 180 GB. In this case, we could end up 
reserving up to 720 GB of
    scratch memory, which is too much. I agree that users can tune this, 
but I think the
    default scale should be more reasonable for powerpc.


2. Fadump also uses the crashkernel kernel command-line argument, but 
its reservation policy
    is different from kdump. The crashkernel base address starts after 
the memory needed for
    fadump. For example, with crashkernel=3G, the base address would be 
3 GB, and the crashkernel
    reservation would be from 3 GB to 6 GB. So, depending on the 
crashkernel size, the reservation
    may or may not fall within low memory. Also, fadump does not support 
crashkernel=xxM,high.

3. I do have a patch [1] to support high crashkernel reservations with 
kdump, but this would not
    work with Hash MMU. With Hash MMU, the kernel image is constrained 
to low memory, whereas with
    crashkernel=,high, all segments would be loaded into high memory.

So, while supporting crashkernel=,high can help address the low-memory 
reservation issue for kdump,
I think there are still some powerpc-specific constraints to consider. I 
would like to explore
whether we can fix the ordering between crashkernel and scratch memory 
reservations, so that we can
avoid unnecessarily large scratch memory reservations and address some 
of the other constraints mentioned
above. Please share your thoughts.

>
> Powerpc doesn't support this right now, but from a quick skim of the
> code, I think it should be simple enough.

I have patch series under review for the same:
[1] 
https://lore.kernel.org/all/20260708143357.673251-1-sourabhjain@linux.ibm.com/

> From
> arch_reserve_crashkernel() you just need to pass a bool * to
> parse_crashkernel(), and then pass the result to
> reserve_crashkernel_generic().

Due to some architecture-specific dependencies (such as RTAS), booting 
the kernel from above 4G with
support for high crashkernel is not as straightforward as on other 
architectures. Patch 2/4 in [1] has the
details.

>
> Solving the ordering of crash reservations and KHO is tricky and comes
> with some difficult tradeoffs. Allocating crash from highmem should be a
> lot simpler.


Could you please elaborate a bit on what makes the ordering tricky and 
what the
main tradeoffs are between crash reservations and KHO? It would help me 
better
understand the concerns here.


>
> And on that note, I don't think you should do a depends on !CRASH_DUMP.
> Even when CONFIG_KEXEC_HANDOVER is enabled, KHO isn't on by default
> (well, unless KEXEC_HANDOVER_ENABLE_DEFAULT is set). You need to enable
> it via cmdline. So it is entirely possible for people using KHO on PPC
> to not use crash and vice versa. This decision can be made at deployment
> time, not at compile time.

Agree. depends on !CRASH_DUMP is temporary and will be removed once
we settle the crashkernel reservation and scratch region handling.

>
>> To work around this and get KHO working on powerpc, this patch:
>>
>> 1. Makes KHO usable on powerpc only when CRASH_DUMP is disabled.
>> 2. Calls kho_populate() from setup_arch(), so it runs before
>>     kho_memory_init() reserves the scratch region.
>>
>> The real fix would be to reserve the KHO scratch region before
>> crashkernel memory instead of after. But scratch reservation happens
>> in generic code (kho_memory_init(), called from mm_core_init()), so
>> this isn't something powerpc can address on its own -- it needs
>> discussion on how to influence the ordering between generic scratch
>> reservation and arch-specific crashkernel reservation. This patch
>> doesn't attempt that; it's meant as a starting point for that
>> discussion.
>>
> [...]
>> diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
>> index 2580e27e4328..61350d3e7a19 100644
>> --- a/arch/powerpc/Kconfig
>> +++ b/arch/powerpc/Kconfig
>> @@ -716,6 +716,11 @@ config ARCH_SELECTS_CRASH_DUMP
>>   	depends on CRASH_DUMP
>>   	select RELOCATABLE if PPC64 || 44x || PPC_85xx
>>   
>> +config ARCH_SUPPORTS_KEXEC_HANDOVER
>> +	def_bool y
>> +	depends on PPC64
>> +	depends on !CRASH_DUMP
>> +
>>   config ARCH_SUPPORTS_CRASH_HOTPLUG
>>   	def_bool y
>>   	depends on PPC64
>> diff --git a/arch/powerpc/kernel/setup-common.c b/arch/powerpc/kernel/setup-common.c
>> index 4afaba19b586..1fee743abdf2 100644
>> --- a/arch/powerpc/kernel/setup-common.c
>> +++ b/arch/powerpc/kernel/setup-common.c
> [...]
>>   }
>>   #endif
>>   
>> +#ifdef CONFIG_PPC64
>> +static void __init init_kho(const void *fdt)
>> +{
>> +	unsigned long node;
>> +	u64 fdt_start, fdt_size, scratch_start, scratch_size;
>> +
>> +	if (!IS_ENABLED(CONFIG_KEXEC_HANDOVER))
>> +		return;
>> +
>> +	/* Find and verify the /chosen node, same as early_init_dt_scan_chosen() does */
>> +	node = fdt_path_offset(fdt, "/chosen");
>> +	if ((long)node < 0)
>> +		node = fdt_path_offset(fdt, "/chosen@0");
>> +	if ((long)node < 0)
>> +		return;
>> +
>> +	if (!of_flat_dt_get_addr_size(node, "linux,kho-fdt",
>> +				      &fdt_start, &fdt_size))
>> +		return;
>> +	if (!of_flat_dt_get_addr_size(node, "linux,kho-scratch",
>> +				      &scratch_start, &scratch_size))
>> +		return;
>> +
>> +	kho_populate(fdt_start, fdt_size, scratch_start, scratch_size);
>> +}
> This looks pretty much a duplicate of early_init_dt_check_kho(). On
> arm64 this is called via early_init_dt_scan(). But from a quick search I
> don't see powerpc calling it.

Yes it is not called on powerpc.

>
> Would it make sense to call this function (or
> early_init_dt_scan_nodes()) for powerpc?
>
> If not, I think it would be a better idea to expose
> early_init_dt_check_kho() and call it from powerpc setup_arch() instead
> of duplicating the logic.

Agree they are identical.

I would prefer making early_init_dt_check_kho() as public function and 
call from arch specific code. Thanks for the review. - Sourabh Jain

>
>> +#endif
>> +
>>   /*
>>    * Called into from start_kernel this initializes memblock, which is used
>>    * to manage page allocation until mem_init is called.
> [...]
>

Re: [RFC PATCH 2/3] powerpc: add support for Kexec HandOver (KHO)

[Sourabh Jain]

Hello Pratyush,

After looking at the x86 crashkernel and KHO scratch memory
allocation order, I realized that the same issue should exist on x86
as well. I ran the same experiment on an x86 guest, and the results
confirmed this.


[root@localhost ~]# dmesg | grep -i -e kho -e crashkernel
[    0.000000] Command line: 
BOOT_IMAGE=(hd0,gpt3)/boot/vmlinuz-6.19.10-300.fc44.x86_64 
no_timer_check console=tty1 console=ttyS0,115200n8 systemd.firstboot=off 
root=UUID=15c26993-ac30-424a-9c4b-faec4434d234 rootflags=subvol=root 
kho=on crashkernel=1G
[    0.004615] crashkernel reserved: 0x000000007f000000 - 
0x00000000bf000000 (1024 MB)
[    0.034746] Kernel command line: 
BOOT_IMAGE=(hd0,gpt3)/boot/vmlinuz-6.19.10-300.fc44.x86_64 
no_timer_check console=tty1 console=ttyS0,115200n8 systemd.firstboot=off 
root=UUID=15c26993-ac30-424a-9c4b-faec4434d234 rootflags=subvol=root 
kho=on crashkernel=1G
[    0.095761] KHO: Failed to reserve scratch area, disabling kexec handover

I agree that this can be avoided by using the crashkernel=xxM,high option.
However, I just wanted to share the same problem currently exists on x86 
as well.

While reading the KHO code, I came across the following commit, which 
handles
a somewhat similar problem with HugeTLB:

fdd843f2be1a ("kho: exclude hugetlb memory from scratch size calculation")

The approach is to first account for HugeTLB allocations as a separate 
memblock
reservation type and then exclude them when calculating the scratch memory
reservation.

How about using the same approach to avoid accounting for crashkernel memory
when calculating the scratch memory size?

- Sourabh Jain


On 21/08/26 17:26, Pratyush Yadav wrote:
> On Fri, Aug 21 2026, Sourabh Jain wrote:
>
>> Add the architecture bits needed to enable CONFIG_KEXEC_HANDOVER on
>> powerpc.
>>
>> Set ARCH_SUPPORTS_KEXEC_HANDOVER for PPC64, following the existing
>> pattern used by ARCH_SUPPORTS_KEXEC and ARCH_SUPPORTS_KEXEC_FILE.
>>
>> On the boot path, parse the "linux,kho-fdt" and "linux,kho-scratch"
>> properties from /chosen and pass them to kho_populate(). This lets a
>> kernel booted via KHO kexec recover the FDT and scratch region left
>> behind by the previous kernel. The call is placed early in
>> setup_arch(), before unflatten_device_tree().
>>
>> Open issues:
>> ============
>>
>> This patch also adds "depends on !CRASH_DUMP" to
>> ARCH_SUPPORTS_KEXEC_HANDOVER. This is needed because of an ordering
>> conflict between crashkernel reservation and KHO scratch reservation
>> on powerpc.
>>
>> Crashkernel memory is reserved very early in boot, from arch-specific
>> code: head.S -> early_setup() -> early_init_devtree() ->
>> arch_reserve_crashkernel() / fadump_reserve_mem(). KHO's scratch
>> region is reserved later, from generic code: start_kernel() ->
>> mm_core_init() -> kho_memory_init(). So on powerpc, crashkernel
>> memory is always reserved first.
>>
>> This ordering causes a real failure. In the common case, crashkernel
>> reservation on powerpc starts at a 512M offset (the exact offset can
>> vary, but 512M is typical). So with crashkernel=3G, the reservation
>> occupies memory from 512M up to 3.5G -- roughly 75% of the entire low
>> 4G area.
>>
>> Since crashkernel reservation always happens first, that 3G is
>> already committed by the time kho_memory_init() runs. It then tries
>> to reserve a low scratch region sized at 200% of whatever is already
>> reserved below 4G. With ~75% of that 4G area already taken by
>> crashkernel memory, 200% of that easily exceeds the remaining space
>> -- and since the low scratch region is itself capped at 4G, there's
>> no room left to fit it. The reservation fails.
> crashkernel has the variant "crashkernel=size[KMG],high", which ensures
> memory is allocated above 4G. Unless powerpc has some requirement for
> strictly having the crashkernel below 4G, I think it will make a lot of
> sense to enable support for this feature. So KHO users can specify this
> to get crashkernel working with KHO.
>
> Powerpc doesn't support this right now, but from a quick skim of the
> code, I think it should be simple enough. From
> arch_reserve_crashkernel() you just need to pass a bool * to
> parse_crashkernel(), and then pass the result to
> reserve_crashkernel_generic().
>
> Solving the ordering of crash reservations and KHO is tricky and comes
> with some difficult tradeoffs. Allocating crash from highmem should be a
> lot simpler.
>
> And on that note, I don't think you should do a depends on !CRASH_DUMP.
> Even when CONFIG_KEXEC_HANDOVER is enabled, KHO isn't on by default
> (well, unless KEXEC_HANDOVER_ENABLE_DEFAULT is set). You need to enable
> it via cmdline. So it is entirely possible for people using KHO on PPC
> to not use crash and vice versa. This decision can be made at deployment
> time, not at compile time.
>
>> To work around this and get KHO working on powerpc, this patch:
>>
>> 1. Makes KHO usable on powerpc only when CRASH_DUMP is disabled.
>> 2. Calls kho_populate() from setup_arch(), so it runs before
>>     kho_memory_init() reserves the scratch region.
>>
>> The real fix would be to reserve the KHO scratch region before
>> crashkernel memory instead of after. But scratch reservation happens
>> in generic code (kho_memory_init(), called from mm_core_init()), so
>> this isn't something powerpc can address on its own -- it needs
>> discussion on how to influence the ordering between generic scratch
>> reservation and arch-specific crashkernel reservation. This patch
>> doesn't attempt that; it's meant as a starting point for that
>> discussion.
>>
> [...]
>> diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
>> index 2580e27e4328..61350d3e7a19 100644
>> --- a/arch/powerpc/Kconfig
>> +++ b/arch/powerpc/Kconfig
>> @@ -716,6 +716,11 @@ config ARCH_SELECTS_CRASH_DUMP
>>   	depends on CRASH_DUMP
>>   	select RELOCATABLE if PPC64 || 44x || PPC_85xx
>>   
>> +config ARCH_SUPPORTS_KEXEC_HANDOVER
>> +	def_bool y
>> +	depends on PPC64
>> +	depends on !CRASH_DUMP
>> +
>>   config ARCH_SUPPORTS_CRASH_HOTPLUG
>>   	def_bool y
>>   	depends on PPC64
>> diff --git a/arch/powerpc/kernel/setup-common.c b/arch/powerpc/kernel/setup-common.c
>> index 4afaba19b586..1fee743abdf2 100644
>> --- a/arch/powerpc/kernel/setup-common.c
>> +++ b/arch/powerpc/kernel/setup-common.c
> [...]
>>   }
>>   #endif
>>   
>> +#ifdef CONFIG_PPC64
>> +static void __init init_kho(const void *fdt)
>> +{
>> +	unsigned long node;
>> +	u64 fdt_start, fdt_size, scratch_start, scratch_size;
>> +
>> +	if (!IS_ENABLED(CONFIG_KEXEC_HANDOVER))
>> +		return;
>> +
>> +	/* Find and verify the /chosen node, same as early_init_dt_scan_chosen() does */
>> +	node = fdt_path_offset(fdt, "/chosen");
>> +	if ((long)node < 0)
>> +		node = fdt_path_offset(fdt, "/chosen@0");
>> +	if ((long)node < 0)
>> +		return;
>> +
>> +	if (!of_flat_dt_get_addr_size(node, "linux,kho-fdt",
>> +				      &fdt_start, &fdt_size))
>> +		return;
>> +	if (!of_flat_dt_get_addr_size(node, "linux,kho-scratch",
>> +				      &scratch_start, &scratch_size))
>> +		return;
>> +
>> +	kho_populate(fdt_start, fdt_size, scratch_start, scratch_size);
>> +}
> This looks pretty much a duplicate of early_init_dt_check_kho(). On
> arm64 this is called via early_init_dt_scan(). But from a quick search I
> don't see powerpc calling it.
>
> Would it make sense to call this function (or
> early_init_dt_scan_nodes()) for powerpc?
>
> If not, I think it would be a better idea to expose
> early_init_dt_check_kho() and call it from powerpc setup_arch() instead
> of duplicating the logic.
>
>> +#endif
>> +
>>   /*
>>    * Called into from start_kernel this initializes memblock, which is used
>>    * to manage page allocation until mem_init is called.
> [...]
>

Re: [RFC PATCH 2/3] powerpc: add support for Kexec HandOver (KHO)

[Pratyush Yadav]

On Sun, Aug 23 2026, Sourabh Jain wrote:

> On 21/08/26 17:26, Pratyush Yadav wrote:
>> On Fri, Aug 21 2026, Sourabh Jain wrote:
>>
>>> Add the architecture bits needed to enable CONFIG_KEXEC_HANDOVER on
>>> powerpc.
>>>
>>> Set ARCH_SUPPORTS_KEXEC_HANDOVER for PPC64, following the existing
>>> pattern used by ARCH_SUPPORTS_KEXEC and ARCH_SUPPORTS_KEXEC_FILE.
>>>
>>> On the boot path, parse the "linux,kho-fdt" and "linux,kho-scratch"
>>> properties from /chosen and pass them to kho_populate(). This lets a
>>> kernel booted via KHO kexec recover the FDT and scratch region left
>>> behind by the previous kernel. The call is placed early in
>>> setup_arch(), before unflatten_device_tree().
>>>
>>> Open issues:
>>> ============
>>>
>>> This patch also adds "depends on !CRASH_DUMP" to
>>> ARCH_SUPPORTS_KEXEC_HANDOVER. This is needed because of an ordering
>>> conflict between crashkernel reservation and KHO scratch reservation
>>> on powerpc.
>>>
>>> Crashkernel memory is reserved very early in boot, from arch-specific
>>> code: head.S -> early_setup() -> early_init_devtree() ->
>>> arch_reserve_crashkernel() / fadump_reserve_mem(). KHO's scratch
>>> region is reserved later, from generic code: start_kernel() ->
>>> mm_core_init() -> kho_memory_init(). So on powerpc, crashkernel
>>> memory is always reserved first.
>>>
>>> This ordering causes a real failure. In the common case, crashkernel
>>> reservation on powerpc starts at a 512M offset (the exact offset can
>>> vary, but 512M is typical). So with crashkernel=3G, the reservation
>>> occupies memory from 512M up to 3.5G -- roughly 75% of the entire low
>>> 4G area.
>>>
>>> Since crashkernel reservation always happens first, that 3G is
>>> already committed by the time kho_memory_init() runs. It then tries
>>> to reserve a low scratch region sized at 200% of whatever is already
>>> reserved below 4G. With ~75% of that 4G area already taken by
>>> crashkernel memory, 200% of that easily exceeds the remaining space
>>> -- and since the low scratch region is itself capped at 4G, there's
>>> no room left to fit it. The reservation fails.
>> crashkernel has the variant "crashkernel=size[KMG],high", which ensures
>> memory is allocated above 4G. Unless powerpc has some requirement for
>> strictly having the crashkernel below 4G, I think it will make a lot of
>> sense to enable support for this feature. So KHO users can specify this
>> to get crashkernel working with KHO.
>
> I agree that this is one way to work around the low-memory reservation problem.
> However, there are a few things that come into play here:
>
> 1. On powerpc, the crashkernel reservation can go up to 64 GB for kdump. With
> the
>    current default scratch memory reservation policy, this could result in
> reserving
>    up to 256 GB of scratch memory: 200% for the high-memory reservation and
> another
>    200% for per-node memory.

That calculation looks off. It _should_ be 200% once not twice. So 128
GB total. If the allocation came out via the global area, it should
_only_ be accounted to the global scratch size. Similarly, only the
allocations made specifically on that node should be counted for the
per-node scratch size.

But I have also noticed this problem on some of the systems Google has.
Which makes me wonder if scratch_size_update() is broken and
over-calculating. I have this on my TODO list and have been meaning to
look into it, but other things keep intervening.

If you are interested, feel free to take it off my hands.

>    For fadump, which is the powerpc-specific memory dump capture mechanism, the
> crashkernel
>    reservation can go up to 180 GB. In this case, we could end up reserving up
> to 720 GB of
>    scratch memory, which is too much. I agree that users can tune this, but I
> think the
>    default scale should be more reasonable for powerpc.

Once we fix scratch_size_update() to actually use 200% and not 400%,
perhaps that alone will be enough? If not, we can discuss reducing the
default scratch scale to maybe 150%. But I'd rather do it for all
platforms if we do it at all, because this problem doesn't seem specific
to PowerPC.

>
>
> 2. Fadump also uses the crashkernel kernel command-line argument, but its
> reservation policy
>    is different from kdump. The crashkernel base address starts after the memory
> needed for
>    fadump. For example, with crashkernel=3G, the base address would be 3 GB, and
> the crashkernel
>    reservation would be from 3 GB to 6 GB. So, depending on the crashkernel
> size, the reservation
>    may or may not fall within low memory. Also, fadump does not support
> crashkernel=xxM,high.
>
> 3. I do have a patch [1] to support high crashkernel reservations with kdump,
> but this would not
>    work with Hash MMU. With Hash MMU, the kernel image is constrained to low
> memory, whereas with
>    crashkernel=,high, all segments would be loaded into high memory.

Oh, nice!

>
> So, while supporting crashkernel=,high can help address the low-memory
> reservation issue for kdump,
> I think there are still some powerpc-specific constraints to consider. I would
> like to explore
> whether we can fix the ordering between crashkernel and scratch memory
> reservations, so that we can
> avoid unnecessarily large scratch memory reservations and address some of the
> other constraints mentioned
> above. Please share your thoughts.
>
>>
>> Powerpc doesn't support this right now, but from a quick skim of the
>> code, I think it should be simple enough.
>
> I have patch series under review for the same:
> [1]
> https://lore.kernel.org/all/20260708143357.673251-1-sourabhjain@linux.ibm.com/
>
>> From
>> arch_reserve_crashkernel() you just need to pass a bool * to
>> parse_crashkernel(), and then pass the result to
>> reserve_crashkernel_generic().
>
> Due to some architecture-specific dependencies (such as RTAS), booting the
> kernel from above 4G with
> support for high crashkernel is not as straightforward as on other
> architectures. Patch 2/4 in [1] has the
> details.
>
>>
>> Solving the ordering of crash reservations and KHO is tricky and comes
>> with some difficult tradeoffs. Allocating crash from highmem should be a
>> lot simpler.
>
>
> Could you please elaborate a bit on what makes the ordering tricky and what the
> main tradeoffs are between crash reservations and KHO? It would help me better
> understand the concerns here.

The problem today is that kho_preserved_memory_reserve() (called by
kho_mem_retrieve()) does a memblock_reserve() for each preserved folio.
So if you have a lot of order-0 (or, 4k) folios, you end up with a lot
of reservations in memblock. The large number of reservations can slow
down later memblock operations like allocations too since memblock might
have to walk through a lot of ranges to find free memory.

We kind of work around this problem by calling kho_mem_retrieve() as
pretty much the last thing in the MM init. So all allocations prior to
this have already been fulfilled from scratch without any of the
reservations added, so it should be pretty fast. You only take the
performance hit at the end, where the only thing left is to release
pages to buddy.

Even then, the memblock reservations can get pretty damn slow. In some
of my testing with under-load systems, preserving a 2G memfd with 4k
pages can go over **5 minutes** in only kho_mem_retrieve() if the folios
of the memfd are fragmented enough. Plus there is the memory overhead of
the regions in memblock.reserved.

So long-term, I would like to get rid of the memblock reservations
entirely and use scratch-only mode all the way until buddy comes up. And
I would like to modify buddy init (free_low_memory_core_early() and
deferred_init_memmap_chunk()) to be KHO-aware and directly skip the KHO
pages.

This vision goes in the opposite direction of turning scratch-only mode
off _earlier_. And turning off scratch-only mode earlier is necessary
for doing crash reservations outside of scratch.

That is the tradeoff I mentioned. Hope I was clear enough.

>>
>> And on that note, I don't think you should do a depends on !CRASH_DUMP.
>> Even when CONFIG_KEXEC_HANDOVER is enabled, KHO isn't on by default
>> (well, unless KEXEC_HANDOVER_ENABLE_DEFAULT is set). You need to enable
>> it via cmdline. So it is entirely possible for people using KHO on PPC
>> to not use crash and vice versa. This decision can be made at deployment
>> time, not at compile time.
>
> Agree. depends on !CRASH_DUMP is temporary and will be removed once
> we settle the crashkernel reservation and scratch region handling.

My point is that !CRASH_DUMP can be removed _even if_ we don't settle
the reservation thing, because it is perfectly valid for the same kernel
to use either KHO or crash but not at the same time. These both can be
enabled/disabled at runtime.

[...]

-- 
Regards,
Pratyush Yadav

Re: [RFC PATCH 2/3] powerpc: add support for Kexec HandOver (KHO)

[Sourabh Jain]

On 02/09/26 16:04, Pratyush Yadav wrote:
> On Sun, Aug 23 2026, Sourabh Jain wrote:
>
>> On 21/08/26 17:26, Pratyush Yadav wrote:
>>> On Fri, Aug 21 2026, Sourabh Jain wrote:
>>>
>>>> Add the architecture bits needed to enable CONFIG_KEXEC_HANDOVER on
>>>> powerpc.
>>>>
>>>> Set ARCH_SUPPORTS_KEXEC_HANDOVER for PPC64, following the existing
>>>> pattern used by ARCH_SUPPORTS_KEXEC and ARCH_SUPPORTS_KEXEC_FILE.
>>>>
>>>> On the boot path, parse the "linux,kho-fdt" and "linux,kho-scratch"
>>>> properties from /chosen and pass them to kho_populate(). This lets a
>>>> kernel booted via KHO kexec recover the FDT and scratch region left
>>>> behind by the previous kernel. The call is placed early in
>>>> setup_arch(), before unflatten_device_tree().
>>>>
>>>> Open issues:
>>>> ============
>>>>
>>>> This patch also adds "depends on !CRASH_DUMP" to
>>>> ARCH_SUPPORTS_KEXEC_HANDOVER. This is needed because of an ordering
>>>> conflict between crashkernel reservation and KHO scratch reservation
>>>> on powerpc.
>>>>
>>>> Crashkernel memory is reserved very early in boot, from arch-specific
>>>> code: head.S -> early_setup() -> early_init_devtree() ->
>>>> arch_reserve_crashkernel() / fadump_reserve_mem(). KHO's scratch
>>>> region is reserved later, from generic code: start_kernel() ->
>>>> mm_core_init() -> kho_memory_init(). So on powerpc, crashkernel
>>>> memory is always reserved first.
>>>>
>>>> This ordering causes a real failure. In the common case, crashkernel
>>>> reservation on powerpc starts at a 512M offset (the exact offset can
>>>> vary, but 512M is typical). So with crashkernel=3G, the reservation
>>>> occupies memory from 512M up to 3.5G -- roughly 75% of the entire low
>>>> 4G area.
>>>>
>>>> Since crashkernel reservation always happens first, that 3G is
>>>> already committed by the time kho_memory_init() runs. It then tries
>>>> to reserve a low scratch region sized at 200% of whatever is already
>>>> reserved below 4G. With ~75% of that 4G area already taken by
>>>> crashkernel memory, 200% of that easily exceeds the remaining space
>>>> -- and since the low scratch region is itself capped at 4G, there's
>>>> no room left to fit it. The reservation fails.
>>> crashkernel has the variant "crashkernel=size[KMG],high", which ensures
>>> memory is allocated above 4G. Unless powerpc has some requirement for
>>> strictly having the crashkernel below 4G, I think it will make a lot of
>>> sense to enable support for this feature. So KHO users can specify this
>>> to get crashkernel working with KHO.
>> I agree that this is one way to work around the low-memory reservation problem.
>> However, there are a few things that come into play here:
>>
>> 1. On powerpc, the crashkernel reservation can go up to 64 GB for kdump. With
>> the
>>     current default scratch memory reservation policy, this could result in
>> reserving
>>     up to 256 GB of scratch memory: 200% for the high-memory reservation and
>> another
>>     200% for per-node memory.
> That calculation looks off. It _should_ be 200% once not twice. So 128
> GB total. If the allocation came out via the global area, it should
> _only_ be accounted to the global scratch size. Similarly, only the
> allocations made specifically on that node should be counted for the
> per-node scratch size.

For example, if a system has only one node and 64 GB is allocated from
that node before the kernel starts calculating the per-node and global
allocations for scratch memory, wouldn't the per-node allocation also be 
64 GB?

If so, wouldn't that result in 200% of 64 GB being allocated for the global
area and another 200% of 64 GB for the per-node area, resulting in 256 GB
of total scratch memory allocation? Or am I missing something here?


>
> But I have also noticed this problem on some of the systems Google has.
> Which makes me wonder if scratch_size_update() is broken and
> over-calculating. I have this on my TODO list and have been meaning to
> look into it, but other things keep intervening.
>
> If you are interested, feel free to take it off my hands.

Yes, I can take this up and propose patches to make crashkernel and
scratch reservations work together.

Based on my current testing, a Linux partition (powerpc) with 16 CPUs 
and 30 GB
of RAM needs only 16 MB of scratch memory in the low-memory area when
crashkernel=xxM is not specified.

16 MB is not much. I am also trying to get a larger Linux partition with 
1000+
CPUs to get a better idea of the limits for low-memory reservations.

BTW, do you know the rationale behind the 200% value?

I couldn't find any explanation for it in the commit message of
3dc92c311498c ("kexec: add Kexec HandOver (KHO) generation helpers")


>
>>     For fadump, which is the powerpc-specific memory dump capture mechanism, the
>> crashkernel
>>     reservation can go up to 180 GB. In this case, we could end up reserving up
>> to 720 GB of
>>     scratch memory, which is too much. I agree that users can tune this, but I
>> think the
>>     default scale should be more reasonable for powerpc.
> Once we fix scratch_size_update() to actually use 200% and not 400%,
> perhaps that alone will be enough? If not, we can discuss reducing the
> default scratch scale to maybe 150%. But I'd rather do it for all
> platforms if we do it at all, because this problem doesn't seem specific
> to PowerPC.

Yes, it makes sense to have a general fix that works for all architectures.

BTW, I was able to reproduce this issue on x86 as well. Please have a
look at this:

https://lore.kernel.org/all/008fe00e-fd52-4010-86ca-f0ab80a65a46@linux.ibm.com/ 
<https://lore.kernel.org/all/008fe00e-fd52-4010-86ca-f0ab80a65a46@linux.ibm.com/>

I have also suggested an approach to handle this issue which is similar 
how you handle
huge pages. Please share your thoughts on it.


>
>>
>> 2. Fadump also uses the crashkernel kernel command-line argument, but its
>> reservation policy
>>     is different from kdump. The crashkernel base address starts after the memory
>> needed for
>>     fadump. For example, with crashkernel=3G, the base address would be 3 GB, and
>> the crashkernel
>>     reservation would be from 3 GB to 6 GB. So, depending on the crashkernel
>> size, the reservation
>>     may or may not fall within low memory. Also, fadump does not support
>> crashkernel=xxM,high.
>>
>> 3. I do have a patch [1] to support high crashkernel reservations with kdump,
>> but this would not
>>     work with Hash MMU. With Hash MMU, the kernel image is constrained to low
>> memory, whereas with
>>     crashkernel=,high, all segments would be loaded into high memory.
> Oh, nice!
>
>> So, while supporting crashkernel=,high can help address the low-memory
>> reservation issue for kdump,
>> I think there are still some powerpc-specific constraints to consider. I would
>> like to explore
>> whether we can fix the ordering between crashkernel and scratch memory
>> reservations, so that we can
>> avoid unnecessarily large scratch memory reservations and address some of the
>> other constraints mentioned
>> above. Please share your thoughts.
>>
>>> Powerpc doesn't support this right now, but from a quick skim of the
>>> code, I think it should be simple enough.
>> I have patch series under review for the same:
>> [1]
>> https://lore.kernel.org/all/20260708143357.673251-1-sourabhjain@linux.ibm.com/
>>
>>> From
>>> arch_reserve_crashkernel() you just need to pass a bool * to
>>> parse_crashkernel(), and then pass the result to
>>> reserve_crashkernel_generic().
>> Due to some architecture-specific dependencies (such as RTAS), booting the
>> kernel from above 4G with
>> support for high crashkernel is not as straightforward as on other
>> architectures. Patch 2/4 in [1] has the
>> details.
>>
>>> Solving the ordering of crash reservations and KHO is tricky and comes
>>> with some difficult tradeoffs. Allocating crash from highmem should be a
>>> lot simpler.
>>
>> Could you please elaborate a bit on what makes the ordering tricky and what the
>> main tradeoffs are between crash reservations and KHO? It would help me better
>> understand the concerns here.
> The problem today is that kho_preserved_memory_reserve() (called by
> kho_mem_retrieve()) does a memblock_reserve() for each preserved folio.
> So if you have a lot of order-0 (or, 4k) folios, you end up with a lot
> of reservations in memblock. The large number of reservations can slow
> down later memblock operations like allocations too since memblock might
> have to walk through a lot of ranges to find free memory.
>
> We kind of work around this problem by calling kho_mem_retrieve() as
> pretty much the last thing in the MM init. So all allocations prior to
> this have already been fulfilled from scratch without any of the
> reservations added, so it should be pretty fast. You only take the
> performance hit at the end, where the only thing left is to release
> pages to buddy.

Ah, okay, that makes sense. Thanks for the clarification.


> Even then, the memblock reservations can get pretty damn slow. In some
> of my testing with under-load systems, preserving a 2G memfd with 4k
> pages can go over **5 minutes** in only kho_mem_retrieve() if the folios
> of the memfd are fragmented enough. Plus there is the memory overhead of
> the regions in memblock.reserved.

5 minutes in kho_mem_retrieve(), which is primarily marking a bunch
of memory as reserved using memblock, seems like quite a lot. If you
have the test case handy somewhere, I would be interested in trying it
myself, just to get a better feel for the issue.

Regardless, I understand the concern now. From my perspective also, the
current ordering of crashkernel and scratch memory reservations seems
reasonable, because crashkernel is not as flexible as scratch reservation
atleast on powerpc.

On powerpc, the crashkernel offset is determined first, and the
corresponding memory region is reserved. To make sure that no
other reservation falls within the crashkernel region, the crashkernel
reservation is one of the first reservations we make on powerpc.

If we change this ordering, there is a possibility that a scratch
reservation could end up in a region where the crashkernel is supposed
to be placed. That would lead to crashkernel reservation failure.

Also, reserving scratch memory at a location where the crashkernel
cannot be placed could be problematic. Each architecture has its own
constraints on where the crashkernel can be placed, so the available
memory for scratch reservation may need to account for those constraints.
Which I think too much to take care off...

And, of course, moving kho_mem_retrieve()earlier during boot would
also mean taking the performance hit you mentioned earlier.

So let's keep the current ordering and find a way to make both
reservations work with it: reserve the crashkernel first, and then
reserve the scratch memory.


> So long-term, I would like to get rid of the memblock reservations
> entirely and use scratch-only mode all the way until buddy comes up. And
> I would like to modify buddy init (free_low_memory_core_early() and
> deferred_init_memmap_chunk()) to be KHO-aware and directly skip the KHO
> pages.
>
> This vision goes in the opposite direction of turning scratch-only mode
> off _earlier_. And turning off scratch-only mode earlier is necessary
> for doing crash reservations outside of scratch.
>
> That is the tradeoff I mentioned. Hope I was clear enough.

Yes, that makes sense. I understand the tradeoff you're pointing out
now.

>
>>> And on that note, I don't think you should do a depends on !CRASH_DUMP.
>>> Even when CONFIG_KEXEC_HANDOVER is enabled, KHO isn't on by default
>>> (well, unless KEXEC_HANDOVER_ENABLE_DEFAULT is set). You need to enable
>>> it via cmdline. So it is entirely possible for people using KHO on PPC
>>> to not use crash and vice versa. This decision can be made at deployment
>>> time, not at compile time.
>> Agree. depends on !CRASH_DUMP is temporary and will be removed once
>> we settle the crashkernel reservation and scratch region handling.
> My point is that !CRASH_DUMP can be removed _even if_ we don't settle
> the reservation thing, because it is perfectly valid for the same kernel
> to use either KHO or crash but not at the same time. These both can be
> enabled/disabled at runtime.

Agree I will drop the !CRASH_DUMP dependency...

- Sourabh Jain

Re: [RFC PATCH 2/3] powerpc: add support for Kexec HandOver (KHO)

[Pratyush Yadav]

On Fri, Sep 04 2026, Sourabh Jain wrote:

> On 02/09/26 16:04, Pratyush Yadav wrote:
>> On Sun, Aug 23 2026, Sourabh Jain wrote:
>>
>>> On 21/08/26 17:26, Pratyush Yadav wrote:
>>>> On Fri, Aug 21 2026, Sourabh Jain wrote:
[...]
>>> I agree that this is one way to work around the low-memory reservation problem.
>>> However, there are a few things that come into play here:
>>>
>>> 1. On powerpc, the crashkernel reservation can go up to 64 GB for kdump. With
>>> the
>>>     current default scratch memory reservation policy, this could result in
>>> reserving
>>>     up to 256 GB of scratch memory: 200% for the high-memory reservation and
>>> another
>>>     200% for per-node memory.
>> That calculation looks off. It _should_ be 200% once not twice. So 128
>> GB total. If the allocation came out via the global area, it should
>> _only_ be accounted to the global scratch size. Similarly, only the
>> allocations made specifically on that node should be counted for the
>> per-node scratch size.
>
> For example, if a system has only one node and 64 GB is allocated from
> that node before the kernel starts calculating the per-node and global
> allocations for scratch memory, wouldn't the per-node allocation also be 64 GB?
>
> If so, wouldn't that result in 200% of 64 GB being allocated for the global
> area and another 200% of 64 GB for the per-node area, resulting in 256 GB
> of total scratch memory allocation? Or am I missing something here?

It shouldn't. If the 64 GB of allocation was done with NUMA_NO_NODE, and
it _happened_ to land on node X, it should not be counted for per-node
sizing. It should count towards the global pool. Only allocations that
were explicitly requested with node X should be count for that node's
scratch size.

So on a one node system where 64G of memory is allocated with
NUMA_NO_NODE and 8G is allocated with node X, we should get 128G of
global scratch and 16G of per-node scratch, giving us a total of 144G.

I took a quick look and it looks like the problem might be that the
calculation for global scratch includes _all_ nodes in it. See
memblock_reserved_kern_size():

	for_each_reserved_mem_region(r) {
		...
                
		if (nid == memblock_get_region_node(r) || !numa_valid_node(nid))
			if (r->flags & MEMBLOCK_RSRV_KERN)
				total += size;
	}

And for global scratch we pass nid as NUMA_NO_NODE.

For KHO we could just drop the || !numa_valid_node(), but
memblock_estimated_nr_free_pages() seems to depend on that behaviour. It
wants to get _all_ allocations across all nodes. KHO only wants
allocations explicitly made with NUMA_NO_NODE.

But disclaimer: all this is from reading the code for maybe 15 minutes.
I didn't run anything and might be missing something. So please
double-check what I am saying.

Not sure how to fix this. Since memblock_estimated_nr_free_pages() needs
all the reservations anyway, perhaps open code a simple counting loop
there? And the drop the || !numa_valid_node() from
memblock_reserved_kern_size().

But yeah, it would be much appreciated if you'd care to fix this.

The fix should be a separete patch, since it fixes problems on all
platforms, and not just PowerPC.

>> But I have also noticed this problem on some of the systems Google has.
>> Which makes me wonder if scratch_size_update() is broken and
>> over-calculating. I have this on my TODO list and have been meaning to
>> look into it, but other things keep intervening.
>>
>> If you are interested, feel free to take it off my hands.
>
> Yes, I can take this up and propose patches to make crashkernel and
> scratch reservations work together.
>
> Based on my current testing, a Linux partition (powerpc) with 16 CPUs and 30 GB
> of RAM needs only 16 MB of scratch memory in the low-memory area when
> crashkernel=xxM is not specified.
>
> 16 MB is not much. I am also trying to get a larger Linux partition with 1000+
> CPUs to get a better idea of the limits for low-memory reservations.
>
> BTW, do you know the rationale behind the 200% value?
>
> I couldn't find any explanation for it in the commit message of
> 3dc92c311498c ("kexec: add Kexec HandOver (KHO) generation helpers")

We need to ask Alex (or maybe Mike?; I forget who added this).

But if I were to guess, I don't think there is much science involved
behind the number. Since the scratch lives across all kexecs, it needs
to be large enough in case the next kernel uses more memory. 200% sounds
"large enough".

>>>     For fadump, which is the powerpc-specific memory dump capture mechanism, the
>>> crashkernel
>>>     reservation can go up to 180 GB. In this case, we could end up reserving up
>>> to 720 GB of
>>>     scratch memory, which is too much. I agree that users can tune this, but I
>>> think the
>>>     default scale should be more reasonable for powerpc.
>> Once we fix scratch_size_update() to actually use 200% and not 400%,
>> perhaps that alone will be enough? If not, we can discuss reducing the
>> default scratch scale to maybe 150%. But I'd rather do it for all
>> platforms if we do it at all, because this problem doesn't seem specific
>> to PowerPC.
>
> Yes, it makes sense to have a general fix that works for all architectures.
>
> BTW, I was able to reproduce this issue on x86 as well. Please have a
> look at this:
>
> https://lore.kernel.org/all/008fe00e-fd52-4010-86ca-f0ab80a65a46@linux.ibm.com/
>
> I have also suggested an approach to handle this issue which is similar how you
> handle
> huge pages. Please share your thoughts on it.

I missed this.

We can exclude HugeTLB pages from scratch accounting because the series
updates HugeTLB to use a new routine called memblock_alloc_hugetlb() to
allocate pages. This special allocator makes sure the pages are
_outside_ of scratch even if scratch-only mode is used. Since HugeTLB
pages come outside of scratch, they don't get counted in scratch sizing.

We need to do this for HugeTLB mainly because we want to preserve
HugeTLB pages in the future, and pages from scratch can't be preserved.

We could perhaps do so for crash as well, but I need to think more about
this.

But at first glance, if you need to allocate crash super early, perhaps
memblock won't be able to cope. Because allocating crash outside of
scratch would depend on kho_extend_scratch() and that needs
memblock_allow_resize() to be called to be able to cope with multiple
memory regions.

[...]
>>> Could you please elaborate a bit on what makes the ordering tricky and what the
>>> main tradeoffs are between crash reservations and KHO? It would help me better
>>> understand the concerns here.
>> The problem today is that kho_preserved_memory_reserve() (called by
>> kho_mem_retrieve()) does a memblock_reserve() for each preserved folio.
>> So if you have a lot of order-0 (or, 4k) folios, you end up with a lot
>> of reservations in memblock. The large number of reservations can slow
>> down later memblock operations like allocations too since memblock might
>> have to walk through a lot of ranges to find free memory.
>>
>> We kind of work around this problem by calling kho_mem_retrieve() as
>> pretty much the last thing in the MM init. So all allocations prior to
>> this have already been fulfilled from scratch without any of the
>> reservations added, so it should be pretty fast. You only take the
>> performance hit at the end, where the only thing left is to release
>> pages to buddy.
>
> Ah, okay, that makes sense. Thanks for the clarification.
>
>
>> Even then, the memblock reservations can get pretty damn slow. In some
>> of my testing with under-load systems, preserving a 2G memfd with 4k
>> pages can go over **5 minutes** in only kho_mem_retrieve() if the folios
>> of the memfd are fragmented enough. Plus there is the memory overhead of
>> the regions in memblock.reserved.
>
> 5 minutes in kho_mem_retrieve(), which is primarily marking a bunch
> of memory as reserved using memblock, seems like quite a lot. If you
> have the test case handy somewhere, I would be interested in trying it
> myself, just to get a better feel for the issue.

I do, but unfortunately based on downstream code so it is neither useful
to you nor something I can share I think.

But your friendly neighbourhood LLM can help here. Ask it to preserve
you a memfd but fragment/shatter buddy blocks first. That's pretty much
how I wrote my test.

But also see [0] which fixes the problem. Maybe Tarun (+Cc) has a test
based on upstream that he can share?

[0] https://lore.kernel.org/kexec/20260903155907.1065681-1-tarunsahu@google.com/

>
> Regardless, I understand the concern now. From my perspective also, the
> current ordering of crashkernel and scratch memory reservations seems
> reasonable, because crashkernel is not as flexible as scratch reservation
> atleast on powerpc.
>
> On powerpc, the crashkernel offset is determined first, and the
> corresponding memory region is reserved. To make sure that no
> other reservation falls within the crashkernel region, the crashkernel
> reservation is one of the first reservations we make on powerpc.
>
> If we change this ordering, there is a possibility that a scratch
> reservation could end up in a region where the crashkernel is supposed
> to be placed. That would lead to crashkernel reservation failure.
>
> Also, reserving scratch memory at a location where the crashkernel
> cannot be placed could be problematic. Each architecture has its own
> constraints on where the crashkernel can be placed, so the available
> memory for scratch reservation may need to account for those constraints.
> Which I think too much to take care off...

That's a real problem. But I am hoping the restrictions are something
along the lines of "crash kernel must be in lowmem", so the lowmem
scratch already solves that problem?

>
> And, of course, moving kho_mem_retrieve()earlier during boot would
> also mean taking the performance hit you mentioned earlier.
>
> So let's keep the current ordering and find a way to make both
> reservations work with it: reserve the crashkernel first, and then
> reserve the scratch memory.
[...]

-- 
Regards,
Pratyush Yadav