[PATCH v2 0/2] x86/crash: fix kexec_file_load(2) -EINVAL on machines with many possible CPUs

[PATCH v2 0/2] x86/crash: fix kexec_file_load(2) -EINVAL on machines with many possible CPUs

[Ionut Nechita (Wind River)]

From: Ionut Nechita <ionut.nechita@windriver.com>

Hi,

On x86 with CONFIG_CRASH_HOTPLUG=y and CONFIG_MEMORY_HOTPLUG=n, the
crash elfcorehdr segment is reserved for

    nr_mem_ranges + 2 + CONFIG_NR_CPUS_DEFAULT

program headers, while the header that crash_prepare_elf64_headers()
actually builds carries

    nr_mem_ranges + 2 + num_possible_cpus()

of them.  num_possible_cpus() is bounded by CONFIG_NR_CPUS, not by
CONFIG_NR_CPUS_DEFAULT, so on configs that raise CONFIG_NR_CPUS above
the arch default without MAXSMP the header outgrows its reservation once
the excess exceeds the page padding, and kexec_file_load(2) is rejected
by sanity_check_segment_list() with -EINVAL.  The classic kexec_load(2)
path is unaffected because user space builds the elfcorehdr without the
hotplug over-allocation.

Patch 1 reserves the elfcorehdr for CONFIG_NR_CPUS, the compile-time
upper bound of num_possible_cpus(), so the reservation always covers the
header that is actually generated.

Patch 2 is a documentation-only follow-up requested during v1 review: it
updates the two remaining places that still name NR_CPUS_DEFAULT when
describing the elfcorehdr sizing (the comment above
crash_handle_hotplug_event() and the CRASH_MAX_MEMORY_RANGES help text).

Based on linux-next (next-20260824, base-commit 4b18edbd8e70f).

Verified on a single-socket Xeon 6776P running a PREEMPT_RT kernel with:

    # CONFIG_MAXSMP is not set
    # CONFIG_MEMORY_HOTPLUG is not set
    CONFIG_NR_CPUS_RANGE_BEGIN=2
    CONFIG_NR_CPUS_RANGE_END=512
    CONFIG_NR_CPUS_DEFAULT=64
    CONFIG_NR_CPUS=256
    CONFIG_CRASH_MAX_MEMORY_RANGES=8192

- 144 possible CPUs: excess is (144 - 64) * 56 = 4480 bytes, more than
  the 4096 bytes of page padding, so 'kexec -p -s' fails with
  "kexec_file_load failed: Invalid argument"
- 72 possible CPUs (reduced via firmware): excess is (72 - 64) * 56 =
  448 bytes, still absorbed by the page rounding, and the load succeeds

Changes since v1:
- Patch 1/2: the changelog has been substantially rewritten.  v1
  described the reservation as "2 + CONFIG_NR_CPUS_DEFAULT" and did not
  explain where the CPU term goes missing.  The actual mechanism is that
  the @pnum the reservation builds on is crash_prepare_headers()'s
  @nr_mem_ranges out parameter - a memory range count, not a phdr count
  - so the CONFIG_NR_CPUS_DEFAULT term is the entire CPU allowance, and
  it is exceeded as soon as num_possible_cpus() > CONFIG_NR_CPUS_DEFAULT
  (modulo the page rounding on both sides).  The changelog now spells
  this out, including why the CONFIG_MEMORY_HOTPLUG=y path does not fail
  in practice with the default CONFIG_CRASH_MAX_MEMORY_RANGES=8192.
- Patch 1/2: the Fixes: tag was wrong in v1.  The two lines being
  changed were introduced by ea53ad9cf73b ("x86/crash: add x86 crash
  hotplug support"), not by a72bbec70da2 ("crash: hotplug support for
  kexec_load()") - which is the kexec_load(2) commit, i.e. the path this
  patch does not touch.  Corrected.
- Patch 1/2: added a paragraph on memory impact, and the reproducer now
  states CONFIG_MEMORY_HOTPLUG and CONFIG_CRASH_MAX_MEMORY_RANGES, which
  v1 omitted and which are needed to reproduce.
- Added patch 2/2 updating the stale NR_CPUS_DEFAULT references in the
  crash_handle_hotplug_event() comment (Jinjie Ruan) and in the
  CRASH_MAX_MEMORY_RANGES Kconfig help text (Bradley Morgan).
- Rebased from next-20260811 onto next-20260824.

Jinjie, Bradley: the code in patch 1/2 is byte-identical to v1, so I
carried your Reviewed-by tags, but given how much the changelog and the
Fixes: tag changed please re-check that you are still happy with them
and I will drop them if not.

On the num_possible_cpus() alternative raised by Jinjie in v1: it would
give a tighter reservation and powerpc's arch_crash_get_elfcorehdr_size()
already does exactly that.  I kept CONFIG_NR_CPUS here because it is the
minimal, easy-to-backport fix - it just swaps the wrong compile-time
constant for the correct compile-time upper bound, and keeps both
branches of the expression homogeneous alongside the equally
compile-time CONFIG_CRASH_MAX_MEMORY_RANGES term.  I am happy to respin
with num_possible_cpus() if maintainers prefer that.

v1: https://lore.kernel.org/lkml/20260812170433.533845-1-ionut.nechita@windriver.com/

Ionut Nechita (2):
  x86/crash: reserve elfcorehdr for CONFIG_NR_CPUS, not
    CONFIG_NR_CPUS_DEFAULT
  crash: update stale NR_CPUS_DEFAULT references in elfcorehdr sizing
    docs

 arch/x86/kernel/crash.c | 6 +++---
 kernel/Kconfig.kexec    | 2 +-
 kernel/crash_core.c     | 2 +-
 3 files changed, 5 insertions(+), 5 deletions(-)


base-commit: 4b18edbd8e70f7e6860d56370f13244896d0f95c
--
2.55.0

[PATCH v2 1/2] x86/crash: reserve elfcorehdr for CONFIG_NR_CPUS, not CONFIG_NR_CPUS_DEFAULT

[Ionut Nechita (Wind River)]

From: Ionut Nechita <ionut.nechita@windriver.com>

kexec_file_load(2) fails with -EINVAL when loading a crash kernel on a
machine whose number of possible CPUs exceeds CONFIG_NR_CPUS_DEFAULT,
even though the classic kexec_load(2) path succeeds on the same machine.

With CONFIG_CRASH_HOTPLUG=y the elfcorehdr segment is over-allocated so
it can be updated in place on CPU/memory hotplug.  On the
!CONFIG_MEMORY_HOTPLUG path, crash_load_segments() sizes that
reservation as:

	ret = crash_prepare_headers(..., &kbuf.bufsz, &pnum);
	...
	pnum += 2 + CONFIG_NR_CPUS_DEFAULT;

The value that lands in @pnum is crash_prepare_headers()'s
@nr_mem_ranges out parameter, i.e. cmem->nr_ranges - the number of
memory ranges only, not a phdr count.  The header that
crash_prepare_elf64_headers() actually builds adds one phdr per
*possible* CPU on top of those ranges:

	nr_phdr = nr_cpus + 1;		/* + vmcoreinfo */
	nr_phdr += mem->nr_ranges;
	nr_phdr++;			/* + kernel text map */

So the reservation covers

	nr_ranges + 2 + CONFIG_NR_CPUS_DEFAULT

phdrs while the buffer holds

	nr_ranges + 2 + num_possible_cpus()

phdrs, and the buffer exceeds the reservation by

	(num_possible_cpus() - CONFIG_NR_CPUS_DEFAULT) * sizeof(Elf64_Phdr)

bytes as soon as num_possible_cpus() grows past CONFIG_NR_CPUS_DEFAULT.
num_possible_cpus() is bounded by CONFIG_NR_CPUS, not by
CONFIG_NR_CPUS_DEFAULT, so this is reachable on any config that raises
CONFIG_NR_CPUS above the arch default without CONFIG_MAXSMP.

crash_prepare_elf64_headers() rounds bufsz up to ELF_CORE_HEADER_ALIGN
and kexec_add_buffer() rounds memsz up to PAGE_SIZE (both 4096), so the
excess is invisible until it outgrows that padding.  Once it does,
sanity_check_segment_list() rejects the image:

	if (image->segment[i].bufsz > image->segment[i].memsz)
		return -EINVAL;

kexec_load(2) is unaffected because user space builds the elfcorehdr
without the hotplug over-allocation.

Observed on a single-socket Xeon 6776P (144 possible CPUs) running a
PREEMPT_RT kernel with:

	# CONFIG_MAXSMP is not set
	# CONFIG_MEMORY_HOTPLUG is not set
	CONFIG_NR_CPUS_RANGE_BEGIN=2
	CONFIG_NR_CPUS_RANGE_END=512
	CONFIG_NR_CPUS_DEFAULT=64
	CONFIG_NR_CPUS=256

At 144 possible CPUs the buffer exceeds the reservation by
(144 - 64) * 56 = 4480 bytes.  That is more than the 4096 bytes of page
padding, so the overflow is guaranteed and kexec -p -s fails with
"kexec_file_load failed: Invalid argument".  Reducing the possible CPU
count to 72 leaves an excess of (72 - 64) * 56 = 448 bytes, which the
page rounding still absorbs, and the load succeeds - confirming the
reservation is the limiting factor.

Reserve the elfcorehdr for CONFIG_NR_CPUS, the compile-time upper bound
of num_possible_cpus(), so the reservation always covers the header that
is actually generated.

The CONFIG_MEMORY_HOTPLUG=y path discards @pnum and reserves
2 + CONFIG_NR_CPUS_DEFAULT + CONFIG_CRASH_MAX_MEMORY_RANGES phdrs
instead.  With the default CONFIG_CRASH_MAX_MEMORY_RANGES=8192 the
memory range allowance dwarfs the CPU shortfall, so that path does not
fail in practice; it is switched to CONFIG_NR_CPUS as well for
consistency and to stay correct for small CONFIG_CRASH_MAX_MEMORY_RANGES
values.

This does not change the reservation for defconfig-like builds, since
CONFIG_NR_CPUS defaults to CONFIG_NR_CPUS_DEFAULT.  Only configs that
raise CONFIG_NR_CPUS reserve more, and the worst case is bounded by the
top of the range (CONFIG_NR_CPUS=8192 with CONFIG_CPUMASK_OFFSTACK=y),
which is exactly what CONFIG_MAXSMP already reserves today.

Fixes: ea53ad9cf73b ("x86/crash: add x86 crash hotplug support")
Signed-off-by: Ionut Nechita <ionut.nechita@windriver.com>
Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com>
Reviewed-by: Bradley Morgan <include@grrlz.net>
---
 arch/x86/kernel/crash.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c
index e681ec9cf1dc..e6f23933a6df 100644
--- a/arch/x86/kernel/crash.c
+++ b/arch/x86/kernel/crash.c
@@ -369,9 +369,9 @@ int crash_load_segments(struct kimage *image)
 	 * maximum CPUs and maximum memory ranges.
 	 */
 	if (IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
-		pnum = 2 + CONFIG_NR_CPUS_DEFAULT + CONFIG_CRASH_MAX_MEMORY_RANGES;
+		pnum = 2 + CONFIG_NR_CPUS + CONFIG_CRASH_MAX_MEMORY_RANGES;
 	else
-		pnum += 2 + CONFIG_NR_CPUS_DEFAULT;
+		pnum += 2 + CONFIG_NR_CPUS;
 
 	if (pnum < (unsigned long)PN_XNUM) {
 		kbuf.memsz = pnum * sizeof(Elf64_Phdr);
@@ -430,7 +430,7 @@ unsigned int arch_crash_get_elfcorehdr_size(void)
 	unsigned int sz;
 
 	/* kernel_map, VMCOREINFO and maximum CPUs */
-	sz = 2 + CONFIG_NR_CPUS_DEFAULT;
+	sz = 2 + CONFIG_NR_CPUS;
 	if (IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
 		sz += CONFIG_CRASH_MAX_MEMORY_RANGES;
 	sz *= sizeof(Elf64_Phdr);

base-commit: 4b18edbd8e70f7e6860d56370f13244896d0f95c
-- 
2.55.0

[PATCH v2 2/2] crash: update stale NR_CPUS_DEFAULT references in elfcorehdr sizing docs

[Ionut Nechita (Wind River)]

From: Ionut Nechita <ionut.nechita@windriver.com>

The elfcorehdr over-allocation for crash hotplug is now computed from
CONFIG_NR_CPUS rather than CONFIG_NR_CPUS_DEFAULT, but two pieces of
documentation still name the old symbol: the comment above
crash_handle_hotplug_event() and the CRASH_MAX_MEMORY_RANGES help text.

Update both so they describe what the code actually does and do not
mislead people sizing CRASH_MAX_MEMORY_RANGES.

Documentation only, no functional change.

Suggested-by: Jinjie Ruan <ruanjinjie@huawei.com>
Suggested-by: Bradley Morgan <include@grrlz.net>
Signed-off-by: Ionut Nechita <ionut.nechita@windriver.com>
---
 kernel/Kconfig.kexec | 2 +-
 kernel/crash_core.c  | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/Kconfig.kexec b/kernel/Kconfig.kexec
index 15632358bcf7..a97ed9605602 100644
--- a/kernel/Kconfig.kexec
+++ b/kernel/Kconfig.kexec
@@ -167,7 +167,7 @@ config CRASH_MAX_MEMORY_RANGES
 	  memory regions that the elfcorehdr buffer/segment can accommodate.
 	  These regions are obtained via walk_system_ram_res(); eg. the
 	  'System RAM' entries in /proc/iomem.
-	  This value is combined with NR_CPUS_DEFAULT and multiplied by
+	  This value is combined with NR_CPUS and multiplied by
 	  sizeof(Elf64_Phdr) to determine the final elfcorehdr memory buffer/
 	  segment size.
 	  The value 8192, for example, covers a (sparsely populated) 1TiB system
diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index 2b36aa9fade0..d0bd2d0cf899 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -648,7 +648,7 @@ int crash_check_hotplug_support(void)
  * new list of CPUs and memory. To make changes to the elfcorehdr, it
  * should be large enough to permit a growing number of CPU and Memory
  * resources. One can estimate the elfcorehdr memory size based on
- * NR_CPUS_DEFAULT and CRASH_MAX_MEMORY_RANGES. The elfcorehdr is
+ * NR_CPUS and CRASH_MAX_MEMORY_RANGES. The elfcorehdr is
  * excluded from SHA verification by default if the architecture
  * supports crash hotplug.
  */
-- 
2.55.0

Re: [PATCH v2 1/2] x86/crash: reserve elfcorehdr for CONFIG_NR_CPUS, not CONFIG_NR_CPUS_DEFAULT

[Baoquan He]

On 08/25/26 at 10:50am, Ionut Nechita (Wind River) wrote:
> From: Ionut Nechita <ionut.nechita@windriver.com>
> 
> kexec_file_load(2) fails with -EINVAL when loading a crash kernel on a
> machine whose number of possible CPUs exceeds CONFIG_NR_CPUS_DEFAULT,
> even though the classic kexec_load(2) path succeeds on the same machine.
> 
> With CONFIG_CRASH_HOTPLUG=y the elfcorehdr segment is over-allocated so
> it can be updated in place on CPU/memory hotplug.  On the
> !CONFIG_MEMORY_HOTPLUG path, crash_load_segments() sizes that
> reservation as:
> 
> 	ret = crash_prepare_headers(..., &kbuf.bufsz, &pnum);
> 	...
> 	pnum += 2 + CONFIG_NR_CPUS_DEFAULT;
> 
> The value that lands in @pnum is crash_prepare_headers()'s
> @nr_mem_ranges out parameter, i.e. cmem->nr_ranges - the number of
> memory ranges only, not a phdr count.  The header that
> crash_prepare_elf64_headers() actually builds adds one phdr per
> *possible* CPU on top of those ranges:
> 
> 	nr_phdr = nr_cpus + 1;		/* + vmcoreinfo */
> 	nr_phdr += mem->nr_ranges;
> 	nr_phdr++;			/* + kernel text map */
> 
> So the reservation covers
> 
> 	nr_ranges + 2 + CONFIG_NR_CPUS_DEFAULT
> 
> phdrs while the buffer holds
> 
> 	nr_ranges + 2 + num_possible_cpus()
> 
> phdrs, and the buffer exceeds the reservation by
> 
> 	(num_possible_cpus() - CONFIG_NR_CPUS_DEFAULT) * sizeof(Elf64_Phdr)
> 
> bytes as soon as num_possible_cpus() grows past CONFIG_NR_CPUS_DEFAULT.
> num_possible_cpus() is bounded by CONFIG_NR_CPUS, not by
> CONFIG_NR_CPUS_DEFAULT, so this is reachable on any config that raises
> CONFIG_NR_CPUS above the arch default without CONFIG_MAXSMP.
> 
> crash_prepare_elf64_headers() rounds bufsz up to ELF_CORE_HEADER_ALIGN
> and kexec_add_buffer() rounds memsz up to PAGE_SIZE (both 4096), so the
> excess is invisible until it outgrows that padding.  Once it does,
> sanity_check_segment_list() rejects the image:
> 
> 	if (image->segment[i].bufsz > image->segment[i].memsz)
> 		return -EINVAL;
> 
> kexec_load(2) is unaffected because user space builds the elfcorehdr
> without the hotplug over-allocation.
> 
> Observed on a single-socket Xeon 6776P (144 possible CPUs) running a
> PREEMPT_RT kernel with:
> 
> 	# CONFIG_MAXSMP is not set
> 	# CONFIG_MEMORY_HOTPLUG is not set
> 	CONFIG_NR_CPUS_RANGE_BEGIN=2
> 	CONFIG_NR_CPUS_RANGE_END=512
> 	CONFIG_NR_CPUS_DEFAULT=64
> 	CONFIG_NR_CPUS=256
> 
> At 144 possible CPUs the buffer exceeds the reservation by
> (144 - 64) * 56 = 4480 bytes.  That is more than the 4096 bytes of page
> padding, so the overflow is guaranteed and kexec -p -s fails with
> "kexec_file_load failed: Invalid argument".  Reducing the possible CPU
> count to 72 leaves an excess of (72 - 64) * 56 = 448 bytes, which the
> page rounding still absorbs, and the load succeeds - confirming the
> reservation is the limiting factor.
> 
> Reserve the elfcorehdr for CONFIG_NR_CPUS, the compile-time upper bound
> of num_possible_cpus(), so the reservation always covers the header that
> is actually generated.
> 
> The CONFIG_MEMORY_HOTPLUG=y path discards @pnum and reserves
> 2 + CONFIG_NR_CPUS_DEFAULT + CONFIG_CRASH_MAX_MEMORY_RANGES phdrs
> instead.  With the default CONFIG_CRASH_MAX_MEMORY_RANGES=8192 the
> memory range allowance dwarfs the CPU shortfall, so that path does not
> fail in practice; it is switched to CONFIG_NR_CPUS as well for
> consistency and to stay correct for small CONFIG_CRASH_MAX_MEMORY_RANGES
> values.
> 
> This does not change the reservation for defconfig-like builds, since
> CONFIG_NR_CPUS defaults to CONFIG_NR_CPUS_DEFAULT.  Only configs that
> raise CONFIG_NR_CPUS reserve more, and the worst case is bounded by the
> top of the range (CONFIG_NR_CPUS=8192 with CONFIG_CPUMASK_OFFSTACK=y),
> which is exactly what CONFIG_MAXSMP already reserves today.
> 
> Fixes: ea53ad9cf73b ("x86/crash: add x86 crash hotplug support")
> Signed-off-by: Ionut Nechita <ionut.nechita@windriver.com>
> Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com>
> Reviewed-by: Bradley Morgan <include@grrlz.net>
> ---
>  arch/x86/kernel/crash.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c
> index e681ec9cf1dc..e6f23933a6df 100644
> --- a/arch/x86/kernel/crash.c
> +++ b/arch/x86/kernel/crash.c
> @@ -369,9 +369,9 @@ int crash_load_segments(struct kimage *image)
>  	 * maximum CPUs and maximum memory ranges.
>  	 */
>  	if (IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
> -		pnum = 2 + CONFIG_NR_CPUS_DEFAULT + CONFIG_CRASH_MAX_MEMORY_RANGES;
> +		pnum = 2 + CONFIG_NR_CPUS + CONFIG_CRASH_MAX_MEMORY_RANGES;
>  	else
> -		pnum += 2 + CONFIG_NR_CPUS_DEFAULT;
> +		pnum += 2 + CONFIG_NR_CPUS;
>  
>  	if (pnum < (unsigned long)PN_XNUM) {
>  		kbuf.memsz = pnum * sizeof(Elf64_Phdr);
> @@ -430,7 +430,7 @@ unsigned int arch_crash_get_elfcorehdr_size(void)
>  	unsigned int sz;
>  
>  	/* kernel_map, VMCOREINFO and maximum CPUs */
> -	sz = 2 + CONFIG_NR_CPUS_DEFAULT;
> +	sz = 2 + CONFIG_NR_CPUS;
>  	if (IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
>  		sz += CONFIG_CRASH_MAX_MEMORY_RANGES;
>  	sz *= sizeof(Elf64_Phdr);

I didn't dare to read the commit log, but judging from the code, it looks
like a good fix.

Acked-by: Baoquan He <baoquan.he@linux.dev>

> 
> base-commit: 4b18edbd8e70f7e6860d56370f13244896d0f95c
> -- 
> 2.55.0
> 

Re: [PATCH v2 2/2] crash: update stale NR_CPUS_DEFAULT references in elfcorehdr sizing docs

[Bradley Morgan]

On 25 August 2026 08:50:43 BST, "Ionut Nechita (Wind River)"
<ionut.nechita@windriver.com> wrote:
>From: Ionut Nechita <ionut.nechita@windriver.com>
>
>The elfcorehdr over-allocation for crash hotplug is now computed from
>CONFIG_NR_CPUS rather than CONFIG_NR_CPUS_DEFAULT, but two pieces of
>documentation still name the old symbol: the comment above
>crash_handle_hotplug_event() and the CRASH_MAX_MEMORY_RANGES help text.
>
>Update both so they describe what the code actually does and do not
>mislead people sizing CRASH_MAX_MEMORY_RANGES.
>
>Documentation only, no functional change.
>
>Suggested-by: Jinjie Ruan <ruanjinjie@huawei.com>
>Suggested-by: Bradley Morgan <include@grrlz.net>

Reviewed-by: Bradley Morgan <include@grrlz.net>

You shockingly didn't CC me!1!1

>Signed-off-by: Ionut Nechita <ionut.nechita@windriver.com>
>---
> kernel/Kconfig.kexec | 2 +-
> kernel/crash_core.c  | 2 +-
> 2 files changed, 2 insertions(+), 2 deletions(-)
>
>diff --git a/kernel/Kconfig.kexec b/kernel/Kconfig.kexec
>index 15632358bcf7..a97ed9605602 100644
>--- a/kernel/Kconfig.kexec
>+++ b/kernel/Kconfig.kexec
>@@ -167,7 +167,7 @@ config CRASH_MAX_MEMORY_RANGES
> 	  memory regions that the elfcorehdr buffer/segment can accommodate.
> 	  These regions are obtained via walk_system_ram_res(); eg. the
> 	  'System RAM' entries in /proc/iomem.
>-	  This value is combined with NR_CPUS_DEFAULT and multiplied by
>+	  This value is combined with NR_CPUS and multiplied by
> 	  sizeof(Elf64_Phdr) to determine the final elfcorehdr memory buffer/
> 	  segment size.
> 	  The value 8192, for example, covers a (sparsely populated) 1TiB system
>diff --git a/kernel/crash_core.c b/kernel/crash_core.c
>index 2b36aa9fade0..d0bd2d0cf899 100644
>--- a/kernel/crash_core.c
>+++ b/kernel/crash_core.c
>@@ -648,7 +648,7 @@ int crash_check_hotplug_support(void)
>  * new list of CPUs and memory. To make changes to the elfcorehdr, it
>  * should be large enough to permit a growing number of CPU and Memory
>  * resources. One can estimate the elfcorehdr memory size based on
>- * NR_CPUS_DEFAULT and CRASH_MAX_MEMORY_RANGES. The elfcorehdr is
>+ * NR_CPUS and CRASH_MAX_MEMORY_RANGES. The elfcorehdr is
>  * excluded from SHA verification by default if the architecture
>  * supports crash hotplug.
>  */
>


Thanks!

Re: [PATCH v2 1/2] x86/crash: reserve elfcorehdr for CONFIG_NR_CPUS, not CONFIG_NR_CPUS_DEFAULT

[Sourabh Jain]

On 25/08/26 13:20, Ionut Nechita (Wind River) wrote:
> From: Ionut Nechita <ionut.nechita@windriver.com>
>
> kexec_file_load(2) fails with -EINVAL when loading a crash kernel on a
> machine whose number of possible CPUs exceeds CONFIG_NR_CPUS_DEFAULT,
> even though the classic kexec_load(2) path succeeds on the same machine.

It was surprising because the kexec tool with kexec_load also uses the
same size for elfcorehdr, which is exported via 
/sys/kernel/crash_elfcorehdr_size.

Code snippet from load_crashdump_segments() - 
kexec/arch/i386/crashdump-x86.c:

|/* For hotplug suppoBut I think we should handle the above issue 
separately.rt, override the minimum necessary size just * computed with 
the value from /sys/kernel/crash_elfcorehdr_size. * Properly align the 
size as well. */ if (do_hotplug) { memsz = _ALIGN(elfcorehdrsz, align); }|

Then I found the following code in add_segment_phys_virt() - kexec/kexec.c:

|if (bufsz > memsz) { bufsz = memsz; }|

when adding the segment. This seems wrong to me. What is the point of 
finding
a memory hole smaller than bufsz? It seems like it should be memsz = 
bufsz instead.

This could be the reason you don't see the problem with the kexec_load 
system call.
The kexec tool is truncating bufsz while finding a hole of size memsz.

So, yes, you didn't observe this issue with the kexec_load syscall while 
loading the
kdump kernel. However, given that the elfcorehdr memsz is truncated, you 
may face
problems during dump collection or with the collected dump.

Another problem I see around setting memsz when crash hotplug support is 
enabled
in both the kernel and kexec tool is that memsz is being overridden 
without checking
its current size.

It is possible that the elfcorehdr buffer prepared by the kernel could 
be larger than the
size calculated statically from the kernel configuration.

So I think if kbuf.bufsz for elfcorehdr is larger than (pnum + 1) * 
(sizeof(Elf64_Phdr) , we
should skip updating kbuf.memsz.

With that said the changes introduce here looks good, so feel free to add:
Reviewed-by: Sourabh Jain <sourabhjain@linux.ibm.com>

But I think we should handle the above issues separately.

- Sourabh Jain
>
> With CONFIG_CRASH_HOTPLUG=y the elfcorehdr segment is over-allocated so
> it can be updated in place on CPU/memory hotplug.  On the
> !CONFIG_MEMORY_HOTPLUG path, crash_load_segments() sizes that
> reservation as:
>
> 	ret = crash_prepare_headers(..., &kbuf.bufsz, &pnum);
> 	...
> 	pnum += 2 + CONFIG_NR_CPUS_DEFAULT;
>
> The value that lands in @pnum is crash_prepare_headers()'s
> @nr_mem_ranges out parameter, i.e. cmem->nr_ranges - the number of
> memory ranges only, not a phdr count.  The header that
> crash_prepare_elf64_headers() actually builds adds one phdr per
> *possible* CPU on top of those ranges:
>
> 	nr_phdr = nr_cpus + 1;		/* + vmcoreinfo */
> 	nr_phdr += mem->nr_ranges;
> 	nr_phdr++;			/* + kernel text map */
>
> So the reservation covers
>
> 	nr_ranges + 2 + CONFIG_NR_CPUS_DEFAULT
>
> phdrs while the buffer holds
>
> 	nr_ranges + 2 + num_possible_cpus()
>
> phdrs, and the buffer exceeds the reservation by
>
> 	(num_possible_cpus() - CONFIG_NR_CPUS_DEFAULT) * sizeof(Elf64_Phdr)
>
> bytes as soon as num_possible_cpus() grows past CONFIG_NR_CPUS_DEFAULT.
> num_possible_cpus() is bounded by CONFIG_NR_CPUS, not by
> CONFIG_NR_CPUS_DEFAULT, so this is reachable on any config that raises
> CONFIG_NR_CPUS above the arch default without CONFIG_MAXSMP.
>
> crash_prepare_elf64_headers() rounds bufsz up to ELF_CORE_HEADER_ALIGN
> and kexec_add_buffer() rounds memsz up to PAGE_SIZE (both 4096), so the
> excess is invisible until it outgrows that padding.  Once it does,
> sanity_check_segment_list() rejects the image:
>
> 	if (image->segment[i].bufsz > image->segment[i].memsz)
> 		return -EINVAL;
>
> kexec_load(2) is unaffected because user space builds the elfcorehdr
> without the hotplug over-allocation.
>
> Observed on a single-socket Xeon 6776P (144 possible CPUs) running a
> PREEMPT_RT kernel with:
>
> 	# CONFIG_MAXSMP is not set
> 	# CONFIG_MEMORY_HOTPLUG is not set
> 	CONFIG_NR_CPUS_RANGE_BEGIN=2
> 	CONFIG_NR_CPUS_RANGE_END=512
> 	CONFIG_NR_CPUS_DEFAULT=64
> 	CONFIG_NR_CPUS=256
>
> At 144 possible CPUs the buffer exceeds the reservation by
> (144 - 64) * 56 = 4480 bytes.  That is more than the 4096 bytes of page
> padding, so the overflow is guaranteed and kexec -p -s fails with
> "kexec_file_load failed: Invalid argument".  Reducing the possible CPU
> count to 72 leaves an excess of (72 - 64) * 56 = 448 bytes, which the
> page rounding still absorbs, and the load succeeds - confirming the
> reservation is the limiting factor.
>
> Reserve the elfcorehdr for CONFIG_NR_CPUS, the compile-time upper bound
> of num_possible_cpus(), so the reservation always covers the header that
> is actually generated.
>
> The CONFIG_MEMORY_HOTPLUG=y path discards @pnum and reserves
> 2 + CONFIG_NR_CPUS_DEFAULT + CONFIG_CRASH_MAX_MEMORY_RANGES phdrs
> instead.  With the default CONFIG_CRASH_MAX_MEMORY_RANGES=8192 the
> memory range allowance dwarfs the CPU shortfall, so that path does not
> fail in practice; it is switched to CONFIG_NR_CPUS as well for
> consistency and to stay correct for small CONFIG_CRASH_MAX_MEMORY_RANGES
> values.
>
> This does not change the reservation for defconfig-like builds, since
> CONFIG_NR_CPUS defaults to CONFIG_NR_CPUS_DEFAULT.  Only configs that
> raise CONFIG_NR_CPUS reserve more, and the worst case is bounded by the
> top of the range (CONFIG_NR_CPUS=8192 with CONFIG_CPUMASK_OFFSTACK=y),
> which is exactly what CONFIG_MAXSMP already reserves today.
>
> Fixes: ea53ad9cf73b ("x86/crash: add x86 crash hotplug support")
> Signed-off-by: Ionut Nechita <ionut.nechita@windriver.com>
> Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com>
> Reviewed-by: Bradley Morgan <include@grrlz.net>
> ---
>   arch/x86/kernel/crash.c | 6 +++---
>   1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c
> index e681ec9cf1dc..e6f23933a6df 100644
> --- a/arch/x86/kernel/crash.c
> +++ b/arch/x86/kernel/crash.c
> @@ -369,9 +369,9 @@ int crash_load_segments(struct kimage *image)
>   	 * maximum CPUs and maximum memory ranges.
>   	 */
>   	if (IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
> -		pnum = 2 + CONFIG_NR_CPUS_DEFAULT + CONFIG_CRASH_MAX_MEMORY_RANGES;
> +		pnum = 2 + CONFIG_NR_CPUS + CONFIG_CRASH_MAX_MEMORY_RANGES;
>   	else
> -		pnum += 2 + CONFIG_NR_CPUS_DEFAULT;
> +		pnum += 2 + CONFIG_NR_CPUS;
>   
>   	if (pnum < (unsigned long)PN_XNUM) {
>   		kbuf.memsz = pnum * sizeof(Elf64_Phdr);
> @@ -430,7 +430,7 @@ unsigned int arch_crash_get_elfcorehdr_size(void)
>   	unsigned int sz;
>   
>   	/* kernel_map, VMCOREINFO and maximum CPUs */
> -	sz = 2 + CONFIG_NR_CPUS_DEFAULT;
> +	sz = 2 + CONFIG_NR_CPUS;
>   	if (IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
>   		sz += CONFIG_CRASH_MAX_MEMORY_RANGES;
>   	sz *= sizeof(Elf64_Phdr);
>
> base-commit: 4b18edbd8e70f7e6860d56370f13244896d0f95c