Re: [PATCH 2/2] kho: fix bootmem sizing

[Pratyush Yadav <pratyush@kernel.org>]

On Tue, Nov 03 2026, Mike Rapoport wrote:

> On Tue, Nov 03, 2026 at 05:01:08PM +0100, Pratyush Yadav wrote:
>> Hi Mike,
>> 
>> On Tue, Nov 03 2026, Mike Rapoport (Microsoft) wrote:
>> 
>> > Bootmem areas are sized from overlapping views of the memory reserved in
>> > memblock:
>> >
>> > * the global area is sized from all reserved memory, including memory
>> >   allocated on specific nodes, so on NUMA systems everything allocated
>> >   on a node is accounted in both the global area and the area of that
>> >   node.
>> > * per-node sizes are calculated after the lowmem and the global areas
>> >   are reserved. With CONFIG_NUMA=n all memblock regions belong to node 0,
>> >   so the lowmem and the global areas are accounted in the size of node 0
>> >   area and scaled once again.
>> >
>> > As a result, bootmem is about twice bigger than it should be on NUMA
>> > systems and about four times bigger with CONFIG_NUMA=n. On small systems
>> > reservation of the bootmem may fail and then KHO gets disabled.
>> >
>> > Make the areas cover separate parts of the reserved memory: the lowmem
>> > area covers allocations below ARCH_LOW_ADDRESS_LIMIT, per-node areas
>> > cover allocations above it on each node and the global area covers
>> > allocations that are not bound to any node. Calculate all sizes before
>> > reserving any of the areas.
>> >
>> > With CONFIG_NUMA=n the global area becomes empty and it is not reserved.
>> >
>> > Bootmem sizes in MiB in x86 QEMU VMs, before -> after:
>> >
>> >                      CONFIG_NUMA=y     CONFIG_NUMA=n
>> >   1G,  1 node         176 ->   92       320 ->   80
>> >   4G,  1 node         508 ->  332      1264 ->  316
>> >   16G, 2 nodes       1268 ->  704      2944 ->  736
>> >   32G, 4 nodes       2296 -> 1224      5196 -> 1300
>> >   64G, 4 nodes       4344 -> 2248      9664 -> 2420
>> >
>> > Fixes: 3dc92c311498 ("kexec: add Kexec HandOver (KHO) generation helpers")
>> > Reported-by: Kexin Liu <liukexin@kylinos.cn>
>> > Reported-by: George Guo <guodongtai@kylinos.cn>
>> > Closes: https://lore.kernel.org/all/20260904025101.9959-1-dongtai.guo@linux.dev
>> > Reported-by: Sourabh Jain <sourabhjain@linux.ibm.com>
>> > Closes: https://lore.kernel.org/all/20260922131217.698809-1-sourabhjain@linux.ibm.com
>> > Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
>> 
>> Thanks for the patch! My LLM caught an interesting problems that is
>> worth considering.
>> 
>> On arm64, ARCH_LOW_ADDRESS_LIMIT is arm64_dma_phys_limit - 1, and
>> dma_limits_init() sets the default value to PHYS_MASK + 1, and it can
>> also be memblock_end_of_dram() in some cases. So on arm64, it is
>> possible for lowmem to span the entire RAM.
>> 
>> Currently, that results in over-allocation of bootmem, and it ends up
>> being 4x what it should be. While obviously problematic, it should still
>> work. With this patch, all memory is in lowmem, and nothing ever goes to
>> per-node bootmem (except for the node that hosts the lowmem area).
>> 
>> All this is from reading the code, I haven't tried anything out yet.
>> 
>> Not sure how to fix this though. Should we perhaps use
>> min(ARCH_LOW_ADDRESS_LIMIT, U32_MAX) for the lowmem limit?
>
> Argh :)
>
> Maybe we reset lowmem to zero if memblock_end_of_DRAM() <
> ARCH_LOW_ADDRESS_LIMIT?

Sounds good.

>  
>> Also, a couple nitpicks below.
>> 
>> > ---
>> >  kernel/liveupdate/kexec_handover.c | 83 ++++++++++++++++++++------------------
>> >  1 file changed, 44 insertions(+), 39 deletions(-)
>> >
>> > +
>> > +		lowmem = reserved_mem_size(ARCH_LOW_ADDRESS_LIMIT, NUMA_NO_NODE);
>> > +		global = reserved_mem_size(MEMBLOCK_ALLOC_ANYWHERE, NUMA_NO_NODE);
>> > +		global -= lowmem;
>> 
>> Nit: Please add a short comment above this to explain why you remove
>> lowmem from global.
>
> I prefer to extend the comment above the function to say that calculations
> for global and node that includes lowmem remove that.

Sure, that works too.

-- 
Regards,
Pratyush Yadav