Re: [PATCH 2/2] kho: fix bootmem sizing

[Mike Rapoport <rppt@kernel.org>]

On Tue, Nov 03, 2026 at 05:01:08PM +0100, Pratyush Yadav wrote:
> Hi Mike,
> 
> On Tue, Nov 03 2026, Mike Rapoport (Microsoft) wrote:
> 
> > Bootmem areas are sized from overlapping views of the memory reserved in
> > memblock:
> >
> > * the global area is sized from all reserved memory, including memory
> >   allocated on specific nodes, so on NUMA systems everything allocated
> >   on a node is accounted in both the global area and the area of that
> >   node.
> > * per-node sizes are calculated after the lowmem and the global areas
> >   are reserved. With CONFIG_NUMA=n all memblock regions belong to node 0,
> >   so the lowmem and the global areas are accounted in the size of node 0
> >   area and scaled once again.
> >
> > As a result, bootmem is about twice bigger than it should be on NUMA
> > systems and about four times bigger with CONFIG_NUMA=n. On small systems
> > reservation of the bootmem may fail and then KHO gets disabled.
> >
> > Make the areas cover separate parts of the reserved memory: the lowmem
> > area covers allocations below ARCH_LOW_ADDRESS_LIMIT, per-node areas
> > cover allocations above it on each node and the global area covers
> > allocations that are not bound to any node. Calculate all sizes before
> > reserving any of the areas.
> >
> > With CONFIG_NUMA=n the global area becomes empty and it is not reserved.
> >
> > Bootmem sizes in MiB in x86 QEMU VMs, before -> after:
> >
> >                      CONFIG_NUMA=y     CONFIG_NUMA=n
> >   1G,  1 node         176 ->   92       320 ->   80
> >   4G,  1 node         508 ->  332      1264 ->  316
> >   16G, 2 nodes       1268 ->  704      2944 ->  736
> >   32G, 4 nodes       2296 -> 1224      5196 -> 1300
> >   64G, 4 nodes       4344 -> 2248      9664 -> 2420
> >
> > Fixes: 3dc92c311498 ("kexec: add Kexec HandOver (KHO) generation helpers")
> > Reported-by: Kexin Liu <liukexin@kylinos.cn>
> > Reported-by: George Guo <guodongtai@kylinos.cn>
> > Closes: https://lore.kernel.org/all/20260904025101.9959-1-dongtai.guo@linux.dev
> > Reported-by: Sourabh Jain <sourabhjain@linux.ibm.com>
> > Closes: https://lore.kernel.org/all/20260922131217.698809-1-sourabhjain@linux.ibm.com
> > Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
> 
> Thanks for the patch! My LLM caught an interesting problems that is
> worth considering.
> 
> On arm64, ARCH_LOW_ADDRESS_LIMIT is arm64_dma_phys_limit - 1, and
> dma_limits_init() sets the default value to PHYS_MASK + 1, and it can
> also be memblock_end_of_dram() in some cases. So on arm64, it is
> possible for lowmem to span the entire RAM.
> 
> Currently, that results in over-allocation of bootmem, and it ends up
> being 4x what it should be. While obviously problematic, it should still
> work. With this patch, all memory is in lowmem, and nothing ever goes to
> per-node bootmem (except for the node that hosts the lowmem area).
> 
> All this is from reading the code, I haven't tried anything out yet.
> 
> Not sure how to fix this though. Should we perhaps use
> min(ARCH_LOW_ADDRESS_LIMIT, U32_MAX) for the lowmem limit?

Argh :)

Maybe we reset lowmem to zero if memblock_end_of_DRAM() <
ARCH_LOW_ADDRESS_LIMIT?
 
> Also, a couple nitpicks below.
> 
> > ---
> >  kernel/liveupdate/kexec_handover.c | 83 ++++++++++++++++++++------------------
> >  1 file changed, 44 insertions(+), 39 deletions(-)
> >
> > +
> > +		lowmem = reserved_mem_size(ARCH_LOW_ADDRESS_LIMIT, NUMA_NO_NODE);
> > +		global = reserved_mem_size(MEMBLOCK_ALLOC_ANYWHERE, NUMA_NO_NODE);
> > +		global -= lowmem;
> 
> Nit: Please add a short comment above this to explain why you remove
> lowmem from global.

I prefer to extend the comment above the function to say that calculations
for global and node that includes lowmem remove that.
 
> -- 
> Regards,
> Pratyush Yadav

-- 
Sincerely yours,
Mike.