Re: [PATCH] kho: Skip zero-sized bootmem areas

[Chen Pei <cp0613@linux.alibaba.com>]

> Do you see zero sized lowmem or global scratch in practice? If yes,
> how? What's your setup?

Yes. Enabling KHO for riscv64 gave me three cases. They differ in which
size ends up 0 and in what it costs, and all three are measured:

  #  condition                     size    cost
  1  all memory below the limit    global  empty area kept
  2  no memory below the limit     lowmem  KHO off
  3  kho_bootmem= with explicit 0  either  empty area kept

Case 1, measured with a temporary pr_info(), not part of the patch.
riscv64 defconfig plus KEXEC_HANDOVER, cmdline "kho=on", four RAM sizes:

  mem   reserved_low   reserved_any   lowmem       global
  1G    0x32d3ea5      0x32d3ea5      0x6800000    0x0
  2G    0x4557164      0x4557165      0x8c00000    0x400000
  4G    0x4000000      0xa7116e5      0x8000000    0xd000000
  8G    0x4000000      0xed121e5      0x8000000    0x15c00000

riscv does not define ARCH_LOW_ADDRESS_LIMIT, so it uses the 0xffffffff
default and every reservation is below it; the two queries then agree
and the global size is a number scaled minus itself. The 1 GiB case is
exactly 0, and from 2 GiB up a single byte of difference between the two
queries is enough to leave the 4 MiB minimum after rounding up. The 0 is
recorded, not just computed - /sys/kernel/debug/kho/out/ on an
unpatched 1 GiB configuration reads

  bootmem_phys:  0xb6400000  0xbcc00000  0xa2c00000
  bootmem_len:   0x6800000   0x0         0x13800000

with the middle pair gone after the patch.

Case 2 is the one that actually costs something. On a riscv64 platform
whose memory all starts above the 0xffffffff default limit, unpatched
and with memblock=debug:

  memblock_phys_alloc_range: 0 bytes align=0x400000 from=0x0
                             max_addr=0x00000000ffffffff
  KHO: Failed to reserve lowmem bootmem
  KHO: Failed to reserve bootmem, disabling kexec handover

The size is 0 because nothing is reserved below a limit that has no
memory under it, and the range below the limit is empty as well, so the
allocation returns 0 and kho_enable is cleared. KHO ends up off on a
machine that needed no lowmem area at all, and /sys/kernel/debug/kho is
not there. With the skip the same machine comes up with two areas and no
error:

  bootmem_phys:  0x1000f3400000  0x1000ca000000
  bootmem_len:   0xdc00000       0x29400000

Case 3 is a cheap way to reproduce the recording of an empty area on any
architecture: the parser rejects the explicit sizes only when their sum
is 0, so "kho_bootmem=0,128M,128M" is accepted and the zero reaches the
allocator. On an unpatched kernel it gives

  bootmem_phys:  0xbcc00000  0xb4c00000  0xacc00000
  bootmem_len:   0x0         0x8000000   0x8000000

and with the patch it stops at the two real areas.

> Note that I already fixed this for per-node areas since those do end
> up with no allocations with movable_node set [0].

That one covers the per-node loop only; lowmem and global are still
unconditional in linux-next today, which is what cases 1, 2 and 3 above
are about. Is there anything I have not considered here? Please point
it out and I will follow up accordingly.

Best regards,
Chen