Re: [RFC PATCH v2] mm: Improve pgdat_balanced() to avoid over-reclamation for higher-order allocation

[Lian Wang <lianux.mm@gmail.com>]

From: "Lian Wang (ProcessMission)" <lianux.mm@gmail.com>

Hi Johannes, Barry, Baolin, Kairui, everyone,

We revisited the thread and Johannes's earlier analysis on the kswapd/kcompactd
handoff. We agree that a single static `compact_gap()` cannot represent
hundreds or thousands of concurrent higher-order requests. In such workloads,
repeated kswapd wakeups often reflect real unsatisfied demand, and stopping
kswapd too early can shift the burden to direct reclaim and direct compaction.

At the same time, related reclaim issues are still appearing in several
environments:

  - Barry observed order-3 network allocations causing persistent kswapd
    activity and device heating on phones, even though requesters can fall back
    quickly to order-0.
  - Baolin and Kairui have seen similar fragmentation-driven reclaim behavior
    on servers.
  - We have reproduced on PCs and VMs that order-0 memory and total available
    memory can remain plentiful while order-3 blocks are scarce, yet kswapd
    consumes substantial CPU and runs heavy reclaim/scan loops. We have not yet
    reproduced this consistently on our server systems.

There is a useful parallel in Johannes's production work on reclaim storms in
defrag_mode:

https://lore.kernel.org/20260626182215.1107966-1-hannes@cmpxchg.org/

The allocation paths differ, but both threads expose a similar feedback problem:
reclaim may report progress in the number of freed pages, while not making useful
allocation progress for the order, migratetype, or fallback intent that triggered
it.

> The compaction gap is sized for a single allocation, but
> kswapd/kcompactd are a shared resource for potentially hundreds or
> thousands of incoming requests.

We share this concern and do not want to replace the existing incremental
pipeline with one fixed value. We would like to separate two possibilities:

  1. One high-order kswapd wakeup overshoots `compact_gap()` substantially
     before `sc->order` is reset and kcompactd is woken.
  2. Each wakeup makes bounded progress, and the total overhead comes from
     repeated wakeups after best-effort callers have already fallen back to
     order-0.

For the first case, limiting the high-order reclaim scan or per-wakeup reclaim
budget when order-0 watermarks are already satisfied may preserve the repeated-
wakeup pipeline while avoiding a large single-pass overshoot. For the second,
we may need stronger accounting of outstanding higher-order demand, instead of
treating every repeated best-effort attempt as a request that still needs
background progress.

Would it make sense to align on a common, parameterized reproducer and trace set
before choosing between changing `pgdat_balanced()`, bounding the high-order
reclaim budget, or improving demand accounting? A useful matrix could cover:

  - best-effort order-3 allocation with immediate order-0 fallback;
  - sustained and concurrent THP or pageblock-order demand;
  - default mode and defrag_mode;
  - phone, PC/VM, and NUMA server environments.

We can share our reproducer and the PC/VM traces, and adapt the workload to
collect per-wakeup `nr_to_reclaim`, `nr_reclaimed`, `nr_scanned`, order reset,
kcompactd activity, allocation success, and fallback path details. It would be
very helpful if Barry, Baolin, Kairui, and Johannes can tell us which parts of
this matrix and which counters best represent the phone and server cases. Then
we can compare scenarios with aligned measurements and work toward a solution that
does not trade mobile reclaim storms for server direct-reclaim/compaction storms.

Thanks,
Lian