Re: [PATCH v4] mm/page_alloc: avoid direct compaction for costly __GFP_NORETRY allocations

[Dave Chinner <dgc@kernel.org>]

On Sat, Sep 05, 2026 at 05:42:39PM -0700, Andrew Morton wrote:
> On Fri, 4 Sep 2026 11:56:28 +0000 Salvatore Dipietro <dipiets@amazon.it> wrote:
> 
> > Commit 5d8edfb900d5 ("iomap: Copy larger chunks from userspace")
> > introduced high-order folio allocations in the iomap buffered write
> > path.  When memory is fragmented, each failed costly-order allocation
> > enters __alloc_pages_slowpath() which runs direct compaction and
> > drain_all_pages(), causing a 0.38x throughput drop on PostgreSQL
> > pgbench (simple-update) with 1024 clients on a 96-vCPU arm64 system.
> 
> Is there anything particularly unusual about this test case?
> 
> > Results (average of 3 runs, TPS):
> > 
> >   Config                   Avg TPS      % vs Baseline
> >   baseline (no patch)       59,408       -
> >   With this patch          155,409      +161.6%
> 
> Is this back to pre-5d8edfb900d5 performance?
> 
> > Link: https://lore.kernel.org/all/20260403193535.9970-1-dipiets@amazon.it/T/#t [v1]
> > Link: https://lore.kernel.org/linux-mm/20260420161404.642-1-dipiets@amazon.it/T/#u [v2]
> > Link: https://lore.kernel.org/all/20260710143437.12379-1-dipiets@amazon.it/T/#u [v3]
> > Fixes: 5d8edfb900d5 ("iomap: Copy larger chunks from userspace")
> 
> That's three years old.  v6.10.  
> 
> This is not good.  Can you think of an exceptional reason why this took
> so long to surface, or is it simply that our testing isn't good?

Neither.  We've know about these excessive high-order allocation->
direct reclaim compaction costs for a lot longer than this, and have
quite a number of filesystem based workloads that have been easily
able to reproduce it on demand.  XFS first worked around direct
compaction overhead back around ~5.17/18.

So why has it taken so long for this to be (kinda) fixed in the
memory reclaim code?

It is not for lack of awareness or trying.

I've responded to about half a dozen "direct compaction is
expensive" bug reports in the past couple of years where I've said
"this needs to be fixed in the reclaim code, not at the individual
call sites". Not just fs stuff, but graphics and other subsystems,
too.

However, convincing the MM manintainers to do anything w.r.t.
changing memory reclaim takes forever, and requires excessive
amounts of persistence before anything happens despite clear
evidence of the issue and the performance numbers showing how costly
direct compaction actually is.

A directly relatable example of this is kvmalloc() semantics. We
needed kvmalloc() use fail fast semantics for the high order
kmalloc() because it is much less costly to fall back to vmalloc()
than do direct compaction....

This was what the XFS changes in 5.17 worked around, and that fix
didn't get pulled into the kvmalloc() implementation until 6.15 with
commit 46459154f997 ("mm: kvmalloc: make kmalloc fast path real fast
path").

IOWs, it often takes years from "bad behaviour known" to "fixed in
MM infrastructure", and most of that time is spent trying to
convince the MM maintainers that the (relatively simple) fix needs
to be done in the MM code itself...

Cheers,

Dave.
-- 
Dave Chinner
dgc@kernel.org