Re: [PATCH v4] mm/page_alloc: avoid direct compaction for costly __GFP_NORETRY allocations

["Zi Yan" <ziy@nvidia.com>]

On Mon Sep 7, 2026 at 3:30 AM EDT, Vlastimil Babka (SUSE) wrote:
> On 9/4/26 17:08, Zi Yan wrote:
>> On Fri Sep 4, 2026 at 10:11 AM EDT, Vlastimil Babka (SUSE) wrote:
>>> On 9/4/26 13:56, Salvatore Dipietro wrote:
>>>> Commit 5d8edfb900d5 ("iomap: Copy larger chunks from userspace")
>>>> introduced high-order folio allocations in the iomap buffered write
>>>> path.  When memory is fragmented, each failed costly-order allocation
>>>> enters __alloc_pages_slowpath() which runs direct compaction and
>>>> drain_all_pages(), causing a 0.38x throughput drop on PostgreSQL
>>>> pgbench (simple-update) with 1024 clients on a 96-vCPU arm64 system.
>>>> 
>>>> The root issue is that direct compaction is too expensive for hot
>>>> allocation paths that have fallbacks to smaller allocations.
>>>> __filemap_get_folio_mpol() already marks higher-order allocations with
>>>> __GFP_NORETRY | __GFP_NOWARN, signalling that the caller can handle
>>>> failure.  However, the page allocator still attempts full direct
>>>> compaction for costly orders with __GFP_NORETRY, which is unnecessarily
>>>> aggressive when the caller will simply retry at a lower order.
>>>> 
>>>> For costly-order allocations with __GFP_NORETRY, clear
>>>> __GFP_DIRECT_RECLAIM at the very start of the slowpath, before
>>>> can_direct_reclaim, can_compact and the nofail checks are evaluated.
>>>> This makes the entire slowpath treat the request as non-blocking: no
>>>> direct reclaim, no direct compaction and no drain_all_pages() IPI
>>>> across every CPU. kswapd (and in turn kcompactd) is still woken further
>>>> down for background defragmentation, so compaction keeps working for
>>>> long-term system health while being removed from the latency-critical
>>>> direct allocation path.
>>>> 
>>>> Allocations that also request __GFP_THISNODE are exempted.  That flag
>>>> pairing identifies the local-node-first THP attempt issued by
>>>> alloc_pages_mpol() (mempolicy.c), which relies on direct compaction to
>>>> form transparent huge pages.
>>>> 
>>>> Test environment:
>>>>   Hardware:  AWS EC2 m8g.24xlarge (96 vCPU, arm64)
>>>>              12x 1TB IO2 32000 IOPS RAID0 XFS
>>>>   OS:        AL2023
>>>>   Kernel:    v7.3-rc1
>>>>   Database:  PostgreSQL 18.4
>>>>   Workload:  pgbench simple-update, 1024 clients, 96 threads, 1200s
>>>> 
>>>> Results (average of 3 runs, TPS):
>>>> 
>>>>   Config                   Avg TPS      % vs Baseline
>>>>   baseline (no patch)       59,408       -
>>>>   With this patch          155,409      +161.6%
>>>> 
>>>> Link: https://lore.kernel.org/all/20260403193535.9970-1-dipiets@amazon.it/T/#t [v1]
>>>> Link: https://lore.kernel.org/linux-mm/20260420161404.642-1-dipiets@amazon.it/T/#u [v2]
>>>> Link: https://lore.kernel.org/all/20260710143437.12379-1-dipiets@amazon.it/T/#u [v3]
>>>> Fixes: 5d8edfb900d5 ("iomap: Copy larger chunks from userspace")
>>>> Cc: stable@vger.kernel.org
>>>> Cc: Andrew Morton <akpm@linux-foundation.org>
>>>> Cc: Vlastimil Babka <vbabka@suse.cz>
>>>> Cc: David Hildenbrand <david@redhat.com>
>>>> Cc: Michal Hocko <mhocko@suse.com>
>>>> Cc: Johannes Weiner <hannes@cmpxchg.org>
>>>> Cc: Matthew Wilcox <willy@infradead.org>
>>>> Cc: Christoph Hellwig <hch@lst.de>
>>>> Cc: Dave Chinner <dgc@kernel.org>
>>>> Cc: Ritesh Harjani <ritesh.list@gmail.com>
>>>> Cc: linux-mm@kvack.org
>>>> Cc: linux-fsdevel@vger.kernel.org
>>>> Cc: linux-xfs@vger.kernel.org
>>>> Signed-off-by: Salvatore Dipietro <dipiets@amazon.it>
>>>
>>> I guess this will have to do until unlikely(we figure out a better API)...
>> 
>> We could add a "new_gfp = gfp_policy(gfp)" to adjust input gfp based on
>> various policies we currently have.
>
> Maybe with the help of alloc_flags to avoid the limitated count of gfp flags.

But alloc_flags are not exposed outside of MM, so alloc_page*() users
are still limited by the existing GFP_* flags. Are you suggesting
alloc_flags could be an addtional input for alloc_page*()?

>
>> Even better if callers can do that
>> instead.
>
> Not sure about burdening the callers (e.g. "every filesystem should do X"
> etc), unless they are some intermediate wrappers within mm itself.

Not all callers. I assume gfp_policy() will only be used by advance
users who want specific page allocation behavior without knowing the
details of the page allocator or changing how the page allocator behave.

> For example, THP (but only anonymous?) is now handled in such a special way,
> it could be a candidate. But I also have a vague feeling this was already
> done in the past. Worth investigating perhaps.
>

The number of wrappers can grow if different users have different needs.
Some might not want direct reclaim, some might not want direct
compaction, some might not want kswapd, and so on. I am not sure we want
to add wrappers for each.

For this patch, the caller can use gfp_policy() to strip out
__GFP_DIRECT_RECLAIM based on the conditions and pass the adjusted gfp
to the page allocator.


-- 
Best Regards,
Yan, Zi