[PATCH] Revert "perf/aux: Allocate non-contiguous AUX pages by default"

[PATCH] Revert "perf/aux: Allocate non-contiguous AUX pages by default"

[Leo Yan]

This reverts commit 18049c8cff9cc89daadc4df6975f7d9069638926.

The original commit claimed that contiguous AUX pages unnecessarily
exacerbate fragmentation for SPE, TRBE and ETR. This is not generally
valid: larger allocations can keep pages with the same lifetime
together, while order-0 allocations can scatter them across large
blocks and prevent free pages from coalescing [1]. The follow-up
discussion [2] has not established how forcing order-0 allocations
addresses the reported problem.

TRBE and SPE write to virtually addressed buffers. The TRBE L1 TLB
has one entry on Neoverse V2 [3] and two on Neoverse V3 [4]. Small
mappings increase translation pressure, and TTW stalls can contribute
to FIFO overflow and trace loss on TRBE.

Larger allocations provide contiguous, aligned chunks for PMD or
contiguous-PTE mappings with the proposed vmap() support [5].
With the proposed vmap() support already applied, the tests in [2]
showed that preferring larger AUX allocations reduced L2 data TLB
refills by 99.68% in the TRBE test and 34.74% in the SPE test.

Restore the previous policy of trying higher-order allocations first.
The allocator already uses __GFP_NORETRY and falls back to smaller
orders, including order 0, when larger allocations fail.

Remove PERF_PMU_CAP_AUX_PREFER_LARGE, since the restored default
provides that preference. Intel PT retains its existing allocation
behaviour, with no functional change.

[1] https://lore.kernel.org/all/aPjrRkjiIt6HmXmT@casper.infradead.org/
[2] https://lore.kernel.org/linux-arm-kernel/20260810-perf_aux_trace_large_granule-v1-0-03306c9339e3@arm.com/
[3] https://documentation-service.arm.com/static/633fe1ec4c59b30b517730b8#page=52
[4] https://documentation-service.arm.com/static/67361bd2c7fc0d1f211da382#page=54
[5] https://lore.kernel.org/linux-mm/20260715120813.3609949-1-jiangwen6@xiaomi.com/

Signed-off-by: Leo Yan <leo.yan@arm.com>
---
 arch/x86/events/intel/pt.c  |  2 --
 include/linux/perf_event.h  |  5 ++---
 kernel/events/ring_buffer.c | 29 +++++++++--------------------
 3 files changed, 11 insertions(+), 25 deletions(-)

diff --git a/arch/x86/events/intel/pt.c b/arch/x86/events/intel/pt.c
index 49349afee6119fb6fad815c847cb3cd9daa97604..3782af40f0832a874a7f0b60961d6652513de241 100644
--- a/arch/x86/events/intel/pt.c
+++ b/arch/x86/events/intel/pt.c
@@ -1896,8 +1896,6 @@ static __init int pt_init(void)
 
 	if (!intel_pt_validate_hw_cap(PT_CAP_topa_multiple_entries))
 		pt_pmu.pmu.capabilities = PERF_PMU_CAP_AUX_NO_SG;
-	else
-		pt_pmu.pmu.capabilities = PERF_PMU_CAP_AUX_PREFER_LARGE;
 
 	pt_pmu.pmu.capabilities		|= PERF_PMU_CAP_EXCLUSIVE |
 					   PERF_PMU_CAP_ITRACE |
diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
index eb3f7bab498ca9bee9ce7193744943b24ab2e0c5..b8d8b005f5c8a7c1def7cad02230bf9ffb876bd6 100644
--- a/include/linux/perf_event.h
+++ b/include/linux/perf_event.h
@@ -304,9 +304,8 @@ struct perf_event_pmu_context;
 #define PERF_PMU_CAP_AUX_OUTPUT		0x0080
 #define PERF_PMU_CAP_EXTENDED_HW_TYPE	0x0100
 #define PERF_PMU_CAP_AUX_PAUSE		0x0200
-#define PERF_PMU_CAP_AUX_PREFER_LARGE	0x0400
-#define PERF_PMU_CAP_MEDIATED_VPMU	0x0800
-#define PERF_PMU_CAP_SIMD_REGS		0x1000
+#define PERF_PMU_CAP_MEDIATED_VPMU	0x0400
+#define PERF_PMU_CAP_SIMD_REGS		0x0800
 
 /**
  * pmu::scope
diff --git a/kernel/events/ring_buffer.c b/kernel/events/ring_buffer.c
index 64d4dbdc6c071b72b670dd044616f52981206ae8..95f4fbdfd2764de5b72d901d778c46302f525fad 100644
--- a/kernel/events/ring_buffer.c
+++ b/kernel/events/ring_buffer.c
@@ -684,15 +684,7 @@ int rb_alloc_aux(struct perf_buffer *rb, struct perf_event *event,
 {
 	bool overwrite = !(flags & RING_BUFFER_WRITABLE);
 	int node = (event->cpu == -1) ? -1 : cpu_to_node(event->cpu);
-	bool use_contiguous_pages = event->pmu->capabilities & (
-		PERF_PMU_CAP_AUX_NO_SG | PERF_PMU_CAP_AUX_PREFER_LARGE);
-	/*
-	 * Initialize max_order to 0 for page allocation. This allocates single
-	 * pages to minimize memory fragmentation. This is overridden if the
-	 * PMU needs or prefers contiguous pages (use_contiguous_pages = true).
-	 */
-	int max_order = 0;
-	int ret = -ENOMEM;
+	int ret = -ENOMEM, max_order;
 
 	if (!has_aux(event))
 		return -EOPNOTSUPP;
@@ -702,8 +694,8 @@ int rb_alloc_aux(struct perf_buffer *rb, struct perf_event *event,
 
 	if (!overwrite) {
 		/*
-		 * Watermark defaults to half the buffer, to aid PMU drivers
-		 * in double buffering.
+		 * Watermark defaults to half the buffer, and so does the
+		 * max_order, to aid PMU drivers in double buffering.
 		 */
 		if (!watermark)
 			watermark = min_t(unsigned long,
@@ -711,19 +703,16 @@ int rb_alloc_aux(struct perf_buffer *rb, struct perf_event *event,
 					  (unsigned long)nr_pages << (PAGE_SHIFT - 1));
 
 		/*
-		 * If using contiguous pages, use aux_watermark as the basis
-		 * for chunking to help PMU drivers honor the watermark.
+		 * Use aux_watermark as the basis for chunking to
+		 * help PMU drivers honor the watermark.
 		 */
-		if (use_contiguous_pages)
-			max_order = get_order(watermark);
+		max_order = get_order(watermark);
 	} else {
 		/*
-		 * If using contiguous pages, we need to start with the
-		 * max_order that fits in nr_pages, not the other way around,
-		 * hence ilog2() and not get_order.
+		 * We need to start with the max_order that fits in nr_pages,
+		 * not the other way around, hence ilog2() and not get_order.
 		 */
-		if (use_contiguous_pages)
-			max_order = ilog2(nr_pages);
+		max_order = ilog2(nr_pages);
 		watermark = 0;
 	}
 

---
base-commit: 21a7f44fd0765fcc5194130b28767786eba1d1e7
change-id: 20261025-perf_revert_prefer_large_alloc-a0177d6379f6

Best regards,
-- 
Leo Yan <leo.yan@arm.com>

Re: [PATCH] Revert "perf/aux: Allocate non-contiguous AUX pages by default"

[James Clark]

On 25/10/2026 16:24, Leo Yan wrote:
> This reverts commit 18049c8cff9cc89daadc4df6975f7d9069638926.
> 
> The original commit claimed that contiguous AUX pages unnecessarily
> exacerbate fragmentation for SPE, TRBE and ETR. This is not generally
> valid: larger allocations can keep pages with the same lifetime
> together, while order-0 allocations can scatter them across large
> blocks and prevent free pages from coalescing [1]. The follow-up
> discussion [2] has not established how forcing order-0 allocations
> addresses the reported problem.
> 
> TRBE and SPE write to virtually addressed buffers. The TRBE L1 TLB
> has one entry on Neoverse V2 [3] and two on Neoverse V3 [4]. Small
> mappings increase translation pressure, and TTW stalls can contribute
> to FIFO overflow and trace loss on TRBE.
> 
> Larger allocations provide contiguous, aligned chunks for PMD or
> contiguous-PTE mappings with the proposed vmap() support [5].
> With the proposed vmap() support already applied, the tests in [2]
> showed that preferring larger AUX allocations reduced L2 data TLB
> refills by 99.68% in the TRBE test and 34.74% in the SPE test.
> 
> Restore the previous policy of trying higher-order allocations first.
> The allocator already uses __GFP_NORETRY and falls back to smaller
> orders, including order 0, when larger allocations fail.
> 
> Remove PERF_PMU_CAP_AUX_PREFER_LARGE, since the restored default
> provides that preference. Intel PT retains its existing allocation
> behaviour, with no functional change.
> 
> [1] https://lore.kernel.org/all/aPjrRkjiIt6HmXmT@casper.infradead.org/
> [2] https://lore.kernel.org/linux-arm-kernel/20260810-perf_aux_trace_large_granule-v1-0-03306c9339e3@arm.com/
> [3] https://documentation-service.arm.com/static/633fe1ec4c59b30b517730b8#page=52
> [4] https://documentation-service.arm.com/static/67361bd2c7fc0d1f211da382#page=54
> [5] https://lore.kernel.org/linux-mm/20260715120813.3609949-1-jiangwen6@xiaomi.com/
> 
> Signed-off-by: Leo Yan <leo.yan@arm.com>
> ---
>   arch/x86/events/intel/pt.c  |  2 --
>   include/linux/perf_event.h  |  5 ++---
>   kernel/events/ring_buffer.c | 29 +++++++++--------------------
>   3 files changed, 11 insertions(+), 25 deletions(-)
> 
> diff --git a/arch/x86/events/intel/pt.c b/arch/x86/events/intel/pt.c
> index 49349afee6119fb6fad815c847cb3cd9daa97604..3782af40f0832a874a7f0b60961d6652513de241 100644
> --- a/arch/x86/events/intel/pt.c
> +++ b/arch/x86/events/intel/pt.c
> @@ -1896,8 +1896,6 @@ static __init int pt_init(void)
>   
>   	if (!intel_pt_validate_hw_cap(PT_CAP_topa_multiple_entries))
>   		pt_pmu.pmu.capabilities = PERF_PMU_CAP_AUX_NO_SG;
> -	else
> -		pt_pmu.pmu.capabilities = PERF_PMU_CAP_AUX_PREFER_LARGE;
>   
>   	pt_pmu.pmu.capabilities		|= PERF_PMU_CAP_EXCLUSIVE |
>   					   PERF_PMU_CAP_ITRACE |
> diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
> index eb3f7bab498ca9bee9ce7193744943b24ab2e0c5..b8d8b005f5c8a7c1def7cad02230bf9ffb876bd6 100644
> --- a/include/linux/perf_event.h
> +++ b/include/linux/perf_event.h
> @@ -304,9 +304,8 @@ struct perf_event_pmu_context;
>   #define PERF_PMU_CAP_AUX_OUTPUT		0x0080
>   #define PERF_PMU_CAP_EXTENDED_HW_TYPE	0x0100
>   #define PERF_PMU_CAP_AUX_PAUSE		0x0200
> -#define PERF_PMU_CAP_AUX_PREFER_LARGE	0x0400
> -#define PERF_PMU_CAP_MEDIATED_VPMU	0x0800
> -#define PERF_PMU_CAP_SIMD_REGS		0x1000
> +#define PERF_PMU_CAP_MEDIATED_VPMU	0x0400
> +#define PERF_PMU_CAP_SIMD_REGS		0x0800
>   
>   /**
>    * pmu::scope
> diff --git a/kernel/events/ring_buffer.c b/kernel/events/ring_buffer.c
> index 64d4dbdc6c071b72b670dd044616f52981206ae8..95f4fbdfd2764de5b72d901d778c46302f525fad 100644
> --- a/kernel/events/ring_buffer.c
> +++ b/kernel/events/ring_buffer.c
> @@ -684,15 +684,7 @@ int rb_alloc_aux(struct perf_buffer *rb, struct perf_event *event,
>   {
>   	bool overwrite = !(flags & RING_BUFFER_WRITABLE);
>   	int node = (event->cpu == -1) ? -1 : cpu_to_node(event->cpu);
> -	bool use_contiguous_pages = event->pmu->capabilities & (
> -		PERF_PMU_CAP_AUX_NO_SG | PERF_PMU_CAP_AUX_PREFER_LARGE);
> -	/*
> -	 * Initialize max_order to 0 for page allocation. This allocates single
> -	 * pages to minimize memory fragmentation. This is overridden if the
> -	 * PMU needs or prefers contiguous pages (use_contiguous_pages = true).
> -	 */
> -	int max_order = 0;
> -	int ret = -ENOMEM;
> +	int ret = -ENOMEM, max_order;
>   
>   	if (!has_aux(event))
>   		return -EOPNOTSUPP;
> @@ -702,8 +694,8 @@ int rb_alloc_aux(struct perf_buffer *rb, struct perf_event *event,
>   
>   	if (!overwrite) {
>   		/*
> -		 * Watermark defaults to half the buffer, to aid PMU drivers
> -		 * in double buffering.
> +		 * Watermark defaults to half the buffer, and so does the
> +		 * max_order, to aid PMU drivers in double buffering.
>   		 */
>   		if (!watermark)
>   			watermark = min_t(unsigned long,
> @@ -711,19 +703,16 @@ int rb_alloc_aux(struct perf_buffer *rb, struct perf_event *event,
>   					  (unsigned long)nr_pages << (PAGE_SHIFT - 1));
>   
>   		/*
> -		 * If using contiguous pages, use aux_watermark as the basis
> -		 * for chunking to help PMU drivers honor the watermark.
> +		 * Use aux_watermark as the basis for chunking to
> +		 * help PMU drivers honor the watermark.
>   		 */
> -		if (use_contiguous_pages)
> -			max_order = get_order(watermark);
> +		max_order = get_order(watermark);
>   	} else {
>   		/*
> -		 * If using contiguous pages, we need to start with the
> -		 * max_order that fits in nr_pages, not the other way around,
> -		 * hence ilog2() and not get_order.
> +		 * We need to start with the max_order that fits in nr_pages,
> +		 * not the other way around, hence ilog2() and not get_order.
>   		 */
> -		if (use_contiguous_pages)
> -			max_order = ilog2(nr_pages);
> +		max_order = ilog2(nr_pages);
>   		watermark = 0;
>   	}
>   
> 
> ---
> base-commit: 21a7f44fd0765fcc5194130b28767786eba1d1e7
> change-id: 20261025-perf_revert_prefer_large_alloc-a0177d6379f6
> 
> Best regards,


Reviewed-by: James Clark <james.clark@linaro.org>

Re: [PATCH] Revert "perf/aux: Allocate non-contiguous AUX pages by default"

[Dev Jain]

On 25/10/26 9:54 pm, Leo Yan wrote:
> This reverts commit 18049c8cff9cc89daadc4df6975f7d9069638926.
> 
> The original commit claimed that contiguous AUX pages unnecessarily
> exacerbate fragmentation for SPE, TRBE and ETR. This is not generally
> valid: larger allocations can keep pages with the same lifetime
> together, while order-0 allocations can scatter them across large
> blocks and prevent free pages from coalescing [1]. The follow-up
> discussion [2] has not established how forcing order-0 allocations
> addresses the reported problem.
> 
> TRBE and SPE write to virtually addressed buffers. The TRBE L1 TLB
> has one entry on Neoverse V2 [3] and two on Neoverse V3 [4]. Small
> mappings increase translation pressure, and TTW stalls can contribute
> to FIFO overflow and trace loss on TRBE.
> 
> Larger allocations provide contiguous, aligned chunks for PMD or
> contiguous-PTE mappings with the proposed vmap() support [5].
> With the proposed vmap() support already applied, the tests in [2]
> showed that preferring larger AUX allocations reduced L2 data TLB
> refills by 99.68% in the TRBE test and 34.74% in the SPE test.
> 
> Restore the previous policy of trying higher-order allocations first.
> The allocator already uses __GFP_NORETRY and falls back to smaller
> orders, including order 0, when larger allocations fail.
> 
> Remove PERF_PMU_CAP_AUX_PREFER_LARGE, since the restored default
> provides that preference. Intel PT retains its existing allocation
> behaviour, with no functional change.
> 
> [1] https://lore.kernel.org/all/aPjrRkjiIt6HmXmT@casper.infradead.org/
> [2] https://lore.kernel.org/linux-arm-kernel/20260810-perf_aux_trace_large_granule-v1-0-03306c9339e3@arm.com/
> [3] https://documentation-service.arm.com/static/633fe1ec4c59b30b517730b8#page=52
> [4] https://documentation-service.arm.com/static/67361bd2c7fc0d1f211da382#page=54
> [5] https://lore.kernel.org/linux-mm/20260715120813.3609949-1-jiangwen6@xiaomi.com/
> 
> Signed-off-by: Leo Yan <leo.yan@arm.com>
> ---

Reviewed-by: Dev Jain <dev.jain@arm.com>


>  arch/x86/events/intel/pt.c  |  2 --
>  include/linux/perf_event.h  |  5 ++---
>  kernel/events/ring_buffer.c | 29 +++++++++--------------------
>  3 files changed, 11 insertions(+), 25 deletions(-)
> 
> diff --git a/arch/x86/events/intel/pt.c b/arch/x86/events/intel/pt.c
> index 49349afee6119fb6fad815c847cb3cd9daa97604..3782af40f0832a874a7f0b60961d6652513de241 100644
> --- a/arch/x86/events/intel/pt.c
> +++ b/arch/x86/events/intel/pt.c
> @@ -1896,8 +1896,6 @@ static __init int pt_init(void)
>  
>  	if (!intel_pt_validate_hw_cap(PT_CAP_topa_multiple_entries))
>  		pt_pmu.pmu.capabilities = PERF_PMU_CAP_AUX_NO_SG;
> -	else
> -		pt_pmu.pmu.capabilities = PERF_PMU_CAP_AUX_PREFER_LARGE;
>  
>  	pt_pmu.pmu.capabilities		|= PERF_PMU_CAP_EXCLUSIVE |
>  					   PERF_PMU_CAP_ITRACE |
> diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
> index eb3f7bab498ca9bee9ce7193744943b24ab2e0c5..b8d8b005f5c8a7c1def7cad02230bf9ffb876bd6 100644
> --- a/include/linux/perf_event.h
> +++ b/include/linux/perf_event.h
> @@ -304,9 +304,8 @@ struct perf_event_pmu_context;
>  #define PERF_PMU_CAP_AUX_OUTPUT		0x0080
>  #define PERF_PMU_CAP_EXTENDED_HW_TYPE	0x0100
>  #define PERF_PMU_CAP_AUX_PAUSE		0x0200
> -#define PERF_PMU_CAP_AUX_PREFER_LARGE	0x0400
> -#define PERF_PMU_CAP_MEDIATED_VPMU	0x0800
> -#define PERF_PMU_CAP_SIMD_REGS		0x1000
> +#define PERF_PMU_CAP_MEDIATED_VPMU	0x0400
> +#define PERF_PMU_CAP_SIMD_REGS		0x0800
>  
>  /**
>   * pmu::scope
> diff --git a/kernel/events/ring_buffer.c b/kernel/events/ring_buffer.c
> index 64d4dbdc6c071b72b670dd044616f52981206ae8..95f4fbdfd2764de5b72d901d778c46302f525fad 100644
> --- a/kernel/events/ring_buffer.c
> +++ b/kernel/events/ring_buffer.c
> @@ -684,15 +684,7 @@ int rb_alloc_aux(struct perf_buffer *rb, struct perf_event *event,
>  {
>  	bool overwrite = !(flags & RING_BUFFER_WRITABLE);
>  	int node = (event->cpu == -1) ? -1 : cpu_to_node(event->cpu);
> -	bool use_contiguous_pages = event->pmu->capabilities & (
> -		PERF_PMU_CAP_AUX_NO_SG | PERF_PMU_CAP_AUX_PREFER_LARGE);
> -	/*
> -	 * Initialize max_order to 0 for page allocation. This allocates single
> -	 * pages to minimize memory fragmentation. This is overridden if the
> -	 * PMU needs or prefers contiguous pages (use_contiguous_pages = true).
> -	 */
> -	int max_order = 0;
> -	int ret = -ENOMEM;
> +	int ret = -ENOMEM, max_order;
>  
>  	if (!has_aux(event))
>  		return -EOPNOTSUPP;
> @@ -702,8 +694,8 @@ int rb_alloc_aux(struct perf_buffer *rb, struct perf_event *event,
>  
>  	if (!overwrite) {
>  		/*
> -		 * Watermark defaults to half the buffer, to aid PMU drivers
> -		 * in double buffering.
> +		 * Watermark defaults to half the buffer, and so does the
> +		 * max_order, to aid PMU drivers in double buffering.
>  		 */
>  		if (!watermark)
>  			watermark = min_t(unsigned long,
> @@ -711,19 +703,16 @@ int rb_alloc_aux(struct perf_buffer *rb, struct perf_event *event,
>  					  (unsigned long)nr_pages << (PAGE_SHIFT - 1));
>  
>  		/*
> -		 * If using contiguous pages, use aux_watermark as the basis
> -		 * for chunking to help PMU drivers honor the watermark.
> +		 * Use aux_watermark as the basis for chunking to
> +		 * help PMU drivers honor the watermark.
>  		 */
> -		if (use_contiguous_pages)
> -			max_order = get_order(watermark);
> +		max_order = get_order(watermark);
>  	} else {
>  		/*
> -		 * If using contiguous pages, we need to start with the
> -		 * max_order that fits in nr_pages, not the other way around,
> -		 * hence ilog2() and not get_order.
> +		 * We need to start with the max_order that fits in nr_pages,
> +		 * not the other way around, hence ilog2() and not get_order.
>  		 */
> -		if (use_contiguous_pages)
> -			max_order = ilog2(nr_pages);
> +		max_order = ilog2(nr_pages);
>  		watermark = 0;
>  	}
>  
> 
> ---
> base-commit: 21a7f44fd0765fcc5194130b28767786eba1d1e7
> change-id: 20261025-perf_revert_prefer_large_alloc-a0177d6379f6
> 
> Best regards,

Re: [PATCH] Revert "perf/aux: Allocate non-contiguous AUX pages by default"

[Yabin Cui]

[Resending in plain text for the mailing lists; sorry for the duplicate.]

Hi Leo, Will, Dev,

On Sun, Oct 25, 2026 at 04:24:06PM +0000, Leo Yan wrote:
> The original commit claimed that contiguous AUX pages unnecessarily
> exacerbate fragmentation for SPE, TRBE and ETR. This is not generally
> valid: larger allocations can keep pages with the same lifetime together,
> while order-0 allocations can scatter them across large blocks and prevent
> free pages from coalescing [1]. The follow-up discussion [2] has not
> established how forcing order-0 allocations addresses the reported problem.
...
> Restore the previous policy of trying higher-order allocations first.
> The allocator already uses __GFP_NORETRY and falls back to smaller
> orders, including order 0, when larger allocations fail.

Sorry for the late reply, and sorry that the commit message in
18049c8cff9c ("perf/aux: Allocate non-contiguous AUX pages by default")
was imprecise about "exacerbating memory fragmentation". You and Dev are
right that grouping same-lifetime pages into higher-order allocations
does not itself worsen fragmentation, and the TLB reduction numbers for
TRBE and SPE with large vmap() mappings make total sense given TRBE's
tiny TLB.

However, a straight revert will reintroduce a severe production issue we
hit on Android devices, so I'd like to share what actually happened and
suggest a way to get large mappings for SPE/TRBE without bringing back
the regression.

### What happened in production (even with __GFP_NORETRY)

On Android Pixel dogfood devices, a background daemon (profcollectd)
periodically records short (1s) system-wide CoreSight ETM traces every
~10 minutes for AutoFDO, using a 4 MiB AUX buffer per CPU with default
aux_watermark = 0 (so watermark = 2 MiB, max_order = 9).

Although PERF_AUX_GFP includes __GFP_NORETRY, it is based on GFP_KERNEL,
which retains __GFP_DIRECT_RECLAIM | __GFP_KSWAPD_RECLAIM. __GFP_NORETRY
only avoids retry loops and the kernel OOM killer at the end of
__alloc_pages_slowpath(). On long-running phones where physical memory is
already fragmented and order 9..4 pages are unavailable, every failed
high-order call in rb_alloc_aux_page() still enters the slowpath and:

  1. Wakes kswapd/kcompactd at that high order.
  2. Runs synchronous direct compaction (__alloc_pages_direct_compact()).
  3. Runs synchronous direct reclaim (__alloc_pages_direct_reclaim()),
     evicting pages and raising memory PSI pressure. On Android, PSI
     reclaim stalls trigger userspace lmkd (Low Memory Killer daemon)
     even though the kernel OOM killer is never invoked.

This is amplified by two loops in rb_alloc_aux():
  - rb_alloc_aux_page() steps down one order at a time (9, 8, ..., 0),
    hitting direct compaction/reclaim at each unavailable order.
  - Worse, the outer loop in rb_alloc_aux() never lowers max_order when
    rb_alloc_aux_page() falls back:

      order = min(max_order, ilog2(nr_pages - rb->aux_nr_pages));
      page = rb_alloc_aux_page(node, order);

    When a 4 MiB buffer (1024 pages) only finds order-0 pages, it retries
    orders 9..1 on almost every iteration -- thousands of failed
    high-order slowpath compaction/reclaim attempts for a single
    perf_mmap() call.

In field telemetry and local ftrace on Pixel devices (prior to
18049c8cff9c, so __GFP_NORETRY was already in place), we observed heavy
rb_alloc_aux() -> __alloc_pages_direct_compact() stalls alongside
kcompactd0 pinned on big cores scanning order-9, and this accounted for
>20% of all perceptible app kills (via lmkd) across dogfood devices
until we stopped high-order AUX allocations.

Initially we worked around this in userspace (simpleperf) by setting
perf_event_attr::aux_watermark = 4096 so get_order(watermark) == 0. That
was fine for CoreSight ETR (which uses its own separate scatter-gather
buffer and only copies into AUX on flush/stop), but when enabling TRBE
(and SPE) it caused hardware maintenance interrupts every 4 KiB instead
of every 2 MiB. Also, in overwrite/snapshot mode rb_alloc_aux() ignores
aux_watermark and uses ilog2(nr_pages). That is why I sent 18049c8cff9c.

### Possible solutions

For PMUs without PERF_PMU_CAP_AUX_NO_SG (such as SPE, TRBE, and modern
Intel PT), higher-order AUX pages are an opportunistic optimization
rather than a functional requirement -- very similar to THP allocations
on fault (GFP_TRANSHUGE_LIGHT) or opportunistic high-order slabs in SLUB
(allocate_slab()).

Instead of reverting to GFP_KERNEL reclaim/compaction for all high-order
attempts, could we do the following?

1. Make opportunistic high-order AUX allocations no-reclaim:
   When !(event->pmu->capabilities & PERF_PMU_CAP_AUX_NO_SG) and
   order > 0, clear __GFP_RECLAIM (and add __GFP_NOMEMALLOC, matching
   SLUB's allocate_slab() and GFP_TRANSHUGE_LIGHT):

     gfp_t gfp = PERF_AUX_GFP;

     if (order && !has_no_sg)
             gfp = (PERF_AUX_GFP | __GFP_NOMEMALLOC) & ~__GFP_RECLAIM;

   When order falls back to 0 (or when PERF_PMU_CAP_AUX_NO_SG is set),
   keep PERF_AUX_GFP so single-page allocations still reclaim normally
   under memory pressure.

   That way, when free order-9 / order-4 blocks exist, SPE and TRBE get
   them immediately for PMD / contiguous-PTE vmap() mappings (and pages
   stay grouped together); when memory is fragmented on a long-running
   device, high-order probes fail fast on the freelist without waking
   kcompactd/kswapd or triggering direct compaction and reclaim.

2. Clamp max_order in rb_alloc_aux() after fallback:
   If rb_alloc_aux_page() requested `order` and had to fall back to a
   smaller order, update `max_order = page_private(page)` in
   rb_alloc_aux() so subsequent iterations for the rest of the buffer do
   not keep re-probing higher orders that just failed.

(Optionally, if server/benchmark workloads ever want synchronous
compaction to form huge AUX mappings even on fragmented hosts, that could
be gated behind an opt-in attr/sysfs knob similar to THP defrag modes,
with light/no-reclaim as the default.)

Would this approach work for your SPE/TRBE use cases? Happy to help test
on our side.

Thanks,
Yabin


On Tue, Nov 3, 2026 at 12:32 PM Yabin Cui <yabinc@google.com> wrote:
>
> Hi Leo, Will, Dev,
>
> Sorry for the late reply, and sorry that the commit message in
> 18049c8cff9c ("perf/aux: Allocate non-contiguous AUX pages by default")
> was imprecise about "exacerbating memory fragmentation". You and Dev are
> right that grouping same-lifetime pages into higher-order allocations
> does not itself worsen fragmentation, and the TLB reduction numbers for
> TRBE and SPE with large vmap() mappings make total sense given TRBE's
> tiny TLB.
>
> However, a straight revert will reintroduce a severe production issue we
> hit on Android devices, so I'd like to share what actually happened and
> suggest a way to get large mappings for SPE/TRBE without bringing back
> the regression.
>
> ### What happened in production (even with __GFP_NORETRY)
>
> On Android Pixel dogfood devices, a background daemon (profcollectd)
> periodically records short (1s) system-wide CoreSight ETM traces every
> ~10 minutes for AutoFDO, using a 4 MiB AUX buffer per CPU with default
> aux_watermark = 0 (so watermark = 2 MiB, max_order = 9).
>
> Although PERF_AUX_GFP includes __GFP_NORETRY, it is based on GFP_KERNEL,
> which retains __GFP_DIRECT_RECLAIM | __GFP_KSWAPD_RECLAIM. __GFP_NORETRY
> only avoids retry loops and the kernel OOM killer at the end of
> __alloc_pages_slowpath(). On long-running phones where physical memory is
> already fragmented and order 9..4 pages are unavailable, every failed
> high-order call in rb_alloc_aux_page() still enters the slowpath and:
>
>   1. Wakes kswapd/kcompactd at that high order.
>   2. Runs synchronous direct compaction (__alloc_pages_direct_compact()).
>   3. Runs synchronous direct reclaim (__alloc_pages_direct_reclaim()),
>      evicting pages and raising memory PSI pressure. On Android, PSI
>      reclaim stalls trigger userspace lmkd (Low Memory Killer daemon)
>      even though the kernel OOM killer is never invoked.
>
> This is amplified by two loops in rb_alloc_aux():
>   - rb_alloc_aux_page() steps down one order at a time (9, 8, ..., 0),
>     hitting direct compaction/reclaim at each unavailable order.
>   - Worse, the outer loop in rb_alloc_aux() never lowers max_order when
>     rb_alloc_aux_page() falls back:
>
>       order = min(max_order, ilog2(nr_pages - rb->aux_nr_pages));
>       page = rb_alloc_aux_page(node, order);
>
>     When a 4 MiB buffer (1024 pages) only finds order-0 pages, it retries
>     orders 9..1 on almost every iteration -- thousands of failed
>     high-order slowpath compaction/reclaim attempts for a single
>     perf_mmap() call.
>
> In field telemetry and local ftrace on Pixel devices (prior to
> 18049c8cff9c, so __GFP_NORETRY was already in place), we observed heavy
> rb_alloc_aux() -> __alloc_pages_direct_compact() stalls alongside
> kcompactd0 pinned on big cores scanning order-9, and this accounted for
> >20% of all perceptible app kills (via lmkd) across dogfood devices
> until we stopped high-order AUX allocations.
>
> Initially we worked around this in userspace (simpleperf) by setting
> perf_event_attr::aux_watermark = 4096 so get_order(watermark) == 0. That
> was fine for CoreSight ETR (which uses its own separate scatter-gather
> buffer and only copies into AUX on flush/stop), but when enabling TRBE
> (and SPE) it caused hardware maintenance interrupts every 4 KiB instead
> of every 2 MiB. Also, in overwrite/snapshot mode rb_alloc_aux() ignores
> aux_watermark and uses ilog2(nr_pages). That is why I sent 18049c8cff9c.
>
> ### Possible solutions
>
> For PMUs without PERF_PMU_CAP_AUX_NO_SG (such as SPE, TRBE, and modern
> Intel PT), higher-order AUX pages are an opportunistic optimization
> rather than a functional requirement -- very similar to THP allocations
> on fault (GFP_TRANSHUGE_LIGHT) or opportunistic high-order slabs in SLUB
> (allocate_slab()).
>
> Instead of reverting to GFP_KERNEL reclaim/compaction for all high-order
> attempts, could we do the following?
>
> 1. Make opportunistic high-order AUX allocations no-reclaim:
>    When !(event->pmu->capabilities & PERF_PMU_CAP_AUX_NO_SG) and
>    order > 0, clear __GFP_RECLAIM (and add __GFP_NOMEMALLOC, matching
>    SLUB's allocate_slab() and GFP_TRANSHUGE_LIGHT):
>
>      gfp_t gfp = PERF_AUX_GFP;
>
>      if (order && !has_no_sg)
>              gfp = (PERF_AUX_GFP | __GFP_NOMEMALLOC) & ~__GFP_RECLAIM;
>
>    When order falls back to 0 (or when PERF_PMU_CAP_AUX_NO_SG is set),
>    keep PERF_AUX_GFP so single-page allocations still reclaim normally
>    under memory pressure.
>
>    That way, when free order-9 / order-4 blocks exist, SPE and TRBE get
>    them immediately for PMD / contiguous-PTE vmap() mappings (and pages
>    stay grouped together); when memory is fragmented on a long-running
>    device, high-order probes fail fast on the freelist without waking
>    kcompactd/kswapd or triggering direct compaction and reclaim.
>
> 2. Clamp max_order in rb_alloc_aux() after fallback:
>    If rb_alloc_aux_page() requested `order` and had to fall back to a
>    smaller order, update `max_order = page_private(page)` in
>    rb_alloc_aux() so subsequent iterations for the rest of the buffer do
>    not keep re-probing higher orders that just failed.
>
> (Optionally, if server/benchmark workloads ever want synchronous
> compaction to form huge AUX mappings even on fragmented hosts, that could
> be gated behind an opt-in attr/sysfs knob similar to THP defrag modes,
> with light/no-reclaim as the default.)
>
> Would this approach work for your SPE/TRBE use cases? Happy to help test
> on our side.
>
> Thanks,
> Yabin
>
>
> On Wed, Oct 28, 2026 at 11:56 PM Dev Jain <dev.jain@arm.com> wrote:
>>
>>
>>
>> On 25/10/26 9:54 pm, Leo Yan wrote:
>> > This reverts commit 18049c8cff9cc89daadc4df6975f7d9069638926.
>> >
>> > The original commit claimed that contiguous AUX pages unnecessarily
>> > exacerbate fragmentation for SPE, TRBE and ETR. This is not generally
>> > valid: larger allocations can keep pages with the same lifetime
>> > together, while order-0 allocations can scatter them across large
>> > blocks and prevent free pages from coalescing [1]. The follow-up
>> > discussion [2] has not established how forcing order-0 allocations
>> > addresses the reported problem.
>> >
>> > TRBE and SPE write to virtually addressed buffers. The TRBE L1 TLB
>> > has one entry on Neoverse V2 [3] and two on Neoverse V3 [4]. Small
>> > mappings increase translation pressure, and TTW stalls can contribute
>> > to FIFO overflow and trace loss on TRBE.
>> >
>> > Larger allocations provide contiguous, aligned chunks for PMD or
>> > contiguous-PTE mappings with the proposed vmap() support [5].
>> > With the proposed vmap() support already applied, the tests in [2]
>> > showed that preferring larger AUX allocations reduced L2 data TLB
>> > refills by 99.68% in the TRBE test and 34.74% in the SPE test.
>> >
>> > Restore the previous policy of trying higher-order allocations first.
>> > The allocator already uses __GFP_NORETRY and falls back to smaller
>> > orders, including order 0, when larger allocations fail.
>> >
>> > Remove PERF_PMU_CAP_AUX_PREFER_LARGE, since the restored default
>> > provides that preference. Intel PT retains its existing allocation
>> > behaviour, with no functional change.
>> >
>> > [1] https://lore.kernel.org/all/aPjrRkjiIt6HmXmT@casper.infradead.org/
>> > [2] https://lore.kernel.org/linux-arm-kernel/20260810-perf_aux_trace_large_granule-v1-0-03306c9339e3@arm.com/
>> > [3] https://documentation-service.arm.com/static/633fe1ec4c59b30b517730b8#page=52
>> > [4] https://documentation-service.arm.com/static/67361bd2c7fc0d1f211da382#page=54
>> > [5] https://lore.kernel.org/linux-mm/20260715120813.3609949-1-jiangwen6@xiaomi.com/
>> >
>> > Signed-off-by: Leo Yan <leo.yan@arm.com>
>> > ---
>>
>> Reviewed-by: Dev Jain <dev.jain@arm.com>
>>
>>
>> >  arch/x86/events/intel/pt.c  |  2 --
>> >  include/linux/perf_event.h  |  5 ++---
>> >  kernel/events/ring_buffer.c | 29 +++++++++--------------------
>> >  3 files changed, 11 insertions(+), 25 deletions(-)
>> >
>> > diff --git a/arch/x86/events/intel/pt.c b/arch/x86/events/intel/pt.c
>> > index 49349afee6119fb6fad815c847cb3cd9daa97604..3782af40f0832a874a7f0b60961d6652513de241 100644
>> > --- a/arch/x86/events/intel/pt.c
>> > +++ b/arch/x86/events/intel/pt.c
>> > @@ -1896,8 +1896,6 @@ static __init int pt_init(void)
>> >
>> >       if (!intel_pt_validate_hw_cap(PT_CAP_topa_multiple_entries))
>> >               pt_pmu.pmu.capabilities = PERF_PMU_CAP_AUX_NO_SG;
>> > -     else
>> > -             pt_pmu.pmu.capabilities = PERF_PMU_CAP_AUX_PREFER_LARGE;
>> >
>> >       pt_pmu.pmu.capabilities         |= PERF_PMU_CAP_EXCLUSIVE |
>> >                                          PERF_PMU_CAP_ITRACE |
>> > diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
>> > index eb3f7bab498ca9bee9ce7193744943b24ab2e0c5..b8d8b005f5c8a7c1def7cad02230bf9ffb876bd6 100644
>> > --- a/include/linux/perf_event.h
>> > +++ b/include/linux/perf_event.h
>> > @@ -304,9 +304,8 @@ struct perf_event_pmu_context;
>> >  #define PERF_PMU_CAP_AUX_OUTPUT              0x0080
>> >  #define PERF_PMU_CAP_EXTENDED_HW_TYPE        0x0100
>> >  #define PERF_PMU_CAP_AUX_PAUSE               0x0200
>> > -#define PERF_PMU_CAP_AUX_PREFER_LARGE        0x0400
>> > -#define PERF_PMU_CAP_MEDIATED_VPMU   0x0800
>> > -#define PERF_PMU_CAP_SIMD_REGS               0x1000
>> > +#define PERF_PMU_CAP_MEDIATED_VPMU   0x0400
>> > +#define PERF_PMU_CAP_SIMD_REGS               0x0800
>> >
>> >  /**
>> >   * pmu::scope
>> > diff --git a/kernel/events/ring_buffer.c b/kernel/events/ring_buffer.c
>> > index 64d4dbdc6c071b72b670dd044616f52981206ae8..95f4fbdfd2764de5b72d901d778c46302f525fad 100644
>> > --- a/kernel/events/ring_buffer.c
>> > +++ b/kernel/events/ring_buffer.c
>> > @@ -684,15 +684,7 @@ int rb_alloc_aux(struct perf_buffer *rb, struct perf_event *event,
>> >  {
>> >       bool overwrite = !(flags & RING_BUFFER_WRITABLE);
>> >       int node = (event->cpu == -1) ? -1 : cpu_to_node(event->cpu);
>> > -     bool use_contiguous_pages = event->pmu->capabilities & (
>> > -             PERF_PMU_CAP_AUX_NO_SG | PERF_PMU_CAP_AUX_PREFER_LARGE);
>> > -     /*
>> > -      * Initialize max_order to 0 for page allocation. This allocates single
>> > -      * pages to minimize memory fragmentation. This is overridden if the
>> > -      * PMU needs or prefers contiguous pages (use_contiguous_pages = true).
>> > -      */
>> > -     int max_order = 0;
>> > -     int ret = -ENOMEM;
>> > +     int ret = -ENOMEM, max_order;
>> >
>> >       if (!has_aux(event))
>> >               return -EOPNOTSUPP;
>> > @@ -702,8 +694,8 @@ int rb_alloc_aux(struct perf_buffer *rb, struct perf_event *event,
>> >
>> >       if (!overwrite) {
>> >               /*
>> > -              * Watermark defaults to half the buffer, to aid PMU drivers
>> > -              * in double buffering.
>> > +              * Watermark defaults to half the buffer, and so does the
>> > +              * max_order, to aid PMU drivers in double buffering.
>> >                */
>> >               if (!watermark)
>> >                       watermark = min_t(unsigned long,
>> > @@ -711,19 +703,16 @@ int rb_alloc_aux(struct perf_buffer *rb, struct perf_event *event,
>> >                                         (unsigned long)nr_pages << (PAGE_SHIFT - 1));
>> >
>> >               /*
>> > -              * If using contiguous pages, use aux_watermark as the basis
>> > -              * for chunking to help PMU drivers honor the watermark.
>> > +              * Use aux_watermark as the basis for chunking to
>> > +              * help PMU drivers honor the watermark.
>> >                */
>> > -             if (use_contiguous_pages)
>> > -                     max_order = get_order(watermark);
>> > +             max_order = get_order(watermark);
>> >       } else {
>> >               /*
>> > -              * If using contiguous pages, we need to start with the
>> > -              * max_order that fits in nr_pages, not the other way around,
>> > -              * hence ilog2() and not get_order.
>> > +              * We need to start with the max_order that fits in nr_pages,
>> > +              * not the other way around, hence ilog2() and not get_order.
>> >                */
>> > -             if (use_contiguous_pages)
>> > -                     max_order = ilog2(nr_pages);
>> > +             max_order = ilog2(nr_pages);
>> >               watermark = 0;
>> >       }
>> >
>> >
>> > ---
>> > base-commit: 21a7f44fd0765fcc5194130b28767786eba1d1e7
>> > change-id: 20261025-perf_revert_prefer_large_alloc-a0177d6379f6
>> >
>> > Best regards,
>>

Re: [PATCH] Revert "perf/aux: Allocate non-contiguous AUX pages by default"

[Will Deacon]

On Tue, Nov 03, 2026 at 12:46:24PM -0800, Yabin Cui wrote:
> On Sun, Oct 25, 2026 at 04:24:06PM +0000, Leo Yan wrote:
> > The original commit claimed that contiguous AUX pages unnecessarily
> > exacerbate fragmentation for SPE, TRBE and ETR. This is not generally
> > valid: larger allocations can keep pages with the same lifetime together,
> > while order-0 allocations can scatter them across large blocks and prevent
> > free pages from coalescing [1]. The follow-up discussion [2] has not
> > established how forcing order-0 allocations addresses the reported problem.
> ...
> > Restore the previous policy of trying higher-order allocations first.
> > The allocator already uses __GFP_NORETRY and falls back to smaller
> > orders, including order 0, when larger allocations fail.
> 
> Sorry for the late reply, and sorry that the commit message in
> 18049c8cff9c ("perf/aux: Allocate non-contiguous AUX pages by default")
> was imprecise about "exacerbating memory fragmentation". You and Dev are
> right that grouping same-lifetime pages into higher-order allocations
> does not itself worsen fragmentation, and the TLB reduction numbers for
> TRBE and SPE with large vmap() mappings make total sense given TRBE's
> tiny TLB.
> 
> However, a straight revert will reintroduce a severe production issue we
> hit on Android devices, so I'd like to share what actually happened and
> suggest a way to get large mappings for SPE/TRBE without bringing back
> the regression.

[...]

Thanks, Yabin. This just hardens my stance [1] that the allocation decision
should be driven by userspace and not second-guessed by the kernel. So the
revert is _not_ the right approach, in my view.

Will

[1] https://lore.kernel.org/all/ar4Iuanz_vA4twGP@willie-the-truck/

Re: [PATCH] Revert "perf/aux: Allocate non-contiguous AUX pages by default"

[Leo Yan]

On Tue, Nov 03, 2026 at 12:46:24PM -0800, Yabin Cui wrote:

...

> In field telemetry and local ftrace on Pixel devices (prior to
> 18049c8cff9c, so __GFP_NORETRY was already in place), we observed heavy
> rb_alloc_aux() -> __alloc_pages_direct_compact() stalls alongside
> kcompactd0 pinned on big cores scanning order-9, and this accounted for
> >20% of all perceptible app kills (via lmkd) across dogfood devices
> until we stopped high-order AUX allocations.

Thanks for the detailed explanation. My understanding is that repeated
reclaim/compaction during AUX allocation raises memory PSI and can lead
to lmkd killing applications. The problem to address is therefore the
cost of obtaining higher-order pages under memory fragmentation.

> Initially we worked around this in userspace (simpleperf) by setting
> perf_event_attr::aux_watermark = 4096 so get_order(watermark) == 0. That
> was fine for CoreSight ETR (which uses its own separate scatter-gather
> buffer and only copies into AUX on flush/stop), but when enabling TRBE
> (and SPE) it caused hardware maintenance interrupts every 4 KiB instead
> of every 2 MiB. Also, in overwrite/snapshot mode rb_alloc_aux() ignores
> aux_watermark and uses ilog2(nr_pages). That is why I sent 18049c8cff9c.

Agreed. Reducing watermark to 4096 also increases maintenance interrupt
frequency for SPE/TRBE. We should refine buffer allocation rather than
using watermark as a workaround.

> ### Possible solutions
> 
> For PMUs without PERF_PMU_CAP_AUX_NO_SG (such as SPE, TRBE, and modern
> Intel PT), higher-order AUX pages are an opportunistic optimization
> rather than a functional requirement -- very similar to THP allocations
> on fault (GFP_TRANSHUGE_LIGHT) or opportunistic high-order slabs in SLUB
> (allocate_slab()).
> 
> Instead of reverting to GFP_KERNEL reclaim/compaction for all high-order
> attempts, could we do the following?
> 
> 1. Make opportunistic high-order AUX allocations no-reclaim:
>    When !(event->pmu->capabilities & PERF_PMU_CAP_AUX_NO_SG) and
>    order > 0, clear __GFP_RECLAIM (and add __GFP_NOMEMALLOC, matching
>    SLUB's allocate_slab() and GFP_TRANSHUGE_LIGHT):
> 
>      gfp_t gfp = PERF_AUX_GFP;
> 
>      if (order && !has_no_sg)
>              gfp = (PERF_AUX_GFP | __GFP_NOMEMALLOC) & ~__GFP_RECLAIM;
> 
>    When order falls back to 0 (or when PERF_PMU_CAP_AUX_NO_SG is set),
>    keep PERF_AUX_GFP so single-page allocations still reclaim normally
>    under memory pressure.
> 
>    That way, when free order-9 / order-4 blocks exist, SPE and TRBE get
>    them immediately for PMD / contiguous-PTE vmap() mappings (and pages
>    stay grouped together); when memory is fragmented on a long-running
>    device, high-order probes fail fast on the freelist without waking
>    kcompactd/kswapd or triggering direct compaction and reclaim.

This looks reasonable for me: try to obtain larger blocks cheaply, then
then retain the PERF_AUX_GFP policy for order-0 allocations. I also
agree with preserving the allocation policy for PERF_PMU_CAP_AUX_NO_SG
case.

Could you compare the memory PSI stall deltas before and after applying
this option? I think measurements from your production devices would be
convinced evidence for benefits.

> 2. Clamp max_order in rb_alloc_aux() after fallback:
>    If rb_alloc_aux_page() requested `order` and had to fall back to a
>    smaller order, update `max_order = page_private(page)` in
>    rb_alloc_aux() so subsequent iterations for the rest of the buffer do
>    not keep re-probing higher orders that just failed.

This also can avoid reclaim repeatedly, but the side effect is subsequent
iterations would no longer try high order, which have no chance to
benefit from large mapping.

I would prefer to evaluate option 1 first.

> (Optionally, if server/benchmark workloads ever want synchronous
> compaction to form huge AUX mappings even on fragmented hosts, that could
> be gated behind an opt-in attr/sysfs knob similar to THP defrag modes,
> with light/no-reclaim as the default.)

I will follow up with Will separately about the userspace interface.

> Would this approach work for your SPE/TRBE use cases? Happy to help test
> on our side.

If option 1 works, we can confidently revert 18049c8cff9c, as it would
restore higher-order allocations for TRBE/SPE without introducing the
regression you mentioned.

Thanks,
Leo

Re: [PATCH] Revert "perf/aux: Allocate non-contiguous AUX pages by default"

[Yabin Cui]

On Thu, Nov 5, 2026 at 9:35 AM Leo Yan <leo.yan@arm.com> wrote:
>
> On Tue, Nov 03, 2026 at 12:46:24PM -0800, Yabin Cui wrote:
>
> ...
>
> > In field telemetry and local ftrace on Pixel devices (prior to
> > 18049c8cff9c, so __GFP_NORETRY was already in place), we observed heavy
> > rb_alloc_aux() -> __alloc_pages_direct_compact() stalls alongside
> > kcompactd0 pinned on big cores scanning order-9, and this accounted for
> > >20% of all perceptible app kills (via lmkd) across dogfood devices
> > until we stopped high-order AUX allocations.
>
> Thanks for the detailed explanation. My understanding is that repeated
> reclaim/compaction during AUX allocation raises memory PSI and can lead
> to lmkd killing applications. The problem to address is therefore the
> cost of obtaining higher-order pages under memory fragmentation.
>
> > Initially we worked around this in userspace (simpleperf) by setting
> > perf_event_attr::aux_watermark = 4096 so get_order(watermark) == 0. That
> > was fine for CoreSight ETR (which uses its own separate scatter-gather
> > buffer and only copies into AUX on flush/stop), but when enabling TRBE
> > (and SPE) it caused hardware maintenance interrupts every 4 KiB instead
> > of every 2 MiB. Also, in overwrite/snapshot mode rb_alloc_aux() ignores
> > aux_watermark and uses ilog2(nr_pages). That is why I sent 18049c8cff9c.
>
> Agreed. Reducing watermark to 4096 also increases maintenance interrupt
> frequency for SPE/TRBE. We should refine buffer allocation rather than
> using watermark as a workaround.
>
> > ### Possible solutions
> >
> > For PMUs without PERF_PMU_CAP_AUX_NO_SG (such as SPE, TRBE, and modern
> > Intel PT), higher-order AUX pages are an opportunistic optimization
> > rather than a functional requirement -- very similar to THP allocations
> > on fault (GFP_TRANSHUGE_LIGHT) or opportunistic high-order slabs in SLUB
> > (allocate_slab()).
> >
> > Instead of reverting to GFP_KERNEL reclaim/compaction for all high-order
> > attempts, could we do the following?
> >
> > 1. Make opportunistic high-order AUX allocations no-reclaim:
> >    When !(event->pmu->capabilities & PERF_PMU_CAP_AUX_NO_SG) and
> >    order > 0, clear __GFP_RECLAIM (and add __GFP_NOMEMALLOC, matching
> >    SLUB's allocate_slab() and GFP_TRANSHUGE_LIGHT):
> >
> >      gfp_t gfp = PERF_AUX_GFP;
> >
> >      if (order && !has_no_sg)
> >              gfp = (PERF_AUX_GFP | __GFP_NOMEMALLOC) & ~__GFP_RECLAIM;
> >
> >    When order falls back to 0 (or when PERF_PMU_CAP_AUX_NO_SG is set),
> >    keep PERF_AUX_GFP so single-page allocations still reclaim normally
> >    under memory pressure.
> >
> >    That way, when free order-9 / order-4 blocks exist, SPE and TRBE get
> >    them immediately for PMD / contiguous-PTE vmap() mappings (and pages
> >    stay grouped together); when memory is fragmented on a long-running
> >    device, high-order probes fail fast on the freelist without waking
> >    kcompactd/kswapd or triggering direct compaction and reclaim.
>
> This looks reasonable for me: try to obtain larger blocks cheaply, then
> then retain the PERF_AUX_GFP policy for order-0 allocations. I also
> agree with preserving the allocation policy for PERF_PMU_CAP_AUX_NO_SG
> case.
>
> Could you compare the memory PSI stall deltas before and after applying
> this option? I think measurements from your production devices would be
> convinced evidence for benefits.

Hi Leo,

Unfortunately I'm tied up with other priorities at the moment and don't
have the bandwidth to put together and upstream the kernel patches
myself.
Whether you and Will decide to go with Option 1, a userspace-controlled
interface, or a combination of both, please feel free to take it forward
and send the patches. Once you have a patch series ready, please CC me
-- I'll be very happy to test it on Android.

Thanks,
Yabin

>
> > 2. Clamp max_order in rb_alloc_aux() after fallback:
> >    If rb_alloc_aux_page() requested `order` and had to fall back to a
> >    smaller order, update `max_order = page_private(page)` in
> >    rb_alloc_aux() so subsequent iterations for the rest of the buffer do
> >    not keep re-probing higher orders that just failed.
>
> This also can avoid reclaim repeatedly, but the side effect is subsequent
> iterations would no longer try high order, which have no chance to
> benefit from large mapping.
>
> I would prefer to evaluate option 1 first.
>
> > (Optionally, if server/benchmark workloads ever want synchronous
> > compaction to form huge AUX mappings even on fragmented hosts, that could
> > be gated behind an opt-in attr/sysfs knob similar to THP defrag modes,
> > with light/no-reclaim as the default.)
>
> I will follow up with Will separately about the userspace interface.
>
> > Would this approach work for your SPE/TRBE use cases? Happy to help test
> > on our side.
>
> If option 1 works, we can confidently revert 18049c8cff9c, as it would
> restore higher-order allocations for TRBE/SPE without introducing the
> regression you mentioned.
>
> Thanks,
> Leo

Re: [PATCH] Revert "perf/aux: Allocate non-contiguous AUX pages by default"

[Leo Yan]

On Wed, Nov 04, 2026 at 12:59:13PM +0000, Will Deacon wrote:

[...]

> > Sorry for the late reply, and sorry that the commit message in
> > 18049c8cff9c ("perf/aux: Allocate non-contiguous AUX pages by default")
> > was imprecise about "exacerbating memory fragmentation". You and Dev are
> > right that grouping same-lifetime pages into higher-order allocations
> > does not itself worsen fragmentation, and the TLB reduction numbers for
> > TRBE and SPE with large vmap() mappings make total sense given TRBE's
> > tiny TLB.
> > 
> > However, a straight revert will reintroduce a severe production issue we
> > hit on Android devices, so I'd like to share what actually happened and
> > suggest a way to get large mappings for SPE/TRBE without bringing back
> > the regression.
> 
> [...]
> 
> Thanks, Yabin. This just hardens my stance [1] that the allocation decision
> should be driven by userspace and not second-guessed by the kernel. So the
> revert is _not_ the right approach, in my view.

If a userspace interface is needed, what remaining reclaim or allocation
policy should userspace control? I am not clear how userspace control
would fix the PSI reclaim stalls described in Yabin's reply.

Alternatively, we could optimize the reported reclaim cost in the kernel
by clearing __GFP_RECLAIM for optional higher-order attempts, while
preserving the existing policy at order 0 and for
PERF_PMU_CAP_AUX_NO_SG PMUs.

If testing confirms this works, combining this mitigation with a revert
of 18049c8cff9c would be a reasonable way to restore higher-order
allocations for SPE/TRBE without adding a userspace interface.

Thanks,
Leo

[tip: perf/urgent] Revert "perf/aux: Allocate non-contiguous AUX pages by default"

[tip-bot2 for Leo Yan]

The following commit has been merged into the perf/urgent branch of tip:

Commit-ID:     526d945ad9c09d42f2482f058e1f41376681627e
Gitweb:        https://git.kernel.org/tip/526d945ad9c09d42f2482f058e1f41376681627e
Author:        Leo Yan <leo.yan@arm.com>
AuthorDate:    Sun, 25 Oct 2026 16:24:06 
Committer:     Ingo Molnar <mingo@kernel.org>
CommitterDate: Fri, 06 Nov 2026 09:50:37 +01:00

Revert "perf/aux: Allocate non-contiguous AUX pages by default"

This reverts commit 18049c8cff9cc89daadc4df6975f7d9069638926.

The original commit claimed that contiguous AUX pages unnecessarily
exacerbate fragmentation for SPE, TRBE and ETR. This is not generally
valid: larger allocations can keep pages with the same lifetime
together, while order-0 allocations can scatter them across large
blocks and prevent free pages from coalescing [1]. The follow-up
discussion [2] has not established how forcing order-0 allocations
addresses the reported problem.

TRBE and SPE write to virtually addressed buffers. The TRBE L1 TLB
has one entry on Neoverse V2 [3] and two on Neoverse V3 [4]. Small
mappings increase translation pressure, and TTW stalls can contribute
to FIFO overflow and trace loss on TRBE.

Larger allocations provide contiguous, aligned chunks for PMD or
contiguous-PTE mappings with the proposed vmap() support [5].
With the proposed vmap() support already applied, the tests in [2]
showed that preferring larger AUX allocations reduced L2 data TLB
refills by 99.68% in the TRBE test and 34.74% in the SPE test.

Restore the previous policy of trying higher-order allocations first.
The allocator already uses __GFP_NORETRY and falls back to smaller
orders, including order 0, when larger allocations fail.

Remove PERF_PMU_CAP_AUX_PREFER_LARGE, since the restored default
provides that preference. Intel PT retains its existing allocation
behaviour, with no functional change.

[1] https://lore.kernel.org/all/aPjrRkjiIt6HmXmT@casper.infradead.org/
[2] https://lore.kernel.org/linux-arm-kernel/20260810-perf_aux_trace_large_granule-v1-0-03306c9339e3@arm.com/
[3] https://documentation-service.arm.com/static/633fe1ec4c59b30b517730b8#page=52
[4] https://documentation-service.arm.com/static/67361bd2c7fc0d1f211da382#page=54
[5] https://lore.kernel.org/linux-mm/20260715120813.3609949-1-jiangwen6@xiaomi.com/

Signed-off-by: Leo Yan <leo.yan@arm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: James Clark <james.clark@linaro.org>
Reviewed-by: Dev Jain <dev.jain@arm.com>
Link: https://patch.msgid.link/20261025-perf_revert_prefer_large_alloc-v1-1-5fb431bbd624@arm.com
---
 arch/x86/events/intel/pt.c  |  2 --
 include/linux/perf_event.h  |  5 ++---
 kernel/events/ring_buffer.c | 29 +++++++++--------------------
 3 files changed, 11 insertions(+), 25 deletions(-)

diff --git a/arch/x86/events/intel/pt.c b/arch/x86/events/intel/pt.c
index 49349af..3782af4 100644
--- a/arch/x86/events/intel/pt.c
+++ b/arch/x86/events/intel/pt.c
@@ -1896,8 +1896,6 @@ static __init int pt_init(void)
 
 	if (!intel_pt_validate_hw_cap(PT_CAP_topa_multiple_entries))
 		pt_pmu.pmu.capabilities = PERF_PMU_CAP_AUX_NO_SG;
-	else
-		pt_pmu.pmu.capabilities = PERF_PMU_CAP_AUX_PREFER_LARGE;
 
 	pt_pmu.pmu.capabilities		|= PERF_PMU_CAP_EXCLUSIVE |
 					   PERF_PMU_CAP_ITRACE |
diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
index 7884489..e1e1150 100644
--- a/include/linux/perf_event.h
+++ b/include/linux/perf_event.h
@@ -304,9 +304,8 @@ struct perf_event_pmu_context;
 #define PERF_PMU_CAP_AUX_OUTPUT		0x0080
 #define PERF_PMU_CAP_EXTENDED_HW_TYPE	0x0100
 #define PERF_PMU_CAP_AUX_PAUSE		0x0200
-#define PERF_PMU_CAP_AUX_PREFER_LARGE	0x0400
-#define PERF_PMU_CAP_MEDIATED_VPMU	0x0800
-#define PERF_PMU_CAP_SIMD_REGS		0x1000
+#define PERF_PMU_CAP_MEDIATED_VPMU	0x0400
+#define PERF_PMU_CAP_SIMD_REGS		0x0800
 
 /**
  * pmu::scope
diff --git a/kernel/events/ring_buffer.c b/kernel/events/ring_buffer.c
index 64d4dbd..95f4fbd 100644
--- a/kernel/events/ring_buffer.c
+++ b/kernel/events/ring_buffer.c
@@ -684,15 +684,7 @@ int rb_alloc_aux(struct perf_buffer *rb, struct perf_event *event,
 {
 	bool overwrite = !(flags & RING_BUFFER_WRITABLE);
 	int node = (event->cpu == -1) ? -1 : cpu_to_node(event->cpu);
-	bool use_contiguous_pages = event->pmu->capabilities & (
-		PERF_PMU_CAP_AUX_NO_SG | PERF_PMU_CAP_AUX_PREFER_LARGE);
-	/*
-	 * Initialize max_order to 0 for page allocation. This allocates single
-	 * pages to minimize memory fragmentation. This is overridden if the
-	 * PMU needs or prefers contiguous pages (use_contiguous_pages = true).
-	 */
-	int max_order = 0;
-	int ret = -ENOMEM;
+	int ret = -ENOMEM, max_order;
 
 	if (!has_aux(event))
 		return -EOPNOTSUPP;
@@ -702,8 +694,8 @@ int rb_alloc_aux(struct perf_buffer *rb, struct perf_event *event,
 
 	if (!overwrite) {
 		/*
-		 * Watermark defaults to half the buffer, to aid PMU drivers
-		 * in double buffering.
+		 * Watermark defaults to half the buffer, and so does the
+		 * max_order, to aid PMU drivers in double buffering.
 		 */
 		if (!watermark)
 			watermark = min_t(unsigned long,
@@ -711,19 +703,16 @@ int rb_alloc_aux(struct perf_buffer *rb, struct perf_event *event,
 					  (unsigned long)nr_pages << (PAGE_SHIFT - 1));
 
 		/*
-		 * If using contiguous pages, use aux_watermark as the basis
-		 * for chunking to help PMU drivers honor the watermark.
+		 * Use aux_watermark as the basis for chunking to
+		 * help PMU drivers honor the watermark.
 		 */
-		if (use_contiguous_pages)
-			max_order = get_order(watermark);
+		max_order = get_order(watermark);
 	} else {
 		/*
-		 * If using contiguous pages, we need to start with the
-		 * max_order that fits in nr_pages, not the other way around,
-		 * hence ilog2() and not get_order.
+		 * We need to start with the max_order that fits in nr_pages,
+		 * not the other way around, hence ilog2() and not get_order.
 		 */
-		if (use_contiguous_pages)
-			max_order = ilog2(nr_pages);
+		max_order = ilog2(nr_pages);
 		watermark = 0;
 	}