mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* [PATCH v2] mm: shmem: limit forced anonymous shmem huge orders to MAX_PAGECACHE_ORDER
@ 2026-11-18  2:45 Jingyi Sun
  2026-11-18  9:31 ` David Hildenbrand (Arm)
  0 siblings, 1 reply; 3+ messages in thread
From: Jingyi Sun @ 2026-11-18  2:45 UTC (permalink / raw)
  To: Hugh Dickins, David Hildenbrand, Andrew Morton
  Cc: Baolin Wang, linux-mm, linux-kernel, stable, Luiz Capitulino

When the top-level shmem_enabled knob is set to "force",
shmem_allowable_huge_orders() returns huge_shmem_orders_inherit for
anonymous shmem without masking it with THP_ORDERS_ALL_FILE_DEFAULT,
unlike the other return path at the end of the same function.

huge_shmem_orders_inherit defaults to BIT(HPAGE_PMD_ORDER). On arm64
with 64K base pages HPAGE_PMD_ORDER is 13, while MAX_PAGECACHE_ORDER
is 11 because of the xarray split limit. With shmem_enabled=force, an
anonymous MAP_SHARED mapping can therefore get a 512M (order-13) folio
in the page cache. Splitting it later, e.g. on swap-out from
shmem_writeout(), triggers

  WARNING: lib/xarray.c:1051 at xas_split_alloc

and the split fails, so the folio can never be swapped out.

This is the same problem that commit 9fd154ba926b ("mm/shmem: disable
PMD-sized page cache if needed") and commit b66b1b71d7ff ("mm: shmem:
avoid allocating huge pages larger than MAX_PAGECACHE_ORDER for shmem")
fixed for the other paths; the force branch was left out. Mask it with
THP_ORDERS_ALL_FILE_DEFAULT as well.

Reproduced on v7.3-rc2 in QEMU (arm64, 64K pages, -cpu cortex-a72,
4G RAM, zram swap): with shmem_enabled=force, a PMD-aligned
MAP_SHARED|MAP_ANONYMOUS mapping of one PMD size got a single
PMD-mapped folio (ShmemPmdMapped: 524288 kB) and MADV_PAGEOUT
triggered the warning above in 3/3 runs. With this change: 0/3.
shmem_enabled=always and 4K pages did not show the warning. The code is
unchanged in v7.4-rc3 and in mm-unstable, but the test has not been
re-run on those trees.

The issue was found with a research static analyzer for
cross-architecture dependency bugs and confirmed with the QEMU
reproducer described above.

Fixes: e7a2ab7b3bb5 ("mm: shmem: add mTHP support for anonymous shmem")
Cc: stable@vger.kernel.org
Cc: Luiz Capitulino <luizcap@redhat.com>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Signed-off-by: Jingyi Sun <1045704958@qq.com>
---
Changes in v2:
- Dropped the Assisted-by tag (Baolin).
- Collected Reviewed-by from Baolin.
- Dropped the note about the overlap with Luiz's mTHP series; Luiz
  confirmed his series will land after this patch.

v1: https://lore.kernel.org/all/20261117121626.1058-1-1045704958@qq.com/

 mm/shmem.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/mm/shmem.c b/mm/shmem.c
index 10cef89b00b0..41bb3f2c6b63 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -2114,7 +2114,8 @@ unsigned long shmem_allowable_huge_orders(struct inode *inode,
 	 * means non-PMD sized THP can not override 'huge' mount option now.
 	 */
 	if (shmem_huge == SHMEM_HUGE_FORCE)
-		return READ_ONCE(huge_shmem_orders_inherit);
+		return THP_ORDERS_ALL_FILE_DEFAULT &
+		       READ_ONCE(huge_shmem_orders_inherit);
 
 	/* Allow mTHP that will be fully within i_size. */
 	mask |= shmem_get_orders_within_size(inode, within_size_orders, index, 0);

base-commit: 666539eecdf4db623f5e3c3e55cf9f2dfad24aa4
-- 
2.43.0


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [PATCH v2] mm: shmem: limit forced anonymous shmem huge orders to MAX_PAGECACHE_ORDER
  2026-11-18  2:45 [PATCH v2] mm: shmem: limit forced anonymous shmem huge orders to MAX_PAGECACHE_ORDER Jingyi Sun
@ 2026-11-18  9:31 ` David Hildenbrand (Arm)
  2026-11-18 10:13   ` Baolin Wang
  0 siblings, 1 reply; 3+ messages in thread
From: David Hildenbrand (Arm) @ 2026-11-18  9:31 UTC (permalink / raw)
  To: Jingyi Sun, Hugh Dickins, Andrew Morton
  Cc: Baolin Wang, linux-mm, linux-kernel, stable, Luiz Capitulino

On 11/18/26 03:45, Jingyi Sun wrote:
> When the top-level shmem_enabled knob is set to "force",
> shmem_allowable_huge_orders() returns huge_shmem_orders_inherit for
> anonymous shmem without masking it with THP_ORDERS_ALL_FILE_DEFAULT,
> unlike the other return path at the end of the same function.
> 
> huge_shmem_orders_inherit defaults to BIT(HPAGE_PMD_ORDER). On arm64
> with 64K base pages HPAGE_PMD_ORDER is 13, while MAX_PAGECACHE_ORDER
> is 11 because of the xarray split limit. With shmem_enabled=force, an
> anonymous MAP_SHARED mapping can therefore get a 512M (order-13) folio
> in the page cache. Splitting it later, e.g. on swap-out from
> shmem_writeout(), triggers
> 
>   WARNING: lib/xarray.c:1051 at xas_split_alloc
> 
> and the split fails, so the folio can never be swapped out.
> 
> This is the same problem that commit 9fd154ba926b ("mm/shmem: disable
> PMD-sized page cache if needed") and commit b66b1b71d7ff ("mm: shmem:
> avoid allocating huge pages larger than MAX_PAGECACHE_ORDER for shmem")
> fixed for the other paths; the force branch was left out. Mask it with
> THP_ORDERS_ALL_FILE_DEFAULT as well.
> 
> Reproduced on v7.3-rc2 in QEMU (arm64, 64K pages, -cpu cortex-a72,
> 4G RAM, zram swap): with shmem_enabled=force, a PMD-aligned
> MAP_SHARED|MAP_ANONYMOUS mapping of one PMD size got a single
> PMD-mapped folio (ShmemPmdMapped: 524288 kB) and MADV_PAGEOUT
> triggered the warning above in 3/3 runs. With this change: 0/3.
> shmem_enabled=always and 4K pages did not show the warning. The code is
> unchanged in v7.4-rc3 and in mm-unstable, but the test has not been
> re-run on those trees.
> 
> The issue was found with a research static analyzer for
> cross-architecture dependency bugs and confirmed with the QEMU
> reproducer described above.
> 
> Fixes: e7a2ab7b3bb5 ("mm: shmem: add mTHP support for anonymous shmem")
> Cc: stable@vger.kernel.org
> Cc: Luiz Capitulino <luizcap@redhat.com>
> Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
> Signed-off-by: Jingyi Sun <1045704958@qq.com>
> ---
> Changes in v2:
> - Dropped the Assisted-by tag (Baolin).
> - Collected Reviewed-by from Baolin.
> - Dropped the note about the overlap with Luiz's mTHP series; Luiz
>   confirmed his series will land after this patch.
> 
> v1: https://lore.kernel.org/all/20261117121626.1058-1-1045704958@qq.com/
> 
>  mm/shmem.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/mm/shmem.c b/mm/shmem.c
> index 10cef89b00b0..41bb3f2c6b63 100644
> --- a/mm/shmem.c
> +++ b/mm/shmem.c
> @@ -2114,7 +2114,8 @@ unsigned long shmem_allowable_huge_orders(struct inode *inode,
>  	 * means non-PMD sized THP can not override 'huge' mount option now.
>  	 */
>  	if (shmem_huge == SHMEM_HUGE_FORCE)
> -		return READ_ONCE(huge_shmem_orders_inherit);
> +		return THP_ORDERS_ALL_FILE_DEFAULT &
> +		       READ_ONCE(huge_shmem_orders_inherit);

could we instead set

	mask = READ_ONCE(huge_shmem_orders_inherit);

And sue a goto to the end of the function  where we have

	return THP_ORDERS_ALL_FILE_DEFAULT & mask;

Avoiding the goto means more churn I guess.

-- 
Cheers,

David

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [PATCH v2] mm: shmem: limit forced anonymous shmem huge orders to MAX_PAGECACHE_ORDER
  2026-11-18  9:31 ` David Hildenbrand (Arm)
@ 2026-11-18 10:13   ` Baolin Wang
  0 siblings, 0 replies; 3+ messages in thread
From: Baolin Wang @ 2026-11-18 10:13 UTC (permalink / raw)
  To: David Hildenbrand (Arm), Jingyi Sun, Hugh Dickins, Andrew Morton
  Cc: linux-mm, linux-kernel, stable, Luiz Capitulino



On 11/18/26 5:31 PM, David Hildenbrand (Arm) wrote:
> On 11/18/26 03:45, Jingyi Sun wrote:
>> When the top-level shmem_enabled knob is set to "force",
>> shmem_allowable_huge_orders() returns huge_shmem_orders_inherit for
>> anonymous shmem without masking it with THP_ORDERS_ALL_FILE_DEFAULT,
>> unlike the other return path at the end of the same function.
>>
>> huge_shmem_orders_inherit defaults to BIT(HPAGE_PMD_ORDER). On arm64
>> with 64K base pages HPAGE_PMD_ORDER is 13, while MAX_PAGECACHE_ORDER
>> is 11 because of the xarray split limit. With shmem_enabled=force, an
>> anonymous MAP_SHARED mapping can therefore get a 512M (order-13) folio
>> in the page cache. Splitting it later, e.g. on swap-out from
>> shmem_writeout(), triggers
>>
>>    WARNING: lib/xarray.c:1051 at xas_split_alloc
>>
>> and the split fails, so the folio can never be swapped out.
>>
>> This is the same problem that commit 9fd154ba926b ("mm/shmem: disable
>> PMD-sized page cache if needed") and commit b66b1b71d7ff ("mm: shmem:
>> avoid allocating huge pages larger than MAX_PAGECACHE_ORDER for shmem")
>> fixed for the other paths; the force branch was left out. Mask it with
>> THP_ORDERS_ALL_FILE_DEFAULT as well.
>>
>> Reproduced on v7.3-rc2 in QEMU (arm64, 64K pages, -cpu cortex-a72,
>> 4G RAM, zram swap): with shmem_enabled=force, a PMD-aligned
>> MAP_SHARED|MAP_ANONYMOUS mapping of one PMD size got a single
>> PMD-mapped folio (ShmemPmdMapped: 524288 kB) and MADV_PAGEOUT
>> triggered the warning above in 3/3 runs. With this change: 0/3.
>> shmem_enabled=always and 4K pages did not show the warning. The code is
>> unchanged in v7.4-rc3 and in mm-unstable, but the test has not been
>> re-run on those trees.
>>
>> The issue was found with a research static analyzer for
>> cross-architecture dependency bugs and confirmed with the QEMU
>> reproducer described above.
>>
>> Fixes: e7a2ab7b3bb5 ("mm: shmem: add mTHP support for anonymous shmem")
>> Cc: stable@vger.kernel.org
>> Cc: Luiz Capitulino <luizcap@redhat.com>
>> Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
>> Signed-off-by: Jingyi Sun <1045704958@qq.com>
>> ---
>> Changes in v2:
>> - Dropped the Assisted-by tag (Baolin).
>> - Collected Reviewed-by from Baolin.
>> - Dropped the note about the overlap with Luiz's mTHP series; Luiz
>>    confirmed his series will land after this patch.
>>
>> v1: https://lore.kernel.org/all/20261117121626.1058-1-1045704958@qq.com/
>>
>>   mm/shmem.c | 3 ++-
>>   1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/mm/shmem.c b/mm/shmem.c
>> index 10cef89b00b0..41bb3f2c6b63 100644
>> --- a/mm/shmem.c
>> +++ b/mm/shmem.c
>> @@ -2114,7 +2114,8 @@ unsigned long shmem_allowable_huge_orders(struct inode *inode,
>>   	 * means non-PMD sized THP can not override 'huge' mount option now.
>>   	 */
>>   	if (shmem_huge == SHMEM_HUGE_FORCE)
>> -		return READ_ONCE(huge_shmem_orders_inherit);
>> +		return THP_ORDERS_ALL_FILE_DEFAULT &
>> +		       READ_ONCE(huge_shmem_orders_inherit);
> 
> could we instead set
> 
> 	mask = READ_ONCE(huge_shmem_orders_inherit);
> 
> And sue a goto to the end of the function  where we have
> 
> 	return THP_ORDERS_ALL_FILE_DEFAULT & mask;
> 
> Avoiding the goto means more churn I guess.

Yes. This fix seems more simple, but no strong opinion on this:) .

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2026-11-18 10:14 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-18  2:45 [PATCH v2] mm: shmem: limit forced anonymous shmem huge orders to MAX_PAGECACHE_ORDER Jingyi Sun
2026-11-18  9:31 ` David Hildenbrand (Arm)
2026-11-18 10:13   ` Baolin Wang

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®