[PATCH v1 0/5] mm/shmem: optimize read with reduced xarray lookups and folio batching

[PATCH v1 0/5] mm/shmem: optimize read with reduced xarray lookups and folio batching

[Chi Zhiling]

From: Chi Zhiling <chizhiling@kylinos.cn>

This series improves shmem read performance by implementing folio
batching in the read path and reducing unnecessary xarray lookups.


Changes since RFC
=================

The RFC version used xas_for_each() in shmem_get_read_batch(), which
introduced about a 1% regression for 4K read workloads.

This v1 addresses the regression in patch 2 by switching to
filemap_get_folios_contig() and optimizing it to avoid the extra
xarray traversal overhead.


Performance Results
===================

Testing was performed with fio sequential read workloads:

  fio --ioengine=sync --rw=read --size=1G --runtime=180


### THP Disabled - Normal Files ###

| Block Size | Baseline  | v1        | Improvement |
| ---------- | --------- | --------- | ----------- |
| 1M         | 11.4GiB/s | 12.7GiB/s | +11.4%      |
| 64k        | 11.2GiB/s | 12.2GiB/s | +8.9%       |
| 4k         | 3809MiB/s | 3838MiB/s | +0.8%       |

### THP Disabled - Fallocated Files ###

| Block Size | Baseline  | v1        | Improvement |
| ---------- | --------- | --------- | ----------- |
| 1M         | 23.7GiB/s | 28.7GiB/s | +21.1%      |
| 64k        | 22.6GiB/s | 27.0GiB/s | +19.5%      |
| 4k         | 4668MiB/s | 4678MiB/s | +0.2%       |

### THP Enabled - Normal Files ###

| Block Size | Baseline  | v1        | Improvement |
| ---------- | --------- | --------- | ----------- |
| 1M         | 13.9GiB/s | 13.9GiB/s | 0%          |
| 64k        | 13.4GiB/s | 13.4GiB/s | 0%          |
| 4k         | 3818MiB/s | 3836MiB/s | +0.5%       |

### THP Enabled - Fallocated Files ###

| Block Size | Baseline  | v1        | Improvement |
| ---------- | --------- | --------- | ----------- |
| 1M         | 24.1GiB/s | 34.9GiB/s | +44.8%      |
| 64k        | 22.9GiB/s | 31.3GiB/s | +36.7%      |
| 4k         | 4721MiB/s | 4708MiB/s | -0.3%       |


rfc:
https://lore.kernel.org/linux-fsdevel/20260515094702.1092355-1-chizhiling@163.com/


Chi Zhiling (5):
  mm/filemap: reduce unnecessary xarray lookups when read cached pages
  mm/filemap: reduce xarray lookups in filemap_get_folios_contig()
  mm/shmem: make SGP_NOALLOC succeed on hole like SGP_READ
  mm/shmem: introduce copy_zero_to_iter() for large zeroing
  mm/shmem: optimize file read with folio batching

 include/linux/shmem_fs.h |  2 +-
 mm/filemap.c             | 34 ++++++++++++------
 mm/khugepaged.c          |  2 +-
 mm/shmem.c               | 75 +++++++++++++++++++++++++++++-----------
 4 files changed, 79 insertions(+), 34 deletions(-)

-- 
2.43.0

[PATCH v1 1/5] mm/filemap: reduce unnecessary xarray lookups when read cached pages

[Chi Zhiling]

From: Chi Zhiling <chizhiling@kylinos.cn>

When reading small amounts of data from the page cache, only a single
folio is typically returned from filemap_read_get_batch(). In this case,
calling xas_advance() or xas_next() after adding the folio to the batch
is unnecessary and only introduces extra branches.

The same issue exists for large reads, where one additional xarray walk
is always performed before termination.

Move the boundary check to after the folio is added to the batch so the
final redundant xarray advancement can be avoided. This significantly
reduces the branch count in the read path.

xas_next() does not update xa_index when xas->xa_node is set to
XAS_RESTART, so checking the boundary before updating xa_index is
sufficient to keep the folio within range. The warning should therefore
never trigger.

The branch count:
654.198 M/sec -> 646.444 M/sec

Performance counter stats for 'fio --ioengine=sync --rw=read --bs=4k --size=1G
--runtime=300 --time_based --group_reporting --name=seq_read_test --filename=file':

before:
READ: bw=2697MiB/s (2828MB/s), 2697MiB/s-2697MiB/s (2828MB/s-2828MB/s), io=790GiB (848GB), run=300001-300001msec
      245602051556      task-clock                       #    0.821 CPUs utilized
             78467      context-switches                 #  319.488 /sec
                40      cpu-migrations                   #    0.163 /sec
              3388      page-faults                      #   13.795 /sec
      758312319204      instructions                     #    0.74  insn per cycle
     1025881497502      cycles                           #    4.177 GHz
      160672383734      branches                         #  654.198 M/sec
         361904512      branch-misses                    #    0.23% of all branches

after:
READ: bw=2709MiB/s (2841MB/s), 2709MiB/s-2709MiB/s (2841MB/s-2841MB/s), io=794GiB (852GB), run=300000-300000msec
      243985503670      task-clock                       #    0.812 CPUs utilized
             79004      context-switches                 #  323.806 /sec
                30      cpu-migrations                   #    0.123 /sec
              3355      page-faults                      #   13.751 /sec
      747830935069      instructions                     #    0.73  insn per cycle
     1019609333322      cycles                           #    4.179 GHz
      157722976668      branches                         #  646.444 M/sec
         348984893      branch-misses                    #    0.22% of all branches

Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
---
 mm/filemap.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/mm/filemap.c b/mm/filemap.c
index 4e636647100c..d54450e529bd 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -2458,12 +2458,16 @@ static void filemap_get_read_batch(struct address_space *mapping,
 {
 	XA_STATE(xas, &mapping->i_pages, index);
 	struct folio *folio;
+	pgoff_t next;
+
+	if (unlikely(index > max))
+		return;
 
 	rcu_read_lock();
 	for (folio = xas_load(&xas); folio; folio = xas_next(&xas)) {
 		if (xas_retry(&xas, folio))
 			continue;
-		if (xas.xa_index > max || xa_is_value(folio))
+		if (xa_is_value(folio) || WARN_ON(xas.xa_index > max))
 			break;
 		if (xa_is_sibling(folio))
 			break;
@@ -2479,7 +2483,11 @@ static void filemap_get_read_batch(struct address_space *mapping,
 			break;
 		if (folio_test_readahead(folio))
 			break;
-		xas_advance(&xas, folio_next_index(folio) - 1);
+
+		next = folio_next_index(folio);
+		if (next > max)
+			break;
+		xas_advance(&xas, next - 1);
 		continue;
 put_folio:
 		folio_put(folio);
-- 
2.43.0

[PATCH v1 2/5] mm/filemap: reduce xarray lookups in filemap_get_folios_contig()

[Chi Zhiling]

From: Chi Zhiling <chizhiling@kylinos.cn>

Apply the same optimization used in filemap_get_read_batch() by moving
the boundary check from the loop condition to before xas_advance(),
avoiding an unnecessary xarray lookup and reducing branches in the fast
path.

Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
---
 mm/filemap.c | 22 +++++++++++++---------
 1 file changed, 13 insertions(+), 9 deletions(-)

diff --git a/mm/filemap.c b/mm/filemap.c
index d54450e529bd..5a5268a7c802 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -2258,11 +2258,14 @@ unsigned filemap_get_folios_contig(struct address_space *mapping,
 	XA_STATE(xas, &mapping->i_pages, *start);
 	unsigned long nr;
 	struct folio *folio;
+	pgoff_t next;
+
+	if (unlikely(*start > end))
+		return 0;
 
 	rcu_read_lock();
 
-	for (folio = xas_load(&xas); folio && xas.xa_index <= end;
-			folio = xas_next(&xas)) {
+	for (folio = xas_load(&xas); folio; folio = xas_next(&xas)) {
 		if (xas_retry(&xas, folio))
 			continue;
 		/*
@@ -2270,11 +2273,11 @@ unsigned filemap_get_folios_contig(struct address_space *mapping,
 		 * No current caller is looking for DAX entries.
 		 */
 		if (xa_is_value(folio))
-			goto update_start;
+			break;
 
 		/* If we landed in the middle of a THP, continue at its end. */
 		if (xa_is_sibling(folio))
-			goto update_start;
+			break;
 
 		if (!folio_try_get(folio))
 			goto retry;
@@ -2282,12 +2285,15 @@ unsigned filemap_get_folios_contig(struct address_space *mapping,
 		if (unlikely(folio != xas_reload(&xas)))
 			goto put_folio;
 
+		next = folio_next_index(folio);
 		if (!folio_batch_add(fbatch, folio)) {
-			nr = folio_nr_pages(folio);
-			*start = folio->index + nr;
+			*start = next;
 			goto out;
 		}
-		xas_advance(&xas, folio_next_index(folio) - 1);
+
+		if (next > end)
+			break;
+		xas_advance(&xas, next - 1);
 		continue;
 put_folio:
 		folio_put(folio);
@@ -2296,9 +2302,7 @@ unsigned filemap_get_folios_contig(struct address_space *mapping,
 		xas_reset(&xas);
 	}
 
-update_start:
 	nr = folio_batch_count(fbatch);
-
 	if (nr) {
 		folio = fbatch->folios[nr - 1];
 		*start = folio_next_index(folio);
-- 
2.43.0

[PATCH v1 3/5] mm/shmem: make SGP_NOALLOC succeed on hole like SGP_READ

[Chi Zhiling]

From: Chi Zhiling <chizhiling@kylinos.cn>

Change SGP_NOALLOC to return 0 with NULL folio on hole, matching
SGP_READ behavior. This simplifies the sgp_type handling by unifying
hole semantics across these types.

Previously, SGP_NOALLOC returned -ENOENT on hole, while SGP_READ
returned 0. This inconsistency required special handling in callers
like khugepaged and userfaultfd.

After this change:
- khugepaged: behavior unchanged (checks both error and NULL folio)
- userfaultfd: behavior unchanged (both -ENOENT and NULL are converted
  to -EFAULT before returning to userspace)

Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
---
 include/linux/shmem_fs.h | 2 +-
 mm/khugepaged.c          | 2 +-
 mm/shmem.c               | 9 +++------
 3 files changed, 5 insertions(+), 8 deletions(-)

diff --git a/include/linux/shmem_fs.h b/include/linux/shmem_fs.h
index 93a0ba872ebe..d461713c095b 100644
--- a/include/linux/shmem_fs.h
+++ b/include/linux/shmem_fs.h
@@ -165,7 +165,7 @@ extern unsigned long shmem_partial_swap_usage(struct address_space *mapping,
 /* Flag allocation requirements to shmem_get_folio */
 enum sgp_type {
 	SGP_READ,	/* don't exceed i_size, don't allocate page */
-	SGP_NOALLOC,	/* similar, but fail on hole or use fallocated page */
+	SGP_NOALLOC,	/* like SGP_READ, but accept fallocated page */
 	SGP_CACHE,	/* don't exceed i_size, may allocate page */
 	SGP_WRITE,	/* may exceed i_size, may allocate !Uptodate page */
 	SGP_FALLOC,	/* like SGP_WRITE, but make existing page Uptodate */
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index b8452dbdb043..3309d1c094df 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -1950,7 +1950,7 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
 				xas_unlock_irq(&xas);
 				/* swap in or instantiate fallocated page */
 				if (shmem_get_folio(mapping->host, index, 0,
-						&folio, SGP_NOALLOC)) {
+						&folio, SGP_NOALLOC) || !folio) {
 					result = SCAN_FAIL;
 					goto xa_unlocked;
 				}
diff --git a/mm/shmem.c b/mm/shmem.c
index 3b5dc21b323c..458853c506ea 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -2524,14 +2524,11 @@ static int shmem_get_folio_gfp(struct inode *inode, pgoff_t index,
 	}
 
 	/*
-	 * SGP_READ: succeed on hole, with NULL folio, letting caller zero.
-	 * SGP_NOALLOC: fail on hole, with NULL folio, letting caller fail.
+	 * SGP_READ/SGP_NOALLOC: succeed on hole, with NULL folio.
 	 */
 	*foliop = NULL;
-	if (sgp == SGP_READ)
+	if (sgp <= SGP_NOALLOC)
 		return 0;
-	if (sgp == SGP_NOALLOC)
-		return -ENOENT;
 
 	/*
 	 * Fast cache lookup and swap lookup did not find it: allocate.
@@ -2657,7 +2654,7 @@ static int shmem_get_folio_gfp(struct inode *inode, pgoff_t index,
  *
  * When no folio is found, the behavior depends on @sgp:
  *  - for SGP_READ, *@foliop is %NULL and 0 is returned
- *  - for SGP_NOALLOC, *@foliop is %NULL and -ENOENT is returned
+ *  - for SGP_NOALLOC, *@foliop is %NULL and 0 is returned
  *  - for all other flags a new folio is allocated, inserted into the
  *    page cache and returned locked in @foliop.
  *
-- 
2.43.0

Re: [PATCH v1 3/5] mm/shmem: make SGP_NOALLOC succeed on hole like SGP_READ

[Baolin Wang]

On 5/20/26 6:15 PM, Chi Zhiling wrote:
> From: Chi Zhiling <chizhiling@kylinos.cn>
> 
> Change SGP_NOALLOC to return 0 with NULL folio on hole, matching
> SGP_READ behavior. This simplifies the sgp_type handling by unifying
> hole semantics across these types.
> 
> Previously, SGP_NOALLOC returned -ENOENT on hole, while SGP_READ
> returned 0. This inconsistency required special handling in callers
> like khugepaged and userfaultfd.

This patch doesn't seem to be a performance optimization, and I'm not 
convinced.

> After this change:
> - khugepaged: behavior unchanged (checks both error and NULL folio)

But this adds an extra check to khugepaged, and I'm not sure it's worth it.

> - userfaultfd: behavior unchanged (both -ENOENT and NULL are converted
>    to -EFAULT before returning to userspace)

No, this will break userfaultfd, cause userfaultfd currently returns an 
error code directly in this case (please update to the latest 
codebase.). See:

static struct folio *shmem_get_folio_noalloc(struct inode *inode, 
pgoff_t pgoff)
{
	struct folio *folio;
	int err;

	err = shmem_get_folio(inode, pgoff, 0, &folio, SGP_NOALLOC);
	if (err)
		return ERR_PTR(err);

	return folio;
}

> Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
> ---
>   include/linux/shmem_fs.h | 2 +-
>   mm/khugepaged.c          | 2 +-
>   mm/shmem.c               | 9 +++------
>   3 files changed, 5 insertions(+), 8 deletions(-)
> 
> diff --git a/include/linux/shmem_fs.h b/include/linux/shmem_fs.h
> index 93a0ba872ebe..d461713c095b 100644
> --- a/include/linux/shmem_fs.h
> +++ b/include/linux/shmem_fs.h
> @@ -165,7 +165,7 @@ extern unsigned long shmem_partial_swap_usage(struct address_space *mapping,
>   /* Flag allocation requirements to shmem_get_folio */
>   enum sgp_type {
>   	SGP_READ,	/* don't exceed i_size, don't allocate page */
> -	SGP_NOALLOC,	/* similar, but fail on hole or use fallocated page */
> +	SGP_NOALLOC,	/* like SGP_READ, but accept fallocated page */
>   	SGP_CACHE,	/* don't exceed i_size, may allocate page */
>   	SGP_WRITE,	/* may exceed i_size, may allocate !Uptodate page */
>   	SGP_FALLOC,	/* like SGP_WRITE, but make existing page Uptodate */
> diff --git a/mm/khugepaged.c b/mm/khugepaged.c
> index b8452dbdb043..3309d1c094df 100644
> --- a/mm/khugepaged.c
> +++ b/mm/khugepaged.c
> @@ -1950,7 +1950,7 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
>   				xas_unlock_irq(&xas);
>   				/* swap in or instantiate fallocated page */
>   				if (shmem_get_folio(mapping->host, index, 0,
> -						&folio, SGP_NOALLOC)) {
> +						&folio, SGP_NOALLOC) || !folio) {
>   					result = SCAN_FAIL;
>   					goto xa_unlocked;
>   				}
> diff --git a/mm/shmem.c b/mm/shmem.c
> index 3b5dc21b323c..458853c506ea 100644
> --- a/mm/shmem.c
> +++ b/mm/shmem.c
> @@ -2524,14 +2524,11 @@ static int shmem_get_folio_gfp(struct inode *inode, pgoff_t index,
>   	}
>   
>   	/*
> -	 * SGP_READ: succeed on hole, with NULL folio, letting caller zero.
> -	 * SGP_NOALLOC: fail on hole, with NULL folio, letting caller fail.
> +	 * SGP_READ/SGP_NOALLOC: succeed on hole, with NULL folio.
>   	 */
>   	*foliop = NULL;
> -	if (sgp == SGP_READ)
> +	if (sgp <= SGP_NOALLOC)
>   		return 0;
> -	if (sgp == SGP_NOALLOC)
> -		return -ENOENT;
>   
>   	/*
>   	 * Fast cache lookup and swap lookup did not find it: allocate.
> @@ -2657,7 +2654,7 @@ static int shmem_get_folio_gfp(struct inode *inode, pgoff_t index,
>    *
>    * When no folio is found, the behavior depends on @sgp:
>    *  - for SGP_READ, *@foliop is %NULL and 0 is returned
> - *  - for SGP_NOALLOC, *@foliop is %NULL and -ENOENT is returned
> + *  - for SGP_NOALLOC, *@foliop is %NULL and 0 is returned
>    *  - for all other flags a new folio is allocated, inserted into the
>    *    page cache and returned locked in @foliop.
>    *

Re: [PATCH v1 3/5] mm/shmem: make SGP_NOALLOC succeed on hole like SGP_READ

[Chi Zhiling]

On 5/25/26 3:10 PM, Baolin Wang wrote:
> On 5/20/26 6:15 PM, Chi Zhiling wrote:
>> From: Chi Zhiling <chizhiling@kylinos.cn>
>>
>> Change SGP_NOALLOC to return 0 with NULL folio on hole, matching
>> SGP_READ behavior. This simplifies the sgp_type handling by unifying
>> hole semantics across these types.
>>
>> Previously, SGP_NOALLOC returned -ENOENT on hole, while SGP_READ
>> returned 0. This inconsistency required special handling in callers
>> like khugepaged and userfaultfd.
> 
> This patch doesn't seem to be a performance optimization, and I'm not 
> convinced.

Hi, baolin

This is not an optimization patch, and it is not related to this patch 
series either.

> 
>> After this change:
>> - khugepaged: behavior unchanged (checks both error and NULL folio)
> 
> But this adds an extra check to khugepaged, and I'm not sure it's worth it.

You are right, I will drop this patch in v2.

> 
>> - userfaultfd: behavior unchanged (both -ENOENT and NULL are converted
>>    to -EFAULT before returning to userspace)
> 
> No, this will break userfaultfd, cause userfaultfd currently returns an 
> error code directly in this case (please update to the latest 
> codebase.). See:
> 
> static struct folio *shmem_get_folio_noalloc(struct inode *inode, 
> pgoff_t pgoff)
> {
>      struct folio *folio;
>      int err;
> 
>      err = shmem_get_folio(inode, pgoff, 0, &folio, SGP_NOALLOC);
>      if (err)
>          return ERR_PTR(err);
> 
>      return folio;
> }

The return value of shmem_get_folio_noalloc() will change, but it seems 
this does not affect the logic in mfill_atomic_pte_continue().

```
	folio = ops->get_folio_noalloc(inode, pgoff);
	/* Our caller expects us to return -EFAULT if we failed to find folio */
	if (IS_ERR_OR_NULL(folio))
		return -EFAULT;
```

No matter whether get_folio_noalloc() returns an error or NULL, the 
error code is converted to -EFAULT here, and mfill_atomic_pte_continue() 
is currently the only caller of get_folio_noalloc().

Did I miss something?


Thanks,

> 
>> Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
>> ---
>>   include/linux/shmem_fs.h | 2 +-
>>   mm/khugepaged.c          | 2 +-
>>   mm/shmem.c               | 9 +++------
>>   3 files changed, 5 insertions(+), 8 deletions(-)
>>
>> diff --git a/include/linux/shmem_fs.h b/include/linux/shmem_fs.h
>> index 93a0ba872ebe..d461713c095b 100644
>> --- a/include/linux/shmem_fs.h
>> +++ b/include/linux/shmem_fs.h
>> @@ -165,7 +165,7 @@ extern unsigned long 
>> shmem_partial_swap_usage(struct address_space *mapping,
>>   /* Flag allocation requirements to shmem_get_folio */
>>   enum sgp_type {
>>       SGP_READ,    /* don't exceed i_size, don't allocate page */
>> -    SGP_NOALLOC,    /* similar, but fail on hole or use fallocated 
>> page */
>> +    SGP_NOALLOC,    /* like SGP_READ, but accept fallocated page */
>>       SGP_CACHE,    /* don't exceed i_size, may allocate page */
>>       SGP_WRITE,    /* may exceed i_size, may allocate !Uptodate page */
>>       SGP_FALLOC,    /* like SGP_WRITE, but make existing page 
>> Uptodate */
>> diff --git a/mm/khugepaged.c b/mm/khugepaged.c
>> index b8452dbdb043..3309d1c094df 100644
>> --- a/mm/khugepaged.c
>> +++ b/mm/khugepaged.c
>> @@ -1950,7 +1950,7 @@ static enum scan_result collapse_file(struct 
>> mm_struct *mm, unsigned long addr,
>>                   xas_unlock_irq(&xas);
>>                   /* swap in or instantiate fallocated page */
>>                   if (shmem_get_folio(mapping->host, index, 0,
>> -                        &folio, SGP_NOALLOC)) {
>> +                        &folio, SGP_NOALLOC) || !folio) {
>>                       result = SCAN_FAIL;
>>                       goto xa_unlocked;
>>                   }
>> diff --git a/mm/shmem.c b/mm/shmem.c
>> index 3b5dc21b323c..458853c506ea 100644
>> --- a/mm/shmem.c
>> +++ b/mm/shmem.c
>> @@ -2524,14 +2524,11 @@ static int shmem_get_folio_gfp(struct inode 
>> *inode, pgoff_t index,
>>       }
>>       /*
>> -     * SGP_READ: succeed on hole, with NULL folio, letting caller zero.
>> -     * SGP_NOALLOC: fail on hole, with NULL folio, letting caller fail.
>> +     * SGP_READ/SGP_NOALLOC: succeed on hole, with NULL folio.
>>        */
>>       *foliop = NULL;
>> -    if (sgp == SGP_READ)
>> +    if (sgp <= SGP_NOALLOC)
>>           return 0;
>> -    if (sgp == SGP_NOALLOC)
>> -        return -ENOENT;
>>       /*
>>        * Fast cache lookup and swap lookup did not find it: allocate.
>> @@ -2657,7 +2654,7 @@ static int shmem_get_folio_gfp(struct inode 
>> *inode, pgoff_t index,
>>    *
>>    * When no folio is found, the behavior depends on @sgp:
>>    *  - for SGP_READ, *@foliop is %NULL and 0 is returned
>> - *  - for SGP_NOALLOC, *@foliop is %NULL and -ENOENT is returned
>> + *  - for SGP_NOALLOC, *@foliop is %NULL and 0 is returned
>>    *  - for all other flags a new folio is allocated, inserted into the
>>    *    page cache and returned locked in @foliop.
>>    *

Re: [PATCH v1 3/5] mm/shmem: make SGP_NOALLOC succeed on hole like SGP_READ

[Baolin Wang]

On 5/25/26 4:14 PM, Chi Zhiling wrote:
> On 5/25/26 3:10 PM, Baolin Wang wrote:
>> On 5/20/26 6:15 PM, Chi Zhiling wrote:
>>> From: Chi Zhiling <chizhiling@kylinos.cn>
>>>
>>> Change SGP_NOALLOC to return 0 with NULL folio on hole, matching
>>> SGP_READ behavior. This simplifies the sgp_type handling by unifying
>>> hole semantics across these types.
>>>
>>> Previously, SGP_NOALLOC returned -ENOENT on hole, while SGP_READ
>>> returned 0. This inconsistency required special handling in callers
>>> like khugepaged and userfaultfd.
>>
>> This patch doesn't seem to be a performance optimization, and I'm not 
>> convinced.
> 
> Hi, baolin
> 
> This is not an optimization patch, and it is not related to this patch 
> series either.
> 
>>
>>> After this change:
>>> - khugepaged: behavior unchanged (checks both error and NULL folio)
>>
>> But this adds an extra check to khugepaged, and I'm not sure it's 
>> worth it.
> 
> You are right, I will drop this patch in v2.
> 
>>
>>> - userfaultfd: behavior unchanged (both -ENOENT and NULL are converted
>>>    to -EFAULT before returning to userspace)
>>
>> No, this will break userfaultfd, cause userfaultfd currently returns 
>> an error code directly in this case (please update to the latest 
>> codebase.). See:
>>
>> static struct folio *shmem_get_folio_noalloc(struct inode *inode, 
>> pgoff_t pgoff)
>> {
>>      struct folio *folio;
>>      int err;
>>
>>      err = shmem_get_folio(inode, pgoff, 0, &folio, SGP_NOALLOC);
>>      if (err)
>>          return ERR_PTR(err);
>>
>>      return folio;
>> }
> 
> The return value of shmem_get_folio_noalloc() will change, but it seems 
> this does not affect the logic in mfill_atomic_pte_continue().
> 
> ```
>      folio = ops->get_folio_noalloc(inode, pgoff);
>      /* Our caller expects us to return -EFAULT if we failed to find 
> folio */
>      if (IS_ERR_OR_NULL(folio))
>          return -EFAULT;
> ```
> 
> No matter whether get_folio_noalloc() returns an error or NULL, the 
> error code is converted to -EFAULT here, and mfill_atomic_pte_continue() 
> is currently the only caller of get_folio_noalloc().
> 
> Did I miss something?

OK.

But I still wonder if it's worth changing the definition of a flag, 
especially when there's no obvious bug here and this is merely a 
cleanup. Yet this cleanup requires adding an extra check for khugepaged, 
which makes me feel it is not worth it. Unless Hugh prefers to change 
the meaning of this flag:).

[PATCH v1 4/5] mm/shmem: introduce copy_zero_to_iter() for large zeroing

[Chi Zhiling]

From: Chi Zhiling <chizhiling@kylinos.cn>

This is a prep patch for shmem folio batching in the read path, where
non-uptodate folios need to be handled in the main iteration loop. A
large non-uptodate folio should be treated as a hole.

Currently, holes larger than PAGE_SIZE cannot be handled because
ZERO_PAGE is limited to a single page. Add copy_zero_to_iter() as a
wrapper to support copying larger zero ranges to the iterator.

Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
---
 mm/shmem.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/mm/shmem.c b/mm/shmem.c
index 458853c506ea..96ea5a4c0aff 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -3336,6 +3336,20 @@ shmem_write_end(const struct kiocb *iocb, struct address_space *mapping,
 	return copied;
 }
 
+static size_t copy_zero_to_iter(size_t bytes, struct iov_iter *i)
+{
+	struct folio *zero = largest_zero_folio();
+	unsigned long nr, ret, written = 0;
+
+	do {
+		nr = min(bytes - written, folio_size(zero));
+		ret = copy_folio_to_iter(zero, 0, nr, i);
+		written += ret;
+	} while (written < bytes && ret == nr);
+
+	return written;
+}
+
 static ssize_t shmem_file_read_iter(struct kiocb *iocb, struct iov_iter *to)
 {
 	struct file *file = iocb->ki_filp;
@@ -3430,7 +3444,7 @@ static ssize_t shmem_file_read_iter(struct kiocb *iocb, struct iov_iter *to)
 			 * clear_user() not so much, that it is noticeably
 			 * faster to copy the zero page instead of clearing.
 			 */
-			ret = copy_page_to_iter(ZERO_PAGE(0), offset, nr, to);
+			ret = copy_zero_to_iter(nr, to);
 		} else {
 			/*
 			 * But submitting the same page twice in a row to
-- 
2.43.0

[PATCH v1 5/5] mm/shmem: optimize file read with folio batching

[Chi Zhiling]

From: Chi Zhiling <chizhiling@kylinos.cn>

Optimize shmem file read by using filemap_get_folios_contig() to
batch fetch contiguous folios from the page cache, reducing the
overhead of repeated shmem_get_folio() calls.

When the folio batch is exhausted, attempt to refill it with
filemap_get_folios_contig(). If no folios are found (hole or swapped
out pages), fall back to shmem_get_folio() to handle these cases
individually.

Additionally:
- Defer folio_put() until the batch is exhausted or on exit
- Add folio_test_uptodate() check before copying to ensure data
  validity

Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
---
 mm/shmem.c | 50 ++++++++++++++++++++++++++++++++++++--------------
 1 file changed, 36 insertions(+), 14 deletions(-)

diff --git a/mm/shmem.c b/mm/shmem.c
index 96ea5a4c0aff..e0eacc23cccd 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -3355,11 +3355,14 @@ static ssize_t shmem_file_read_iter(struct kiocb *iocb, struct iov_iter *to)
 	struct file *file = iocb->ki_filp;
 	struct inode *inode = file_inode(file);
 	struct address_space *mapping = inode->i_mapping;
+	struct folio_batch fbatch;
 	pgoff_t index;
 	unsigned long offset;
 	int error = 0;
 	ssize_t retval = 0;
 
+	folio_batch_init(&fbatch);
+
 	for (;;) {
 		struct folio *folio = NULL;
 		struct page *page = NULL;
@@ -3372,18 +3375,38 @@ static ssize_t shmem_file_read_iter(struct kiocb *iocb, struct iov_iter *to)
 			break;
 
 		index = iocb->ki_pos >> PAGE_SHIFT;
-		error = shmem_get_folio(inode, index, 0, &folio, SGP_READ);
-		if (error) {
-			if (error == -EINVAL)
-				error = 0;
-			break;
+fetch:
+		folio = folio_batch_next(&fbatch);
+		if (!folio) {
+			pgoff_t start = index;
+			pgoff_t end = (iocb->ki_pos + to->count - 1) >> PAGE_SHIFT;
+
+			if (folio_batch_count(&fbatch)) {
+				for (int i = 0; i < folio_batch_count(&fbatch); i++)
+					folio_put(fbatch.folios[i]);
+				folio_batch_reinit(&fbatch);
+			}
+
+			filemap_get_folios_contig(inode->i_mapping, &start, end, &fbatch);
+			if (folio_batch_count(&fbatch))
+				goto fetch;
+
+			error = shmem_get_folio(inode, index, 0, &folio, SGP_READ);
+			if (unlikely(error)) {
+				if (error == -EINVAL)
+					error = 0;
+				break;
+			}
+			if (folio) {
+				folio_unlock(folio);
+				folio_batch_add(&fbatch, folio);
+				fbatch.i++;
+			}
 		}
-		if (folio) {
-			folio_unlock(folio);
 
+		if (folio) {
 			page = folio_file_page(folio, index);
 			if (PageHWPoison(page)) {
-				folio_put(folio);
 				error = -EIO;
 				break;
 			}
@@ -3398,11 +3421,9 @@ static ssize_t shmem_file_read_iter(struct kiocb *iocb, struct iov_iter *to)
 		 * are called without i_rwsem protection against truncate
 		 */
 		i_size = i_size_read(inode);
-		if (unlikely(iocb->ki_pos >= i_size)) {
-			if (folio)
-				folio_put(folio);
+		if (unlikely(iocb->ki_pos >= i_size))
 			break;
-		}
+
 		end_offset = min_t(loff_t, i_size, iocb->ki_pos + to->count);
 		if (folio && likely(!fallback_page_copy))
 			fsize = folio_size(folio);
@@ -3411,7 +3432,7 @@ static ssize_t shmem_file_read_iter(struct kiocb *iocb, struct iov_iter *to)
 		offset = iocb->ki_pos & (fsize - 1);
 		nr = min_t(loff_t, end_offset - iocb->ki_pos, fsize - offset);
 
-		if (folio) {
+		if (folio && folio_test_uptodate(folio)) {
 			/*
 			 * If users can be writing to this page using arbitrary
 			 * virtual addresses, take care about potential aliasing
@@ -3437,7 +3458,6 @@ static ssize_t shmem_file_read_iter(struct kiocb *iocb, struct iov_iter *to)
 				ret = copy_folio_to_iter(folio, offset, nr, to);
 			else
 				ret = copy_page_to_iter(page, offset, nr, to);
-			folio_put(folio);
 		} else if (user_backed_iter(to)) {
 			/*
 			 * Copy to user tends to be so well optimized, but
@@ -3466,6 +3486,8 @@ static ssize_t shmem_file_read_iter(struct kiocb *iocb, struct iov_iter *to)
 		cond_resched();
 	}
 
+	for (int i = 0; i < folio_batch_count(&fbatch); i++)
+		folio_put(fbatch.folios[i]);
 	file_accessed(file);
 	return retval ? retval : error;
 }
-- 
2.43.0

Re: [PATCH v1 5/5] mm/shmem: optimize file read with folio batching

[Baolin Wang]

On 5/20/26 6:15 PM, Chi Zhiling wrote:
> From: Chi Zhiling <chizhiling@kylinos.cn>
> 
> Optimize shmem file read by using filemap_get_folios_contig() to
> batch fetch contiguous folios from the page cache, reducing the
> overhead of repeated shmem_get_folio() calls.
> 
> When the folio batch is exhausted, attempt to refill it with
> filemap_get_folios_contig(). If no folios are found (hole or swapped
> out pages), fall back to shmem_get_folio() to handle these cases
> individually.
> 
> Additionally:
> - Defer folio_put() until the batch is exhausted or on exit
> - Add folio_test_uptodate() check before copying to ensure data
>    validity
> 
> Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
> ---
>   mm/shmem.c | 50 ++++++++++++++++++++++++++++++++++++--------------
>   1 file changed, 36 insertions(+), 14 deletions(-)
> 
> diff --git a/mm/shmem.c b/mm/shmem.c
> index 96ea5a4c0aff..e0eacc23cccd 100644
> --- a/mm/shmem.c
> +++ b/mm/shmem.c
> @@ -3355,11 +3355,14 @@ static ssize_t shmem_file_read_iter(struct kiocb *iocb, struct iov_iter *to)
>   	struct file *file = iocb->ki_filp;
>   	struct inode *inode = file_inode(file);
>   	struct address_space *mapping = inode->i_mapping;
> +	struct folio_batch fbatch;
>   	pgoff_t index;
>   	unsigned long offset;
>   	int error = 0;
>   	ssize_t retval = 0;
>   
> +	folio_batch_init(&fbatch);
> +
>   	for (;;) {
>   		struct folio *folio = NULL;
>   		struct page *page = NULL;
> @@ -3372,18 +3375,38 @@ static ssize_t shmem_file_read_iter(struct kiocb *iocb, struct iov_iter *to)
>   			break;
>   
>   		index = iocb->ki_pos >> PAGE_SHIFT;
> -		error = shmem_get_folio(inode, index, 0, &folio, SGP_READ);
> -		if (error) {
> -			if (error == -EINVAL)
> -				error = 0;
> -			break;
> +fetch:
> +		folio = folio_batch_next(&fbatch);
> +		if (!folio) {
> +			pgoff_t start = index;
> +			pgoff_t end = (iocb->ki_pos + to->count - 1) >> PAGE_SHIFT;
> +
> +			if (folio_batch_count(&fbatch)) {
> +				for (int i = 0; i < folio_batch_count(&fbatch); i++)
> +					folio_put(fbatch.folios[i]);
> +				folio_batch_reinit(&fbatch);
> +			}
> +
> +			filemap_get_folios_contig(inode->i_mapping, &start, end, &fbatch);
> +			if (folio_batch_count(&fbatch))
> +				goto fetch;
> +
> +			error = shmem_get_folio(inode, index, 0, &folio, SGP_READ);
> +			if (unlikely(error)) {
> +				if (error == -EINVAL)
> +					error = 0;
> +				break;
> +			}
> +			if (folio) {
> +				folio_unlock(folio);
> +				folio_batch_add(&fbatch, folio);
> +				fbatch.i++;
> +			}
>   		}
> -		if (folio) {
> -			folio_unlock(folio);
>   
> +		if (folio) {
>   			page = folio_file_page(folio, index);
>   			if (PageHWPoison(page)) {
> -				folio_put(folio);
>   				error = -EIO;
>   				break;
>   			}

I haven't tested it yet, but I'm sure this will break the 
fallback_page_copy mode.

You're fetching from the batch at folio granularity here, but in 
fallback_page_copy mode, we still copy the data at page granularity.

Re: [PATCH v1 5/5] mm/shmem: optimize file read with folio batching

[Chi Zhiling]

On 5/25/26 4:37 PM, Baolin Wang wrote:
> 
> 
> On 5/20/26 6:15 PM, Chi Zhiling wrote:
>> From: Chi Zhiling <chizhiling@kylinos.cn>
>>
>> Optimize shmem file read by using filemap_get_folios_contig() to
>> batch fetch contiguous folios from the page cache, reducing the
>> overhead of repeated shmem_get_folio() calls.
>>
>> When the folio batch is exhausted, attempt to refill it with
>> filemap_get_folios_contig(). If no folios are found (hole or swapped
>> out pages), fall back to shmem_get_folio() to handle these cases
>> individually.
>>
>> Additionally:
>> - Defer folio_put() until the batch is exhausted or on exit
>> - Add folio_test_uptodate() check before copying to ensure data
>>    validity
>>
>> Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
>> ---
>>   mm/shmem.c | 50 ++++++++++++++++++++++++++++++++++++--------------
>>   1 file changed, 36 insertions(+), 14 deletions(-)
>>
>> diff --git a/mm/shmem.c b/mm/shmem.c
>> index 96ea5a4c0aff..e0eacc23cccd 100644
>> --- a/mm/shmem.c
>> +++ b/mm/shmem.c
>> @@ -3355,11 +3355,14 @@ static ssize_t shmem_file_read_iter(struct 
>> kiocb *iocb, struct iov_iter *to)
>>       struct file *file = iocb->ki_filp;
>>       struct inode *inode = file_inode(file);
>>       struct address_space *mapping = inode->i_mapping;
>> +    struct folio_batch fbatch;
>>       pgoff_t index;
>>       unsigned long offset;
>>       int error = 0;
>>       ssize_t retval = 0;
>> +    folio_batch_init(&fbatch);
>> +
>>       for (;;) {
>>           struct folio *folio = NULL;
>>           struct page *page = NULL;
>> @@ -3372,18 +3375,38 @@ static ssize_t shmem_file_read_iter(struct 
>> kiocb *iocb, struct iov_iter *to)
>>               break;
>>           index = iocb->ki_pos >> PAGE_SHIFT;
>> -        error = shmem_get_folio(inode, index, 0, &folio, SGP_READ);
>> -        if (error) {
>> -            if (error == -EINVAL)
>> -                error = 0;
>> -            break;
>> +fetch:
>> +        folio = folio_batch_next(&fbatch);
>> +        if (!folio) {
>> +            pgoff_t start = index;
>> +            pgoff_t end = (iocb->ki_pos + to->count - 1) >> PAGE_SHIFT;
>> +
>> +            if (folio_batch_count(&fbatch)) {
>> +                for (int i = 0; i < folio_batch_count(&fbatch); i++)
>> +                    folio_put(fbatch.folios[i]);
>> +                folio_batch_reinit(&fbatch);
>> +            }
>> +
>> +            filemap_get_folios_contig(inode->i_mapping, &start, end, 
>> &fbatch);
>> +            if (folio_batch_count(&fbatch))
>> +                goto fetch;
>> +
>> +            error = shmem_get_folio(inode, index, 0, &folio, SGP_READ);
>> +            if (unlikely(error)) {
>> +                if (error == -EINVAL)
>> +                    error = 0;
>> +                break;
>> +            }
>> +            if (folio) {
>> +                folio_unlock(folio);
>> +                folio_batch_add(&fbatch, folio);
>> +                fbatch.i++;
>> +            }
>>           }
>> -        if (folio) {
>> -            folio_unlock(folio);
>> +        if (folio) {
>>               page = folio_file_page(folio, index);
>>               if (PageHWPoison(page)) {
>> -                folio_put(folio);
>>                   error = -EIO;
>>                   break;
>>               }
> 
> I haven't tested it yet, but I'm sure this will break the 
> fallback_page_copy mode.
> 
> You're fetching from the batch at folio granularity here, but in 
> fallback_page_copy mode, we still copy the data at page granularity.

Yes, that's a bug, Sashika has already reported it. I’ll fix it in the 
next version.

Sashika also reported another potential issue below, but I think that is 
a misunderstanding, because a folio fetched from the swap cache is 
already marked Uptodate before being added to the page cache. What do 
you think?

~~~
Sashika:

Can this lockless check cause readers to receive spurious zeros instead
of the actual data?

Since filemap_get_folios_contig() fetches locklessly, a folio that is
currently being allocated and initialized, or swapped in by another
thread, might not yet be marked uptodate.

By checking folio_test_uptodate() without holding the folio lock, this falls
through to the zero-fill paths if the folio isn't uptodate yet. The original
code avoided this by using shmem_get_folio() which waits for the folio lock
and I/O completion.
~~~

link: 
https://sashiko.dev/#/patchset/20260520101538.58745-1-chizhiling@163.com

Thanks,

Re: [PATCH v1 5/5] mm/shmem: optimize file read with folio batching

[Baolin Wang]

On 5/25/26 5:47 PM, Chi Zhiling wrote:
> On 5/25/26 4:37 PM, Baolin Wang wrote:
>>
>>
>> On 5/20/26 6:15 PM, Chi Zhiling wrote:
>>> From: Chi Zhiling <chizhiling@kylinos.cn>
>>>
>>> Optimize shmem file read by using filemap_get_folios_contig() to
>>> batch fetch contiguous folios from the page cache, reducing the
>>> overhead of repeated shmem_get_folio() calls.
>>>
>>> When the folio batch is exhausted, attempt to refill it with
>>> filemap_get_folios_contig(). If no folios are found (hole or swapped
>>> out pages), fall back to shmem_get_folio() to handle these cases
>>> individually.
>>>
>>> Additionally:
>>> - Defer folio_put() until the batch is exhausted or on exit
>>> - Add folio_test_uptodate() check before copying to ensure data
>>>    validity
>>>
>>> Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
>>> ---
>>>   mm/shmem.c | 50 ++++++++++++++++++++++++++++++++++++--------------
>>>   1 file changed, 36 insertions(+), 14 deletions(-)
>>>
>>> diff --git a/mm/shmem.c b/mm/shmem.c
>>> index 96ea5a4c0aff..e0eacc23cccd 100644
>>> --- a/mm/shmem.c
>>> +++ b/mm/shmem.c
>>> @@ -3355,11 +3355,14 @@ static ssize_t shmem_file_read_iter(struct 
>>> kiocb *iocb, struct iov_iter *to)
>>>       struct file *file = iocb->ki_filp;
>>>       struct inode *inode = file_inode(file);
>>>       struct address_space *mapping = inode->i_mapping;
>>> +    struct folio_batch fbatch;
>>>       pgoff_t index;
>>>       unsigned long offset;
>>>       int error = 0;
>>>       ssize_t retval = 0;
>>> +    folio_batch_init(&fbatch);
>>> +
>>>       for (;;) {
>>>           struct folio *folio = NULL;
>>>           struct page *page = NULL;
>>> @@ -3372,18 +3375,38 @@ static ssize_t shmem_file_read_iter(struct 
>>> kiocb *iocb, struct iov_iter *to)
>>>               break;
>>>           index = iocb->ki_pos >> PAGE_SHIFT;
>>> -        error = shmem_get_folio(inode, index, 0, &folio, SGP_READ);
>>> -        if (error) {
>>> -            if (error == -EINVAL)
>>> -                error = 0;
>>> -            break;
>>> +fetch:
>>> +        folio = folio_batch_next(&fbatch);
>>> +        if (!folio) {
>>> +            pgoff_t start = index;
>>> +            pgoff_t end = (iocb->ki_pos + to->count - 1) >> PAGE_SHIFT;
>>> +
>>> +            if (folio_batch_count(&fbatch)) {
>>> +                for (int i = 0; i < folio_batch_count(&fbatch); i++)
>>> +                    folio_put(fbatch.folios[i]);
>>> +                folio_batch_reinit(&fbatch);
>>> +            }
>>> +
>>> +            filemap_get_folios_contig(inode->i_mapping, &start, end, 
>>> &fbatch);
>>> +            if (folio_batch_count(&fbatch))
>>> +                goto fetch;
>>> +
>>> +            error = shmem_get_folio(inode, index, 0, &folio, SGP_READ);
>>> +            if (unlikely(error)) {
>>> +                if (error == -EINVAL)
>>> +                    error = 0;
>>> +                break;
>>> +            }
>>> +            if (folio) {
>>> +                folio_unlock(folio);
>>> +                folio_batch_add(&fbatch, folio);
>>> +                fbatch.i++;
>>> +            }
>>>           }
>>> -        if (folio) {
>>> -            folio_unlock(folio);
>>> +        if (folio) {
>>>               page = folio_file_page(folio, index);
>>>               if (PageHWPoison(page)) {
>>> -                folio_put(folio);
>>>                   error = -EIO;
>>>                   break;
>>>               }
>>
>> I haven't tested it yet, but I'm sure this will break the 
>> fallback_page_copy mode.
>>
>> You're fetching from the batch at folio granularity here, but in 
>> fallback_page_copy mode, we still copy the data at page granularity.
> 
> Yes, that's a bug, Sashika has already reported it. I’ll fix it in the 
> next version.

Good.

> Sashika also reported another potential issue below, but I think that is 
> a misunderstanding, because a folio fetched from the swap cache is 
> already marked Uptodate before being added to the page cache. What do 
> you think?

I'm not concerned about swap-in folios, since we already ensure the 
folio is uptodate when swapping out.

What I'm worried about is that this change moves the 
folio_test_uptodate() check outside of the folio lock, which could 
introduce some race? Of course, with the original logic, the folio state 
could also transition from !uptodate to uptodate after the folio
is unlocked (I need more thought).

Another race condition that needs to be addressed is, when getting the 
folio locklessly via filemap_get_folios_contig(), there could be a 
concurrent race with swap-out (because you removed the folio->mapping 
check under folio lock). This could result in reading zero data when it 
shouldn't (the folio actually needs to be swapped in).

Re: [PATCH v1 5/5] mm/shmem: optimize file read with folio batching

[Chi Zhiling]

On 5/26/26 11:11 AM, Baolin Wang wrote:
> 
> 
> On 5/25/26 5:47 PM, Chi Zhiling wrote:
>> On 5/25/26 4:37 PM, Baolin Wang wrote:
>>>
>>>
>>> On 5/20/26 6:15 PM, Chi Zhiling wrote:
>>>> From: Chi Zhiling <chizhiling@kylinos.cn>
>>>>
>>>> Optimize shmem file read by using filemap_get_folios_contig() to
>>>> batch fetch contiguous folios from the page cache, reducing the
>>>> overhead of repeated shmem_get_folio() calls.
>>>>
>>>> When the folio batch is exhausted, attempt to refill it with
>>>> filemap_get_folios_contig(). If no folios are found (hole or swapped
>>>> out pages), fall back to shmem_get_folio() to handle these cases
>>>> individually.
>>>>
>>>> Additionally:
>>>> - Defer folio_put() until the batch is exhausted or on exit
>>>> - Add folio_test_uptodate() check before copying to ensure data
>>>>    validity
>>>>
>>>> Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
>>>> ---
>>>>   mm/shmem.c | 50 ++++++++++++++++++++++++++++++++++++--------------
>>>>   1 file changed, 36 insertions(+), 14 deletions(-)
>>>>
>>>> diff --git a/mm/shmem.c b/mm/shmem.c
>>>> index 96ea5a4c0aff..e0eacc23cccd 100644
>>>> --- a/mm/shmem.c
>>>> +++ b/mm/shmem.c
>>>> @@ -3355,11 +3355,14 @@ static ssize_t shmem_file_read_iter(struct 
>>>> kiocb *iocb, struct iov_iter *to)
>>>>       struct file *file = iocb->ki_filp;
>>>>       struct inode *inode = file_inode(file);
>>>>       struct address_space *mapping = inode->i_mapping;
>>>> +    struct folio_batch fbatch;
>>>>       pgoff_t index;
>>>>       unsigned long offset;
>>>>       int error = 0;
>>>>       ssize_t retval = 0;
>>>> +    folio_batch_init(&fbatch);
>>>> +
>>>>       for (;;) {
>>>>           struct folio *folio = NULL;
>>>>           struct page *page = NULL;
>>>> @@ -3372,18 +3375,38 @@ static ssize_t shmem_file_read_iter(struct 
>>>> kiocb *iocb, struct iov_iter *to)
>>>>               break;
>>>>           index = iocb->ki_pos >> PAGE_SHIFT;
>>>> -        error = shmem_get_folio(inode, index, 0, &folio, SGP_READ);
>>>> -        if (error) {
>>>> -            if (error == -EINVAL)
>>>> -                error = 0;
>>>> -            break;
>>>> +fetch:
>>>> +        folio = folio_batch_next(&fbatch);
>>>> +        if (!folio) {
>>>> +            pgoff_t start = index;
>>>> +            pgoff_t end = (iocb->ki_pos + to->count - 1) >> 
>>>> PAGE_SHIFT;
>>>> +
>>>> +            if (folio_batch_count(&fbatch)) {
>>>> +                for (int i = 0; i < folio_batch_count(&fbatch); i++)
>>>> +                    folio_put(fbatch.folios[i]);
>>>> +                folio_batch_reinit(&fbatch);
>>>> +            }
>>>> +
>>>> +            filemap_get_folios_contig(inode->i_mapping, &start, 
>>>> end, &fbatch);
>>>> +            if (folio_batch_count(&fbatch))
>>>> +                goto fetch;
>>>> +
>>>> +            error = shmem_get_folio(inode, index, 0, &folio, 
>>>> SGP_READ);
>>>> +            if (unlikely(error)) {
>>>> +                if (error == -EINVAL)
>>>> +                    error = 0;
>>>> +                break;
>>>> +            }
>>>> +            if (folio) {
>>>> +                folio_unlock(folio);
>>>> +                folio_batch_add(&fbatch, folio);
>>>> +                fbatch.i++;
>>>> +            }
>>>>           }
>>>> -        if (folio) {
>>>> -            folio_unlock(folio);
>>>> +        if (folio) {
>>>>               page = folio_file_page(folio, index);
>>>>               if (PageHWPoison(page)) {
>>>> -                folio_put(folio);
>>>>                   error = -EIO;
>>>>                   break;
>>>>               }
>>>
>>> I haven't tested it yet, but I'm sure this will break the 
>>> fallback_page_copy mode.
>>>
>>> You're fetching from the batch at folio granularity here, but in 
>>> fallback_page_copy mode, we still copy the data at page granularity.
>>
>> Yes, that's a bug, Sashika has already reported it. I’ll fix it in the 
>> next version.
> 
> Good.
> 
>> Sashika also reported another potential issue below, but I think that 
>> is a misunderstanding, because a folio fetched from the swap cache is 
>> already marked Uptodate before being added to the page cache. What do 
>> you think?
> 
> I'm not concerned about swap-in folios, since we already ensure the 
> folio is uptodate when swapping out.
> 
> What I'm worried about is that this change moves the 
> folio_test_uptodate() check outside of the folio lock, which could 
> introduce some race? Of course, with the original logic, the folio state 
> could also transition from !uptodate to uptodate after the folio
> is unlocked (I need more thought).
> 
> Another race condition that needs to be addressed is, when getting the 
> folio locklessly via filemap_get_folios_contig(), there could be a 
> concurrent race with swap-out (because you removed the folio->mapping 
> check under folio lock). This could result in reading zero data when it 
> shouldn't (the folio actually needs to be swapped in).

Sorry, I don't fully understand what you mean. It seems that swap-out 
does not clear the uptodate flag, and the folio cannot be freed during 
the read operation because its reference count is held. So a swapped-out 
folio should not be mistaken for a zero-filled folio. Right?

Re: [PATCH v1 5/5] mm/shmem: optimize file read with folio batching

[Baolin Wang]

On 5/26/26 3:12 PM, Chi Zhiling wrote:
> On 5/26/26 11:11 AM, Baolin Wang wrote:
>>
>>
>> On 5/25/26 5:47 PM, Chi Zhiling wrote:
>>> On 5/25/26 4:37 PM, Baolin Wang wrote:
>>>>
>>>>
>>>> On 5/20/26 6:15 PM, Chi Zhiling wrote:
>>>>> From: Chi Zhiling <chizhiling@kylinos.cn>
>>>>>
>>>>> Optimize shmem file read by using filemap_get_folios_contig() to
>>>>> batch fetch contiguous folios from the page cache, reducing the
>>>>> overhead of repeated shmem_get_folio() calls.
>>>>>
>>>>> When the folio batch is exhausted, attempt to refill it with
>>>>> filemap_get_folios_contig(). If no folios are found (hole or swapped
>>>>> out pages), fall back to shmem_get_folio() to handle these cases
>>>>> individually.
>>>>>
>>>>> Additionally:
>>>>> - Defer folio_put() until the batch is exhausted or on exit
>>>>> - Add folio_test_uptodate() check before copying to ensure data
>>>>>    validity
>>>>>
>>>>> Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
>>>>> ---
>>>>>   mm/shmem.c | 50 ++++++++++++++++++++++++++++++++++++--------------
>>>>>   1 file changed, 36 insertions(+), 14 deletions(-)
>>>>>
>>>>> diff --git a/mm/shmem.c b/mm/shmem.c
>>>>> index 96ea5a4c0aff..e0eacc23cccd 100644
>>>>> --- a/mm/shmem.c
>>>>> +++ b/mm/shmem.c
>>>>> @@ -3355,11 +3355,14 @@ static ssize_t shmem_file_read_iter(struct 
>>>>> kiocb *iocb, struct iov_iter *to)
>>>>>       struct file *file = iocb->ki_filp;
>>>>>       struct inode *inode = file_inode(file);
>>>>>       struct address_space *mapping = inode->i_mapping;
>>>>> +    struct folio_batch fbatch;
>>>>>       pgoff_t index;
>>>>>       unsigned long offset;
>>>>>       int error = 0;
>>>>>       ssize_t retval = 0;
>>>>> +    folio_batch_init(&fbatch);
>>>>> +
>>>>>       for (;;) {
>>>>>           struct folio *folio = NULL;
>>>>>           struct page *page = NULL;
>>>>> @@ -3372,18 +3375,38 @@ static ssize_t shmem_file_read_iter(struct 
>>>>> kiocb *iocb, struct iov_iter *to)
>>>>>               break;
>>>>>           index = iocb->ki_pos >> PAGE_SHIFT;
>>>>> -        error = shmem_get_folio(inode, index, 0, &folio, SGP_READ);
>>>>> -        if (error) {
>>>>> -            if (error == -EINVAL)
>>>>> -                error = 0;
>>>>> -            break;
>>>>> +fetch:
>>>>> +        folio = folio_batch_next(&fbatch);
>>>>> +        if (!folio) {
>>>>> +            pgoff_t start = index;
>>>>> +            pgoff_t end = (iocb->ki_pos + to->count - 1) >> 
>>>>> PAGE_SHIFT;
>>>>> +
>>>>> +            if (folio_batch_count(&fbatch)) {
>>>>> +                for (int i = 0; i < folio_batch_count(&fbatch); i++)
>>>>> +                    folio_put(fbatch.folios[i]);
>>>>> +                folio_batch_reinit(&fbatch);
>>>>> +            }
>>>>> +
>>>>> +            filemap_get_folios_contig(inode->i_mapping, &start, 
>>>>> end, &fbatch);
>>>>> +            if (folio_batch_count(&fbatch))
>>>>> +                goto fetch;
>>>>> +
>>>>> +            error = shmem_get_folio(inode, index, 0, &folio, 
>>>>> SGP_READ);
>>>>> +            if (unlikely(error)) {
>>>>> +                if (error == -EINVAL)
>>>>> +                    error = 0;
>>>>> +                break;
>>>>> +            }
>>>>> +            if (folio) {
>>>>> +                folio_unlock(folio);
>>>>> +                folio_batch_add(&fbatch, folio);
>>>>> +                fbatch.i++;
>>>>> +            }
>>>>>           }
>>>>> -        if (folio) {
>>>>> -            folio_unlock(folio);
>>>>> +        if (folio) {
>>>>>               page = folio_file_page(folio, index);
>>>>>               if (PageHWPoison(page)) {
>>>>> -                folio_put(folio);
>>>>>                   error = -EIO;
>>>>>                   break;
>>>>>               }
>>>>
>>>> I haven't tested it yet, but I'm sure this will break the 
>>>> fallback_page_copy mode.
>>>>
>>>> You're fetching from the batch at folio granularity here, but in 
>>>> fallback_page_copy mode, we still copy the data at page granularity.
>>>
>>> Yes, that's a bug, Sashika has already reported it. I’ll fix it in 
>>> the next version.
>>
>> Good.
>>
>>> Sashika also reported another potential issue below, but I think that 
>>> is a misunderstanding, because a folio fetched from the swap cache is 
>>> already marked Uptodate before being added to the page cache. What do 
>>> you think?
>>
>> I'm not concerned about swap-in folios, since we already ensure the 
>> folio is uptodate when swapping out.
>>
>> What I'm worried about is that this change moves the 
>> folio_test_uptodate() check outside of the folio lock, which could 
>> introduce some race? Of course, with the original logic, the folio 
>> state could also transition from !uptodate to uptodate after the folio
>> is unlocked (I need more thought).
>>
>> Another race condition that needs to be addressed is, when getting the 
>> folio locklessly via filemap_get_folios_contig(), there could be a 
>> concurrent race with swap-out (because you removed the folio->mapping 
>> check under folio lock). This could result in reading zero data when 
>> it shouldn't (the folio actually needs to be swapped in).
> 
> Sorry, I don't fully understand what you mean. It seems that swap-out 
> does not clear the uptodate flag, and the folio cannot be freed during 
> the read operation because its reference count is held. So a swapped-out 
> folio should not be mistaken for a zero-filled folio. Right?

Sorry for the late reply. Ah, I missed the held refcnt, and now sounds 
reasonable.

Re: [PATCH v1 5/5] mm/shmem: optimize file read with folio batching

[Chi Zhiling]

On 6/1/26 7:20 PM, Baolin Wang wrote:
> 
> 
> On 5/26/26 3:12 PM, Chi Zhiling wrote:
>> On 5/26/26 11:11 AM, Baolin Wang wrote:
>>>
>>>
>>> On 5/25/26 5:47 PM, Chi Zhiling wrote:
>>>> On 5/25/26 4:37 PM, Baolin Wang wrote:
>>>>>
>>>>>
>>>>> On 5/20/26 6:15 PM, Chi Zhiling wrote:
>>>>>> From: Chi Zhiling <chizhiling@kylinos.cn>
>>>>>>
>>>>>> Optimize shmem file read by using filemap_get_folios_contig() to
>>>>>> batch fetch contiguous folios from the page cache, reducing the
>>>>>> overhead of repeated shmem_get_folio() calls.
>>>>>>
>>>>>> When the folio batch is exhausted, attempt to refill it with
>>>>>> filemap_get_folios_contig(). If no folios are found (hole or swapped
>>>>>> out pages), fall back to shmem_get_folio() to handle these cases
>>>>>> individually.
>>>>>>
>>>>>> Additionally:
>>>>>> - Defer folio_put() until the batch is exhausted or on exit
>>>>>> - Add folio_test_uptodate() check before copying to ensure data
>>>>>>    validity
>>>>>>
>>>>>> Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
>>>>>> ---
>>>>>>   mm/shmem.c | 50 ++++++++++++++++++++++++++++++++++++--------------
>>>>>>   1 file changed, 36 insertions(+), 14 deletions(-)
>>>>>>
>>>>>> diff --git a/mm/shmem.c b/mm/shmem.c
>>>>>> index 96ea5a4c0aff..e0eacc23cccd 100644
>>>>>> --- a/mm/shmem.c
>>>>>> +++ b/mm/shmem.c
>>>>>> @@ -3355,11 +3355,14 @@ static ssize_t shmem_file_read_iter(struct 
>>>>>> kiocb *iocb, struct iov_iter *to)
>>>>>>       struct file *file = iocb->ki_filp;
>>>>>>       struct inode *inode = file_inode(file);
>>>>>>       struct address_space *mapping = inode->i_mapping;
>>>>>> +    struct folio_batch fbatch;
>>>>>>       pgoff_t index;
>>>>>>       unsigned long offset;
>>>>>>       int error = 0;
>>>>>>       ssize_t retval = 0;
>>>>>> +    folio_batch_init(&fbatch);
>>>>>> +
>>>>>>       for (;;) {
>>>>>>           struct folio *folio = NULL;
>>>>>>           struct page *page = NULL;
>>>>>> @@ -3372,18 +3375,38 @@ static ssize_t shmem_file_read_iter(struct 
>>>>>> kiocb *iocb, struct iov_iter *to)
>>>>>>               break;
>>>>>>           index = iocb->ki_pos >> PAGE_SHIFT;
>>>>>> -        error = shmem_get_folio(inode, index, 0, &folio, SGP_READ);
>>>>>> -        if (error) {
>>>>>> -            if (error == -EINVAL)
>>>>>> -                error = 0;
>>>>>> -            break;
>>>>>> +fetch:
>>>>>> +        folio = folio_batch_next(&fbatch);
>>>>>> +        if (!folio) {
>>>>>> +            pgoff_t start = index;
>>>>>> +            pgoff_t end = (iocb->ki_pos + to->count - 1) >> 
>>>>>> PAGE_SHIFT;
>>>>>> +
>>>>>> +            if (folio_batch_count(&fbatch)) {
>>>>>> +                for (int i = 0; i < folio_batch_count(&fbatch); i++)
>>>>>> +                    folio_put(fbatch.folios[i]);
>>>>>> +                folio_batch_reinit(&fbatch);
>>>>>> +            }
>>>>>> +
>>>>>> +            filemap_get_folios_contig(inode->i_mapping, &start, 
>>>>>> end, &fbatch);
>>>>>> +            if (folio_batch_count(&fbatch))
>>>>>> +                goto fetch;
>>>>>> +
>>>>>> +            error = shmem_get_folio(inode, index, 0, &folio, 
>>>>>> SGP_READ);
>>>>>> +            if (unlikely(error)) {
>>>>>> +                if (error == -EINVAL)
>>>>>> +                    error = 0;
>>>>>> +                break;
>>>>>> +            }
>>>>>> +            if (folio) {
>>>>>> +                folio_unlock(folio);
>>>>>> +                folio_batch_add(&fbatch, folio);
>>>>>> +                fbatch.i++;
>>>>>> +            }
>>>>>>           }
>>>>>> -        if (folio) {
>>>>>> -            folio_unlock(folio);
>>>>>> +        if (folio) {
>>>>>>               page = folio_file_page(folio, index);
>>>>>>               if (PageHWPoison(page)) {
>>>>>> -                folio_put(folio);
>>>>>>                   error = -EIO;
>>>>>>                   break;
>>>>>>               }
>>>>>
>>>>> I haven't tested it yet, but I'm sure this will break the 
>>>>> fallback_page_copy mode.
>>>>>
>>>>> You're fetching from the batch at folio granularity here, but in 
>>>>> fallback_page_copy mode, we still copy the data at page granularity.
>>>>
>>>> Yes, that's a bug, Sashika has already reported it. I’ll fix it in 
>>>> the next version.
>>>
>>> Good.
>>>
>>>> Sashika also reported another potential issue below, but I think 
>>>> that is a misunderstanding, because a folio fetched from the swap 
>>>> cache is already marked Uptodate before being added to the page 
>>>> cache. What do you think?
>>>
>>> I'm not concerned about swap-in folios, since we already ensure the 
>>> folio is uptodate when swapping out.
>>>
>>> What I'm worried about is that this change moves the 
>>> folio_test_uptodate() check outside of the folio lock, which could 
>>> introduce some race? Of course, with the original logic, the folio 
>>> state could also transition from !uptodate to uptodate after the folio
>>> is unlocked (I need more thought).
>>>
>>> Another race condition that needs to be addressed is, when getting 
>>> the folio locklessly via filemap_get_folios_contig(), there could be 
>>> a concurrent race with swap-out (because you removed the folio- 
>>> >mapping check under folio lock). This could result in reading zero 
>>> data when it shouldn't (the folio actually needs to be swapped in).
>>
>> Sorry, I don't fully understand what you mean. It seems that swap-out 
>> does not clear the uptodate flag, and the folio cannot be freed during 
>> the read operation because its reference count is held. So a swapped- 
>> out folio should not be mistaken for a zero-filled folio. Right?
> 
> Sorry for the late reply. Ah, I missed the held refcnt, and now sounds 
> reasonable.

I've just sent out v2. Feel free to comment. :)

https://lore.kernel.org/linux-fsdevel/20260601055704.167436-1-chizhiling@163.com/

Re: [PATCH v1 0/5] mm/shmem: optimize read with reduced xarray lookups and folio batching

[Andrew Morton]

On Wed, 20 May 2026 18:15:33 +0800 Chi Zhiling <chizhiling@163.com> wrote:

> From: Chi Zhiling <chizhiling@kylinos.cn>
> 
> This series improves shmem read performance by implementing folio
> batching in the read path and reducing unnecessary xarray lookups.
> 

Thanks.

> Performance Results
> ===================
> 
> Testing was performed with fio sequential read workloads:
> 
>   fio --ioengine=sync --rw=read --size=1G --runtime=180
> 
> 
> ### THP Disabled - Normal Files ###
> 
> | Block Size | Baseline  | v1        | Improvement |
> | ---------- | --------- | --------- | ----------- |
> | 1M         | 11.4GiB/s | 12.7GiB/s | +11.4%      |
> | 64k        | 11.2GiB/s | 12.2GiB/s | +8.9%       |
> | 4k         | 3809MiB/s | 3838MiB/s | +0.8%       |
> 
> ### THP Disabled - Fallocated Files ###
> 
> | Block Size | Baseline  | v1        | Improvement |
> | ---------- | --------- | --------- | ----------- |
> | 1M         | 23.7GiB/s | 28.7GiB/s | +21.1%      |
> | 64k        | 22.6GiB/s | 27.0GiB/s | +19.5%      |
> | 4k         | 4668MiB/s | 4678MiB/s | +0.2%       |
> 
> ### THP Enabled - Normal Files ###
> 
> | Block Size | Baseline  | v1        | Improvement |
> | ---------- | --------- | --------- | ----------- |
> | 1M         | 13.9GiB/s | 13.9GiB/s | 0%          |
> | 64k        | 13.4GiB/s | 13.4GiB/s | 0%          |
> | 4k         | 3818MiB/s | 3836MiB/s | +0.5%       |
> 
> ### THP Enabled - Fallocated Files ###
> 
> | Block Size | Baseline  | v1        | Improvement |
> | ---------- | --------- | --------- | ----------- |
> | 1M         | 24.1GiB/s | 34.9GiB/s | +44.8%      |
> | 64k        | 22.9GiB/s | 31.3GiB/s | +36.7%      |
> | 4k         | 4721MiB/s | 4708MiB/s | -0.3%       |

That looks nice.

AI review might have found a few things:
	https://sashiko.dev/#/patchset/20260520101538.58745-1-chizhiling@163.com

I'll skip the patchset for now (unreviewed v1!).

Re: [PATCH v1 0/5] mm/shmem: optimize read with reduced xarray lookups and folio batching

[Chi Zhiling]

On 5/22/26 08:14, Andrew Morton wrote:
> On Wed, 20 May 2026 18:15:33 +0800 Chi Zhiling <chizhiling@163.com> wrote:
> 
>> From: Chi Zhiling <chizhiling@kylinos.cn>
>>
>> This series improves shmem read performance by implementing folio
>> batching in the read path and reducing unnecessary xarray lookups.
>>
> 
> Thanks.
> 
>> Performance Results
>> ===================
>>
>> Testing was performed with fio sequential read workloads:
>>
>>    fio --ioengine=sync --rw=read --size=1G --runtime=180
>>
>>
>> ### THP Disabled - Normal Files ###
>>
>> | Block Size | Baseline  | v1        | Improvement |
>> | ---------- | --------- | --------- | ----------- |
>> | 1M         | 11.4GiB/s | 12.7GiB/s | +11.4%      |
>> | 64k        | 11.2GiB/s | 12.2GiB/s | +8.9%       |
>> | 4k         | 3809MiB/s | 3838MiB/s | +0.8%       |
>>
>> ### THP Disabled - Fallocated Files ###
>>
>> | Block Size | Baseline  | v1        | Improvement |
>> | ---------- | --------- | --------- | ----------- |
>> | 1M         | 23.7GiB/s | 28.7GiB/s | +21.1%      |
>> | 64k        | 22.6GiB/s | 27.0GiB/s | +19.5%      |
>> | 4k         | 4668MiB/s | 4678MiB/s | +0.2%       |
>>
>> ### THP Enabled - Normal Files ###
>>
>> | Block Size | Baseline  | v1        | Improvement |
>> | ---------- | --------- | --------- | ----------- |
>> | 1M         | 13.9GiB/s | 13.9GiB/s | 0%          |
>> | 64k        | 13.4GiB/s | 13.4GiB/s | 0%          |
>> | 4k         | 3818MiB/s | 3836MiB/s | +0.5%       |
>>
>> ### THP Enabled - Fallocated Files ###
>>
>> | Block Size | Baseline  | v1        | Improvement |
>> | ---------- | --------- | --------- | ----------- |
>> | 1M         | 24.1GiB/s | 34.9GiB/s | +44.8%      |
>> | 64k        | 22.9GiB/s | 31.3GiB/s | +36.7%      |
>> | 4k         | 4721MiB/s | 4708MiB/s | -0.3%       |
> 
> That looks nice.
> 
> AI review might have found a few things:
> 	https://sashiko.dev/#/patchset/20260520101538.58745-1-chizhiling@163.com
> 
> I'll skip the patchset for now (unreviewed v1!).

Okay, I will fix those issues in v2.

Thanks!