[PATCH v2 0/1] mm, swap: only prefer nonfull clusters on synchronous devices

[PATCH v2 0/1] mm, swap: only prefer nonfull clusters on synchronous devices

[Alexandre Ghiti]

7.3 regressed swap I/O: for the same swap volume, a workload that
rewrites swapped out memory sends many more requests to the device.
mysql/sysbench in a 256M memcg swapping to an NVMe partition, 7.3
compared to the previous version:

  device read requests per transaction        +59%
  device write requests per transaction       +27%
  throughput, iocost off                      no change
  throughput, iocost on                       -19%

Two commits interact:

- Commit 9a42aed48421 ("mm, swap: prefer nonfull over free clusters"),
  in 6.18, makes allocations fill the nonfull clusters of their order
  before taking a free one.

- Commit 3cedb745d354 ("mm: clarify the folio_free_swap() for
  do_swap_page()"), in 7.3, frees the swap slot on an exclusive write
  fault. 7.0 to 7.2 kept the slot, but only as a side effect of commit
  4b34f1d82c65 ("mm, swap: free the swap cache after folio is
  mapped").

The freed slots leave holes in full clusters, which then go back on
the nonfull list. Reclaim allocates slots one folio at a time and
merges the writes afterwards, so a batch of LRU-adjacent folios gets
scattered over those holes and is written with many small requests.
Readahead later finds those pages scattered as well, and needs more
read requests to bring them back.

The RFC [1] listed three fixes, which perform similarly:

A. Revert 3cedb745d354. This keeps the slot on write fault, but undoes
   the intent of that commit and keeps swap space pinned under dirty
   pages.

B. Below 50% swap usage, make order-0 allocations take a free cluster
   before a nonfull one. This is what the allocator already does, at
   any usage, on swap devices with discard enabled (swapon
   --discard=pages). The threshold is arbitrary and it stops working
   as swap fills up.

C. The RFC patch. Reclaim tells the allocator how many pages of each
   small order the batch is about to swap out, and those allocations
   then follow contiguous free slots.

In the RFC thread, Kairui suggested another option, which he prefers
[2]: keep nonfull-first only on synchronous devices like zram, which
don't merge I/O and where filling holes first keeps the free clusters
for large folios, and go back to preferring free clusters on every
other device, as discard devices already do. This is what v2 does. It
is a few lines, needs no information from reclaim, and performs as
well as C.

Same workload as above, each fix relative to its base kernel:

                                              RFC (C)      v2
  device read requests per transaction        -12%         -14%
  device write requests per transaction       -36%         -48%
  throughput, iocost off                      no change    no change
  throughput, iocost on                       +41%         +33%

Changes since RFC:
- Replace the batch-aware slot allocation with keeping nonfull-first
  for synchronous devices only, as suggested by Kairui.

[1] https://lore.kernel.org/linux-mm/20261022145615.2184343-1-alex@ghiti.fr/
[2] https://lore.kernel.org/linux-mm/CAMgjq7BFbXEKEt9faj_rg5AStfLkpJ0j57EQOOJ+L3-w2xRvtg@mail.gmail.com/

Alexandre Ghiti (1):
  mm, swap: only prefer nonfull clusters on synchronous devices

 mm/swapfile.c | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

base-commit: afb4eea28037fa55c4560313ecbd8a1e1da8369a

[PATCH v2 1/1] mm, swap: only prefer nonfull clusters on synchronous devices

[Alexandre Ghiti]

Commit 9a42aed48421 ("mm, swap: prefer nonfull over free clusters") made
allocations fill the nonfull clusters of their order before taking a
free one, on every device without discard. Since commit 3cedb745d354
("mm: clarify the folio_free_swap() for do_swap_page()"), an exclusive
write fault frees the swap slot, so a workload that rewrites swapped out
memory leaves holes in full clusters, which then go back on the nonfull
list. Reclaim allocates slots one folio at a time, so a batch of folios
gets scattered over those holes, its writes can't be merged and it is
written with many small requests.

So partially revert commit 9a42aed48421 ("mm, swap: prefer nonfull over
free clusters") and prefer free clusters again, except on synchronous
devices like zram: they don't merge I/O, so the contiguity of the slots
doesn't matter to them, and filling the holes first keeps the free
clusters for large folios.

Fixes: 9a42aed48421 ("mm, swap: prefer nonfull over free clusters")
Suggested-by: Kairui Song <kasong@tencent.com>
Signed-off-by: Alexandre Ghiti <alex@ghiti.fr>
---
 mm/swapfile.c | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/mm/swapfile.c b/mm/swapfile.c
index 189f04d8838f..1c1b5d0b724a 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -973,6 +973,17 @@ static bool __swap_cluster_alloc_entries(struct swap_info_struct *si,
 	return true;
 }
 
+/*
+ * Prefer a new cluster over a nonfull one if the device needs discard, to
+ * spread out the writes, or if it is not synchronous, to keep the slots
+ * of a swapout batch contiguous so that its writes can be merged.
+ */
+static bool swap_prefer_free_cluster(struct swap_info_struct *si)
+{
+	return (si->flags & SWP_PAGE_DISCARD) ||
+	       !(si->flags & SWP_SYNCHRONOUS_IO);
+}
+
 /* Try use a new cluster for current CPU and allocate from it. */
 static unsigned int alloc_swap_scan_cluster(struct swap_info_struct *si,
 					    struct swap_cluster_info *ci,
@@ -1143,11 +1154,7 @@ static unsigned long cluster_alloc_swap_entry(struct swap_info_struct *si,
 	}
 
 new_cluster:
-	/*
-	 * If the device need discard, prefer new cluster over nonfull
-	 * to spread out the writes.
-	 */
-	if (si->flags & SWP_PAGE_DISCARD) {
+	if (swap_prefer_free_cluster(si)) {
 		found = alloc_swap_scan_list(si, &si->free_clusters, folio, false);
 		if (found)
 			goto done;
@@ -1159,7 +1166,7 @@ static unsigned long cluster_alloc_swap_entry(struct swap_info_struct *si,
 			goto done;
 	}
 
-	if (!(si->flags & SWP_PAGE_DISCARD)) {
+	if (!swap_prefer_free_cluster(si)) {
 		found = alloc_swap_scan_list(si, &si->free_clusters, folio, false);
 		if (found)
 			goto done;
-- 
2.53.0-Meta

Re: [PATCH v2 1/1] mm, swap: only prefer nonfull clusters on synchronous devices

[Nhat Pham]

On Thu, Nov 5, 2026 at 8:51 AM Alexandre Ghiti <alex@ghiti.fr> wrote:
>
> Commit 9a42aed48421 ("mm, swap: prefer nonfull over free clusters") made
> allocations fill the nonfull clusters of their order before taking a
> free one, on every device without discard. Since commit 3cedb745d354
> ("mm: clarify the folio_free_swap() for do_swap_page()"), an exclusive
> write fault frees the swap slot, so a workload that rewrites swapped out
> memory leaves holes in full clusters, which then go back on the nonfull
> list. Reclaim allocates slots one folio at a time, so a batch of folios
> gets scattered over those holes, its writes can't be merged and it is
> written with many small requests.
>
> So partially revert commit 9a42aed48421 ("mm, swap: prefer nonfull over
> free clusters") and prefer free clusters again, except on synchronous
> devices like zram: they don't merge I/O, so the contiguity of the slots
> doesn't matter to them, and filling the holes first keeps the free
> clusters for large folios.
>
> Fixes: 9a42aed48421 ("mm, swap: prefer nonfull over free clusters")
> Suggested-by: Kairui Song <kasong@tencent.com>
> Signed-off-by: Alexandre Ghiti <alex@ghiti.fr>

LGTM for the most part:

Reviewed-by: Nhat Pham <nphamcs@gmail.com>

Quick comment below:

> ---
>  mm/swapfile.c | 19 +++++++++++++------
>  1 file changed, 13 insertions(+), 6 deletions(-)
>
> diff --git a/mm/swapfile.c b/mm/swapfile.c
> index 189f04d8838f..1c1b5d0b724a 100644
> --- a/mm/swapfile.c
> +++ b/mm/swapfile.c
> @@ -973,6 +973,17 @@ static bool __swap_cluster_alloc_entries(struct swap_info_struct *si,
>         return true;
>  }
>
> +/*
> + * Prefer a new cluster over a nonfull one if the device needs discard, to
> + * spread out the writes, or if it is not synchronous, to keep the slots
> + * of a swapout batch contiguous so that its writes can be merged.
> + */
> +static bool swap_prefer_free_cluster(struct swap_info_struct *si)
> +{
> +       return (si->flags & SWP_PAGE_DISCARD) ||
> +              !(si->flags & SWP_SYNCHRONOUS_IO);
> +}

Side note - I wonder if it's worth it to also prefer non-full clusters
for zswap as well?

It's a bit tricky, but I suppose you can do a best-effort check of the
folio's cgroup's zswappability?

Re: [PATCH v2 1/1] mm, swap: only prefer nonfull clusters on synchronous devices

[Kairui Song]

On Fri, Nov 6, 2026 at 12:05 AM Alexandre Ghiti <alex@ghiti.fr> wrote:
>
> Commit 9a42aed48421 ("mm, swap: prefer nonfull over free clusters") made
> allocations fill the nonfull clusters of their order before taking a
> free one, on every device without discard. Since commit 3cedb745d354
> ("mm: clarify the folio_free_swap() for do_swap_page()"), an exclusive
> write fault frees the swap slot, so a workload that rewrites swapped out
> memory leaves holes in full clusters, which then go back on the nonfull
> list. Reclaim allocates slots one folio at a time, so a batch of folios
> gets scattered over those holes, its writes can't be merged and it is
> written with many small requests.
>
> So partially revert commit 9a42aed48421 ("mm, swap: prefer nonfull over
> free clusters") and prefer free clusters again, except on synchronous
> devices like zram: they don't merge I/O, so the contiguity of the slots
> doesn't matter to them, and filling the holes first keeps the free
> clusters for large folios.
>
> Fixes: 9a42aed48421 ("mm, swap: prefer nonfull over free clusters")

Do we need Cc stable? I guess no because 3cedb745d354 is new?

> Suggested-by: Kairui Song <kasong@tencent.com>
> Signed-off-by: Alexandre Ghiti <alex@ghiti.fr>
> ---
>  mm/swapfile.c | 19 +++++++++++++------
>  1 file changed, 13 insertions(+), 6 deletions(-)
>
> diff --git a/mm/swapfile.c b/mm/swapfile.c
> index 189f04d8838f..1c1b5d0b724a 100644
> --- a/mm/swapfile.c
> +++ b/mm/swapfile.c
> @@ -973,6 +973,17 @@ static bool __swap_cluster_alloc_entries(struct swap_info_struct *si,
>         return true;
>  }
>
> +/*
> + * Prefer a new cluster over a nonfull one if the device needs discard, to
> + * spread out the writes, or if it is not synchronous, to keep the slots
> + * of a swapout batch contiguous so that its writes can be merged.
> + */
> +static bool swap_prefer_free_cluster(struct swap_info_struct *si)
> +{
> +       return (si->flags & SWP_PAGE_DISCARD) ||
> +              !(si->flags & SWP_SYNCHRONOUS_IO);
> +}

I actually wonder if we just want !si->flags & SWP_SYNCHRONOUS_IO
here, but still reasonable to keep the two checks here.

> +
>  /* Try use a new cluster for current CPU and allocate from it. */
>  static unsigned int alloc_swap_scan_cluster(struct swap_info_struct *si,
>                                             struct swap_cluster_info *ci,
> @@ -1143,11 +1154,7 @@ static unsigned long cluster_alloc_swap_entry(struct swap_info_struct *si,
>         }
>
>  new_cluster:
> -       /*
> -        * If the device need discard, prefer new cluster over nonfull
> -        * to spread out the writes.
> -        */
> -       if (si->flags & SWP_PAGE_DISCARD) {
> +       if (swap_prefer_free_cluster(si)) {

That's a pretty nice helper, thanks!

The patch itself looks good to me.

Acked-by: Kairui Song <kasong@tencent.com>

Re: [PATCH v2 1/1] mm, swap: only prefer nonfull clusters on synchronous devices

[Alexandre Ghiti]

Hi Kairui,

On Sat, Nov 7, 2026 at 11:59 AM Kairui Song <ryncsn@gmail.com> wrote:
>
> >
> On Fri, Nov 6, 2026 at 12:05 AM Alexandre Ghiti <alex@ghiti.fr> wrote:
> >
> > Commit 9a42aed48421 ("mm, swap: prefer nonfull over free clusters") made
> > allocations fill the nonfull clusters of their order before taking a
> > free one, on every device without discard. Since commit 3cedb745d354
> > ("mm: clarify the folio_free_swap() for do_swap_page()"), an exclusive
> > write fault frees the swap slot, so a workload that rewrites swapped out
> > memory leaves holes in full clusters, which then go back on the nonfull
> > list. Reclaim allocates slots one folio at a time, so a batch of folios
> > gets scattered over those holes, its writes can't be merged and it is
> > written with many small requests.
> >
> > So partially revert commit 9a42aed48421 ("mm, swap: prefer nonfull over
> > free clusters") and prefer free clusters again, except on synchronous
> > devices like zram: they don't merge I/O, so the contiguity of the slots
> > doesn't matter to them, and filling the holes first keeps the free
> > clusters for large folios.
> >
> > Fixes: 9a42aed48421 ("mm, swap: prefer nonfull over free clusters")
>
> Do we need Cc stable? I guess no because 3cedb745d354 is new?

I think it should go to stable since it was merged in 7.3, I'm adding
the tag below (if it works that way in mm):

Cc: stable@vger.kernel.org

>
> > Suggested-by: Kairui Song <kasong@tencent.com>
> > Signed-off-by: Alexandre Ghiti <alex@ghiti.fr>
> > ---
> >  mm/swapfile.c | 19 +++++++++++++------
> >  1 file changed, 13 insertions(+), 6 deletions(-)
> >
> > diff --git a/mm/swapfile.c b/mm/swapfile.c
> > index 189f04d8838f..1c1b5d0b724a 100644
> > --- a/mm/swapfile.c
> > +++ b/mm/swapfile.c
> > @@ -973,6 +973,17 @@ static bool __swap_cluster_alloc_entries(struct swap_info_struct *si,
> >         return true;
> >  }
> >
> > +/*
> > + * Prefer a new cluster over a nonfull one if the device needs discard, to
> > + * spread out the writes, or if it is not synchronous, to keep the slots
> > + * of a swapout batch contiguous so that its writes can be merged.
> > + */
> > +static bool swap_prefer_free_cluster(struct swap_info_struct *si)
> > +{
> > +       return (si->flags & SWP_PAGE_DISCARD) ||
> > +              !(si->flags & SWP_SYNCHRONOUS_IO);
> > +}
>
> I actually wonder if we just want !si->flags & SWP_SYNCHRONOUS_IO
> here, but still reasonable to keep the two checks here.
>
> > +
> >  /* Try use a new cluster for current CPU and allocate from it. */
> >  static unsigned int alloc_swap_scan_cluster(struct swap_info_struct *si,
> >                                             struct swap_cluster_info *ci,
> > @@ -1143,11 +1154,7 @@ static unsigned long cluster_alloc_swap_entry(struct swap_info_struct *si,
> >         }
> >
> >  new_cluster:
> > -       /*
> > -        * If the device need discard, prefer new cluster over nonfull
> > -        * to spread out the writes.
> > -        */
> > -       if (si->flags & SWP_PAGE_DISCARD) {
> > +       if (swap_prefer_free_cluster(si)) {
>
> That's a pretty nice helper, thanks!
>
> The patch itself looks good to me.
>
> Acked-by: Kairui Song <kasong@tencent.com>

Thanks,

Alex

>

Re: [PATCH v2 1/1] mm, swap: only prefer nonfull clusters on synchronous devices

[Andrew Morton]

On Mon, 9 Nov 2026 10:00:45 +0100 Alexandre Ghiti <alexghiti@meta.com> wrote:

> > The patch itself looks good to me.
> >
> > Acked-by: Kairui Song <kasong@tencent.com>
> 
> Thanks,

Thanks all.

Please avoid single-patch patch series?  Present them as a unified
singleton.

Here's what I ended up with.  Please check.


From: Alexandre Ghiti <alex@ghiti.fr>
Subject: mm, swap: only prefer nonfull clusters on synchronous devices
Date: Mon, 09 Nov 2026 15:18:32 -0800

Commit 9a42aed48421 ("mm, swap: prefer nonfull over free clusters") made
the allocator fill nonfull clusters before taking free ones on swap
devices without discard.  Commit 3cedb745d354 ("mm: clarify the
folio_free_swap() for do_swap_page()") frees the swap slot on an exclusive
write fault.  Such faults now leave holes in formerly full clusters,
putting those clusters back on the nonfull list.  Reclaim allocates slots
one folio at a time, scattering a batch across the holes instead of
producing mergeable writes.  Later swapin can also require more read
requests.

On a mysql/sysbench workload in a 256 MiB memcg swapping to an NVMe
partition, 7.3 compared with the previous version produced 59% more device
read requests and 27% more write requests per transaction.  Throughput was
unchanged without iocost and fell 19% with iocost.

Prefer free clusters first on nonsynchronous devices, preserving
contiguous swap slots for merging I/O.  Keep nonfull-first on synchronous
devices such as zram: they do not merge I/O, and filling holes saves free
clusters for large folios.  Keep free-cluster preference on discard
devices to spread writes.

In the same workload, v2 reduced read requests per transaction by 14% and
write requests by 48% against its base kernel.  Throughput was unchanged
without iocost and improved 33% with iocost.  The RFC's batch-aware
allocator also improved this workload, but this policy needs no batch
information from reclaim.  Reverting the slot-freeing change would keep
swap space pinned under dirty pages; a proposed 50% swap-usage cutoff for
free-cluster preference was arbitrary and would stop helping as swap
filled.

The 7.0 to 7.2 kernels retained the swap slot after a write fault as a
side effect of 4b34f1d82c65 ("mm, swap: free the swap cache after folio is
mapped").  The observed regression requires both 9a42aed48421 and
3cedb745d354; stable backports should target trees containing both.

Link: https://lore.kernel.org/r/20261105155032.240457-1-alex@ghiti.fr
Link: https://lore.kernel.org/r/20261105155032.240457-2-alex@ghiti.fr
Link: https://lore.kernel.org/r/20261022145615.2184343-1-alex@ghiti.fr
Link: https://lore.kernel.org/r/CAMgjq7BFbXEKEt9faj_rg5AStfLkpJ0j57EQOOJ+L3-w2xRvtg@mail.gmail.com
Fixes: 9a42aed48421 ("mm, swap: prefer nonfull over free clusters")
Signed-off-by: Alexandre Ghiti <alex@ghiti.fr>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Suggested-by: Kairui Song <kasong@tencent.com>
Acked-by: Kairui Song <kasong@tencent.com>
Reviewed-by: Nhat Pham <nphamcs@gmail.com>
Cc: <stable@vger.kernel.org>
---

 mm/swapfile.c |   19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

--- a/mm/swapfile.c~mm-swap-only-prefer-nonfull-clusters-on-synchronous-devices
+++ a/mm/swapfile.c
@@ -973,6 +973,17 @@ static bool __swap_cluster_alloc_entries
 	return true;
 }
 
+/*
+ * Prefer a new cluster over a nonfull one if the device needs discard, to
+ * spread out the writes, or if it is not synchronous, to keep the slots
+ * of a swapout batch contiguous so that its writes can be merged.
+ */
+static bool swap_prefer_free_cluster(struct swap_info_struct *si)
+{
+	return (si->flags & SWP_PAGE_DISCARD) ||
+	       !(si->flags & SWP_SYNCHRONOUS_IO);
+}
+
 /* Try use a new cluster for current CPU and allocate from it. */
 static unsigned int alloc_swap_scan_cluster(struct swap_info_struct *si,
 					    struct swap_cluster_info *ci,
@@ -1143,11 +1154,7 @@ static unsigned long cluster_alloc_swap_
 	}
 
 new_cluster:
-	/*
-	 * If the device need discard, prefer new cluster over nonfull
-	 * to spread out the writes.
-	 */
-	if (si->flags & SWP_PAGE_DISCARD) {
+	if (swap_prefer_free_cluster(si)) {
 		found = alloc_swap_scan_list(si, &si->free_clusters, folio, false);
 		if (found)
 			goto done;
@@ -1159,7 +1166,7 @@ new_cluster:
 			goto done;
 	}
 
-	if (!(si->flags & SWP_PAGE_DISCARD)) {
+	if (!swap_prefer_free_cluster(si)) {
 		found = alloc_swap_scan_list(si, &si->free_clusters, folio, false);
 		if (found)
 			goto done;
_