[RFC PATCH v2 0/9] mm: support zswap-backed large folio swapin

[RFC PATCH v2 0/9] mm: support zswap-backed large folio swapin

[fujunjie]

Hi,

This RFC explores large-folio swapin for ranges that are still fully backed
by zswap.

Large swapin is currently disabled once zswap is in the picture. Anonymous
faults stop considering large orders after zswap has ever been enabled,
shmem does the same, and zswap_load() refuses large swapcache folios. That
keeps mixed zswap/disk cases safe, but it also loses the dense case where
every slot in an aligned 64K range is still resident in zswap.

The series keeps the policy in common swapin code:

  - zswap reports backend facts and provides the large-folio load helper.
  - swapin_sync() filters candidate orders by backend range.
  - all-disk and zeromap ranges keep the existing Kairui large-swapin path.
  - mixed zswap/disk ranges stay order-0.
  - all-zswap ranges may use a 64K folio after locality admission.
  - anon provides locality evidence from VMA hints and PTE young density.
  - shmem starts with explicit VMA-hint evidence only.
  - swap readahead uses its existing VMA/cluster window as locality
    evidence; it does not also run the anon PTE-young rule.

The backend range probe is only a snapshot. If the backend changes after a
fresh large swapcache folio is allocated, the common path drops that folio
and falls back to order-0. zswap_load() can also return -EAGAIN for the
same retry path. If a late fault retry keeps the large folio in swapcache
instead of deleting it, the cgroup v1 memsw swap owner is committed before
returning.

This is mTHP/large-folio swapin. The mappings installed by do_swap_page()
are still PTE mappings, not PMD mappings. The expected win is fewer faults,
batched PTE/rmap work, and preserving the large folio across zswapin
instead of rebuilding the working set as order-0 pages.

Prior art: Usama Arif posted a related RFC on 2024-10-18:

  mm: zswap: add support for zswapin of large folios
  https://lore.kernel.org/linux-mm/20241018105026.2521366-1-usamaarif642@gmail.com/

This RFC keeps the same broad goal, but moves admission into common swapin
code. zswap does not decide the policy. Mixed zswap/disk ranges are
rejected before large IO, and the first cap is 64K.

This is a rewrite of the RFC posted on 2026-05-08:

  [RFC PATCH 0/5] mm: support zswap-backed anonymous large folio swapin
  https://lore.kernel.org/linux-mm/tencent_8B437BE4F586C162950BF71954316C1EDB05@qq.com/

The v1 series was anonymous-only and kept too much of the policy near the
anon fault and zswap paths. This version is rebuilt on top of Kairui Song's
common swapin infrastructure. It keeps admission in common swapin code,
rejects mixed zswap/disk large ranges, and adds separate locality producers
for anon, shmem and swap readahead.

Performance and behavior
========================

The A/B tables are 10-run measurements. Elapsed values are seconds,
shown as mean +/- sample standard deviation. "phase" or "refault" is the
measured refault subphase. "zswpin" counts zswap loads. "pswpin" counts
swap-ins from the real swap device; pswpin=0 means the refaults were served
by zswap even when a disk swap device was configured. "RFC 64K" is the mean
number of successful 64K swapins.

The numbers below show where the large path is used and where it is
rejected.

zram-backed zswap microbench, 64K mTHP, 8G guest:

+-----------------+----------------+----------------+--------+--------+--------+----------+
| workload        | base elapsed   | RFC elapsed    | delta  | phase  | zswpin | RFC 64K  |
+-----------------+----------------+----------------+--------+--------+--------+----------+
| usama_1g        | 11.260+/-0.235 | 10.301+/-0.140 | -8.5%  | -22.2% | 1.000x | 16381.1  |
| nohint_seq64    |  4.398+/-0.085 |  4.025+/-0.022 | -8.5%  | -21.1% | 1.000x |  6221.1  |
| seqhint_seq64   |  4.283+/-0.060 |  3.948+/-0.062 | -7.8%  | -20.6% | 1.000x |  6223.5  |
| stride64_sparse |  3.095+/-0.051 |  3.086+/-0.025 | -0.3%  |  +5.8% | 1.002x |     1.0  |
| random64_sparse |  3.095+/-0.046 |  3.076+/-0.016 | -0.6%  |  +0.7% | 1.001x |     0.0  |
| random64_full   |  4.423+/-0.067 |  4.405+/-0.018 | -0.4%  |  +0.1% | 1.000x |     0.0  |
+-----------------+----------------+----------------+--------+--------+--------+----------+

The usama_1g row follows the shape of the 2024 RFC benchmark: allocate 1G,
fill it with compressible per-page data, reclaim it through memory.reclaim,
then time the full integrity-check refault. The seq64 rows use a 512M
target and 768M of pressure. "sparse" touches one 4K page per 64K region, while
"full" touches every 4K page. "seqhint" uses MADV_SEQUENTIAL; "nohint" does
not.

Virtio-block swap device present, zswap enabled:

+-----------------+---------------+---------------+--------+---------+--------+--------+---------+
| workload        | base elapsed  | RFC elapsed   | delta  | refault | pswpin | zswpin | RFC 64K |
+-----------------+---------------+---------------+--------+---------+--------+--------+---------+
| seq64           | 4.399+/-0.100 | 4.279+/-0.216 | -2.7%  | -10.5%  | 0      | 1.000x | 3110.7  |
| stride64_sparse | 3.103+/-0.047 | 3.119+/-0.086 | +0.5%  |  +6.2%  | 0      | 0.999x |    0.0  |
| random64_sparse | 3.142+/-0.112 | 3.097+/-0.030 | -1.4%  |  -2.2%  | 0      | 0.999x |    0.1  |
| random64_full   | 4.473+/-0.147 | 4.445+/-0.088 | -0.6%  |  +0.9%  | 0      | 1.000x |    0.4  |
+-----------------+---------------+---------------+--------+---------+--------+--------+---------+

This run uses a real block swap device, but the refaulted data stayed in
zswap. It covers the all-zswap hit path with disk swap configured, not disk
read IO.

Virtio-block pressure/mixed run, zswap max_pool_percent=1,
low-compressibility full fill:

+-------------------------------+---------------+---------------+--------+---------+----------------+------------+---------+----------+
| workload                      | base elapsed  | RFC elapsed   | delta  | refault | pswpin base/RFC | RFC zswpin | RFC 64K | fallback |
+-------------------------------+---------------+---------------+--------+---------+----------------+------------+---------+----------+
| seq64_full_pressure           | 5.908+/-0.195 | 5.790+/-0.235 | -2.0%  |  +3.0%  | 90258/99038    | 20327      |   0.0   | 3730     |
| random64_sparse_full_pressure | 5.104+/-0.069 | 5.068+/-0.090 | -0.7%  |  -9.1%  |  6201/6196     |  1297      |   0.0   |    0     |
+-------------------------------+---------------+---------------+--------+---------+----------------+------------+---------+----------+

This run reaches the disk-backed path: pswpin is non-zero in both base and
RFC. It is mainly fallback coverage. The RFC does not install 64K folios
for these disk/mixed-heavy ranges.

Policy matrix, virtio-block swap device present:

+------------------------------+----+------+--------+--------+-------+----------+
| case                         | pc | hint | pswpin | zswpin | zswpwb| 64K in   |
+------------------------------+----+------+--------+--------+-------+----------+
| pc0_seq                      | 0  | none | 0      | 99559  | 0     | 0        |
| pc3_seq                      | 3  | none | 0      | 99498  | 0     | 0        |
| pc4_seq                      | 4  | none | 0      | 99512  | 0     | 3109     |
| pc5_seq                      | 5  | none | 0      | 99657  | 0     | 3113     |
| hint_none_random_sparse      | 5  | none | 0      |  6265  | 0     | 0        |
| hint_random_seq              | 5  | rand | 0      | 99488  | 0     | 0        |
| mixed_seq_full               | 5  | none | 97725  | 20147  | 84    | 569      |
| mixed_random_sparse_full     | 5  | none |  6230  |  1302  | 0     | 0        |
+------------------------------+----+------+--------+--------+-------+----------+

The pc rows show the readahead-window gate. The hint_random_seq row shows
the explicit random hint veto. The mixed rows use a small zswap pool to
force disk/zswap split backing; most mixed ranges are rejected, while any
remaining 64K successes were all-zswap at the time of the fault.

Kbuild pressure, zram swap, 384M memcg:

+----------------------+----------+----------+--------+--------+----------+
| setup                | base     | RFC      | delta  | zswpin | RFC 64K  |
+----------------------+----------+----------+--------+--------+----------+
| zram swap, 384M memcg| 2060.323 | 2047.516 | -0.6%  | 0.991x | 2797     |
+----------------------+----------+----------+--------+--------+----------+

This is a single-run zram pressure smoke. It did not show Kbuild
regression, and the RFC run installed 64K zswap-backed folios. The result
should not be read as a tuned-performance claim.

Kbuild pressure, virtio-block swap device, 512M memcg:

+-------------------------+----------+----------+--------+--------+----------+
| setup                   | base     | RFC      | delta  | pswpin | RFC 64K  |
+-------------------------+----------+----------+--------+--------+----------+
| disk swap, 512M memcg   | 1420.671 | 1409.263 | -0.8%  | 0      | 7497     |
+-------------------------+----------+----------+--------+--------+----------+

This is a single-run pressure smoke. The disk-swap Kbuild run also stayed
on the all-zswap hit path, so it is pressure coverage with a disk swap device
present rather than proof of disk-read large swapin.

Shmem smoke, tmpfs huge=always, 64K shmem mTHP:

+----------------------------+---------------+---------+-------------+----------+
| case                       | refault hint  | touched | 64K shmem   | 64K in   |
+----------------------------+---------------+---------+-------------+----------+
| nohint_seq                 | none          | 65536   | 4096        | 0        |
| seq_refault_hint           | sequential    | 65536   | 4096        | 4096     |
| random_refault_hint_sparse | random        |  4096   | 4096        | 0        |
+----------------------------+---------------+---------+-------------+----------+

That matches the current shmem producer: explicit sequential refault hints
allow large zswap swapin; no hint and random hints do not.

What this RFC does not establish
================================

The 64K cap is deliberate, but it is not tuned. The anon PTE-young rule is
only anon evidence. Shmem has the framework and explicit VMA hints in this
RFC, not a page-cache locality producer. For larger orders, the anon
producer should probably use bounded sampling instead of walking every PTE
in a 1M or larger candidate range. The mixed-backend tests cover fallback
behavior, but this series does not add mixed zswap/disk large IO.

Changes since RFC v1:

  - rebuilt the series on Kairui Song's common swapin/swap-table work;
  - moved large-swapin admission into common swapin code;
  - made zswap provide range facts and fully-zswap-backed folio loads;
  - rejected mixed zswap/disk large ranges before large IO;
  - capped zswap-backed swapin at 64K for this RFC;
  - added locality producers for anon, shmem hints and swap readahead;
  - covered cgroup v1 memsw accounting in speculative large-swapcache
    fallback paths;
  - added 10-run microbench data, mixed-backend pressure tests, shmem
    smoke tests, and zram/disk Kbuild pressure data.

fujunjie (9):
  mm/zswap: expose range state for swapin policy
  mm: let swap_read_folio() report retryable zswap races
  mm/zswap: support fully zswap-backed large folio loads
  mm: admit large swapin by backend range in swapin_sync()
  mm: add common locality admission for zswap large swapin
  mm: provide anon locality evidence for zswap large swapin
  mm/shmem: provide VMA-hint locality for zswap large swapin
  mm: try all-zswap large swapin within swap readahead windows
  docs: mm: update THP swapin counter descriptions

 Documentation/admin-guide/mm/transhuge.rst |  11 +-
 include/linux/zswap.h                      |  26 +
 mm/memcontrol-v1.c                         |   8 +-
 mm/memory.c                                | 269 +++++++-
 mm/page_io.c                               |  19 +-
 mm/shmem.c                                 |  42 +-
 mm/swap.h                                  |  21 +-
 mm/swap_state.c                            | 681 +++++++++++++++++++--
 mm/swapfile.c                              |   2 +-
 mm/zswap.c                                 | 149 ++++-
 10 files changed, 1099 insertions(+), 129 deletions(-)


base-commit: 404fb4f38e8f38469dfff4df0205c9d18eeb1f57
-- 
2.34.1

[RFC PATCH v2 1/9] mm/zswap: expose range state for swapin policy

[fujunjie]

Large folio swapin needs to know whether a candidate swap range is fully
backed by zswap before it can choose an order. That decision should stay
in common swapin code, not inside zswap.

Export two zswap facts for that caller: a lockless range occupancy snapshot
and the current zswap reclaim-pressure state. The range state is
advisory only. Writeback or invalidation can change the backend after the
snapshot, so users must recheck before issuing large-folio IO.

Signed-off-by: fujunjie <fujunjie1@qq.com>
---
 include/linux/zswap.h | 26 +++++++++++++++++++++++++
 mm/zswap.c            | 44 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 70 insertions(+)

diff --git a/include/linux/zswap.h b/include/linux/zswap.h
index 30c193a1207e..8f9aee97517c 100644
--- a/include/linux/zswap.h
+++ b/include/linux/zswap.h
@@ -9,6 +9,18 @@ struct lruvec;
 
 extern atomic_long_t zswap_stored_pages;
 
+/*
+ * Advisory zswap occupancy snapshot for a swap range. This is not a complete
+ * backend classifier; callers must recheck before depending on ALL_ZSWAP for
+ * large-folio IO.
+ */
+enum zswap_range_state {
+	ZSWAP_RANGE_NEVER_ENABLED,
+	ZSWAP_RANGE_NO_ZSWAP,
+	ZSWAP_RANGE_ALL_ZSWAP,
+	ZSWAP_RANGE_MIXED,
+};
+
 #ifdef CONFIG_ZSWAP
 
 struct zswap_lruvec_state {
@@ -27,6 +39,9 @@ struct zswap_lruvec_state {
 unsigned long zswap_total_pages(void);
 bool zswap_store(struct folio *folio);
 int zswap_load(struct folio *folio);
+enum zswap_range_state zswap_probe_range(swp_entry_t swp,
+					 unsigned int nr_pages);
+bool zswap_pool_reclaim_pressure(void);
 void zswap_invalidate(swp_entry_t swp);
 int zswap_swapon(int type, unsigned long nr_pages);
 void zswap_swapoff(int type);
@@ -49,6 +64,17 @@ static inline int zswap_load(struct folio *folio)
 	return -ENOENT;
 }
 
+static inline enum zswap_range_state zswap_probe_range(swp_entry_t swp,
+						       unsigned int nr_pages)
+{
+	return ZSWAP_RANGE_NEVER_ENABLED;
+}
+
+static inline bool zswap_pool_reclaim_pressure(void)
+{
+	return false;
+}
+
 static inline void zswap_invalidate(swp_entry_t swp) {}
 static inline int zswap_swapon(int type, unsigned long nr_pages)
 {
diff --git a/mm/zswap.c b/mm/zswap.c
index 761cd699e0a3..da5297f7bd69 100644
--- a/mm/zswap.c
+++ b/mm/zswap.c
@@ -506,6 +506,19 @@ unsigned long zswap_total_pages(void)
 	return total;
 }
 
+/*
+ * Expose whether zswap reclaim pressure is active. This is a backend fact:
+ * zswap_check_limits() sets the state once the pool reaches the hard limit and
+ * keeps it set until the pool falls below the accept threshold.
+ */
+bool zswap_pool_reclaim_pressure(void)
+{
+	if (zswap_never_enabled())
+		return false;
+
+	return READ_ONCE(zswap_pool_reached_full);
+}
+
 static bool zswap_check_limits(void)
 {
 	unsigned long cur_pages = zswap_total_pages();
@@ -1559,6 +1572,37 @@ bool zswap_store(struct folio *folio)
 	return ret;
 }
 
+enum zswap_range_state zswap_probe_range(swp_entry_t swp,
+					 unsigned int nr_pages)
+{
+	unsigned int type = swp_type(swp);
+	pgoff_t offset = swp_offset(swp);
+	bool present = false, missing = false;
+	unsigned int i;
+
+	/*
+	 * This is an advisory, lockless snapshot for common swapin admission.
+	 * Callers must recheck before depending on an all-zswap range for IO:
+	 * concurrent writeback or invalidation can change the backend state.
+	 */
+	if (zswap_never_enabled())
+		return ZSWAP_RANGE_NEVER_ENABLED;
+
+	for (i = 0; i < nr_pages; i++) {
+		struct xarray *tree = swap_zswap_tree(swp_entry(type, offset + i));
+
+		if (xa_load(tree, offset + i))
+			present = true;
+		else
+			missing = true;
+
+		if (present && missing)
+			return ZSWAP_RANGE_MIXED;
+	}
+
+	return present ? ZSWAP_RANGE_ALL_ZSWAP : ZSWAP_RANGE_NO_ZSWAP;
+}
+
 /**
  * zswap_load() - load a folio from zswap
  * @folio: folio to load
-- 
2.34.1

[RFC PATCH v2 2/9] mm: let swap_read_folio() report retryable zswap races

[fujunjie]

Large zswap loads need a way to ask the caller to drop a speculative large
swapcache folio and retry order-0. A void swap_read_folio() cannot express
that without turning a backend race into an IO failure.

Return int from swap_read_folio() and reserve -EAGAIN for retryable large
zswap races. Existing order-0 paths keep treating the read as before; the
synchronous swapin path only warns for now. A later patch will consume
-EAGAIN and retry order-0.

Signed-off-by: fujunjie <fujunjie1@qq.com>
---
 mm/page_io.c    | 19 +++++++++++++++++--
 mm/swap.h       |  5 +++--
 mm/swap_state.c | 13 +++++++++++--
 3 files changed, 31 insertions(+), 6 deletions(-)

diff --git a/mm/page_io.c b/mm/page_io.c
index f2d8fe7fd057..16724bdfb400 100644
--- a/mm/page_io.c
+++ b/mm/page_io.c
@@ -653,13 +653,21 @@ static void swap_read_folio_bdev_async(struct folio *folio,
 	submit_bio(bio);
 }
 
-void swap_read_folio(struct folio *folio, struct swap_iocb **plug)
+/*
+ * Return -EAGAIN only when a locked large swapcache folio hit a retryable
+ * zswap backend race. The caller owns that still-locked folio and must drop or
+ * retry it. Other zswap errors are still reported through the usual folio
+ * state: the folio is unlocked without PG_uptodate and the fault path will
+ * turn that into an I/O error.
+ */
+int swap_read_folio(struct folio *folio, struct swap_iocb **plug)
 {
 	struct swap_info_struct *sis = __swap_entry_to_info(folio->swap);
 	bool synchronous = sis->flags & SWP_SYNCHRONOUS_IO;
 	bool workingset = folio_test_workingset(folio);
 	unsigned long pflags;
 	bool in_thrashing;
+	int ret = 0;
 
 	VM_BUG_ON_FOLIO(!folio_test_swapcache(folio) && !synchronous, folio);
 	VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio);
@@ -681,8 +689,14 @@ void swap_read_folio(struct folio *folio, struct swap_iocb **plug)
 		goto finish;
 	}
 
-	if (zswap_load(folio) != -ENOENT)
+	ret = zswap_load(folio);
+	if (ret == -EAGAIN) {
+		VM_WARN_ON_ONCE_FOLIO(!folio_test_large(folio), folio);
 		goto finish;
+	}
+	if (ret != -ENOENT)
+		goto finish;
+	ret = 0;
 
 	/* We have to read from slower devices. Increase zswap protection. */
 	zswap_folio_swapin(folio);
@@ -701,6 +715,7 @@ void swap_read_folio(struct folio *folio, struct swap_iocb **plug)
 		psi_memstall_leave(&pflags);
 	}
 	delayacct_swapin_end();
+	return ret;
 }
 
 void __swap_read_unplug(struct swap_iocb *sio)
diff --git a/mm/swap.h b/mm/swap.h
index 77d2d14eda42..ea7e1f3c4410 100644
--- a/mm/swap.h
+++ b/mm/swap.h
@@ -241,7 +241,7 @@ extern void __swap_cluster_free_entries(struct swap_info_struct *si,
 /* linux/mm/page_io.c */
 int sio_pool_init(void);
 struct swap_iocb;
-void swap_read_folio(struct folio *folio, struct swap_iocb **plug);
+int swap_read_folio(struct folio *folio, struct swap_iocb **plug);
 void __swap_read_unplug(struct swap_iocb *plug);
 static inline void swap_read_unplug(struct swap_iocb *plug)
 {
@@ -381,8 +381,9 @@ static inline void folio_put_swap(struct folio *folio, struct page *page)
 {
 }
 
-static inline void swap_read_folio(struct folio *folio, struct swap_iocb **plug)
+static inline int swap_read_folio(struct folio *folio, struct swap_iocb **plug)
 {
+	return 0;
 }
 
 static inline void swap_write_unplug(struct swap_iocb *sio)
diff --git a/mm/swap_state.c b/mm/swap_state.c
index 04f5ce992401..d37097913b30 100644
--- a/mm/swap_state.c
+++ b/mm/swap_state.c
@@ -628,6 +628,7 @@ static struct folio *swap_cache_read_folio(swp_entry_t entry, gfp_t gfp,
 					   struct swap_iocb **plug, bool readahead)
 {
 	struct folio *folio;
+	int ret;
 
 	do {
 		folio = swap_cache_get_folio(entry);
@@ -639,7 +640,13 @@ static struct folio *swap_cache_read_folio(swp_entry_t entry, gfp_t gfp,
 	if (IS_ERR_OR_NULL(folio))
 		return NULL;
 
-	swap_read_folio(folio, plug);
+	ret = swap_read_folio(folio, plug);
+	/*
+	 * Swap readahead allocates order-0 folios. -EAGAIN is reserved for
+	 * retryable large zswap backend races and must be handled by the
+	 * synchronous common swapin path.
+	 */
+	VM_WARN_ON_ONCE(ret == -EAGAIN);
 	if (readahead) {
 		folio_set_readahead(folio);
 		count_vm_event(SWAP_RA);
@@ -668,6 +675,7 @@ struct folio *swapin_sync(swp_entry_t entry, gfp_t gfp, unsigned long orders,
 			   struct vm_fault *vmf, struct mempolicy *mpol, pgoff_t ilx)
 {
 	struct folio *folio;
+	int ret;
 
 	do {
 		folio = swap_cache_get_folio(entry);
@@ -679,7 +687,8 @@ struct folio *swapin_sync(swp_entry_t entry, gfp_t gfp, unsigned long orders,
 	if (IS_ERR(folio))
 		return folio;
 
-	swap_read_folio(folio, NULL);
+	ret = swap_read_folio(folio, NULL);
+	VM_WARN_ON_ONCE(ret == -EAGAIN);
 	return folio;
 }
 
-- 
2.34.1

[RFC PATCH v2 3/9] mm/zswap: support fully zswap-backed large folio loads

[fujunjie]

zswap currently refuses large swapcache folios. That is correct for mixed
backend ranges, but it also prevents the common swapin path from loading a
range that is still fully backed by zswap.

Teach zswap_load() to fill a locked large swapcache folio by decompressing
each base-page entry into the matching folio offset, then flushing the
folio once. A missing entry after zswap data has been seen is reported as
-EAGAIN so the caller can drop the speculative large folio and retry
order-0.

The large load keeps the zswap entries in place. It is a clean speculative
fill: until the swap slots are freed, zswap remains the backing copy if
reclaim drops the large folio before PTEs are installed.

Signed-off-by: fujunjie <fujunjie1@qq.com>
---
 mm/zswap.c | 105 ++++++++++++++++++++++++++++++++++++++++++++---------
 1 file changed, 87 insertions(+), 18 deletions(-)

diff --git a/mm/zswap.c b/mm/zswap.c
index da5297f7bd69..94ba112a2982 100644
--- a/mm/zswap.c
+++ b/mm/zswap.c
@@ -15,6 +15,8 @@
 
 #include <linux/module.h>
 #include <linux/cpu.h>
+#include <linux/mm.h>
+#include <linux/huge_mm.h>
 #include <linux/highmem.h>
 #include <linux/slab.h>
 #include <linux/spinlock.h>
@@ -934,7 +936,8 @@ static bool zswap_compress(struct page *page, struct zswap_entry *entry,
 	return comp_ret == 0 && alloc_ret == 0;
 }
 
-static bool zswap_decompress(struct zswap_entry *entry, struct folio *folio)
+static bool zswap_decompress(struct zswap_entry *entry, struct folio *folio,
+			     unsigned int page_idx, bool flush_dcache)
 {
 	struct zswap_pool *pool = entry->pool;
 	struct scatterlist input[2]; /* zsmalloc returns an SG list 1-2 entries */
@@ -952,14 +955,15 @@ static bool zswap_decompress(struct zswap_entry *entry, struct folio *folio)
 
 		WARN_ON_ONCE(input->length != PAGE_SIZE);
 
-		dst = kmap_local_folio(folio, 0);
+		dst = kmap_local_folio(folio, page_idx * PAGE_SIZE);
 		memcpy_from_sglist(dst, input, 0, PAGE_SIZE);
 		dlen = PAGE_SIZE;
 		kunmap_local(dst);
-		flush_dcache_folio(folio);
+		if (flush_dcache)
+			flush_dcache_folio(folio);
 	} else {
 		sg_init_table(&output, 1);
-		sg_set_folio(&output, folio, PAGE_SIZE, 0);
+		sg_set_folio(&output, folio, PAGE_SIZE, page_idx * PAGE_SIZE);
 		acomp_request_set_params(acomp_ctx->req, input, &output,
 					 entry->length, PAGE_SIZE);
 		ret = crypto_acomp_decompress(acomp_ctx->req);
@@ -1042,7 +1046,7 @@ static int zswap_writeback_entry(struct zswap_entry *entry,
 		goto out;
 	}
 
-	if (!zswap_decompress(entry, folio)) {
+	if (!zswap_decompress(entry, folio, 0, true)) {
 		ret = -EIO;
 		goto out;
 	}
@@ -1615,10 +1619,9 @@ enum zswap_range_state zswap_probe_range(swp_entry_t swp,
  *  NOT marked up-to-date, so that an IO error is emitted (e.g. do_swap_page()
  *  will SIGBUS).
  *
- *  -EINVAL: if the swapped out content was in zswap, but the page belongs
- *  to a large folio, which is not supported by zswap. The folio is unlocked,
- *  but NOT marked up-to-date, so that an IO error is emitted (e.g.
- *  do_swap_page() will SIGBUS).
+ *  -EAGAIN: if the swapped out content belongs to a large folio, but the
+ *  range is mixed or raced with writeback. The folio remains locked so the
+ *  caller can drop the large swapcache folio and retry order-0.
  *
  *  -ENOENT: if the swapped out content was not in zswap. The folio remains
  *  locked on return.
@@ -1626,9 +1629,12 @@ enum zswap_range_state zswap_probe_range(swp_entry_t swp,
 int zswap_load(struct folio *folio)
 {
 	swp_entry_t swp = folio->swap;
+	unsigned int nr_pages = folio_nr_pages(folio);
+	unsigned int type = swp_type(swp);
 	pgoff_t offset = swp_offset(swp);
-	struct xarray *tree = swap_zswap_tree(swp);
+	struct xarray *tree;
 	struct zswap_entry *entry;
+	unsigned int i;
 
 	VM_WARN_ON_ONCE(!folio_test_locked(folio));
 	VM_WARN_ON_ONCE(!folio_test_swapcache(folio));
@@ -1636,21 +1642,84 @@ int zswap_load(struct folio *folio)
 	if (zswap_never_enabled())
 		return -ENOENT;
 
-	/*
-	 * Large folios should not be swapped in while zswap is being used, as
-	 * they are not properly handled. Zswap does not properly load large
-	 * folios, and a large folio may only be partially in zswap.
-	 */
-	if (WARN_ON_ONCE(folio_test_large(folio))) {
+	if (folio_test_large(folio)) {
+		struct obj_cgroup *first_objcg = NULL;
+		bool same_objcg = true;
+		bool saw_zswap = false;
+		bool saw_non_zswap = false;
+
+		/*
+		 * The locked large swapcache folio now covers the range and
+		 * conflicts with zswap writeback's order-0 swapcache allocation.
+		 * If the range is mixed or an entry disappears, retry order-0.
+		 */
+		for (i = 0; i < nr_pages; i++) {
+			tree = swap_zswap_tree(swp_entry(type, offset + i));
+			entry = xa_load(tree, offset + i);
+			if (!entry) {
+				if (saw_zswap)
+					return -EAGAIN;
+				saw_non_zswap = true;
+				continue;
+			}
+			if (saw_non_zswap)
+				return -EAGAIN;
+
+			if (!saw_zswap)
+				first_objcg = entry->objcg;
+			else if (entry->objcg != first_objcg)
+				same_objcg = false;
+			saw_zswap = true;
+		}
+		if (!saw_zswap)
+			return -ENOENT;
+
+		for (i = 0; i < nr_pages; i++) {
+			tree = swap_zswap_tree(swp_entry(type, offset + i));
+			entry = xa_load(tree, offset + i);
+			if (!entry)
+				return -EAGAIN;
+
+			if (!zswap_decompress(entry, folio, i, false)) {
+				folio_unlock(folio);
+				return -EIO;
+			}
+		}
+
+		flush_dcache_folio(folio);
+		/*
+		 * Keep zswap entries until swap slots are freed. This is a clean
+		 * speculative fill; zswap remains the backing copy if reclaim
+		 * drops the large folio before PTEs are installed.
+		 */
+		folio_mark_uptodate(folio);
+		count_vm_events(ZSWPIN, nr_pages);
+		count_mthp_stat(folio_order(folio), MTHP_STAT_SWPIN);
+
+		if (same_objcg) {
+			if (first_objcg)
+				count_objcg_events(first_objcg, ZSWPIN, nr_pages);
+		} else {
+			for (i = 0; i < nr_pages; i++) {
+				tree = swap_zswap_tree(swp_entry(type, offset + i));
+				entry = xa_load(tree, offset + i);
+				if (WARN_ON_ONCE(!entry))
+					continue;
+				if (entry->objcg)
+					count_objcg_events(entry->objcg, ZSWPIN, 1);
+			}
+		}
+
 		folio_unlock(folio);
-		return -EINVAL;
+		return 0;
 	}
 
+	tree = swap_zswap_tree(swp);
 	entry = xa_load(tree, offset);
 	if (!entry)
 		return -ENOENT;
 
-	if (!zswap_decompress(entry, folio)) {
+	if (!zswap_decompress(entry, folio, 0, true)) {
 		folio_unlock(folio);
 		return -EIO;
 	}
-- 
2.34.1

[RFC PATCH v2 4/9] mm: admit large swapin by backend range in swapin_sync()

[fujunjie]

A large swapin can only read one folio when the whole range has compatible
backing. Mixed zswap/disk ranges must not reach large-folio IO, and zswap
range probes are only snapshots.

Filter the orders passed to swap_cache_alloc_folio() in swapin_sync().
Uniform zeromap ranges and all-disk ranges keep the existing large swapin
path. Fully zswap-backed ranges may be tried. Mixed zswap/disk ranges fall
back before allocation.

After a large swapcache folio is installed, recheck the zswap range and
drop the fresh folio if it became mixed. Also consume -EAGAIN from
swap_read_folio() the same way. Both cases retry order-0, where each slot
can resolve its current backend independently.

Signed-off-by: fujunjie <fujunjie1@qq.com>
---
 mm/memcontrol-v1.c |   8 ++-
 mm/memory.c        |  31 ++++++++-
 mm/swap_state.c    | 169 ++++++++++++++++++++++++++++++++++++++++++---
 3 files changed, 194 insertions(+), 14 deletions(-)

diff --git a/mm/memcontrol-v1.c b/mm/memcontrol-v1.c
index 765069211567..5b11b8055c66 100644
--- a/mm/memcontrol-v1.c
+++ b/mm/memcontrol-v1.c
@@ -682,8 +682,8 @@ void __memcg1_swapout(struct folio *folio, struct swap_cluster_info *ci)
  * memcg1_swapin - uncharge swap slot on swapin
  * @folio: folio being swapped in
  *
- * Call this function after successfully adding the charged
- * folio to swapcache.
+ * Call this after the charged folio has been added to swapcache and the caller
+ * is no longer going to drop it back to swapped-out state.
  *
  * Context: The folio has to be in swap cache and locked.
  */
@@ -721,7 +721,9 @@ void memcg1_swapin(struct folio *folio)
 	id = __swap_cgroup_clear(ci, swp_cluster_offset(folio->swap),
 				 nr_pages);
 	swap_cluster_unlock(ci);
-	mem_cgroup_uncharge_swap(id, nr_pages);
+
+	if (id)
+		mem_cgroup_uncharge_swap(id, nr_pages);
 }
 #endif
 
diff --git a/mm/memory.c b/mm/memory.c
index 5a365492a9a2..d73a19692dea 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -4538,6 +4538,24 @@ static inline bool should_try_to_free_swap(struct swap_info_struct *si,
 		folio_ref_count(folio) == (extra_refs + folio_nr_pages(folio));
 }
 
+static void memcg1_swapin_retry_folio(struct folio *folio,
+				      struct vm_fault *vmf)
+{
+	if (!folio_test_large(folio) || !folio_test_swapcache(folio))
+		return;
+
+	if (vmf->flags & FAULT_FLAG_RETRY_NOWAIT) {
+		if (!folio_trylock(folio))
+			return;
+	} else {
+		folio_lock(folio);
+	}
+
+	if (folio_test_large(folio) && folio_test_swapcache(folio))
+		memcg1_swapin(folio);
+	folio_unlock(folio);
+}
+
 static vm_fault_t pte_marker_clear(struct vm_fault *vmf)
 {
 	vmf->pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd,
@@ -4857,8 +4875,10 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
 
 	swapcache = folio;
 	ret |= folio_lock_or_retry(folio, vmf);
-	if (ret & VM_FAULT_RETRY)
+	if (ret & VM_FAULT_RETRY) {
+		memcg1_swapin_retry_folio(folio, vmf);
 		goto out_release;
+	}
 
 	page = folio_file_page(folio, swp_offset(entry));
 	/*
@@ -5067,6 +5087,8 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
 	if (unlikely(folio != swapcache)) {
 		folio_add_new_anon_rmap(folio, vma, address, RMAP_EXCLUSIVE);
 		folio_add_lru_vma(folio, vma);
+		if (folio_test_large(swapcache))
+			memcg1_swapin(swapcache);
 		folio_put_swap(swapcache, NULL);
 	} else if (!folio_test_anon(folio)) {
 		/*
@@ -5076,6 +5098,8 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
 		VM_WARN_ON_ONCE_FOLIO(folio_nr_pages(folio) != nr_pages, folio);
 		VM_WARN_ON_ONCE_FOLIO(folio_mapped(folio), folio);
 		folio_add_new_anon_rmap(folio, vma, address, rmap_flags);
+		if (folio_test_large(folio))
+			memcg1_swapin(folio);
 		folio_put_swap(folio, NULL);
 	} else {
 		VM_WARN_ON_ONCE(nr_pages != 1 && nr_pages != folio_nr_pages(folio));
@@ -5132,8 +5156,11 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
 	if (vmf->pte)
 		pte_unmap_unlock(vmf->pte, vmf->ptl);
 out_page:
-	if (folio_test_swapcache(folio))
+	if (folio_test_swapcache(folio)) {
+		if (folio_test_large(folio))
+			memcg1_swapin(folio);
 		folio_free_swap(folio);
+	}
 	folio_unlock(folio);
 out_release:
 	folio_put(folio);
diff --git a/mm/swap_state.c b/mm/swap_state.c
index d37097913b30..f03ad4832f16 100644
--- a/mm/swap_state.c
+++ b/mm/swap_state.c
@@ -21,6 +21,7 @@
 #include <linux/migrate.h>
 #include <linux/vmalloc.h>
 #include <linux/huge_mm.h>
+#include <linux/zswap.h>
 #include <linux/shmem_fs.h>
 #include "internal.h"
 #include "swap_table.h"
@@ -403,7 +404,8 @@ void __swap_cache_replace_folio(struct swap_cluster_info *ci,
 static struct folio *__swap_cache_alloc(struct swap_cluster_info *ci,
 					swp_entry_t targ_entry, gfp_t gfp,
 					unsigned int order, struct vm_fault *vmf,
-					struct mempolicy *mpol, pgoff_t ilx)
+					struct mempolicy *mpol, pgoff_t ilx,
+					bool defer_memcg1_swapin)
 {
 	int err;
 	swp_entry_t entry;
@@ -466,7 +468,8 @@ static struct folio *__swap_cache_alloc(struct swap_cluster_info *ci,
 	}
 
 	/* memsw uncharges swap when folio is added to swap cache */
-	memcg1_swapin(folio);
+	if (!defer_memcg1_swapin || !order)
+		memcg1_swapin(folio);
 	if (shadow)
 		workingset_refault(folio, shadow);
 
@@ -495,9 +498,12 @@ static struct folio *__swap_cache_alloc(struct swap_cluster_info *ci,
  * Return: Returns the folio if allocation succeeded and folio is in the swap
  * cache. Returns error code if failed due to race, OOM or invalid arguments.
  */
-struct folio *swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp,
-				     unsigned long orders, struct vm_fault *vmf,
-				     struct mempolicy *mpol, pgoff_t ilx)
+static struct folio *__swap_cache_alloc_folio(swp_entry_t targ_entry,
+					      gfp_t gfp, unsigned long orders,
+					      struct vm_fault *vmf,
+					      struct mempolicy *mpol,
+					      pgoff_t ilx,
+					      bool defer_memcg1_swapin)
 {
 	int order, err;
 	struct folio *ret;
@@ -512,7 +518,8 @@ struct folio *swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp,
 
 	do {
 		ret = __swap_cache_alloc(ci, targ_entry, gfp, order,
-					 vmf, mpol, ilx);
+					 vmf, mpol, ilx,
+					 defer_memcg1_swapin);
 		if (!IS_ERR(ret))
 			break;
 		err = PTR_ERR(ret);
@@ -525,6 +532,124 @@ struct folio *swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp,
 	return ret;
 }
 
+struct folio *swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp,
+				     unsigned long orders, struct vm_fault *vmf,
+				     struct mempolicy *mpol, pgoff_t ilx)
+{
+	return __swap_cache_alloc_folio(targ_entry, gfp, orders, vmf,
+					mpol, ilx, false);
+}
+
+static struct folio *swap_cache_alloc_speculative_folio(swp_entry_t targ_entry,
+							gfp_t gfp,
+							unsigned long orders,
+							struct vm_fault *vmf,
+							struct mempolicy *mpol,
+							pgoff_t ilx)
+{
+	/*
+	 * Speculative large swapin may drop this fresh swapcache folio and
+	 * retry order-0 after backend or page-table revalidation. Keep the
+	 * cgroup v1 memsw swap owner until the caller commits the folio.
+	 */
+	return __swap_cache_alloc_folio(targ_entry, gfp, orders, vmf,
+					mpol, ilx, true);
+}
+
+static bool swapin_zeromap_same(swp_entry_t entry, unsigned int nr_pages)
+{
+	unsigned int ci_start = swp_cluster_offset(entry);
+	struct swap_cluster_info *ci = __swap_entry_to_cluster(entry);
+	bool is_zero;
+	unsigned int i;
+
+	if (ci_start + nr_pages > SWAPFILE_CLUSTER) {
+		VM_WARN_ON_ONCE(1);
+		return false;
+	}
+
+	rcu_read_lock();
+	if (!rcu_dereference(ci->table)) {
+		rcu_read_unlock();
+		return true;
+	}
+
+	is_zero = __swap_table_test_zero(ci, ci_start);
+	for (i = 1; i < nr_pages; i++) {
+		if (is_zero != __swap_table_test_zero(ci, ci_start + i)) {
+			rcu_read_unlock();
+			return false;
+		}
+	}
+	rcu_read_unlock();
+
+	return true;
+}
+
+static unsigned long swapin_admit_orders(swp_entry_t entry,
+					 unsigned long orders)
+{
+	unsigned long candidates = orders & ~BIT(0);
+	unsigned long admitted = orders & BIT(0);
+	int order;
+
+	if (!candidates)
+		return orders;
+
+	while (candidates) {
+		enum zswap_range_state state;
+		unsigned int nr_pages;
+		swp_entry_t range_entry;
+		bool admit = false;
+
+		order = fls_long(candidates) - 1;
+		if (order > MAX_PAGE_ORDER) {
+			candidates &= ~BIT(order);
+			continue;
+		}
+
+		nr_pages = 1U << order;
+		range_entry = swp_entry(swp_type(entry),
+					round_down(swp_offset(entry), nr_pages));
+		if (!swapin_zeromap_same(range_entry, nr_pages))
+			goto next;
+
+		state = zswap_probe_range(range_entry, nr_pages);
+		switch (state) {
+		case ZSWAP_RANGE_MIXED:
+			break;
+		case ZSWAP_RANGE_ALL_ZSWAP:
+		case ZSWAP_RANGE_NEVER_ENABLED:
+		case ZSWAP_RANGE_NO_ZSWAP:
+			admit = true;
+			break;
+		}
+
+next:
+		if (admit)
+			admitted |= BIT(order);
+		else
+			count_mthp_stat(order, MTHP_STAT_SWPIN_FALLBACK);
+		candidates &= ~BIT(order);
+	}
+
+	return admitted ? admitted : BIT(0);
+}
+
+static bool zswap_needs_order0_retry(struct folio *folio)
+{
+	if (!folio_test_large(folio))
+		return false;
+
+	/*
+	 * Admission sees only an advisory zswap snapshot. Recheck after the
+	 * large swapcache folio is installed; if the range became mixed, drop
+	 * the fresh folio before IO and let order-0 handle each slot.
+	 */
+	return zswap_probe_range(folio->swap, folio_nr_pages(folio)) ==
+	       ZSWAP_RANGE_MIXED;
+}
+
 /*
  * If we are the only user, then try to free up the swap cache.
  *
@@ -634,7 +759,8 @@ static struct folio *swap_cache_read_folio(swp_entry_t entry, gfp_t gfp,
 		folio = swap_cache_get_folio(entry);
 		if (folio)
 			return folio;
-		folio = swap_cache_alloc_folio(entry, gfp, BIT(0), NULL, mpol, ilx);
+		folio = swap_cache_alloc_folio(entry, gfp, BIT(0), NULL,
+					       mpol, ilx);
 	} while (PTR_ERR(folio) == -EEXIST);
 
 	if (IS_ERR_OR_NULL(folio))
@@ -677,18 +803,43 @@ struct folio *swapin_sync(swp_entry_t entry, gfp_t gfp, unsigned long orders,
 	struct folio *folio;
 	int ret;
 
+	orders = swapin_admit_orders(entry, orders);
+again:
 	do {
 		folio = swap_cache_get_folio(entry);
 		if (folio)
 			return folio;
-		folio = swap_cache_alloc_folio(entry, gfp, orders, vmf, mpol, ilx);
+		folio = swap_cache_alloc_speculative_folio(entry, gfp, orders,
+							   vmf, mpol, ilx);
 	} while (PTR_ERR(folio) == -EEXIST);
 
 	if (IS_ERR(folio))
 		return folio;
 
+	if (zswap_needs_order0_retry(folio)) {
+		count_mthp_stat(folio_order(folio), MTHP_STAT_SWPIN_FALLBACK);
+		/*
+		 * The folio is newly allocated, locked, clean and not uptodate;
+		 * no data has been read into it. Removing it only restores the
+		 * swap table entries so order-0 swapin can resolve a backend
+		 * race without attempting speculative large-folio zswapin.
+		 */
+		swap_cache_del_folio(folio);
+		folio_unlock(folio);
+		folio_put(folio);
+		orders = BIT(0);
+		goto again;
+	}
+
 	ret = swap_read_folio(folio, NULL);
-	VM_WARN_ON_ONCE(ret == -EAGAIN);
+	if (ret == -EAGAIN) {
+		count_mthp_stat(folio_order(folio), MTHP_STAT_SWPIN_FALLBACK);
+		swap_cache_del_folio(folio);
+		folio_unlock(folio);
+		folio_put(folio);
+		orders = BIT(0);
+		goto again;
+	}
 	return folio;
 }
 
-- 
2.34.1

[RFC PATCH v2 5/9] mm: add common locality admission for zswap large swapin

[fujunjie]

Fully zswap-backed ranges are safe to load as a large folio only when
the caller has a reason to expect the neighbouring slots to be useful.
Otherwise a sparse refault can turn one 4K demand fault into a 64K
decompression and swapcache fill.

Add a common admission gate for zswap-backed large swapin. The common
layer keeps backend checks, the 64K cap, recent-refault rejection, and
zswap reclaim-pressure rejection. It consumes a caller-provided locality
order mask instead of looking at anon or shmem state directly.

Callers pass no locality evidence for now, so this patch only installs
the common policy hook. Later patches add anon and shmem producers.

Signed-off-by: fujunjie <fujunjie1@qq.com>
---
 mm/memory.c     |   2 +-
 mm/shmem.c      |   2 +-
 mm/swap.h       |   8 ++--
 mm/swap_state.c | 118 ++++++++++++++++++++++++++++++++++++++++++++----
 4 files changed, 117 insertions(+), 13 deletions(-)

diff --git a/mm/memory.c b/mm/memory.c
index d73a19692dea..92a82008d583 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -4849,7 +4849,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
 		if (data_race(si->flags & SWP_SYNCHRONOUS_IO))
 			folio = swapin_sync(entry, GFP_HIGHUSER_MOVABLE,
 					    thp_swapin_suitable_orders(vmf) | BIT(0),
-					    vmf, NULL, 0);
+					    0, vmf, NULL, 0);
 		else
 			folio = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, vmf);
 
diff --git a/mm/shmem.c b/mm/shmem.c
index 56c23a7b15c7..fa99b48ed62b 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -2031,7 +2031,7 @@ static struct folio *shmem_swap_alloc_folio(struct inode *inode,
 
 again:
 	mpol = shmem_get_pgoff_policy(info, index, order, &ilx);
-	folio = swapin_sync(entry, gfp, BIT(order), vmf, mpol, ilx);
+	folio = swapin_sync(entry, gfp, BIT(order), 0, vmf, mpol, ilx);
 	mpol_cond_put(mpol);
 
 	if (!IS_ERR(folio))
diff --git a/mm/swap.h b/mm/swap.h
index ea7e1f3c4410..dd35a310d06d 100644
--- a/mm/swap.h
+++ b/mm/swap.h
@@ -323,9 +323,10 @@ struct folio *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask,
 struct folio *swap_cluster_readahead(swp_entry_t entry, gfp_t flag,
 		struct mempolicy *mpol, pgoff_t ilx);
 struct folio *swapin_readahead(swp_entry_t entry, gfp_t flag,
-		struct vm_fault *vmf);
+			struct vm_fault *vmf);
 struct folio *swapin_sync(swp_entry_t entry, gfp_t flag, unsigned long orders,
-			   struct vm_fault *vmf, struct mempolicy *mpol, pgoff_t ilx);
+			  unsigned long locality_orders, struct vm_fault *vmf,
+			  struct mempolicy *mpol, pgoff_t ilx);
 void swap_update_readahead(struct folio *folio, struct vm_area_struct *vma,
 			   unsigned long addr);
 
@@ -418,7 +419,8 @@ static inline struct folio *swapin_readahead(swp_entry_t swp, gfp_t gfp_mask,
 
 static inline struct folio *swapin_sync(
 	swp_entry_t entry, gfp_t flag, unsigned long orders,
-	struct vm_fault *vmf, struct mempolicy *mpol, pgoff_t ilx)
+	unsigned long locality_orders, struct vm_fault *vmf,
+	struct mempolicy *mpol, pgoff_t ilx)
 {
 	return NULL;
 }
diff --git a/mm/swap_state.c b/mm/swap_state.c
index f03ad4832f16..5a4ca289009a 100644
--- a/mm/swap_state.c
+++ b/mm/swap_state.c
@@ -21,6 +21,7 @@
 #include <linux/migrate.h>
 #include <linux/vmalloc.h>
 #include <linux/huge_mm.h>
+#include <linux/sizes.h>
 #include <linux/zswap.h>
 #include <linux/shmem_fs.h>
 #include "internal.h"
@@ -556,6 +557,24 @@ static struct folio *swap_cache_alloc_speculative_folio(swp_entry_t targ_entry,
 					mpol, ilx, true);
 }
 
+/*
+ * Initial conservative cap for speculative zswap large swapin. Locality
+ * evidence is supplied by the caller or by generic VMA hints; the common
+ * swapin layer keeps backend safety and pressure decisions here.
+ */
+#define SWAPIN_ZSWAP_MAX_SIZE			SZ_64K
+#if PAGE_SIZE < SWAPIN_ZSWAP_MAX_SIZE
+#define SWAPIN_ZSWAP_MAX_ORDER			\
+	ilog2(SWAPIN_ZSWAP_MAX_SIZE / PAGE_SIZE)
+#else
+#define SWAPIN_ZSWAP_MAX_ORDER			0
+#endif
+
+struct zswap_admit_ctx {
+	bool pressure_checked;
+	bool reclaim_pressure;
+};
+
 static bool swapin_zeromap_same(swp_entry_t entry, unsigned int nr_pages)
 {
 	unsigned int ci_start = swp_cluster_offset(entry);
@@ -586,11 +605,84 @@ static bool swapin_zeromap_same(swp_entry_t entry, unsigned int nr_pages)
 	return true;
 }
 
+static bool swapin_zswap_locality(struct vm_fault *vmf, unsigned int order,
+				  unsigned long locality_orders)
+{
+	struct vm_area_struct *vma = vmf ? vmf->vma : NULL;
+
+	if (!order || order > MAX_PAGE_ORDER)
+		return false;
+
+	if (vma && (vma->vm_flags & VM_RAND_READ))
+		return false;
+
+	return locality_orders & BIT(order);
+}
+
+static bool swapin_zswap_refaulted(swp_entry_t entry, unsigned int nr_pages)
+{
+	unsigned int type = swp_type(entry);
+	pgoff_t offset = swp_offset(entry);
+	unsigned int i;
+
+	for (i = 0; i < nr_pages; i++) {
+		bool workingset;
+		void *shadow;
+
+		shadow = swap_cache_get_shadow(swp_entry(type, offset + i));
+		if (!shadow)
+			continue;
+		if (workingset_test_recent(shadow, false, &workingset, false) &&
+		    workingset)
+			return true;
+	}
+
+	return false;
+}
+
+static bool swapin_zswap_admit(swp_entry_t entry,
+			       unsigned int order, unsigned int nr_pages,
+			       struct vm_fault *vmf,
+			       unsigned long locality_orders,
+			       struct zswap_admit_ctx *ctx)
+{
+	if (order > SWAPIN_ZSWAP_MAX_ORDER)
+		return false;
+
+	/*
+	 * Treat zswap-backed large swapin as speculative. The common layer
+	 * consumes caller-provided locality orders, but does not inspect
+	 * anon-specific PTE state or shmem-specific mapping state directly.
+	 */
+	if (!swapin_zswap_locality(vmf, order, locality_orders))
+		return false;
+
+	/*
+	 * A recent workingset refault shadow in the target range means reclaim
+	 * already saw churn there. Keep the refault path narrow instead of
+	 * speculatively decompressing neighbouring slots.
+	 */
+	if (swapin_zswap_refaulted(entry, nr_pages))
+		return false;
+
+	if (!ctx->pressure_checked) {
+		ctx->reclaim_pressure = zswap_pool_reclaim_pressure();
+		ctx->pressure_checked = true;
+	}
+	if (ctx->reclaim_pressure)
+		return false;
+
+	return true;
+}
+
 static unsigned long swapin_admit_orders(swp_entry_t entry,
-					 unsigned long orders)
+					 unsigned long orders,
+					 struct vm_fault *vmf,
+					 unsigned long locality_orders)
 {
 	unsigned long candidates = orders & ~BIT(0);
 	unsigned long admitted = orders & BIT(0);
+	struct zswap_admit_ctx zswap_ctx = {};
 	int order;
 
 	if (!candidates)
@@ -616,9 +708,14 @@ static unsigned long swapin_admit_orders(swp_entry_t entry,
 
 		state = zswap_probe_range(range_entry, nr_pages);
 		switch (state) {
+		case ZSWAP_RANGE_ALL_ZSWAP:
+			admit = swapin_zswap_admit(range_entry, order,
+						   nr_pages, vmf,
+						   locality_orders,
+						   &zswap_ctx);
+			break;
 		case ZSWAP_RANGE_MIXED:
 			break;
-		case ZSWAP_RANGE_ALL_ZSWAP:
 		case ZSWAP_RANGE_NEVER_ENABLED:
 		case ZSWAP_RANGE_NO_ZSWAP:
 			admit = true;
@@ -769,8 +866,8 @@ static struct folio *swap_cache_read_folio(swp_entry_t entry, gfp_t gfp,
 	ret = swap_read_folio(folio, plug);
 	/*
 	 * Swap readahead allocates order-0 folios. -EAGAIN is reserved for
-	 * retryable large zswap backend races and must be handled by the
-	 * synchronous common swapin path.
+	 * retryable large zswap backend races and should never escape to this
+	 * order-0 path.
 	 */
 	VM_WARN_ON_ONCE(ret == -EAGAIN);
 	if (readahead) {
@@ -786,6 +883,7 @@ static struct folio *swap_cache_read_folio(swp_entry_t entry, gfp_t gfp,
  * @entry: swap entry indicating the target slot
  * @gfp: memory allocation flags
  * @orders: allocation orders
+ * @locality_orders: orders with caller-provided locality evidence
  * @vmf: fault information
  * @mpol: NUMA memory allocation policy to be applied
  * @ilx: NUMA interleave index, for use only when MPOL_INTERLEAVE
@@ -794,16 +892,20 @@ static struct folio *swap_cache_read_folio(swp_entry_t entry, gfp_t gfp,
  * existing folio in the swap cache for @entry. This initiates the IO, too,
  * if needed. @entry is rounded down if @orders allow large allocation.
  *
- * Context: Caller must ensure @entry is valid and pin the swap device with refcount.
+ * Context: Caller must ensure @entry is valid and pin the swap device with
+ * refcount.
  * Return: Returns the folio on success, error code if failed.
  */
-struct folio *swapin_sync(swp_entry_t entry, gfp_t gfp, unsigned long orders,
-			   struct vm_fault *vmf, struct mempolicy *mpol, pgoff_t ilx)
+struct folio *swapin_sync(swp_entry_t entry, gfp_t gfp,
+			  unsigned long orders,
+			  unsigned long locality_orders,
+			  struct vm_fault *vmf, struct mempolicy *mpol,
+			  pgoff_t ilx)
 {
 	struct folio *folio;
 	int ret;
 
-	orders = swapin_admit_orders(entry, orders);
+	orders = swapin_admit_orders(entry, orders, vmf, locality_orders);
 again:
 	do {
 		folio = swap_cache_get_folio(entry);
-- 
2.34.1

[RFC PATCH v2 6/9] mm: provide anon locality evidence for zswap large swapin

[fujunjie]

The common zswap large-swapin policy needs locality evidence from
callers before it can admit a large folio. For anonymous faults, provide
that evidence from existing VMA hints and from the PTE young state left
by earlier zswap-backed large swapins.

Keep non-faulting PTEs old when mapping a speculative all-zswap large
folio. A later fault can then require a dense young previous range before
admitting another large swapin without adding VMA state.

This also removes the old zswap-enabled guard from the THP swapin
candidate scan. The common swapin path now classifies the backend range
and falls back to order-0 for mixed zswap/disk ranges or races.

Signed-off-by: fujunjie <fujunjie1@qq.com>
---
 mm/memory.c     | 234 +++++++++++++++++++++++++++++++++++++++++++-----
 mm/swap.h       |   6 ++
 mm/swap_state.c |  15 ++++
 3 files changed, 235 insertions(+), 20 deletions(-)

diff --git a/mm/memory.c b/mm/memory.c
index 92a82008d583..7bbb89632000 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -4556,6 +4556,35 @@ static void memcg1_swapin_retry_folio(struct folio *folio,
 	folio_unlock(folio);
 }
 
+static void set_swapin_ptes(struct vm_area_struct *vma,
+			    unsigned long address, pte_t *ptep, pte_t pte,
+			    unsigned int nr_pages, unsigned int fault_pte_idx,
+			    bool fault_only_young)
+{
+	struct mm_struct *mm = vma->vm_mm;
+	pte_t old_pte;
+
+	if (!fault_only_young || nr_pages == 1) {
+		set_ptes(mm, address, ptep, pte, nr_pages);
+		return;
+	}
+
+	old_pte = pte_mkold(pte);
+	if (fault_pte_idx)
+		set_ptes(mm, address, ptep, old_pte, fault_pte_idx);
+
+	set_pte_at(mm, address + fault_pte_idx * PAGE_SIZE,
+		   ptep + fault_pte_idx,
+		   pte_mkyoung(pte_advance_pfn(pte, fault_pte_idx)));
+
+	fault_pte_idx++;
+	if (fault_pte_idx < nr_pages)
+		set_ptes(mm, address + fault_pte_idx * PAGE_SIZE,
+			 ptep + fault_pte_idx,
+			 pte_advance_pfn(old_pte, fault_pte_idx),
+			 nr_pages - fault_pte_idx);
+}
+
 static vm_fault_t pte_marker_clear(struct vm_fault *vmf)
 {
 	vmf->pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd,
@@ -4628,6 +4657,157 @@ static vm_fault_t handle_pte_marker(struct vm_fault *vmf)
 }
 
 #ifdef CONFIG_TRANSPARENT_HUGEPAGE
+#define SWAPIN_ANON_YOUNG_MIN_PERCENT		75
+#define SWAPIN_ANON_MAX_FAULT_SKIP_SHIFT	2
+
+static bool swapin_anon_prev_young_dense(struct vm_fault *vmf,
+					 unsigned int order)
+{
+	struct vm_area_struct *vma;
+	unsigned int nr_pages;
+	unsigned int threshold;
+	unsigned long size;
+	unsigned long base, prev, addr;
+	struct folio *first = NULL;
+	unsigned int present = 0;
+	unsigned int young = 0;
+	pmd_t *pmd;
+	pmd_t pmdval;
+	spinlock_t *ptl; /* protects the previous PTE range */
+	pte_t *ptep;
+	unsigned int i;
+
+	if (!IS_ENABLED(CONFIG_MMU) || !arch_has_hw_pte_young() || !vmf ||
+	    !vmf->vma || !vmf->pmd || !order || order > MAX_PAGE_ORDER)
+		return false;
+
+	nr_pages = 1U << order;
+	threshold = DIV_ROUND_UP(nr_pages *
+				 SWAPIN_ANON_YOUNG_MIN_PERCENT, 100);
+	size = PAGE_SIZE << order;
+
+	vma = vmf->vma;
+	base = ALIGN_DOWN(vmf->address, size);
+	if (base < size)
+		return false;
+
+	prev = base - size;
+	if (prev < vma->vm_start || prev + size > vma->vm_end)
+		return false;
+
+	pmd = vmf->pmd;
+	if ((prev & PMD_MASK) != (base & PMD_MASK)) {
+		pmd = mm_find_pmd(vma->vm_mm, prev);
+		if (!pmd)
+			return false;
+	}
+
+	pmdval = pmdp_get_lockless(pmd);
+	if (!pmd_present(pmdval) || pmd_leaf(pmdval))
+		return false;
+
+	ptep = pte_offset_map_lock(vma->vm_mm, pmd, prev, &ptl);
+	if (!ptep)
+		return false;
+
+	for (i = 0, addr = prev; i < nr_pages; i++, addr += PAGE_SIZE) {
+		struct folio *folio;
+		pte_t pte = ptep_get(ptep + i);
+
+		if (!pte_present(pte))
+			break;
+
+		folio = vm_normal_folio(vma, addr, pte);
+		if (!folio || folio_order(folio) != order)
+			break;
+		if (!first)
+			first = folio;
+		else if (folio != first)
+			break;
+
+		present++;
+		if (pte_young(pte))
+			young++;
+	}
+
+	pte_unmap_unlock(ptep, ptl);
+	if (present != nr_pages)
+		return false;
+
+	return young >= threshold;
+}
+
+static bool swapin_anon_accessed_neighbour(struct vm_fault *vmf,
+					   unsigned int order)
+{
+	unsigned long size;
+	unsigned long base;
+	unsigned long fault_idx;
+	unsigned long max_skip;
+
+	if (!vmf || !vmf->vma || !order || order > MAX_PAGE_ORDER)
+		return false;
+
+	size = PAGE_SIZE << order;
+	base = ALIGN_DOWN(vmf->address, size);
+
+	/*
+	 * Without a sequential hint, require prior young-density evidence and
+	 * only allow faults near the start of the candidate range.
+	 */
+	fault_idx = (vmf->address - base) >> PAGE_SHIFT;
+	max_skip = (1UL << order) >> SWAPIN_ANON_MAX_FAULT_SKIP_SHIFT;
+	if (fault_idx > max_skip)
+		return false;
+
+	return swapin_anon_prev_young_dense(vmf, order);
+}
+
+static bool swapin_anon_fault_starts_range(struct vm_fault *vmf,
+					   unsigned int order)
+{
+	struct vm_area_struct *vma;
+	unsigned long size;
+	unsigned long base;
+	unsigned long first;
+
+	if (!vmf || !vmf->vma || !order || order > MAX_PAGE_ORDER)
+		return false;
+
+	vma = vmf->vma;
+	size = PAGE_SIZE << order;
+	base = ALIGN_DOWN(vmf->address, size);
+	first = ALIGN(vma->vm_start, size);
+
+	return base == first && vmf->address == base &&
+	       base + size <= vma->vm_end;
+}
+
+static unsigned long swapin_anon_locality_orders(struct vm_fault *vmf,
+						 unsigned long orders)
+{
+	struct vm_area_struct *vma = vmf ? vmf->vma : NULL;
+	unsigned long locality_orders = 0;
+	unsigned long candidates = orders & ~BIT(0);
+	int order;
+
+	if (vma && (vma->vm_flags & VM_RAND_READ))
+		return 0;
+
+	if (vma && (vma->vm_flags & VM_SEQ_READ))
+		return candidates;
+
+	while (candidates) {
+		order = fls_long(candidates) - 1;
+		if (swapin_anon_fault_starts_range(vmf, order) ||
+		    swapin_anon_accessed_neighbour(vmf, order))
+			locality_orders |= BIT(order);
+		candidates &= ~BIT(order);
+	}
+
+	return locality_orders;
+}
+
 /*
  * Check if the PTEs within a range are contiguous swap entries.
  */
@@ -4644,9 +4824,9 @@ static bool can_swapin_thp(struct vm_fault *vmf, pte_t *ptep, int nr_pages)
 	if (!pte_same(pte, pte_move_swp_offset(vmf->orig_pte, -idx)))
 		return false;
 	/*
-	 * swap_read_folio() can't handle the case a large folio is hybridly
-	 * from different backends. And they are likely corner cases. Similar
-	 * things might be added once zswap support large folios.
+	 * swap_read_folio() can't do mixed-backend large folio IO. The common
+	 * synchronous swapin path will recheck backend state and fall back to
+	 * order-0 if a zswap/disk race makes the range mixed.
 	 */
 	if (swap_pte_batch(ptep, nr_pages, pte) != nr_pages)
 		return false;
@@ -4693,14 +4873,6 @@ static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)
 	if (unlikely(userfaultfd_armed(vma)))
 		return 0;
 
-	/*
-	 * A large swapped out folio could be partially or fully in zswap. We
-	 * lack handling for such cases, so fallback to swapping in order-0
-	 * folio.
-	 */
-	if (!zswap_never_enabled())
-		return 0;
-
 	entry = softleaf_from_pte(vmf->orig_pte);
 	/*
 	 * Get a list of all the (large) orders below PMD_ORDER that are enabled
@@ -4708,10 +4880,13 @@ static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)
 	 */
 	orders = thp_vma_allowable_orders(vma, vma->vm_flags, TVA_PAGEFAULT,
 					  BIT(PMD_ORDER) - 1);
+	if (!orders)
+		return 0;
 	orders = thp_vma_suitable_orders(vma, vmf->address, orders);
+	if (!orders)
+		return 0;
 	orders = thp_swap_suitable_orders(swp_offset(entry),
 					  vmf->address, orders);
-
 	if (!orders)
 		return 0;
 
@@ -4741,6 +4916,12 @@ static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)
 {
 	return 0;
 }
+
+static unsigned long swapin_anon_locality_orders(struct vm_fault *vmf,
+						 unsigned long orders)
+{
+	return 0;
+}
 #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
 
 /* Sanity check that a folio is fully exclusive */
@@ -4777,6 +4958,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
 	unsigned long page_idx;
 	unsigned long address;
 	pte_t *ptep;
+	bool fault_only_young = false;
 
 	if (!pte_unmap_same(vmf))
 		goto out;
@@ -4845,13 +5027,22 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
 	if (folio)
 		swap_update_readahead(folio, vma, vmf->address);
 	if (!folio) {
-		/* Swapin bypasses readahead for SWP_SYNCHRONOUS_IO devices */
-		if (data_race(si->flags & SWP_SYNCHRONOUS_IO))
+		/*
+		 * Swapin bypasses readahead for SWP_SYNCHRONOUS_IO devices.
+		 * The swap device is pinned while checking the flag, matching
+		 * the existing fault path.
+		 */
+		if (data_race(si->flags & SWP_SYNCHRONOUS_IO)) {
+			unsigned long swapin_orders = thp_swapin_suitable_orders(vmf);
+			unsigned long locality_orders =
+				swapin_anon_locality_orders(vmf, swapin_orders);
+
 			folio = swapin_sync(entry, GFP_HIGHUSER_MOVABLE,
-					    thp_swapin_suitable_orders(vmf) | BIT(0),
-					    0, vmf, NULL, 0);
-		else
+					    swapin_orders | BIT(0),
+					    locality_orders, vmf, NULL, 0);
+		} else {
 			folio = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, vmf);
+		}
 
 		if (IS_ERR_OR_NULL(folio)) {
 			/*
@@ -5110,9 +5301,12 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
 
 	VM_BUG_ON(!folio_test_anon(folio) ||
 			(pte_write(pte) && !PageAnonExclusive(page)));
-	set_ptes(vma->vm_mm, address, ptep, pte, nr_pages);
-	arch_do_swap_page_nr(vma->vm_mm, vma, address,
-			pte, pte, nr_pages);
+	if (folio == swapcache && nr_pages == folio_nr_pages(folio) &&
+	    arch_has_hw_pte_young())
+		fault_only_young = swapin_fault_only_young(folio);
+	set_swapin_ptes(vma, address, ptep, pte, nr_pages, page_idx,
+			fault_only_young);
+	arch_do_swap_page_nr(vma->vm_mm, vma, address, pte, pte, nr_pages);
 
 	/*
 	 * Remove the swap entry and conditionally try to free up the swapcache.
diff --git a/mm/swap.h b/mm/swap.h
index dd35a310d06d..5d1c81ab49b9 100644
--- a/mm/swap.h
+++ b/mm/swap.h
@@ -327,6 +327,7 @@ struct folio *swapin_readahead(swp_entry_t entry, gfp_t flag,
 struct folio *swapin_sync(swp_entry_t entry, gfp_t flag, unsigned long orders,
 			  unsigned long locality_orders, struct vm_fault *vmf,
 			  struct mempolicy *mpol, pgoff_t ilx);
+bool swapin_fault_only_young(struct folio *folio);
 void swap_update_readahead(struct folio *folio, struct vm_area_struct *vma,
 			   unsigned long addr);
 
@@ -430,6 +431,11 @@ static inline void swap_update_readahead(struct folio *folio,
 {
 }
 
+static inline bool swapin_fault_only_young(struct folio *folio)
+{
+	return false;
+}
+
 static inline int swap_writeout(struct folio *folio,
 		struct swap_iocb **swap_plug)
 {
diff --git a/mm/swap_state.c b/mm/swap_state.c
index 5a4ca289009a..80dff6a1ee65 100644
--- a/mm/swap_state.c
+++ b/mm/swap_state.c
@@ -747,6 +747,21 @@ static bool zswap_needs_order0_retry(struct folio *folio)
 	       ZSWAP_RANGE_MIXED;
 }
 
+/*
+ * A speculative large swapin may install PTEs for pages that did not fault.
+ * Keep those non-faulting PTEs old so a later anon fault can report
+ * PTE-young density as caller-provided locality evidence without storing
+ * state in the VMA.
+ */
+bool swapin_fault_only_young(struct folio *folio)
+{
+	if (!folio_test_large(folio) || !folio_test_swapcache(folio))
+		return false;
+
+	return zswap_probe_range(folio->swap, folio_nr_pages(folio)) ==
+	       ZSWAP_RANGE_ALL_ZSWAP;
+}
+
 /*
  * If we are the only user, then try to free up the swap cache.
  *
-- 
2.34.1

[RFC PATCH v2 7/9] mm/shmem: provide VMA-hint locality for zswap large swapin

[fujunjie]

Let the shmem swap fault path pass locality evidence into the common
zswap large-swapin policy. Shmem does not have anon PTE-young density
evidence, so this first step only treats explicit VM_SEQ_READ as
positive evidence and VM_RAND_READ as a veto.

The non-fault shmem readahead path remains unchanged. This keeps large
zswap swapin limited to synchronous shmem faults where the caller
supplies a VMA and the common policy can still fall back to order-0.

Signed-off-by: fujunjie <fujunjie1@qq.com>
---
 mm/shmem.c | 42 +++++++++++++++++++++++++++++++++++++-----
 1 file changed, 37 insertions(+), 5 deletions(-)

diff --git a/mm/shmem.c b/mm/shmem.c
index fa99b48ed62b..a5ac35ac85fb 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -30,6 +30,7 @@
 #include <linux/fileattr.h>
 #include <linux/filelock.h>
 #include <linux/mm.h>
+#include <linux/memcontrol.h>
 #include <linux/random.h>
 #include <linux/sched/signal.h>
 #include <linux/export.h>
@@ -1791,6 +1792,29 @@ static struct folio *shmem_swapin_cluster(swp_entry_t swap, gfp_t gfp,
 	return folio;
 }
 
+static unsigned long shmem_swapin_locality_orders(struct vm_fault *vmf,
+						  unsigned long orders)
+{
+	struct vm_area_struct *vma = vmf ? vmf->vma : NULL;
+	unsigned long candidates = orders & ~BIT(0);
+
+	/*
+	 * Shmem does not have anon-style PTE young density evidence. Start with
+	 * explicit VMA access hints; future shmem/page-cache readahead evidence
+	 * can be folded into this producer without changing common swapin policy.
+	 */
+	if (!vma)
+		return 0;
+
+	if (vma->vm_flags & VM_RAND_READ)
+		return 0;
+
+	if (vma->vm_flags & VM_SEQ_READ)
+		return candidates;
+
+	return 0;
+}
+
 #ifdef CONFIG_TRANSPARENT_HUGEPAGE
 bool shmem_hpage_pmd_enabled(void)
 {
@@ -2020,18 +2044,22 @@ static struct folio *shmem_swap_alloc_folio(struct inode *inode,
 		struct vm_fault *vmf, pgoff_t index,
 		swp_entry_t entry, int order, gfp_t gfp)
 {
+	unsigned long locality_orders;
+	unsigned long orders;
 	pgoff_t ilx;
 	struct folio *folio;
 	struct mempolicy *mpol;
 	struct shmem_inode_info *info = SHMEM_I(inode);
 
-	if ((vmf && unlikely(userfaultfd_armed(vmf->vma))) ||
-	     !zswap_never_enabled())
+	if (vmf && unlikely(userfaultfd_armed(vmf->vma)))
 		order = 0;
 
 again:
+	orders = BIT(order);
+	locality_orders = shmem_swapin_locality_orders(vmf, orders);
 	mpol = shmem_get_pgoff_policy(info, index, order, &ilx);
-	folio = swapin_sync(entry, gfp, BIT(order), 0, vmf, mpol, ilx);
+	folio = swapin_sync(entry, gfp, orders, locality_orders, vmf, mpol,
+			    ilx);
 	mpol_cond_put(mpol);
 
 	if (!IS_ERR(folio))
@@ -2339,7 +2367,7 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index,
 	if (!folio_matches_swap_entry(folio, swap) ||
 	    shmem_confirm_swap(mapping, index, swap) < 0) {
 		error = -EEXIST;
-		goto unlock;
+		goto failed_swapcache;
 	}
 	if (!folio_test_uptodate(folio)) {
 		error = -EIO;
@@ -2369,6 +2397,8 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index,
 	if (sgp == SGP_WRITE)
 		folio_mark_accessed(folio);
 
+	if (folio_test_large(folio))
+		memcg1_swapin(folio);
 	folio_put_swap(folio, NULL);
 	swap_cache_del_folio(folio);
 	folio_mark_dirty(folio);
@@ -2379,9 +2409,11 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index,
 failed:
 	if (shmem_confirm_swap(mapping, index, swap) < 0)
 		error = -EEXIST;
+failed_swapcache:
+	if (folio && folio_test_large(folio) && folio_test_swapcache(folio))
+		memcg1_swapin(folio);
 	if (error == -EIO)
 		shmem_set_folio_swapin_error(inode, index, folio, swap);
-unlock:
 	if (folio)
 		folio_unlock(folio);
 failed_nolock:
-- 
2.34.1

[RFC PATCH v2 8/9] mm: try all-zswap large swapin within swap readahead windows

[fujunjie]

The non-synchronous swap fault path already computes either a VMA-based
or cluster-based readahead window. Use that existing window as locality
evidence for zswap-backed large swapin instead of mixing it with the
synchronous anon/shmem evidence.

The path first prepares the normal readahead window. If the faulting
aligned range is fully covered by that window and is still all-zswap, it
may be loaded as one large folio. If the large attempt fails or a backend
race is detected, the precomputed order-0 readahead window is used
without updating readahead state again.

Mixed zswap/disk ranges remain order-0 only. Disk-backed large swapin is
not added by this change.

Signed-off-by: fujunjie <fujunjie1@qq.com>
---
 mm/memory.c     |   6 +-
 mm/swap.h       |   4 +-
 mm/swap_state.c | 434 +++++++++++++++++++++++++++++++++++++++---------
 mm/swapfile.c   |   2 +-
 4 files changed, 360 insertions(+), 86 deletions(-)

diff --git a/mm/memory.c b/mm/memory.c
index 7bbb89632000..451375090d83 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -5027,13 +5027,14 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
 	if (folio)
 		swap_update_readahead(folio, vma, vmf->address);
 	if (!folio) {
+		unsigned long swapin_orders = thp_swapin_suitable_orders(vmf);
+
 		/*
 		 * Swapin bypasses readahead for SWP_SYNCHRONOUS_IO devices.
 		 * The swap device is pinned while checking the flag, matching
 		 * the existing fault path.
 		 */
 		if (data_race(si->flags & SWP_SYNCHRONOUS_IO)) {
-			unsigned long swapin_orders = thp_swapin_suitable_orders(vmf);
 			unsigned long locality_orders =
 				swapin_anon_locality_orders(vmf, swapin_orders);
 
@@ -5041,7 +5042,8 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
 					    swapin_orders | BIT(0),
 					    locality_orders, vmf, NULL, 0);
 		} else {
-			folio = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, vmf);
+			folio = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE,
+						 swapin_orders, vmf);
 		}
 
 		if (IS_ERR_OR_NULL(folio)) {
diff --git a/mm/swap.h b/mm/swap.h
index 5d1c81ab49b9..0e1bf9218b5e 100644
--- a/mm/swap.h
+++ b/mm/swap.h
@@ -323,7 +323,7 @@ struct folio *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask,
 struct folio *swap_cluster_readahead(swp_entry_t entry, gfp_t flag,
 		struct mempolicy *mpol, pgoff_t ilx);
 struct folio *swapin_readahead(swp_entry_t entry, gfp_t flag,
-			struct vm_fault *vmf);
+			       unsigned long orders, struct vm_fault *vmf);
 struct folio *swapin_sync(swp_entry_t entry, gfp_t flag, unsigned long orders,
 			  unsigned long locality_orders, struct vm_fault *vmf,
 			  struct mempolicy *mpol, pgoff_t ilx);
@@ -413,7 +413,7 @@ static inline struct folio *swap_cluster_readahead(swp_entry_t entry,
 }
 
 static inline struct folio *swapin_readahead(swp_entry_t swp, gfp_t gfp_mask,
-			struct vm_fault *vmf)
+				unsigned long orders, struct vm_fault *vmf)
 {
 	return NULL;
 }
diff --git a/mm/swap_state.c b/mm/swap_state.c
index 80dff6a1ee65..4f1eb0a7f9f5 100644
--- a/mm/swap_state.c
+++ b/mm/swap_state.c
@@ -678,20 +678,24 @@ static bool swapin_zswap_admit(swp_entry_t entry,
 static unsigned long swapin_admit_orders(swp_entry_t entry,
 					 unsigned long orders,
 					 struct vm_fault *vmf,
-					 unsigned long locality_orders)
+					 unsigned long locality_orders,
+					 bool zswap_only)
 {
 	unsigned long candidates = orders & ~BIT(0);
-	unsigned long admitted = orders & BIT(0);
+	unsigned long admitted = zswap_only ? 0 : orders & BIT(0);
+	enum zswap_range_state fault_zswap_state = ZSWAP_RANGE_NEVER_ENABLED;
 	struct zswap_admit_ctx zswap_ctx = {};
+	bool fault_zswap_checked = false;
 	int order;
 
 	if (!candidates)
-		return orders;
+		return zswap_only ? 0 : orders;
 
 	while (candidates) {
 		enum zswap_range_state state;
 		unsigned int nr_pages;
 		swp_entry_t range_entry;
+		bool zswap_locality;
 		bool admit = false;
 
 		order = fls_long(candidates) - 1;
@@ -703,6 +707,29 @@ static unsigned long swapin_admit_orders(swp_entry_t entry,
 		nr_pages = 1U << order;
 		range_entry = swp_entry(swp_type(entry),
 					round_down(swp_offset(entry), nr_pages));
+		zswap_locality = order <= SWAPIN_ZSWAP_MAX_ORDER &&
+				 swapin_zswap_locality(vmf, order,
+						       locality_orders);
+		/*
+		 * If the faulting slot is already in zswap but this order has
+		 * no zswap locality evidence, a larger range covering the fault
+		 * cannot be admitted: it is either all-zswap or mixed, and both
+		 * require zswap locality. Avoid scanning the whole range on
+		 * sparse/random zswap refaults. If the faulting slot is not in
+		 * zswap, keep the full classification so all-disk large swapin
+		 * can follow the existing policy.
+		 */
+		if (!zswap_locality) {
+			if (zswap_only)
+				goto next;
+			if (!fault_zswap_checked) {
+				fault_zswap_state = zswap_probe_range(entry, 1);
+				fault_zswap_checked = true;
+			}
+			if (fault_zswap_state == ZSWAP_RANGE_ALL_ZSWAP)
+				goto next;
+		}
+
 		if (!swapin_zeromap_same(range_entry, nr_pages))
 			goto next;
 
@@ -718,7 +745,7 @@ static unsigned long swapin_admit_orders(swp_entry_t entry,
 			break;
 		case ZSWAP_RANGE_NEVER_ENABLED:
 		case ZSWAP_RANGE_NO_ZSWAP:
-			admit = true;
+			admit = !zswap_only;
 			break;
 		}
 
@@ -730,21 +757,32 @@ static unsigned long swapin_admit_orders(swp_entry_t entry,
 		candidates &= ~BIT(order);
 	}
 
-	return admitted ? admitted : BIT(0);
+	return admitted ? admitted : (zswap_only ? 0 : BIT(0));
 }
 
-static bool zswap_needs_order0_retry(struct folio *folio)
+static bool zswap_folio_all_zswap(struct folio *folio)
 {
+	return zswap_probe_range(folio->swap, folio_nr_pages(folio)) ==
+	       ZSWAP_RANGE_ALL_ZSWAP;
+}
+
+static bool zswap_needs_fallback(struct folio *folio, bool zswap_only)
+{
+	enum zswap_range_state state;
+
 	if (!folio_test_large(folio))
 		return false;
 
+	state = zswap_probe_range(folio->swap, folio_nr_pages(folio));
+	if (zswap_only)
+		return state != ZSWAP_RANGE_ALL_ZSWAP;
+
 	/*
 	 * Admission sees only an advisory zswap snapshot. Recheck after the
 	 * large swapcache folio is installed; if the range became mixed, drop
 	 * the fresh folio before IO and let order-0 handle each slot.
 	 */
-	return zswap_probe_range(folio->swap, folio_nr_pages(folio)) ==
-	       ZSWAP_RANGE_MIXED;
+	return state == ZSWAP_RANGE_MIXED;
 }
 
 /*
@@ -758,8 +796,7 @@ bool swapin_fault_only_young(struct folio *folio)
 	if (!folio_test_large(folio) || !folio_test_swapcache(folio))
 		return false;
 
-	return zswap_probe_range(folio->swap, folio_nr_pages(folio)) ==
-	       ZSWAP_RANGE_ALL_ZSWAP;
+	return zswap_folio_all_zswap(folio);
 }
 
 /*
@@ -893,34 +930,15 @@ static struct folio *swap_cache_read_folio(swp_entry_t entry, gfp_t gfp,
 	return folio;
 }
 
-/**
- * swapin_sync - swap-in one or multiple entries skipping readahead.
- * @entry: swap entry indicating the target slot
- * @gfp: memory allocation flags
- * @orders: allocation orders
- * @locality_orders: orders with caller-provided locality evidence
- * @vmf: fault information
- * @mpol: NUMA memory allocation policy to be applied
- * @ilx: NUMA interleave index, for use only when MPOL_INTERLEAVE
- *
- * This allocates a folio suitable for given @orders, or returns the
- * existing folio in the swap cache for @entry. This initiates the IO, too,
- * if needed. @entry is rounded down if @orders allow large allocation.
- *
- * Context: Caller must ensure @entry is valid and pin the swap device with
- * refcount.
- * Return: Returns the folio on success, error code if failed.
- */
-struct folio *swapin_sync(swp_entry_t entry, gfp_t gfp,
-			  unsigned long orders,
-			  unsigned long locality_orders,
-			  struct vm_fault *vmf, struct mempolicy *mpol,
-			  pgoff_t ilx)
+static struct folio *swapin_alloc_read(swp_entry_t entry, gfp_t gfp,
+				       unsigned long orders,
+				       struct vm_fault *vmf,
+				       struct mempolicy *mpol, pgoff_t ilx,
+				       bool retry_order0, bool zswap_only)
 {
 	struct folio *folio;
 	int ret;
 
-	orders = swapin_admit_orders(entry, orders, vmf, locality_orders);
 again:
 	do {
 		folio = swap_cache_get_folio(entry);
@@ -931,19 +949,21 @@ struct folio *swapin_sync(swp_entry_t entry, gfp_t gfp,
 	} while (PTR_ERR(folio) == -EEXIST);
 
 	if (IS_ERR(folio))
-		return folio;
+		return retry_order0 ? folio : NULL;
 
-	if (zswap_needs_order0_retry(folio)) {
+	if (zswap_needs_fallback(folio, zswap_only)) {
 		count_mthp_stat(folio_order(folio), MTHP_STAT_SWPIN_FALLBACK);
 		/*
 		 * The folio is newly allocated, locked, clean and not uptodate;
 		 * no data has been read into it. Removing it only restores the
-		 * swap table entries so order-0 swapin can resolve a backend
+		 * swap table entries so the fallback path can resolve a backend
 		 * race without attempting speculative large-folio zswapin.
 		 */
 		swap_cache_del_folio(folio);
 		folio_unlock(folio);
 		folio_put(folio);
+		if (!retry_order0)
+			return NULL;
 		orders = BIT(0);
 		goto again;
 	}
@@ -954,12 +974,62 @@ struct folio *swapin_sync(swp_entry_t entry, gfp_t gfp,
 		swap_cache_del_folio(folio);
 		folio_unlock(folio);
 		folio_put(folio);
+		if (!retry_order0)
+			return NULL;
 		orders = BIT(0);
 		goto again;
 	}
 	return folio;
 }
 
+/**
+ * swapin_sync - swap-in one or multiple entries skipping readahead.
+ * @entry: swap entry indicating the target slot
+ * @gfp: memory allocation flags
+ * @orders: allocation orders
+ * @locality_orders: orders with caller-provided locality evidence
+ * @vmf: fault information
+ * @mpol: NUMA memory allocation policy to be applied
+ * @ilx: NUMA interleave index, for use only when MPOL_INTERLEAVE
+ *
+ * This allocates a folio suitable for given @orders, or returns the
+ * existing folio in the swap cache for @entry. This initiates the IO, too,
+ * if needed. @entry is rounded down if @orders allow large allocation.
+ *
+ * Context: Caller must ensure @entry is valid and pin the swap device with
+ * refcount.
+ * Return: Returns the folio on success, error code if failed.
+ */
+struct folio *swapin_sync(swp_entry_t entry, gfp_t gfp,
+			  unsigned long orders,
+			  unsigned long locality_orders,
+			  struct vm_fault *vmf, struct mempolicy *mpol,
+			  pgoff_t ilx)
+{
+	orders = swapin_admit_orders(entry, orders, vmf,
+				     locality_orders, false);
+	return swapin_alloc_read(entry, gfp, orders, vmf, mpol, ilx,
+				 true, false);
+}
+
+static struct folio *swapin_zswap_large(swp_entry_t entry, gfp_t gfp,
+					unsigned long orders,
+					unsigned long locality_orders,
+					struct vm_fault *vmf,
+					struct mempolicy *mpol, pgoff_t ilx)
+{
+	if (READ_ONCE(page_cluster) <= 0)
+		return NULL;
+
+	orders = swapin_admit_orders(entry, orders, vmf,
+				     locality_orders, true);
+	if (!orders)
+		return NULL;
+
+	return swapin_alloc_read(entry, gfp, orders, vmf, mpol, ilx,
+				 false, true);
+}
+
 /*
  * Locate a page of swap in physical memory, reserving swap cache space
  * and reading the disk if it is not already cached.
@@ -1048,12 +1118,88 @@ static unsigned long swapin_nr_pages(unsigned long offset)
 	return pages;
 }
 
+struct swap_cluster_ra {
+	unsigned long start_offset;
+	unsigned long end_offset;
+	bool readahead;
+};
+
+static void swap_cluster_ra_prepare(swp_entry_t entry,
+				    struct swap_cluster_ra *ra)
+{
+	struct swap_info_struct *si = __swap_entry_to_info(entry);
+	unsigned long entry_offset = swp_offset(entry);
+	unsigned long mask;
+
+	mask = swapin_nr_pages(entry_offset) - 1;
+	ra->readahead = !!mask;
+	ra->start_offset = entry_offset;
+	ra->end_offset = entry_offset;
+	if (!mask)
+		return;
+
+	/* Read a page_cluster sized and aligned cluster around offset. */
+	ra->start_offset = entry_offset & ~mask;
+	ra->end_offset = entry_offset | mask;
+	if (!ra->start_offset)	/* First page is swap header. */
+		ra->start_offset++;
+	if (ra->end_offset >= si->max)
+		ra->end_offset = si->max - 1;
+}
+
+static unsigned long swap_cluster_ra_orders(swp_entry_t entry,
+					    unsigned long orders,
+					    const struct swap_cluster_ra *ra)
+{
+	unsigned long admitted = 0;
+	unsigned long candidates = orders & ~BIT(0);
+	unsigned long entry_offset = swp_offset(entry);
+	int order;
+
+	if (!ra->readahead)
+		return 0;
+
+	while (candidates) {
+		unsigned long nr_pages;
+		unsigned long start_offset;
+		unsigned long end_offset;
+
+		order = fls_long(candidates) - 1;
+		if (order > MAX_PAGE_ORDER) {
+			candidates &= ~BIT(order);
+			continue;
+		}
+
+		nr_pages = 1UL << order;
+		start_offset = round_down(entry_offset, nr_pages);
+		end_offset = start_offset + nr_pages - 1;
+		if (start_offset >= ra->start_offset &&
+		    end_offset <= ra->end_offset)
+			admitted |= BIT(order);
+		candidates &= ~BIT(order);
+	}
+
+	return admitted;
+}
+
+static bool swapin_readahead_skip(unsigned long index,
+				  unsigned long skip_start,
+				  unsigned long skip_end)
+{
+	return skip_start < skip_end &&
+	       index >= skip_start && index < skip_end;
+}
+
 /**
- * swap_cluster_readahead - swap in pages in hope we need them soon
+ * swap_cluster_readahead_win - swap in pages from a prepared swap window
  * @entry: swap entry of this memory
  * @gfp_mask: memory allocation flags
  * @mpol: NUMA memory allocation policy to be applied
  * @ilx: NUMA interleave index, for use only when MPOL_INTERLEAVE
+ * @ra: readahead window prepared by swap_cluster_ra_prepare()
+ * @skip_start: first offset already covered by @target_folio
+ * @skip_end: offset after the already covered range
+ * @target_folio: target folio to return after queueing the rest of the window
  *
  * Returns the struct folio for entry and addr, after queueing swapin.
  *
@@ -1066,33 +1212,38 @@ static unsigned long swapin_nr_pages(unsigned long offset)
  * are used for every page of the readahead: neighbouring pages on swap
  * are fairly likely to have been swapped out from the same node.
  */
-struct folio *swap_cluster_readahead(swp_entry_t entry, gfp_t gfp_mask,
-				     struct mempolicy *mpol, pgoff_t ilx)
+static struct folio *swap_cluster_readahead_win(swp_entry_t entry,
+						gfp_t gfp_mask,
+						struct mempolicy *mpol,
+						pgoff_t ilx,
+						const struct swap_cluster_ra *ra,
+						unsigned long skip_start,
+						unsigned long skip_end,
+						struct folio *target_folio)
 {
 	struct folio *folio;
 	unsigned long entry_offset = swp_offset(entry);
-	unsigned long offset = entry_offset;
-	unsigned long start_offset, end_offset;
-	unsigned long mask;
-	struct swap_info_struct *si = __swap_entry_to_info(entry);
+	unsigned long offset;
 	struct blk_plug plug;
 	struct swap_iocb *splug = NULL;
 	swp_entry_t ra_entry;
 
-	mask = swapin_nr_pages(offset) - 1;
-	if (!mask)
+	if (!ra->readahead)
 		goto skip;
 
-	/* Read a page_cluster sized and aligned cluster around offset. */
-	start_offset = offset & ~mask;
-	end_offset = offset | mask;
-	if (!start_offset)	/* First page is swap header. */
-		start_offset++;
-	if (end_offset >= si->max)
-		end_offset = si->max - 1;
+	if (target_folio &&
+	    skip_start <= ra->start_offset && skip_end > ra->end_offset)
+		goto skip;
 
 	blk_start_plug(&plug);
-	for (offset = start_offset; offset <= end_offset ; offset++) {
+	for (offset = ra->start_offset; offset <= ra->end_offset; offset++) {
+		if (swapin_readahead_skip(offset, skip_start, skip_end)) {
+			if (skip_end > ra->end_offset)
+				break;
+			offset = skip_end - 1;
+			continue;
+		}
+
 		/* Ok, do the async read-ahead now */
 		ra_entry = swp_entry(swp_type(entry), offset);
 		folio = swap_cache_read_folio(ra_entry, gfp_mask, mpol, ilx,
@@ -1105,10 +1256,29 @@ struct folio *swap_cluster_readahead(swp_entry_t entry, gfp_t gfp_mask,
 	swap_read_unplug(splug);
 	lru_add_drain();	/* Push any new pages onto the LRU now */
 skip:
+	if (target_folio)
+		return target_folio;
+
 	/* The page was likely read above, so no need for plugging here */
 	return swap_cache_read_folio(entry, gfp_mask, mpol, ilx, NULL, false);
 }
 
+struct folio *swap_cluster_readahead(swp_entry_t entry, gfp_t gfp_mask,
+				     struct mempolicy *mpol, pgoff_t ilx)
+{
+	struct swap_cluster_ra ra;
+
+	swap_cluster_ra_prepare(entry, &ra);
+	return swap_cluster_readahead_win(entry, gfp_mask, mpol, ilx, &ra,
+					 0, 0, NULL);
+}
+
+struct swap_vma_ra {
+	unsigned long start;
+	unsigned long end;
+	int win;
+};
+
 static int swap_vma_ra_win(struct vm_fault *vmf, unsigned long *start,
 			   unsigned long *end)
 {
@@ -1147,35 +1317,69 @@ static int swap_vma_ra_win(struct vm_fault *vmf, unsigned long *start,
 	return win;
 }
 
-/**
- * swap_vma_readahead - swap in pages in hope we need them soon
- * @targ_entry: swap entry of the targeted memory
- * @gfp_mask: memory allocation flags
- * @mpol: NUMA memory allocation policy to be applied
- * @targ_ilx: NUMA interleave index, for use only when MPOL_INTERLEAVE
- * @vmf: fault information
- *
- * Returns the struct folio for entry and addr, after queueing swapin.
- *
- * Primitive swap readahead code. We simply read in a few pages whose
- * virtual addresses are around the fault address in the same vma.
- *
- * Caller must hold read mmap_lock if vmf->vma is not NULL.
- *
+static unsigned long swap_vma_ra_orders(struct vm_fault *vmf,
+					unsigned long orders,
+					const struct swap_vma_ra *ra)
+{
+	unsigned long admitted = 0;
+	unsigned long candidates = orders & ~BIT(0);
+	int order;
+
+	if (ra->win <= 1)
+		return 0;
+
+	while (candidates) {
+		unsigned long size;
+		unsigned long start;
+		unsigned long end;
+
+		order = fls_long(candidates) - 1;
+		if (order > MAX_PAGE_ORDER) {
+			candidates &= ~BIT(order);
+			continue;
+		}
+
+		size = PAGE_SIZE << order;
+		start = ALIGN_DOWN(vmf->address, size);
+		end = start + size;
+		if (start >= ra->start && end <= ra->end)
+			admitted |= BIT(order);
+		candidates &= ~BIT(order);
+	}
+
+	return admitted;
+}
+
+/*
+ * Queue swapin for a precomputed VMA readahead window. The window has already
+ * been accounted in vma->swap_readahead_info, so fallback after a failed
+ * zswap-large attempt does not update readahead state a second time. If
+ * @target_folio is already populated, queue only the part of the window outside
+ * [@skip_start, @skip_end) and return @target_folio.
  */
-static struct folio *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask,
-		struct mempolicy *mpol, pgoff_t targ_ilx, struct vm_fault *vmf)
+static struct folio *swap_vma_readahead_win(swp_entry_t targ_entry,
+					    gfp_t gfp_mask,
+					    struct mempolicy *mpol,
+					    pgoff_t targ_ilx,
+					    struct vm_fault *vmf,
+					    const struct swap_vma_ra *ra,
+					    unsigned long skip_start,
+					    unsigned long skip_end,
+					    struct folio *target_folio)
 {
 	struct blk_plug plug;
 	struct swap_iocb *splug = NULL;
 	struct folio *folio;
 	pte_t *pte = NULL, pentry;
-	int win;
 	unsigned long start, end, addr;
 	pgoff_t ilx = targ_ilx;
 
-	win = swap_vma_ra_win(vmf, &start, &end);
-	if (win == 1)
+	if (ra->win <= 1)
+		goto skip;
+
+	start = ra->start;
+	end = ra->end;
+	if (target_folio && skip_start <= start && skip_end >= end)
 		goto skip;
 
 	ilx = targ_ilx - PFN_DOWN(vmf->address - start);
@@ -1185,6 +1389,18 @@ static struct folio *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask,
 		struct swap_info_struct *si = NULL;
 		softleaf_t entry;
 
+		if (swapin_readahead_skip(addr, skip_start, skip_end)) {
+			unsigned long next = min(skip_end, end);
+
+			if (pte) {
+				pte_unmap(pte);
+				pte = NULL;
+			}
+			ilx += PFN_DOWN(next - addr) - 1;
+			addr = next - PAGE_SIZE;
+			continue;
+		}
+
 		if (!pte++) {
 			pte = pte_offset_map(vmf->pmd, addr);
 			if (!pte)
@@ -1220,6 +1436,9 @@ static struct folio *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask,
 	swap_read_unplug(splug);
 	lru_add_drain();
 skip:
+	if (target_folio)
+		return target_folio;
+
 	/* The folio was likely read above, so no need for plugging here */
 	folio = swap_cache_read_folio(targ_entry, gfp_mask, mpol, targ_ilx,
 				      NULL, false);
@@ -1230,25 +1449,78 @@ static struct folio *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask,
  * swapin_readahead - swap in pages in hope we need them soon
  * @entry: swap entry of this memory
  * @gfp_mask: memory allocation flags
+ * @orders: large folio orders suitable for the faulting entry
  * @vmf: fault information
  *
  * Returns the struct folio for entry and addr, after queueing swapin.
  *
- * It's a main entry function for swap readahead. By the configuration,
- * it will read ahead blocks by cluster-based(ie, physical disk based)
- * or vma-based(ie, virtual address based on faulty address) readahead.
+ * This first computes the normal VMA or cluster readahead window. If the
+ * window fully covers an aligned all-zswap range containing the fault, that
+ * range may be swapped in as one large folio. The remaining window is still
+ * queued through the original order-0 readahead path, skipping the already
+ * covered target range and without updating readahead state a second time.
  */
 struct folio *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask,
-				struct vm_fault *vmf)
+				unsigned long orders, struct vm_fault *vmf)
 {
 	struct mempolicy *mpol;
 	pgoff_t ilx;
 	struct folio *folio;
+	unsigned long ra_orders;
+	bool vma_ra;
 
 	mpol = get_vma_policy(vmf->vma, vmf->address, 0, &ilx);
-	folio = swap_use_vma_readahead() ?
-		swap_vma_readahead(entry, gfp_mask, mpol, ilx, vmf) :
-		swap_cluster_readahead(entry, gfp_mask, mpol, ilx);
+	vma_ra = swap_use_vma_readahead();
+	if (vma_ra) {
+		struct swap_vma_ra ra = {};
+		unsigned long skip_start = 0;
+		unsigned long skip_end = 0;
+
+		ra.win = swap_vma_ra_win(vmf, &ra.start, &ra.end);
+		ra_orders = swap_vma_ra_orders(vmf, orders, &ra);
+		if (ra_orders) {
+			folio = swapin_zswap_large(entry, gfp_mask, ra_orders,
+						   ra_orders, vmf, mpol, ilx);
+			if (folio) {
+				skip_start = ALIGN_DOWN(vmf->address,
+							folio_size(folio));
+				skip_end = skip_start + folio_size(folio);
+				folio = swap_vma_readahead_win(entry, gfp_mask,
+							       mpol, ilx, vmf,
+							       &ra, skip_start,
+							       skip_end, folio);
+				goto out;
+			}
+		}
+		folio = swap_vma_readahead_win(entry, gfp_mask, mpol, ilx,
+					       vmf, &ra, 0, 0, NULL);
+	} else {
+		struct swap_cluster_ra ra;
+		unsigned long skip_start = 0;
+		unsigned long skip_end = 0;
+
+		swap_cluster_ra_prepare(entry, &ra);
+		ra_orders = swap_cluster_ra_orders(entry, orders, &ra);
+		if (ra_orders) {
+			folio = swapin_zswap_large(entry, gfp_mask, ra_orders,
+						   ra_orders, vmf, mpol, ilx);
+			if (folio) {
+				skip_start = swp_offset(folio->swap);
+				skip_end = skip_start + folio_nr_pages(folio);
+				folio = swap_cluster_readahead_win(entry,
+								   gfp_mask,
+								   mpol, ilx,
+								   &ra,
+								   skip_start,
+								   skip_end,
+								   folio);
+				goto out;
+			}
+		}
+		folio = swap_cluster_readahead_win(entry, gfp_mask, mpol, ilx,
+						   &ra, 0, 0, NULL);
+	}
+out:
 	mpol_cond_put(mpol);
 
 	return folio;
diff --git a/mm/swapfile.c b/mm/swapfile.c
index 615d90867111..3b7e7d8ae89d 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -2452,7 +2452,7 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd,
 			};
 
 			folio = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE,
-						&vmf);
+						 0, &vmf);
 		}
 		if (!folio) {
 			swp_tb = swap_table_get(__swap_entry_to_cluster(entry),
-- 
2.34.1

[RFC PATCH v2 9/9] docs: mm: update THP swapin counter descriptions

[fujunjie]

The THP swapin counter descriptions still describe large swapin as
coming only from non-zswap swap devices. Update them now that
zswap-backed large folio swapin can also increment swpin.

Also describe policy and backend rejection as swpin_fallback cases,
since speculative zswap large swapin can intentionally fall back before
doing large IO.

Signed-off-by: fujunjie <fujunjie1@qq.com>
---
 Documentation/admin-guide/mm/transhuge.rst | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst
index 23f8d13c2629..59b7a0d09243 100644
--- a/Documentation/admin-guide/mm/transhuge.rst
+++ b/Documentation/admin-guide/mm/transhuge.rst
@@ -667,13 +667,14 @@ zswpout
 	piece without splitting.
 
 swpin
-	is incremented every time a huge page is swapped in from a non-zswap
-	swap device in one piece.
+	is incremented every time a huge page is swapped in from swap or
+	zswap in one piece.
 
 swpin_fallback
-	is incremented if swapin fails to allocate or charge a huge page
-	and instead falls back to using huge pages with lower orders or
-	small pages.
+	is incremented if swapin cannot use a huge page and instead falls
+	back to using huge pages with lower orders or small pages. This can
+	happen because allocation or charging fails, or because policy or
+	backend state rejects a speculative large swapin.
 
 swpin_fallback_charge
 	is incremented if swapin fails to charge a huge page and instead
-- 
2.34.1

Re: [RFC PATCH v2 4/9] mm: admit large swapin by backend range in swapin_sync()

[Kairui Song]

On Fri, May 29, 2026 at 8:26 PM fujunjie <fujunjie1@qq.com> wrote:
>
> A large swapin can only read one folio when the whole range has compatible
> backing. Mixed zswap/disk ranges must not reach large-folio IO, and zswap
> range probes are only snapshots.
>
> Filter the orders passed to swap_cache_alloc_folio() in swapin_sync().
> Uniform zeromap ranges and all-disk ranges keep the existing large swapin
> path. Fully zswap-backed ranges may be tried. Mixed zswap/disk ranges fall
> back before allocation.
>
> After a large swapcache folio is installed, recheck the zswap range and
> drop the fresh folio if it became mixed. Also consume -EAGAIN from
> swap_read_folio() the same way. Both cases retry order-0, where each slot
> can resolve its current backend independently.
>
> Signed-off-by: fujunjie <fujunjie1@qq.com>
> ---
>  mm/memcontrol-v1.c |   8 ++-
>  mm/memory.c        |  31 ++++++++-
>  mm/swap_state.c    | 169 ++++++++++++++++++++++++++++++++++++++++++---
>  3 files changed, 194 insertions(+), 14 deletions(-)
>
> diff --git a/mm/memcontrol-v1.c b/mm/memcontrol-v1.c
> index 765069211567..5b11b8055c66 100644
> --- a/mm/memcontrol-v1.c
> +++ b/mm/memcontrol-v1.c
> @@ -682,8 +682,8 @@ void __memcg1_swapout(struct folio *folio, struct swap_cluster_info *ci)
>   * memcg1_swapin - uncharge swap slot on swapin
>   * @folio: folio being swapped in
>   *
> - * Call this function after successfully adding the charged
> - * folio to swapcache.
> + * Call this after the charged folio has been added to swapcache and the caller
> + * is no longer going to drop it back to swapped-out state.
>   *
>   * Context: The folio has to be in swap cache and locked.
>   */
> @@ -721,7 +721,9 @@ void memcg1_swapin(struct folio *folio)
>         id = __swap_cgroup_clear(ci, swp_cluster_offset(folio->swap),
>                                  nr_pages);
>         swap_cluster_unlock(ci);
> -       mem_cgroup_uncharge_swap(id, nr_pages);
> +
> +       if (id)
> +               mem_cgroup_uncharge_swap(id, nr_pages);
>  }
>  #endif
>
> diff --git a/mm/memory.c b/mm/memory.c
> index 5a365492a9a2..d73a19692dea 100644
> --- a/mm/memory.c
> +++ b/mm/memory.c
> @@ -4538,6 +4538,24 @@ static inline bool should_try_to_free_swap(struct swap_info_struct *si,
>                 folio_ref_count(folio) == (extra_refs + folio_nr_pages(folio));
>  }
>
> +static void memcg1_swapin_retry_folio(struct folio *folio,
> +                                     struct vm_fault *vmf)
> +{
> +       if (!folio_test_large(folio) || !folio_test_swapcache(folio))
> +               return;
> +
> +       if (vmf->flags & FAULT_FLAG_RETRY_NOWAIT) {
> +               if (!folio_trylock(folio))
> +                       return;
> +       } else {
> +               folio_lock(folio);
> +       }
> +
> +       if (folio_test_large(folio) && folio_test_swapcache(folio))
> +               memcg1_swapin(folio);
> +       folio_unlock(folio);
> +}
> +
>  static vm_fault_t pte_marker_clear(struct vm_fault *vmf)
>  {
>         vmf->pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd,
> @@ -4857,8 +4875,10 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
>
>         swapcache = folio;
>         ret |= folio_lock_or_retry(folio, vmf);
> -       if (ret & VM_FAULT_RETRY)
> +       if (ret & VM_FAULT_RETRY) {
> +               memcg1_swapin_retry_folio(folio, vmf);
>                 goto out_release;
> +       }
>
>         page = folio_file_page(folio, swp_offset(entry));
>         /*
> @@ -5067,6 +5087,8 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
>         if (unlikely(folio != swapcache)) {
>                 folio_add_new_anon_rmap(folio, vma, address, RMAP_EXCLUSIVE);
>                 folio_add_lru_vma(folio, vma);
> +               if (folio_test_large(swapcache))
> +                       memcg1_swapin(swapcache);
>                 folio_put_swap(swapcache, NULL);
>         } else if (!folio_test_anon(folio)) {
>                 /*
> @@ -5076,6 +5098,8 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
>                 VM_WARN_ON_ONCE_FOLIO(folio_nr_pages(folio) != nr_pages, folio);
>                 VM_WARN_ON_ONCE_FOLIO(folio_mapped(folio), folio);
>                 folio_add_new_anon_rmap(folio, vma, address, rmap_flags);
> +               if (folio_test_large(folio))
> +                       memcg1_swapin(folio);
>                 folio_put_swap(folio, NULL);
>         } else {
>                 VM_WARN_ON_ONCE(nr_pages != 1 && nr_pages != folio_nr_pages(folio));
> @@ -5132,8 +5156,11 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
>         if (vmf->pte)
>                 pte_unmap_unlock(vmf->pte, vmf->ptl);
>  out_page:
> -       if (folio_test_swapcache(folio))
> +       if (folio_test_swapcache(folio)) {
> +               if (folio_test_large(folio))
> +                       memcg1_swapin(folio);
>                 folio_free_swap(folio);
> +       }
>         folio_unlock(folio);
>  out_release:
>         folio_put(folio);
> diff --git a/mm/swap_state.c b/mm/swap_state.c
> index d37097913b30..f03ad4832f16 100644
> --- a/mm/swap_state.c
> +++ b/mm/swap_state.c
> @@ -21,6 +21,7 @@
>  #include <linux/migrate.h>
>  #include <linux/vmalloc.h>
>  #include <linux/huge_mm.h>
> +#include <linux/zswap.h>
>  #include <linux/shmem_fs.h>
>  #include "internal.h"
>  #include "swap_table.h"
> @@ -403,7 +404,8 @@ void __swap_cache_replace_folio(struct swap_cluster_info *ci,
>  static struct folio *__swap_cache_alloc(struct swap_cluster_info *ci,
>                                         swp_entry_t targ_entry, gfp_t gfp,
>                                         unsigned int order, struct vm_fault *vmf,
> -                                       struct mempolicy *mpol, pgoff_t ilx)
> +                                       struct mempolicy *mpol, pgoff_t ilx,
> +                                       bool defer_memcg1_swapin)

Hi Fujunjie,

Thanks for the update, but this whole defer_memcg1_swapin thing is so
ugly I don't think this is the right way at all.

If you really need this, maybe you can always defer the memcg1
uncharge, I don't see why we need to treat large folio differently.
This charge doesn't effect the memory pressure, the reason we uncharge
memcg1's swap counter is to avoid long pinning swap cache holding the
swap cache of a cgroup so the cgroup will no longer be able to swap
out more folios. Deferring it won't hurt.

>  {
>         int err;
>         swp_entry_t entry;
> @@ -466,7 +468,8 @@ static struct folio *__swap_cache_alloc(struct swap_cluster_info *ci,
>         }
>
>         /* memsw uncharges swap when folio is added to swap cache */
> -       memcg1_swapin(folio);
> +       if (!defer_memcg1_swapin || !order)
> +               memcg1_swapin(folio);
>         if (shadow)
>                 workingset_refault(folio, shadow);
>
> @@ -495,9 +498,12 @@ static struct folio *__swap_cache_alloc(struct swap_cluster_info *ci,
>   * Return: Returns the folio if allocation succeeded and folio is in the swap
>   * cache. Returns error code if failed due to race, OOM or invalid arguments.
>   */
> -struct folio *swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp,
> -                                    unsigned long orders, struct vm_fault *vmf,
> -                                    struct mempolicy *mpol, pgoff_t ilx)
> +static struct folio *__swap_cache_alloc_folio(swp_entry_t targ_entry,
> +                                             gfp_t gfp, unsigned long orders,
> +                                             struct vm_fault *vmf,
> +                                             struct mempolicy *mpol,
> +                                             pgoff_t ilx,
> +                                             bool defer_memcg1_swapin)
>  {
>         int order, err;
>         struct folio *ret;
> @@ -512,7 +518,8 @@ struct folio *swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp,
>
>         do {
>                 ret = __swap_cache_alloc(ci, targ_entry, gfp, order,
> -                                        vmf, mpol, ilx);
> +                                        vmf, mpol, ilx,
> +                                        defer_memcg1_swapin);
>                 if (!IS_ERR(ret))
>                         break;
>                 err = PTR_ERR(ret);
> @@ -525,6 +532,124 @@ struct folio *swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp,
>         return ret;
>  }
>
> +struct folio *swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp,
> +                                    unsigned long orders, struct vm_fault *vmf,
> +                                    struct mempolicy *mpol, pgoff_t ilx)
> +{
> +       return __swap_cache_alloc_folio(targ_entry, gfp, orders, vmf,
> +                                       mpol, ilx, false);
> +}
> +
> +static struct folio *swap_cache_alloc_speculative_folio(swp_entry_t targ_entry,
> +                                                       gfp_t gfp,
> +                                                       unsigned long orders,
> +                                                       struct vm_fault *vmf,
> +                                                       struct mempolicy *mpol,
> +                                                       pgoff_t ilx)
> +{
> +       /*
> +        * Speculative large swapin may drop this fresh swapcache folio and
> +        * retry order-0 after backend or page-table revalidation. Keep the
> +        * cgroup v1 memsw swap owner until the caller commits the folio.
> +        */
> +       return __swap_cache_alloc_folio(targ_entry, gfp, orders, vmf,
> +                                       mpol, ilx, true);
> +}
> +
> +static bool swapin_zeromap_same(swp_entry_t entry, unsigned int nr_pages)
> +{
> +       unsigned int ci_start = swp_cluster_offset(entry);
> +       struct swap_cluster_info *ci = __swap_entry_to_cluster(entry);
> +       bool is_zero;
> +       unsigned int i;
> +
> +       if (ci_start + nr_pages > SWAPFILE_CLUSTER) {
> +               VM_WARN_ON_ONCE(1);
> +               return false;
> +       }
> +
> +       rcu_read_lock();
> +       if (!rcu_dereference(ci->table)) {
> +               rcu_read_unlock();
> +               return true;
> +       }
> +
> +       is_zero = __swap_table_test_zero(ci, ci_start);
> +       for (i = 1; i < nr_pages; i++) {
> +               if (is_zero != __swap_table_test_zero(ci, ci_start + i)) {
> +                       rcu_read_unlock();
> +                       return false;
> +               }
> +       }
> +       rcu_read_unlock();
> +
> +       return true;
> +}
> +
> +static unsigned long swapin_admit_orders(swp_entry_t entry,
> +                                        unsigned long orders)

And this swapin_admit_orders chunk doesn't look good either...

> +{
> +       unsigned long candidates = orders & ~BIT(0);
> +       unsigned long admitted = orders & BIT(0);
> +       int order;
> +
> +       if (!candidates)
> +               return orders;
> +
> +       while (candidates) {
> +               enum zswap_range_state state;
> +               unsigned int nr_pages;
> +               swp_entry_t range_entry;
> +               bool admit = false;
> +
> +               order = fls_long(candidates) - 1;
> +               if (order > MAX_PAGE_ORDER) {
> +                       candidates &= ~BIT(order);
> +                       continue;
> +               }
> +
> +               nr_pages = 1U << order;
> +               range_entry = swp_entry(swp_type(entry),
> +                                       round_down(swp_offset(entry), nr_pages));
> +               if (!swapin_zeromap_same(range_entry, nr_pages))
> +                       goto next;

I think you don't need to test zeromap at all? __swap_cache_alloc
handles that already.

> +
> +               state = zswap_probe_range(range_entry, nr_pages);

If you just move the zswap_probe_range into __swap_cache_alloc and do
fallback there (or maybe you can shrink the order faster), then this
two new helpers are all redundant.

> +               switch (state) {
> +               case ZSWAP_RANGE_MIXED:
> +                       break;
> +               case ZSWAP_RANGE_ALL_ZSWAP:
> +               case ZSWAP_RANGE_NEVER_ENABLED:
> +               case ZSWAP_RANGE_NO_ZSWAP:
> +                       admit = true;
> +                       break;
> +               }
> +
> +next:
> +               if (admit)
> +                       admitted |= BIT(order);
> +               else
> +                       count_mthp_stat(order, MTHP_STAT_SWPIN_FALLBACK);
> +               candidates &= ~BIT(order);
> +       }
> +
> +       return admitted ? admitted : BIT(0);
> +}
> +
> +static bool zswap_needs_order0_retry(struct folio *folio)
> +{
> +       if (!folio_test_large(folio))
> +               return false;
> +
> +       /*
> +        * Admission sees only an advisory zswap snapshot. Recheck after the
> +        * large swapcache folio is installed; if the range became mixed, drop
> +        * the fresh folio before IO and let order-0 handle each slot.
> +        */
> +       return zswap_probe_range(folio->swap, folio_nr_pages(folio)) ==
> +              ZSWAP_RANGE_MIXED;
> +}
> +

Again, I think you can just probe the suitable size in
__swap_cache_alloc directly, that way, we avoid the diverge of sync /
non-sync device, and avoid the whole chunk making the code much
simplier too, just like what we are alreadying doing for zero map in
__swap_cache_alloc, or am I over simpliying it?

Re: [RFC PATCH v2 4/9] mm: admit large swapin by backend range in swapin_sync()

[Kairui Song]

On Fri, May 29, 2026 at 10:43 PM Kairui Song <ryncsn@gmail.com> wrote:
>
> Hi Fujunjie,
>
> Thanks for the update, but this whole defer_memcg1_swapin thing is so
> ugly I don't think this is the right way at all.
>
> If you really need this, maybe you can always defer the memcg1

Oh and I'm not saying I'm against this series or the idea, I'm just
saying this particular design of this one patch needs some improvement
:)

Re: [RFC PATCH v2 0/9] mm: support zswap-backed large folio swapin

[Nhat Pham]

On Fri, May 29, 2026 at 5:17 AM fujunjie <fujunjie1@qq.com> wrote:
>
> Hi,
>
> This RFC explores large-folio swapin for ranges that are still fully backed
> by zswap.
>
> Large swapin is currently disabled once zswap is in the picture. Anonymous
> faults stop considering large orders after zswap has ever been enabled,
> shmem does the same, and zswap_load() refuses large swapcache folios. That
> keeps mixed zswap/disk cases safe, but it also loses the dense case where
> every slot in an aligned 64K range is still resident in zswap.
>
> The series keeps the policy in common swapin code:
>
>   - zswap reports backend facts and provides the large-folio load helper.
>   - swapin_sync() filters candidate orders by backend range.
>   - all-disk and zeromap ranges keep the existing Kairui large-swapin path.
>   - mixed zswap/disk ranges stay order-0.
>   - all-zswap ranges may use a 64K folio after locality admission.
>   - anon provides locality evidence from VMA hints and PTE young density.
>   - shmem starts with explicit VMA-hint evidence only.
>   - swap readahead uses its existing VMA/cluster window as locality
>     evidence; it does not also run the anon PTE-young rule.
>
> The backend range probe is only a snapshot. If the backend changes after a
> fresh large swapcache folio is allocated, the common path drops that folio
> and falls back to order-0. zswap_load() can also return -EAGAIN for the
> same retry path. If a late fault retry keeps the large folio in swapcache
> instead of deleting it, the cgroup v1 memsw swap owner is committed before
> returning.
>
> This is mTHP/large-folio swapin. The mappings installed by do_swap_page()
> are still PTE mappings, not PMD mappings. The expected win is fewer faults,
> batched PTE/rmap work, and preserving the large folio across zswapin
> instead of rebuilding the working set as order-0 pages.
>
> Prior art: Usama Arif posted a related RFC on 2024-10-18:
>
>   mm: zswap: add support for zswapin of large folios
>   https://lore.kernel.org/linux-mm/20241018105026.2521366-1-usamaarif642@gmail.com/
>
> This RFC keeps the same broad goal, but moves admission into common swapin
> code. zswap does not decide the policy. Mixed zswap/disk ranges are
> rejected before large IO, and the first cap is 64K.
>
> This is a rewrite of the RFC posted on 2026-05-08:
>
>   [RFC PATCH 0/5] mm: support zswap-backed anonymous large folio swapin
>   https://lore.kernel.org/linux-mm/tencent_8B437BE4F586C162950BF71954316C1EDB05@qq.com/
>
> The v1 series was anonymous-only and kept too much of the policy near the
> anon fault and zswap paths. This version is rebuilt on top of Kairui Song's
> common swapin infrastructure. It keeps admission in common swapin code,
> rejects mixed zswap/disk large ranges, and adds separate locality producers
> for anon, shmem and swap readahead.
>
> Performance and behavior
> ========================
>
> The A/B tables are 10-run measurements. Elapsed values are seconds,
> shown as mean +/- sample standard deviation. "phase" or "refault" is the
> measured refault subphase. "zswpin" counts zswap loads. "pswpin" counts
> swap-ins from the real swap device; pswpin=0 means the refaults were served
> by zswap even when a disk swap device was configured. "RFC 64K" is the mean
> number of successful 64K swapins.
>
> The numbers below show where the large path is used and where it is
> rejected.
>
> zram-backed zswap microbench, 64K mTHP, 8G guest:
>
> +-----------------+----------------+----------------+--------+--------+--------+----------+
> | workload        | base elapsed   | RFC elapsed    | delta  | phase  | zswpin | RFC 64K  |
> +-----------------+----------------+----------------+--------+--------+--------+----------+
> | usama_1g        | 11.260+/-0.235 | 10.301+/-0.140 | -8.5%  | -22.2% | 1.000x | 16381.1  |
> | nohint_seq64    |  4.398+/-0.085 |  4.025+/-0.022 | -8.5%  | -21.1% | 1.000x |  6221.1  |
> | seqhint_seq64   |  4.283+/-0.060 |  3.948+/-0.062 | -7.8%  | -20.6% | 1.000x |  6223.5  |
> | stride64_sparse |  3.095+/-0.051 |  3.086+/-0.025 | -0.3%  |  +5.8% | 1.002x |     1.0  |
> | random64_sparse |  3.095+/-0.046 |  3.076+/-0.016 | -0.6%  |  +0.7% | 1.001x |     0.0  |
> | random64_full   |  4.423+/-0.067 |  4.405+/-0.018 | -0.4%  |  +0.1% | 1.000x |     0.0  |
> +-----------------+----------------+----------------+--------+--------+--------+----------+
>
> The usama_1g row follows the shape of the 2024 RFC benchmark: allocate 1G,
> fill it with compressible per-page data, reclaim it through memory.reclaim,
> then time the full integrity-check refault. The seq64 rows use a 512M
> target and 768M of pressure. "sparse" touches one 4K page per 64K region, while
> "full" touches every 4K page. "seqhint" uses MADV_SEQUENTIAL; "nohint" does
> not.
>
> Virtio-block swap device present, zswap enabled:
>
> +-----------------+---------------+---------------+--------+---------+--------+--------+---------+
> | workload        | base elapsed  | RFC elapsed   | delta  | refault | pswpin | zswpin | RFC 64K |
> +-----------------+---------------+---------------+--------+---------+--------+--------+---------+
> | seq64           | 4.399+/-0.100 | 4.279+/-0.216 | -2.7%  | -10.5%  | 0      | 1.000x | 3110.7  |
> | stride64_sparse | 3.103+/-0.047 | 3.119+/-0.086 | +0.5%  |  +6.2%  | 0      | 0.999x |    0.0  |
> | random64_sparse | 3.142+/-0.112 | 3.097+/-0.030 | -1.4%  |  -2.2%  | 0      | 0.999x |    0.1  |
> | random64_full   | 4.473+/-0.147 | 4.445+/-0.088 | -0.6%  |  +0.9%  | 0      | 1.000x |    0.4  |
> +-----------------+---------------+---------------+--------+---------+--------+--------+---------+
>
> This run uses a real block swap device, but the refaulted data stayed in
> zswap. It covers the all-zswap hit path with disk swap configured, not disk
> read IO.
>
> Virtio-block pressure/mixed run, zswap max_pool_percent=1,
> low-compressibility full fill:
>
> +-------------------------------+---------------+---------------+--------+---------+----------------+------------+---------+----------+
> | workload                      | base elapsed  | RFC elapsed   | delta  | refault | pswpin base/RFC | RFC zswpin | RFC 64K | fallback |
> +-------------------------------+---------------+---------------+--------+---------+----------------+------------+---------+----------+
> | seq64_full_pressure           | 5.908+/-0.195 | 5.790+/-0.235 | -2.0%  |  +3.0%  | 90258/99038    | 20327      |   0.0   | 3730     |
> | random64_sparse_full_pressure | 5.104+/-0.069 | 5.068+/-0.090 | -0.7%  |  -9.1%  |  6201/6196     |  1297      |   0.0   |    0     |
> +-------------------------------+---------------+---------------+--------+---------+----------------+------------+---------+----------+
>
> This run reaches the disk-backed path: pswpin is non-zero in both base and
> RFC. It is mainly fallback coverage. The RFC does not install 64K folios
> for these disk/mixed-heavy ranges.

Ok this results above look good. Basically, if we don't have spatial
locality in access patterns, we don't do THP zswapin. Nice.

>
> Policy matrix, virtio-block swap device present:
>
> +------------------------------+----+------+--------+--------+-------+----------+
> | case                         | pc | hint | pswpin | zswpin | zswpwb| 64K in   |
> +------------------------------+----+------+--------+--------+-------+----------+
> | pc0_seq                      | 0  | none | 0      | 99559  | 0     | 0        |
> | pc3_seq                      | 3  | none | 0      | 99498  | 0     | 0        |
> | pc4_seq                      | 4  | none | 0      | 99512  | 0     | 3109     |
> | pc5_seq                      | 5  | none | 0      | 99657  | 0     | 3113     |
> | hint_none_random_sparse      | 5  | none | 0      |  6265  | 0     | 0        |
> | hint_random_seq              | 5  | rand | 0      | 99488  | 0     | 0        |
> | mixed_seq_full               | 5  | none | 97725  | 20147  | 84    | 569      |
> | mixed_random_sparse_full     | 5  | none |  6230  |  1302  | 0     | 0        |
> +------------------------------+----+------+--------+--------+-------+----------+
>
> The pc rows show the readahead-window gate. The hint_random_seq row shows
> the explicit random hint veto. The mixed rows use a small zswap pool to
> force disk/zswap split backing; most mixed ranges are rejected, while any
> remaining 64K successes were all-zswap at the time of the fault.
>
> Kbuild pressure, zram swap, 384M memcg:
>
> +----------------------+----------+----------+--------+--------+----------+
> | setup                | base     | RFC      | delta  | zswpin | RFC 64K  |
> +----------------------+----------+----------+--------+--------+----------+
> | zram swap, 384M memcg| 2060.323 | 2047.516 | -0.6%  | 0.991x | 2797     |
> +----------------------+----------+----------+--------+--------+----------+
>
> This is a single-run zram pressure smoke. It did not show Kbuild
> regression, and the RFC run installed 64K zswap-backed folios. The result
> should not be read as a tuned-performance claim.
>
> Kbuild pressure, virtio-block swap device, 512M memcg:
>
> +-------------------------+----------+----------+--------+--------+----------+
> | setup                   | base     | RFC      | delta  | pswpin | RFC 64K  |
> +-------------------------+----------+----------+--------+--------+----------+
> | disk swap, 512M memcg   | 1420.671 | 1409.263 | -0.8%  | 0      | 7497     |
> +-------------------------+----------+----------+--------+--------+----------+
>
> This is a single-run pressure smoke. The disk-swap Kbuild run also stayed
> on the all-zswap hit path, so it is pressure coverage with a disk swap device
> present rather than proof of disk-read large swapin.

Why a single-run?

>
> Shmem smoke, tmpfs huge=always, 64K shmem mTHP:
>
> +----------------------------+---------------+---------+-------------+----------+
> | case                       | refault hint  | touched | 64K shmem   | 64K in   |
> +----------------------------+---------------+---------+-------------+----------+
> | nohint_seq                 | none          | 65536   | 4096        | 0        |
> | seq_refault_hint           | sequential    | 65536   | 4096        | 4096     |
> | random_refault_hint_sparse | random        |  4096   | 4096        | 0        |
> +----------------------------+---------------+---------+-------------+----------+
>
> That matches the current shmem producer: explicit sequential refault hints
> allow large zswap swapin; no hint and random hints do not.
>
> What this RFC does not establish
> ================================
>
> The 64K cap is deliberate, but it is not tuned. The anon PTE-young rule is
> only anon evidence. Shmem has the framework and explicit VMA hints in this
> RFC, not a page-cache locality producer. For larger orders, the anon
> producer should probably use bounded sampling instead of walking every PTE
> in a 1M or larger candidate range. The mixed-backend tests cover fallback
> behavior, but this series does not add mixed zswap/disk large IO.

The mixed IO can be deferred, but I think we should figure out a rule
to extend this hint to arbitrarily sized ranges, and preferrably shmem
too.

Re: [RFC PATCH v2 3/9] mm/zswap: support fully zswap-backed large folio loads

[Nhat Pham]

On Fri, May 29, 2026 at 5:19 AM fujunjie <fujunjie1@qq.com> wrote:
>
> zswap currently refuses large swapcache folios. That is correct for mixed
> backend ranges, but it also prevents the common swapin path from loading a
> range that is still fully backed by zswap.
>
> Teach zswap_load() to fill a locked large swapcache folio by decompressing
> each base-page entry into the matching folio offset, then flushing the
> folio once. A missing entry after zswap data has been seen is reported as
> -EAGAIN so the caller can drop the speculative large folio and retry
> order-0.
>
> The large load keeps the zswap entries in place. It is a clean speculative
> fill: until the swap slots are freed, zswap remains the backing copy if
> reclaim drops the large folio before PTEs are installed.
>
> Signed-off-by: fujunjie <fujunjie1@qq.com>
> ---
>  mm/zswap.c | 105 ++++++++++++++++++++++++++++++++++++++++++++---------
>  1 file changed, 87 insertions(+), 18 deletions(-)
>
> diff --git a/mm/zswap.c b/mm/zswap.c
> index da5297f7bd69..94ba112a2982 100644
> --- a/mm/zswap.c
> +++ b/mm/zswap.c
> @@ -15,6 +15,8 @@
>
>  #include <linux/module.h>
>  #include <linux/cpu.h>
> +#include <linux/mm.h>
> +#include <linux/huge_mm.h>
>  #include <linux/highmem.h>
>  #include <linux/slab.h>
>  #include <linux/spinlock.h>
> @@ -934,7 +936,8 @@ static bool zswap_compress(struct page *page, struct zswap_entry *entry,
>         return comp_ret == 0 && alloc_ret == 0;
>  }
>
> -static bool zswap_decompress(struct zswap_entry *entry, struct folio *folio)
> +static bool zswap_decompress(struct zswap_entry *entry, struct folio *folio,
> +                            unsigned int page_idx, bool flush_dcache)
>  {
>         struct zswap_pool *pool = entry->pool;
>         struct scatterlist input[2]; /* zsmalloc returns an SG list 1-2 entries */
> @@ -952,14 +955,15 @@ static bool zswap_decompress(struct zswap_entry *entry, struct folio *folio)
>
>                 WARN_ON_ONCE(input->length != PAGE_SIZE);
>
> -               dst = kmap_local_folio(folio, 0);
> +               dst = kmap_local_folio(folio, page_idx * PAGE_SIZE);
>                 memcpy_from_sglist(dst, input, 0, PAGE_SIZE);
>                 dlen = PAGE_SIZE;
>                 kunmap_local(dst);
> -               flush_dcache_folio(folio);
> +               if (flush_dcache)
> +                       flush_dcache_folio(folio);
>         } else {
>                 sg_init_table(&output, 1);
> -               sg_set_folio(&output, folio, PAGE_SIZE, 0);
> +               sg_set_folio(&output, folio, PAGE_SIZE, page_idx * PAGE_SIZE);
>                 acomp_request_set_params(acomp_ctx->req, input, &output,
>                                          entry->length, PAGE_SIZE);
>                 ret = crypto_acomp_decompress(acomp_ctx->req);
> @@ -1042,7 +1046,7 @@ static int zswap_writeback_entry(struct zswap_entry *entry,
>                 goto out;
>         }
>
> -       if (!zswap_decompress(entry, folio)) {
> +       if (!zswap_decompress(entry, folio, 0, true)) {
>                 ret = -EIO;
>                 goto out;
>         }
> @@ -1615,10 +1619,9 @@ enum zswap_range_state zswap_probe_range(swp_entry_t swp,
>   *  NOT marked up-to-date, so that an IO error is emitted (e.g. do_swap_page()
>   *  will SIGBUS).
>   *
> - *  -EINVAL: if the swapped out content was in zswap, but the page belongs
> - *  to a large folio, which is not supported by zswap. The folio is unlocked,
> - *  but NOT marked up-to-date, so that an IO error is emitted (e.g.
> - *  do_swap_page() will SIGBUS).
> + *  -EAGAIN: if the swapped out content belongs to a large folio, but the
> + *  range is mixed or raced with writeback. The folio remains locked so the
> + *  caller can drop the large swapcache folio and retry order-0.
>   *
>   *  -ENOENT: if the swapped out content was not in zswap. The folio remains
>   *  locked on return.
> @@ -1626,9 +1629,12 @@ enum zswap_range_state zswap_probe_range(swp_entry_t swp,
>  int zswap_load(struct folio *folio)
>  {
>         swp_entry_t swp = folio->swap;
> +       unsigned int nr_pages = folio_nr_pages(folio);
> +       unsigned int type = swp_type(swp);
>         pgoff_t offset = swp_offset(swp);
> -       struct xarray *tree = swap_zswap_tree(swp);
> +       struct xarray *tree;
>         struct zswap_entry *entry;
> +       unsigned int i;
>
>         VM_WARN_ON_ONCE(!folio_test_locked(folio));
>         VM_WARN_ON_ONCE(!folio_test_swapcache(folio));
> @@ -1636,21 +1642,84 @@ int zswap_load(struct folio *folio)
>         if (zswap_never_enabled())
>                 return -ENOENT;
>
> -       /*
> -        * Large folios should not be swapped in while zswap is being used, as
> -        * they are not properly handled. Zswap does not properly load large
> -        * folios, and a large folio may only be partially in zswap.
> -        */
> -       if (WARN_ON_ONCE(folio_test_large(folio))) {
> +       if (folio_test_large(folio)) {
> +               struct obj_cgroup *first_objcg = NULL;
> +               bool same_objcg = true;
> +               bool saw_zswap = false;
> +               bool saw_non_zswap = false;
> +
> +               /*
> +                * The locked large swapcache folio now covers the range and
> +                * conflicts with zswap writeback's order-0 swapcache allocation.
> +                * If the range is mixed or an entry disappears, retry order-0.
> +                */
> +               for (i = 0; i < nr_pages; i++) {
> +                       tree = swap_zswap_tree(swp_entry(type, offset + i));
> +                       entry = xa_load(tree, offset + i);
> +                       if (!entry) {
> +                               if (saw_zswap)
> +                                       return -EAGAIN;
> +                               saw_non_zswap = true;
> +                               continue;
> +                       }

Can we use xas_load API here instead of traversing down the tree again
and again?

> +                       if (saw_non_zswap)
> +                               return -EAGAIN;
> +
> +                       if (!saw_zswap)
> +                               first_objcg = entry->objcg;
> +                       else if (entry->objcg != first_objcg)
> +                               same_objcg = false;

Can we get different objcg at this point?

> +                       saw_zswap = true;
> +               }
> +               if (!saw_zswap)
> +                       return -ENOENT;
> +
> +               for (i = 0; i < nr_pages; i++) {
> +                       tree = swap_zswap_tree(swp_entry(type, offset + i));
> +                       entry = xa_load(tree, offset + i);
> +                       if (!entry)
> +                               return -EAGAIN;
> +
> +                       if (!zswap_decompress(entry, folio, i, false)) {
> +                               folio_unlock(folio);
> +                               return -EIO;
> +                       }
> +               }
> +
> +               flush_dcache_folio(folio);
> +               /*
> +                * Keep zswap entries until swap slots are freed. This is a clean
> +                * speculative fill; zswap remains the backing copy if reclaim
> +                * drops the large folio before PTEs are installed.
> +                */
> +               folio_mark_uptodate(folio);
> +               count_vm_events(ZSWPIN, nr_pages);
> +               count_mthp_stat(folio_order(folio), MTHP_STAT_SWPIN);
> +
> +               if (same_objcg) {
> +                       if (first_objcg)
> +                               count_objcg_events(first_objcg, ZSWPIN, nr_pages);
> +               } else {
> +                       for (i = 0; i < nr_pages; i++) {
> +                               tree = swap_zswap_tree(swp_entry(type, offset + i));
> +                               entry = xa_load(tree, offset + i);
> +                               if (WARN_ON_ONCE(!entry))
> +                                       continue;
> +                               if (entry->objcg)
> +                                       count_objcg_events(entry->objcg, ZSWPIN, 1);

xas_load() here too?


> +                       }
> +               }
> +
>                 folio_unlock(folio);
> -               return -EINVAL;
> +               return 0;
>         }

>
> +       tree = swap_zswap_tree(swp);
>         entry = xa_load(tree, offset);
>         if (!entry)
>                 return -ENOENT;
>
> -       if (!zswap_decompress(entry, folio)) {
> +       if (!zswap_decompress(entry, folio, 0, true)) {
>                 folio_unlock(folio);
>                 return -EIO;
>         }

I wonder how much of these two paths (order 0 and larger order) can be
unified...

> --
> 2.34.1
>

Re: [RFC PATCH v2 4/9] mm: admit large swapin by backend range in swapin_sync()

[Nhat Pham]

On Fri, May 29, 2026 at 5:19 AM fujunjie <fujunjie1@qq.com> wrote:
>
> A large swapin can only read one folio when the whole range has compatible
> backing. Mixed zswap/disk ranges must not reach large-folio IO, and zswap
> range probes are only snapshots.
>
> Filter the orders passed to swap_cache_alloc_folio() in swapin_sync().
> Uniform zeromap ranges and all-disk ranges keep the existing large swapin
> path. Fully zswap-backed ranges may be tried. Mixed zswap/disk ranges fall
> back before allocation.
>
> After a large swapcache folio is installed, recheck the zswap range and
> drop the fresh folio if it became mixed. Also consume -EAGAIN from
> swap_read_folio() the same way. Both cases retry order-0, where each slot
> can resolve its current backend independently.
>
> Signed-off-by: fujunjie <fujunjie1@qq.com>
> ---
>  mm/memcontrol-v1.c |   8 ++-
>  mm/memory.c        |  31 ++++++++-
>  mm/swap_state.c    | 169 ++++++++++++++++++++++++++++++++++++++++++---
>  3 files changed, 194 insertions(+), 14 deletions(-)
>
> diff --git a/mm/memcontrol-v1.c b/mm/memcontrol-v1.c
> index 765069211567..5b11b8055c66 100644
> --- a/mm/memcontrol-v1.c
> +++ b/mm/memcontrol-v1.c
> @@ -682,8 +682,8 @@ void __memcg1_swapout(struct folio *folio, struct swap_cluster_info *ci)
>   * memcg1_swapin - uncharge swap slot on swapin
>   * @folio: folio being swapped in
>   *
> - * Call this function after successfully adding the charged
> - * folio to swapcache.
> + * Call this after the charged folio has been added to swapcache and the caller
> + * is no longer going to drop it back to swapped-out state.
>   *
>   * Context: The folio has to be in swap cache and locked.
>   */
> @@ -721,7 +721,9 @@ void memcg1_swapin(struct folio *folio)
>         id = __swap_cgroup_clear(ci, swp_cluster_offset(folio->swap),
>                                  nr_pages);
>         swap_cluster_unlock(ci);
> -       mem_cgroup_uncharge_swap(id, nr_pages);
> +
> +       if (id)
> +               mem_cgroup_uncharge_swap(id, nr_pages);
>  }
>  #endif
>
> diff --git a/mm/memory.c b/mm/memory.c
> index 5a365492a9a2..d73a19692dea 100644
> --- a/mm/memory.c
> +++ b/mm/memory.c
> @@ -4538,6 +4538,24 @@ static inline bool should_try_to_free_swap(struct swap_info_struct *si,
>                 folio_ref_count(folio) == (extra_refs + folio_nr_pages(folio));
>  }
>
> +static void memcg1_swapin_retry_folio(struct folio *folio,
> +                                     struct vm_fault *vmf)
> +{
> +       if (!folio_test_large(folio) || !folio_test_swapcache(folio))
> +               return;
> +
> +       if (vmf->flags & FAULT_FLAG_RETRY_NOWAIT) {
> +               if (!folio_trylock(folio))
> +                       return;
> +       } else {
> +               folio_lock(folio);
> +       }
> +
> +       if (folio_test_large(folio) && folio_test_swapcache(folio))
> +               memcg1_swapin(folio);
> +       folio_unlock(folio);
> +}
> +
>  static vm_fault_t pte_marker_clear(struct vm_fault *vmf)
>  {
>         vmf->pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd,
> @@ -4857,8 +4875,10 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
>
>         swapcache = folio;
>         ret |= folio_lock_or_retry(folio, vmf);
> -       if (ret & VM_FAULT_RETRY)
> +       if (ret & VM_FAULT_RETRY) {
> +               memcg1_swapin_retry_folio(folio, vmf);
>                 goto out_release;
> +       }
>
>         page = folio_file_page(folio, swp_offset(entry));
>         /*
> @@ -5067,6 +5087,8 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
>         if (unlikely(folio != swapcache)) {
>                 folio_add_new_anon_rmap(folio, vma, address, RMAP_EXCLUSIVE);
>                 folio_add_lru_vma(folio, vma);
> +               if (folio_test_large(swapcache))
> +                       memcg1_swapin(swapcache);
>                 folio_put_swap(swapcache, NULL);
>         } else if (!folio_test_anon(folio)) {
>                 /*
> @@ -5076,6 +5098,8 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
>                 VM_WARN_ON_ONCE_FOLIO(folio_nr_pages(folio) != nr_pages, folio);
>                 VM_WARN_ON_ONCE_FOLIO(folio_mapped(folio), folio);
>                 folio_add_new_anon_rmap(folio, vma, address, rmap_flags);
> +               if (folio_test_large(folio))
> +                       memcg1_swapin(folio);
>                 folio_put_swap(folio, NULL);
>         } else {
>                 VM_WARN_ON_ONCE(nr_pages != 1 && nr_pages != folio_nr_pages(folio));
> @@ -5132,8 +5156,11 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
>         if (vmf->pte)
>                 pte_unmap_unlock(vmf->pte, vmf->ptl);
>  out_page:
> -       if (folio_test_swapcache(folio))
> +       if (folio_test_swapcache(folio)) {
> +               if (folio_test_large(folio))
> +                       memcg1_swapin(folio);
>                 folio_free_swap(folio);
> +       }
>         folio_unlock(folio);
>  out_release:
>         folio_put(folio);
> diff --git a/mm/swap_state.c b/mm/swap_state.c
> index d37097913b30..f03ad4832f16 100644
> --- a/mm/swap_state.c
> +++ b/mm/swap_state.c
> @@ -21,6 +21,7 @@
>  #include <linux/migrate.h>
>  #include <linux/vmalloc.h>
>  #include <linux/huge_mm.h>
> +#include <linux/zswap.h>
>  #include <linux/shmem_fs.h>
>  #include "internal.h"
>  #include "swap_table.h"
> @@ -403,7 +404,8 @@ void __swap_cache_replace_folio(struct swap_cluster_info *ci,
>  static struct folio *__swap_cache_alloc(struct swap_cluster_info *ci,
>                                         swp_entry_t targ_entry, gfp_t gfp,
>                                         unsigned int order, struct vm_fault *vmf,
> -                                       struct mempolicy *mpol, pgoff_t ilx)
> +                                       struct mempolicy *mpol, pgoff_t ilx,
> +                                       bool defer_memcg1_swapin)
>  {
>         int err;
>         swp_entry_t entry;
> @@ -466,7 +468,8 @@ static struct folio *__swap_cache_alloc(struct swap_cluster_info *ci,
>         }
>
>         /* memsw uncharges swap when folio is added to swap cache */
> -       memcg1_swapin(folio);
> +       if (!defer_memcg1_swapin || !order)
> +               memcg1_swapin(folio);
>         if (shadow)
>                 workingset_refault(folio, shadow);
>
> @@ -495,9 +498,12 @@ static struct folio *__swap_cache_alloc(struct swap_cluster_info *ci,
>   * Return: Returns the folio if allocation succeeded and folio is in the swap
>   * cache. Returns error code if failed due to race, OOM or invalid arguments.
>   */
> -struct folio *swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp,
> -                                    unsigned long orders, struct vm_fault *vmf,
> -                                    struct mempolicy *mpol, pgoff_t ilx)
> +static struct folio *__swap_cache_alloc_folio(swp_entry_t targ_entry,
> +                                             gfp_t gfp, unsigned long orders,
> +                                             struct vm_fault *vmf,
> +                                             struct mempolicy *mpol,
> +                                             pgoff_t ilx,
> +                                             bool defer_memcg1_swapin)
>  {
>         int order, err;
>         struct folio *ret;
> @@ -512,7 +518,8 @@ struct folio *swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp,
>
>         do {
>                 ret = __swap_cache_alloc(ci, targ_entry, gfp, order,
> -                                        vmf, mpol, ilx);
> +                                        vmf, mpol, ilx,
> +                                        defer_memcg1_swapin);
>                 if (!IS_ERR(ret))
>                         break;
>                 err = PTR_ERR(ret);
> @@ -525,6 +532,124 @@ struct folio *swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp,
>         return ret;
>  }
>
> +struct folio *swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp,
> +                                    unsigned long orders, struct vm_fault *vmf,
> +                                    struct mempolicy *mpol, pgoff_t ilx)
> +{
> +       return __swap_cache_alloc_folio(targ_entry, gfp, orders, vmf,
> +                                       mpol, ilx, false);
> +}
> +
> +static struct folio *swap_cache_alloc_speculative_folio(swp_entry_t targ_entry,
> +                                                       gfp_t gfp,
> +                                                       unsigned long orders,
> +                                                       struct vm_fault *vmf,
> +                                                       struct mempolicy *mpol,
> +                                                       pgoff_t ilx)
> +{
> +       /*
> +        * Speculative large swapin may drop this fresh swapcache folio and
> +        * retry order-0 after backend or page-table revalidation. Keep the
> +        * cgroup v1 memsw swap owner until the caller commits the folio.
> +        */
> +       return __swap_cache_alloc_folio(targ_entry, gfp, orders, vmf,
> +                                       mpol, ilx, true);
> +}
> +
> +static bool swapin_zeromap_same(swp_entry_t entry, unsigned int nr_pages)
> +{
> +       unsigned int ci_start = swp_cluster_offset(entry);
> +       struct swap_cluster_info *ci = __swap_entry_to_cluster(entry);
> +       bool is_zero;
> +       unsigned int i;
> +
> +       if (ci_start + nr_pages > SWAPFILE_CLUSTER) {
> +               VM_WARN_ON_ONCE(1);
> +               return false;
> +       }
> +
> +       rcu_read_lock();
> +       if (!rcu_dereference(ci->table)) {
> +               rcu_read_unlock();
> +               return true;
> +       }
> +
> +       is_zero = __swap_table_test_zero(ci, ci_start);
> +       for (i = 1; i < nr_pages; i++) {
> +               if (is_zero != __swap_table_test_zero(ci, ci_start + i)) {
> +                       rcu_read_unlock();
> +                       return false;
> +               }
> +       }
> +       rcu_read_unlock();
> +
> +       return true;
> +}
> +
> +static unsigned long swapin_admit_orders(swp_entry_t entry,
> +                                        unsigned long orders)
> +{
> +       unsigned long candidates = orders & ~BIT(0);
> +       unsigned long admitted = orders & BIT(0);
> +       int order;
> +
> +       if (!candidates)
> +               return orders;
> +
> +       while (candidates) {
> +               enum zswap_range_state state;
> +               unsigned int nr_pages;
> +               swp_entry_t range_entry;
> +               bool admit = false;
> +
> +               order = fls_long(candidates) - 1;
> +               if (order > MAX_PAGE_ORDER) {
> +                       candidates &= ~BIT(order);
> +                       continue;
> +               }
> +
> +               nr_pages = 1U << order;
> +               range_entry = swp_entry(swp_type(entry),
> +                                       round_down(swp_offset(entry), nr_pages));
> +               if (!swapin_zeromap_same(range_entry, nr_pages))
> +                       goto next;
> +
> +               state = zswap_probe_range(range_entry, nr_pages);
> +               switch (state) {
> +               case ZSWAP_RANGE_MIXED:
> +                       break;
> +               case ZSWAP_RANGE_ALL_ZSWAP:
> +               case ZSWAP_RANGE_NEVER_ENABLED:
> +               case ZSWAP_RANGE_NO_ZSWAP:
> +                       admit = true;
> +                       break;
> +               }
> +
> +next:
> +               if (admit)
> +                       admitted |= BIT(order);
> +               else
> +                       count_mthp_stat(order, MTHP_STAT_SWPIN_FALLBACK);
> +               candidates &= ~BIT(order);
> +       }
> +
> +       return admitted ? admitted : BIT(0);
> +}
> +
> +static bool zswap_needs_order0_retry(struct folio *folio)
> +{
> +       if (!folio_test_large(folio))
> +               return false;
> +
> +       /*
> +        * Admission sees only an advisory zswap snapshot. Recheck after the
> +        * large swapcache folio is installed; if the range became mixed, drop
> +        * the fresh folio before IO and let order-0 handle each slot.
> +        */
> +       return zswap_probe_range(folio->swap, folio_nr_pages(folio)) ==
> +              ZSWAP_RANGE_MIXED;
> +}
> +
>  /*
>   * If we are the only user, then try to free up the swap cache.
>   *
> @@ -634,7 +759,8 @@ static struct folio *swap_cache_read_folio(swp_entry_t entry, gfp_t gfp,
>                 folio = swap_cache_get_folio(entry);
>                 if (folio)
>                         return folio;
> -               folio = swap_cache_alloc_folio(entry, gfp, BIT(0), NULL, mpol, ilx);
> +               folio = swap_cache_alloc_folio(entry, gfp, BIT(0), NULL,
> +                                              mpol, ilx);
>         } while (PTR_ERR(folio) == -EEXIST);
>
>         if (IS_ERR_OR_NULL(folio))
> @@ -677,18 +803,43 @@ struct folio *swapin_sync(swp_entry_t entry, gfp_t gfp, unsigned long orders,
>         struct folio *folio;
>         int ret;
>
> +       orders = swapin_admit_orders(entry, orders);
> +again:
>         do {
>                 folio = swap_cache_get_folio(entry);
>                 if (folio)
>                         return folio;
> -               folio = swap_cache_alloc_folio(entry, gfp, orders, vmf, mpol, ilx);
> +               folio = swap_cache_alloc_speculative_folio(entry, gfp, orders,
> +                                                          vmf, mpol, ilx);
>         } while (PTR_ERR(folio) == -EEXIST);
>
>         if (IS_ERR(folio))
>                 return folio;
>
> +       if (zswap_needs_order0_retry(folio)) {
> +               count_mthp_stat(folio_order(folio), MTHP_STAT_SWPIN_FALLBACK);
> +               /*
> +                * The folio is newly allocated, locked, clean and not uptodate;
> +                * no data has been read into it. Removing it only restores the
> +                * swap table entries so order-0 swapin can resolve a backend
> +                * race without attempting speculative large-folio zswapin.
> +                */
> +               swap_cache_del_folio(folio);
> +               folio_unlock(folio);
> +               folio_put(folio);
> +               orders = BIT(0);
> +               goto again;
> +       }
> +
>         ret = swap_read_folio(folio, NULL);
> -       VM_WARN_ON_ONCE(ret == -EAGAIN);
> +       if (ret == -EAGAIN) {

Can this happen? After you add the entire swap range to swap cache,
backend is locked. Zswap writeback bails out if it fails to add the
page to swap cache.

I think you can just check (zswap_probe_range or wev) before
swap_read_folio(). If the range is still fully backed by zswap, you
are good to go. Otherwise, bail here immediately.

Then you don't need all the complexity with extending swap_read_folio
to handle mixed range errors (for now at least).

Re: [RFC PATCH v2 1/9] mm/zswap: expose range state for swapin policy

[Nhat Pham]

On Fri, May 29, 2026 at 5:19 AM fujunjie <fujunjie1@qq.com> wrote:
>
> Large folio swapin needs to know whether a candidate swap range is fully
> backed by zswap before it can choose an order. That decision should stay
> in common swapin code, not inside zswap.
>
> Export two zswap facts for that caller: a lockless range occupancy snapshot
> and the current zswap reclaim-pressure state. The range state is
> advisory only. Writeback or invalidation can change the backend after the
> snapshot, so users must recheck before issuing large-folio IO.
>
> Signed-off-by: fujunjie <fujunjie1@qq.com>
> ---
>  include/linux/zswap.h | 26 +++++++++++++++++++++++++
>  mm/zswap.c            | 44 +++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 70 insertions(+)
>
> diff --git a/include/linux/zswap.h b/include/linux/zswap.h
> index 30c193a1207e..8f9aee97517c 100644
> --- a/include/linux/zswap.h
> +++ b/include/linux/zswap.h
> @@ -9,6 +9,18 @@ struct lruvec;
>
>  extern atomic_long_t zswap_stored_pages;
>
> +/*
> + * Advisory zswap occupancy snapshot for a swap range. This is not a complete
> + * backend classifier; callers must recheck before depending on ALL_ZSWAP for
> + * large-folio IO.
> + */
> +enum zswap_range_state {
> +       ZSWAP_RANGE_NEVER_ENABLED,
> +       ZSWAP_RANGE_NO_ZSWAP,
> +       ZSWAP_RANGE_ALL_ZSWAP,
> +       ZSWAP_RANGE_MIXED,
> +};
> +
>  #ifdef CONFIG_ZSWAP
>
>  struct zswap_lruvec_state {
> @@ -27,6 +39,9 @@ struct zswap_lruvec_state {
>  unsigned long zswap_total_pages(void);
>  bool zswap_store(struct folio *folio);
>  int zswap_load(struct folio *folio);
> +enum zswap_range_state zswap_probe_range(swp_entry_t swp,
> +                                        unsigned int nr_pages);
> +bool zswap_pool_reclaim_pressure(void);
>  void zswap_invalidate(swp_entry_t swp);
>  int zswap_swapon(int type, unsigned long nr_pages);
>  void zswap_swapoff(int type);
> @@ -49,6 +64,17 @@ static inline int zswap_load(struct folio *folio)
>         return -ENOENT;
>  }
>
> +static inline enum zswap_range_state zswap_probe_range(swp_entry_t swp,
> +                                                      unsigned int nr_pages)
> +{
> +       return ZSWAP_RANGE_NEVER_ENABLED;
> +}
> +
> +static inline bool zswap_pool_reclaim_pressure(void)
> +{
> +       return false;
> +}
> +
>  static inline void zswap_invalidate(swp_entry_t swp) {}
>  static inline int zswap_swapon(int type, unsigned long nr_pages)
>  {
> diff --git a/mm/zswap.c b/mm/zswap.c
> index 761cd699e0a3..da5297f7bd69 100644
> --- a/mm/zswap.c
> +++ b/mm/zswap.c
> @@ -506,6 +506,19 @@ unsigned long zswap_total_pages(void)
>         return total;
>  }
>
> +/*
> + * Expose whether zswap reclaim pressure is active. This is a backend fact:
> + * zswap_check_limits() sets the state once the pool reaches the hard limit and
> + * keeps it set until the pool falls below the accept threshold.
> + */
> +bool zswap_pool_reclaim_pressure(void)
> +{
> +       if (zswap_never_enabled())
> +               return false;
> +
> +       return READ_ONCE(zswap_pool_reached_full);
> +}
> +
>  static bool zswap_check_limits(void)
>  {
>         unsigned long cur_pages = zswap_total_pages();
> @@ -1559,6 +1572,37 @@ bool zswap_store(struct folio *folio)
>         return ret;
>  }
>
> +enum zswap_range_state zswap_probe_range(swp_entry_t swp,
> +                                        unsigned int nr_pages)
> +{
> +       unsigned int type = swp_type(swp);
> +       pgoff_t offset = swp_offset(swp);
> +       bool present = false, missing = false;
> +       unsigned int i;
> +
> +       /*
> +        * This is an advisory, lockless snapshot for common swapin admission.
> +        * Callers must recheck before depending on an all-zswap range for IO:
> +        * concurrent writeback or invalidation can change the backend state.
> +        */
> +       if (zswap_never_enabled())
> +               return ZSWAP_RANGE_NEVER_ENABLED;
> +
> +       for (i = 0; i < nr_pages; i++) {
> +               struct xarray *tree = swap_zswap_tree(swp_entry(type, offset + i));
> +
> +               if (xa_load(tree, offset + i))
> +                       present = true;
> +               else
> +                       missing = true;
> +
> +               if (present && missing)
> +                       return ZSWAP_RANGE_MIXED;
> +       }

Can we use xas_load() to make this check more efficient? IIUC,
xa_load() walks the tree every time.

(We used to use a bitmap here back in frontswap days. Good times....)

Re: [RFC PATCH v2 9/9] docs: mm: update THP swapin counter descriptions

[Nhat Pham]

On Fri, May 29, 2026 at 5:19 AM fujunjie <fujunjie1@qq.com> wrote:
>
> The THP swapin counter descriptions still describe large swapin as
> coming only from non-zswap swap devices. Update them now that
> zswap-backed large folio swapin can also increment swpin.
>
> Also describe policy and backend rejection as swpin_fallback cases,
> since speculative zswap large swapin can intentionally fall back before
> doing large IO.
>
> Signed-off-by: fujunjie <fujunjie1@qq.com>
> ---
>  Documentation/admin-guide/mm/transhuge.rst | 11 ++++++-----
>  1 file changed, 6 insertions(+), 5 deletions(-)
>
> diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst
> index 23f8d13c2629..59b7a0d09243 100644
> --- a/Documentation/admin-guide/mm/transhuge.rst
> +++ b/Documentation/admin-guide/mm/transhuge.rst
> @@ -667,13 +667,14 @@ zswpout
>         piece without splitting.
>
>  swpin
> -       is incremented every time a huge page is swapped in from a non-zswap
> -       swap device in one piece.
> +       is incremented every time a huge page is swapped in from swap or
> +       zswap in one piece.
>
>  swpin_fallback
> -       is incremented if swapin fails to allocate or charge a huge page
> -       and instead falls back to using huge pages with lower orders or
> -       small pages.
> +       is incremented if swapin cannot use a huge page and instead falls
> +       back to using huge pages with lower orders or small pages. This can
> +       happen because allocation or charging fails, or because policy or
> +       backend state rejects a speculative large swapin.

I think we should add separate zswpin and zswpin fallback counter for
THP rather than overloading swpin. We already do that for zswpout vs
swpout.

Re: [RFC PATCH v2 5/9] mm: add common locality admission for zswap large swapin

[Nhat Pham]

On Fri, May 29, 2026 at 5:19 AM fujunjie <fujunjie1@qq.com> wrote:
>
> Fully zswap-backed ranges are safe to load as a large folio only when
> the caller has a reason to expect the neighbouring slots to be useful.
> Otherwise a sparse refault can turn one 4K demand fault into a 64K
> decompression and swapcache fill.
>
> Add a common admission gate for zswap-backed large swapin. The common
> layer keeps backend checks, the 64K cap, recent-refault rejection, and
> zswap reclaim-pressure rejection. It consumes a caller-provided locality
> order mask instead of looking at anon or shmem state directly.

Can you add more documentation about these policies, both in patch
changelog and in code? I'm pretty confused by the
zswap_pool_reclaim_pressure heuristics, for e.g

>
> Callers pass no locality evidence for now, so this patch only installs
> the common policy hook. Later patches add anon and shmem producers.
>
> Signed-off-by: fujunjie <fujunjie1@qq.com>
> ---
>  mm/memory.c     |   2 +-
>  mm/shmem.c      |   2 +-
>  mm/swap.h       |   8 ++--
>  mm/swap_state.c | 118 ++++++++++++++++++++++++++++++++++++++++++++----
>  4 files changed, 117 insertions(+), 13 deletions(-)
>
> diff --git a/mm/memory.c b/mm/memory.c
> index d73a19692dea..92a82008d583 100644
> --- a/mm/memory.c
> +++ b/mm/memory.c
> @@ -4849,7 +4849,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
>                 if (data_race(si->flags & SWP_SYNCHRONOUS_IO))
>                         folio = swapin_sync(entry, GFP_HIGHUSER_MOVABLE,
>                                             thp_swapin_suitable_orders(vmf) | BIT(0),
> -                                           vmf, NULL, 0);
> +                                           0, vmf, NULL, 0);
>                 else
>                         folio = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, vmf);
>
> diff --git a/mm/shmem.c b/mm/shmem.c
> index 56c23a7b15c7..fa99b48ed62b 100644
> --- a/mm/shmem.c
> +++ b/mm/shmem.c
> @@ -2031,7 +2031,7 @@ static struct folio *shmem_swap_alloc_folio(struct inode *inode,
>
>  again:
>         mpol = shmem_get_pgoff_policy(info, index, order, &ilx);
> -       folio = swapin_sync(entry, gfp, BIT(order), vmf, mpol, ilx);
> +       folio = swapin_sync(entry, gfp, BIT(order), 0, vmf, mpol, ilx);
>         mpol_cond_put(mpol);
>
>         if (!IS_ERR(folio))
> diff --git a/mm/swap.h b/mm/swap.h
> index ea7e1f3c4410..dd35a310d06d 100644
> --- a/mm/swap.h
> +++ b/mm/swap.h
> @@ -323,9 +323,10 @@ struct folio *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask,
>  struct folio *swap_cluster_readahead(swp_entry_t entry, gfp_t flag,
>                 struct mempolicy *mpol, pgoff_t ilx);
>  struct folio *swapin_readahead(swp_entry_t entry, gfp_t flag,
> -               struct vm_fault *vmf);
> +                       struct vm_fault *vmf);
>  struct folio *swapin_sync(swp_entry_t entry, gfp_t flag, unsigned long orders,
> -                          struct vm_fault *vmf, struct mempolicy *mpol, pgoff_t ilx);
> +                         unsigned long locality_orders, struct vm_fault *vmf,
> +                         struct mempolicy *mpol, pgoff_t ilx);
>  void swap_update_readahead(struct folio *folio, struct vm_area_struct *vma,
>                            unsigned long addr);
>
> @@ -418,7 +419,8 @@ static inline struct folio *swapin_readahead(swp_entry_t swp, gfp_t gfp_mask,
>
>  static inline struct folio *swapin_sync(
>         swp_entry_t entry, gfp_t flag, unsigned long orders,
> -       struct vm_fault *vmf, struct mempolicy *mpol, pgoff_t ilx)
> +       unsigned long locality_orders, struct vm_fault *vmf,
> +       struct mempolicy *mpol, pgoff_t ilx)
>  {
>         return NULL;
>  }
> diff --git a/mm/swap_state.c b/mm/swap_state.c
> index f03ad4832f16..5a4ca289009a 100644
> --- a/mm/swap_state.c
> +++ b/mm/swap_state.c
> @@ -21,6 +21,7 @@
>  #include <linux/migrate.h>
>  #include <linux/vmalloc.h>
>  #include <linux/huge_mm.h>
> +#include <linux/sizes.h>
>  #include <linux/zswap.h>
>  #include <linux/shmem_fs.h>
>  #include "internal.h"
> @@ -556,6 +557,24 @@ static struct folio *swap_cache_alloc_speculative_folio(swp_entry_t targ_entry,
>                                         mpol, ilx, true);
>  }
>
> +/*
> + * Initial conservative cap for speculative zswap large swapin. Locality
> + * evidence is supplied by the caller or by generic VMA hints; the common
> + * swapin layer keeps backend safety and pressure decisions here.
> + */
> +#define SWAPIN_ZSWAP_MAX_SIZE                  SZ_64K
> +#if PAGE_SIZE < SWAPIN_ZSWAP_MAX_SIZE
> +#define SWAPIN_ZSWAP_MAX_ORDER                 \
> +       ilog2(SWAPIN_ZSWAP_MAX_SIZE / PAGE_SIZE)
> +#else
> +#define SWAPIN_ZSWAP_MAX_ORDER                 0
> +#endif
> +
> +struct zswap_admit_ctx {
> +       bool pressure_checked;
> +       bool reclaim_pressure;
> +};
> +
>  static bool swapin_zeromap_same(swp_entry_t entry, unsigned int nr_pages)
>  {
>         unsigned int ci_start = swp_cluster_offset(entry);
> @@ -586,11 +605,84 @@ static bool swapin_zeromap_same(swp_entry_t entry, unsigned int nr_pages)
>         return true;
>  }
>
> +static bool swapin_zswap_locality(struct vm_fault *vmf, unsigned int order,
> +                                 unsigned long locality_orders)
> +{
> +       struct vm_area_struct *vma = vmf ? vmf->vma : NULL;
> +
> +       if (!order || order > MAX_PAGE_ORDER)
> +               return false;
> +
> +       if (vma && (vma->vm_flags & VM_RAND_READ))
> +               return false;

what about VM_SEQ_READ?

> +
> +       return locality_orders & BIT(order);
> +}
> +
> +static bool swapin_zswap_refaulted(swp_entry_t entry, unsigned int nr_pages)

nit: this does not seem zswap-specific. Just call it
swapin_range_refaulted or sth like that, maybe?

> +{
> +       unsigned int type = swp_type(entry);
> +       pgoff_t offset = swp_offset(entry);
> +       unsigned int i;
> +
> +       for (i = 0; i < nr_pages; i++) {
> +               bool workingset;
> +               void *shadow;
> +
> +               shadow = swap_cache_get_shadow(swp_entry(type, offset + i));

This seems inefficient. Can't we just lock the swap cluster once,
check all the shadow in the range, instead of repeatedly getting then
dropping the swap cluster lock?

> +               if (!shadow)
> +                       continue;
> +               if (workingset_test_recent(shadow, false, &workingset, false) &&
> +                   workingset)
> +                       return true;
> +       }
> +
> +       return false;
> +}
> +
> +static bool swapin_zswap_admit(swp_entry_t entry,
> +                              unsigned int order, unsigned int nr_pages,
> +                              struct vm_fault *vmf,
> +                              unsigned long locality_orders,
> +                              struct zswap_admit_ctx *ctx)
> +{
> +       if (order > SWAPIN_ZSWAP_MAX_ORDER)
> +               return false;
> +
> +       /*
> +        * Treat zswap-backed large swapin as speculative. The common layer
> +        * consumes caller-provided locality orders, but does not inspect
> +        * anon-specific PTE state or shmem-specific mapping state directly.
> +        */
> +       if (!swapin_zswap_locality(vmf, order, locality_orders))
> +               return false;
> +
> +       /*
> +        * A recent workingset refault shadow in the target range means reclaim
> +        * already saw churn there. Keep the refault path narrow instead of
> +        * speculatively decompressing neighbouring slots.
> +        */
> +       if (swapin_zswap_refaulted(entry, nr_pages))
> +               return false;

Hmm this depends. If it's just a refault coming from a speculative
read (readhead or THP (z)swpin), which is then promptly discarded,
then yeah we should backoff here. But maybe the refaulted page is
workingset one?

But yeah I guess it is better to be cautious when you are uncertain :)


> +
> +       if (!ctx->pressure_checked) {
> +               ctx->reclaim_pressure = zswap_pool_reclaim_pressure();
> +               ctx->pressure_checked = true;
> +       }

Why do we backoff if there is zswap_pool_reclaim_pressure (which only
check if the pool is full ONCE in its lifetime)? What's the rationale
here?

> +       if (ctx->reclaim_pressure)
> +               return false;
> +
> +       return true;
> +}
> +
>  static unsigned long swapin_admit_orders(swp_entry_t entry,
> -                                        unsigned long orders)
> +                                        unsigned long orders,
> +                                        struct vm_fault *vmf,
> +                                        unsigned long locality_orders)
>  {
>         unsigned long candidates = orders & ~BIT(0);
>         unsigned long admitted = orders & BIT(0);
> +       struct zswap_admit_ctx zswap_ctx = {};
>         int order;
>
>         if (!candidates)
> @@ -616,9 +708,14 @@ static unsigned long swapin_admit_orders(swp_entry_t entry,
>
>                 state = zswap_probe_range(range_entry, nr_pages);
>                 switch (state) {
> +               case ZSWAP_RANGE_ALL_ZSWAP:
> +                       admit = swapin_zswap_admit(range_entry, order,
> +                                                  nr_pages, vmf,
> +                                                  locality_orders,
> +                                                  &zswap_ctx);
> +                       break;
>                 case ZSWAP_RANGE_MIXED:
>                         break;
> -               case ZSWAP_RANGE_ALL_ZSWAP:
>                 case ZSWAP_RANGE_NEVER_ENABLED:
>                 case ZSWAP_RANGE_NO_ZSWAP:
>                         admit = true;
> @@ -769,8 +866,8 @@ static struct folio *swap_cache_read_folio(swp_entry_t entry, gfp_t gfp,
>         ret = swap_read_folio(folio, plug);
>         /*
>          * Swap readahead allocates order-0 folios. -EAGAIN is reserved for
> -        * retryable large zswap backend races and must be handled by the
> -        * synchronous common swapin path.
> +        * retryable large zswap backend races and should never escape to this
> +        * order-0 path.
>          */
>         VM_WARN_ON_ONCE(ret == -EAGAIN);
>         if (readahead) {
> @@ -786,6 +883,7 @@ static struct folio *swap_cache_read_folio(swp_entry_t entry, gfp_t gfp,
>   * @entry: swap entry indicating the target slot
>   * @gfp: memory allocation flags
>   * @orders: allocation orders
> + * @locality_orders: orders with caller-provided locality evidence
>   * @vmf: fault information
>   * @mpol: NUMA memory allocation policy to be applied
>   * @ilx: NUMA interleave index, for use only when MPOL_INTERLEAVE
> @@ -794,16 +892,20 @@ static struct folio *swap_cache_read_folio(swp_entry_t entry, gfp_t gfp,
>   * existing folio in the swap cache for @entry. This initiates the IO, too,
>   * if needed. @entry is rounded down if @orders allow large allocation.
>   *
> - * Context: Caller must ensure @entry is valid and pin the swap device with refcount.
> + * Context: Caller must ensure @entry is valid and pin the swap device with
> + * refcount.
>   * Return: Returns the folio on success, error code if failed.
>   */
> -struct folio *swapin_sync(swp_entry_t entry, gfp_t gfp, unsigned long orders,
> -                          struct vm_fault *vmf, struct mempolicy *mpol, pgoff_t ilx)
> +struct folio *swapin_sync(swp_entry_t entry, gfp_t gfp,
> +                         unsigned long orders,
> +                         unsigned long locality_orders,
> +                         struct vm_fault *vmf, struct mempolicy *mpol,
> +                         pgoff_t ilx)
>  {
>         struct folio *folio;
>         int ret;
>
> -       orders = swapin_admit_orders(entry, orders);
> +       orders = swapin_admit_orders(entry, orders, vmf, locality_orders);
>  again:
>         do {
>                 folio = swap_cache_get_folio(entry);
> --
> 2.34.1
>

Re: [RFC PATCH v2 6/9] mm: provide anon locality evidence for zswap large swapin

[Nhat Pham]

On Fri, May 29, 2026 at 5:19 AM fujunjie <fujunjie1@qq.com> wrote:
>
> The common zswap large-swapin policy needs locality evidence from
> callers before it can admit a large folio. For anonymous faults, provide
> that evidence from existing VMA hints and from the PTE young state left
> by earlier zswap-backed large swapins.
>
> Keep non-faulting PTEs old when mapping a speculative all-zswap large
> folio. A later fault can then require a dense young previous range before
> admitting another large swapin without adding VMA state.

Makes sense to me.

>
> This also removes the old zswap-enabled guard from the THP swapin
> candidate scan. The common swapin path now classifies the backend range
> and falls back to order-0 for mixed zswap/disk ranges or races.
>
> Signed-off-by: fujunjie <fujunjie1@qq.com>
> ---
>  mm/memory.c     | 234 +++++++++++++++++++++++++++++++++++++++++++-----
>  mm/swap.h       |   6 ++
>  mm/swap_state.c |  15 ++++
>  3 files changed, 235 insertions(+), 20 deletions(-)
>
> diff --git a/mm/memory.c b/mm/memory.c
> index 92a82008d583..7bbb89632000 100644
> --- a/mm/memory.c
> +++ b/mm/memory.c
> @@ -4556,6 +4556,35 @@ static void memcg1_swapin_retry_folio(struct folio *folio,
>         folio_unlock(folio);
>  }
>
> +static void set_swapin_ptes(struct vm_area_struct *vma,
> +                           unsigned long address, pte_t *ptep, pte_t pte,
> +                           unsigned int nr_pages, unsigned int fault_pte_idx,
> +                           bool fault_only_young)
> +{
> +       struct mm_struct *mm = vma->vm_mm;
> +       pte_t old_pte;
> +
> +       if (!fault_only_young || nr_pages == 1) {
> +               set_ptes(mm, address, ptep, pte, nr_pages);
> +               return;
> +       }
> +
> +       old_pte = pte_mkold(pte);
> +       if (fault_pte_idx)
> +               set_ptes(mm, address, ptep, old_pte, fault_pte_idx);
> +
> +       set_pte_at(mm, address + fault_pte_idx * PAGE_SIZE,
> +                  ptep + fault_pte_idx,
> +                  pte_mkyoung(pte_advance_pfn(pte, fault_pte_idx)));

Hmm, does this mean that without THP swapin, the faulting PTE is not
marked young, but it is marked young if there is a THP swapin. That's
a behavioral change right? Would this throw off other heuristics
relying on this bit, or any justification that this is fine?

> +
> +       fault_pte_idx++;
> +       if (fault_pte_idx < nr_pages)
> +               set_ptes(mm, address + fault_pte_idx * PAGE_SIZE,
> +                        ptep + fault_pte_idx,
> +                        pte_advance_pfn(old_pte, fault_pte_idx),
> +                        nr_pages - fault_pte_idx);
> +}
> +
>  static vm_fault_t pte_marker_clear(struct vm_fault *vmf)
>  {
>         vmf->pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd,
> @@ -4628,6 +4657,157 @@ static vm_fault_t handle_pte_marker(struct vm_fault *vmf)
>  }
>
>  #ifdef CONFIG_TRANSPARENT_HUGEPAGE
> +#define SWAPIN_ANON_YOUNG_MIN_PERCENT          75
> +#define SWAPIN_ANON_MAX_FAULT_SKIP_SHIFT       2
> +
> +static bool swapin_anon_prev_young_dense(struct vm_fault *vmf,
> +                                        unsigned int order)
> +{
> +       struct vm_area_struct *vma;
> +       unsigned int nr_pages;
> +       unsigned int threshold;
> +       unsigned long size;
> +       unsigned long base, prev, addr;
> +       struct folio *first = NULL;
> +       unsigned int present = 0;
> +       unsigned int young = 0;
> +       pmd_t *pmd;
> +       pmd_t pmdval;
> +       spinlock_t *ptl; /* protects the previous PTE range */
> +       pte_t *ptep;
> +       unsigned int i;
> +
> +       if (!IS_ENABLED(CONFIG_MMU) || !arch_has_hw_pte_young() || !vmf ||
> +           !vmf->vma || !vmf->pmd || !order || order > MAX_PAGE_ORDER)
> +               return false;
> +
> +       nr_pages = 1U << order;
> +       threshold = DIV_ROUND_UP(nr_pages *
> +                                SWAPIN_ANON_YOUNG_MIN_PERCENT, 100);
> +       size = PAGE_SIZE << order;
> +
> +       vma = vmf->vma;
> +       base = ALIGN_DOWN(vmf->address, size);
> +       if (base < size)
> +               return false;
> +
> +       prev = base - size;
> +       if (prev < vma->vm_start || prev + size > vma->vm_end)
> +               return false;
> +
> +       pmd = vmf->pmd;
> +       if ((prev & PMD_MASK) != (base & PMD_MASK)) {
> +               pmd = mm_find_pmd(vma->vm_mm, prev);
> +               if (!pmd)
> +                       return false;
> +       }
> +
> +       pmdval = pmdp_get_lockless(pmd);
> +       if (!pmd_present(pmdval) || pmd_leaf(pmdval))
> +               return false;
> +
> +       ptep = pte_offset_map_lock(vma->vm_mm, pmd, prev, &ptl);
> +       if (!ptep)
> +               return false;
> +
> +       for (i = 0, addr = prev; i < nr_pages; i++, addr += PAGE_SIZE) {
> +               struct folio *folio;
> +               pte_t pte = ptep_get(ptep + i);
> +
> +               if (!pte_present(pte))
> +                       break;
> +
> +               folio = vm_normal_folio(vma, addr, pte);
> +               if (!folio || folio_order(folio) != order)
> +                       break;
> +               if (!first)
> +                       first = folio;
> +               else if (folio != first)
> +                       break;
> +
> +               present++;
> +               if (pte_young(pte))
> +                       young++;
> +       }
> +
> +       pte_unmap_unlock(ptep, ptl);
> +       if (present != nr_pages)
> +               return false;
> +
> +       return young >= threshold;
> +}
> +
> +static bool swapin_anon_accessed_neighbour(struct vm_fault *vmf,
> +                                          unsigned int order)
> +{
> +       unsigned long size;
> +       unsigned long base;
> +       unsigned long fault_idx;
> +       unsigned long max_skip;
> +
> +       if (!vmf || !vmf->vma || !order || order > MAX_PAGE_ORDER)
> +               return false;
> +
> +       size = PAGE_SIZE << order;
> +       base = ALIGN_DOWN(vmf->address, size);
> +
> +       /*
> +        * Without a sequential hint, require prior young-density evidence and
> +        * only allow faults near the start of the candidate range.
> +        */
> +       fault_idx = (vmf->address - base) >> PAGE_SHIFT;
> +       max_skip = (1UL << order) >> SWAPIN_ANON_MAX_FAULT_SKIP_SHIFT;
> +       if (fault_idx > max_skip)
> +               return false;
> +
> +       return swapin_anon_prev_young_dense(vmf, order);
> +}
> +
> +static bool swapin_anon_fault_starts_range(struct vm_fault *vmf,
> +                                          unsigned int order)
> +{
> +       struct vm_area_struct *vma;
> +       unsigned long size;
> +       unsigned long base;
> +       unsigned long first;
> +
> +       if (!vmf || !vmf->vma || !order || order > MAX_PAGE_ORDER)
> +               return false;
> +
> +       vma = vmf->vma;
> +       size = PAGE_SIZE << order;
> +       base = ALIGN_DOWN(vmf->address, size);
> +       first = ALIGN(vma->vm_start, size);
> +
> +       return base == first && vmf->address == base &&
> +              base + size <= vma->vm_end;
> +}
> +
> +static unsigned long swapin_anon_locality_orders(struct vm_fault *vmf,
> +                                                unsigned long orders)
> +{
> +       struct vm_area_struct *vma = vmf ? vmf->vma : NULL;
> +       unsigned long locality_orders = 0;
> +       unsigned long candidates = orders & ~BIT(0);
> +       int order;
> +
> +       if (vma && (vma->vm_flags & VM_RAND_READ))
> +               return 0;
> +
> +       if (vma && (vma->vm_flags & VM_SEQ_READ))
> +               return candidates;
> +
> +       while (candidates) {
> +               order = fls_long(candidates) - 1;
> +               if (swapin_anon_fault_starts_range(vmf, order) ||
> +                   swapin_anon_accessed_neighbour(vmf, order))
> +                       locality_orders |= BIT(order);
> +               candidates &= ~BIT(order);
> +       }
> +
> +       return locality_orders;
> +}
> +
>  /*
>   * Check if the PTEs within a range are contiguous swap entries.
>   */
> @@ -4644,9 +4824,9 @@ static bool can_swapin_thp(struct vm_fault *vmf, pte_t *ptep, int nr_pages)
>         if (!pte_same(pte, pte_move_swp_offset(vmf->orig_pte, -idx)))
>                 return false;
>         /*
> -        * swap_read_folio() can't handle the case a large folio is hybridly
> -        * from different backends. And they are likely corner cases. Similar
> -        * things might be added once zswap support large folios.
> +        * swap_read_folio() can't do mixed-backend large folio IO. The common
> +        * synchronous swapin path will recheck backend state and fall back to
> +        * order-0 if a zswap/disk race makes the range mixed.
>          */
>         if (swap_pte_batch(ptep, nr_pages, pte) != nr_pages)
>                 return false;
> @@ -4693,14 +4873,6 @@ static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)
>         if (unlikely(userfaultfd_armed(vma)))
>                 return 0;
>
> -       /*
> -        * A large swapped out folio could be partially or fully in zswap. We
> -        * lack handling for such cases, so fallback to swapping in order-0
> -        * folio.
> -        */
> -       if (!zswap_never_enabled())
> -               return 0;
> -
>         entry = softleaf_from_pte(vmf->orig_pte);
>         /*
>          * Get a list of all the (large) orders below PMD_ORDER that are enabled
> @@ -4708,10 +4880,13 @@ static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)
>          */
>         orders = thp_vma_allowable_orders(vma, vma->vm_flags, TVA_PAGEFAULT,
>                                           BIT(PMD_ORDER) - 1);
> +       if (!orders)
> +               return 0;
>         orders = thp_vma_suitable_orders(vma, vmf->address, orders);
> +       if (!orders)
> +               return 0;
>         orders = thp_swap_suitable_orders(swp_offset(entry),
>                                           vmf->address, orders);
> -
>         if (!orders)
>                 return 0;
>
> @@ -4741,6 +4916,12 @@ static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)
>  {
>         return 0;
>  }
> +
> +static unsigned long swapin_anon_locality_orders(struct vm_fault *vmf,
> +                                                unsigned long orders)
> +{
> +       return 0;
> +}
>  #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
>
>  /* Sanity check that a folio is fully exclusive */
> @@ -4777,6 +4958,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
>         unsigned long page_idx;
>         unsigned long address;
>         pte_t *ptep;
> +       bool fault_only_young = false;
>
>         if (!pte_unmap_same(vmf))
>                 goto out;
> @@ -4845,13 +5027,22 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
>         if (folio)
>                 swap_update_readahead(folio, vma, vmf->address);
>         if (!folio) {
> -               /* Swapin bypasses readahead for SWP_SYNCHRONOUS_IO devices */
> -               if (data_race(si->flags & SWP_SYNCHRONOUS_IO))
> +               /*
> +                * Swapin bypasses readahead for SWP_SYNCHRONOUS_IO devices.
> +                * The swap device is pinned while checking the flag, matching
> +                * the existing fault path.
> +                */
> +               if (data_race(si->flags & SWP_SYNCHRONOUS_IO)) {
> +                       unsigned long swapin_orders = thp_swapin_suitable_orders(vmf);
> +                       unsigned long locality_orders =
> +                               swapin_anon_locality_orders(vmf, swapin_orders);
> +
>                         folio = swapin_sync(entry, GFP_HIGHUSER_MOVABLE,
> -                                           thp_swapin_suitable_orders(vmf) | BIT(0),
> -                                           0, vmf, NULL, 0);
> -               else
> +                                           swapin_orders | BIT(0),
> +                                           locality_orders, vmf, NULL, 0);
> +               } else {
>                         folio = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, vmf);
> +               }
>
>                 if (IS_ERR_OR_NULL(folio)) {
>                         /*
> @@ -5110,9 +5301,12 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
>
>         VM_BUG_ON(!folio_test_anon(folio) ||
>                         (pte_write(pte) && !PageAnonExclusive(page)));
> -       set_ptes(vma->vm_mm, address, ptep, pte, nr_pages);
> -       arch_do_swap_page_nr(vma->vm_mm, vma, address,
> -                       pte, pte, nr_pages);
> +       if (folio == swapcache && nr_pages == folio_nr_pages(folio) &&
> +           arch_has_hw_pte_young())
> +               fault_only_young = swapin_fault_only_young(folio);
> +       set_swapin_ptes(vma, address, ptep, pte, nr_pages, page_idx,
> +                       fault_only_young);
> +       arch_do_swap_page_nr(vma->vm_mm, vma, address, pte, pte, nr_pages);
>
>         /*
>          * Remove the swap entry and conditionally try to free up the swapcache.
> diff --git a/mm/swap.h b/mm/swap.h
> index dd35a310d06d..5d1c81ab49b9 100644
> --- a/mm/swap.h
> +++ b/mm/swap.h
> @@ -327,6 +327,7 @@ struct folio *swapin_readahead(swp_entry_t entry, gfp_t flag,
>  struct folio *swapin_sync(swp_entry_t entry, gfp_t flag, unsigned long orders,
>                           unsigned long locality_orders, struct vm_fault *vmf,
>                           struct mempolicy *mpol, pgoff_t ilx);
> +bool swapin_fault_only_young(struct folio *folio);
>  void swap_update_readahead(struct folio *folio, struct vm_area_struct *vma,
>                            unsigned long addr);
>
> @@ -430,6 +431,11 @@ static inline void swap_update_readahead(struct folio *folio,
>  {
>  }
>
> +static inline bool swapin_fault_only_young(struct folio *folio)
> +{
> +       return false;
> +}
> +
>  static inline int swap_writeout(struct folio *folio,
>                 struct swap_iocb **swap_plug)
>  {
> diff --git a/mm/swap_state.c b/mm/swap_state.c
> index 5a4ca289009a..80dff6a1ee65 100644
> --- a/mm/swap_state.c
> +++ b/mm/swap_state.c
> @@ -747,6 +747,21 @@ static bool zswap_needs_order0_retry(struct folio *folio)
>                ZSWAP_RANGE_MIXED;
>  }
>
> +/*
> + * A speculative large swapin may install PTEs for pages that did not fault.
> + * Keep those non-faulting PTEs old so a later anon fault can report
> + * PTE-young density as caller-provided locality evidence without storing
> + * state in the VMA.
> + */
> +bool swapin_fault_only_young(struct folio *folio)
> +{
> +       if (!folio_test_large(folio) || !folio_test_swapcache(folio))
> +               return false;
> +
> +       return zswap_probe_range(folio->swap, folio_nr_pages(folio)) ==
> +              ZSWAP_RANGE_ALL_ZSWAP;
> +}
> +
>  /*
>   * If we are the only user, then try to free up the swap cache.
>   *
> --
> 2.34.1
>

Re: [RFC PATCH v2 4/9] mm: admit large swapin by backend range in swapin_sync()

[Fujunjie]

On 5/29/2026 10:43 PM, Kairui Song wrote:
> On Fri, May 29, 2026 at 8:26 PM fujunjie <fujunjie1@qq.com> wrote:
>>
>> A large swapin can only read one folio when the whole range has compatible
>> backing. Mixed zswap/disk ranges must not reach large-folio IO, and zswap
>> range probes are only snapshots.
>>
>> Filter the orders passed to swap_cache_alloc_folio() in swapin_sync().
>> Uniform zeromap ranges and all-disk ranges keep the existing large swapin
>> path. Fully zswap-backed ranges may be tried. Mixed zswap/disk ranges fall
>> back before allocation.
>>
>> After a large swapcache folio is installed, recheck the zswap range and
>> drop the fresh folio if it became mixed. Also consume -EAGAIN from
>> swap_read_folio() the same way. Both cases retry order-0, where each slot
>> can resolve its current backend independently.
>>
>> Signed-off-by: fujunjie <fujunjie1@qq.com>
>> ---
>>  mm/memcontrol-v1.c |   8 ++-
>>  mm/memory.c        |  31 ++++++++-
>>  mm/swap_state.c    | 169 ++++++++++++++++++++++++++++++++++++++++++---
>>  3 files changed, 194 insertions(+), 14 deletions(-)
>>
>> diff --git a/mm/memcontrol-v1.c b/mm/memcontrol-v1.c
>> index 765069211567..5b11b8055c66 100644
>> --- a/mm/memcontrol-v1.c
>> +++ b/mm/memcontrol-v1.c
>> @@ -682,8 +682,8 @@ void __memcg1_swapout(struct folio *folio, struct swap_cluster_info *ci)
>>   * memcg1_swapin - uncharge swap slot on swapin
>>   * @folio: folio being swapped in
>>   *
>> - * Call this function after successfully adding the charged
>> - * folio to swapcache.
>> + * Call this after the charged folio has been added to swapcache and the caller
>> + * is no longer going to drop it back to swapped-out state.
>>   *
>>   * Context: The folio has to be in swap cache and locked.
>>   */
>> @@ -721,7 +721,9 @@ void memcg1_swapin(struct folio *folio)
>>         id = __swap_cgroup_clear(ci, swp_cluster_offset(folio->swap),
>>                                  nr_pages);
>>         swap_cluster_unlock(ci);
>> -       mem_cgroup_uncharge_swap(id, nr_pages);
>> +
>> +       if (id)
>> +               mem_cgroup_uncharge_swap(id, nr_pages);
>>  }
>>  #endif
>>
>> diff --git a/mm/memory.c b/mm/memory.c
>> index 5a365492a9a2..d73a19692dea 100644
>> --- a/mm/memory.c
>> +++ b/mm/memory.c
>> @@ -4538,6 +4538,24 @@ static inline bool should_try_to_free_swap(struct swap_info_struct *si,
>>                 folio_ref_count(folio) == (extra_refs + folio_nr_pages(folio));
>>  }
>>
>> +static void memcg1_swapin_retry_folio(struct folio *folio,
>> +                                     struct vm_fault *vmf)
>> +{
>> +       if (!folio_test_large(folio) || !folio_test_swapcache(folio))
>> +               return;
>> +
>> +       if (vmf->flags & FAULT_FLAG_RETRY_NOWAIT) {
>> +               if (!folio_trylock(folio))
>> +                       return;
>> +       } else {
>> +               folio_lock(folio);
>> +       }
>> +
>> +       if (folio_test_large(folio) && folio_test_swapcache(folio))
>> +               memcg1_swapin(folio);
>> +       folio_unlock(folio);
>> +}
>> +
>>  static vm_fault_t pte_marker_clear(struct vm_fault *vmf)
>>  {
>>         vmf->pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd,
>> @@ -4857,8 +4875,10 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
>>
>>         swapcache = folio;
>>         ret |= folio_lock_or_retry(folio, vmf);
>> -       if (ret & VM_FAULT_RETRY)
>> +       if (ret & VM_FAULT_RETRY) {
>> +               memcg1_swapin_retry_folio(folio, vmf);
>>                 goto out_release;
>> +       }
>>
>>         page = folio_file_page(folio, swp_offset(entry));
>>         /*
>> @@ -5067,6 +5087,8 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
>>         if (unlikely(folio != swapcache)) {
>>                 folio_add_new_anon_rmap(folio, vma, address, RMAP_EXCLUSIVE);
>>                 folio_add_lru_vma(folio, vma);
>> +               if (folio_test_large(swapcache))
>> +                       memcg1_swapin(swapcache);
>>                 folio_put_swap(swapcache, NULL);
>>         } else if (!folio_test_anon(folio)) {
>>                 /*
>> @@ -5076,6 +5098,8 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
>>                 VM_WARN_ON_ONCE_FOLIO(folio_nr_pages(folio) != nr_pages, folio);
>>                 VM_WARN_ON_ONCE_FOLIO(folio_mapped(folio), folio);
>>                 folio_add_new_anon_rmap(folio, vma, address, rmap_flags);
>> +               if (folio_test_large(folio))
>> +                       memcg1_swapin(folio);
>>                 folio_put_swap(folio, NULL);
>>         } else {
>>                 VM_WARN_ON_ONCE(nr_pages != 1 && nr_pages != folio_nr_pages(folio));
>> @@ -5132,8 +5156,11 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
>>         if (vmf->pte)
>>                 pte_unmap_unlock(vmf->pte, vmf->ptl);
>>  out_page:
>> -       if (folio_test_swapcache(folio))
>> +       if (folio_test_swapcache(folio)) {
>> +               if (folio_test_large(folio))
>> +                       memcg1_swapin(folio);
>>                 folio_free_swap(folio);
>> +       }
>>         folio_unlock(folio);
>>  out_release:
>>         folio_put(folio);
>> diff --git a/mm/swap_state.c b/mm/swap_state.c
>> index d37097913b30..f03ad4832f16 100644
>> --- a/mm/swap_state.c
>> +++ b/mm/swap_state.c
>> @@ -21,6 +21,7 @@
>>  #include <linux/migrate.h>
>>  #include <linux/vmalloc.h>
>>  #include <linux/huge_mm.h>
>> +#include <linux/zswap.h>
>>  #include <linux/shmem_fs.h>
>>  #include "internal.h"
>>  #include "swap_table.h"
>> @@ -403,7 +404,8 @@ void __swap_cache_replace_folio(struct swap_cluster_info *ci,
>>  static struct folio *__swap_cache_alloc(struct swap_cluster_info *ci,
>>                                         swp_entry_t targ_entry, gfp_t gfp,
>>                                         unsigned int order, struct vm_fault *vmf,
>> -                                       struct mempolicy *mpol, pgoff_t ilx)
>> +                                       struct mempolicy *mpol, pgoff_t ilx,
>> +                                       bool defer_memcg1_swapin)
> 
> Hi Fujunjie,
> 
> Thanks for the update, but this whole defer_memcg1_swapin thing is so
> ugly I don't think this is the right way at all.
> 
> If you really need this, maybe you can always defer the memcg1
> uncharge, I don't see why we need to treat large folio differently.
> This charge doesn't effect the memory pressure, the reason we uncharge
> memcg1's swap counter is to avoid long pinning swap cache holding the
> swap cache of a cgroup so the cgroup will no longer be able to swap
> out more folios. Deferring it won't hurt.

Yes, I think you are right.

I added defer_memcg1_swapin because I was still treating the freshly
allocated large swapcache folio as something that might be dropped after it
was installed, so I tried to avoid clearing the cgroup v1 swap owner too
early.

Nhat Pham also pointed out that this is probably the wrong model. Once the whole
range is covered by the large swapcache folio, zswap writeback should not be
able to turn one subslot into disk-backed state, since it has to allocate an
order-0 swapcache folio first and that should fail.

So the deferred memcg1 handling is likely self-inflicted complexity. I'll
drop this flag and rework the mixed-backend check so we fail the current
order before we need this late abort path. If any memcg1 timing issue remains
after that, I'll try to handle it with a uniform rule rather than a
large-folio-specific flag.
> 
>>  {
>>         int err;
>>         swp_entry_t entry;
>> @@ -466,7 +468,8 @@ static struct folio *__swap_cache_alloc(struct swap_cluster_info *ci,
>>         }
>>
>>         /* memsw uncharges swap when folio is added to swap cache */
>> -       memcg1_swapin(folio);
>> +       if (!defer_memcg1_swapin || !order)
>> +               memcg1_swapin(folio);
>>         if (shadow)
>>                 workingset_refault(folio, shadow);
>>
>> @@ -495,9 +498,12 @@ static struct folio *__swap_cache_alloc(struct swap_cluster_info *ci,
>>   * Return: Returns the folio if allocation succeeded and folio is in the swap
>>   * cache. Returns error code if failed due to race, OOM or invalid arguments.
>>   */
>> -struct folio *swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp,
>> -                                    unsigned long orders, struct vm_fault *vmf,
>> -                                    struct mempolicy *mpol, pgoff_t ilx)
>> +static struct folio *__swap_cache_alloc_folio(swp_entry_t targ_entry,
>> +                                             gfp_t gfp, unsigned long orders,
>> +                                             struct vm_fault *vmf,
>> +                                             struct mempolicy *mpol,
>> +                                             pgoff_t ilx,
>> +                                             bool defer_memcg1_swapin)
>>  {
>>         int order, err;
>>         struct folio *ret;
>> @@ -512,7 +518,8 @@ struct folio *swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp,
>>
>>         do {
>>                 ret = __swap_cache_alloc(ci, targ_entry, gfp, order,
>> -                                        vmf, mpol, ilx);
>> +                                        vmf, mpol, ilx,
>> +                                        defer_memcg1_swapin);
>>                 if (!IS_ERR(ret))
>>                         break;
>>                 err = PTR_ERR(ret);
>> @@ -525,6 +532,124 @@ struct folio *swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp,
>>         return ret;
>>  }
>>
>> +struct folio *swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp,
>> +                                    unsigned long orders, struct vm_fault *vmf,
>> +                                    struct mempolicy *mpol, pgoff_t ilx)
>> +{
>> +       return __swap_cache_alloc_folio(targ_entry, gfp, orders, vmf,
>> +                                       mpol, ilx, false);
>> +}
>> +
>> +static struct folio *swap_cache_alloc_speculative_folio(swp_entry_t targ_entry,
>> +                                                       gfp_t gfp,
>> +                                                       unsigned long orders,
>> +                                                       struct vm_fault *vmf,
>> +                                                       struct mempolicy *mpol,
>> +                                                       pgoff_t ilx)
>> +{
>> +       /*
>> +        * Speculative large swapin may drop this fresh swapcache folio and
>> +        * retry order-0 after backend or page-table revalidation. Keep the
>> +        * cgroup v1 memsw swap owner until the caller commits the folio.
>> +        */
>> +       return __swap_cache_alloc_folio(targ_entry, gfp, orders, vmf,
>> +                                       mpol, ilx, true);
>> +}
>> +
>> +static bool swapin_zeromap_same(swp_entry_t entry, unsigned int nr_pages)
>> +{
>> +       unsigned int ci_start = swp_cluster_offset(entry);
>> +       struct swap_cluster_info *ci = __swap_entry_to_cluster(entry);
>> +       bool is_zero;
>> +       unsigned int i;
>> +
>> +       if (ci_start + nr_pages > SWAPFILE_CLUSTER) {
>> +               VM_WARN_ON_ONCE(1);
>> +               return false;
>> +       }
>> +
>> +       rcu_read_lock();
>> +       if (!rcu_dereference(ci->table)) {
>> +               rcu_read_unlock();
>> +               return true;
>> +       }
>> +
>> +       is_zero = __swap_table_test_zero(ci, ci_start);
>> +       for (i = 1; i < nr_pages; i++) {
>> +               if (is_zero != __swap_table_test_zero(ci, ci_start + i)) {
>> +                       rcu_read_unlock();
>> +                       return false;
>> +               }
>> +       }
>> +       rcu_read_unlock();
>> +
>> +       return true;
>> +}
>> +
>> +static unsigned long swapin_admit_orders(swp_entry_t entry,
>> +                                        unsigned long orders)
> 
> And this swapin_admit_orders chunk doesn't look good either...、

Yes, this helper is doing too much.
I wanted to keep mixed zswap/disk ranges away from large-folio IO, but this
ended up mixing policy with range feasibility checks.

> 
>> +{
>> +       unsigned long candidates = orders & ~BIT(0);
>> +       unsigned long admitted = orders & BIT(0);
>> +       int order;
>> +
>> +       if (!candidates)
>> +               return orders;
>> +
>> +       while (candidates) {
>> +               enum zswap_range_state state;
>> +               unsigned int nr_pages;
>> +               swp_entry_t range_entry;
>> +               bool admit = false;
>> +
>> +               order = fls_long(candidates) - 1;
>> +               if (order > MAX_PAGE_ORDER) {
>> +                       candidates &= ~BIT(order);
>> +                       continue;
>> +               }
>> +
>> +               nr_pages = 1U << order;
>> +               range_entry = swp_entry(swp_type(entry),
>> +                                       round_down(swp_offset(entry), nr_pages));
>> +               if (!swapin_zeromap_same(range_entry, nr_pages))
>> +                       goto next;
> 
> I think you don't need to test zeromap at all? __swap_cache_alloc
> handles that already.

I am sorry for missed that this is already covered by __swap_cache_add_check().
I'll drop the explicit zeromap scan in v3 version.

> 
>> +
>> +               state = zswap_probe_range(range_entry, nr_pages);
> 
> If you just move the zswap_probe_range into __swap_cache_alloc and do
> fallback there (or maybe you can shrink the order faster), then this
> two new helpers are all redundant.
> 
>> +               switch (state) {
>> +               case ZSWAP_RANGE_MIXED:
>> +                       break;
>> +               case ZSWAP_RANGE_ALL_ZSWAP:
>> +               case ZSWAP_RANGE_NEVER_ENABLED:
>> +               case ZSWAP_RANGE_NO_ZSWAP:
>> +                       admit = true;
>> +                       break;
>> +               }
>> +
>> +next:
>> +               if (admit)
>> +                       admitted |= BIT(order);
>> +               else
>> +                       count_mthp_stat(order, MTHP_STAT_SWPIN_FALLBACK);
>> +               candidates &= ~BIT(order);
>> +       }
>> +
>> +       return admitted ? admitted : BIT(0);
>> +}
>> +
>> +static bool zswap_needs_order0_retry(struct folio *folio)
>> +{
>> +       if (!folio_test_large(folio))
>> +               return false;
>> +
>> +       /*
>> +        * Admission sees only an advisory zswap snapshot. Recheck after the
>> +        * large swapcache folio is installed; if the range became mixed, drop
>> +        * the fresh folio before IO and let order-0 handle each slot.
>> +        */
>> +       return zswap_probe_range(folio->swap, folio_nr_pages(folio)) ==
>> +              ZSWAP_RANGE_MIXED;
>> +}
>> +
> 
> Again, I think you can just probe the suitable size in
> __swap_cache_alloc directly, that way, we avoid the diverge of sync /
> non-sync device, and avoid the whole chunk making the code much
> simplier too, just like what we are alreadying doing for zero map in
> __swap_cache_alloc, or am I over simpliying it?Thanks for your review! I will try it.

Re: [RFC PATCH v2 0/9] mm: support zswap-backed large folio swapin

[Fujunjie]

On 5/30/2026 2:06 AM, Nhat Pham wrote:
> On Fri, May 29, 2026 at 5:17 AM fujunjie <fujunjie1@qq.com> wrote:
>>
>> Hi,
>>
>> This RFC explores large-folio swapin for ranges that are still fully backed
>> by zswap.
>>
>> Large swapin is currently disabled once zswap is in the picture. Anonymous
>> faults stop considering large orders after zswap has ever been enabled,
>> shmem does the same, and zswap_load() refuses large swapcache folios. That
>> keeps mixed zswap/disk cases safe, but it also loses the dense case where
>> every slot in an aligned 64K range is still resident in zswap.
>>
>> The series keeps the policy in common swapin code:
>>
>>   - zswap reports backend facts and provides the large-folio load helper.
>>   - swapin_sync() filters candidate orders by backend range.
>>   - all-disk and zeromap ranges keep the existing Kairui large-swapin path.
>>   - mixed zswap/disk ranges stay order-0.
>>   - all-zswap ranges may use a 64K folio after locality admission.
>>   - anon provides locality evidence from VMA hints and PTE young density.
>>   - shmem starts with explicit VMA-hint evidence only.
>>   - swap readahead uses its existing VMA/cluster window as locality
>>     evidence; it does not also run the anon PTE-young rule.
>>
>> The backend range probe is only a snapshot. If the backend changes after a
>> fresh large swapcache folio is allocated, the common path drops that folio
>> and falls back to order-0. zswap_load() can also return -EAGAIN for the
>> same retry path. If a late fault retry keeps the large folio in swapcache
>> instead of deleting it, the cgroup v1 memsw swap owner is committed before
>> returning.
>>
>> This is mTHP/large-folio swapin. The mappings installed by do_swap_page()
>> are still PTE mappings, not PMD mappings. The expected win is fewer faults,
>> batched PTE/rmap work, and preserving the large folio across zswapin
>> instead of rebuilding the working set as order-0 pages.
>>
>> Prior art: Usama Arif posted a related RFC on 2024-10-18:
>>
>>   mm: zswap: add support for zswapin of large folios
>>   https://lore.kernel.org/linux-mm/20241018105026.2521366-1-usamaarif642@gmail.com/
>>
>> This RFC keeps the same broad goal, but moves admission into common swapin
>> code. zswap does not decide the policy. Mixed zswap/disk ranges are
>> rejected before large IO, and the first cap is 64K.
>>
>> This is a rewrite of the RFC posted on 2026-05-08:
>>
>>   [RFC PATCH 0/5] mm: support zswap-backed anonymous large folio swapin
>>   https://lore.kernel.org/linux-mm/tencent_8B437BE4F586C162950BF71954316C1EDB05@qq.com/
>>
>> The v1 series was anonymous-only and kept too much of the policy near the
>> anon fault and zswap paths. This version is rebuilt on top of Kairui Song's
>> common swapin infrastructure. It keeps admission in common swapin code,
>> rejects mixed zswap/disk large ranges, and adds separate locality producers
>> for anon, shmem and swap readahead.
>>
>> Performance and behavior
>> ========================
>>
>> The A/B tables are 10-run measurements. Elapsed values are seconds,
>> shown as mean +/- sample standard deviation. "phase" or "refault" is the
>> measured refault subphase. "zswpin" counts zswap loads. "pswpin" counts
>> swap-ins from the real swap device; pswpin=0 means the refaults were served
>> by zswap even when a disk swap device was configured. "RFC 64K" is the mean
>> number of successful 64K swapins.
>>
>> The numbers below show where the large path is used and where it is
>> rejected.
>>
>> zram-backed zswap microbench, 64K mTHP, 8G guest:
>>
>> +-----------------+----------------+----------------+--------+--------+--------+----------+
>> | workload        | base elapsed   | RFC elapsed    | delta  | phase  | zswpin | RFC 64K  |
>> +-----------------+----------------+----------------+--------+--------+--------+----------+
>> | usama_1g        | 11.260+/-0.235 | 10.301+/-0.140 | -8.5%  | -22.2% | 1.000x | 16381.1  |
>> | nohint_seq64    |  4.398+/-0.085 |  4.025+/-0.022 | -8.5%  | -21.1% | 1.000x |  6221.1  |
>> | seqhint_seq64   |  4.283+/-0.060 |  3.948+/-0.062 | -7.8%  | -20.6% | 1.000x |  6223.5  |
>> | stride64_sparse |  3.095+/-0.051 |  3.086+/-0.025 | -0.3%  |  +5.8% | 1.002x |     1.0  |
>> | random64_sparse |  3.095+/-0.046 |  3.076+/-0.016 | -0.6%  |  +0.7% | 1.001x |     0.0  |
>> | random64_full   |  4.423+/-0.067 |  4.405+/-0.018 | -0.4%  |  +0.1% | 1.000x |     0.0  |
>> +-----------------+----------------+----------------+--------+--------+--------+----------+
>>
>> The usama_1g row follows the shape of the 2024 RFC benchmark: allocate 1G,
>> fill it with compressible per-page data, reclaim it through memory.reclaim,
>> then time the full integrity-check refault. The seq64 rows use a 512M
>> target and 768M of pressure. "sparse" touches one 4K page per 64K region, while
>> "full" touches every 4K page. "seqhint" uses MADV_SEQUENTIAL; "nohint" does
>> not.
>>
>> Virtio-block swap device present, zswap enabled:
>>
>> +-----------------+---------------+---------------+--------+---------+--------+--------+---------+
>> | workload        | base elapsed  | RFC elapsed   | delta  | refault | pswpin | zswpin | RFC 64K |
>> +-----------------+---------------+---------------+--------+---------+--------+--------+---------+
>> | seq64           | 4.399+/-0.100 | 4.279+/-0.216 | -2.7%  | -10.5%  | 0      | 1.000x | 3110.7  |
>> | stride64_sparse | 3.103+/-0.047 | 3.119+/-0.086 | +0.5%  |  +6.2%  | 0      | 0.999x |    0.0  |
>> | random64_sparse | 3.142+/-0.112 | 3.097+/-0.030 | -1.4%  |  -2.2%  | 0      | 0.999x |    0.1  |
>> | random64_full   | 4.473+/-0.147 | 4.445+/-0.088 | -0.6%  |  +0.9%  | 0      | 1.000x |    0.4  |
>> +-----------------+---------------+---------------+--------+---------+--------+--------+---------+
>>
>> This run uses a real block swap device, but the refaulted data stayed in
>> zswap. It covers the all-zswap hit path with disk swap configured, not disk
>> read IO.
>>
>> Virtio-block pressure/mixed run, zswap max_pool_percent=1,
>> low-compressibility full fill:
>>
>> +-------------------------------+---------------+---------------+--------+---------+----------------+------------+---------+----------+
>> | workload                      | base elapsed  | RFC elapsed   | delta  | refault | pswpin base/RFC | RFC zswpin | RFC 64K | fallback |
>> +-------------------------------+---------------+---------------+--------+---------+----------------+------------+---------+----------+
>> | seq64_full_pressure           | 5.908+/-0.195 | 5.790+/-0.235 | -2.0%  |  +3.0%  | 90258/99038    | 20327      |   0.0   | 3730     |
>> | random64_sparse_full_pressure | 5.104+/-0.069 | 5.068+/-0.090 | -0.7%  |  -9.1%  |  6201/6196     |  1297      |   0.0   |    0     |
>> +-------------------------------+---------------+---------------+--------+---------+----------------+------------+---------+----------+
>>
>> This run reaches the disk-backed path: pswpin is non-zero in both base and
>> RFC. It is mainly fallback coverage. The RFC does not install 64K folios
>> for these disk/mixed-heavy ranges.
> 
> Ok this results above look good. Basically, if we don't have spatial
> locality in access patterns, we don't do THP zswapin. Nice.
> 
>>
>> Policy matrix, virtio-block swap device present:
>>
>> +------------------------------+----+------+--------+--------+-------+----------+
>> | case                         | pc | hint | pswpin | zswpin | zswpwb| 64K in   |
>> +------------------------------+----+------+--------+--------+-------+----------+
>> | pc0_seq                      | 0  | none | 0      | 99559  | 0     | 0        |
>> | pc3_seq                      | 3  | none | 0      | 99498  | 0     | 0        |
>> | pc4_seq                      | 4  | none | 0      | 99512  | 0     | 3109     |
>> | pc5_seq                      | 5  | none | 0      | 99657  | 0     | 3113     |
>> | hint_none_random_sparse      | 5  | none | 0      |  6265  | 0     | 0        |
>> | hint_random_seq              | 5  | rand | 0      | 99488  | 0     | 0        |
>> | mixed_seq_full               | 5  | none | 97725  | 20147  | 84    | 569      |
>> | mixed_random_sparse_full     | 5  | none |  6230  |  1302  | 0     | 0        |
>> +------------------------------+----+------+--------+--------+-------+----------+
>>
>> The pc rows show the readahead-window gate. The hint_random_seq row shows
>> the explicit random hint veto. The mixed rows use a small zswap pool to
>> force disk/zswap split backing; most mixed ranges are rejected, while any
>> remaining 64K successes were all-zswap at the time of the fault.
>>
>> Kbuild pressure, zram swap, 384M memcg:
>>
>> +----------------------+----------+----------+--------+--------+----------+
>> | setup                | base     | RFC      | delta  | zswpin | RFC 64K  |
>> +----------------------+----------+----------+--------+--------+----------+
>> | zram swap, 384M memcg| 2060.323 | 2047.516 | -0.6%  | 0.991x | 2797     |
>> +----------------------+----------+----------+--------+--------+----------+
>>
>> This is a single-run zram pressure smoke. It did not show Kbuild
>> regression, and the RFC run installed 64K zswap-backed folios. The result
>> should not be read as a tuned-performance claim.
>>
>> Kbuild pressure, virtio-block swap device, 512M memcg:
>>
>> +-------------------------+----------+----------+--------+--------+----------+
>> | setup                   | base     | RFC      | delta  | pswpin | RFC 64K  |
>> +-------------------------+----------+----------+--------+--------+----------+
>> | disk swap, 512M memcg   | 1420.671 | 1409.263 | -0.8%  | 0      | 7497     |
>> +-------------------------+----------+----------+--------+--------+----------+
>>
>> This is a single-run pressure smoke. The disk-swap Kbuild run also stayed
>> on the all-zswap hit path, so it is pressure coverage with a disk swap device
>> present rather than proof of disk-read large swapin.
> 
> Why a single-run?

I did run Kbuild a few times while debugging the series and did not see a
significant difference either way. Because of that I only kept one fresh run
with the final tree before sending the RFC, so this should be read only as a
smoke test, not as performance evidence.

For the next version I will rerun Kbuild properly with multiple fresh
iterations and report it, so it can be used as a more reliable
performance comparison instead of just smoke coverage.

> 
>>
>> Shmem smoke, tmpfs huge=always, 64K shmem mTHP:
>>
>> +----------------------------+---------------+---------+-------------+----------+
>> | case                       | refault hint  | touched | 64K shmem   | 64K in   |
>> +----------------------------+---------------+---------+-------------+----------+
>> | nohint_seq                 | none          | 65536   | 4096        | 0        |
>> | seq_refault_hint           | sequential    | 65536   | 4096        | 4096     |
>> | random_refault_hint_sparse | random        |  4096   | 4096        | 0        |
>> +----------------------------+---------------+---------+-------------+----------+
>>
>> That matches the current shmem producer: explicit sequential refault hints
>> allow large zswap swapin; no hint and random hints do not.
>>
>> What this RFC does not establish
>> ================================
>>
>> The 64K cap is deliberate, but it is not tuned. The anon PTE-young rule is
>> only anon evidence. Shmem has the framework and explicit VMA hints in this
>> RFC, not a page-cache locality producer. For larger orders, the anon
>> producer should probably use bounded sampling instead of walking every PTE
>> in a 1M or larger candidate range. The mixed-backend tests cover fallback
>> behavior, but this series does not add mixed zswap/disk large IO.
> 
> The mixed IO can be deferred, but I think we should figure out a rule
> to extend this hint to arbitrarily sized ranges, and preferrably shmem
> too.

That makes sense.

The current 64K cap was intentionally conservative, but the locality rule is
too tied to that size. For v3 I will look at making the admission rule
order-independent, probably with bounded sampling rather than walking every
PTE for larger ranges.

For shmem, this RFC only uses explicit VMA hints, so it does not yet have a
real page-cache locality producer. I will think through how to add a shmem
producer with similar semantics, so the rule is not anon-only.

Thanks,
Fujunjie

Re: [RFC PATCH v2 4/9] mm: admit large swapin by backend range in swapin_sync()

[Fujunjie]

On 5/29/2026 10:45 PM, Kairui Song wrote:
> On Fri, May 29, 2026 at 10:43 PM Kairui Song <ryncsn@gmail.com> wrote:
>>
>> Hi Fujunjie,
>>
>> Thanks for the update, but this whole defer_memcg1_swapin thing is so
>> ugly I don't think this is the right way at all.
>>
>> If you really need this, maybe you can always defer the memcg1
> 
> Oh and I'm not saying I'm against this series or the idea, I'm just
> saying this particular design of this one patch needs some improvement
> :)

Thanks for your review! I will improve the implementation.

Re: [RFC PATCH v2 6/9] mm: provide anon locality evidence for zswap large swapin

[Fujunjie]

On 5/30/2026 3:22 AM, Nhat Pham wrote:
> On Fri, May 29, 2026 at 5:19 AM fujunjie <fujunjie1@qq.com> wrote:
>>
>> The common zswap large-swapin policy needs locality evidence from
>> callers before it can admit a large folio. For anonymous faults, provide
>> that evidence from existing VMA hints and from the PTE young state left
>> by earlier zswap-backed large swapins.
>>
>> Keep non-faulting PTEs old when mapping a speculative all-zswap large
>> folio. A later fault can then require a dense young previous range before
>> admitting another large swapin without adding VMA state.
> 
> Makes sense to me.
> 
>>
>> This also removes the old zswap-enabled guard from the THP swapin
>> candidate scan. The common swapin path now classifies the backend range
>> and falls back to order-0 for mixed zswap/disk ranges or races.
>>
>> Signed-off-by: fujunjie <fujunjie1@qq.com>
>> ---
>>  mm/memory.c     | 234 +++++++++++++++++++++++++++++++++++++++++++-----
>>  mm/swap.h       |   6 ++
>>  mm/swap_state.c |  15 ++++
>>  3 files changed, 235 insertions(+), 20 deletions(-)
>>
>> diff --git a/mm/memory.c b/mm/memory.c
>> index 92a82008d583..7bbb89632000 100644
>> --- a/mm/memory.c
>> +++ b/mm/memory.c
>> @@ -4556,6 +4556,35 @@ static void memcg1_swapin_retry_folio(struct folio *folio,
>>         folio_unlock(folio);
>>  }
>>
>> +static void set_swapin_ptes(struct vm_area_struct *vma,
>> +                           unsigned long address, pte_t *ptep, pte_t pte,
>> +                           unsigned int nr_pages, unsigned int fault_pte_idx,
>> +                           bool fault_only_young)
>> +{
>> +       struct mm_struct *mm = vma->vm_mm;
>> +       pte_t old_pte;
>> +
>> +       if (!fault_only_young || nr_pages == 1) {
>> +               set_ptes(mm, address, ptep, pte, nr_pages);
>> +               return;
>> +       }
>> +
>> +       old_pte = pte_mkold(pte);
>> +       if (fault_pte_idx)
>> +               set_ptes(mm, address, ptep, old_pte, fault_pte_idx);
>> +
>> +       set_pte_at(mm, address + fault_pte_idx * PAGE_SIZE,
>> +                  ptep + fault_pte_idx,
>> +                  pte_mkyoung(pte_advance_pfn(pte, fault_pte_idx)));
> 
> Hmm, does this mean that without THP swapin, the faulting PTE is not
> marked young, but it is marked young if there is a THP swapin. That's
> a behavioral change right? Would this throw off other heuristics
> relying on this bit, or any justification that this is fine?

Thanks.

The intent was not to make the faulting PTE behave differently from the
normal swapin path. In do_swap_page() we first build the PTE with:

		pte = mk_pte(page, vma->vm_page_prot);

and on the common architectures I checked, the normal user pgprot already
contains the accessed/young bit. For example arm64 PAGE_SHARED/PAGE_READONLY
are based on _PAGE_DEFAULT, which includes PTE_AF, and x86 user page
protections also include the accessed bit. So in practice the faulting PTE is
already young after mk_pte() there.Therefore, the default path is originally marked as young.

What I really wanted here is only to keep the speculative neighbouring PTEs
old. A large zswapin may install PTEs for pages that did not fault, and those
should not all look accessed just because mk_pte() produced a young PTE.

But, the explicit pte_mkyoung() on the faulting PTE makes this look
like THP swapin is adding a new behavior.I will try to improve it 
in a way that is less ambiguous.

Re: [RFC PATCH v2 4/9] mm: admit large swapin by backend range in swapin_sync()

[Fujunjie]

On 5/30/2026 2:34 AM, Nhat Pham wrote:
> On Fri, May 29, 2026 at 5:19 AM fujunjie <fujunjie1@qq.com> wrote:
>>
>> A large swapin can only read one folio when the whole range has compatible
>> backing. Mixed zswap/disk ranges must not reach large-folio IO, and zswap
>> range probes are only snapshots.
>>
>> Filter the orders passed to swap_cache_alloc_folio() in swapin_sync().
>> Uniform zeromap ranges and all-disk ranges keep the existing large swapin
>> path. Fully zswap-backed ranges may be tried. Mixed zswap/disk ranges fall
>> back before allocation.
>>
>> After a large swapcache folio is installed, recheck the zswap range and
>> drop the fresh folio if it became mixed. Also consume -EAGAIN from
>> swap_read_folio() the same way. Both cases retry order-0, where each slot
>> can resolve its current backend independently.
>>
>> Signed-off-by: fujunjie <fujunjie1@qq.com>
>> ---
>>  mm/memcontrol-v1.c |   8 ++-
>>  mm/memory.c        |  31 ++++++++-
>>  mm/swap_state.c    | 169 ++++++++++++++++++++++++++++++++++++++++++---
>>  3 files changed, 194 insertions(+), 14 deletions(-)
>>
>> diff --git a/mm/memcontrol-v1.c b/mm/memcontrol-v1.c
>> index 765069211567..5b11b8055c66 100644
>> --- a/mm/memcontrol-v1.c
>> +++ b/mm/memcontrol-v1.c
>> @@ -682,8 +682,8 @@ void __memcg1_swapout(struct folio *folio, struct swap_cluster_info *ci)
>>   * memcg1_swapin - uncharge swap slot on swapin
>>   * @folio: folio being swapped in
>>   *
>> - * Call this function after successfully adding the charged
>> - * folio to swapcache.
>> + * Call this after the charged folio has been added to swapcache and the caller
>> + * is no longer going to drop it back to swapped-out state.
>>   *
>>   * Context: The folio has to be in swap cache and locked.
>>   */
>> @@ -721,7 +721,9 @@ void memcg1_swapin(struct folio *folio)
>>         id = __swap_cgroup_clear(ci, swp_cluster_offset(folio->swap),
>>                                  nr_pages);
>>         swap_cluster_unlock(ci);
>> -       mem_cgroup_uncharge_swap(id, nr_pages);
>> +
>> +       if (id)
>> +               mem_cgroup_uncharge_swap(id, nr_pages);
>>  }
>>  #endif
>>
>> diff --git a/mm/memory.c b/mm/memory.c
>> index 5a365492a9a2..d73a19692dea 100644
>> --- a/mm/memory.c
>> +++ b/mm/memory.c
>> @@ -4538,6 +4538,24 @@ static inline bool should_try_to_free_swap(struct swap_info_struct *si,
>>                 folio_ref_count(folio) == (extra_refs + folio_nr_pages(folio));
>>  }
>>
>> +static void memcg1_swapin_retry_folio(struct folio *folio,
>> +                                     struct vm_fault *vmf)
>> +{
>> +       if (!folio_test_large(folio) || !folio_test_swapcache(folio))
>> +               return;
>> +
>> +       if (vmf->flags & FAULT_FLAG_RETRY_NOWAIT) {
>> +               if (!folio_trylock(folio))
>> +                       return;
>> +       } else {
>> +               folio_lock(folio);
>> +       }
>> +
>> +       if (folio_test_large(folio) && folio_test_swapcache(folio))
>> +               memcg1_swapin(folio);
>> +       folio_unlock(folio);
>> +}
>> +
>>  static vm_fault_t pte_marker_clear(struct vm_fault *vmf)
>>  {
>>         vmf->pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd,
>> @@ -4857,8 +4875,10 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
>>
>>         swapcache = folio;
>>         ret |= folio_lock_or_retry(folio, vmf);
>> -       if (ret & VM_FAULT_RETRY)
>> +       if (ret & VM_FAULT_RETRY) {
>> +               memcg1_swapin_retry_folio(folio, vmf);
>>                 goto out_release;
>> +       }
>>
>>         page = folio_file_page(folio, swp_offset(entry));
>>         /*
>> @@ -5067,6 +5087,8 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
>>         if (unlikely(folio != swapcache)) {
>>                 folio_add_new_anon_rmap(folio, vma, address, RMAP_EXCLUSIVE);
>>                 folio_add_lru_vma(folio, vma);
>> +               if (folio_test_large(swapcache))
>> +                       memcg1_swapin(swapcache);
>>                 folio_put_swap(swapcache, NULL);
>>         } else if (!folio_test_anon(folio)) {
>>                 /*
>> @@ -5076,6 +5098,8 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
>>                 VM_WARN_ON_ONCE_FOLIO(folio_nr_pages(folio) != nr_pages, folio);
>>                 VM_WARN_ON_ONCE_FOLIO(folio_mapped(folio), folio);
>>                 folio_add_new_anon_rmap(folio, vma, address, rmap_flags);
>> +               if (folio_test_large(folio))
>> +                       memcg1_swapin(folio);
>>                 folio_put_swap(folio, NULL);
>>         } else {
>>                 VM_WARN_ON_ONCE(nr_pages != 1 && nr_pages != folio_nr_pages(folio));
>> @@ -5132,8 +5156,11 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
>>         if (vmf->pte)
>>                 pte_unmap_unlock(vmf->pte, vmf->ptl);
>>  out_page:
>> -       if (folio_test_swapcache(folio))
>> +       if (folio_test_swapcache(folio)) {
>> +               if (folio_test_large(folio))
>> +                       memcg1_swapin(folio);
>>                 folio_free_swap(folio);
>> +       }
>>         folio_unlock(folio);
>>  out_release:
>>         folio_put(folio);
>> diff --git a/mm/swap_state.c b/mm/swap_state.c
>> index d37097913b30..f03ad4832f16 100644
>> --- a/mm/swap_state.c
>> +++ b/mm/swap_state.c
>> @@ -21,6 +21,7 @@
>>  #include <linux/migrate.h>
>>  #include <linux/vmalloc.h>
>>  #include <linux/huge_mm.h>
>> +#include <linux/zswap.h>
>>  #include <linux/shmem_fs.h>
>>  #include "internal.h"
>>  #include "swap_table.h"
>> @@ -403,7 +404,8 @@ void __swap_cache_replace_folio(struct swap_cluster_info *ci,
>>  static struct folio *__swap_cache_alloc(struct swap_cluster_info *ci,
>>                                         swp_entry_t targ_entry, gfp_t gfp,
>>                                         unsigned int order, struct vm_fault *vmf,
>> -                                       struct mempolicy *mpol, pgoff_t ilx)
>> +                                       struct mempolicy *mpol, pgoff_t ilx,
>> +                                       bool defer_memcg1_swapin)
>>  {
>>         int err;
>>         swp_entry_t entry;
>> @@ -466,7 +468,8 @@ static struct folio *__swap_cache_alloc(struct swap_cluster_info *ci,
>>         }
>>
>>         /* memsw uncharges swap when folio is added to swap cache */
>> -       memcg1_swapin(folio);
>> +       if (!defer_memcg1_swapin || !order)
>> +               memcg1_swapin(folio);
>>         if (shadow)
>>                 workingset_refault(folio, shadow);
>>
>> @@ -495,9 +498,12 @@ static struct folio *__swap_cache_alloc(struct swap_cluster_info *ci,
>>   * Return: Returns the folio if allocation succeeded and folio is in the swap
>>   * cache. Returns error code if failed due to race, OOM or invalid arguments.
>>   */
>> -struct folio *swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp,
>> -                                    unsigned long orders, struct vm_fault *vmf,
>> -                                    struct mempolicy *mpol, pgoff_t ilx)
>> +static struct folio *__swap_cache_alloc_folio(swp_entry_t targ_entry,
>> +                                             gfp_t gfp, unsigned long orders,
>> +                                             struct vm_fault *vmf,
>> +                                             struct mempolicy *mpol,
>> +                                             pgoff_t ilx,
>> +                                             bool defer_memcg1_swapin)
>>  {
>>         int order, err;
>>         struct folio *ret;
>> @@ -512,7 +518,8 @@ struct folio *swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp,
>>
>>         do {
>>                 ret = __swap_cache_alloc(ci, targ_entry, gfp, order,
>> -                                        vmf, mpol, ilx);
>> +                                        vmf, mpol, ilx,
>> +                                        defer_memcg1_swapin);
>>                 if (!IS_ERR(ret))
>>                         break;
>>                 err = PTR_ERR(ret);
>> @@ -525,6 +532,124 @@ struct folio *swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp,
>>         return ret;
>>  }
>>
>> +struct folio *swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp,
>> +                                    unsigned long orders, struct vm_fault *vmf,
>> +                                    struct mempolicy *mpol, pgoff_t ilx)
>> +{
>> +       return __swap_cache_alloc_folio(targ_entry, gfp, orders, vmf,
>> +                                       mpol, ilx, false);
>> +}
>> +
>> +static struct folio *swap_cache_alloc_speculative_folio(swp_entry_t targ_entry,
>> +                                                       gfp_t gfp,
>> +                                                       unsigned long orders,
>> +                                                       struct vm_fault *vmf,
>> +                                                       struct mempolicy *mpol,
>> +                                                       pgoff_t ilx)
>> +{
>> +       /*
>> +        * Speculative large swapin may drop this fresh swapcache folio and
>> +        * retry order-0 after backend or page-table revalidation. Keep the
>> +        * cgroup v1 memsw swap owner until the caller commits the folio.
>> +        */
>> +       return __swap_cache_alloc_folio(targ_entry, gfp, orders, vmf,
>> +                                       mpol, ilx, true);
>> +}
>> +
>> +static bool swapin_zeromap_same(swp_entry_t entry, unsigned int nr_pages)
>> +{
>> +       unsigned int ci_start = swp_cluster_offset(entry);
>> +       struct swap_cluster_info *ci = __swap_entry_to_cluster(entry);
>> +       bool is_zero;
>> +       unsigned int i;
>> +
>> +       if (ci_start + nr_pages > SWAPFILE_CLUSTER) {
>> +               VM_WARN_ON_ONCE(1);
>> +               return false;
>> +       }
>> +
>> +       rcu_read_lock();
>> +       if (!rcu_dereference(ci->table)) {
>> +               rcu_read_unlock();
>> +               return true;
>> +       }
>> +
>> +       is_zero = __swap_table_test_zero(ci, ci_start);
>> +       for (i = 1; i < nr_pages; i++) {
>> +               if (is_zero != __swap_table_test_zero(ci, ci_start + i)) {
>> +                       rcu_read_unlock();
>> +                       return false;
>> +               }
>> +       }
>> +       rcu_read_unlock();
>> +
>> +       return true;
>> +}
>> +
>> +static unsigned long swapin_admit_orders(swp_entry_t entry,
>> +                                        unsigned long orders)
>> +{
>> +       unsigned long candidates = orders & ~BIT(0);
>> +       unsigned long admitted = orders & BIT(0);
>> +       int order;
>> +
>> +       if (!candidates)
>> +               return orders;
>> +
>> +       while (candidates) {
>> +               enum zswap_range_state state;
>> +               unsigned int nr_pages;
>> +               swp_entry_t range_entry;
>> +               bool admit = false;
>> +
>> +               order = fls_long(candidates) - 1;
>> +               if (order > MAX_PAGE_ORDER) {
>> +                       candidates &= ~BIT(order);
>> +                       continue;
>> +               }
>> +
>> +               nr_pages = 1U << order;
>> +               range_entry = swp_entry(swp_type(entry),
>> +                                       round_down(swp_offset(entry), nr_pages));
>> +               if (!swapin_zeromap_same(range_entry, nr_pages))
>> +                       goto next;
>> +
>> +               state = zswap_probe_range(range_entry, nr_pages);
>> +               switch (state) {
>> +               case ZSWAP_RANGE_MIXED:
>> +                       break;
>> +               case ZSWAP_RANGE_ALL_ZSWAP:
>> +               case ZSWAP_RANGE_NEVER_ENABLED:
>> +               case ZSWAP_RANGE_NO_ZSWAP:
>> +                       admit = true;
>> +                       break;
>> +               }
>> +
>> +next:
>> +               if (admit)
>> +                       admitted |= BIT(order);
>> +               else
>> +                       count_mthp_stat(order, MTHP_STAT_SWPIN_FALLBACK);
>> +               candidates &= ~BIT(order);
>> +       }
>> +
>> +       return admitted ? admitted : BIT(0);
>> +}
>> +
>> +static bool zswap_needs_order0_retry(struct folio *folio)
>> +{
>> +       if (!folio_test_large(folio))
>> +               return false;
>> +
>> +       /*
>> +        * Admission sees only an advisory zswap snapshot. Recheck after the
>> +        * large swapcache folio is installed; if the range became mixed, drop
>> +        * the fresh folio before IO and let order-0 handle each slot.
>> +        */
>> +       return zswap_probe_range(folio->swap, folio_nr_pages(folio)) ==
>> +              ZSWAP_RANGE_MIXED;
>> +}
>> +
>>  /*
>>   * If we are the only user, then try to free up the swap cache.
>>   *
>> @@ -634,7 +759,8 @@ static struct folio *swap_cache_read_folio(swp_entry_t entry, gfp_t gfp,
>>                 folio = swap_cache_get_folio(entry);
>>                 if (folio)
>>                         return folio;
>> -               folio = swap_cache_alloc_folio(entry, gfp, BIT(0), NULL, mpol, ilx);
>> +               folio = swap_cache_alloc_folio(entry, gfp, BIT(0), NULL,
>> +                                              mpol, ilx);
>>         } while (PTR_ERR(folio) == -EEXIST);
>>
>>         if (IS_ERR_OR_NULL(folio))
>> @@ -677,18 +803,43 @@ struct folio *swapin_sync(swp_entry_t entry, gfp_t gfp, unsigned long orders,
>>         struct folio *folio;
>>         int ret;
>>
>> +       orders = swapin_admit_orders(entry, orders);
>> +again:
>>         do {
>>                 folio = swap_cache_get_folio(entry);
>>                 if (folio)
>>                         return folio;
>> -               folio = swap_cache_alloc_folio(entry, gfp, orders, vmf, mpol, ilx);
>> +               folio = swap_cache_alloc_speculative_folio(entry, gfp, orders,
>> +                                                          vmf, mpol, ilx);
>>         } while (PTR_ERR(folio) == -EEXIST);
>>
>>         if (IS_ERR(folio))
>>                 return folio;
>>
>> +       if (zswap_needs_order0_retry(folio)) {
>> +               count_mthp_stat(folio_order(folio), MTHP_STAT_SWPIN_FALLBACK);
>> +               /*
>> +                * The folio is newly allocated, locked, clean and not uptodate;
>> +                * no data has been read into it. Removing it only restores the
>> +                * swap table entries so order-0 swapin can resolve a backend
>> +                * race without attempting speculative large-folio zswapin.
>> +                */
>> +               swap_cache_del_folio(folio);
>> +               folio_unlock(folio);
>> +               folio_put(folio);
>> +               orders = BIT(0);
>> +               goto again;
>> +       }
>> +
>>         ret = swap_read_folio(folio, NULL);
>> -       VM_WARN_ON_ONCE(ret == -EAGAIN);
>> +       if (ret == -EAGAIN) {
> 
> Can this happen? After you add the entire swap range to swap cache,
> backend is locked. Zswap writeback bails out if it fails to add the
> page to swap cache.
> 
> I think you can just check (zswap_probe_range or wev) before
> swap_read_folio(). If the range is still fully backed by zswap, you
> are good to go. Otherwise, bail here immediately.
> 
> Then you don't need all the complexity with extending swap_read_folio
> to handle mixed range errors (for now at least).

Yes, I think you are right.

I missed that property of zswap writeback. Once the whole range is covered by
the large swapcache folio, writeback should not be able to move a subslot to
disk because it has to allocate an order-0 swapcache folio first, and that
should fail.

Sorry for adding this extra complexity. I will rework this in a more unified way for the
next version. 

Re: [RFC PATCH v2 9/9] docs: mm: update THP swapin counter descriptions

[Fujunjie]

On 5/30/2026 2:37 AM, Nhat Pham wrote:
> On Fri, May 29, 2026 at 5:19 AM fujunjie <fujunjie1@qq.com> wrote:
>>
>> The THP swapin counter descriptions still describe large swapin as
>> coming only from non-zswap swap devices. Update them now that
>> zswap-backed large folio swapin can also increment swpin.
>>
>> Also describe policy and backend rejection as swpin_fallback cases,
>> since speculative zswap large swapin can intentionally fall back before
>> doing large IO.
>>
>> Signed-off-by: fujunjie <fujunjie1@qq.com>
>> ---
>>  Documentation/admin-guide/mm/transhuge.rst | 11 ++++++-----
>>  1 file changed, 6 insertions(+), 5 deletions(-)
>>
>> diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst
>> index 23f8d13c2629..59b7a0d09243 100644
>> --- a/Documentation/admin-guide/mm/transhuge.rst
>> +++ b/Documentation/admin-guide/mm/transhuge.rst
>> @@ -667,13 +667,14 @@ zswpout
>>         piece without splitting.
>>
>>  swpin
>> -       is incremented every time a huge page is swapped in from a non-zswap
>> -       swap device in one piece.
>> +       is incremented every time a huge page is swapped in from swap or
>> +       zswap in one piece.
>>
>>  swpin_fallback
>> -       is incremented if swapin fails to allocate or charge a huge page
>> -       and instead falls back to using huge pages with lower orders or
>> -       small pages.
>> +       is incremented if swapin cannot use a huge page and instead falls
>> +       back to using huge pages with lower orders or small pages. This can
>> +       happen because allocation or charging fails, or because policy or
>> +       backend state rejects a speculative large swapin.
> 
> I think we should add separate zswpin and zswpin fallback counter for
> THP rather than overloading swpin. We already do that for zswpout vs
> swpout.

that makes sense.

Re: [RFC PATCH v2 1/9] mm/zswap: expose range state for swapin policy

[Fujunjie]

On 5/30/2026 2:35 AM, Nhat Pham wrote:
> On Fri, May 29, 2026 at 5:19 AM fujunjie <fujunjie1@qq.com> wrote:
>>
>> Large folio swapin needs to know whether a candidate swap range is fully
>> backed by zswap before it can choose an order. That decision should stay
>> in common swapin code, not inside zswap.
>>
>> Export two zswap facts for that caller: a lockless range occupancy snapshot
>> and the current zswap reclaim-pressure state. The range state is
>> advisory only. Writeback or invalidation can change the backend after the
>> snapshot, so users must recheck before issuing large-folio IO.
>>
>> Signed-off-by: fujunjie <fujunjie1@qq.com>
>> ---
>>  include/linux/zswap.h | 26 +++++++++++++++++++++++++
>>  mm/zswap.c            | 44 +++++++++++++++++++++++++++++++++++++++++++
>>  2 files changed, 70 insertions(+)
>>
>> diff --git a/include/linux/zswap.h b/include/linux/zswap.h
>> index 30c193a1207e..8f9aee97517c 100644
>> --- a/include/linux/zswap.h
>> +++ b/include/linux/zswap.h
>> @@ -9,6 +9,18 @@ struct lruvec;
>>
>>  extern atomic_long_t zswap_stored_pages;
>>
>> +/*
>> + * Advisory zswap occupancy snapshot for a swap range. This is not a complete
>> + * backend classifier; callers must recheck before depending on ALL_ZSWAP for
>> + * large-folio IO.
>> + */
>> +enum zswap_range_state {
>> +       ZSWAP_RANGE_NEVER_ENABLED,
>> +       ZSWAP_RANGE_NO_ZSWAP,
>> +       ZSWAP_RANGE_ALL_ZSWAP,
>> +       ZSWAP_RANGE_MIXED,
>> +};
>> +
>>  #ifdef CONFIG_ZSWAP
>>
>>  struct zswap_lruvec_state {
>> @@ -27,6 +39,9 @@ struct zswap_lruvec_state {
>>  unsigned long zswap_total_pages(void);
>>  bool zswap_store(struct folio *folio);
>>  int zswap_load(struct folio *folio);
>> +enum zswap_range_state zswap_probe_range(swp_entry_t swp,
>> +                                        unsigned int nr_pages);
>> +bool zswap_pool_reclaim_pressure(void);
>>  void zswap_invalidate(swp_entry_t swp);
>>  int zswap_swapon(int type, unsigned long nr_pages);
>>  void zswap_swapoff(int type);
>> @@ -49,6 +64,17 @@ static inline int zswap_load(struct folio *folio)
>>         return -ENOENT;
>>  }
>>
>> +static inline enum zswap_range_state zswap_probe_range(swp_entry_t swp,
>> +                                                      unsigned int nr_pages)
>> +{
>> +       return ZSWAP_RANGE_NEVER_ENABLED;
>> +}
>> +
>> +static inline bool zswap_pool_reclaim_pressure(void)
>> +{
>> +       return false;
>> +}
>> +
>>  static inline void zswap_invalidate(swp_entry_t swp) {}
>>  static inline int zswap_swapon(int type, unsigned long nr_pages)
>>  {
>> diff --git a/mm/zswap.c b/mm/zswap.c
>> index 761cd699e0a3..da5297f7bd69 100644
>> --- a/mm/zswap.c
>> +++ b/mm/zswap.c
>> @@ -506,6 +506,19 @@ unsigned long zswap_total_pages(void)
>>         return total;
>>  }
>>
>> +/*
>> + * Expose whether zswap reclaim pressure is active. This is a backend fact:
>> + * zswap_check_limits() sets the state once the pool reaches the hard limit and
>> + * keeps it set until the pool falls below the accept threshold.
>> + */
>> +bool zswap_pool_reclaim_pressure(void)
>> +{
>> +       if (zswap_never_enabled())
>> +               return false;
>> +
>> +       return READ_ONCE(zswap_pool_reached_full);
>> +}
>> +
>>  static bool zswap_check_limits(void)
>>  {
>>         unsigned long cur_pages = zswap_total_pages();
>> @@ -1559,6 +1572,37 @@ bool zswap_store(struct folio *folio)
>>         return ret;
>>  }
>>
>> +enum zswap_range_state zswap_probe_range(swp_entry_t swp,
>> +                                        unsigned int nr_pages)
>> +{
>> +       unsigned int type = swp_type(swp);
>> +       pgoff_t offset = swp_offset(swp);
>> +       bool present = false, missing = false;
>> +       unsigned int i;
>> +
>> +       /*
>> +        * This is an advisory, lockless snapshot for common swapin admission.
>> +        * Callers must recheck before depending on an all-zswap range for IO:
>> +        * concurrent writeback or invalidation can change the backend state.
>> +        */
>> +       if (zswap_never_enabled())
>> +               return ZSWAP_RANGE_NEVER_ENABLED;
>> +
>> +       for (i = 0; i < nr_pages; i++) {
>> +               struct xarray *tree = swap_zswap_tree(swp_entry(type, offset + i));
>> +
>> +               if (xa_load(tree, offset + i))
>> +                       present = true;
>> +               else
>> +                       missing = true;
>> +
>> +               if (present && missing)
>> +                       return ZSWAP_RANGE_MIXED;
>> +       }
> 
> Can we use xas_load() to make this check more efficient? IIUC,
> xa_load() walks the tree every time.
> 
> (We used to use a bitmap here back in frontswap days. Good times....)

Thanks for your review.

I'll switch this to xas_load() in the v3 version.

Re: [RFC PATCH v2 3/9] mm/zswap: support fully zswap-backed large folio loads

[Fujunjie]

On 5/30/2026 2:25 AM, Nhat Pham wrote:
> On Fri, May 29, 2026 at 5:19 AM fujunjie <fujunjie1@qq.com> wrote:
>>
>> zswap currently refuses large swapcache folios. That is correct for mixed
>> backend ranges, but it also prevents the common swapin path from loading a
>> range that is still fully backed by zswap.
>>
>> Teach zswap_load() to fill a locked large swapcache folio by decompressing
>> each base-page entry into the matching folio offset, then flushing the
>> folio once. A missing entry after zswap data has been seen is reported as
>> -EAGAIN so the caller can drop the speculative large folio and retry
>> order-0.
>>
>> The large load keeps the zswap entries in place. It is a clean speculative
>> fill: until the swap slots are freed, zswap remains the backing copy if
>> reclaim drops the large folio before PTEs are installed.
>>
>> Signed-off-by: fujunjie <fujunjie1@qq.com>
>> ---
>>  mm/zswap.c | 105 ++++++++++++++++++++++++++++++++++++++++++++---------
>>  1 file changed, 87 insertions(+), 18 deletions(-)
>>
>> diff --git a/mm/zswap.c b/mm/zswap.c
>> index da5297f7bd69..94ba112a2982 100644
>> --- a/mm/zswap.c
>> +++ b/mm/zswap.c
>> @@ -15,6 +15,8 @@
>>
>>  #include <linux/module.h>
>>  #include <linux/cpu.h>
>> +#include <linux/mm.h>
>> +#include <linux/huge_mm.h>
>>  #include <linux/highmem.h>
>>  #include <linux/slab.h>
>>  #include <linux/spinlock.h>
>> @@ -934,7 +936,8 @@ static bool zswap_compress(struct page *page, struct zswap_entry *entry,
>>         return comp_ret == 0 && alloc_ret == 0;
>>  }
>>
>> -static bool zswap_decompress(struct zswap_entry *entry, struct folio *folio)
>> +static bool zswap_decompress(struct zswap_entry *entry, struct folio *folio,
>> +                            unsigned int page_idx, bool flush_dcache)
>>  {
>>         struct zswap_pool *pool = entry->pool;
>>         struct scatterlist input[2]; /* zsmalloc returns an SG list 1-2 entries */
>> @@ -952,14 +955,15 @@ static bool zswap_decompress(struct zswap_entry *entry, struct folio *folio)
>>
>>                 WARN_ON_ONCE(input->length != PAGE_SIZE);
>>
>> -               dst = kmap_local_folio(folio, 0);
>> +               dst = kmap_local_folio(folio, page_idx * PAGE_SIZE);
>>                 memcpy_from_sglist(dst, input, 0, PAGE_SIZE);
>>                 dlen = PAGE_SIZE;
>>                 kunmap_local(dst);
>> -               flush_dcache_folio(folio);
>> +               if (flush_dcache)
>> +                       flush_dcache_folio(folio);
>>         } else {
>>                 sg_init_table(&output, 1);
>> -               sg_set_folio(&output, folio, PAGE_SIZE, 0);
>> +               sg_set_folio(&output, folio, PAGE_SIZE, page_idx * PAGE_SIZE);
>>                 acomp_request_set_params(acomp_ctx->req, input, &output,
>>                                          entry->length, PAGE_SIZE);
>>                 ret = crypto_acomp_decompress(acomp_ctx->req);
>> @@ -1042,7 +1046,7 @@ static int zswap_writeback_entry(struct zswap_entry *entry,
>>                 goto out;
>>         }
>>
>> -       if (!zswap_decompress(entry, folio)) {
>> +       if (!zswap_decompress(entry, folio, 0, true)) {
>>                 ret = -EIO;
>>                 goto out;
>>         }
>> @@ -1615,10 +1619,9 @@ enum zswap_range_state zswap_probe_range(swp_entry_t swp,
>>   *  NOT marked up-to-date, so that an IO error is emitted (e.g. do_swap_page()
>>   *  will SIGBUS).
>>   *
>> - *  -EINVAL: if the swapped out content was in zswap, but the page belongs
>> - *  to a large folio, which is not supported by zswap. The folio is unlocked,
>> - *  but NOT marked up-to-date, so that an IO error is emitted (e.g.
>> - *  do_swap_page() will SIGBUS).
>> + *  -EAGAIN: if the swapped out content belongs to a large folio, but the
>> + *  range is mixed or raced with writeback. The folio remains locked so the
>> + *  caller can drop the large swapcache folio and retry order-0.
>>   *
>>   *  -ENOENT: if the swapped out content was not in zswap. The folio remains
>>   *  locked on return.
>> @@ -1626,9 +1629,12 @@ enum zswap_range_state zswap_probe_range(swp_entry_t swp,
>>  int zswap_load(struct folio *folio)
>>  {
>>         swp_entry_t swp = folio->swap;
>> +       unsigned int nr_pages = folio_nr_pages(folio);
>> +       unsigned int type = swp_type(swp);
>>         pgoff_t offset = swp_offset(swp);
>> -       struct xarray *tree = swap_zswap_tree(swp);
>> +       struct xarray *tree;
>>         struct zswap_entry *entry;
>> +       unsigned int i;
>>
>>         VM_WARN_ON_ONCE(!folio_test_locked(folio));
>>         VM_WARN_ON_ONCE(!folio_test_swapcache(folio));
>> @@ -1636,21 +1642,84 @@ int zswap_load(struct folio *folio)
>>         if (zswap_never_enabled())
>>                 return -ENOENT;
>>
>> -       /*
>> -        * Large folios should not be swapped in while zswap is being used, as
>> -        * they are not properly handled. Zswap does not properly load large
>> -        * folios, and a large folio may only be partially in zswap.
>> -        */
>> -       if (WARN_ON_ONCE(folio_test_large(folio))) {
>> +       if (folio_test_large(folio)) {
>> +               struct obj_cgroup *first_objcg = NULL;
>> +               bool same_objcg = true;
>> +               bool saw_zswap = false;
>> +               bool saw_non_zswap = false;
>> +
>> +               /*
>> +                * The locked large swapcache folio now covers the range and
>> +                * conflicts with zswap writeback's order-0 swapcache allocation.
>> +                * If the range is mixed or an entry disappears, retry order-0.
>> +                */
>> +               for (i = 0; i < nr_pages; i++) {
>> +                       tree = swap_zswap_tree(swp_entry(type, offset + i));
>> +                       entry = xa_load(tree, offset + i);
>> +                       if (!entry) {
>> +                               if (saw_zswap)
>> +                                       return -EAGAIN;
>> +                               saw_non_zswap = true;
>> +                               continue;
>> +                       }
> 
> Can we use xas_load API here instead of traversing down the tree again
> and again?

I'll rework it to use xas_load(), while handling zswap tree boundaries correctly.

> 
>> +                       if (saw_non_zswap)
>> +                               return -EAGAIN;
>> +
>> +                       if (!saw_zswap)
>> +                               first_objcg = entry->objcg;
>> +                       else if (entry->objcg != first_objcg)
>> +                               same_objcg = false;
> 
> Can we get different objcg at this point?

The objcg pointers can be different in principle, for example if
the range is assembled from entries that came from different per-node objcgs
of the same memcg.

But for this accounting path, count_objcg_events() ultimately charges the
event to obj_cgroup_memcg(entry->objcg). Since the large swapcache allocation
has already checked compatible swap ownership for the range, the final memcg
accounting target should be the same even if the objcg pointers differ.

I will simplify this in v3 and avoid the extra objcg equality pass.

> 
>> +                       saw_zswap = true;
>> +               }
>> +               if (!saw_zswap)
>> +                       return -ENOENT;
>> +
>> +               for (i = 0; i < nr_pages; i++) {
>> +                       tree = swap_zswap_tree(swp_entry(type, offset + i));
>> +                       entry = xa_load(tree, offset + i);
>> +                       if (!entry)
>> +                               return -EAGAIN;
>> +
>> +                       if (!zswap_decompress(entry, folio, i, false)) {
>> +                               folio_unlock(folio);
>> +                               return -EIO;
>> +                       }
>> +               }
>> +
>> +               flush_dcache_folio(folio);
>> +               /*
>> +                * Keep zswap entries until swap slots are freed. This is a clean
>> +                * speculative fill; zswap remains the backing copy if reclaim
>> +                * drops the large folio before PTEs are installed.
>> +                */
>> +               folio_mark_uptodate(folio);
>> +               count_vm_events(ZSWPIN, nr_pages);
>> +               count_mthp_stat(folio_order(folio), MTHP_STAT_SWPIN);
>> +
>> +               if (same_objcg) {
>> +                       if (first_objcg)
>> +                               count_objcg_events(first_objcg, ZSWPIN, nr_pages);
>> +               } else {
>> +                       for (i = 0; i < nr_pages; i++) {
>> +                               tree = swap_zswap_tree(swp_entry(type, offset + i));
>> +                               entry = xa_load(tree, offset + i);
>> +                               if (WARN_ON_ONCE(!entry))
>> +                                       continue;
>> +                               if (entry->objcg)
>> +                                       count_objcg_events(entry->objcg, ZSWPIN, 1);
> 
> xas_load() here too?

Yes, same issue here. 

> 
> 
>> +                       }
>> +               }
>> +
>>                 folio_unlock(folio);
>> -               return -EINVAL;
>> +               return 0;
>>         }
> 
>>
>> +       tree = swap_zswap_tree(swp);
>>         entry = xa_load(tree, offset);
>>         if (!entry)
>>                 return -ENOENT;
>>
>> -       if (!zswap_decompress(entry, folio)) {
>> +       if (!zswap_decompress(entry, folio, 0, true)) {
>>                 folio_unlock(folio);
>>                 return -EIO;
>>         }
> 
> I wonder how much of these two paths (order 0 and larger order) can be
> unified...

I think more of this can be unified than this version does.

I split the paths this way because I treated the large-folio load as a
speculative fill and kept the zswap entries as the backing copy. But with
your point that an installed large swapcache folio should block zswap
writeback from turning the range mixed, I should revisit that completion rule
instead of baking it into a separate path.

For the v3 version I will try to collapse the common load path. If the large-folio
case still needs different entry lifetime rules, I will make that distinction
explicit.

> 
>> --
>> 2.34.1
>>

Re: [RFC PATCH v2 5/9] mm: add common locality admission for zswap large swapin

[Fujunjie]

On 5/30/2026 3:00 AM, Nhat Pham wrote:
> On Fri, May 29, 2026 at 5:19 AM fujunjie <fujunjie1@qq.com> wrote:
>>
>> Fully zswap-backed ranges are safe to load as a large folio only when
>> the caller has a reason to expect the neighbouring slots to be useful.
>> Otherwise a sparse refault can turn one 4K demand fault into a 64K
>> decompression and swapcache fill.
>>
>> Add a common admission gate for zswap-backed large swapin. The common
>> layer keeps backend checks, the 64K cap, recent-refault rejection, and
>> zswap reclaim-pressure rejection. It consumes a caller-provided locality
>> order mask instead of looking at anon or shmem state directly.
> 
> Can you add more documentation about these policies, both in patch
> changelog and in code? I'm pretty confused by the
> zswap_pool_reclaim_pressure heuristics, for e.g
> 
You're right, this should have been documented much better, and I think the
heuristic itself needs another look.

The intent was not to block demand swapin when zswap is under pressure. It was
only meant to block speculative large zswapin. Even after locality_orders
admits an order, that is still only evidence that the neighbouring slots are
likely useful; it does not prove that the whole large folio will be consumed
soon enough to justify the extra resident memory under pressure.

In this RFC, the cost is also higher because of my late mixed-backend fallback
design. I kept the zswap entries after filling the large swapcache folio, so
that the fresh large folio could still be dropped and retried as order-0 if a
late mixed-backend race was detected. With that design, a large zswapin does
not immediately reduce the zswap pool: the uncompressed large swapcache folio
and the compressed zswap entries can exist at the same time until the swap
slots are freed.

I used zswap_pool_reclaim_pressure() as a rough signal for avoiding that extra
speculative expansion. My assumption was that a zswap pool that has reached
its limit is often correlated with memory pressure. But that is not a strict
relationship, and I did not validate this heuristic rigorously. Sorry, this
was too under-explained and probably too ad hoc in this RFC.

Your earlier point also makes me think the design is over-defensive. Once the
large swapcache folio is installed, zswap writeback should not be able to turn
one slot in the range into disk-backed state, since it first has to allocate
an order-0 swapcache folio. For v3 I will revisit the completion rule first:
drop the late mixed-race -EAGAIN path, decide whether a successful large load
should consume zswap entries like the order-0 path, and then either remove
this pressure heuristic or keep it only with clearer documentation and
specific experiments.

>>
>> Callers pass no locality evidence for now, so this patch only installs
>> the common policy hook. Later patches add anon and shmem producers.
>>
>> Signed-off-by: fujunjie <fujunjie1@qq.com>
>> ---
>>  mm/memory.c     |   2 +-
>>  mm/shmem.c      |   2 +-
>>  mm/swap.h       |   8 ++--
>>  mm/swap_state.c | 118 ++++++++++++++++++++++++++++++++++++++++++++----
>>  4 files changed, 117 insertions(+), 13 deletions(-)
>>
>> diff --git a/mm/memory.c b/mm/memory.c
>> index d73a19692dea..92a82008d583 100644
>> --- a/mm/memory.c
>> +++ b/mm/memory.c
>> @@ -4849,7 +4849,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
>>                 if (data_race(si->flags & SWP_SYNCHRONOUS_IO))
>>                         folio = swapin_sync(entry, GFP_HIGHUSER_MOVABLE,
>>                                             thp_swapin_suitable_orders(vmf) | BIT(0),
>> -                                           vmf, NULL, 0);
>> +                                           0, vmf, NULL, 0);
>>                 else
>>                         folio = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, vmf);
>>
>> diff --git a/mm/shmem.c b/mm/shmem.c
>> index 56c23a7b15c7..fa99b48ed62b 100644
>> --- a/mm/shmem.c
>> +++ b/mm/shmem.c
>> @@ -2031,7 +2031,7 @@ static struct folio *shmem_swap_alloc_folio(struct inode *inode,
>>
>>  again:
>>         mpol = shmem_get_pgoff_policy(info, index, order, &ilx);
>> -       folio = swapin_sync(entry, gfp, BIT(order), vmf, mpol, ilx);
>> +       folio = swapin_sync(entry, gfp, BIT(order), 0, vmf, mpol, ilx);
>>         mpol_cond_put(mpol);
>>
>>         if (!IS_ERR(folio))
>> diff --git a/mm/swap.h b/mm/swap.h
>> index ea7e1f3c4410..dd35a310d06d 100644
>> --- a/mm/swap.h
>> +++ b/mm/swap.h
>> @@ -323,9 +323,10 @@ struct folio *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask,
>>  struct folio *swap_cluster_readahead(swp_entry_t entry, gfp_t flag,
>>                 struct mempolicy *mpol, pgoff_t ilx);
>>  struct folio *swapin_readahead(swp_entry_t entry, gfp_t flag,
>> -               struct vm_fault *vmf);
>> +                       struct vm_fault *vmf);
>>  struct folio *swapin_sync(swp_entry_t entry, gfp_t flag, unsigned long orders,
>> -                          struct vm_fault *vmf, struct mempolicy *mpol, pgoff_t ilx);
>> +                         unsigned long locality_orders, struct vm_fault *vmf,
>> +                         struct mempolicy *mpol, pgoff_t ilx);
>>  void swap_update_readahead(struct folio *folio, struct vm_area_struct *vma,
>>                            unsigned long addr);
>>
>> @@ -418,7 +419,8 @@ static inline struct folio *swapin_readahead(swp_entry_t swp, gfp_t gfp_mask,
>>
>>  static inline struct folio *swapin_sync(
>>         swp_entry_t entry, gfp_t flag, unsigned long orders,
>> -       struct vm_fault *vmf, struct mempolicy *mpol, pgoff_t ilx)
>> +       unsigned long locality_orders, struct vm_fault *vmf,
>> +       struct mempolicy *mpol, pgoff_t ilx)
>>  {
>>         return NULL;
>>  }
>> diff --git a/mm/swap_state.c b/mm/swap_state.c
>> index f03ad4832f16..5a4ca289009a 100644
>> --- a/mm/swap_state.c
>> +++ b/mm/swap_state.c
>> @@ -21,6 +21,7 @@
>>  #include <linux/migrate.h>
>>  #include <linux/vmalloc.h>
>>  #include <linux/huge_mm.h>
>> +#include <linux/sizes.h>
>>  #include <linux/zswap.h>
>>  #include <linux/shmem_fs.h>
>>  #include "internal.h"
>> @@ -556,6 +557,24 @@ static struct folio *swap_cache_alloc_speculative_folio(swp_entry_t targ_entry,
>>                                         mpol, ilx, true);
>>  }
>>
>> +/*
>> + * Initial conservative cap for speculative zswap large swapin. Locality
>> + * evidence is supplied by the caller or by generic VMA hints; the common
>> + * swapin layer keeps backend safety and pressure decisions here.
>> + */
>> +#define SWAPIN_ZSWAP_MAX_SIZE                  SZ_64K
>> +#if PAGE_SIZE < SWAPIN_ZSWAP_MAX_SIZE
>> +#define SWAPIN_ZSWAP_MAX_ORDER                 \
>> +       ilog2(SWAPIN_ZSWAP_MAX_SIZE / PAGE_SIZE)
>> +#else
>> +#define SWAPIN_ZSWAP_MAX_ORDER                 0
>> +#endif
>> +
>> +struct zswap_admit_ctx {
>> +       bool pressure_checked;
>> +       bool reclaim_pressure;
>> +};
>> +
>>  static bool swapin_zeromap_same(swp_entry_t entry, unsigned int nr_pages)
>>  {
>>         unsigned int ci_start = swp_cluster_offset(entry);
>> @@ -586,11 +605,84 @@ static bool swapin_zeromap_same(swp_entry_t entry, unsigned int nr_pages)
>>         return true;
>>  }
>>
>> +static bool swapin_zswap_locality(struct vm_fault *vmf, unsigned int order,
>> +                                 unsigned long locality_orders)
>> +{
>> +       struct vm_area_struct *vma = vmf ? vmf->vma : NULL;
>> +
>> +       if (!order || order > MAX_PAGE_ORDER)
>> +               return false;
>> +
>> +       if (vma && (vma->vm_flags & VM_RAND_READ))
>> +               return false;
> 
> what about VM_SEQ_READ?
This helper is meant to consume locality_orders, not to produce locality
evidence itself. VM_SEQ_READ is handled by the caller-side locality producer:
the anon producer returns all candidate orders for VM_SEQ_READ, and the shmem
producer does the same for now.

I kept the check in the common helper mostly as a common veto for the readahead path where ra_orders are passed as locality_orders directly.

I think the cleaner fix is to move this into the producer side as well.

> 
>> +
>> +       return locality_orders & BIT(order);
>> +}
>> +
>> +static bool swapin_zswap_refaulted(swp_entry_t entry, unsigned int nr_pages)
> 
> nit: this does not seem zswap-specific. Just call it
> swapin_range_refaulted or sth like that, maybe?
> 
>> +       for (i = 0; i < nr_pages; i++) {
>> +               bool workingset;
>> +               void *shadow;
>> +
>> +               shadow = swap_cache_get_shadow(swp_entry(type, offset + i));
> 
> This seems inefficient. Can't we just lock the swap cluster once,
> check all the shadow in the range, instead of repeatedly getting then
> dropping the swap cluster lock?
both points make sense.

The refault check is not zswap-specific, so I will rename it to something like
swapin_range_refaulted().

And yes, the current implementation is too expensive. The range is already
bounded and contiguous, so I should check the swap table under one cluster lock
instead of calling swap_cache_get_shadow() for every slot. I will rework that
in v3.

> 
>> +               if (!shadow)
>> +                       continue;
>> +               if (workingset_test_recent(shadow, false, &workingset, false) &&
>> +                   workingset)
>> +                       return true;
>> +       }
>> +
>> +       return false;
>> +}
>> +
>> +static bool swapin_zswap_admit(swp_entry_t entry,
>> +                              unsigned int order, unsigned int nr_pages,
>> +                              struct vm_fault *vmf,
>> +                              unsigned long locality_orders,
>> +                              struct zswap_admit_ctx *ctx)
>> +{
>> +       if (order > SWAPIN_ZSWAP_MAX_ORDER)
>> +               return false;
>> +
>> +       /*
>> +        * Treat zswap-backed large swapin as speculative. The common layer
>> +        * consumes caller-provided locality orders, but does not inspect
>> +        * anon-specific PTE state or shmem-specific mapping state directly.
>> +        */
>> +       if (!swapin_zswap_locality(vmf, order, locality_orders))
>> +               return false;
>> +
>> +       /*
>> +        * A recent workingset refault shadow in the target range means reclaim
>> +        * already saw churn there. Keep the refault path narrow instead of
>> +        * speculatively decompressing neighbouring slots.
>> +        */
>> +       if (swapin_zswap_refaulted(entry, nr_pages))
>> +               return false;
> 
> Hmm this depends. If it's just a refault coming from a speculative
> read (readhead or THP (z)swpin), which is then promptly discarded,
> then yeah we should backoff here. But maybe the refaulted page is
> workingset one?
> 
> But yeah I guess it is better to be cautious when you are uncertain :)
> 
> 
>> +
>> +       if (!ctx->pressure_checked) {
>> +               ctx->reclaim_pressure = zswap_pool_reclaim_pressure();
>> +               ctx->pressure_checked = true;
>> +       }
> 
> Why do we backoff if there is zswap_pool_reclaim_pressure (which only
> check if the pool is full ONCE in its lifetime)? What's the rationale
> here?
The ctx is only local to one swapin_admit_orders() call. It is initialized
before walking the candidate orders, so pressure_checked is not persistent
across faults and is not tied to the lifetime of the zswap pool.

It is also not there because zswap_pool_reclaim_pressure() is expensive. That
helper is cheap. The point was to use one consistent pressure snapshot while
evaluating all candidate orders for the same fault, instead of letting
different orders observe different pressure states.

The zswap state being read is not "full once forever". It is
hysteretic: zswap_check_limits() sets zswap_pool_reached_full when the pool
reaches the max limit, and clears it after the pool drops below the accept
threshold.

The rationale for backing off was conservative: I treated zswap pool pressure
as correlated with memory/reclaim pressure. With this RFC's current design,
large zswapin can temporarily keep both the compressed zswap entries and the
uncompressed large swapcache folio, so I wanted to avoid that speculative
expansion when the compressed pool already looks stressed.

> 
>> +       if (ctx->reclaim_pressure)
>> +               return false;
>> +
>> +       return true;
>> +}
>> +
>>  static unsigned long swapin_admit_orders(swp_entry_t entry,
>> -                                        unsigned long orders)
>> +                                        unsigned long orders,
>> +                                        struct vm_fault *vmf,
>> +                                        unsigned long locality_orders)
>>  {
>>         unsigned long candidates = orders & ~BIT(0);
>>         unsigned long admitted = orders & BIT(0);
>> +       struct zswap_admit_ctx zswap_ctx = {};
>>         int order;
>>
>>         if (!candidates)
>> @@ -616,9 +708,14 @@ static unsigned long swapin_admit_orders(swp_entry_t entry,
>>
>>                 state = zswap_probe_range(range_entry, nr_pages);
>>                 switch (state) {
>> +               case ZSWAP_RANGE_ALL_ZSWAP:
>> +                       admit = swapin_zswap_admit(range_entry, order,
>> +                                                  nr_pages, vmf,
>> +                                                  locality_orders,
>> +                                                  &zswap_ctx);
>> +                       break;
>>                 case ZSWAP_RANGE_MIXED:
>>                         break;
>> -               case ZSWAP_RANGE_ALL_ZSWAP:
>>                 case ZSWAP_RANGE_NEVER_ENABLED:
>>                 case ZSWAP_RANGE_NO_ZSWAP:
>>                         admit = true;
>> @@ -769,8 +866,8 @@ static struct folio *swap_cache_read_folio(swp_entry_t entry, gfp_t gfp,
>>         ret = swap_read_folio(folio, plug);
>>         /*
>>          * Swap readahead allocates order-0 folios. -EAGAIN is reserved for
>> -        * retryable large zswap backend races and must be handled by the
>> -        * synchronous common swapin path.
>> +        * retryable large zswap backend races and should never escape to this
>> +        * order-0 path.
>>          */
>>         VM_WARN_ON_ONCE(ret == -EAGAIN);
>>         if (readahead) {
>> @@ -786,6 +883,7 @@ static struct folio *swap_cache_read_folio(swp_entry_t entry, gfp_t gfp,
>>   * @entry: swap entry indicating the target slot
>>   * @gfp: memory allocation flags
>>   * @orders: allocation orders
>> + * @locality_orders: orders with caller-provided locality evidence
>>   * @vmf: fault information
>>   * @mpol: NUMA memory allocation policy to be applied
>>   * @ilx: NUMA interleave index, for use only when MPOL_INTERLEAVE
>> @@ -794,16 +892,20 @@ static struct folio *swap_cache_read_folio(swp_entry_t entry, gfp_t gfp,
>>   * existing folio in the swap cache for @entry. This initiates the IO, too,
>>   * if needed. @entry is rounded down if @orders allow large allocation.
>>   *
>> - * Context: Caller must ensure @entry is valid and pin the swap device with refcount.
>> + * Context: Caller must ensure @entry is valid and pin the swap device with
>> + * refcount.
>>   * Return: Returns the folio on success, error code if failed.
>>   */
>> -struct folio *swapin_sync(swp_entry_t entry, gfp_t gfp, unsigned long orders,
>> -                          struct vm_fault *vmf, struct mempolicy *mpol, pgoff_t ilx)
>> +struct folio *swapin_sync(swp_entry_t entry, gfp_t gfp,
>> +                         unsigned long orders,
>> +                         unsigned long locality_orders,
>> +                         struct vm_fault *vmf, struct mempolicy *mpol,
>> +                         pgoff_t ilx)
>>  {
>>         struct folio *folio;
>>         int ret;
>>
>> -       orders = swapin_admit_orders(entry, orders);
>> +       orders = swapin_admit_orders(entry, orders, vmf, locality_orders);
>>  again:
>>         do {
>>                 folio = swap_cache_get_folio(entry);
>> --
>> 2.34.1
>>