From: Baoquan He <baoquan.he@linux.dev>
To: linux-mm@kvack.org
Cc: akpm@linux-foundation.org, david@kernel.org, urezki@gmail.com,
chrisl@kernel.org, kasong@tencent.com, nphamcs@gmail.com,
baohua@kernel.org, youngjun.park@lge.com,
shikemeng@huaweicloud.com, hannes@cmpxchg.org, yosry@kernel.org,
chengming.zhou@linux.dev, corbet@lwn.net, kunwu.chan@gmail.com,
klarasmodin@gmail.com, linux-kernel@vger.kernel.org,
baoquan.he@linux.dev, Kunwu Chan <chentao@kylinos.cn>,
Baoquan He <hebaoquan@kylinos.cn>
Subject: [PATCH v6 02/14] mm: xswap support for zswap
Date: Sun, 22 Nov 2026 20:39:06 +0800 [thread overview]
Message-ID: <20261122123918.3486916-3-baoquan.he@linux.dev> (raw)
In-Reply-To: <20261122123918.3486916-1-baoquan.he@linux.dev>
From: Chris Li <chrisl@kernel.org>
Introduce extendable swap device support - xswap.
An xswap device has no backing storage and no swap data section, so
it wastes no disk space. Creation is via swapon(2) with the name
"xswap", added in a later patch.
Zswap writeback is gated on whether a real (non-xswap) swap device is
active. nr_real_swapfiles counts such devices and is maintained at
swapon/swapoff only, so the gate reflects "a device exists to write
back to" rather than "a device currently has free slots". This keeps
writeback working even when the real swap device is full, and avoids
a double decrement when a full device is swapped off.
An xswap entry is not added to the zswap writeback LRU: with no backing
store there is nothing to write it back to. zswap_lru_del() tolerates
that, since an entry that was never added is on no list.
An incompressible page is stored uncompressed only because it can be
written back later, in LRU order. In this phase an xswap entry has no
writeback target and never reaches the LRU, so such a page would be
pinned in the pool for the lifetime of the entry. Reject it instead,
the same way a writeback-disabled memcg is handled. A later phase gives
xswap a physical backend, and the test becomes "this device has a
writeback target".
A VMA readahead walks the page table and can reach an entry on another
device. Skip an xswap device there: its entries live only in zswap, so
there is no IO to read ahead and a speculative read would take the entry
out of zswap.
An xswap entry is kept out of the zswap writeback LRU but still counts
towards the pool size. With memcg disabled the shrinker walks the root
LRU, finds no entry it can write back and is told -ENOENT without
counting a failure, so the accept-threshold loop would spin. Count that
as a failure; the worker then stops after its retry budget.
Reviewed-by: Kunwu Chan <chentao@kylinos.cn>
Co-developed-by: Baoquan He <hebaoquan@kylinos.cn>
Signed-off-by: Baoquan He <hebaoquan@kylinos.cn>
Signed-off-by: Chris Li <chrisl@kernel.org>
---
include/linux/swap.h | 2 ++
mm/page_io.c | 13 +++++++++++++
mm/swap_state.c | 24 +++++++++++++++++++-----
mm/swapfile.c | 34 ++++++++++++++++++++++++++++------
mm/zswap.c | 34 +++++++++++++++++++++++++++-------
5 files changed, 89 insertions(+), 18 deletions(-)
diff --git a/include/linux/swap.h b/include/linux/swap.h
index 4f686709d71f..9073d29377d8 100644
--- a/include/linux/swap.h
+++ b/include/linux/swap.h
@@ -201,6 +201,7 @@ enum {
SWP_STABLE_WRITES = (1 << 11), /* no overwrite PG_writeback pages */
SWP_SYNCHRONOUS_IO = (1 << 12), /* synchronous IO is efficient */
SWP_HIBERNATION = (1 << 13), /* pinned for hibernation */
+ SWP_XSWAP = (1 << 14), /* extendable swap device */
/* add others here before... */
};
@@ -378,6 +379,7 @@ void free_folio_and_swap_cache(struct folio *folio);
void free_pages_and_swap_cache(struct encoded_page **, int);
/* linux/mm/swapfile.c */
extern atomic_long_t nr_swap_pages;
+extern atomic_t nr_real_swapfiles;
extern long total_swap_pages;
extern atomic_t nr_rotate_swap;
diff --git a/mm/page_io.c b/mm/page_io.c
index a57cdcf2892d..4b541c50cd59 100644
--- a/mm/page_io.c
+++ b/mm/page_io.c
@@ -258,6 +258,12 @@ int swap_writeout(struct swap_io_ctx *ctx, struct folio *folio)
}
rcu_read_unlock();
+ /* No backing store: keep the folio; ctx->sis is not set yet. */
+ if (unlikely(__swap_entry_to_info(folio->swap)->flags & SWP_XSWAP)) {
+ folio_mark_dirty(folio);
+ return AOP_WRITEPAGE_ACTIVATE;
+ }
+
__swap_writeout(ctx, folio);
return 0;
out_unlock:
@@ -515,6 +521,13 @@ void swap_read_folio(struct swap_io_ctx *ctx, struct folio *folio)
if (zswap_load(folio) != -ENOENT)
return;
+ if (unlikely(sis->flags & SWP_XSWAP)) {
+ /* An xswap entry only lives in zswap: zswap_load() must hit. */
+ WARN_ON_ONCE(1);
+ folio_unlock(folio);
+ return;
+ }
+
/* We have to read from slower devices. Increase zswap protection. */
zswap_folio_swapin(folio);
swap_add_folio(ctx, folio, READ);
diff --git a/mm/swap_state.c b/mm/swap_state.c
index da65d75ceead..90cd99c25dfe 100644
--- a/mm/swap_state.c
+++ b/mm/swap_state.c
@@ -899,6 +899,10 @@ struct folio *swap_cluster_readahead(swp_entry_t entry, gfp_t gfp_mask,
struct blk_plug plug;
swp_entry_t ra_entry;
+ /* xswap entries live only in zswap; readahead does not help. */
+ if (si->flags & SWP_XSWAP)
+ goto skip;
+
mask = swapin_nr_pages(offset) - 1;
if (!mask)
goto skip;
@@ -984,6 +988,7 @@ static int swap_vma_ra_win(struct vm_fault *vmf, unsigned long *start,
static struct folio *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask,
struct mempolicy *mpol, pgoff_t targ_ilx, struct vm_fault *vmf)
{
+ struct swap_info_struct *si = __swap_entry_to_info(targ_entry);
struct swap_io_ctx ctx = {};
struct blk_plug plug;
struct folio *folio;
@@ -992,6 +997,10 @@ static struct folio *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask,
unsigned long start, end, addr;
pgoff_t ilx = targ_ilx;
+ /* xswap entries live only in zswap; readahead does not help. */
+ if (si->flags & SWP_XSWAP)
+ goto skip;
+
win = swap_vma_ra_win(vmf, &start, &end);
if (win == 1)
goto skip;
@@ -1000,7 +1009,7 @@ static struct folio *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask,
blk_start_plug(&plug);
for (addr = start; addr < end; ilx++, addr += PAGE_SIZE) {
- struct swap_info_struct *si = NULL;
+ struct swap_info_struct *entry_si = NULL;
softleaf_t entry;
if (!pte++) {
@@ -1020,14 +1029,19 @@ static struct folio *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask,
* holding a reference to, try to grab a reference, or skip.
*/
if (swp_type(entry) != swp_type(targ_entry)) {
- si = get_swap_device(entry);
- if (IS_ERR_OR_NULL(si))
+ entry_si = get_swap_device(entry);
+ if (IS_ERR_OR_NULL(entry_si))
+ continue;
+ /* An xswap entry lives only in zswap; there is no IO. */
+ if (entry_si->flags & SWP_XSWAP) {
+ put_swap_device(entry_si);
continue;
+ }
}
folio = swap_cache_read_folio(&ctx, entry, gfp_mask, mpol, ilx,
addr != vmf->address);
- if (si)
- put_swap_device(si);
+ if (entry_si)
+ put_swap_device(entry_si);
if (!folio)
continue;
folio_put(folio);
diff --git a/mm/swapfile.c b/mm/swapfile.c
index d59fa8220862..558c4b75bf92 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -66,6 +66,7 @@ static void move_cluster(struct swap_info_struct *si,
static DEFINE_SPINLOCK(swap_lock);
static unsigned int nr_swapfiles;
atomic_long_t nr_swap_pages;
+atomic_t nr_real_swapfiles;
/*
* Some modules use swappable objects and may try to swap them out under
* memory pressure (via the shrinker). Before doing so, they may wish to
@@ -775,7 +776,8 @@ static void free_cluster(struct swap_info_struct *si, struct swap_cluster_info *
/*
* If the swap is discardable, prepare discard the cluster
* instead of free it immediately. The cluster will be freed
- * after discard.
+ * after discard. xswap has no bdev and never sets
+ * SWP_PAGE_DISCARD, so it always takes the free path below.
*/
if ((si->flags & (SWP_WRITEOK | SWP_PAGE_DISCARD)) ==
(SWP_WRITEOK | SWP_PAGE_DISCARD)) {
@@ -1277,6 +1279,9 @@ static void del_from_avail_list(struct swap_info_struct *si, bool swapoff)
*/
lockdep_assert_held(&si->lock);
si->flags &= ~SWP_WRITEOK;
+ /* Count active devices, not merely those on the avail list. */
+ if (!(si->flags & SWP_XSWAP))
+ atomic_sub(1, &nr_real_swapfiles);
atomic_long_or(SWAP_USAGE_OFFLIST_BIT, &si->inuse_pages);
} else {
/*
@@ -1310,6 +1315,9 @@ static void add_to_avail_list(struct swap_info_struct *si, bool swapon)
if (swapon) {
lockdep_assert_held(&si->lock);
si->flags |= SWP_WRITEOK;
+ /* A full device is still active and skips the add below. */
+ if (!(si->flags & SWP_XSWAP))
+ atomic_add(1, &nr_real_swapfiles);
} else {
if (!(READ_ONCE(si->flags) & SWP_WRITEOK))
goto skip;
@@ -1323,10 +1331,11 @@ static void add_to_avail_list(struct swap_info_struct *si, bool swapon)
/*
* When device is full and device is on the plist, only one updater will
* see (inuse_pages == si->pages) and will call del_from_avail_list. If
- * that updater happen to be here, just skip adding.
+ * that updater happen to be here, just skip adding. @val still has
+ * the off-list bit, so compare its counter part.
*/
pages = si->pages;
- if (val == pages) {
+ if ((val & SWAP_USAGE_COUNTER_MASK) == pages) {
/* Just like the cmpxchg in del_from_avail_list */
if (atomic_long_try_cmpxchg(&si->inuse_pages, &pages,
pages | SWAP_USAGE_OFFLIST_BIT))
@@ -3330,7 +3339,8 @@ SYSCALL_DEFINE1(swapoff, const char __user *, specialfile)
destroy_swap_extents(p, p->swap_file);
- if (!(p->flags & SWP_SOLIDSTATE))
+ if (!(p->flags & SWP_XSWAP) &&
+ !(p->flags & SWP_SOLIDSTATE))
atomic_dec(&nr_rotate_swap);
mutex_lock(&swapon_mutex);
@@ -3440,6 +3450,19 @@ static void swap_stop(struct seq_file *swap, void *v)
mutex_unlock(&swapon_mutex);
}
+static const char *swap_type_str(struct swap_info_struct *si)
+{
+ struct file *file = si->swap_file;
+
+ if (si->flags & SWP_XSWAP)
+ return "xswap\t";
+
+ if (S_ISBLK(file_inode(file)->i_mode))
+ return "partition";
+
+ return "file\t";
+}
+
static int swap_show(struct seq_file *swap, void *v)
{
struct swap_info_struct *si = v;
@@ -3459,8 +3482,7 @@ static int swap_show(struct seq_file *swap, void *v)
len = seq_file_path(swap, file, " \t\n\\");
seq_printf(swap, "%*s%s\t%lu\t%s%lu\t%s%d\n",
len < 40 ? 40 - len : 1, " ",
- S_ISBLK(file_inode(file)->i_mode) ?
- "partition" : "file\t",
+ swap_type_str(si),
bytes, bytes < 10000000 ? "\t" : "",
inuse, inuse < 10000000 ? "\t" : "",
si->prio);
diff --git a/mm/zswap.c b/mm/zswap.c
index a2e0314db1cf..5603d283f9b5 100644
--- a/mm/zswap.c
+++ b/mm/zswap.c
@@ -885,7 +885,8 @@ static int zswap_cpu_comp_prepare(unsigned int cpu, struct hlist_node *node)
}
static bool zswap_compress(struct folio *folio, long index,
- struct zswap_entry *entry, struct zswap_pool *pool)
+ struct zswap_entry *entry, struct zswap_pool *pool,
+ bool can_writeback)
{
struct zswap_comp_ctx *comp_ctx;
struct scatterlist input, output;
@@ -934,9 +935,13 @@ static bool zswap_compress(struct folio *folio, long index,
* save the content as is without a compression, to keep the LRU order
* of writebacks. If writeback is disabled, reject the page since it
* only adds metadata overhead. swap_writeout() will put the page back
- * to the active LRU list in the case.
+ * to the active LRU list in the case. An xswap entry in this phase
+ * is never put on the LRU, so an incompressible page stored for it
+ * could never be reclaimed: reject it the same way.
*/
if (comp_ret) {
+ if (!can_writeback)
+ goto unlock;
rcu_read_lock();
if (!mem_cgroup_zswap_writeback_enabled(folio_memcg(folio))) {
rcu_read_unlock();
@@ -1093,6 +1098,11 @@ static int zswap_writeback_entry(struct swap_io_ctx *ctx,
if (IS_ERR_OR_NULL(si))
return -ENOENT;
+ if (si->flags & SWP_XSWAP) {
+ put_swap_device(si);
+ return -EINVAL;
+ }
+
mpol = get_task_policy(current);
folio = __swap_cache_alloc_folio(swpentry, GFP_KERNEL, BIT(0), NULL, mpol,
NO_INTERLEAVE_INDEX);
@@ -1508,10 +1518,15 @@ static void shrink_worker(struct work_struct *w)
* There are no writeback-candidate pages in the memcg.
* This is not an issue as long as we can find another memcg
* with pages in zswap. Skip this without incrementing attempts
- * and failures.
+ * and failures. With memcg disabled there is no other memcg,
+ * so an empty walk is the last one and has to count.
*/
- if (ret == -ENOENT)
+ if (ret == -ENOENT) {
+ if (mem_cgroup_disabled() &&
+ ++failures == MAX_RECLAIM_RETRIES)
+ break;
goto resched;
+ }
++attempts;
if (ret && ++failures == MAX_RECLAIM_RETRIES)
@@ -1531,6 +1546,7 @@ static bool zswap_store_page(struct folio *folio, long index,
{
swp_entry_t page_swpentry = folio_swap_entry(folio, index);
struct zswap_entry *entry, *old;
+ bool can_writeback;
/* allocate entry */
entry = zswap_entry_cache_alloc(GFP_KERNEL, folio_nid(folio));
@@ -1539,7 +1555,10 @@ static bool zswap_store_page(struct folio *folio, long index,
return false;
}
- if (!zswap_compress(folio, index, entry, pool))
+ /* In this phase xswap has no device to write the page back to. */
+ can_writeback = !(__swap_entry_to_info(page_swpentry)->flags & SWP_XSWAP);
+
+ if (!zswap_compress(folio, index, entry, pool, can_writeback))
goto compress_failed;
/*
@@ -1599,7 +1618,8 @@ static bool zswap_store_page(struct folio *folio, long index,
entry->referenced = true;
if (entry->length) {
INIT_LIST_HEAD(&entry->lru);
- zswap_lru_add(entry);
+ if (can_writeback)
+ zswap_lru_add(entry);
}
return true;
@@ -1669,7 +1689,7 @@ bool zswap_store(struct folio *folio)
zswap_pool_put(pool);
put_objcg:
obj_cgroup_put(objcg);
- if (!ret && zswap_pool_reached_full)
+ if (!ret && zswap_pool_reached_full && atomic_read(&nr_real_swapfiles))
queue_work(shrink_wq, &zswap_shrink_work);
check_old:
/*
--
2.54.0
next prev parent reply other threads:[~2026-11-22 12:39 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-22 12:39 [PATCH v6 00/14] mm, swap: extendable swap devices (xswap phase I) Baoquan He
2026-11-22 12:39 ` [PATCH v6 01/14] mm/vmalloc: support vmalloc-backed VM_SPARSE areas Baoquan He
2026-11-22 12:39 ` Baoquan He [this message]
2026-11-22 12:39 ` [PATCH v6 03/14] mm, swap: refactor free_swap_cluster_info to take swap_info_struct Baoquan He
2026-11-22 12:39 ` [PATCH v6 04/14] mm, swap: use a VM_SPARSE cluster array for xswap devices Baoquan He
2026-11-22 12:39 ` [PATCH v6 05/14] mm, swap: widen swap_info_struct max/pages to unsigned long Baoquan He
2026-11-22 12:39 ` [PATCH v6 06/14] mm, swap: create the xswap device from swapon(2) Baoquan He
2026-11-22 12:39 ` [PATCH v6 07/14] mm, swap: refactor swapoff and add xswap_destroy Baoquan He
2026-11-22 12:39 ` [PATCH v6 08/14] mm, swap: require zswap for xswap devices Baoquan He
2026-11-22 12:39 ` [PATCH v6 09/14] mm, swap: let the command line set the xswap device size Baoquan He
2026-11-22 12:48 ` [PATCH v6 10/14] mm, swap: add CONFIG_XSWAP_DEFAULT_MAX Baoquan He
2026-11-22 12:48 ` [PATCH v6 11/14] mm, swap: add xswap grow trigger on cluster allocation Baoquan He
2026-11-22 12:48 ` [PATCH v6 12/14] mm, swap: add debugfs counters for xswap Baoquan He
2026-11-22 12:48 ` [PATCH v6 13/14] mm, swap: do not give an xswap slot to a cgroup that cannot zswap Baoquan He
2026-11-22 12:48 ` [PATCH v6 14/14] mm, swap: skip the overcommit probe when removing xswap Baoquan He
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261122123918.3486916-3-baoquan.he@linux.dev \
--to=baoquan.he@linux.dev \
--cc=akpm@linux-foundation.org \
--cc=baohua@kernel.org \
--cc=chengming.zhou@linux.dev \
--cc=chentao@kylinos.cn \
--cc=chrisl@kernel.org \
--cc=corbet@lwn.net \
--cc=david@kernel.org \
--cc=hannes@cmpxchg.org \
--cc=hebaoquan@kylinos.cn \
--cc=kasong@tencent.com \
--cc=klarasmodin@gmail.com \
--cc=kunwu.chan@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=nphamcs@gmail.com \
--cc=shikemeng@huaweicloud.com \
--cc=urezki@gmail.com \
--cc=yosry@kernel.org \
--cc=youngjun.park@lge.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®