Re: [PATCH v5 00/11] Virtual Swap Space (Swap Table Edition)

[Klara Modin <klarasmodin@gmail.com>]

On 2026-09-18 11:02:30 -0700, Nhat Pham wrote:
> Changelog:
> * v4 [v4] -> v5
>     * OVERCOMMIT_GUESS now allows for 3xRAM margin when vswap is enabled,
>       to take into account swapfile-less zswap usage (proposed by
>       Johannes Weiner).
>     * Limit vswap swapfile size to 8TB and drop the last patch, to avoid
>       memcg private id refcnt saturation.
>     * More assorted cleanups and fixes: anon swappability check, etc.
>     * RFC: Replace the xarray with a new data structure (vmalloc array)
>       (new patch 11).
>     * Rebased onto mm-unstable.
> * v3 [v3] -> v4
>     * Replaced the runtime sysctl with a cmdline param, and remove
>       CONFIG_VSWAP (suggested by Johannes Weiner). CONFIG_VSWAP_DEFAULT_ON
>       now only gives the default value of the vswap cmdline parameter.
>     * Refactor swap-related memcg operations into composable building
>       blocks: reference acquisitions, charging, etc. (patch 8,
>       suggested by Johannes Weiner).
>     * Fixed vtable UAF bug reported by syzbot and Kunwu.
>     * Rebased onto mm-unstable (minimal merge conflicts).
>     * Re-run benchmarks (no signal change).
> * v2 [v2] -> v3:
>     * Rebased onto current mm-unstable.
>     * Add a runtime vm.vswap_enabled sysctl and CONFIG_VSWAP_DEFAULT_ON
>       to gate vswap allocation.
>     * More cleanups and small bug fixes.
>     * Split THP swapin enablement into its own patch (patch 5).
>     * Add production workload benchmark results, and drop RFC tag.
> * v1 [v1] -> v2:
>     * Rebased to a newer mm-unstable tip.
>     * Fix a bunch of assorted issues (incorrect zswap store failure
>       rollback, vswap_init() failure handling, rmap-encoding collision,
>       etc.) and clean up the code (rename a bunch of functions to
>       more closely follow existing patterns, etc.).
>     * Some more code clean up and simplification: some renamings to more
>       closely follow existing patterns, move vswap backing check to
>       __swap_cache_add_check, store zero state in the swap_table for
>       vswap entries, etc.. Many of these are proposed by Kairui Song
>       in [1].
>     * Defer memcg_table allocation on physical clusters until the first
>       vswap-backing slot installs. Saves ~512 bytes per physical cluster
>       that only serves vswap-backing slots (this is the new patch 8).
>     * Widen swap_info_struct->max and ->pages (and the swapoff unuse-path
>       index) so vswap supports ~8 PB of swap space (this is the new
>       patch 9).
>     * Split the physical-swap-backend patch into three for reviewability:
>       the core backend (patch 3), zswap writeback to physical swap
>       (patch 4), and reclaim of cache-only physical slots (patch 5). No
>       functional change.
>     * Add kerneldoc for the vswap API.
>     * Add some benchmark numbers for zswap case.
> 
> 
> Patch 11 is an RFC. It swaps vswap's cluster xarray for the VM_SPARSE
> vmalloc array Baoquan He designed for xswap, to show that the data
> structure and the device model are separable: moving to his is one
> self-contained patch that adds no userspace interface. Note that
> per Baoquan's commentary (see [5]), I have skipped shrink for now, only
> freeing vtable when the cluster becomes free to minimize metadata
> overhead while keeping the skeleton in the free list. I have not done
> performance testing on this patch yet (the number is from the old design),
> but I have run a suite of simple stress tests.
> 
> It is adapted almost entirely from Baoquan's code (see [6]), so I have
> kept Baoquan's Co-developed-by and Signed-off-by tag.
> 
> 
> I. Context and Motivation
> =========================
> 
> Currently, when an anon page is swapped out, a slot in a backing swap
> device is allocated and stored in the page table entries that refer to
> the original page. This slot is also used as the "key" to find the
> swapped out content, as well as the index to swap data structures, such
> as the swap cache, or the swap cgroup mapping. Tying a swap entry to its
> backing slot in this way is performant and efficient when swap is purely
> just disk space, and swapoff is rare.
> 
> However, the advent of many swap optimizations has exposed major
> drawbacks of this design. The first problem is that we occupy a physical
> slot in the swap space, even for pages that are NEVER expected to hit
> the disk: pages compressed and stored in the zswap pool, zero-filled
> pages, or pages rejected by both of these optimizations when zswap
> writeback is disabled. This is arguably the central shortcoming of
> zswap:
> * Resource-wise, it is hugely wasteful in terms of disk usage. At Meta,
>   we size swapfile in the order of 25-50% of host RAM, depending on flash
>   availability. This is a lot of flash for a fleet of our size, and
>   with universal zswap enablement, most of this is wasted for zswap
>   entries.
> 
> * In deployments when no disk space can be afforded for swap (such as
>   mobile and embedded devices), users cannot adopt zswap, and are forced
>   to use zram. This is confusing for users, and creates extra burdens
>   for developers, having to develop and maintain similar features for
>   two separate swap backends (writeback, cgroup charging, THP support,
>   etc.). For instance, see the discussion in [2].
> 
> * Tying zswap (and more generally, other in-memory swap backends) to
>   the current physical swapfile infrastructure makes zswap implicitly
>   statically sized. This does not make sense, as unlike disk swap, in
>   which we consume a limited resource (disk space or swapfile space) to
>   save another resource (memory), zswap consumes the same resource it is
>   saving (memory). The more we zswap, the more memory we have available,
>   not less. We are not rationing a limited resource when we limit
>   the size of the zswap pool, but rather we are capping the resource
>   (memory) saving potential of zswap. Under memory pressure, using
>   more zswap is almost always better than the alternative (disk IOs, or
>   even worse, OOMs), and dynamically sizing the zswap pool on demand
>   allows the system to flexibly respond to these precarious scenarios.
> 
> * Operationally, static provisioning the swapfile for zswap poses
>   significant challenges, because the sysadmin has to prescribe how
>   much swap is needed a priori, for each combination of
>   (memory size x disk space x workload usage). It is even more
>   complicated when we take into account the variance of memory
>   compression, which changes the reclaim dynamics (and as a result,
>   swap space size requirement). The problem is further exacerbated for
>   users who rely on swap utilization (and exhaustion) as an OOM signal.
> 
>   All of these factors make it very difficult to configure the swapfile
>   for zswap: too small of a swapfile and we risk preventable OOMs and
>   limit the memory saving potentials of zswap; too big of a swapfile
>   and we waste disk space and memory due to swap metadata overhead.
>   This dilemma becomes more drastic in high memory systems, which can
>   have up to TBs worth of memory.
> 
> Swap virtualization is the answer to these issues, with three properties:
> 
> 1. Decoupled backends. For zswap in particular, this means we eliminate
>    the unused storage space, and allows zswap to be used in systems that
>    do not have enough storage capacity for physical swap (without having
>    to resort to silly hacks). Zero-filled swap pages and swap-cache-only
>    folios also benefit here.
> 
> 2. Dynamic swap space. Since virtual swap is not tied to any physical
>    resource, we can make it effectively infinite and dynamically grow it
>    on demand.
>    This massively simplifies operational provisioning, and increases the
>    utilization of compressed swap backends (zswap). Dynamicity also
>    reduces overhead on unused swap capacity.
> 
> 3. Efficient backend transfer. The virtualization scheme should not
>    introduce PTE/rmap walking overhead for backend transfer. This
>    is crucial for systems that want to support multiple swap backends
>    in a tiering fashion (for e.g zswap -> disk swap).
> 
> For more historical contexts and references, please take a look at
> the cover letter of the older vswap submissions ([3] and [v2]).
> 
> II. Design
> ==========
> 
> When vswap is enabled (via vswap=on cmdline parameter), a special vswap
> device is allocated at boot time. Anon pages that can be zswapped will
> obtain a vswap slot at swap allocation time.
> 
> These swap entries can subsequently acquire backend on-demand, such as
> a zswap entry, or a slot on a physical swap device (as a fallback option
> or at zswap writeback time).
> 
> We repurpose much of the existing swap_table infrastructure and
> swapfile allocator for this new vswap device, with two notable
> differences:
> * Clusters are dynamically allocated on demand and managed through
>   an xarray. This in turn allows us to avoid static provisioning and
>   let swap space grow dynamically.
> 
> * Each cluster of this new vswap device has a virtual_table that stores
>   the backend information of the entries in the cluster (see below).
> 
> Diagrams:
> 
>   Case 1: vswap entry (virtualized)
> 
>   PTE                  swap_cluster_info_dynamic
>   vswap_entry          +---------------------------------+
>   (swp_entry_t) ------>| swap_cluster_info (ci)          |
>                        | +----------------------------+  |
>                        | | swap_table                 |  |
>                        | |   PFN / Shadow             |  |
>                        | | memcg_table                |  |
>                        | | count,flags,order          |  |
>                        | | lock, list                 |  |
>                        | +----------------------------+  |
>                        |                                 |
>                        | virtual_table                   |
>                        | +----------------------------+  |
>                        | | NONE                       |  |
>                        | | SWAPFILE(swp_entry_t)      |  |
>                        | | ZSWAP(struct zswap_entry*) |  |
>                        | +----------------------------+  |
>                        +---------------------------------+
>                               |
>                               | SWAPFILE resolves to
>                               v
>                        PHYSICAL CLUSTER (swap_cluster_info)
>                        +--------------------------+
>                        | swap_table per-slot:     |
>                        |   NULL   - free          |
>                        |   PFN    - cached folio  |
>                        |   Shadow - swapped out   |
>                        |   Pointer- vswap rmap    |
>                        |   Bad    - unusable      |
>                        |                          |
>                        | Vswap-backing slot:      |
>                        |   Pointer(C|swp_entry_t) |
>                        |     rmap back to vswap   |
>                        +--------------------------+
> 
>   Case 2: direct-mapped physical entry (no vswap)
> 
>   PTE                  PHYSICAL CLUSTER (swap_cluster_info)
>   phys_entry           +--------------------------+
>   (swp_entry_t) ------>| swap_table per-slot:     |
>                        |   NULL   - free          |
>                        |   PFN    - cached folio  |
>                        |   Shadow - swapped out   |
>                        |   Bad    - unusable      |
>                        +--------------------------+
> 
> struct swap_cluster_info_dynamic {
>     struct swap_cluster_info ci;       /* swap_table, lock, etc. */
>     unsigned int index;                /* position in xarray */
>     struct rcu_head rcu;               /* kfree_rcu deferred free */
>     atomic_long_t *virtual_table;      /* backend info, 8 B/slot */
> };
> 
> Each vswap cluster (swap_cluster_info_dynamic) extends the classic
> swap_cluster_info struct with a virtual_table array that stores the
> backend information for each virtual swap entry in the cluster. Each
> entry is tag-encoded in the low 3 bits to indicate the backend type:
> 
>   NONE:     |----- 0000 ------|000|  free / unbacked
>   ZSWAP:    |--- zswap_entry* |001|  compressed in zswap
>   SWAPFILE: |- type:5,off:56 -|010|  on a physical swapfile
> 
> Other design highlights:
> 
> * Note that for the vswap device, we have merged the zswap xarray tree
>   with the swapfile-level clusters. This means that for zswap only users,
>   we have negligible extra space overhead.
> 
> * Both vswap entries (Case 1) and directly-mapped physical entries
>   (Case 2) coexist as first-class citizens.
> 
> * Backend transitions in the virtual_table are synchronized through the
>   swap cache and the folio lock - the same mechanism that already
>   serializes ordinary swap operations (swapin, swapout, zswap
>   writeback, swap cache reclaim). IOW, we can only assume that the
>   backend of a vswap entry is stable through swap cache/folio lock.
>   Looking at the backend without this should be done at best for
>   optimization purposes, as there is no guarantee that the backend
>   will not change under the observer.
> 
> * Pointer-tagged swap_table entries on physical clusters provide the
>   rmap (physical -> virtual) lookup.
> 
> * Virtual swap slots not backed by physical swap are not charged to
>   memcg swap counters - only physical backing is charged (I made the
>   case for this in [4]).
> 
> III. Benchmarks
> ===============
> 
> Note that the goal is not to match vswap performance with baseline on
> every single case yet - running with vswap off is still supported. We
> can optimize further once we have landed this new feature.
> 
> A. Production Workload: Instagram
> =================================
> 
> To test vswap's stability and performance, I ran an A/B experiment on
> Instagram (django) workload, with zswap as the swap backend. On these
> hosts, the swapfiles' size is 50% of RAM.
> 
> Compared to baseline, vswap gives:
> 
> * On par request throughput.
> * Lower request serving latency (by about 1-3%).
> * Lower memory pressure in the system service cgroups running alongside
>   the workload. PSI-based proactive reclaimer can therefore recover more
>   from them, lowering their overall memory footprint, allowing the main
>   workload to expand.
> * Elimination of swapfile footprint for all zswap users in the host.
> 
> B. Semi-synthetic Workloads (memhog, usemem, kernel build)
> ==========================================================
> 
> All values are mean +/- standard deviation across rounds.
> 
> Test system: x86_64, 52 cores, 64 GB swapfile for all 3 benchmarks.
> Swap backend: zswap (zstd) with the traditional active/inactive LRU. We
> focus on zswap here because it is the motivating use case for vswap.
> 
> For each benchmark, we test 3 kernels:
> * Baseline: mm-unstable, no vswap patches.
> * VSS off: vswap series applied, vswap=off, to verify that there is no
>   regression to existing swap paths when we disable vswap.
> * VSS on: vswap series applied, vswap=on.
> 
> 1. Memhog: single-threaded, 48GB allocation on a host with 16GB RAM,
>    20 rounds.
> 
>                     Baseline           VSS off            VSS on
>    real (s)        124.05 +/- 11.64   122.31 +/- 10.29   118.57 +/- 15.68
>    sys (s)         106.75 +/- 10.86   105.01 +/- 9.64    101.34 +/- 13.97
>    user (s)        10.81 +/- 0.11     10.85 +/- 0.09     10.79 +/- 0.09
>    delta real              -              -1.4%              -4.4%
>    delta sys               -              -1.6%              -5.1%
> 
> Dropping the best and the worst round to reduce variance:
> 
>    memhog              Baseline           VSS off            VSS on
>    real (s)        123.75 +/- 10.39   122.04 +/- 9.10    116.06 +/- 8.22
>    sys (s)         106.80 +/- 10.21   104.99 +/- 8.86    99.29 +/- 8.27
>    user (s)        10.82 +/- 0.11     10.85 +/- 0.08     10.79 +/- 0.10
>    delta real              -              -1.4%              -6.2%
>    delta sys               -              -1.7%              -7.0%
> 
> 
> 2. Usemem single-threaded: 56GB allocation on a host with 32GB RAM,
>    16 rounds.
> 
>                     Baseline           VSS off            VSS on
>    real (s)        178.75 +/- 6.47    178.95 +/- 6.56    175.97 +/- 7.74
>    sys (s)         127.03 +/- 6.56    128.08 +/- 6.59    124.71 +/- 7.90
>    tput (KB/s)     386662 +/- 14648   386264 +/- 15150   390443 +/- 17532
>    free (ms)       7669 +/- 146       7678 +/- 136       6439 +/- 111
>    delta real              -              +0.1%              -1.6%
>    delta sys               -              +0.8%              -1.8%
>    delta tput              -              -0.1%              +1.0%
>    delta free              -              +0.1%             -16.0%
> 
> 3. Kernel build: 52 workers (one per processor), memory.max=3GB, 10 rounds.
> 
>                     Baseline           VSS off            VSS on
>    real (s)        165.58 +/- 0.45    165.83 +/- 0.49    166.01 +/- 0.58
>    sys (s)         694.24 +/- 26.13   710.76 +/- 19.83   705.06 +/- 21.40
>    user (s)       5132.62 +/- 1.12   5133.69 +/- 1.57   5134.68 +/- 1.68
>    delta real              -              +0.2%              +0.3%
>    delta sys               -              +2.4%              +1.6%
>    delta user              -              +0.0%              +0.0%
> 
> 
> For zswap backend, vswap outperforms baseline on usemem freeing, and is
> on par with baseline on the rest.

I have been using versions of this series consistently since about
August and occasionally before with no apparent issues, so I think it's
time for

Tested-by: Klara Modin <klarasmodin@gmail.com>

> 
> IV. References
> ==============
> 
> [v1]: https://lore.kernel.org/all/20260528212955.1912856-1-nphamcs@gmail.com/
> [v2]: https://lore.kernel.org/all/20260612193738.2183968-1-nphamcs@gmail.com/
> [v3]: https://lore.kernel.org/all/20260806184254.3790858-1-nphamcs@gmail.com/
> [v4]: https://lore.kernel.org/all/20260825153238.2695446-1-nphamcs@gmail.com/
> [1]: https://lore.kernel.org/all/CAMgjq7BhOn48xEyC=2j837R7qddfjeBVHMiRqdx8no4ZEBpBLg@mail.gmail.com/
> [2]: https://lore.kernel.org/all/Zqe_Nab-Df1CN7iW@infradead.org/
> [3]: https://lore.kernel.org/all/20260505153854.1612033-1-nphamcs@gmail.com/
> [4]: https://lore.kernel.org/linux-mm/CAKEwX=P4syV38jAVCWq198r2OHXXc=xA-fx1dk6+qYef6yzxWQ@mail.gmail.com/
> [5]: https://lore.kernel.org/all/aqjpYHbZ14A8xRtK@fedora/
> [6]: https://lore.kernel.org/all/20260916101929.149106-1-hebaoquan@kylinos.cn/
> 
> Nhat Pham (11):
>   mm, swap: add virtual swap device infrastructure
>   mm, swap: support zswap and zero-filled swap pages as vswap backends
>   mm, swap: prepare the swap IO path for vswap
>   mm, swap: support physical swap as a vswap backend
>   mm, swap: enable THP swapin for vswap entries
>   mm, swap: write back vswap zswap entries to physical swap
>   mm, swap: reclaim physical slots backing cache-only vswap entries
>   mm, swap: only charge physical swap entries
>   mm, swap: add debugfs counters for vswap
>   mm, swap: defer memcg_table allocation for physical swap clusters
>   mm, swap: back vswap clusters with a VM_SPARSE array
> 
>  .../admin-guide/cgroup-v1/memcg_test.rst      |    2 +-
>  Documentation/admin-guide/cgroup-v2.rst       |   46 +-
>  .../admin-guide/kernel-parameters.txt         |    7 +
>  MAINTAINERS                                   |    1 +
>  include/linux/memcontrol.h                    |    6 +
>  include/linux/swap.h                          |   88 +-
>  include/linux/swap_ops.h                      |    9 +-
>  include/linux/zswap.h                         |    4 +
>  mm/Kconfig                                    |   20 +
>  mm/memcontrol-v1.c                            |   10 +-
>  mm/memcontrol.c                               |  147 +-
>  mm/memory.c                                   |   21 +-
>  mm/page_io.c                                  |  101 +-
>  mm/shmem.c                                    |    4 +-
>  mm/swap.h                                     |   28 +-
>  mm/swap_state.c                               |   50 +-
>  mm/swap_table.h                               |   64 +-
>  mm/swapfile.c                                 | 1254 +++++++++++++++--
>  mm/util.c                                     |   13 +-
>  mm/vmscan.c                                   |   18 +-
>  mm/vswap.h                                    |  422 ++++++
>  mm/workingset.c                               |    2 +-
>  mm/zswap.c                                    |  132 +-
>  23 files changed, 2180 insertions(+), 269 deletions(-)
>  create mode 100644 mm/vswap.h
> 
> 
> base-commit: 27e4e1835109ef599d72abe6c09711e0b1916033
> -- 
> 2.53.0-Meta
>