Re: Path forward for Virtualized Swap?

[Kairui Song <ryncsn@gmail.com>]

On Mon, Sep 21, 2026 at 09:27:01AM +0100, Gregory Price wrote:
> On Mon, Sep 21, 2026 at 12:01:18PM +0200, Kairui Song wrote:
> > > First of all, the traditional swap counter has a very well-defined
> > > meaning. It is the size of the memory that, when accessed, requires a
> > > page fault. A page fault adds significant latency to memory access
> > 
> > Yeah I agree on this. Swap just about makes resources not directly
> > accessible by the CPU act as RAM, whether that is storage on disk,
> > compressed memory, or a network resource, all accessed through a page
> > fault. I hope we won't make this fuzzy in the future by introducing
> > too many magics.
> 
> Hm.  The counters are already fuzzy - the accounting is already doing
> two different jobs.
> 
> Suppose we want to allow 24GB of logically swapped memory, backed by up
> to 8GB of compressed RAM at a 3:1 ratio, but permit only 4GB of physical
> swap.
> 
> Today we have:
>   memory.swap.max  = ?     /* logical memory requiring a fault */
>   memory.zswap.max = 8 GB  /* RAM consumed by compressed data */
> 
> If memory.swap.max is 4 GB, zswap stops after 4 GB of logical pages,
> despite consuming only 1.33GB of RAM.
> 
> If memory.swap.max is 24 GB, zswap may reach 8GB of memory consumption,
> but the cgroup may also consume up to 24GB of physical swap instead of
> the desired 4GB limit.

Hmm, I mean, this sound a limitation of the zswap.max and not swap.max,
swap.max is doing a prefect job here limiting the logically swapped
out memory.

> 
> memory.swap.max is simply overloaded - there is no way for us to express
> both limits.

Limitation of physical layer is some swap tiering issue I believe. We
don't have swap tiering at the moment, so we can't do that, right?

With tiering limit setting swap.max = 24G seems totally fine here.

> Could we preserve the existing swap semantics and add a physical swap
> counter instead?
> 
>   memory.swap.max  = 24GB  /* logical swapped memory */
>   memory.zswap.max =  8GB  /* compressed RAM limit */
>   memory.pswap.max =  4GB  /* physical storage limit */
> 
> These limits would be independent and compose naturally.
> 
> For a zswap-only workload where we care about RAM consumption but cannot
> predict the compression ratio:
> 
>   memory.swap.max  = max
>   memory.zswap.max = 8GB
>   memory.pswap.max = 0
> 
> For a workload that performs poorly after more than 7GB of its logical
> memory requires swap faults:
> 
>   memory.swap.max  = 7GB   /* workload-specific latency/SLO limit */
>   memory.zswap.max = 8GB   /* uniform compressed-RAM allowance */
>   memory.pswap.max = 0     /* zswap only */
> 
> In short:
> 
>   memory.swap   = logical swap       - workload/SLO limit
>   memory.pswap  = physical swap      - storage limit
>   memory.zswap  = compressed memory  - RAM limit
> 
> This would preserve the existing memory.swap semantics while allowing
> both backing resources to be constrained independently.

Yeah, this part seems better, but a pswap vs zswap still seems maybe
too specific for one single usage? Or too board to be over rided. :)
A actual tiering limit seems better to me.