[PATCH v3 0/4] mm/zsmalloc: reduce lock contention in zs_free()

[PATCH v3 0/4] mm/zsmalloc: reduce lock contention in zs_free()

[Wenchao Hao]

From: Wenchao Hao <haowenchao@xiaomi.com>

This series reduces lock contention in zs_free(), which dominates the
unmap path under memory pressure on Android (LMK kills) and on x86
servers running zswap-heavy workloads.

The current zs_free() takes pool->lock (rwlock, read side) just to
look up the size_class for a handle, then takes class->lock and holds
it across __free_zspage() which can call into the buddy allocator and
acquire zone->lock.  Two costs follow:

  * pool->lock reader-counter cacheline bouncing among concurrent
    zs_free() callers.
  * class->lock held across folio_put(), so any zone->lock wait
    fans out to every other zs_free() on the same class.

The series tackles both:

  Patch 1: encode size_class index into obj alongside PFN and obj_idx,
           so zs_free() can locate the class without pool->lock.
  Patch 2: drop pool->lock from zs_free() on 64-bit; 32-bit unchanged.
  Patch 3: move zspage page-freeing out of class->lock.
  Patch 4: document the three free_zspage helper variants that result
           from the split in patch 3.

Performance results:

Test: each process independently mmap 256MB, write data, madvise
MADV_PAGEOUT to swap out via zram (lzo-rle), then concurrent munmap.

Raspberry Pi 4B (4-core ARM64 Cortex-A72):

  mode        Base       Patched     Speedup
  single      59.0ms     56.0ms      1.05x
  multi 2p    94.6ms     66.7ms      1.42x
  multi 4p    202.9ms    110.6ms     1.83x

x86 (20-core Intel i7-12700, 16 concurrent processes):

  mode        Base       Patched     Speedup
  single      11.7ms     9.8ms       1.19x
  multi 2p    24.1ms     17.2ms      1.40x
  multi 4p    63.0ms     45.3ms      1.39x

[1] https://lore.kernel.org/linux-mm/20260508061910.3882831-1-haowenchao@xiaomi.com/
[2] https://lore.kernel.org/linux-mm/20260527115930.3138213-1-haowenchao@xiaomi.com/

Changes since v2:
- 3/4: drop likely() hint and tidy up else-branch braces, per review
  nits from Nhat Pham and Joshua Hahn.
- 4/4 (new): document free_zspage helper variants in a single comment
  block, per Nhat Pham.
- Pick up Reviewed-by tags from Nhat Pham (1/4, 2/4, 3/4) and
  Joshua Hahn (3/4).
- Fix patch 1/4 and 2/4 From: to match Signed-off-by, per Barry Song.

Changes since v1:
- Rename obj-encoding macros for clarity.
- Make 32-bit / 64-bit handling transparent at call sites
  (no #ifdef in callers).
- Extract a helper to keep zs_free() unified.


Wenchao Hao (3):
  mm/zsmalloc: encode class index in obj value for lockless class lookup
  mm/zsmalloc: drop pool->lock from zs_free on 64-bit systems
  mm/zsmalloc: document free_zspage helper variants

Xueyuan Chen (1):
  mm/zsmalloc: drop class lock before freeing zspage

 mm/zsmalloc.c | 191 +++++++++++++++++++++++++++++++++++++++++---------
 1 file changed, 159 insertions(+), 32 deletions(-)

-- 
2.34.1

[PATCH v3 1/4] mm/zsmalloc: encode class index in obj value for lockless class lookup

[Wenchao Hao]

From: Wenchao Hao <haowenchao@xiaomi.com>

Encode the size_class index (class_idx) into the obj value so that
zs_free() can determine the correct size_class without dereferencing
the handle->obj->PFN->zpdesc->zspage->class chain under pool->lock.
class_idx is invariant across page migration (only PFN is rewritten),
so a lockless read of obj always yields a valid class_idx.

The space below the PFN field in obj is over-provisioned on 64-bit
systems, with more bits than obj_idx needs. Split that space into
class_idx and obj_idx subfields:

 |<-- _PFN_BITS -->|<-- ZS_OBJ_CLASS_BITS -->|<-- ZS_OBJ_IDX_BITS -->|
 +-----------------+-------------------------+-----------------------+
 |       PFN       |        class_idx        |        obj_idx        |
 +-----------------+-------------------------+-----------------------+
MSB                ^                                                LSB
                   |
                   +-- ZS_OBJ_PFN_SHIFT

The macro layout changes as follows:

    Before            After               Meaning
    ----------------  ------------------  ----------------------------
    OBJ_INDEX_BITS    ZS_OBJ_IDX_BITS     width of obj_idx subfield
    OBJ_INDEX_MASK    ZS_OBJ_IDX_MASK     mask  of obj_idx subfield
    (n/a)             ZS_OBJ_CLASS_BITS   width of class_idx subfield
    (n/a)             ZS_OBJ_CLASS_MASK   mask  of class_idx subfield
    (n/a)             ZS_OBJ_PFN_SHIFT    bit offset of PFN in obj

On 32-bit systems there is no spare room for class_idx, so the
encoding is disabled (ZS_OBJ_CLASS_BITS = 0) and the obj layout
remains [PFN | obj_idx].

Reviewed-by: Nhat Pham <nphamcs@gmail.com>
Signed-off-by: Wenchao Hao <haowenchao@xiaomi.com>
---
 mm/zsmalloc.c | 80 ++++++++++++++++++++++++++++++++++++++++++---------
 1 file changed, 66 insertions(+), 14 deletions(-)

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 63128ddb7959..6b0014b43408 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -67,8 +67,8 @@
 #define MAX_POSSIBLE_PHYSMEM_BITS MAX_PHYSMEM_BITS
 #else
 /*
- * If this definition of MAX_PHYSMEM_BITS is used, OBJ_INDEX_BITS will just
- * be PAGE_SHIFT
+ * If this definition of MAX_PHYSMEM_BITS is used, ZS_OBJ_PFN_SHIFT will
+ * just be PAGE_SHIFT
  */
 #define MAX_POSSIBLE_PHYSMEM_BITS BITS_PER_LONG
 #endif
@@ -88,8 +88,27 @@
 #define OBJ_TAG_BITS	1
 #define OBJ_TAG_MASK	OBJ_ALLOCATED_TAG
 
-#define OBJ_INDEX_BITS	(BITS_PER_LONG - _PFN_BITS)
-#define OBJ_INDEX_MASK	((_AC(1, UL) << OBJ_INDEX_BITS) - 1)
+/*
+ * obj is encoded as [PFN | class_idx | obj_idx] within an unsigned long:
+ *
+ *   |<-- _PFN_BITS -->|<-- ZS_OBJ_CLASS_BITS -->|<-- ZS_OBJ_IDX_BITS -->|
+ *   +-----------------+-------------------------+-----------------------+
+ *   |       PFN       |        class_idx        |        obj_idx        |
+ *   +-----------------+-------------------------+-----------------------+
+ *  MSB                ^                                                LSB
+ *                     |
+ *                     +-- ZS_OBJ_PFN_SHIFT
+ *
+ * Encoding class_idx into obj lets zs_free() locate the size_class
+ * without holding pool->lock; class_idx is invariant across page
+ * migration (only PFN changes), so a lockless read of the obj value
+ * always yields a valid class_idx.
+ *
+ * On 32-bit systems there is no spare room for class_idx, so
+ * ZS_OBJ_CLASS_BITS is 0 and the layout collapses to the original
+ * [PFN | obj_idx] without any ifdef in callers.
+ */
+#define ZS_OBJ_PFN_SHIFT	(BITS_PER_LONG - _PFN_BITS)
 
 #define HUGE_BITS	1
 #define FULLNESS_BITS	4
@@ -98,9 +117,29 @@
 
 #define ZS_MAX_PAGES_PER_ZSPAGE	(_AC(CONFIG_ZSMALLOC_CHAIN_SIZE, UL))
 
+/*
+ * Reuse the width that struct zspage already reserves for its
+ * class field (zspage->class:CLASS_BITS + 1) for the class_idx
+ * field encoded in obj.  On 32-bit there is no spare room, so set
+ * it to 0; the encoded class_idx then folds to a constant 0 and
+ * the layout collapses back to [PFN | obj_idx].
+ */
+#if BITS_PER_LONG >= 64
+#define ZS_OBJ_CLASS_BITS	(CLASS_BITS + 1)
+#else
+#define ZS_OBJ_CLASS_BITS	0
+#endif
+#define ZS_OBJ_CLASS_MASK	((_AC(1, UL) << ZS_OBJ_CLASS_BITS) - 1)
+
+#define ZS_OBJ_IDX_BITS		(ZS_OBJ_PFN_SHIFT - ZS_OBJ_CLASS_BITS)
+#define ZS_OBJ_IDX_MASK		((_AC(1, UL) << ZS_OBJ_IDX_BITS) - 1)
+
+static_assert(ZS_OBJ_IDX_BITS > 0,
+	      "zsmalloc: PFN + class_idx leave no room for obj_idx");
+
 /* ZS_MIN_ALLOC_SIZE must be multiple of ZS_ALIGN */
 #define ZS_MIN_ALLOC_SIZE \
-	MAX(32, (ZS_MAX_PAGES_PER_ZSPAGE << PAGE_SHIFT >> OBJ_INDEX_BITS))
+	MAX(32, (ZS_MAX_PAGES_PER_ZSPAGE << PAGE_SHIFT >> ZS_OBJ_IDX_BITS))
 /* each chunk includes extra space to keep handle */
 #define ZS_MAX_ALLOC_SIZE	PAGE_SIZE
 
@@ -721,26 +760,38 @@ static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc)
 static void obj_to_location(unsigned long obj, struct zpdesc **zpdesc,
 				unsigned int *obj_idx)
 {
-	*zpdesc = pfn_zpdesc(obj >> OBJ_INDEX_BITS);
-	*obj_idx = (obj & OBJ_INDEX_MASK);
+	*zpdesc = pfn_zpdesc(obj >> ZS_OBJ_PFN_SHIFT);
+	*obj_idx = (obj & ZS_OBJ_IDX_MASK);
 }
 
 static void obj_to_zpdesc(unsigned long obj, struct zpdesc **zpdesc)
 {
-	*zpdesc = pfn_zpdesc(obj >> OBJ_INDEX_BITS);
+	*zpdesc = pfn_zpdesc(obj >> ZS_OBJ_PFN_SHIFT);
+}
+
+/*
+ * On 32-bit systems ZS_OBJ_CLASS_BITS is 0 and ZS_OBJ_CLASS_MASK is 0,
+ * so this collapses to a constant 0.  No ifdef needed at the call site.
+ */
+static unsigned int obj_to_class_idx(unsigned long obj)
+{
+	return (obj >> ZS_OBJ_IDX_BITS) & ZS_OBJ_CLASS_MASK;
 }
 
 /**
- * location_to_obj - get obj value encoded from (<zpdesc>, <obj_idx>)
+ * location_to_obj - encode (<zpdesc>, <obj_idx>, <class_idx>) into obj value
  * @zpdesc: zpdesc object resides in zspage
  * @obj_idx: object index
+ * @class_idx: size class index; ignored on 32-bit (ZS_OBJ_CLASS_BITS == 0)
  */
-static unsigned long location_to_obj(struct zpdesc *zpdesc, unsigned int obj_idx)
+static unsigned long location_to_obj(struct zpdesc *zpdesc, unsigned int obj_idx,
+				     unsigned int class_idx)
 {
 	unsigned long obj;
 
-	obj = zpdesc_pfn(zpdesc) << OBJ_INDEX_BITS;
-	obj |= obj_idx & OBJ_INDEX_MASK;
+	obj  = zpdesc_pfn(zpdesc) << ZS_OBJ_PFN_SHIFT;
+	obj |= (unsigned long)(class_idx & ZS_OBJ_CLASS_MASK) << ZS_OBJ_IDX_BITS;
+	obj |= obj_idx & ZS_OBJ_IDX_MASK;
 
 	return obj;
 }
@@ -1276,7 +1327,7 @@ static unsigned long obj_malloc(struct zs_pool *pool,
 	kunmap_local(vaddr);
 	mod_zspage_inuse(zspage, 1);
 
-	obj = location_to_obj(m_zpdesc, obj);
+	obj = location_to_obj(m_zpdesc, obj, zspage->class);
 	record_obj(handle, obj);
 
 	return obj;
@@ -1762,7 +1813,8 @@ static int zs_page_migrate(struct page *newpage, struct page *page,
 
 			old_obj = handle_to_obj(handle);
 			obj_to_location(old_obj, &dummy, &obj_idx);
-			new_obj = (unsigned long)location_to_obj(newzpdesc, obj_idx);
+			new_obj = location_to_obj(newzpdesc, obj_idx,
+						  obj_to_class_idx(old_obj));
 			record_obj(handle, new_obj);
 		}
 	}
-- 
2.34.1

[PATCH v3 2/4] mm/zsmalloc: drop pool->lock from zs_free on 64-bit systems

[Wenchao Hao]

From: Wenchao Hao <haowenchao@xiaomi.com>

With class_idx encoded in obj, zs_free() can locate the size_class
without holding pool->lock on 64-bit systems.  Page migration also
takes class->lock and only rewrites the PFN field of obj, so:

  1. read obj locklessly,
  2. lock the size_class derived from obj's class_idx,
  3. re-read obj under class->lock to get a stable PFN.

This eliminates the rwlock read-side cacheline bouncing between
zs_free() and migration/compaction on multi-core systems.

On 32-bit systems pool->lock is preserved.

Reviewed-by: Nhat Pham <nphamcs@gmail.com>
Signed-off-by: Wenchao Hao <haowenchao@xiaomi.com>
---
 mm/zsmalloc.c | 67 ++++++++++++++++++++++++++++++++++++++++++---------
 1 file changed, 55 insertions(+), 12 deletions(-)

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 6b0014b43408..88d10f814da9 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -21,6 +21,10 @@
  *	pool->lock
  *	class->lock
  *	zspage->lock
+ *
+ * On 64-bit systems zs_free() does not take pool->lock; it locates
+ * the size_class using class_idx encoded in obj and serializes against
+ * page migration via class->lock.
  */
 
 #include <linux/module.h>
@@ -1432,10 +1436,59 @@ static void obj_free(int class_size, unsigned long obj)
 	mod_zspage_inuse(zspage, -1);
 }
 
+/*
+ * Resolve @handle to its zspage / size_class and acquire class->lock.
+ *
+ * 64-bit: class_idx is encoded in obj and is invariant under page
+ * migration, so the handle can be read locklessly to pick the
+ * size_class.  Once class->lock is held migration is blocked and the
+ * handle is re-read to obtain a stable PFN.
+ *
+ * 32-bit: the unlocked load of *(unsigned long *)handle is not
+ * single-copy atomic and class_idx is not encoded (ZS_OBJ_CLASS_BITS
+ * == 0), so fall back to pool->lock for the lookup.
+ */
+#if BITS_PER_LONG >= 64
+static inline void obj_handle_class_lock(struct zs_pool *pool, unsigned long handle,
+					 unsigned long *objp, struct zspage **zspagep,
+					 struct size_class **classp)
+	__acquires(&(*classp)->lock)
+{
+	struct zpdesc *f_zpdesc;
+	unsigned long obj;
+
+	obj = handle_to_obj(handle);
+	*classp = pool->size_class[obj_to_class_idx(obj)];
+	spin_lock(&(*classp)->lock);
+	/* Re-read under class->lock: PFN is now stable vs migration. */
+	obj = handle_to_obj(handle);
+	obj_to_zpdesc(obj, &f_zpdesc);
+	*zspagep = get_zspage(f_zpdesc);
+	*objp = obj;
+}
+#else
+static inline void obj_handle_class_lock(struct zs_pool *pool, unsigned long handle,
+					 unsigned long *objp, struct zspage **zspagep,
+					 struct size_class **classp)
+	__acquires(&(*classp)->lock)
+{
+	struct zpdesc *f_zpdesc;
+	unsigned long obj;
+
+	read_lock(&pool->lock);
+	obj = handle_to_obj(handle);
+	obj_to_zpdesc(obj, &f_zpdesc);
+	*zspagep = get_zspage(f_zpdesc);
+	*classp = zspage_class(pool, *zspagep);
+	spin_lock(&(*classp)->lock);
+	read_unlock(&pool->lock);
+	*objp = obj;
+}
+#endif
+
 void zs_free(struct zs_pool *pool, unsigned long handle)
 {
 	struct zspage *zspage;
-	struct zpdesc *f_zpdesc;
 	unsigned long obj;
 	struct size_class *class;
 	int fullness;
@@ -1443,17 +1496,7 @@ void zs_free(struct zs_pool *pool, unsigned long handle)
 	if (IS_ERR_OR_NULL((void *)handle))
 		return;
 
-	/*
-	 * The pool->lock protects the race with zpage's migration
-	 * so it's safe to get the page from handle.
-	 */
-	read_lock(&pool->lock);
-	obj = handle_to_obj(handle);
-	obj_to_zpdesc(obj, &f_zpdesc);
-	zspage = get_zspage(f_zpdesc);
-	class = zspage_class(pool, zspage);
-	spin_lock(&class->lock);
-	read_unlock(&pool->lock);
+	obj_handle_class_lock(pool, handle, &obj, &zspage, &class);
 
 	class_stat_sub(class, ZS_OBJS_INUSE, 1);
 	obj_free(class->size, obj);
-- 
2.34.1

[PATCH v3 3/4] mm/zsmalloc: drop class lock before freeing zspage

[Wenchao Hao]

From: Xueyuan Chen <xueyuan.chen21@gmail.com>

Currently in zs_free(), the class->lock is held until the zspage is
completely freed and the counters are updated. However, freeing pages
back to the buddy allocator requires acquiring the zone lock.

Under heavy memory pressure, zone lock contention can be severe. When
this happens, the CPU holding the class->lock will stall waiting for
the zone lock, thereby blocking all other CPUs attempting to acquire
the same class->lock.

This patch shrinks the critical section of the class->lock to reduce
lock contention. By moving the actual page freeing process outside the
class->lock, we can improve the concurrency performance of zs_free().

Testing on the RADXA O6 platform shows that with 12 CPUs concurrently
performing zs_free() operations, the execution time is reduced by 20%.

Signed-off-by: Xueyuan Chen <xueyuan.chen21@gmail.com>
Reviewed-by: Nhat Pham <nphamcs@gmail.com>
Reviewed-by: Joshua Hahn <joshua.hahnjy@gmail.com>
Signed-off-by: Wenchao Hao <haowenchao@xiaomi.com>
---
 mm/zsmalloc.c | 29 +++++++++++++++++++++++------
 1 file changed, 23 insertions(+), 6 deletions(-)

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 88d10f814da9..2ecdf79cea03 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -856,13 +856,10 @@ static int trylock_zspage(struct zspage *zspage)
 	return 0;
 }
 
-static void __free_zspage(struct zs_pool *pool, struct size_class *class,
-				struct zspage *zspage)
+static inline void __free_zspage_lockless(struct zs_pool *pool, struct zspage *zspage)
 {
 	struct zpdesc *zpdesc, *next;
 
-	assert_spin_locked(&class->lock);
-
 	VM_BUG_ON(get_zspage_inuse(zspage));
 	VM_BUG_ON(zspage->fullness != ZS_INUSE_RATIO_0);
 
@@ -878,7 +875,13 @@ static void __free_zspage(struct zs_pool *pool, struct size_class *class,
 	} while (zpdesc != NULL);
 
 	cache_free_zspage(zspage);
+}
 
+static void __free_zspage(struct zs_pool *pool, struct size_class *class,
+			  struct zspage *zspage)
+{
+	assert_spin_locked(&class->lock);
+	__free_zspage_lockless(pool, zspage);
 	class_stat_sub(class, ZS_OBJS_ALLOCATED, class->objs_per_zspage);
 	atomic_long_sub(class->pages_per_zspage, &pool->pages_allocated);
 }
@@ -1492,6 +1495,7 @@ void zs_free(struct zs_pool *pool, unsigned long handle)
 	unsigned long obj;
 	struct size_class *class;
 	int fullness;
+	struct zspage *zspage_to_free = NULL;
 
 	if (IS_ERR_OR_NULL((void *)handle))
 		return;
@@ -1502,10 +1506,23 @@ void zs_free(struct zs_pool *pool, unsigned long handle)
 	obj_free(class->size, obj);
 
 	fullness = fix_fullness_group(class, zspage);
-	if (fullness == ZS_INUSE_RATIO_0)
-		free_zspage(pool, class, zspage);
+	if (fullness == ZS_INUSE_RATIO_0) {
+		if (trylock_zspage(zspage)) {
+			remove_zspage(class, zspage);
+			class_stat_sub(class, ZS_OBJS_ALLOCATED,
+				       class->objs_per_zspage);
+			zspage_to_free = zspage;
+		} else {
+			kick_deferred_free(pool);
+		}
+	}
 
 	spin_unlock(&class->lock);
+
+	if (zspage_to_free) {
+		__free_zspage_lockless(pool, zspage_to_free);
+		atomic_long_sub(class->pages_per_zspage, &pool->pages_allocated);
+	}
 	cache_free_handle(handle);
 }
 EXPORT_SYMBOL_GPL(zs_free);
-- 
2.34.1

[PATCH v3 4/4] mm/zsmalloc: document free_zspage helper variants

[Wenchao Hao]

From: Wenchao Hao <haowenchao@xiaomi.com>

After splitting __free_zspage() into a lockless core and a wrapper that
does the class-stat bookkeeping, three similarly-named helpers coexist:
free_zspage / __free_zspage / __free_zspage_lockless.

Add a comment block above them describing what each does and where it
is used, so the names are not easy to confuse.

No functional change.

Suggested-by: Nhat Pham <nphamcs@gmail.com>
Signed-off-by: Wenchao Hao <haowenchao@xiaomi.com>
---
 mm/zsmalloc.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 2ecdf79cea03..9f588b63ec0d 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -856,6 +856,21 @@ static int trylock_zspage(struct zspage *zspage)
 	return 0;
 }
 
+/*
+ * Three free helpers, kept apart here:
+ *
+ * __free_zspage_lockless(): bare core; walks zpdescs and returns pages
+ *   to the buddy allocator.  Caller owns all zpdesc locks and has
+ *   removed the zspage from its class list.  Used by zs_free() outside
+ *   class->lock so the buddy-side work does not stall the class.
+ *
+ * __free_zspage(): __free_zspage_lockless() + per-class accounting,
+ *   under class->lock.  Used by async_free_zspage().
+ *
+ * free_zspage(): full wrapper - trylock zpdescs, remove from class
+ *   list, call __free_zspage(); kicks deferred free on contention.
+ *   Used by compaction.
+ */
 static inline void __free_zspage_lockless(struct zs_pool *pool, struct zspage *zspage)
 {
 	struct zpdesc *zpdesc, *next;
-- 
2.34.1

Re: [PATCH v3 4/4] mm/zsmalloc: document free_zspage helper variants

[Nhat Pham]

On Fri, Jun 5, 2026 at 1:43 AM Wenchao Hao <haowenchao22@gmail.com> wrote:
>
> From: Wenchao Hao <haowenchao@xiaomi.com>
>
> After splitting __free_zspage() into a lockless core and a wrapper that
> does the class-stat bookkeeping, three similarly-named helpers coexist:
> free_zspage / __free_zspage / __free_zspage_lockless.
>
> Add a comment block above them describing what each does and where it
> is used, so the names are not easy to confuse.
>
> No functional change.
>
> Suggested-by: Nhat Pham <nphamcs@gmail.com>
> Signed-off-by: Wenchao Hao <haowenchao@xiaomi.com>

I think it LGTM, but I'll let the zsmalloc maintainers have their say :)

Feel free to add:
Reviewed-by: Nhat Pham <nphamcs@gmail.com>



> ---
>  mm/zsmalloc.c | 15 +++++++++++++++
>  1 file changed, 15 insertions(+)
>
> diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> index 2ecdf79cea03..9f588b63ec0d 100644
> --- a/mm/zsmalloc.c
> +++ b/mm/zsmalloc.c
> @@ -856,6 +856,21 @@ static int trylock_zspage(struct zspage *zspage)
>         return 0;
>  }
>
> +/*
> + * Three free helpers, kept apart here:
> + *
> + * __free_zspage_lockless(): bare core; walks zpdescs and returns pages
> + *   to the buddy allocator.  Caller owns all zpdesc locks and has
> + *   removed the zspage from its class list.  Used by zs_free() outside
> + *   class->lock so the buddy-side work does not stall the class.
> + *
> + * __free_zspage(): __free_zspage_lockless() + per-class accounting,
> + *   under class->lock.  Used by async_free_zspage().

nit: maybe a short note on when async_free_zspage is needed?

(IIUC, only when trylock_zspage() fails in zs_free(), correct?)

> + *
> + * free_zspage(): full wrapper - trylock zpdescs, remove from class
> + *   list, call __free_zspage(); kicks deferred free on contention.
> + *   Used by compaction.
> + */
>  static inline void __free_zspage_lockless(struct zs_pool *pool, struct zspage *zspage)
>  {
>         struct zpdesc *zpdesc, *next;
> --
> 2.34.1
>

Re: [PATCH v3 0/4] mm/zsmalloc: reduce lock contention in zs_free()

[Andrew Morton]

On Fri,  5 Jun 2026 16:42:38 +0800 Wenchao Hao <haowenchao22@gmail.com> wrote:

> This series reduces lock contention in zs_free(), which dominates the
> unmap path under memory pressure on Android (LMK kills) and on x86
> servers running zswap-heavy workloads.

Thanks.  It's late in the cycle so I'd prefer to merge this after 7.2-rc1, please.

AI review said a few things:
	https://sashiko.dev/#/patchset/20260605084242.1549811-1-haowenchao22@gmail.com

Re: [PATCH v3 0/4] mm/zsmalloc: reduce lock contention in zs_free()

[Wenchao Hao]

On Sat, Jun 6, 2026 at 9:19 AM Andrew Morton <akpm@linux-foundation.org> wrote:
>
> On Fri,  5 Jun 2026 16:42:38 +0800 Wenchao Hao <haowenchao22@gmail.com> wrote:
>
> > This series reduces lock contention in zs_free(), which dominates the
> > unmap path under memory pressure on Android (LMK kills) and on x86
> > servers running zswap-heavy workloads.
>
> Thanks.  It's late in the cycle so I'd prefer to merge this after 7.2-rc1, please.
>
> AI review said a few things:
>         https://sashiko.dev/#/patchset/20260605084242.1549811-1-haowenchao22@gmail.com

Thanks Andrew.  Both AI findings are valid and have been fixed in v4
(just sent):

 - 1/4: on 64-bit systems without sparsemem (e.g. UML) obj had no spare
   bits for class_idx; gate ZS_OBJ_CLASS_BITS on a spare-bit check so
   such configs fall back to [PFN | obj_idx].

 - 2/4: annotate the lockless handle access with READ_ONCE/WRITE_ONCE.

Wenchao

Re: [PATCH v3 4/4] mm/zsmalloc: document free_zspage helper variants

[Wenchao Hao]

On Sat, Jun 6, 2026 at 12:35 AM Nhat Pham <nphamcs@gmail.com> wrote:
>
> On Fri, Jun 5, 2026 at 1:43 AM Wenchao Hao <haowenchao22@gmail.com> wrote:
> >
> > From: Wenchao Hao <haowenchao@xiaomi.com>
> >
> > After splitting __free_zspage() into a lockless core and a wrapper that
> > does the class-stat bookkeeping, three similarly-named helpers coexist:
> > free_zspage / __free_zspage / __free_zspage_lockless.
> >
> > Add a comment block above them describing what each does and where it
> > is used, so the names are not easy to confuse.
> >
> > No functional change.
> >
> > Suggested-by: Nhat Pham <nphamcs@gmail.com>
> > Signed-off-by: Wenchao Hao <haowenchao@xiaomi.com>
>
> I think it LGTM, but I'll let the zsmalloc maintainers have their say :)
>
> Feel free to add:
> Reviewed-by: Nhat Pham <nphamcs@gmail.com>
>
>
>
> > ---
> >  mm/zsmalloc.c | 15 +++++++++++++++
> >  1 file changed, 15 insertions(+)
> >
> > diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> > index 2ecdf79cea03..9f588b63ec0d 100644
> > --- a/mm/zsmalloc.c
> > +++ b/mm/zsmalloc.c
> > @@ -856,6 +856,21 @@ static int trylock_zspage(struct zspage *zspage)
> >         return 0;
> >  }
> >
> > +/*
> > + * Three free helpers, kept apart here:
> > + *
> > + * __free_zspage_lockless(): bare core; walks zpdescs and returns pages
> > + *   to the buddy allocator.  Caller owns all zpdesc locks and has
> > + *   removed the zspage from its class list.  Used by zs_free() outside
> > + *   class->lock so the buddy-side work does not stall the class.
> > + *
> > + * __free_zspage(): __free_zspage_lockless() + per-class accounting,
> > + *   under class->lock.  Used by async_free_zspage().
>
> nit: maybe a short note on when async_free_zspage is needed?
>
> (IIUC, only when trylock_zspage() fails in zs_free(), correct?)
>

Thanks Nhat.  Yes, that's exactly it.  In v4 the comment now reads:

  __free_zspage(): __free_zspage_lockless() + per-class accounting,
    under class->lock.  Used by async_free_zspage(), the worker for
    zspages whose trylock_zspage() failed.

Your Reviewed-by is picked up.  Note that 1/4 and 2/4 have non-trivial
rework in v4 in response to the sashiko AI review, so I dropped your
R-b on those two patches -- a re-review would be appreciated.

Wenchao