From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2A84B37C108 for ; Tue, 16 Jun 2026 03:13:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.179 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781579613; cv=none; b=qJA/jMZNKJQGCikRaG3QQvTRxrsPqr3ANCYkm0RQrMrr8RFT4MmUNjSJD2Fi4TefoHx7CUjfuBkOtIhlEHgFUqO7V/1ha837tToUT/sThA7MyWcCFkpQ2sI8Gbajg0+yav1hTcMM/r4V/qiiYWtOGxKLrKkA2zSJh5cAzwZbl50= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781579613; c=relaxed/simple; bh=ctpKUucN5caNKVtEk1cFw44Ve1jNeI46Y/pIDAf5Kwc=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=BtAildG1NBmyf2fGJbv+xV5S5r17+Jd8+WYgmdTA1/3FoaXOQynfoThegGX2pFHzSosRzF2u7AKH6G35bN6c1BYNw8nA+xu4DQQzg8Cm31g+czzZrbMm+tN2ZVoK7LAu/GwVLGfMmL9OZ5wzU3oBIuRI+9McqHVlhwC2lhiqHZ4= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=AEpHnha5; arc=none smtp.client-ip=209.85.214.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="AEpHnha5" Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-2c0c35980fdso41642895ad.2 for ; Mon, 15 Jun 2026 20:13:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1781579611; x=1782184411; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=tGxSkaNfy75vw30Wuf7uLMuhDYY8QKbdwifEr0WCcPA=; b=AEpHnha5vOQiWPmPjjYoB4i4I9qb8Si2gT0ltIhgEyKEAtaoQSOis7aocxb1pY1KiN 3OuGF/tfWi2avhz/s25znEClZuXN0SqSgXvhOzeKjVnrt+g89kJw4FabQ4ot1HX6ogx8 k908h6tMVIqXfntDxue7gm+z9SqsLZcm7P969mVINk8PP8C/VliZ7mcpYg/HrbJNoBU0 PheTWtK578KnG9waC+wOQm9TWncpzfMWuwoEWyVfY1hXn4EZIV7z7EG2ku0dvnWfIc7A e3vNHOd2fVERk8wivEnAL2VnI9FteuyhGq57y2hLa0YWwtv1s+NTLPL5o1G6SQ5LYVRh 8xQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781579611; x=1782184411; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=tGxSkaNfy75vw30Wuf7uLMuhDYY8QKbdwifEr0WCcPA=; b=q0gmItT429hG49L4ScDRXrwHAAAa4iLa9CNJGaVkSYJdmwuKb2oaDZbl6IGrYkFAJ2 B3UeLXqTSLAa4UDm9Tfvb3JfFPfYAvaoTLIXmbKTFMUl2nd1NL1AKK6W6maDwWTmCMNv 2gI8ii0xF9PF2UjTwemC7Xbj0Tu6uEQo/Ar/L0m0TK2wYi7SCmGqkCeL3ZF27vgi0v74 SFUybhUW3fMD0o5fup+fOr6XIkRPMXqEf4eUtB1LcUwsC/tnRWi8aVHs6YgJwTByQbeN maiBvh3iDJHFO42OYH273s16tfDthN/fbmEtScxFbv/KFiy9zyG1/Ho8n1qG7VSUjzHS YTCg== X-Forwarded-Encrypted: i=1; AFNElJ9MToDTpFUlQihlg6gq6Br410SB4LPmscwUDUxrVVhuEFw7N4J7TgBa7Ra9IGPmiF+8atkC68DJSQWjxf4=@vger.kernel.org X-Gm-Message-State: AOJu0YwlKA/BF1n3tqMEmwyY0DWeEGi/jC8VXxVx+dFqScaIQiragTvh RR/nQEvZd+1YkKmmAmpR7usqdfA7iie6QWR3wdNflALjVUY136MJyI9bhMaj6Q== X-Gm-Gg: Acq92OFdaQFTEba6oGwhXc6DDRTLIOWGL4roNZO72m17DI8wD3GXeyPfUj3NfV6xJjE u4XaPADv4XeOxU5EY587NA1sNarzceUaz3iivDFLaRzObo4itTFo2wGP7EvSmlDiYMoEwbZn6Nd QI2xsrpqkIMhx3GLK//nHzMjq6H4En9Ut0ZlSp7iR1bN1yBYnMTc6Rpw0Y/HOahJbFlMpyjtnGg Q7wA7uz8765hFJYoV6Tjeq6S9Z36OT+LvydFcgbwMSKHW1mBFyL769sD3ySfqZFNi0f0nRTxcWz rSfbkAq68AiemqWB12USah7lBJaI1ml+slCYjt/xy7jFptFnXvhXNdTSfYYGiXkcu948GZ839kJ INwz6LJKwu7m5ltvuonmvitSV8/w2w0CMomNyInlnxlAT/amD/EuWCV0uzp2CsMVx99vzXWKqz6 +bU3GZZbnEudauw52347ar+6GT52ycto1IJQ+r2tarefR41OD42Q== X-Received: by 2002:a17:903:3b86:b0:2bf:114b:924 with SMTP id d9443c01a7336-2c69a1c83ddmr15707525ad.34.1781579611441; Mon, 15 Jun 2026 20:13:31 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2c42f2e5644sm118923385ad.4.2026.06.15.20.13.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Jun 2026 20:13:31 -0700 (PDT) From: Wenchao Hao To: Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Sergey Senozhatsky Cc: Nhat Pham , Joshua Hahn , Barry Song , Wenchao Hao Subject: [PATCH v5 0/4] mm/zsmalloc: reduce lock contention in zs_free() Date: Tue, 16 Jun 2026 11:13:05 +0800 Message-Id: <20260616031309.3036234-1-haowenchao22@gmail.com> X-Mailer: git-send-email 2.34.1 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Wenchao Hao This series reduces lock contention in zs_free(), which dominates the unmap path under memory pressure on Android (LMK kills) and on x86 servers running zswap-heavy workloads. The current zs_free() takes pool->lock (rwlock, read side) just to look up the size_class for a handle, then takes class->lock and holds it across __free_zspage() which can call into the buddy allocator and acquire zone->lock. Two costs follow: * pool->lock reader-counter cacheline bouncing among concurrent zs_free() callers. * class->lock held across folio_put(), so any zone->lock wait fans out to every other zs_free() on the same class. The series tackles both: Patch 1: encode size_class index into obj alongside PFN and obj_idx, so zs_free() can locate the class without pool->lock. Patch 2: drop pool->lock from zs_free() on 64-bit; 32-bit unchanged. Patch 3: move zspage page-freeing out of class->lock. Patch 4: document the three free_zspage helper variants that result from the split in patch 3. Performance results: Test: each process independently mmap 256MB, write data, madvise MADV_PAGEOUT to swap out via zram (lzo-rle), then concurrent munmap. Raspberry Pi 4B (4-core ARM64 Cortex-A72): mode Base Patched Speedup single 59.0ms 56.0ms 1.05x multi 2p 94.6ms 66.7ms 1.42x multi 4p 202.9ms 110.6ms 1.83x x86 (20-core Intel i7-12700, 16 concurrent processes): mode Base Patched Speedup single 11.7ms 9.8ms 1.19x multi 2p 24.1ms 17.2ms 1.40x multi 4p 63.0ms 45.3ms 1.39x [1] https://lore.kernel.org/linux-mm/20260508061910.3882831-1-haowenchao@xiaomi.com/ [2] https://lore.kernel.org/linux-mm/20260527115930.3138213-1-haowenchao@xiaomi.com/ [3] https://lore.kernel.org/linux-mm/20260605084242.1549811-1-haowenchao22@gmail.com/ [4] https://lore.kernel.org/linux-mm/20260609113520.3507783-1-haowenchao22@gmail.com/ Changes since v4: - 1/4: rename macros to make their intent clearer, following the kernel's *_BITS convention (Nhat Pham, Barry Song). - 3/4: drop the unused pool argument from __free_zspage_lockless(). Changes since v3: - 1/4: gate ZS_OBJ_CLASS_BITS on a new spare-bit check so unsuitable configs (e.g. UML, no sparsemem) fall back to plain obj layout instead of breaking the build. (sashiko AI review) - 2/4: annotate handle_to_obj()/record_obj() with READ_ONCE/WRITE_ONCE to silence KCSAN; sync the #if predicate with patch 1. (sashiko AI review) - 4/4: clarify when async_free_zspage() runs; pick up Reviewed-by. (Nhat Pham) - Drop Reviewed-by from Nhat Pham on 1/4 and 2/4: substantial logic changes since v3, please re-review. Changes since v2: - 3/4: drop likely() hint and tidy up else-branch braces, per review nits from Nhat Pham and Joshua Hahn. - 4/4 (new): document free_zspage helper variants in a single comment block, per Nhat Pham. - Pick up Reviewed-by tags from Nhat Pham (1/4, 2/4, 3/4) and Joshua Hahn (3/4). - Fix patch 1/4 and 2/4 From: to match Signed-off-by, per Barry Song. Changes since v1: - Rename obj-encoding macros for clarity. - Make 32-bit / 64-bit handling transparent at call sites (no #ifdef in callers). - Extract a helper to keep zs_free() unified. Wenchao Hao (3): mm/zsmalloc: encode class index in obj value for lockless class lookup mm/zsmalloc: drop pool->lock from zs_free on 64-bit systems mm/zsmalloc: document free_zspage helper variants Xueyuan Chen (1): mm/zsmalloc: drop class lock before freeing zspage mm/zsmalloc.c | 225 ++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 190 insertions(+), 35 deletions(-) -- 2.34.1