From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4B1033DB64D for ; Fri, 5 Jun 2026 08:43:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.50 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780648987; cv=none; b=BXm7v1Yxo0eXT3u7vjloz/d1HiCR61VaUP7iuwvZfQLZQXfmbwA3YrY0AXfOqsAXIXH/Hf7ZymTrcz1l8arqkh4AvSm8boEMkNUUw7v8YPtmduPnay76OEap1dOKGVVkgXVTMN6N7K+IER/6JuZDY5s5FHLja+h3aQD4mxyC+9k= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780648987; c=relaxed/simple; bh=/llJvMgKR9KsvWq57gKJfI6fALdQTJtp5pcsTACyqSo=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=HfkGltM7/H0ejmPIJD9eTkKesSfbBd0mZJbXXA6YBEv7DAf5PPgui/E7seKFKHXj1Kxfg7f0+v7T8BOe7ZRNujXsS64//ALHL8e91XV0IY2y5/p7X8R1eSav9naIYOhyFflnGZ+O1ew+j6Qrey00clUyjy/6pZv7Ge2Rb00/yPU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=WN/HBzJq; arc=none smtp.client-ip=209.85.216.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="WN/HBzJq" Received: by mail-pj1-f50.google.com with SMTP id 98e67ed59e1d1-36b95eb4bb4so1118046a91.3 for ; Fri, 05 Jun 2026 01:43:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1780648985; x=1781253785; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=Uu2aS+uKGdtdYwfqjUd0y2UwAmJFu2eezsVO6MOi2Y4=; b=WN/HBzJqjmD+7lt4863RbvLSACIkAHEOD4NvfbZ6CSEvua3Hy2suUDuy4TxA+ZFqo3 bh4wFwIdn9YFycAku5+D+wVNvwzCXuqCDJw00DasIj5oLH+lTnF0N/wlKkMk3gtXFF+s /iUuUfH+mUZ64nhuzQjWibWRYz+RdaS3f2PZGilIrSbAmc/i6E7NFUpswZhTouVflyXA uNbzllZzQXERCiFysDzUCuyVxjCWbZZNKBYrCQswY74L3eIF6eM8+KwNmrT6fej5JIjH xE5kg5JlxXVPTaF5S50ma/2qS+8LBJDuYK1bZQUv9Mc7u4EnhaUND1Fdq+S3uWB7iVVd DUzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780648985; x=1781253785; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Uu2aS+uKGdtdYwfqjUd0y2UwAmJFu2eezsVO6MOi2Y4=; b=CUiHqO+RAuzog8EsUzAHP7dvzhsm1dscqRk3o1CuW2Ah4zsVLz+1AsRuJ/huSGXP63 0YzPYpC8fvWEvewKWPoU8xhjN3FzUpirJR/S/z7RFP0+uyXueCu0VdezZH621eAMgbnz 3mIFZmfjP0BdbTJhu6JPI2tELIecrxBNeK9ir85HpOgaCfrN26X+VuHty0Amgy/za5pF sO0bGGgQDxT6qfaLuseQuZl76HhVvaOV2ZsKGOsmk3ERxa5Rc7qoCJ/s7hPfAVxw+ar8 iSpcx5A93vXQThsCBlnthBIApxxbjTfjvBFIDE4VlsuigGCJIbuo3PbDlpgRDIdpMN2h QWXw== X-Forwarded-Encrypted: i=1; AFNElJ9WBgvuZq49XyZgLNdBa4lNyIXssNc3cGKOdbn7qN12EezQySP2ArmQqhfHs60fnl0Ow9rVxsPKByLUWBg=@vger.kernel.org X-Gm-Message-State: AOJu0YwzydKiQ5avydOoQfMhVaHvmFWT+SqMJGbiYgZuBSYvSanQ58Ay x1Qop/p4DnsJRu2n8T85YChCSKr3W1nbl5HOyVL5iOM33+5TX3krqceM X-Gm-Gg: Acq92OGoqudWqwfhGjvlE5v3ileppVHutLE84dXyGp4xgD0MuItcSOTOXcKoceKqa8e fcoPTt/RCbHtCNXAc+eTdt/LOsog0esUCeN4sTHowK0nzhbJMDbkhhTb06DqEyII5+zGZcBmBO6 4IdwvuWFFCnoiunmQAzXls65As5iKIdowUGc8v6gDvHHTfsHGmYnQM1G1YWy6sC2lIBVp1Y/kVp csPyBUT7VRtL85OoETGJWiEHDrjvhPJWFYzPGnL1wwci5R9z65OZqD5K1Psw3zENnfwlWV5U0HE Xwfm/gb1PjPnoXmuT5J3eS2LEnKn2w2T/Jxq4zZRkxjL9VWOHF29IhkcbSAkmrNfYpjz3/Evy19 QEY3nm8yUqt8ujlN9xM/Km4RtV4QBmkncqLApFYweaop6T5T1HJgS6mi/mV0tdMwxpJBdkOJeLm In+hyIS2LRAZQ8TpwOHcJqSX+UG6eFtZvNBcAwvF6ex+CrmdnmgdJRyoMo0v8B X-Received: by 2002:a17:90b:2810:b0:366:2e1f:393 with SMTP id 98e67ed59e1d1-370f038831dmr2621326a91.21.1780648985426; Fri, 05 Jun 2026 01:43:05 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3712fcb2607sm926632a91.0.2026.06.05.01.43.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 05 Jun 2026 01:43:05 -0700 (PDT) From: Wenchao Hao To: Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Sergey Senozhatsky Cc: Nhat Pham , Joshua Hahn , Barry Song , Wenchao Hao Subject: [PATCH v3 0/4] mm/zsmalloc: reduce lock contention in zs_free() Date: Fri, 5 Jun 2026 16:42:38 +0800 Message-Id: <20260605084242.1549811-1-haowenchao22@gmail.com> X-Mailer: git-send-email 2.34.1 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Wenchao Hao This series reduces lock contention in zs_free(), which dominates the unmap path under memory pressure on Android (LMK kills) and on x86 servers running zswap-heavy workloads. The current zs_free() takes pool->lock (rwlock, read side) just to look up the size_class for a handle, then takes class->lock and holds it across __free_zspage() which can call into the buddy allocator and acquire zone->lock. Two costs follow: * pool->lock reader-counter cacheline bouncing among concurrent zs_free() callers. * class->lock held across folio_put(), so any zone->lock wait fans out to every other zs_free() on the same class. The series tackles both: Patch 1: encode size_class index into obj alongside PFN and obj_idx, so zs_free() can locate the class without pool->lock. Patch 2: drop pool->lock from zs_free() on 64-bit; 32-bit unchanged. Patch 3: move zspage page-freeing out of class->lock. Patch 4: document the three free_zspage helper variants that result from the split in patch 3. Performance results: Test: each process independently mmap 256MB, write data, madvise MADV_PAGEOUT to swap out via zram (lzo-rle), then concurrent munmap. Raspberry Pi 4B (4-core ARM64 Cortex-A72): mode Base Patched Speedup single 59.0ms 56.0ms 1.05x multi 2p 94.6ms 66.7ms 1.42x multi 4p 202.9ms 110.6ms 1.83x x86 (20-core Intel i7-12700, 16 concurrent processes): mode Base Patched Speedup single 11.7ms 9.8ms 1.19x multi 2p 24.1ms 17.2ms 1.40x multi 4p 63.0ms 45.3ms 1.39x [1] https://lore.kernel.org/linux-mm/20260508061910.3882831-1-haowenchao@xiaomi.com/ [2] https://lore.kernel.org/linux-mm/20260527115930.3138213-1-haowenchao@xiaomi.com/ Changes since v2: - 3/4: drop likely() hint and tidy up else-branch braces, per review nits from Nhat Pham and Joshua Hahn. - 4/4 (new): document free_zspage helper variants in a single comment block, per Nhat Pham. - Pick up Reviewed-by tags from Nhat Pham (1/4, 2/4, 3/4) and Joshua Hahn (3/4). - Fix patch 1/4 and 2/4 From: to match Signed-off-by, per Barry Song. Changes since v1: - Rename obj-encoding macros for clarity. - Make 32-bit / 64-bit handling transparent at call sites (no #ifdef in callers). - Extract a helper to keep zs_free() unified. Wenchao Hao (3): mm/zsmalloc: encode class index in obj value for lockless class lookup mm/zsmalloc: drop pool->lock from zs_free on 64-bit systems mm/zsmalloc: document free_zspage helper variants Xueyuan Chen (1): mm/zsmalloc: drop class lock before freeing zspage mm/zsmalloc.c | 191 +++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 159 insertions(+), 32 deletions(-) -- 2.34.1