From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pj1-f46.google.com (mail-pj1-f46.google.com [209.85.216.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AF8E433D6D2 for ; Tue, 9 Jun 2026 11:35:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.46 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781004955; cv=none; b=cT6wyENeH7KwBy5G5vnn/ujshhKqQ5SpK1kEWmdSFkrgBVY1JTuW4i2d4lOhLa93whtOGiZ6IHjZXKvzV5+q/yq6KLEhzEXFsPDg4tYlx40xsYttQap1PoKaSkO9eLWdO4lqGZIxdN3UjIYkUwZLFH85WWiCj28E0Mrkf8erfIs= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781004955; c=relaxed/simple; bh=OGGbrSlCROhWt8Z0cOuCs0yk99+ngt8H1pgEJ4yeylw=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=ch1XQrAAXageWbkjdiQwx0TdcRqamaQr6kLGLdTVDVoUGVYDC3VblPozq0XhLaTMtys4/X/zKkdTa0rDVZqSe/8jR2Pwm27DAg79paTjAuZy5XCyd/1Kq1+MfJf8NvXLMMz3S8Nc89+yYvj4/IxScbZDb5FAKkqFODixMjg4duI= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=sJMMDaxw; arc=none smtp.client-ip=209.85.216.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="sJMMDaxw" Received: by mail-pj1-f46.google.com with SMTP id 98e67ed59e1d1-36bb3551f6eso4758111a91.1 for ; Tue, 09 Jun 2026 04:35:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1781004954; x=1781609754; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=tTVbdNvFsHy/4J7oFF22y14P3b+5jff+s5QxYuRgUYw=; b=sJMMDaxwaObyjGjEt3XjGWueUiJxk02vig/uYQie8vKWbM5MsvhiOdsPlGrjtfGjgH 5qFqV67XOXctDBtWflAGjM71feMU0vA3y3i6+ro/LyKAH7D6yaE+aZQqg76IR8m+snjl gZNKsTf2hJ9YzFp+LTwvtPukAaQlZPJhjP02rZ5m+dEL34hOtHfKdlDLGvRXSU3SFgI2 URwEn9iBWee9lNj0QA0e7vY1GO69ENhDOvNKL1aGBlGwj2L/94IdPDper/gR769OqoOy LjZdR+zAdWqHD9EcdFUzDC1Rpnzu5B+nCjBp8FNznsJSokBi3y6cG7lKL7nrPuLoTYAh YALg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781004954; x=1781609754; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=tTVbdNvFsHy/4J7oFF22y14P3b+5jff+s5QxYuRgUYw=; b=EFwMFwpJjPqHsnB08CSBDxV0oMV7XguZpx/tCQVmPTgwUaCahoDpraMfsyQNffI+Ey M2XuIWiNHPWcalrFnt/9X/Xo5mSgFS6Lb/I9r6PX/LlQ0Nsb8tMCuqruCpDZfRM0XKzc G6+/1aUQawH2T3F3V7RQqctm4gxdeYI/5g63ncE4e4UzLxUaGR/8CsMvJOczie78foSv dEnRoXXgusfKnj92xAO1HkwLqqQAdVFQHPYWDnC5Nggga1AF3P0MpnWdxrYA9aDFh4h5 hYFVRl9c6f8ZCW+83ymndc8xDI6HIR2BDuM/A6iLKF+QAD8nj50uOAB/F8peEZ2Cjc0j cYEw== X-Forwarded-Encrypted: i=1; AFNElJ8I59mAxa9OKTfVpxgtIhIPgPmsVZrrFYCNEfGn/PJqlANx2j7WOrLbotJi9JAhdJIMBlA23OJa9aPq35A=@vger.kernel.org X-Gm-Message-State: AOJu0YyVs2scKx9siQCzJCQ26wi3w86+V6HLiIxFMcXinanYO1M1RKOS MFUEm3QeRuFbIpNQCU409SvCQubGqVpu0k2JqXhHy/+GRqxyTPauqen+ X-Gm-Gg: Acq92OHRXFbK2Qi78q0tVZKEDdAGUkHLceSuoDznkUfznG2EPhxS4y0sh2Wc8Wgt6yj 2SOW7h/+vrEQHc0UhDHwDtJCBCHtebjHbLtxtbsY2IhuxgvZ22HXDXLVvVRVSNWFveCW40Kfycl iiq5/jeJ+kwgWBc8Oi1e5Lvy7OhVOuXMpQ14AQBLmdtTNSgmUqcsk6TJNoFC9IlirV6H0D6ejs2 EjzeUs5dV0jsGNub7AjZnaObG9OfOnOxxg5bEnOUj3aHlP6MYhdYK5cCDuh/9lHWxQFwGGKUVxS fhZPnUpG7+QIpx1q+BGzdZL/iYKgk0mBHUuMe0n2vRT6bBzedwD5/pvPcW1F4AhQCLfPLSMW8BU UTJfU+NDu3oPG9t9FtM+4UK+tZ0IHZEs8uU94wN+t9BvVD7+w9lQVpvu0beywpuLaK4Bqi4HjpE iuWiAVU7F9Fic33WxY74y321UTIRQzjQDtclj1L0N8Fdzw4u0Siw== X-Received: by 2002:a17:90b:5345:b0:35c:30a8:330 with SMTP id 98e67ed59e1d1-370ebff342fmr20517545a91.0.1781004954013; Tue, 09 Jun 2026 04:35:54 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-36f6c668e98sm19655292a91.5.2026.06.09.04.35.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Jun 2026 04:35:52 -0700 (PDT) From: Wenchao Hao To: Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Sergey Senozhatsky Cc: Nhat Pham , Joshua Hahn , Barry Song , Wenchao Hao Subject: [PATCH v4 0/4] mm/zsmalloc: reduce lock contention in zs_free() Date: Tue, 9 Jun 2026 19:35:16 +0800 Message-Id: <20260609113520.3507783-1-haowenchao22@gmail.com> X-Mailer: git-send-email 2.34.1 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Wenchao Hao This series reduces lock contention in zs_free(), which dominates the unmap path under memory pressure on Android (LMK kills) and on x86 servers running zswap-heavy workloads. The current zs_free() takes pool->lock (rwlock, read side) just to look up the size_class for a handle, then takes class->lock and holds it across __free_zspage() which can call into the buddy allocator and acquire zone->lock. Two costs follow: * pool->lock reader-counter cacheline bouncing among concurrent zs_free() callers. * class->lock held across folio_put(), so any zone->lock wait fans out to every other zs_free() on the same class. The series tackles both: Patch 1: encode size_class index into obj alongside PFN and obj_idx, so zs_free() can locate the class without pool->lock. Patch 2: drop pool->lock from zs_free() on 64-bit; 32-bit unchanged. Patch 3: move zspage page-freeing out of class->lock. Patch 4: document the three free_zspage helper variants that result from the split in patch 3. Performance results: Test: each process independently mmap 256MB, write data, madvise MADV_PAGEOUT to swap out via zram (lzo-rle), then concurrent munmap. Raspberry Pi 4B (4-core ARM64 Cortex-A72): mode Base Patched Speedup single 59.0ms 56.0ms 1.05x multi 2p 94.6ms 66.7ms 1.42x multi 4p 202.9ms 110.6ms 1.83x x86 (20-core Intel i7-12700, 16 concurrent processes): mode Base Patched Speedup single 11.7ms 9.8ms 1.19x multi 2p 24.1ms 17.2ms 1.40x multi 4p 63.0ms 45.3ms 1.39x [1] https://lore.kernel.org/linux-mm/20260508061910.3882831-1-haowenchao@xiaomi.com/ [2] https://lore.kernel.org/linux-mm/20260527115930.3138213-1-haowenchao@xiaomi.com/ [3] https://lore.kernel.org/linux-mm/20260605084242.1549811-1-haowenchao22@gmail.com/ [4] https://lore.kernel.org/linux-mm/20260605084242.1549811-1-haowenchao22@gmail.com/ Changes since v3: - 1/4: gate ZS_OBJ_CLASS_BITS on a new spare-bit check so unsuitable configs (e.g. UML, no sparsemem) fall back to plain obj layout instead of breaking the build. (sashiko AI review) - 2/4: annotate handle_to_obj()/record_obj() with READ_ONCE/WRITE_ONCE to silence KCSAN; sync the #if predicate with patch 1. (sashiko AI review) - 4/4: clarify when async_free_zspage() runs; pick up Reviewed-by. (Nhat Pham) - Drop Reviewed-by from Nhat Pham on 1/4 and 2/4: substantial logic changes since v3, please re-review. Changes since v2: - 3/4: drop likely() hint and tidy up else-branch braces, per review nits from Nhat Pham and Joshua Hahn. - 4/4 (new): document free_zspage helper variants in a single comment block, per Nhat Pham. - Pick up Reviewed-by tags from Nhat Pham (1/4, 2/4, 3/4) and Joshua Hahn (3/4). - Fix patch 1/4 and 2/4 From: to match Signed-off-by, per Barry Song. Changes since v1: - Rename obj-encoding macros for clarity. - Make 32-bit / 64-bit handling transparent at call sites (no #ifdef in callers). - Extract a helper to keep zs_free() unified. Wenchao Hao (3): mm/zsmalloc: encode class index in obj value for lockless class lookup mm/zsmalloc: drop pool->lock from zs_free on 64-bit systems mm/zsmalloc: document free_zspage helper variants Xueyuan Chen (1): mm/zsmalloc: drop class lock before freeing zspage mm/zsmalloc.c | 219 ++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 184 insertions(+), 35 deletions(-) -- 2.34.1