From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 88A0B3C5836 for ; Tue, 16 Jun 2026 03:13:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.171 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781579622; cv=none; b=jVcOv858fIiV/dD6D+PhboQtfyl6SDsnJ0YIDM72MW/fHIwR7A1PSA6KP4znUCNDtKHC/6Zinndu4sY7EdzWI/LuU/JJymFckgfXgSARuuC2e57g+1H6eThYiTrBaA5JNFvQEsmwHc9CoZLemwZtg+Qq4fFUG2oohVKTUEf3z84= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781579622; c=relaxed/simple; bh=FK1Bc1wL01z6MIDrydl0Y2wN14P2laIgAGGwrpJ/CQI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=diPpH6dUsg6PK9sZDNcKEDIK13KSorv//ZhIU/GECOgAyJ2g1ggACJkeSMX3bY0UpqdaSRcj7nkVYJYg1zl9YDTOsRsI5i3gdi8bYOq1j7vm3OlHQQG81+FwfSNMvGxLRq3TSOvC95k3CNHrVwyZvT7Vsg9TDVV/vToWkapqf3U= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=hW8JTjrH; arc=none smtp.client-ip=209.85.214.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hW8JTjrH" Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-2bf30d530bdso40104285ad.3 for ; Mon, 15 Jun 2026 20:13:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1781579621; x=1782184421; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YS1IcSMum0RF5BraEGhpud+NVT65eKWpTnhBO1AgKGw=; b=hW8JTjrHa0C/3C4FuVIms1TQ+0YdmqOtbaWCSacZeLNMmwllZtM2PN8qKx5+f6aQ/c 3XHTYSrlpWWhgOrsg9hFd9Jhs9bv9A7Ri13lhZSZ1QHlGo6cxErOwmQPKIOBdaYS5xzN k65W1vqG/9jim08ko/qrpFbe+FD6WrSpbzAdQwoPgacGqOwfJkVX4D3E1SjH0r3/B3BT 7BizDFXH4iFAaOCXcFt7nl5yUiG8NNde5d4NIiW1QM0rsgcQoUJ4STUGFhYEQ8byfYdf vWw5cjETigvzXxwuh5r5OsYbeTEy4mNKAbM8/4g3GA9wGg5SY3EZBsiFa8v1RjguCVE6 Hrmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781579621; x=1782184421; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=YS1IcSMum0RF5BraEGhpud+NVT65eKWpTnhBO1AgKGw=; b=QgrfUCmrMjGOUOC+h2tZGTFToGDoMrQsED/h6rWyK9yIWrMs5AquoLIsmDzm3mOwbI UEzByQKsGintYm7gUdKBWRzOsI8OeCR1/ipeQvhE+SMKnHtbywLaKROV4vt9vwfA5LPd idrzCIL986slQCAlIExrEyRHfqlaFX563rCoYC63wZZpn5VYHx6zu/eS50xqi/AVwa0Q aQ7/TKxGjztjdv8WK0GCLTzjA/dY4R5TYnDjQhV5y2z4w7yXEet3W176B/99yi5uTWfI HzziOjnXutJ850O3dsSHUUWSH399wEhOm8yl5OuW+4TAe0MAtDW+nwppZPvYP2Fjs78H ztmg== X-Forwarded-Encrypted: i=1; AFNElJ++aSjsDePkGhTmkwXY7apyO1oTop04xXNSx9vqmkX5+gXJrg11O6dxvRW/H6JAIG8+qida5xrnJTMHodY=@vger.kernel.org X-Gm-Message-State: AOJu0Yyg36t9nybp5ct3xRmcd5HOdLAG7ABiOQpY7V036KaLMlukNYY7 YZEyrTc/1kPQvCSdYbXiJ0GZvTnFaZzcf8dyYZfIo5Wv4nHboedBYk+8 X-Gm-Gg: Acq92OGAbmF9/oGrs06N6qM7HYMIZj8cfMMCe89eoN/fNIlPGYPaZzaSaHCVChLQ40S FLtLParzTfncP7UsEnIWrJc8NEcni6l/hys0Wzz7gjOOkSm/EUv8GdFQ3W6Gq6shBF24Lx4a1OI Wxt+FEndrQB2fsPDoFZ53crIh4Lv2HkyhqTCyJrlhHOBzMbPcC9phjdlR6LmtRUCVGBdQu4zygK Igyq8AQf/Kw/kylGMHh1HzVzXW9dloeOP3jvr28z/txEVN7imCLLBEPznfu+vMgjlVuzmzXeZiI YPSrTY0WKH9biKymREzxrpzy3EmlX39lrFzpSiNPxmFfJZy5CPbnwHcHQGD/4L7F3LmmM7TSupj EwMt6mgUfnIfB4xEdo7GC/db6PIWY0fma6wejU02A44qk/px2dxjkpGZkPY5AVH4lAyCxX0U7/c VBoVIaMuQna0FFqPn+iVHi9DJD/6ifS9MkAUnhvdY= X-Received: by 2002:a17:903:3bcf:b0:2be:39bd:8dd8 with SMTP id d9443c01a7336-2c66429433bmr142968875ad.33.1781579620838; Mon, 15 Jun 2026 20:13:40 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2c42f2e5644sm118923385ad.4.2026.06.15.20.13.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Jun 2026 20:13:40 -0700 (PDT) From: Wenchao Hao To: Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Sergey Senozhatsky Cc: Nhat Pham , Joshua Hahn , Barry Song , Xueyuan Chen , Wenchao Hao Subject: [PATCH v5 3/4] mm/zsmalloc: drop class lock before freeing zspage Date: Tue, 16 Jun 2026 11:13:08 +0800 Message-Id: <20260616031309.3036234-4-haowenchao22@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260616031309.3036234-1-haowenchao22@gmail.com> References: <20260616031309.3036234-1-haowenchao22@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Xueyuan Chen Currently in zs_free(), the class->lock is held until the zspage is completely freed and the counters are updated. However, freeing pages back to the buddy allocator requires acquiring the zone lock. Under heavy memory pressure, zone lock contention can be severe. When this happens, the CPU holding the class->lock will stall waiting for the zone lock, thereby blocking all other CPUs attempting to acquire the same class->lock. This patch shrinks the critical section of the class->lock to reduce lock contention. By moving the actual page freeing process outside the class->lock, we can improve the concurrency performance of zs_free(). Testing on the RADXA O6 platform shows that with 12 CPUs concurrently performing zs_free() operations, the execution time is reduced by 20%. Signed-off-by: Xueyuan Chen Reviewed-by: Nhat Pham Reviewed-by: Joshua Hahn Signed-off-by: Wenchao Hao --- mm/zsmalloc.c | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 1e9a150cd255..8a0c0c68121d 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -884,13 +884,10 @@ static int trylock_zspage(struct zspage *zspage) return 0; } -static void __free_zspage(struct zs_pool *pool, struct size_class *class, - struct zspage *zspage) +static inline void __free_zspage_lockless(struct zspage *zspage) { struct zpdesc *zpdesc, *next; - assert_spin_locked(&class->lock); - VM_BUG_ON(get_zspage_inuse(zspage)); VM_BUG_ON(zspage->fullness != ZS_INUSE_RATIO_0); @@ -906,7 +903,13 @@ static void __free_zspage(struct zs_pool *pool, struct size_class *class, } while (zpdesc != NULL); cache_free_zspage(zspage); +} +static void __free_zspage(struct zs_pool *pool, struct size_class *class, + struct zspage *zspage) +{ + assert_spin_locked(&class->lock); + __free_zspage_lockless(zspage); class_stat_sub(class, ZS_OBJS_ALLOCATED, class->objs_per_zspage); atomic_long_sub(class->pages_per_zspage, &pool->pages_allocated); } @@ -1519,6 +1522,7 @@ void zs_free(struct zs_pool *pool, unsigned long handle) unsigned long obj; struct size_class *class; int fullness; + struct zspage *zspage_to_free = NULL; if (IS_ERR_OR_NULL((void *)handle)) return; @@ -1529,10 +1533,23 @@ void zs_free(struct zs_pool *pool, unsigned long handle) obj_free(class->size, obj); fullness = fix_fullness_group(class, zspage); - if (fullness == ZS_INUSE_RATIO_0) - free_zspage(pool, class, zspage); + if (fullness == ZS_INUSE_RATIO_0) { + if (trylock_zspage(zspage)) { + remove_zspage(class, zspage); + class_stat_sub(class, ZS_OBJS_ALLOCATED, + class->objs_per_zspage); + zspage_to_free = zspage; + } else { + kick_deferred_free(pool); + } + } spin_unlock(&class->lock); + + if (zspage_to_free) { + __free_zspage_lockless(zspage_to_free); + atomic_long_sub(class->pages_per_zspage, &pool->pages_allocated); + } cache_free_handle(handle); } EXPORT_SYMBOL_GPL(zs_free); -- 2.34.1