From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pj1-f51.google.com (mail-pj1-f51.google.com [209.85.216.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9057B40629E for ; Fri, 5 Jun 2026 08:43:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.51 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780648998; cv=none; b=LUdt1rCRjI653/BIbrkuoTYgWSSd9QtNdkVV+GKHkGsM4llv+tyikqNK5KHFcO84qNyroWBIDZnu+VIe7dJsRZH8q03ZpofhC3EXNwO5Pc+cOft36whJWAv37/XCql1D4MsbytGdf5CcS3eoYrgQTemZzuRp7J+aLdLnhd0y4/Y= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780648998; c=relaxed/simple; bh=CUtHBCNi8z+cxNsqQp/7jAzIf9QD7guzxjv+fVd/asw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=EJx3SmkOtOk9gg6STzuXPP1aSkm3ONtzXu+z05QeSiRPHTY9cB2NoHwkA3R4GlyrsUE7GjpHxdCbY+pH6CdIGfTSF1zP9irzJzv8jp1o7MJX26eXjcCCTSKT6MuvcPygsOirGIGFKNnt+uIUCTJeFWmdF7IvvigvVnJfvcb+3rE= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=PSdJMMjJ; arc=none smtp.client-ip=209.85.216.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="PSdJMMjJ" Received: by mail-pj1-f51.google.com with SMTP id 98e67ed59e1d1-36ba3ea5c46so1019629a91.1 for ; Fri, 05 Jun 2026 01:43:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1780648996; x=1781253796; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=n185DjbA+W+3OzU5ke/OSZXijWXnafvOkeAJL8VmlKA=; b=PSdJMMjJuHEEtsJ5weOMor17w933HhLuEGNJlEica8iOwZnt3IEBQqAp0oHAy+uH4n MId4dSgm1bs9eKouYxEBF/vrqHQP6bdQ7gOhpVyyfEDgMCsC0y3/tYd6Nn0M2Bwjk149 yl+AOWPjVkZ7ojnbgmvePy4wrre6G2crTcxSdafYlmqXlk+ZIqEN7Tqh3P7hT+DWFjzm 5R5TCDigUSALGcTu9Z+NHxC0E7yvd7Rn+UWnnJh8Jnn4+J6J/dvK1nhjdHgAbdWbTeBq DZgkckBShOlAnUATx4qQy0nNDoeYz+uqLS1J3nZo0iekbPTewlkQrDkaqMw1P3Jk1Rf8 mWxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780648996; x=1781253796; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=n185DjbA+W+3OzU5ke/OSZXijWXnafvOkeAJL8VmlKA=; b=bCdGJZsjKFhbPnNZgm6mtMS+rEu7UHRUwrDq1Ph6O6W9lge10AbbiiWtBdhxE2jwHj Vl2V7w81cB8JOp3eNMPFI5p8mY2z1sXNPVt8M2wIeV21g43htCQrZPvKiZ/z88zT33SP 4s7GYE5YmkH+iQb20XuL2TIETDYc3MxXJZIm6YzjG1aio3h/P68lWLQsoTGgY2ToUOUL gtkvqMxbYj80YMj0Uk6UQ4o8zfry3leaKuuwzbqNVytOBo4HpM1SjpNSAwH/hoeD3czM uOIHveHuCFsLtWhiEu28iCtscpqcHjsYDEy6KmTfLiRIe2uQVdll4buHOBLiSG4d8y0L Kz8g== X-Forwarded-Encrypted: i=1; AFNElJ/meJxZjYcggsDG2OKz4Vf2neTWh/RD4w39CkJhhcev+J+YBX+JLBLrnbwSQkyWzS4nmE4ZVqNrvXJn7nE=@vger.kernel.org X-Gm-Message-State: AOJu0YxoV6sGsf3khmn/WmXosUMSpXMFwdkmevM8ZcXK7eoVWpyLTl1B +gAMCx6vA27TqvALgz95SgG9/j6d9H8dQUKJ9xOClF3kiQPRpPRAhQtJ X-Gm-Gg: Acq92OEL8lBpkyi+rIAtZmKceH4Y3Ak7IdT53SNN7loC+E5revTF4tuTaRj2dozRX4p kipQREx8ZXgz86IdGaRaZbaAzEqNYDajdx/coomijwhNk/GxpxLyqOUL1Og8K962JPz9H26p5v/ o8RmDVQqbURfcPumk6YnY3isGauwUdceXd5EFukIH7tiN5QN4ttddApHg2VmVfcqPIA4F0sS+Cp XA44YQPFqNCAq3yW0O6aX8Y3elSB3VmBnjrimbPYVV4VWn1Q+hiLQZRCbMF/2WZlJWHo69L2BGs lLzOjkEffEunP8t6KeCt7NXHylpMK7K0RgCZmuTQKQWUOKhHn7WLYS7na4G+8Q0K1I/Acck84Xo 02WFVsKr9ghpngLAjXfqNsBildSWJ72Ji38OfH3QQ3gayhZ162nMY8+8/XABDdzeJaC89lvPzti 5lXxeag/3RQ475WMBGFZy4h71qzzT0g30hetUB/XKqYfdbNauhqQ== X-Received: by 2002:a17:90b:57c4:b0:35f:b647:d98a with SMTP id 98e67ed59e1d1-370ee64373dmr2739847a91.5.1780648995812; Fri, 05 Jun 2026 01:43:15 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3712fcb2607sm926632a91.0.2026.06.05.01.43.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 05 Jun 2026 01:43:15 -0700 (PDT) From: Wenchao Hao To: Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Sergey Senozhatsky Cc: Nhat Pham , Joshua Hahn , Barry Song , Xueyuan Chen , Wenchao Hao Subject: [PATCH v3 3/4] mm/zsmalloc: drop class lock before freeing zspage Date: Fri, 5 Jun 2026 16:42:41 +0800 Message-Id: <20260605084242.1549811-4-haowenchao22@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260605084242.1549811-1-haowenchao22@gmail.com> References: <20260605084242.1549811-1-haowenchao22@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Xueyuan Chen Currently in zs_free(), the class->lock is held until the zspage is completely freed and the counters are updated. However, freeing pages back to the buddy allocator requires acquiring the zone lock. Under heavy memory pressure, zone lock contention can be severe. When this happens, the CPU holding the class->lock will stall waiting for the zone lock, thereby blocking all other CPUs attempting to acquire the same class->lock. This patch shrinks the critical section of the class->lock to reduce lock contention. By moving the actual page freeing process outside the class->lock, we can improve the concurrency performance of zs_free(). Testing on the RADXA O6 platform shows that with 12 CPUs concurrently performing zs_free() operations, the execution time is reduced by 20%. Signed-off-by: Xueyuan Chen Reviewed-by: Nhat Pham Reviewed-by: Joshua Hahn Signed-off-by: Wenchao Hao --- mm/zsmalloc.c | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 88d10f814da9..2ecdf79cea03 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -856,13 +856,10 @@ static int trylock_zspage(struct zspage *zspage) return 0; } -static void __free_zspage(struct zs_pool *pool, struct size_class *class, - struct zspage *zspage) +static inline void __free_zspage_lockless(struct zs_pool *pool, struct zspage *zspage) { struct zpdesc *zpdesc, *next; - assert_spin_locked(&class->lock); - VM_BUG_ON(get_zspage_inuse(zspage)); VM_BUG_ON(zspage->fullness != ZS_INUSE_RATIO_0); @@ -878,7 +875,13 @@ static void __free_zspage(struct zs_pool *pool, struct size_class *class, } while (zpdesc != NULL); cache_free_zspage(zspage); +} +static void __free_zspage(struct zs_pool *pool, struct size_class *class, + struct zspage *zspage) +{ + assert_spin_locked(&class->lock); + __free_zspage_lockless(pool, zspage); class_stat_sub(class, ZS_OBJS_ALLOCATED, class->objs_per_zspage); atomic_long_sub(class->pages_per_zspage, &pool->pages_allocated); } @@ -1492,6 +1495,7 @@ void zs_free(struct zs_pool *pool, unsigned long handle) unsigned long obj; struct size_class *class; int fullness; + struct zspage *zspage_to_free = NULL; if (IS_ERR_OR_NULL((void *)handle)) return; @@ -1502,10 +1506,23 @@ void zs_free(struct zs_pool *pool, unsigned long handle) obj_free(class->size, obj); fullness = fix_fullness_group(class, zspage); - if (fullness == ZS_INUSE_RATIO_0) - free_zspage(pool, class, zspage); + if (fullness == ZS_INUSE_RATIO_0) { + if (trylock_zspage(zspage)) { + remove_zspage(class, zspage); + class_stat_sub(class, ZS_OBJS_ALLOCATED, + class->objs_per_zspage); + zspage_to_free = zspage; + } else { + kick_deferred_free(pool); + } + } spin_unlock(&class->lock); + + if (zspage_to_free) { + __free_zspage_lockless(pool, zspage_to_free); + atomic_long_sub(class->pages_per_zspage, &pool->pages_allocated); + } cache_free_handle(handle); } EXPORT_SYMBOL_GPL(zs_free); -- 2.34.1