From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pj1-f45.google.com (mail-pj1-f45.google.com [209.85.216.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3168D3FA5D8 for ; Tue, 9 Jun 2026 11:36:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.45 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781004966; cv=none; b=DqasEE1xcxdzAIO0iiZtPnuOGFES8iv7qPtWnbZEOGPmz+Lfn8+htTeAPmlkx+oJWEYE93NOwIdN8WCwnY8FbbQZUHT7uzwO5g8zFzVKa1fX6ZBwLHtivHIgDmeKNYRPHlUQbx1JR/6TQEW/z+O62+Os/KYIqhxxa+SMgFVBN9s= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781004966; c=relaxed/simple; bh=LTHsAjXU2tuHJhc4xIn70WBn3xyeFoMsOm5VDayBji8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=I23HAoHjp2cuSERWHdxlqsrYZAamTvOH22GPBYEEhAD0GF+9y9fn9LlXHu5JMDG1kpjB8s6i9JdekYRSgK5D7l0+s+A1k7ryt9ZI1drP31sko6KdQ49prqRhVBCNu+yHeletnp/T9nwBdCkNwoDiXL94Y244dvnGymePNVAvTTA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=nXisYCDc; arc=none smtp.client-ip=209.85.216.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="nXisYCDc" Received: by mail-pj1-f45.google.com with SMTP id 98e67ed59e1d1-37474afe908so759208a91.1 for ; Tue, 09 Jun 2026 04:36:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1781004964; x=1781609764; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6/nTjntuZjnrvE8ZyTpIYFayYnVSKdUVLKXvluFBzrU=; b=nXisYCDc+sawcwatQh9bWEhfpy5oFg+OBNGNdv4y0mnkiNGnl1Z0hBqZPhcGqGY7kg y8JmTECDsRc/X5qX/2tMciLI2Mk/9zcSrO3doHRJZn9dMmKgjbacp2upSf3Nyln39a50 g1r0S8FeqvmxGyzah5uCb6CWqyBWV3IrpYNdd69PodvEjq8hYTFvuMWnleqADcuAT9tc dZP1TsgY1WbU3uY2alFc25w3Rxk7hmcwDHho6pD+GGtxXLDA/P5J4W925Np0aVMW1z76 bTaztRvkARJRrn5ZDJ6s+j1vo4hZUh2cBe+22cdjDBUD88DlqL1O6pJCNC0kwtP2PxSH xtHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781004964; x=1781609764; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=6/nTjntuZjnrvE8ZyTpIYFayYnVSKdUVLKXvluFBzrU=; b=JYglL84MNGELDlE1zU9kO8msQBo1nTUaEnESGyotaNXGBEzycOTXfT8fBm9+blA/SB qbPHRokakcs9SzT0V6HhaezlzmIrbF43SW35nijiQgPrecZ2uawV6Ze8cNE6YExp11tH WaO61rDF0gsuv6d9iS9nKSvaisHbO89uT32tC20n6cFLq0SGct3wIpOwWFdycbrVKLXj Uk1PdkPLgrJ+7eFkMCbEVnMLle3hyEzVxe642barU6O04vJDHsJebi9+6lnLUjaxwRmj sDrgnpMzIjEJ1o0u/NIDr5hX5vBmYIWp8VAkVpkZJwfwlV8lQmt6qwOxmi0ZV7xmZwH9 TXYQ== X-Forwarded-Encrypted: i=1; AFNElJ/SiwHqJqKoZTWNL2YIKNy0FmgOM+kNrOxgzs+ciTcPsQMpfwdhSboUqXJ5oXl/Uv5s7qOHkgd0aKoRp5s=@vger.kernel.org X-Gm-Message-State: AOJu0YziXcc3wn1SYjeUwmMquWVkIFXWkPcwveo7A13tkyP/G1J52zfY 9hJBpfJesBNqVSf96JlKaeCVBxsHhbYQt8OtBkZd04a2I661Y6eAv8kV X-Gm-Gg: Acq92OEgWwpClodvzrGZpqzrbJcE8dBhS67v8h8DXWK/CQajMcg8B8bLJyhQfhgqvg+ dNTd2P+gEbmDrdZUkanNz+YxGJx9rsoHk0T1Nl6AM+vWC+pnCUuYk78OJkx1zdjT5ZKYaiq0qtL KyXU6IhEykkbdTE1iI0eIk9+qgJxB6VBf5TbV4iipvua5OJR682jmNZ+PJP03OxY/i9zlq5Uywb mG9CIn/7REXl15OMwBVzDXpDILWUCh7Q9PdyA2FCIW4VXQ77IyQmhwnLW/uFFMeZiOP5mktZRw+ 69s9pqoZJkQpGLGAq2HLT8eHG//Laetb5ImytI+qFVzFgV7yUREENNKY2ZteivYKg173bwMtwQV LX8HHNmv9HbzMs6iq4NxZL5tw9+SGhMmJx3sKMqzACg2bZ9MJnGmMdLcFJA/8Utma7mcNgxqpe5 M8gOjnKT1RDj+nM82xWaj5S9F1k7c4tPLtiPAvKLJ+tabra+7Zrw== X-Received: by 2002:a17:90b:2742:b0:36a:c4b:76cf with SMTP id 98e67ed59e1d1-370f0b55e45mr21925890a91.23.1781004964510; Tue, 09 Jun 2026 04:36:04 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-36f6c668e98sm19655292a91.5.2026.06.09.04.36.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Jun 2026 04:36:03 -0700 (PDT) From: Wenchao Hao To: Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Sergey Senozhatsky Cc: Nhat Pham , Joshua Hahn , Barry Song , Xueyuan Chen , Wenchao Hao Subject: [PATCH v4 3/4] mm/zsmalloc: drop class lock before freeing zspage Date: Tue, 9 Jun 2026 19:35:19 +0800 Message-Id: <20260609113520.3507783-4-haowenchao22@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260609113520.3507783-1-haowenchao22@gmail.com> References: <20260609113520.3507783-1-haowenchao22@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Xueyuan Chen Currently in zs_free(), the class->lock is held until the zspage is completely freed and the counters are updated. However, freeing pages back to the buddy allocator requires acquiring the zone lock. Under heavy memory pressure, zone lock contention can be severe. When this happens, the CPU holding the class->lock will stall waiting for the zone lock, thereby blocking all other CPUs attempting to acquire the same class->lock. This patch shrinks the critical section of the class->lock to reduce lock contention. By moving the actual page freeing process outside the class->lock, we can improve the concurrency performance of zs_free(). Testing on the RADXA O6 platform shows that with 12 CPUs concurrently performing zs_free() operations, the execution time is reduced by 20%. Signed-off-by: Xueyuan Chen Reviewed-by: Nhat Pham Reviewed-by: Joshua Hahn Signed-off-by: Wenchao Hao --- mm/zsmalloc.c | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index fe20ab297542..1602ccb6d814 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -878,13 +878,10 @@ static int trylock_zspage(struct zspage *zspage) return 0; } -static void __free_zspage(struct zs_pool *pool, struct size_class *class, - struct zspage *zspage) +static inline void __free_zspage_lockless(struct zs_pool *pool, struct zspage *zspage) { struct zpdesc *zpdesc, *next; - assert_spin_locked(&class->lock); - VM_BUG_ON(get_zspage_inuse(zspage)); VM_BUG_ON(zspage->fullness != ZS_INUSE_RATIO_0); @@ -900,7 +897,13 @@ static void __free_zspage(struct zs_pool *pool, struct size_class *class, } while (zpdesc != NULL); cache_free_zspage(zspage); +} +static void __free_zspage(struct zs_pool *pool, struct size_class *class, + struct zspage *zspage) +{ + assert_spin_locked(&class->lock); + __free_zspage_lockless(pool, zspage); class_stat_sub(class, ZS_OBJS_ALLOCATED, class->objs_per_zspage); atomic_long_sub(class->pages_per_zspage, &pool->pages_allocated); } @@ -1513,6 +1516,7 @@ void zs_free(struct zs_pool *pool, unsigned long handle) unsigned long obj; struct size_class *class; int fullness; + struct zspage *zspage_to_free = NULL; if (IS_ERR_OR_NULL((void *)handle)) return; @@ -1523,10 +1527,23 @@ void zs_free(struct zs_pool *pool, unsigned long handle) obj_free(class->size, obj); fullness = fix_fullness_group(class, zspage); - if (fullness == ZS_INUSE_RATIO_0) - free_zspage(pool, class, zspage); + if (fullness == ZS_INUSE_RATIO_0) { + if (trylock_zspage(zspage)) { + remove_zspage(class, zspage); + class_stat_sub(class, ZS_OBJS_ALLOCATED, + class->objs_per_zspage); + zspage_to_free = zspage; + } else { + kick_deferred_free(pool); + } + } spin_unlock(&class->lock); + + if (zspage_to_free) { + __free_zspage_lockless(pool, zspage_to_free); + atomic_long_sub(class->pages_per_zspage, &pool->pages_allocated); + } cache_free_handle(handle); } EXPORT_SYMBOL_GPL(zs_free); -- 2.34.1