From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7701F3CC30F for ; Wed, 27 May 2026 12:01:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779883307; cv=none; b=fOC4Ujhf/4QM0sliiS4rwU9eyLu7EdBw2sAucJSbAi0PDQSQiP+lJxXejm/4c85SunIReWmy6yMJQCUbDOF79sh/6946NfJIJslV94OmP3tXuIFsB+fG3LG9OO2GCh+dFUncJA4TEa3K1IrhvzMxVtARTEH/xAIRfYdUTttXfYE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779883307; c=relaxed/simple; bh=mqCjFoA3Bl5H+Kl1p6vF/4bkM7qtfYlpBLnONaydEDU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=cQ8RjvWmhtBPDLdhBBOobqfGqeQFAvwB60BeO3ugovcvWvGkFw/uwQqxhe19xSgUrjr8ipTbduaPMOJtZeF4v/0TzXi/ZUddaJwV1w9/CRyNIl0G/lyba7bGzAivZG2wqLcD7Oh1ysRLJy24oaYw0056l4emYA05o5mjC1olZBg= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=hkqOkRFs; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hkqOkRFs" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-2ba6485d219so89507885ad.3 for ; Wed, 27 May 2026 05:01:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1779883304; x=1780488104; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3mefnb7wtXl9fo2oJQ/q5HPl8VRiw71wTKLdzGgpyQU=; b=hkqOkRFsOg3GJrO/VF/1vjXQmhvmSrh5nMUEwj08AG37y2vHIto0mOvq+qHi9BCO3E J7WaqoC5fYuvZRRtjCsVZuwD85Gb4KTuypOngLRfZ+kMbjJQryzSLRTUuPOPJMsPHy/A PFPGP9mAdAsWbExCdw9GysPEyI9OFSmokajeKChOrbFUjiE/SoxxN+A5satscYRnchLH oQRc2sIUjblPHrXVptaT5yeczFuabJU0/WbI7ffk5lw4I3apowHUajAbjR0f4PDOxlAQ /cYYiVd10tJ/Dpgd3KH8CtoSFuDAUv9jy7yebl0OMixcw5WA4+BIVxs9Tu993Jd+/BQB dR0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779883304; x=1780488104; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=3mefnb7wtXl9fo2oJQ/q5HPl8VRiw71wTKLdzGgpyQU=; b=E5UxUYneJU7AI8VJZPE5tWwYIK/BSLcBtJM9N23ovRSr06aIdRa15Rm46AnuRuIA6n krirAbnNcWUNK1F5u1sEGbfDSEcL5mfEWuQaKEkIZmKuyy1pwrYWlEcSFeMznAwdrfMP iPsU58SttYm+PvEk9wlRGIzHsqwStFDYBWoCoUJc3KtLrn9iVzcoYKj72cqexcZ83Zpl b6sPXNeGEPDPozksl9YxWrWEhZM1Kun4sUFF2Sn8fRRS/s+5qu7A/MA3nmUPeiKe7l1Q eymOJ2TfQ5Dq/wrZubuJAPuoZ1344ULqDnrwnfIcxDS1NvKGEsodpU0eT7V+xHG0Xh3F uX8g== X-Forwarded-Encrypted: i=1; AFNElJ+W+p3xwmyIGwr/bkO4NjMwB8JXq7YZB48YJ6JFu0/i7Teb6wazzTKxoOTd8ssHB7j1DY5kMCYVRmMdhJ0=@vger.kernel.org X-Gm-Message-State: AOJu0YxGTdWVrNUOt2egB9Uy2Ht1WnlXy8pZH4qcLcwU6m6TBuONzMcE 9agCm4L57kJ3CmAtSxoGeiPigIgkEwsreLJWMFqzy5lM827yqROvlUC7WPr1nw== X-Gm-Gg: Acq92OF/JFLwIM/1O7goPu8I3WM9tv65d+nPATYAMWBOgi13JlSkvHS2Q4Q9pL2uPaB MDGJnle1fJWgJW5UVSQYtm2EqEKl1Yg2w3lU1nqGoSaAN7MrA9VQmqXJgThyQh39kE8PxAjtRrO Vz+nOYs4+g/ZjrmjE3tk390pC7x9gXxdnuvx27cnON9BRSeg2Rh5ktAM7TNWOJ+RuBSN51X6w0J E7M7gfj3ZK/kvgT224ER7ccm5Gd4bM/4CQ6ljowgBf/8wdqJpiHBngS7EgysWYqMtB/hZp7djxY KfXMpl4RQlST/av4dNvmiqvA9tRYgTe6XjF4QA0zDOlfY15jYfyaoOYQSktGcI4Mx8rWTXQ2I9h ft0S0WGwhkY+yTUIhNnNMzp/KDIbecq8we6Eakd5AmsT+43L4iKQFt2ixL8woT5lsb63flBBldC dT0fKXKoDJoGRnFESG+jpbpWLBmnrUhmmEDCB9e/U463wvOUAk X-Received: by 2002:a17:903:b85:b0:2ba:bfc:7699 with SMTP id d9443c01a7336-2beb05908eamr256392215ad.17.1779883304577; Wed, 27 May 2026 05:01:44 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2beb56b5886sm136250465ad.20.2026.05.27.05.01.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 May 2026 05:01:44 -0700 (PDT) From: Wenchao Hao X-Google-Original-From: Wenchao Hao To: Andrew Morton , Nhat Pham , Yosry Ahmed , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Sergey Senozhatsky Cc: Xueyuan Chen , Wenchao Hao Subject: [RFC PATCH v2 3/3] mm/zsmalloc: drop class lock before freeing zspage Date: Wed, 27 May 2026 19:59:30 +0800 Message-Id: <20260527115930.3138213-4-haowenchao@xiaomi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260527115930.3138213-1-haowenchao@xiaomi.com> References: <20260527115930.3138213-1-haowenchao@xiaomi.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Xueyuan Chen Currently in zs_free(), the class->lock is held until the zspage is completely freed and the counters are updated. However, freeing pages back to the buddy allocator requires acquiring the zone lock. Under heavy memory pressure, zone lock contention can be severe. When this happens, the CPU holding the class->lock will stall waiting for the zone lock, thereby blocking all other CPUs attempting to acquire the same class->lock. This patch shrinks the critical section of the class->lock to reduce lock contention. By moving the actual page freeing process outside the class->lock, we can improve the concurrency performance of zs_free(). Testing on the RADXA O6 platform shows that with 12 CPUs concurrently performing zs_free() operations, the execution time is reduced by 20%. Signed-off-by: Xueyuan Chen Signed-off-by: Wenchao Hao --- mm/zsmalloc.c | 28 ++++++++++++++++++++++------ 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 88d10f814da9..5511c347d00b 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -856,13 +856,10 @@ static int trylock_zspage(struct zspage *zspage) return 0; } -static void __free_zspage(struct zs_pool *pool, struct size_class *class, - struct zspage *zspage) +static inline void __free_zspage_lockless(struct zs_pool *pool, struct zspage *zspage) { struct zpdesc *zpdesc, *next; - assert_spin_locked(&class->lock); - VM_BUG_ON(get_zspage_inuse(zspage)); VM_BUG_ON(zspage->fullness != ZS_INUSE_RATIO_0); @@ -878,7 +875,13 @@ static void __free_zspage(struct zs_pool *pool, struct size_class *class, } while (zpdesc != NULL); cache_free_zspage(zspage); +} +static void __free_zspage(struct zs_pool *pool, struct size_class *class, + struct zspage *zspage) +{ + assert_spin_locked(&class->lock); + __free_zspage_lockless(pool, zspage); class_stat_sub(class, ZS_OBJS_ALLOCATED, class->objs_per_zspage); atomic_long_sub(class->pages_per_zspage, &pool->pages_allocated); } @@ -1492,6 +1495,7 @@ void zs_free(struct zs_pool *pool, unsigned long handle) unsigned long obj; struct size_class *class; int fullness; + struct zspage *zspage_to_free = NULL; if (IS_ERR_OR_NULL((void *)handle)) return; @@ -1502,10 +1506,22 @@ void zs_free(struct zs_pool *pool, unsigned long handle) obj_free(class->size, obj); fullness = fix_fullness_group(class, zspage); - if (fullness == ZS_INUSE_RATIO_0) - free_zspage(pool, class, zspage); + if (fullness == ZS_INUSE_RATIO_0) { + if (trylock_zspage(zspage)) { + remove_zspage(class, zspage); + class_stat_sub(class, ZS_OBJS_ALLOCATED, + class->objs_per_zspage); + zspage_to_free = zspage; + } else + kick_deferred_free(pool); + } spin_unlock(&class->lock); + + if (likely(zspage_to_free)) { + __free_zspage_lockless(pool, zspage_to_free); + atomic_long_sub(class->pages_per_zspage, &pool->pages_allocated); + } cache_free_handle(handle); } EXPORT_SYMBOL_GPL(zs_free); -- 2.34.1