From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=74.125.82.47 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1795148366; b=qzn6/UL3PI6iJxuaJfUr0Uk9OJSqIoeEZzI+ThyHsOl6/WjFUMgQVL56HANQY+1u2zyW 85gvX2BNtI4sbjVYKUCbq6SQX3PAJIif0+cP5QcZCVFjgztofQUFRObqcR8NHg6ubHpwj ZX5gjM3YuVevyR2zbcMl/KX+h2zlUsWpYq88CqyPiQmQU8MILU76hApcGAsaL16M3D4I5 dKJVAeqW9ZI6n1N84Sy86ooVvKj1Tj8kxXLuXm6JufoY9cMLwXaKk8sl245yxlmuO8g5+ x/f1uFZCcgBDxg2VvniKi8zvxTQlyg3UQdfJnsBnjgYWM6RVHhGGV7Q8RK8HTr7G/4A== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1795148366; h=DMARC-Filter:Received:DKIM-Signature:X-Google-DKIM-Signature: X-Forwarded-Encrypted:X-Gm-Message-State:X-Gm-Gg:X-Received:Received: From:To:Cc:Subject:Date:Message-Id:X-Mailer:In-Reply-To:References: MIME-Version:Content-Transfer-Encoding; bh=Tw2IAI5wSBNupIKIxL5Ig7aobgIjCUpdJvEt5vs8Kl4=; b=qqG6AIChWdWKxX/cp/5BSKYlE87Ofxb8HXwdCzPdIFEyrS1433LiflKaehpdZZJ11Av8 YMrl1UBOJKgFwzaRcsDGGKoxBarIH0Y1tWWQbEB4jwkWG6XhtxvsIUTHBaHWeH/gJIAq4 RYhIyEIHJjLzkzGBfKGGbWdXTQkWDoxd4LqtvCyTT+feFnYpw4JVg2mUxWu0kFnrLqwLO vKmO3WvJPfH5sYAgX4u3fZFtc9WIt4NCADLFtGZEBzvrmW8Fs2n4U1NQTDPGFmHCCJvvc 0LX1YNzz5h5GgvsY+02jMLcYZlsGsyuSrilinZZOhtnrxMCwH2UgY2rQH6J+DgTR++w== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20260707 header.b=vxTRE8OP; arc=none smtp.remote-ip=74.125.82.47 Received: from mail-dl1-f47.google.com (mail-dl1-f47.google.com [74.125.82.47]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5E50C60A3812 for ; Fri, 20 Nov 2026 04:19:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.2 smtp.subspace.kernel.org 5E50C60A3812 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20260707 header.b=vxTRE8OP Received: by mail-dl1-f47.google.com with SMTP id a92af1059eb24-1b3ada81853so542592c88.0 for ; Thu, 19 Nov 2026 20:19:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1795148364; x=1795753164; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=Tw2IAI5wSBNupIKIxL5Ig7aobgIjCUpdJvEt5vs8Kl4=; b=vxTRE8OPeHqrGUKmjXMjN6bclXHCrqiOHW5v7f+vC0IOosiXZaakFI0plwoiny3h4V 45LTBXRpY0wENCMkhZvI9eCoDmrlHMB0gufo6mZIqoq7m45MMF9PS5Hp/ymqcnmqz+Y9 +dxze6qqVjJDisRHHuuIvtO7TYWJsnZe/BTIQkFuLCdZuDUBx3bC2kXidV4OLfHLY1+A Wyn1lcbWgqARr1yz1isfJcLUhgw1HrwR1BJJwtvX7cCqEe3dND/3cT/tMFs6MPX9TMH3 NcGSVfHKTge6t5AO4/t2ALWlh+aj9TCk8QpLh20JHlOUA9xQdEVI3CiJlfJ2sabNsGba rmRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1795148364; x=1795753164; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=Tw2IAI5wSBNupIKIxL5Ig7aobgIjCUpdJvEt5vs8Kl4=; b=dN1M5D5+gXc3fHP528Zy4gi1uI6Ad2ni/8DJF2w9lVDaV2ktzu6PpvN+NhKfdP9vRJ uTVA9A7AAPWXBf/sXByLn2HIS98IR1D3pRrJ3cn/R16y2hPQQGyrt5ZJXD853xSsWIHe SPz5xlcZpcZWkRtNadiEHZRnkcMOsHzXxxy/KnQdbi1sAUXm4IFRh211bK1RNIEf4yS8 Lm3H5wpD4NZz4SGbBmj5dw9b/7VaBGRMQy+hYZQyEvSfp7JIZ1EpeOr02DshcsMcKTiI NNDerNf4nEK94wB+cnWGK3i1Z3cOB1L8tPqd4drZ6QoQILrq6UEerWQGhLifnmqGGCNB enDw== X-Forwarded-Encrypted: i=1; AL+hErFgl7dvHIJOM3RGEQikSDK0BYAGFzd1E5Z37eGsrG2Jrgo3FcJEwix2lmdF4MaryLCQoztpVJdaFytdOag=@vger.kernel.org X-Gm-Message-State: APuU4Z/OFKorp25euEy70UNOzsCNx9fFSt8s4Yhj5HZYixBXVEjUppuJ OhK4wy7Y0iBhShvzMobLM53sluVJmTVAOYo/2bj/ycetM1Ze6FxfuttN X-Gm-Gg: AYBFou0zymeTCYqmmB9O7dT6B8k+fJ6EjVam7e7HEcG8WQXq23RHej5w/s3bOeqy4BB Nx+gDJoQqOVkjLfgKB7/2M7TvfRbaggAMoq3q+epDk2HLBFpdxH7Dxrc3eQiZl+uv2ic/6CUmo1 2oiv+WVHlOAuRgMkNVvmkP2Q1goA0ctmOxUPrBSbasHlKTeRRde5J6MD6fDqfyDPTEp3+rAbckg zPe0LclayOHZ7iyNTn7DtEqU5BScn05MEsVOQdokqJSiPxX+zjsyXvp5sofFXQkWcKEu8wyL9oS XGG9jeei4WBcsG6QhKkTB09H/6PeFivDlFe+HhuNd/CT+9Jn7gcHdf6S/tapghOoHa+fbieRIVs zuU38pdcwj3eXBDVrvfraQzg3iuOtFc2qUHSd0j/8yUH03MmjmNhwk7RvqprCuc4R/zdlHxOEL2 dLc2DZwg1RFC/V6eKCV3Br3A+Dgo4FXLGpr6tKp2PuzgO+zqWf5mQbWjJYmc4VMockg2KAauwY4 v/BLKfz6gsqsn+iVbDmZm1zrR+NfFVl7qqIn5ZQW5MturjMVSxtUrTgu+mZbLxuxSgrVthWbIwr n02A8QxD9q/IxlsELWkBIVW5Z5UOTWw+RFyvpnsGcQ== X-Received: by 2002:a05:7022:170c:b0:1ae:59d6:91cd with SMTP id a92af1059eb24-1b5568f522fmr1440476c88.38.1795148364438; Thu, 19 Nov 2026 20:19:24 -0800 (PST) Received: from penguin.lxd ([12.187.136.98]) by smtp.gmail.com with ESMTPSA id a92af1059eb24-1b5567ffa8csm3124341c88.7.2026.11.19.20.19.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Nov 2026 20:19:23 -0800 (PST) From: "Kanchana P. Sridhar" To: hannes@cmpxchg.org, yosry@kernel.org, nphamcs@gmail.com, chengming.zhou@linux.dev, akpm@linux-foundation.org, linux-mm@kvack.org, herbert@gondor.apana.org.au, davem@davemloft.net, linux-crypto@vger.kernel.org, kanchanapsridhar2026@gmail.com, linux-kernel@vger.kernel.org Cc: rientjes@google.com, willy@infradead.org, senozhatsky@chromium.org Subject: [PATCH v4 5/8] mm: zswap: Store the folio's node id in the zswap_entry. Date: Thu, 19 Nov 2026 20:19:03 -0800 Message-Id: <20261120041906.5785-6-kanchanapsridhar2026@gmail.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20261120041906.5785-1-kanchanapsridhar2026@gmail.com> References: <20261120041906.5785-1-kanchanapsridhar2026@gmail.com> X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Transfer-Encoding: 8bit When storing a folio in zswap, save the folio's node id in the zswap entry instead of relying on the zswap_entry being allocated on the same node. This enables optimizations such as batch allocation of zswap entries for large folios using kmem_cache_alloc_bulk() that does not support specifying a node id. Accordingly, replace the SLUB allocation in zswap_entry_cache_alloc() to be node-agnostic. The size of the zswap_entry remains unchanged as 'referenced' is lumped in with the 'length' (as it doesn't need a full unsigned int anyway). A build time assertion checks for the zswap_entry's 'length' bitfield being large enough to represent PAGE_SIZE, which can be 256KB on some architectures. Suggested-by: Nhat Pham Suggested-by: Yosry Ahmed Signed-off-by: Kanchana P Sridhar Signed-off-by: Kanchana P. Sridhar --- mm/zswap.c | 33 ++++++++++++++++++--------------- 1 file changed, 18 insertions(+), 15 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 9a323862bd59..78d9694f75b5 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -181,6 +181,7 @@ struct zswap_pool { */ #define ZSWAP_FIRST_POOL_ID 1 #define ZSWAP_MAX_POOL_ID U8_MAX +#define ZSWAP_MAX_COMPR_LENGTH_SHIFT 19U static DEFINE_XARRAY_FLAGS(zswap_pools, XA_FLAGS_ALLOC1 | XA_FLAGS_LOCK_BH); static struct zswap_pool __rcu *zswap_current_pool; @@ -207,15 +208,19 @@ static struct shrinker *zswap_shrinker; * logic if referenced is unset. See comments in the shrinker * section for context. * pool_idx - id of the zswap_pool that the entry's data is in. + * nid - NUMA node id of the page for which this is the zswap entry. * handle - zsmalloc allocation handle that stores the compressed page data * objcg - the obj_cgroup that the compressed memory is charged to * lru - handle to the pool's lru used to evict pages. */ struct zswap_entry { swp_entry_t swpentry; - unsigned int length; - bool referenced; - u8 pool_idx; + struct { + unsigned int length:ZSWAP_MAX_COMPR_LENGTH_SHIFT; + bool referenced:1U; + u8 pool_idx; + }; + int nid; struct zs_handle handle; struct obj_cgroup *objcg; struct list_head lru; @@ -680,14 +685,8 @@ static int zswap_enabled_param_set(const char *val, * lru functions **********************************/ -static inline int entry_to_nid(struct zswap_entry *entry) -{ - return page_to_nid(virt_to_page(entry)); -} - static void zswap_lru_add(struct zswap_entry *entry) { - int nid = entry_to_nid(entry); struct mem_cgroup *memcg; /* @@ -704,19 +703,18 @@ static void zswap_lru_add(struct zswap_entry *entry) rcu_read_lock(); memcg = obj_cgroup_memcg(entry->objcg); /* will always succeed */ - list_lru_add(&zswap_list_lru, &entry->lru, nid, memcg); + list_lru_add(&zswap_list_lru, &entry->lru, entry->nid, memcg); rcu_read_unlock(); } static void zswap_lru_del(struct zswap_entry *entry) { - int nid = entry_to_nid(entry); struct mem_cgroup *memcg; rcu_read_lock(); memcg = obj_cgroup_memcg(entry->objcg); /* will always succeed */ - list_lru_del(&zswap_list_lru, &entry->lru, nid, memcg); + list_lru_del(&zswap_list_lru, &entry->lru, entry->nid, memcg); rcu_read_unlock(); } @@ -764,10 +762,11 @@ void zswap_memcg_offline_cleanup(struct mem_cgroup *memcg) **********************************/ static struct kmem_cache *zswap_entry_cache; -static struct zswap_entry *zswap_entry_cache_alloc(gfp_t gfp, int nid) +static struct zswap_entry *zswap_entry_cache_alloc(gfp_t gfp) { struct zswap_entry *entry; - entry = kmem_cache_alloc_node(zswap_entry_cache, gfp, nid); + + entry = kmem_cache_alloc(zswap_entry_cache, gfp); if (!entry) return NULL; zs_handle_init(&entry->handle); @@ -1537,9 +1536,10 @@ static bool zswap_store_page(struct folio *folio, long index, { swp_entry_t page_swpentry = folio_swap_entry(folio, index); struct zswap_entry *entry, *old; + int nid = folio_nid(folio); /* allocate entry */ - entry = zswap_entry_cache_alloc(GFP_KERNEL, folio_nid(folio)); + entry = zswap_entry_cache_alloc(GFP_KERNEL); if (!entry) { zswap_reject_kmemcache_fail++; return false; @@ -1554,6 +1554,7 @@ static bool zswap_store_page(struct folio *folio, long index, * to an unrelated live pool. */ entry->pool_idx = pool->idx; + entry->nid = nid; old = xa_store(swap_zswap_tree(page_swpentry), swp_offset(page_swpentry), @@ -1930,6 +1931,8 @@ static int zswap_setup(void) struct zswap_pool *pool; int ret; + BUILD_BUG_ON(ZSWAP_MAX_COMPR_LENGTH_SHIFT <= PAGE_SHIFT); + zswap_entry_cache = KMEM_CACHE(zswap_entry, 0); if (!zswap_entry_cache) { pr_err("entry cache creation failed\n"); -- 2.39.5