From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-dy1-f170.google.com (mail-dy1-f170.google.com [74.125.82.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6F991282F17 for ; Tue, 9 Jun 2026 13:20:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.82.170 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781011210; cv=none; b=R45YQEr8uXQaBzNY8yR/7JUk6MEwYw795VOXJz0zZ7RKaibW67CL9xK/JVdw0oLztob7QyCHBaSWYtjLseyCQ2caYwF4Icjkji+flkGQvzhe2/VnIX9CfZ/0m3ExdL52XEgZ3c3W+obqoXe7b33bn7c7kZGFB/JaSTplQSjMZ0Y= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781011210; c=relaxed/simple; bh=XlIxkLv1z3DX0vJpcDqsDcRiYRXlXP8MqjWgR2enkww=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ac6AgWPIzpGSn5UzN+dzO0C8doKHfdLrv9wwBNEnlUeiy7aVI0Q3XaAEles4JA6cN8e6iBRfyMma3aHQvJ2vapFQsLrJTMZrnVNDYIo6wKu4ZJpBHA+XTfEU47SwIgTKC96r3rpW9C6JEY79EXIsqgWq20b/T8szNoO3RsnL5yI= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=YE1CidtS; arc=none smtp.client-ip=74.125.82.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="YE1CidtS" Received: by mail-dy1-f170.google.com with SMTP id 5a478bee46e88-307d0405e07so993847eec.1 for ; Tue, 09 Jun 2026 06:20:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1781011208; x=1781616008; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rzZH85xAJ3bnOHXxZ2IlyON4zc70knGPmBexsKaQByk=; b=YE1CidtS7ICtMNHUm437muHNsHOCERbkrnT2X+M+Q8MkNP1bv6U7J5E6gtMuc4ouu3 VZCwOr2cXV9VUAej8DAOy56j32Sh48z4z2S8EVVLQR7WUcTTDN6mtE+BaNyvh9wb0XLT w1t/Bzmdzmnya/vv1okO2OSyJ+CTl2RfIv4s2DeDiu09HhaIVsTWgz1qVeb1KQdIExZ/ 8/+C7m2KF6snlJ53XTWAbEmSbBKFyA5VJYyHnLzFPS3UAjixV9+yOKhKNanLwBC2Y1ce SFOyEOcgwmufL5dnl3CI/uzmkCkX/LSpvPCPT26UeXiL8IWa/BFE/jCYXINyNPVBynzA 2vig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781011208; x=1781616008; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=rzZH85xAJ3bnOHXxZ2IlyON4zc70knGPmBexsKaQByk=; b=gD9yiUOgnhUI9KKKiiVfo4DYduqwemeLchQrz2YrjBsmuFenO4oVE/WpvHIu/50BiY /OFzKMLeUQUM7pcsPrum9hfIYH+r7oSWzmddqxCChPc6wT6nQ/I6ak7J2YpwkPkyjCO1 NXZhHxshQ/XLuIRWg2qRkqEWHaC2V9XoNY8/AY64+I6Kp3t5Ow83bSEkoJ11g1FB7Jil vAnt8Nu84cn/D5UaR+NNPnF5WTTmuZILaAIFRr+xsRdC3TLtB8WwfboLDlcy6FSa1kgF 8aZpHSV/IYsdsOKV6b8pvMYpvTlqkQ3Zz7N2FgQKOhcUsYyLy6g5/jk56yBkW+GN+V8a MLMA== X-Forwarded-Encrypted: i=1; AFNElJ8QZS5K6OwlFJdiwnjH2vPhMDgKH4AVAUxLEeAqBPJxH22Re4PDhCYmExZGGjqYtEKD08X11Taqi1A7qOE=@vger.kernel.org X-Gm-Message-State: AOJu0YwGRhkg6xh1pvy/RWumBVxbnJ7PsDA6QnNLcPI3Zn/7pZmpLr4w 66kYI53Ic5nOkUmnlxdAXgFw1SubHd17xZdU2SqCAufwt1mLkqb8ktpt X-Gm-Gg: Acq92OEy0ke92Emy+9hJXDFAKzb07LwBPfH83jyGmeFpXSmmyiqj+pUNJMtR4NLLnWj Qm1VF9b5uyda6V0X2sYEGjD6uGVFY/pUreMwbwfYTs0cDOBuiwp5yIo56ppAMikFo75Ee9OEWrx DJeVkco0GDxLO6Y7wkx+faY+CTS3eCUKp5pKhMlhBVGsb2Mqwk7ae6Y7NrASFm5oo13ySHXs+Hx LTgxBjcmqpp8MxwOmBtCShuVslyh/3pj15KXap4RPD+SC8GOmX81FdvEco5RLjpky+vtBx5Xejr YFl7hMG42/05Vd1/5H465Ftjw1csVRLXODqurOGXKgDn2sLVOi3TBG+TlxhguV8UfNQRaKGGeil fx16kFiyfDDLzULPZ+C+d3PC9Ap2oXXLmEmGUGnvzCyeHJPnVQ7m63pwHrzKCrtet23fFIHUsCp yeSYU63OGW7of4zWvz9RENFFymX7eB4XrGaJ8HEpPMm/JAbW5dnSVpXcNBXTmk X-Received: by 2002:a05:693c:3945:b0:2ea:c085:44b1 with SMTP id 5a478bee46e88-3077b22a000mr11912732eec.19.1781011208262; Tue, 09 Jun 2026 06:20:08 -0700 (PDT) Received: from localhost.localdomain ([129.41.58.2]) by smtp.gmail.com with ESMTPSA id 5a478bee46e88-3074db55f60sm27162467eec.6.2026.06.09.06.20.02 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 09 Jun 2026 06:20:07 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linux-mm@kvack.org Cc: Madhavan Srinivasan , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , Barry Song , Youngjun Park , David Hildenbrand , linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, Sayali Patil , "Ritesh Harjani (IBM)" Subject: [RFC 2/4] mm, swap: make SWAPFILE_CLUSTER runtime Date: Tue, 9 Jun 2026 18:49:32 +0530 Message-ID: X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit This makes SWAPFILE_CLUSTER as a runtime value. Architectures like powerpc book3s64 has HPAGE_PMD_NR, which is derived at runtime depending upon which chosen mmu. Hence this patch initializes SWAPFILE_CLUSTER at runtime and also modifies swap_table and swap_memcg_table which were earlier using this macro for defining the number of table entries. Signed-off-by: Ritesh Harjani (IBM) --- mm/swap.h | 5 +++-- mm/swap_table.h | 6 ++---- mm/swapfile.c | 27 ++++++++++++++++++++++----- 3 files changed, 27 insertions(+), 11 deletions(-) diff --git a/mm/swap.h b/mm/swap.h index 77d2d14eda42..956879a69ddd 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -26,11 +26,12 @@ extern int page_cluster; #define SWAP_TABLE_HAS_ZEROFLAG ((BITS_PER_LONG - SWAP_CACHE_PFN_MARK_BITS - \ SWAP_CACHE_PFN_BITS) > SWAP_COUNT_MIN_BITS) +extern unsigned int swap_slots_in_cluster __read_mostly; +#define SWAPFILE_CLUSTER swap_slots_in_cluster + #ifdef CONFIG_THP_SWAP -#define SWAPFILE_CLUSTER HPAGE_PMD_NR #define swap_entry_order(order) (order) #else -#define SWAPFILE_CLUSTER 256 #define swap_entry_order(order) 0 #endif diff --git a/mm/swap_table.h b/mm/swap_table.h index e6613e62f8d0..90e2a7852300 100644 --- a/mm/swap_table.h +++ b/mm/swap_table.h @@ -8,16 +8,14 @@ /* A typical flat array in each cluster as swap table */ struct swap_table { - atomic_long_t entries[SWAPFILE_CLUSTER]; + DECLARE_FLEX_ARRAY(atomic_long_t, entries); }; /* For storing memcg private id */ struct swap_memcg_table { - unsigned short id[SWAPFILE_CLUSTER]; + DECLARE_FLEX_ARRAY(unsigned short, id); }; -#define SWP_TABLE_USE_PAGE (sizeof(struct swap_table) == PAGE_SIZE) - /* * A swap table entry represents the status of a swap slot on a swap * (physical or virtual) device. The swap table in each cluster is a diff --git a/mm/swapfile.c b/mm/swapfile.c index 78b49b0658ad..016a5aa0cb93 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -129,6 +129,17 @@ static DEFINE_PER_CPU(struct percpu_swap_cluster, percpu_swap_cluster) = { .lock = INIT_LOCAL_LOCK(), }; +unsigned int swap_slots_in_cluster __read_mostly; +bool swap_table_use_page __read_mostly; + +static unsigned int generic_swap_slots_in_clusters(void) +{ + if (IS_ENABLED(CONFIG_THP_SWAP)) + return HPAGE_PMD_NR; + else + return 256; +} + /* May return NULL on invalid type, caller must check for NULL return */ static struct swap_info_struct *swap_type_to_info(int type) { @@ -437,7 +448,7 @@ static void swap_cluster_free_table(struct swap_cluster_info *ci) return; rcu_assign_pointer(ci->table, NULL); - if (!SWP_TABLE_USE_PAGE) { + if (!swap_table_use_page) { kmem_cache_free(swap_table_cachep, table); return; } @@ -456,7 +467,7 @@ static int swap_cluster_alloc_table(struct swap_cluster_info *ci, gfp_t gfp) if (rcu_access_pointer(ci->table)) return 0; - if (SWP_TABLE_USE_PAGE) { + if (swap_table_use_page) { folio = folio_alloc(gfp | __GFP_ZERO, 0); if (folio) table = folio_address(folio); @@ -471,7 +482,8 @@ static int swap_cluster_alloc_table(struct swap_cluster_info *ci, gfp_t gfp) #ifdef CONFIG_MEMCG if (!mem_cgroup_disabled()) { VM_WARN_ON_ONCE(ci->memcg_table); - ci->memcg_table = kzalloc_obj(*ci->memcg_table, gfp); + ci->memcg_table = kzalloc_flex(*ci->memcg_table, id, + SWAPFILE_CLUSTER, gfp); if (!ci->memcg_table) { swap_cluster_free_table(ci); return -ENOMEM; @@ -3912,14 +3924,19 @@ static int __init swapfile_init(void) { swapfile_maximum_size = arch_max_swapfile_size(); + swap_slots_in_cluster = generic_swap_slots_in_clusters(); + swap_table_use_page = + (swap_slots_in_cluster * sizeof(atomic_long_t) == PAGE_SIZE); + /* * Once a cluster is freed, it's swap table content is read * only, and all swap cache readers (swap_cache_*) verifies * the content before use. So it's safe to use RCU slab here. */ - if (!SWP_TABLE_USE_PAGE) + if (!swap_table_use_page) swap_table_cachep = kmem_cache_create("swap_table", - sizeof(struct swap_table), + struct_size_t(struct swap_table, entries, + SWAPFILE_CLUSTER), 0, SLAB_PANIC | SLAB_TYPESAFE_BY_RCU, NULL); #ifdef CONFIG_MIGRATION -- 2.39.5