From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ed1-f53.google.com (mail-ed1-f53.google.com [209.85.208.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B0F2341DDE4 for ; Mon, 17 Aug 2026 12:43:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.53 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786970592; cv=none; b=VxOWcWMwm+w3vAHGz/v56Jht06dekH/2NTSZDvlbX3icIl/cfMVbXMfxqLWfzyX4bqJbofyMxCYJJ3QnGIAS7JNrHATt4BcJKzjbcGxp8VXW+szdYiGhPZydgyOF7a0eWySjkg4Jd6neaM6ZmWJGbLRGkfuzs0im/XW3MdbliwM= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786970592; c=relaxed/simple; bh=OhqGYzOOkx2OUOWbEylHlwtk71aCWuaxYpNokA4l+LY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=MBOne7P34/0Maomajjy7WaT7D3dEtjXTwpHSZHlDLbabg8mBdARqoROhFBMiNvSnUY6KH6I6SeNEMrrWXdjzgpaAuUI/pFsf+gWgyCNEFL2x2OlmOFNQ/8L6jjFxX/+EmiC4CyOsbGujWmLmL5qNLodLRpU0mdH+EaMPR/O+ORc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=h0oSRTVg; arc=none smtp.client-ip=209.85.208.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="h0oSRTVg" Received: by mail-ed1-f53.google.com with SMTP id 4fb4d7f45d1cf-6a374bea882so3024650a12.1 for ; Mon, 17 Aug 2026 05:43:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1786970588; x=1787575388; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=MTB1+FCNUBbtk/wxJTJyqubMSGpfxVMNiUg/WWAjSjU=; b=h0oSRTVgf5CIw2bH/vE9mq6fkL73C/34HiMSbzNH/4cGLYiR0ucpTdYUlq5zL02MGT XBsQZXU8D4Xg9qevRWY+yaXqP/VU5sHgRHYKuC3SJcBMsrr1SgycXN6AWE0vK+1mVZwg VZsfhimCmzOWxNHQ/IvFLIxTeZqoF3JOldK+R6zda9UwdOYO4Z754c/DAid6J/YChz7D oBNP2+7bNzJG6H4bSQwRejtuxDwczw7U50G9xnDvgLINN7xN10dQQZsshtP9dN7johpH YWGNNcYXkrVm+gWnLnb1exdrEQGQeQMhGQB0ShN6+T2EZjU9THPYSqInQ7+2/gsyoDXG LMWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786970588; x=1787575388; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=MTB1+FCNUBbtk/wxJTJyqubMSGpfxVMNiUg/WWAjSjU=; b=ZVBNSp4aozFf3cpAOyIbg0346cosU9b0RS/X3opfBM98srNEj7ZIwUXQp4hwFQjwdG /zpqT3ZZs06p7Fm+/oAykOM0JmIC5dl+I0Nh+ENGTM3VEW8SHOBFgNro86SSpT419ROu 8nLIfFSVxpfzNAbVTbFwu0hgiZtAupEnQXePQqJqZAij0anzZ4OPZTL90GjjICTifynk d//1PW9lcDXmFzaQ3MzkHouAD8SKK2XFuCV+0HvbMjUvjvTTI9LXmfjBa1MSa1zegNJd AID53xOR2+1cfrbIHnhoY9WcxopkWNvPpg8gtPukWnwNwO8tjC5pfX3ZwRVgh/YL1pIC F9jg== X-Forwarded-Encrypted: i=1; AHgh+RpRJEMTSORu3be/dkPKuSM9IrfxarYmhFG+XSJ/Rpc+OsBtIDHpLvD84Ou5c3IcKH5jf/PSK3rFqwJBUoE=@vger.kernel.org X-Gm-Message-State: AOJu0YyVxT4AsPhX/pG8cxOWzKmrShjol9+eo7pn8WkdKng/9KrI7VPj QMwo5iid0ozjh9ho22byf145/3GAE57FzcmbEFvwt5mzqvacGbYI8t/2 X-Gm-Gg: AR+sD13i/j3xk8V0S498saU4E/TyTOQ60muX/0tZ6ybiMPr6xToZ3VZFE8hlRts+S5N 48OcJDBdv2byAbcq5xvH9kWEnPyf9x+M9Lr64g8ck+xNpL9ia/TYDIutlz8jHmEwuCFBJs286Qw /UFJvOVUDwl5cyQ8P0VQqPO0NXU0wKrSKNrmm24P/Gf8ZVaNdjHwXE1VZG9HX4oiT1KIW6M514C qowDq6E0vJF7hgsjsEezizqUkTRIeh9jznNccq2Nl0lMTSuoYA54c9mUJIWoVXQzesDqkRv952C SfOV5FYY3DWGeRhdQ+zyVj8599nyFJOxY8zBj82WogVOnW5LMzJso7gfPE1zruPNznUEZExqYHM CZojDS19TPgINPMYwLrFGV4fjI2/QBCY2k8erkVWe2+Saa9NWyFAvXNxi8d3n2GYytK8q5Cz92u bI0nAMZqtyTXLejyer/9isBo274DmMR3nmF3KzUqS/fN84yfDqDg== X-Received: by 2002:a17:907:c788:b0:c15:d0b6:495c with SMTP id a640c23a62f3a-c212a2452admr1247772466b.29.1786970587948; Mon, 17 Aug 2026 05:43:07 -0700 (PDT) Received: from [127.0.0.1] ([2a09:bac6:3861:1e5a::306:a]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-c217feb99d4sm52626266b.22.2026.08.17.05.43.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Aug 2026 05:43:07 -0700 (PDT) From: Caleb Kan Date: Mon, 17 Aug 2026 13:42:49 +0100 Subject: [PATCH RFC 9/9] stackdepot: add boot-time activation for trie storage Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20260817-stackdepot-trie-v1-9-53870ca1651b@cloudflare.com> References: <20260817-stackdepot-trie-v1-0-53870ca1651b@cloudflare.com> In-Reply-To: <20260817-stackdepot-trie-v1-0-53870ca1651b@cloudflare.com> To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, Vlastimil Babka , Alexander Potapenko , Marco Elver , Dmitry Vyukov , Andrey Konovalov , Oscar Salvador , Caleb Kan , kernel-team@cloudflare.com X-Mailer: b4 0.16.0 From: Caleb Kan Trie storage cannot be activated while persistent stack depot consumers can pass trie-backed handles to hash-only access paths. The preceding patches make those paths backend-independent, keep the corresponding saves explicitly hash-backed, or make the GDB helper reject trie-backed handles. The trie can now be activated without exposing incompatible handles. Add the boot-only stackdepot.trie_enabled parameter. Keep it disabled by default because lookup-only constrained misses can lose traces and the GDB helper does not decode trie handles. Guard backend selection with a static key so the existing hash-only path does not take a normal runtime branch when the parameter is absent. Document the parameter and its handle-namespace requirements. The available trie ID space depends on stack_depot_max_pools because hash and trie handles share the pool-index field. Configurations that consume the entire field cannot enable the backend. In particular, a 64 KiB page configuration using the default maximum of 8,191 pools must lower stack_depot_max_pools to leave trie ID space. For early stack depot initialization, allocate the trie side-table root, first directory, and first chunk through memblock. For later initialization, allocate the root with kvzalloc and grow directory and chunk pages lazily. Enable the static key only after initialization succeeds. Treat trie initialization as optional. If the handle namespace is empty or metadata allocation fails, warn, clear the request, and continue using the initialized hash backend at its configured capacity. Hash and trie storage continue to share stack_pools and the configured physical pool limit. Pools consumed by trie slots are therefore unavailable to refcounted and countable hash records. Once enabled, saves without STACK_DEPOT_FLAG_GET or STACK_DEPOT_FLAG_COUNTABLE use the trie. GET and COUNTABLE saves remain hash-backed; trie-eligible saves that cannot allocate perform a single lockless lookup, and per-save trie insertion failures do not fall back to hash storage. Signed-off-by: Caleb Kan --- Documentation/admin-guide/kernel-parameters.txt | 7 ++ lib/stackdepot.c | 100 ++++++++++++++++++++++-- 2 files changed, 101 insertions(+), 6 deletions(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 1af62cd16c9d..ebb7b7e1867f 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -7387,6 +7387,13 @@ Kernel parameters stack traces. Pools are allocated on-demand up to this limit. Default value is 8191 pools. + stackdepot.trie_enabled= [KNL] + Format: + Enable trie storage for persistent, non-refcounted + stack depot records at boot. Disabled by default. + stack_depot_max_pools must leave unused pool-index + values for trie handles. + stacktrace [FTRACE] Enable the stack tracer on boot up. diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 1e5b9fc44618..1a002063a948 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -28,6 +28,7 @@ #include #include #include +#include #include #include #include @@ -100,8 +101,8 @@ static const char *const counter_names[] = { [DEPOT_COUNTER_REFD_FREES] = "refcounted_frees", [DEPOT_COUNTER_REFD_INUSE] = "refcounted_in_use", [DEPOT_COUNTER_FREELIST_SIZE] = "freelist_size", - [DEPOT_COUNTER_PERSIST_COUNT] = "persistent_count", - [DEPOT_COUNTER_PERSIST_BYTES] = "persistent_bytes", + [DEPOT_COUNTER_PERSIST_COUNT] = "hash_persistent_count", + [DEPOT_COUNTER_PERSIST_BYTES] = "hash_persistent_bytes", }; static_assert(ARRAY_SIZE(counter_names) == DEPOT_COUNTER_COUNT); @@ -180,6 +181,10 @@ static_assert(STACK_DEPOT_TRIE_POOL_FIRST_SLOT < STACK_DEPOT_TRIE_POOL_SLOTS); static DEFINE_STATIC_KEY_FALSE(stack_depot_trie_enabled); static const struct stack_depot_trie_children __rcu *stack_depot_trie_root; static DEFINE_RAW_SPINLOCK(stack_depot_trie_writer_lock); +static bool stack_depot_trie_requested; + +module_param_named(trie_enabled, stack_depot_trie_requested, bool, 0); +MODULE_PARM_DESC(trie_enabled, "Enable stack depot trie storage at boot"); #define DEPOT_POOL_INDEX_MASK ((1U << DEPOT_POOL_INDEX_BITS) - 1) #define DEPOT_OFFSET_MASK ((1U << DEPOT_OFFSET_BITS) - 1) @@ -236,8 +241,9 @@ static u32 trie_stack_id(depot_stack_handle_t handle) /* * Trie handles encode a dense stack ID. The side table maps that ID to a node * pointer for lockless fetch and print paths, which can run from diagnostic - * contexts where taking a lock would be unsafe. Additional directories and - * chunks are published lazily as stack IDs grow. + * contexts where taking a lock would be unsafe. Initialization installs the + * root; early initialization also installs the first directory and chunk. + * Additional directories and chunks are published lazily as stack IDs grow. */ #define STACK_DEPOT_TRIE_SIDE_TABLE_CHUNK_SIZE \ (PAGE_SIZE / sizeof(struct stack_depot_trie_node *)) @@ -375,6 +381,75 @@ trie_side_table_prepare_stack_slot(struct stack_depot_trie_side_prealloc *preall return id; } +static inline unsigned int trie_side_table_root_size_for_max_id(u32 max_stack_id) +{ + unsigned int top_size; + + top_size = DIV_ROUND_UP(max_stack_id, + STACK_DEPOT_TRIE_SIDE_TABLE_CHUNK_SIZE); + return DIV_ROUND_UP(top_size, STACK_DEPOT_TRIE_SIDE_TABLE_DIR_SIZE); +} + +static int __init stack_depot_trie_init_memblock(void) +{ + struct stack_depot_trie_side_root *root_vec; + struct stack_depot_trie_side_dir *first_dir; + const struct stack_depot_trie_node __rcu **first_chunk; + size_t root_bytes; + u32 max_stack_id; + unsigned int root_size; + + max_stack_id = trie_max_stack_id(); + if (!max_stack_id) + return -EINVAL; + root_size = trie_side_table_root_size_for_max_id(max_stack_id); + root_bytes = struct_size_t(struct stack_depot_trie_side_root, dirs, + root_size); + + root_vec = memblock_alloc(root_bytes, __alignof__(*root_vec)); + if (!root_vec) + return -ENOMEM; + first_dir = memblock_alloc(PAGE_SIZE, PAGE_SIZE); + if (!first_dir) { + memblock_free(root_vec, root_bytes); + return -ENOMEM; + } + first_chunk = memblock_alloc(PAGE_SIZE, PAGE_SIZE); + if (!first_chunk) { + memblock_free(first_dir, PAGE_SIZE); + memblock_free(root_vec, root_bytes); + return -ENOMEM; + } + + root_vec->dir_capacity = root_size; + RCU_INIT_POINTER(root_vec->dirs[0], first_dir); + RCU_INIT_POINTER(first_dir->chunks[0], first_chunk); + trie_side_table_root = root_vec; + static_branch_enable(&stack_depot_trie_enabled); + return 0; +} + +static int stack_depot_trie_init(void) +{ + struct stack_depot_trie_side_root *root_vec; + unsigned int root_size; + u32 max_stack_id; + + max_stack_id = trie_max_stack_id(); + if (!max_stack_id) + return -EINVAL; + + root_size = trie_side_table_root_size_for_max_id(max_stack_id); + root_vec = kvzalloc_flex(*root_vec, dirs, root_size); + if (!root_vec) + return -ENOMEM; + + root_vec->dir_capacity = root_size; + trie_side_table_root = root_vec; + static_branch_enable(&stack_depot_trie_enabled); + return 0; +} + static int trie_side_table_get_prealloc(gfp_t gfp_flags, struct stack_depot_trie_side_prealloc *prealloc) { @@ -702,7 +777,7 @@ static void init_stack_table(unsigned long entries) INIT_LIST_HEAD(&stack_table[i]); } -/* Allocates a hash table via memblock. Can only be used during early boot. */ +/* Initializes hash and optional trie storage during early boot. */ int __init stack_depot_early_init(void) { unsigned long entries = 0; @@ -776,11 +851,15 @@ int __init stack_depot_early_init(void) stack_depot_disabled = true; return -ENOMEM; } + if (stack_depot_trie_requested && stack_depot_trie_init_memblock()) { + pr_warn("trie storage initialization failed, disabling trie storage\n"); + stack_depot_trie_requested = false; + } return 0; } -/* Allocates a hash table via kvcalloc. Can be used after boot. */ +/* Initializes hash and optional trie storage after boot. */ int stack_depot_init(void) { static DEFINE_MUTEX(stack_depot_init_mutex); @@ -834,6 +913,15 @@ int stack_depot_init(void) kvfree(stack_table); stack_depot_disabled = true; ret = -ENOMEM; + goto out_unlock; + } + if (stack_depot_trie_requested) { + ret = stack_depot_trie_init(); + if (ret) { + pr_warn("trie storage initialization failed, disabling trie storage\n"); + stack_depot_trie_requested = false; + ret = 0; + } } out_unlock: -- Git-155)