From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4B992423A83 for ; Mon, 29 Jun 2026 14:42:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782744168; cv=none; b=mg6hnCVh6nD632uDwtHq+QKQwzmcM9YvehU+e7ojbODkSS/mcxkwjb4zR3V+eTcdwBA4dIHFmyfTGosOC2sH8MRK5zGuoNEuAkCm1BGnd+/WNRDlm7zKqxJOx4+vDHhISoaudhUnUYpbTs81sR8zmr9fVwdl2tmowON6Zaz/eic= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782744168; c=relaxed/simple; bh=2aD4CtYzBW99H9vdJ6c+MNnP2pR3wqJZG6xi41NJaPA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rzBKXew/2QkoxAhPMG+1U8FLvBuxSbejWVy7YPNtxrkiUoga+0/NcXHd98Dh9HywY7iFyY7StZqUbF8FZJJTNpmarXxTcQ0C3TQJcHq9kfDbgm2HNhAV72EfHJ4jVfH3N/WyJIDZyZblSBZI2vjOKqQStUBh8PAtpgl7ecHECfg= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=infradead.org; spf=pass smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=y/yBz0Rc; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="y/yBz0Rc" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=kWtISXVTdaT95P9Ai8aXDGZfdrCS/RaIS8ba+R7GxKE=; b=y/yBz0RcdINQl7RTVz7XFdvO89 7yyMIFSHWRWFFYGjqnZN3au9+qljknh6ry+52LTkfQFTmgFLzxsnTJUdjvVxBmwrQTR8NEInec1Cs 831OOzkHxH/Hg019AWZjzoBvtUchGha5DcnXrj81SfXoGjuaNO7ZTgJNleqQjuXeEiPrCrsyMZOoq 8nGUSg/s+RztOJ7SznMe47RunhIf3aL+JS4GokcokmKCWC4neh/pRXGeMjt0w1p/eB+8Rn7wPPdje lec0oVXoP2vbqsEReJ9VPbn9jQ02XevZTvSqoDIiLiTHq8UIbktaGq2/o6LGE/jKiSwrWQaQVo1OT 3q0hexHg==; Received: from [38.23.173.23] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.99.1 #2 (Red Hat Linux)) id 1weDCA-0000000Eub1-2snP; Mon, 29 Jun 2026 14:42:46 +0000 From: "Liam R. Howlett (Oracle)" To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, maple-tree@lists.infradead.org, "Liam R. Howlett (Oracle)" Subject: [PATCH 09/18] maple_tree: Optimise mas_wr_node_store() when not in rcu mode Date: Mon, 29 Jun 2026 10:41:36 -0400 Message-ID: <20260629144145.1572283-10-liam@infradead.org> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260629144145.1572283-1-liam@infradead.org> References: <20260629144145.1572283-1-liam@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Clearing the entire node on the stack is unnecessary since most of the node will be overwritten anyways. Just clear what isn't used after the data is in place. Benchmarking shows a speedup of 0.67% on a height 4 tree with 2048 entries. Signed-off-by: Liam R. Howlett (Oracle) --- lib/maple_tree.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/lib/maple_tree.c b/lib/maple_tree.c index fe10ff83e4937..a2b96ef05cb32 100644 --- a/lib/maple_tree.c +++ b/lib/maple_tree.c @@ -3183,7 +3183,7 @@ static void mas_wr_spanning_store(struct ma_wr_state *wr_mas) static inline void mas_wr_node_store(struct ma_wr_state *wr_mas) { unsigned char dst_offset, offset_end; - unsigned char copy_size, node_pivots; + unsigned char copy_size, node_pivots, node_slots; struct maple_node reuse, *newnode; unsigned long *dst_pivots; void __rcu **dst_slots; @@ -3196,6 +3196,7 @@ static inline void mas_wr_node_store(struct ma_wr_state *wr_mas) in_rcu = mt_in_rcu(mas->tree); offset_end = wr_mas->offset_end; node_pivots = mt_pivots[wr_mas->type]; + node_slots = mt_slots[wr_mas->type]; /* Assume last adds an entry */ new_end = mas->end + 1 - offset_end + mas->offset; if (mas->last == wr_mas->end_piv) { @@ -3207,7 +3208,6 @@ static inline void mas_wr_node_store(struct ma_wr_state *wr_mas) if (in_rcu) { newnode = mas_pop_node(mas); } else { - memset(&reuse, 0, sizeof(struct maple_node)); newnode = &reuse; } @@ -3251,7 +3251,17 @@ static inline void mas_wr_node_store(struct ma_wr_state *wr_mas) dst_pivots[new_end] = mas->max; done: - mas_leaf_set_meta(newnode, maple_leaf_64, new_end); + if (!in_rcu && new_end + 2 < node_slots) { + unsigned char clear_from = new_end + 1; + + memset(dst_slots + clear_from, 0, + sizeof(void __rcu *) * (node_slots - clear_from)); + if (clear_from < node_pivots) + memset(dst_pivots + clear_from, 0, + sizeof(unsigned long) * (node_pivots - clear_from)); + } + + mas_leaf_set_meta(newnode, wr_mas->type, new_end); if (in_rcu) { struct maple_enode *old_enode = mas->node; -- 2.47.3