From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F015C3E5562 for ; Tue, 30 Jun 2026 19:09:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782846544; cv=none; b=QkycFnRqQhpmKNj4Lg5K2KOEbwuDMkAVmfJyA5uWl0nQveMhHImwG2VcOrQ7QjEl1NlTsul3Mqkd1KFL/Hesu/umLLG1sAk40lgFkwMGXHgaqe2VlE1Mnz62kGITTYnP34bHrJRQ0mbKLV1d6ijrAM4f4hYDglFmSeac7G5Fsjg= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782846544; c=relaxed/simple; bh=m06AMpQLSBXs4YpFXhVB0RkBMaF9CUxWDxd7NWm3ZbM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=UVR729V9VZbY3Y5s5QANitTNvI5CQJ/kEGQ/j0zjadm2vP2hRp+aW2Kp09a2kluq+G9vg9Z/i8gNnJUiEwSJP4BIjatTiCQLLTHU5nrP0odUjm52MTLKX07oJqw6ytoQzLLFUmNirSQxfjpUM6sKzgQmMaBYOl7uWOFnZ32v3v4= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=infradead.org; spf=pass smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=DN1i6EsS; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="DN1i6EsS" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=UXutp8Kq6mfQ34jQx0qtQAgm1oKAPTTpjQhpDOqSy/s=; b=DN1i6EsSiLU0IuSKqEjc9GtJ0Z C5NN283rS3lWfB1CSBAnz6GI2rbQOJmKT3Hu4uv2/WstRfaFxw16iWa1qgjJaJdme/XJ5uyuePXK9 nGTAUBDMY7w1acOGgKx5+qbZT/qJg9ERhS6EEygCEafuw5AvEEdX3AqmMbJHGp8QJAg5fPzIWrKHd 4hrvlVOtgSk8tnid/QSE86//rvFsgNlFaZgKr8v0kn3EX5QmbL9ZdO0MA9KcN/dn9AkA8zFFq/cF7 LaMXcfSozTiOZfVIm4jiJN6ar65EU7YrF+q7nTX9RQIDCDcg9y0SMdi2CpXpLWtQUYiCoIRNONjgR zMH7bSLQ==; Received: from [38.23.173.23] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.99.1 #2 (Red Hat Linux)) id 1wedpO-000000005us-1iZk; Tue, 30 Jun 2026 19:09:02 +0000 From: "Liam R. Howlett (Oracle)" To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, maple-tree@lists.infradead.org, "Liam R. Howlett (Oracle)" Subject: [PATCH v2 09/19] maple_tree: Optimise mas_wr_node_store() when not in rcu mode Date: Tue, 30 Jun 2026 15:08:33 -0400 Message-ID: <20260630190843.3563858-10-liam@infradead.org> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260630190843.3563858-1-liam@infradead.org> References: <20260630190843.3563858-1-liam@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Clearing the entire node on the stack is unnecessary since most of the node will be overwritten anyways. Just clear what isn't used after the data is in place. Benchmarking shows a speedup of 0.67% on a height 4 tree with 2048 entries. Signed-off-by: Liam R. Howlett (Oracle) --- lib/maple_tree.c | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) diff --git a/lib/maple_tree.c b/lib/maple_tree.c index 8f5a7fa10cf4c..1c96b3a5dab60 100644 --- a/lib/maple_tree.c +++ b/lib/maple_tree.c @@ -3186,7 +3186,7 @@ static void mas_wr_spanning_store(struct ma_wr_state *wr_mas) static inline void mas_wr_node_store(struct ma_wr_state *wr_mas) { unsigned char dst_offset, offset_end; - unsigned char copy_size, node_pivots; + unsigned char copy_size, node_pivots, node_slots; struct maple_node reuse, *newnode; unsigned long *dst_pivots; void __rcu **dst_slots; @@ -3199,6 +3199,7 @@ static inline void mas_wr_node_store(struct ma_wr_state *wr_mas) in_rcu = mt_in_rcu(mas->tree); offset_end = wr_mas->offset_end; node_pivots = mt_pivots[wr_mas->type]; + node_slots = mt_slots[wr_mas->type]; /* Assume last adds an entry */ new_end = mas->end + 1 - offset_end + mas->offset; if (mas->last == wr_mas->end_piv) { @@ -3210,7 +3211,6 @@ static inline void mas_wr_node_store(struct ma_wr_state *wr_mas) if (in_rcu) { newnode = mas_pop_node(mas); } else { - memset(&reuse, 0, sizeof(struct maple_node)); newnode = &reuse; } @@ -3254,7 +3254,21 @@ static inline void mas_wr_node_store(struct ma_wr_state *wr_mas) dst_pivots[new_end] = mas->max; done: - mas_leaf_set_meta(newnode, maple_leaf_64, new_end); + if (!in_rcu && new_end + 2 < node_slots) { + unsigned char clear_from = new_end + 1; + + /* + * Note that the last slot is never cleared, since the metadata + * will be stored there or it has a value. + */ + memset(dst_slots + clear_from, 0, + sizeof(void __rcu *) * (node_slots - clear_from)); + if (clear_from < node_pivots) + memset(dst_pivots + clear_from, 0, + sizeof(unsigned long) * (node_pivots - clear_from)); + } + + mas_leaf_set_meta(newnode, wr_mas->type, new_end); if (in_rcu) { struct maple_enode *old_enode = mas->node; -- 2.47.3