From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2651E395DBE for ; Mon, 13 Jul 2026 07:52:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783929160; cv=none; b=cjNZ9Vw8NpkxiKCdt3EiCurGcNdc0/U/fsoO3cEjPALorXr1MnsLsu98SZXyh4g6duLeZ+KutTNNw8GmYMUbnhCqyFY8F3NaUdYRMmYYV1dV3iOOXatM2EcBcfFxQPgQjI7hIlr8E/EUKb+m6dtpcjV4cMB/12yQHnFFDe62keU= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783929160; c=relaxed/simple; bh=7HIrXJ5nEgp0SEwg3bJKrvyss7mGFs/5pwhhD0866b0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:To:Cc; b=gQKIqhrv6/I4UhJsAH4RsVyNV0GLST6jtT0z0I2xMF5wp5QGuMuP/nkiGGAbQxB34PDZKWf/5qKZ0/OcxufZkc7+OLktd1ctoeoTUdiNqbveE+dxwIm8F+RmzA4NpjdZFhfoErMBHk2v2BnRnkwnQ0IL0DYGIw6ysNAfZr/haxU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=KjoePid7; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="KjoePid7" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 815E61F000E9; Mon, 13 Jul 2026 07:52:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1783929158; bh=JG3wfdlgkpMM8tt+kWWFpTDhSu/cA7QAayvWlQD2jTA=; h=From:Date:Subject:To:Cc; b=KjoePid7SP2n4JqJFiVvkr4H36woTeOhh8kIVatO6/xv9frgAc82eUYEkKRqi1nDX ccMp81BMtkGx3pGKkCVBPHd1a2ZsKhdckbWxNzbn2mOR94fJhuLTpYjB1A6OXOYH0y FurC9mjUd+VPWSzHPe8Tgl8E65Q3m2zxlO2f1oanVOCRL7TOM/t1BKe7Qk0jPaXRpB HORtQSUDGJRqmDdHdGt42GOKeqoMdXj0zQDx0dbSkCv2goR5uaM02RRBGnicEjIkiQ IF/1UFFA9eSnBTCM3lAYJWYvsOIjfPijxzrF7/mjtQIJ+IBMvowynidGLhg+eHUJOp Mkinh9rrbszXQ== From: "Vlastimil Babka (SUSE)" Date: Mon, 13 Jul 2026 09:52:30 +0200 Subject: [PATCH slab/for-next v2] mm/slab: simplify freeing remote objects in free_to_pcs_bulk() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20260713-bulk_free_remote-v2-1-24ee24771c2f@kernel.org> X-B4-Tracking: v=1; b=H4sIAD2ZVGoC/3WOyw6CMBREf4XctVVawyOu/A9DSEtvoYKtuS0EQ /h3AdcuJ5NzZhYISBYD3JIFCCcbrHdbEKcEmk66FpnVWwaRijwteMrUOPS1IcSa8OUjssIozM1 VKpNp2LA3obHzoXxAGKS6GE/M4Ryh+tVhVE9s4q7dgc6G6OlzXJj4gf1fmzjjTJZFqXPdZEJn9 x7J4XD21EK1rusXQm/HgdIAAAA= X-Change-ID: 20260710-bulk_free_remote-7fbe6f3abf5d To: Harry Yoo , Andrew Morton Cc: Pedro Falcato , Hao Li , Shengming Hu , Christoph Lameter , David Rientjes , Roman Gushchin , linux-mm@kvack.org, linux-kernel@vger.kernel.org, "Vlastimil Babka (SUSE)" X-Mailer: b4 0.15.2 We have been moving remote objects to an on-stack array and flushing it when full. Instead, we can swap them towards the beginning of the supplied array and bulk-free it just once. Also add a comment to explain the rationale of freeing remote objects last, because now it would appear to be simpler to free them first. Reviewed-by: Pedro Falcato Reviewed-by: Shengming Hu Reviewed-by: Harry Yoo (Oracle) Reviewed-by: Hao Li Signed-off-by: Vlastimil Babka (SUSE) --- Here's v2 with updated comment, will add to slab/for-next right away. Thanks all! One more cleanup I realized it's possible when processing feedback to the previous one [1] [1] https://lore.kernel.org/all/20260707-slab-simplify-bulk-pcs-v1-1-4850dbe0d904@kernel.org/ --- Changes in v2: - Add comment about the partitioning, based on Shengming's suggestion. - Add R-b tags, thanks! - Link to v1: https://patch.msgid.link/20260710-bulk_free_remote-v1-1-a878d6dc52d5@kernel.org --- mm/slub.c | 30 ++++++++++++++++++++---------- 1 file changed, 20 insertions(+), 10 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index 22045dc919ef..b650c1729437 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -6255,9 +6255,17 @@ static unsigned int __free_to_pcs_batch(struct kmem_cache *s, size_t size, void static void free_to_pcs_bulk(struct kmem_cache *s, size_t size, void **p) { bool init = slab_want_init_on_free(s); - void *remote_objects[PCS_BATCH_MAX]; + void **remote_objects = p; unsigned int remote_nr = 0; + /* + * Process the free hooks and separate out remote objects by + * partitioning the 'p' array in place: + * + * [0, remote_nr) - processed remote objects + * [remote_nr, i) - processed local objects + * [i, size) - unprocessed objects + */ for (unsigned int i = 0; i < size;) { struct slab *slab = virt_to_slab(p[i]); @@ -6270,19 +6278,17 @@ static void free_to_pcs_bulk(struct kmem_cache *s, size_t size, void **p) } if (unlikely(!can_free_to_pcs(slab))) { - remote_objects[remote_nr] = p[i]; - p[i] = p[--size]; - if (++remote_nr >= PCS_BATCH_MAX) { - __kmem_cache_free_bulk(s, remote_nr, &remote_objects[0]); - stat_add(s, FREE_SLOWPATH, remote_nr); - remote_nr = 0; - } - continue; + if (i != remote_nr) + swap(remote_objects[remote_nr], p[i]); + remote_nr++; } i++; } + p += remote_nr; + size -= remote_nr; + while (size) { unsigned int batch_freed = __free_to_pcs_batch(s, size, p); @@ -6296,8 +6302,12 @@ static void free_to_pcs_bulk(struct kmem_cache *s, size_t size, void **p) size -= batch_freed; } + /* + * Processing remote objects last decreases the chances of cpu migration + * while freeing to sheaves and compromising object locality + */ if (remote_nr) { - __kmem_cache_free_bulk(s, remote_nr, &remote_objects[0]); + __kmem_cache_free_bulk(s, remote_nr, remote_objects); stat_add(s, FREE_SLOWPATH, remote_nr); } } --- base-commit: e1fa26489025d6deac76d1dbfe2e0720a3ad84b1 change-id: 20260710-bulk_free_remote-7fbe6f3abf5d