From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 28CB43A6B8D for ; Tue, 26 May 2026 13:41:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779802920; cv=none; b=HidTwkMuQIISmizUlwljoUxEIoLFF2fLbJCeT/eNYbTJP8vH+ItqCM8n/yWymp+a/uHPRcsPCsD4q6PN0ZCMKh/CjFVmQkkI5WHtZo7b4pj/0z6Bnw7lMlkgrYu2IG/faNbNtkQ/ZkNl/vjbEwutQx+JEZrifFZ809k7uJGVY+k= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779802920; c=relaxed/simple; bh=Zkt+vSo8dWmpgDQnd8iC1iTeyQ2yzUR5cktOuVfojp0=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=D+EKYdrJQ6MK1ZkgYY9M1CWBpesdnlRBnWG4p4SgoAko4XhVQqK73UMrTT+cy7ryYOPj3HcbvzgKmjZkhDKcXZf4smrmutm3vbTx+i+lQXBxho9OEsaRtqG0+YgIio4B3xx1FO1JW3aJD+ySLTqYRnxExumEJpYOTuK69U49HUw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=jB9JBhYm; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="jB9JBhYm" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 020EF1F000E9; Tue, 26 May 2026 13:41:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1779802918; bh=whrQ5TAuNm8jjG/nB8vNOlNyW3idicERerbUe4OrMjs=; h=Date:Subject:To:Cc:References:From:In-Reply-To; b=jB9JBhYmnH3a0lFXTpYA6VJQfdh32Rhf5rZbVuN5o8J7dTSGI9q5ZpmfUy2xERyVE XTAsFXH4vDYtZNlpmJO3dIk1EiNqF0W3Z9zmhx1JEor2OO/B8SwJ+WqfIIfrZwl2Zx ErzWCOB+B52x9BmlV1UO8JmgP4qLDtXJzi3LE7a0aCus+5JaDeKJBf/v2cVTYIT86h arB0UIvdmRsd1xR04Y36tMVDwwYJ01DqnuyYzlvMFvFDg6KGcMiAck9rGrbNAoN55O sSuFCB2lz8aRncVfSYL2hQc8PMmqwUd0R9p0Wg8NQSsc2HmZyrLBPA60wU+KI6OnIT NNSbIyXSNHWTQ== Message-ID: <3bfbb8cd-6291-43e8-9f0e-b8b0b3c568d1@kernel.org> Date: Tue, 26 May 2026 15:41:55 +0200 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] mm/slub: batch-detach node partial slabs To: Harry Yoo , Hao Li , akpm@linux-foundation.org Cc: cl@gentwo.org, rientjes@google.com, roman.gushchin@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <20260525032233.10847-1-hao.li@linux.dev> From: "Vlastimil Babka (SUSE)" Content-Language: en-US In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit On 5/26/26 9:37 AM, Harry Yoo wrote: > > > On 5/25/26 12:22 PM, Hao Li wrote: >> get_partial_node_bulk() used to move each selected slab from the node >> partial list to the local pc->slabs list using a remove_partial() and >> list_add() pair. In practice, the loop often detaches several adjacent >> slabs, so this repeatedly manipulates list pointers while holding >> n->list_lock, which causes unnecessary churn. >> >> Instead, track contiguous runs of matching slabs and move each run with >> list_bulk_move_tail() in one operation. > > TIL list_bulk_move_tail() :D > >> This reduces list pointer churn> inside the lock critical section. Nice! > Similar to this, can we return all slabs in pc->slabs at once when > returning those slabs to the list? ... I see Vlastimil removed 'nr of > empty slabs' check in the other series already. > > Now that it inserts slabs to the tail with Vlastimil's patchset, let's > do a list_splice_tail() instead? Why not, although it should be rare to return more than one slab in that path. So in case this change gets tricky for some reason, we can leave it. BTW, if you also get the same idea as I had and try to replace the current "remove from pc.slabs initially, reinsert to pc.slabs if it was a partial refill" with a "keep in pc.slabs and only remove if the refill was full" to reduce pointer churn, don't try that, it crashes rather quickly :D >> The mmap2 testcase shows a 5% improvement after applying this patch. >> >> Signed-off-by: Hao Li >> --- >>   mm/slub.c | 22 ++++++++++++++++++---- >>   1 file changed, 18 insertions(+), 4 deletions(-) >> >> diff --git a/mm/slub.c b/mm/slub.c >> index 04692a6f9128..180973a4a3d2 100644 >> --- a/mm/slub.c >> +++ b/mm/slub.c >> @@ -3775,15 +3783,21 @@ static bool get_partial_node_bulk(struct >> kmem_cache *s, >>               && total_free + slab_free > pc->max_objects) >>               break; >>   -        remove_partial(n, slab); >> - >> -        list_add(&slab->slab_list, &pc->slabs); >> +        if (!first) >> +            first = slab; >> +        last = slab; > >> +        slab_clear_node_partial(slab); >> +        n->nr_partial--; > > Perhaps factor out those two statements into to a common function and > call it in get_partial_node_bulk() and remove_partial()? >>           total_free += slab_free; >>           if (total_free >= pc->max_objects) >>               break; >>       } >>   +    if (first) >> +        list_bulk_move_tail(&pc->slabs, &first->slab_list, >> +                    &last->slab_list); >> + >>       spin_unlock_irqrestore(&n->list_lock, flags); >>       return total_free > 0; >>   } >