Re: [RFC PATCH 2/2] mm/slub: introduce slab parking to reduce list_lock contention

["Vlastimil Babka (SUSE)" <vbabka@kernel.org>]

On 8/24/26 14:25, Hao Li wrote:
> Introduce a mechanism called parking to mitigate lock contention in the
> free slowpath.

Interesting!

> In the free slowpath, when __slab_free() transitions a full slab into a
> partial/empty slab through a free operation, it must acquire the list
> lock to add these newly freed partial/empty slabs to the partial list.
> 
> Why must partial and empty slabs converted from full slabs be added to
> the partial list? Because only by doing so can the sheaf refill or alloc
> slowpath see these partial slabs and allocate from them. Therefore, the
> list insertion must be performed, which requires acquiring the lock and
> leads to heavy lock contention under high concurrency.
> 
> Analysis of profiling data from the will-it-scale mmap1 benchmark shows
> that full -> partial transitions account for a large proportion, second
> only to partial -> partial.
> 
> With extra instrumentation added to __slab_free(), the following data
> was collected for the maple_node cache (in counts):
> 
>   partial->partial    843017414
>   full->partial       550719384
>   partial->empty      17459564
>   full->empty         2

That's a lot inded. I'd be careful if it's some specific aspect of the test,
i.e. lots of parallel allocations followed by lots of frees, that wouldn't
be that common in realistic workloads. But worth looking into at least.
If it's this kind of pathologic behavior, then I expect changing sheaf size
as suggested by Pedro wouldn't help much.

> Since the fundamental purpose of __slab_free() is to make newly freed
> partial/empty slabs visible to the sheaf refill or alloc slowpath, these
> slabs can be temporarily stored in a staging area in a lockless manner
> instead of making the free slowpath contend for the lock. The sheaf

However the lockless manipulation is still going to contend on the
llist_head. But perhaps it's limited enough in both users and lenght of
operations to make a difference.

> refill or alloc slowpath then checks this staging area first when
> allocating objects. This achieves the goal of making these slabs visible
> to the sheaf refill or alloc slowpath while allowing the free slowpath
> to operate locklessly. This process is called "parking".

I'm gonna pull a David Hildenbrand trick here and question the name :)
It seems to me it's an (extension of the) partial list, but lockless.
Parking would suggest to me that it's put somewhere aside not to be used, or
something.

> Parking occurs in only one case: when __slab_free() encounters a full ->
> partial/empty transition and the trylock fails. In this case,
> __slab_free() attaches the slab to an llist locklessly, instead of
> waiting for the lock unnecessarily.

It could be interesting to also see if skipping the trylock completely
(another cache contending operation) helps even more. Also whether moving
the llist_node to a different cache line than list_lock (and fields
protected by it) helps even more, or not.

> Conversely, the process of moving these parked slabs from the llist back
> to the partial list is called "unpark". Unpark occurs in four cases:
> 
> 1. Sheaf refill or alloc slowpath: This is the core case. The sheaf
>    refill or alloc slowpath must see the parked slabs, so the first
>    thing done after acquiring the lock in the sheaf refill or alloc
>    slowpath is unpark.
> 2. Cache shrinking: shrinking also needs to see slabs in the parked
>    state.
> 3. Cache destruction: kmem_cache_destroy() must also see parked slabs,
>    which is obvious, otherwise memory would leak.
> 4. delayed_work (see corner case b below)
> 
> Why is this scheme correct? Because paths entering the sheaf refill or
> alloc slowpath can see both slabs on the partial list and slabs on the
> parked llist, while allocation paths that do not enter the sheaf refill
> or alloc slowpath would not check the partial list in the first place
> and naturally do not need to care about parked slabs. Therefore, whether
> an allocation takes the sheaf refill or the alloc slowpath or not, slabs
> on the parked llist and slabs on the partial list make no difference to
> the allocator. This visibility equivalence is the core of the scheme.
> This analysis also shows that the scheme does not affect the utilization
> of partial slabs or lead to increased fragmentation.
> 
> Corner cases to handle:
> a. Parked slabs may become completely empty. Therefore, unpark must also
>    check min_partial and free excess empty slabs instead of adding them
>    back to the partial list.
> 
> b. In rare cases, the system may go idle immediately after slabs are
>    parked, and the sheaf refill or alloc slowpath may never run
>    again. These parked slabs would then remain in the llist until the
>    next sheaf refill or alloc slowpath performs an unpark. To
>    solve this problem, add a delayed_work named unpark_work to add
>    parked slabs back to the partial list when no other path unparks
>    them.

OK, but is this a problem that needs the delayed work? If the slabs are
still partial, they would just sit on the partial list rather than on the
llist, but it would cause no extra bloat?
It could be a problem only if free slab(s) got stuck on the llist.

So I'd try to avoid the delayed work as it's quite a red flag. Periodic
flushing of alien arrays used to be a very unpopular part of SLAB
implementation. I think there might be two ways:

1) submit the delayed work only when transitioning partial->empty slab on
the llist. Would likely require flagging slabs that are on the llist.
Hopefully this will limit the submissions to negligible amounts.

2) remove the delayed work completely, instead __slab_free() would perform
the "unpark" immediately when detecting partial->empty slab transition on
the llist. Would need flagging the slabs as well.

I guess 2) would be preferred unless it compromises the benefits too much.

> On will-it-scale mmap1 with 192 processes, throughput increases from
> 29237910 to 35585663 (+21.7%). native_queued_spin_lock_slowpath drops
> from 44% to 30% of cycles and __slab_free() disappears from the lock
> profile. alloc_slab,free_slab drop by 52%, and park_slab equals
> unpark_slab exactly, confirming no parked slabs are left stranded.
> 
> Signed-off-by: Hao Li <hao.li@linux.dev>