[PATCH v2] block: try slab allocation in bio_alloc_bioset() before mempool

[PATCH v2] block: try slab allocation in bio_alloc_bioset() before mempool

[Joseph Qi]

When the per-CPU bio cache is enabled but empty, bio_alloc_percpu_cache()
returns NULL and bio_alloc_bioset() falls straight through to the mempool
fallback:

    if (unlikely(!bio)) {
        if (!(saved_gfp & __GFP_DIRECT_RECLAIM))
            return NULL;
        ...
    }

For non-sleeping allocations (no __GFP_DIRECT_RECLAIM) this returns NULL
without ever attempting a slab allocation, even when there is plenty of
free memory.

Commit b520c4eef83d ("block: split bio_alloc_bioset more clearly into a
fast and slowpath") introduced this. Before it, a percpu cache miss fell
through to mempool_alloc(), which attempted the underlying slab allocation
first and only failed when that slab allocation failed. The restructuring
dropped the slab attempt that non-sleeping callers of a cache-enabled
bioset (such as the default fs_bio_set used by bio_alloc()) relied on.

Try a slab allocation with optimistic GFP_ flags before falling back to
the mempool whenever the bio is still NULL, so both the cache-empty and
non-cache paths share the same slab attempt. This restores the previous
behavior for non-sleeping allocations.

Fixes: b520c4eef83d ("block: split bio_alloc_bioset more clearly into a fast and slowpath")
Suggested-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Joseph Qi <joseph.qi@linux.alibaba.com>
---
 block/bio.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/block/bio.c b/block/bio.c
index f2a5f4d0a9672..982f2e47a4218 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -555,6 +555,14 @@ struct bio *bio_alloc_bioset(struct block_device *bdev, unsigned short nr_vecs,
 		bio = bio_alloc_percpu_cache(bs);
 	} else {
 		opf &= ~REQ_ALLOC_CACHE;
+	}
+
+	/*
+	 * For a bioset without a percpu cache, or when the percpu cache was
+	 * empty, try a slab allocation with optimistic GFP_ flags before
+	 * falling back to the mempool.
+	 */
+	if (!bio) {
 		p = kmem_cache_alloc(bs->bio_slab, gfp);
 		if (p)
 			bio = p + bs->front_pad;
-- 
2.39.3

Re: [PATCH v2] block: try slab allocation in bio_alloc_bioset() before mempool

[Christoph Hellwig]

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH v2] block: try slab allocation in bio_alloc_bioset() before mempool

[Joseph Qi]

Gentle ping...

On 7/9/26 10:01 AM, Joseph Qi wrote:
> When the per-CPU bio cache is enabled but empty, bio_alloc_percpu_cache()
> returns NULL and bio_alloc_bioset() falls straight through to the mempool
> fallback:
> 
>     if (unlikely(!bio)) {
>         if (!(saved_gfp & __GFP_DIRECT_RECLAIM))
>             return NULL;
>         ...
>     }
> 
> For non-sleeping allocations (no __GFP_DIRECT_RECLAIM) this returns NULL
> without ever attempting a slab allocation, even when there is plenty of
> free memory.
> 
> Commit b520c4eef83d ("block: split bio_alloc_bioset more clearly into a
> fast and slowpath") introduced this. Before it, a percpu cache miss fell
> through to mempool_alloc(), which attempted the underlying slab allocation
> first and only failed when that slab allocation failed. The restructuring
> dropped the slab attempt that non-sleeping callers of a cache-enabled
> bioset (such as the default fs_bio_set used by bio_alloc()) relied on.
> 
> Try a slab allocation with optimistic GFP_ flags before falling back to
> the mempool whenever the bio is still NULL, so both the cache-empty and
> non-cache paths share the same slab attempt. This restores the previous
> behavior for non-sleeping allocations.
> 
> Fixes: b520c4eef83d ("block: split bio_alloc_bioset more clearly into a fast and slowpath")
> Suggested-by: Christoph Hellwig <hch@lst.de>
> Signed-off-by: Joseph Qi <joseph.qi@linux.alibaba.com>
> ---
>  block/bio.c | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/block/bio.c b/block/bio.c
> index f2a5f4d0a9672..982f2e47a4218 100644
> --- a/block/bio.c
> +++ b/block/bio.c
> @@ -555,6 +555,14 @@ struct bio *bio_alloc_bioset(struct block_device *bdev, unsigned short nr_vecs,
>  		bio = bio_alloc_percpu_cache(bs);
>  	} else {
>  		opf &= ~REQ_ALLOC_CACHE;
> +	}
> +
> +	/*
> +	 * For a bioset without a percpu cache, or when the percpu cache was
> +	 * empty, try a slab allocation with optimistic GFP_ flags before
> +	 * falling back to the mempool.
> +	 */
> +	if (!bio) {
>  		p = kmem_cache_alloc(bs->bio_slab, gfp);
>  		if (p)
>  			bio = p + bs->front_pad;

Re: [PATCH v2] block: try slab allocation in bio_alloc_bioset() before mempool

[Jens Axboe]

On Thu, 09 Jul 2026 10:01:45 +0800, Joseph Qi wrote:
> When the per-CPU bio cache is enabled but empty, bio_alloc_percpu_cache()
> returns NULL and bio_alloc_bioset() falls straight through to the mempool
> fallback:
> 
>     if (unlikely(!bio)) {
>         if (!(saved_gfp & __GFP_DIRECT_RECLAIM))
>             return NULL;
>         ...
>     }
> 
> [...]

Applied, thanks!

[1/1] block: try slab allocation in bio_alloc_bioset() before mempool
      commit: 447cfed6d700bfbd5a7120f8ea5821a0e1191667

Best regards,
-- 
Jens Axboe