[PATCH 0/2] crypto: use vmalloc() for large stream workspaces

[PATCH 0/2] crypto: use vmalloc() for large stream workspaces

[Usama Arif]

The zstd and deflate acomp implementations retain per-CPU stream
workspaces until the algorithms are unregistered. On x86-64 with 4 KiB
pages, kvmalloc() can back them with order-9 and order-6 kmalloc()
blocks, respectively.

Use vmalloc() and vfree() instead. For a server with 384 CPUs, if
we have streams on all of the CPUs and kvmalloc() takes the
kmalloc() path for every stream, their workspaces use 193.5 MiB more
for zstd and 42 MiB for deflate.

In five before/after boot pairs on a 96-vCPU x86-64 VM with 8 GiB RAM,
the test populated both algorithms crypto workspaces on every CPU. All
192 baseline workspaces used kmalloc() without these patches. The patches
reduced workspace backing by 48.375 MiB for zstd and 10.5 MiB for deflate
—58.875 MiB combined—in each pair.

In five before/after runs in a 4-vCPU x86-64 VM, median compression
and decompression throughput changed by less than 1% (-0.93% to +0.64%).
Inputs were 4 KiB and 64 KiB synthetic buffers containing pseudorandom
data or a 50/50 mix of repeating-pattern and pseudorandom bytes.
The differences were within the observed run-to-run variation.
 
Usama Arif (2):
  crypto: zstd - use vmalloc() for stream workspaces
  crypto: deflate - use vmalloc() for stream workspaces

 crypto/deflate.c | 5 +++--
 crypto/zstd.c    | 4 ++--
 2 files changed, 5 insertions(+), 4 deletions(-)

-- 
2.53.0-Meta

[PATCH 1/2] crypto: zstd - use vmalloc() for stream workspaces

[Usama Arif]

The zstd acomp streams retain their workspaces after the last transform
has been freed. The first is allocated during transform initialization;
additional per-CPU streams are allocated on demand. They are released
when the algorithm is unregistered.

In a 4 KiB-page x86-64 build, each stream requests 1,566,281 bytes.
If kvmalloc() succeeds via kmalloc(), it occupies an order-9 block
(2 MiB). vmalloc() needs 383 backing pages (1,532 KiB), 516 KiB fewer
per stream. Across 384 populated streams, that is 193.5 MiB less
workspace backing if all would have used kmalloc(). There is no such
gain when kvmalloc() already falls back to vmalloc().

Use vmalloc() and vfree() directly to avoid the high-order allocation.
This uses vmalloc address space, page tables and bookkeeping; the
figures above compare workspace backing only.

Signed-off-by: Usama Arif <usama.arif@linux.dev>
---
 crypto/zstd.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/crypto/zstd.c b/crypto/zstd.c
index 76bdaca327330..f012293919368 100644
--- a/crypto/zstd.c
+++ b/crypto/zstd.c
@@ -44,7 +44,7 @@ static void *zstd_alloc_stream(void)
 	if (!wksp_size)
 		return ERR_PTR(-EINVAL);
 
-	ctx = kvmalloc_flex(*ctx, wksp, wksp_size);
+	ctx = vmalloc(struct_size(ctx, wksp, wksp_size));
 	if (!ctx)
 		return ERR_PTR(-ENOMEM);
 
@@ -56,7 +56,7 @@ static void *zstd_alloc_stream(void)
 
 static void zstd_free_stream(void *ctx)
 {
-	kvfree(ctx);
+	vfree(ctx);
 }
 
 static struct crypto_acomp_streams zstd_streams = {
-- 
2.53.0-Meta

Re: [PATCH 1/2] crypto: zstd - use vmalloc() for stream workspaces

[Rik van Riel]

On Wed, 2026-11-11 at 07:07 -0800, Usama Arif wrote:
> The zstd acomp streams retain their workspaces after the last
> transform
> has been freed. The first is allocated during transform
> initialization;
> additional per-CPU streams are allocated on demand. They are released
> when the algorithm is unregistered.
> 
> In a 4 KiB-page x86-64 build, each stream requests 1,566,281 bytes.
> If kvmalloc() succeeds via kmalloc(), it occupies an order-9 block
> (2 MiB). vmalloc() needs 383 backing pages (1,532 KiB), 516 KiB fewer
> per stream. Across 384 populated streams, that is 193.5 MiB less
> 
Woah. Nice catch

> Signed-off-by: Usama Arif <usama.arif@linux.dev>
> 
Reviewed-by: Rik van Riel <riel@surriel.com>

-- 
All Rights Reversed.

Re: [PATCH 1/2] crypto: zstd - use vmalloc() for stream workspaces

[Johannes Weiner]

On Wed, Nov 11, 2026 at 07:07:39AM -0800, Usama Arif wrote:
> The zstd acomp streams retain their workspaces after the last transform
> has been freed. The first is allocated during transform initialization;
> additional per-CPU streams are allocated on demand. They are released
> when the algorithm is unregistered.
> 
> In a 4 KiB-page x86-64 build, each stream requests 1,566,281 bytes.
> If kvmalloc() succeeds via kmalloc(), it occupies an order-9 block
> (2 MiB). vmalloc() needs 383 backing pages (1,532 KiB), 516 KiB fewer
> per stream. Across 384 populated streams, that is 193.5 MiB less
> workspace backing if all would have used kmalloc(). There is no such
> gain when kvmalloc() already falls back to vmalloc().
> 
> Use vmalloc() and vfree() directly to avoid the high-order allocation.
> This uses vmalloc address space, page tables and bookkeeping; the
> figures above compare workspace backing only.
> 
> Signed-off-by: Usama Arif <usama.arif@linux.dev>

Nice catch. Given the lifetime of these streams, there really isn't
enough turnover to justify the kmalloc optimization to begin with.

Reviewed-by: Johannes Weiner <hannes@cmpxchg.org>

Re: [PATCH 1/2] crypto: zstd - use vmalloc() for stream workspaces

[Chris Down]

Hi Usama,

Usama Arif writes:
>Use vmalloc() and vfree() directly to avoid the high-order allocation.

vmalloc() doesn't avoid high-order allocations any more. Since
a06157804399 ("mm/vmalloc: request large order pages from buddy
allocator"), vm_area_alloc_pages() first tries the largest order that
fits, and splits the block up afterwards. So for these 383 pages it still
asks for an order-8 block first, then order 6, 5, 4, 3, 2 and 1.

What the patch does avoid is kmalloc() rounding the 1.5 MiB up to a 2 MiB
block, which is where the 516 KiB per stream comes from. Maybe say that
instead?

Once that's fixed, please feel free to add:

Reviewed-by: Chris Down <chris@chrisdown.name>
Tested-by: Chris Down <chris@chrisdown.name>

Chris

[PATCH 2/2] crypto: deflate - use vmalloc() for stream workspaces

[Usama Arif]

The deflate acomp streams outlive the transforms that use them. One
stream is allocated during transform initialization; additional
per-CPU streams are allocated on demand. They are freed only when
the algorithm is unregistered.

In a 4 KiB-page x86-64 build, each stream requests 145,320 bytes.
If kvmalloc() succeeds via kmalloc(), it occupies an order-6 block
(256 KiB). vmalloc() needs 36 backing pages (144 KiB), 112 KiB fewer
per stream. Across 384 populated streams, that is 42 MiB less
workspace backing if all would have used kmalloc(). There is no such
gain when kvmalloc() already falls back to vmalloc().

Use vmalloc() and vfree() directly, as the crypto lz4 implementations
already do. This uses vmalloc address space, page tables and
bookkeeping; the figures above compare workspace backing only.

Signed-off-by: Usama Arif <usama.arif@linux.dev>
---
 crypto/deflate.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/crypto/deflate.c b/crypto/deflate.c
index 710ebba7ce850..a8692532a65da 100644
--- a/crypto/deflate.c
+++ b/crypto/deflate.c
@@ -20,6 +20,7 @@
 #include <linux/scatterlist.h>
 #include <linux/slab.h>
 #include <linux/spinlock.h>
+#include <linux/vmalloc.h>
 #include <linux/zlib.h>
 
 #define DEFLATE_DEF_LEVEL		Z_DEFAULT_COMPRESSION
@@ -40,7 +41,7 @@ static void *deflate_alloc_stream(void)
 						     DEFLATE_DEF_MEMLEVEL));
 	struct deflate_stream *ctx;
 
-	ctx = kvmalloc_flex(*ctx, workspace, size);
+	ctx = vmalloc(struct_size(ctx, workspace, size));
 	if (!ctx)
 		return ERR_PTR(-ENOMEM);
 
@@ -51,7 +52,7 @@ static void *deflate_alloc_stream(void)
 
 static void deflate_free_stream(void *ctx)
 {
-	kvfree(ctx);
+	vfree(ctx);
 }
 
 static struct crypto_acomp_streams deflate_streams = {
-- 
2.53.0-Meta

Re: [PATCH 2/2] crypto: deflate - use vmalloc() for stream workspaces

[Rik van Riel]

On Wed, 2026-11-11 at 07:07 -0800, Usama Arif wrote:
> The deflate acomp streams outlive the transforms that use them. One
> stream is allocated during transform initialization; additional
> per-CPU streams are allocated on demand. They are freed only when
> the algorithm is unregistered.
> 
> In a 4 KiB-page x86-64 build, each stream requests 145,320 bytes.
> If kvmalloc() succeeds via kmalloc(), it occupies an order-6 block
> (256 KiB). vmalloc() needs 36 backing pages (144 KiB), 112 KiB fewer
> per stream. Across 384 populated streams, that is 42 MiB less
> workspace backing if all would have used kmalloc(). There is no such
> gain when kvmalloc() already falls back to vmalloc().
> 
> Use vmalloc() and vfree() directly, as the crypto lz4 implementations
> already do. This uses vmalloc address space, page tables and
> bookkeeping; the figures above compare workspace backing only.
> 
> Signed-off-by: Usama Arif <usama.arif@linux.dev>

Reviewed-by: Rik van Riel <riel@surriel.com>

-- 
All Rights Reversed.

Re: [PATCH 2/2] crypto: deflate - use vmalloc() for stream workspaces

[Johannes Weiner]

On Wed, Nov 11, 2026 at 07:07:40AM -0800, Usama Arif wrote:
> The deflate acomp streams outlive the transforms that use them. One
> stream is allocated during transform initialization; additional
> per-CPU streams are allocated on demand. They are freed only when
> the algorithm is unregistered.
> 
> In a 4 KiB-page x86-64 build, each stream requests 145,320 bytes.
> If kvmalloc() succeeds via kmalloc(), it occupies an order-6 block
> (256 KiB). vmalloc() needs 36 backing pages (144 KiB), 112 KiB fewer
> per stream. Across 384 populated streams, that is 42 MiB less
> workspace backing if all would have used kmalloc(). There is no such
> gain when kvmalloc() already falls back to vmalloc().
> 
> Use vmalloc() and vfree() directly, as the crypto lz4 implementations
> already do. This uses vmalloc address space, page tables and
> bookkeeping; the figures above compare workspace backing only.
> 
> Signed-off-by: Usama Arif <usama.arif@linux.dev>

Reviewed-by: Johannes Weiner <hannes@cmpxchg.org>

Re: [PATCH 2/2] crypto: deflate - use vmalloc() for stream workspaces

[Chris Down]

Hi Usama,

Thanks! I've tested this with zswap using deflate, and it works well.

Reviewed-by: Chris Down <chris@chrisdown.name>
Tested-by: Chris Down <chris@chrisdown.name>

Chris