From: Eric Biggers <ebiggers@kernel.org>
To: Xueyuan Chen <xueyuan.chen21@gmail.com>
Cc: Minchan Kim <minchan@kernel.org>,
Sergey Senozhatsky <senozhatsky@chromium.org>,
Jens Axboe <axboe@kernel.dk>,
linux-kernel@vger.kernel.org, linux-block@vger.kernel.org
Subject: Re: [PATCH RFC 0/3] zram: avoid copying spanning LZ4 objects before decompression
Date: Sat, 21 Nov 2026 12:15:11 -0800 [thread overview]
Message-ID: <20261121201511.GG1975@quark> (raw)
In-Reply-To: <20261122-zram-split-decomp-mm-rfc-v1-0-4e69b08c0fed@gmail.com>
Hi Xueyuan,
On Sun, Nov 22, 2026 at 01:09:23AM +0800, Xueyuan Chen wrote:
> I measured the time to swap in 1 GiB of synthetic data on arm64 with
> 4 KiB pages. Each page started with pseudorandom data of one of 16
> lengths, followed by zeros. On this workload, 32.6% of compressed
> reads spanned two pages with LZ4 and 34.4% with LZ4HC.
>
> The table shows median swap-in times over seven pairs. The last two
> columns summarize the percentage change within each pair.
>
> Codec Workers Baseline (ms) Patched (ms) Median Range
> LZ4 1 1403.81 1344.45 -4.21% -4.64% to -1.61%
> LZ4 8 222.57 216.36 -2.73% -4.19% to -1.97%
> LZ4HC 1 1403.35 1336.99 -4.74% -5.07% to -2.90%
> LZ4HC 8 222.10 215.77 -3.16% -4.39% to -1.27%
>
> These timings include both skipping the copy and switching decoders.
>
> I used Codex to help implement this series.
> Do you think the swap-in improvement justifies maintaining a second
> LZ4 decoder?
>
> Assisted-by: LLM
> Signed-off-by: Xueyuan Chen <xueyuan.chen21@gmail.com>
LZ4_decompress_safe_split() isn't actually very well optimized. While
it avoids the copy of the source data into a contiguous buffer, it
copies every match and literal run with a variable-length memcpy(),
which on most data is a very slow way to implement LZ decompression.
I suspect it looked good on your synthetic benchmark (pseudorandom data
followed by 0's) only because that tended to generate 1 very long
literal run and 1 very long match. Whereas normal data would have lots
of interleaved literal runs and matches of much shorter lengths where an
external memcpy() call does very poorly; that is why the existing LZ4
decompression code does not do it that way.
To some extent the literal run and match copy performance is fixable
with a better implementation. However, having to handle two different
source buffers still introduces overhead that cannot go away.
So I feel this approach is probably not worth it. If you'd like to
continue to explore it, it would need to be tested on different data and
would need to be properly optimized.
- Eric
prev parent reply other threads:[~2026-11-21 20:15 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-21 17:09 Xueyuan Chen
2026-11-21 17:09 ` [PATCH RFC 1/3] zram: allow backends to decompress from two input buffers Xueyuan Chen
2026-11-21 17:09 ` [PATCH RFC 2/3] lib/lz4: add safe decompression " Xueyuan Chen
2026-11-21 17:09 ` [PATCH RFC 3/3] zram: use split-input decompression for LZ4 and LZ4HC Xueyuan Chen
2026-11-21 20:15 ` Eric Biggers [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261121201511.GG1975@quark \
--to=ebiggers@kernel.org \
--cc=axboe@kernel.dk \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=minchan@kernel.org \
--cc=senozhatsky@chromium.org \
--cc=xueyuan.chen21@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®