[PATCH v1 0/3] lib/842: reject malformed streams before invalid memory accesses

[Karl Mehltretter <kmehltretter@gmail.com>]

sw842_decompress() backs the crypto 842 algorithm, the NX-842 software
fallback and zram's 842 backend. Two malformed-stream checks are missing:

  - indexed and short-data templates can write beyond the caller's stated
    output capacity; and
  - a repeat after one to seven bytes of short data reads before the
    output buffer.

The first underflows the unsigned remaining length, so every later capacity
check passes. The decoder keeps writing past the destination and can
return success with an output length the caller will trust.

The decoder already returns errors for format, capacity and CRC problems;
these two cases perform invalid accesses instead.

Patches 1 and 2 add the checks. After them every output write site is
capacity-checked, repeat cannot read before the output, and all input reads
stay bounded. Patch 3 adds KUnit cases for the three malformed streams and
for three valid streams at the exact acceptance boundaries, so the new
rejections cannot regress into off-by-one.

The KMSAN reports from
syzbot+e774233ff687aada969e and syzbot+8f77ff6144a73f0cf71b are unrelated.
They trace poison from uninitialized swapped-page storage, not these bounds
failures.

KUnit under QEMU 10.2.1 TCG, two vCPUs:

                  baseline      fixed
  i386              3/6          6/6
  x86_64            3/6          6/6

Baseline runs applied patch 3 alone: the three boundary cases pass, the
three malformed streams wrongly return success. A fixed x86_64 lockdep
kernel also passes 6/6.

Separately, an x86_64 KASAN QEMU run exercised all three malformed cases
through zram's compressed writeback path by corrupting the backing-disk
contents after writeback. The baseline reported two out-of-bounds writes
and one out-of-bounds read; fixed readback returned -EIO in all three cases
without a KASAN report.

Karl Mehltretter (3):
  lib/842: reject output overflows from index and short data
  lib/842: require a complete history block for repeat templates
  lib/842: add KUnit tests for the decompressor

 MAINTAINERS                      |   1 +
 lib/842/842_decompress.c         |   7 +-
 lib/Kconfig.debug                |  15 ++++
 lib/tests/842_decompress_kunit.c | 144 +++++++++++++++++++++++++++++++
 lib/tests/Makefile               |   1 +
 5 files changed, 167 insertions(+), 1 deletion(-)
 create mode 100644 lib/tests/842_decompress_kunit.c


base-commit: cf72cbb39da84b6f02f90c07f33b102fc10b16f0
-- 
2.53.0