mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Nick Terrell <terrelln@fb.com>
To: Oleksandr Natalenko <oleksandr@natalenko.name>
Cc: Piotr Gorski <lucjan.lucjanov@gmail.com>,
	LKML <linux-kernel@vger.kernel.org>,
	Linux Kbuild mailing list <linux-kbuild@vger.kernel.org>,
	Masahiro Yamada <masahiroy@kernel.org>,
	Michal Marek <michal.lkml@markovi.net>,
	Andrew Morton <akpm@linux-foundation.org>,
	Kees Cook <keescook@chromium.org>,
	"Nathan Chancellor" <nathan@kernel.org>,
	Daniel Borkmann <daniel@iogearbox.net>,
	Valentin Schneider <valentin.schneider@arm.com>,
	"Jan Alexander Steffens (heftig)" <jan.steffens@gmail.com>,
	David Howells <dhowells@redhat.com>,
	Johannes Weiner <hannes@cmpxchg.org>
Subject: Re: [PATCH] init: add support for zstd compressed modules
Date: Wed, 31 Mar 2021 17:39:25 +0000	[thread overview]
Message-ID: <33D7F6C9-6F53-471A-A146-7F128F6918F5@fb.com> (raw)
In-Reply-To: <20210330115023.qt742qsdekwiroey@spock.localdomain>



> On Mar 30, 2021, at 4:50 AM, Oleksandr Natalenko <oleksandr@natalenko.name> wrote:
> 
> On Tue, Mar 30, 2021 at 01:32:35PM +0200, Piotr Gorski wrote:
>> kmod 28 supports modules compressed in zstd format so let's add this possibility to kernel.
>> 
>> Signed-off-by: Piotr Gorski <lucjan.lucjanov@gmail.com>
>> ---
>> Makefile     | 7 +++++--
>> init/Kconfig | 9 ++++++---
>> 2 files changed, 11 insertions(+), 5 deletions(-)
>> 
>> diff --git a/Makefile b/Makefile
>> index 5160ff8903c1..82f4f4cc2955 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -1156,8 +1156,8 @@ endif # INSTALL_MOD_STRIP
>> export mod_strip_cmd
>> 
>> # CONFIG_MODULE_COMPRESS, if defined, will cause module to be compressed
>> -# after they are installed in agreement with CONFIG_MODULE_COMPRESS_GZIP
>> -# or CONFIG_MODULE_COMPRESS_XZ.
>> +# after they are installed in agreement with CONFIG_MODULE_COMPRESS_GZIP,
>> +# CONFIG_MODULE_COMPRESS_XZ, or CONFIG_MODULE_COMPRESS_ZSTD.
>> 
>> mod_compress_cmd = true
>> ifdef CONFIG_MODULE_COMPRESS
>> @@ -1167,6 +1167,9 @@ ifdef CONFIG_MODULE_COMPRESS
>>   ifdef CONFIG_MODULE_COMPRESS_XZ
>>     mod_compress_cmd = $(XZ) --lzma2=dict=2MiB -f
>>   endif # CONFIG_MODULE_COMPRESS_XZ
>> +  ifdef CONFIG_MODULE_COMPRESS_ZSTD
>> +    mod_compress_cmd = $(ZSTD) -T0 --rm -f -q

This will use the default zstd level, level 3. I think it would make more sense to use a high
compression level. Level 19 would probably be a good choice. That will choose a window
size of up to 8MB, meaning the decompressor needs to allocate that much memory. If that
is unacceptable, you could use `zstd -T0 --rm -f -q -19 --zstd=wlog=21`, which will use a
window size of up to 2MB, to match the XZ command. Note that if the file is smaller than
the window size, it will be shrunk to the smallest power of two at least as large as the file.

Best,
Nick Terrell

>> +  endif # CONFIG_MODULE_COMPRESS_ZSTD
>> endif # CONFIG_MODULE_COMPRESS
>> export mod_compress_cmd
>> 
>> diff --git a/init/Kconfig b/init/Kconfig
>> index 8c2cfd88f6ef..86a452bc2747 100644
>> --- a/init/Kconfig
>> +++ b/init/Kconfig
>> @@ -2250,8 +2250,8 @@ config MODULE_COMPRESS
>> 	bool "Compress modules on installation"
>> 	help
>> 
>> -	  Compresses kernel modules when 'make modules_install' is run; gzip or
>> -	  xz depending on "Compression algorithm" below.
>> +	  Compresses kernel modules when 'make modules_install' is run; gzip,
>> +	  xz, or zstd depending on "Compression algorithm" below.
>> 
>> 	  module-init-tools MAY support gzip, and kmod MAY support gzip and xz.
>> 
>> @@ -2273,7 +2273,7 @@ choice
>> 	  This determines which sort of compression will be used during
>> 	  'make modules_install'.
>> 
>> -	  GZIP (default) and XZ are supported.
>> +	  GZIP (default), XZ, and ZSTD are supported.
>> 
>> config MODULE_COMPRESS_GZIP
>> 	bool "GZIP"
>> @@ -2281,6 +2281,9 @@ config MODULE_COMPRESS_GZIP
>> config MODULE_COMPRESS_XZ
>> 	bool "XZ"
>> 
>> +config MODULE_COMPRESS_ZSTD
>> +	bool "ZSTD"
>> +
>> endchoice
>> 
>> config MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS
>> -- 
>> 2.31.0.97.g1424303384
>> 
> 
> Great!
> 
> Reviewed-by: Oleksandr Natalenko <oleksandr@natalenko.name>
> 
> This works perfectly fine in Arch Linux if accompanied by the
> following mkinitcpio amendment: [1].
> 
> I'm also Cc'ing other people from get_maintainers output just
> to make this submission more visible.
> 
> Thanks.
> 
> [1] https://github.com/archlinux/mkinitcpio/pull/43
> 
> -- 
>  Oleksandr Natalenko (post-factum)


  reply	other threads:[~2021-03-31 17:41 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-30 11:32 Piotr Gorski
2021-03-30 11:50 ` Oleksandr Natalenko
2021-03-31 17:39   ` Nick Terrell [this message]
2021-03-31 17:48     ` Oleksandr Natalenko
2021-03-31 19:21       ` Nick Terrell
2021-03-31 20:28         ` Oleksandr Natalenko
2021-04-07 13:53         ` Masahiro Yamada
2021-04-07 17:45           ` Nick Terrell
2021-04-07 14:32 ` Masahiro Yamada
     [not found] <722602000.5829.1617263646814@office.mailbox.org>
2021-04-01  8:01 ` torvic9
2021-04-01 19:18 ` Nick Terrell

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=33D7F6C9-6F53-471A-A146-7F128F6918F5@fb.com \
    --to=terrelln@fb.com \
    --cc=akpm@linux-foundation.org \
    --cc=daniel@iogearbox.net \
    --cc=dhowells@redhat.com \
    --cc=hannes@cmpxchg.org \
    --cc=jan.steffens@gmail.com \
    --cc=keescook@chromium.org \
    --cc=linux-kbuild@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lucjan.lucjanov@gmail.com \
    --cc=masahiroy@kernel.org \
    --cc=michal.lkml@markovi.net \
    --cc=nathan@kernel.org \
    --cc=oleksandr@natalenko.name \
    --cc=valentin.schneider@arm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®