mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Manuel Ebner <manuelebnerli@mailbox.org>
To: "Lorenzo Stoakes (ARM)" <ljs@kernel.org>,
	"Linus Torvalds" <torvalds@linux-foundation.org>,
	"Nathan Chancellor" <nathan@kernel.org>,
	"Nicolas Schier" <nsc@kernel.org>,
	"Nick Desaulniers" <ndesaulniers@google.com>,
	"Bill Wendling" <morbo@google.com>,
	"Justin Stitt" <justinstitt@google.com>,
	"Masahiro Yamada" <masahiroy@kernel.org>,
	"Alexey Gladkov" <legion@kernel.org>,
	"Thomas Gleixner" <tglx@kernel.org>,
	"Ingo Molnar" <mingo@redhat.com>,
	"Borislav Petkov" <bp@alien8.de>,
	"Dave Hansen" <dave.hansen@linux.intel.com>,
	x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>,
	"Paul Walmsley" <pjw@kernel.org>,
	"Palmer Dabbelt" <palmer@dabbelt.com>,
	"Albert Ou" <aou@eecs.berkeley.edu>,
	"Alexandre Ghiti" <alex@ghiti.fr>,
	"Arnd Bergmann" <arnd@arndb.de>,
	"Catalin Marinas" <catalin.marinas@arm.com>,
	"Will Deacon" <will@kernel.org>,
	"Mark Rutland" <mark.rutland@arm.com>,
	"Ard Biesheuvel" <ardb@kernel.org>,
	"Ilias Apalodimas" <ilias.apalodimas@linaro.org>,
	"Josh Poimboeuf" <jpoimboe@kernel.org>,
	"Peter Zijlstra" <peterz@infradead.org>,
	"Miguel Ojeda" <ojeda@kernel.org>,
	"Boqun Feng" <boqun@kernel.org>, "Gary Guo" <gary@garyguo.net>,
	"Björn Roy Baron" <bjorn3_gh@protonmail.com>,
	"Benno Lossin" <lossin@kernel.org>,
	"Andreas Hindborg" <a.hindborg@kernel.org>,
	"Alice Ryhl" <aliceryhl@google.com>,
	"Trevor Gross" <tmgross@umich.edu>,
	"Danilo Krummrich" <dakr@kernel.org>,
	"Daniel Almeida" <daniel.almeida@collabora.com>,
	"Tamir Duberstein" <tamird@kernel.org>,
	"Alexandre Courbot" <acourbot@nvidia.com>,
	"Onur Özkan" <work@onurozkan.dev>,
	"Jonathan Corbet" <corbet@lwn.net>,
	"Randy Dunlap" <rdunlap@infradead.org>,
	"Kees Cook" <kees@kernel.org>,
	"Gustavo A. R. Silva" <gustavoars@kernel.org>
Cc: linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org,
	 llvm@lists.linux.dev, linux-riscv@lists.infradead.org,
	 linux-arch@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org,  linux-efi@vger.kernel.org,
	rust-for-linux@vger.kernel.org,  linux-doc@vger.kernel.org,
	Jens Axboe <axboe@kernel.dk>,
	 linux-hardening@vger.kernel.org
Subject: Re: [PATCH v3 20/20] kbuild: compress the kernel with pigz if available
Date: Fri, 18 Sep 2026 16:27:39 +0200	[thread overview]
Message-ID: <3be3dca3bee5695972e5419a1198bfee40a1ede7.camel@mailbox.org> (raw)
In-Reply-To: <20260917-build-speedup-v3-20-9ecf4163ff36@kernel.org>

Hi Lorenzo,

On Thu, 2026-09-17 at 17:06 +0100, Lorenzo Stoakes (ARM) wrote:
> The gzip step of a kernel build is very lengthy, especially for larger
> builds such as allmodconfig.
> 
> ...
> 
> Whole build, 128-thread Threadripper 9980X, best of N runs:
> 
>                                          before   after     delta
>                                          -------------------------------
>   x86 defconfig, touch mm/vma.c, gcc        7.7s     6.0s     -1.7s (-22%)
>   x86 defconfig, touch mm/vma.c, clang      6.9s     5.4s     -1.5s (-21%)
>   x86 defconfig, clean, gcc                28.2s    26.3s     -1.8s (-6%)
>   x86 defconfig, clean, clang              28.7s    27.2s     -1.5s (-5%)
>   x86 allmodconfig, touch mm/vma.c, gcc    23.7s    15.4s     -8.3s (-35%)
>   x86 allmodconfig, touch mm/vma.c, clang  22.5s    15.6s     -7.0s (-31%)
>   x86 allmodconfig, clean, gcc            294.1s   278.6s     -15.5s (-5%)
>   x86 allmodconfig, clean, clang          283.9s   266.8s     -17.1s (-6%)

That's good to hear.


> Suggested-by: Kees Cook <kees@kernel.org>
> Link: https://zlib.net/pigz/
> Assisted-by: LLM
> Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
> ---
>  Documentation/kbuild/kbuild.rst   | 17 +++++++++++++++++
>  Documentation/process/changes.rst |  8 ++++++++
>  Makefile                          |  5 ++++-
>  kernel/Makefile                   |  6 +++++-
>  scripts/Makefile.lib              |  4 ++--

Please add a line to
scripts/ver_linux

>  scripts/jobserver-exec            |  3 ++-
>  tools/lib/python/jobserver.py     | 13 +++++++++++++
>  7 files changed, 51 insertions(+), 5 deletions(-)
> 
> diff --git a/Documentation/kbuild/kbuild.rst b/Documentation/kbuild/kbuild.rst
> index 5a9013bacfb7..e71a692a559a 100644
> --- a/Documentation/kbuild/kbuild.rst
> +++ b/Documentation/kbuild/kbuild.rst
> @@ -110,6 +110,23 @@ HOSTLDLIBS
>  ----------
>  Additional libraries to link against when building host programs.
>  
> +KGZIP
> +-----
> +The gzip program used for compressed modules, packaging and the embedded

Could you also add gzip to changes.rst and ver_linux while you are at it?
Rest looks good to me.
Reviewed-by: Manuel Ebner <manuelebnerli@mailbox.org>

Thanks
 Manuel

> +kernel configuration. Defaults to gzip.
> +
> +KPGZIP
> +------
> +The gzip program used for the compressed kernel image. Defaults to pigz (a
> +parallel implementation of gzip), run under scripts/jobserver-exec so that it
> +uses the job slots make has free, if pigz is installed, otherwise to
> +``KGZIP``. Set ``KPGZIP=`` on the make command line to use another program.
> +
> +KBZIP2, KLZOP, LZMA, LZ4, XZ, ZSTD
> +----------------------------------
> +The compressor programs for the other formats. Each defaults to the program
> +of the same name.
> +
>  .. _userkbuildflags:
>  
>  USERCFLAGS
> diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst
> index 0aa232b117b5..28f2ca63318e 100644
> --- a/Documentation/process/changes.rst
> +++ b/Documentation/process/changes.rst
> @@ -55,6 +55,7 @@ nfs-utils              1.0.5            showmount --version

+ gzip

>  openssl & libcrypto    1.0.0            openssl version
>  pahole                 1.26             pahole --version
>  pcmciautils            004              pccardctl -V
> +pigz (optional)        2.4              pigz --version
>  PPP                    2.4.0            pppd --version
>  procps                 3.2.0            ps --version
>  Python                 3.9.x            python3 --version
> @@ -206,6 +207,13 @@ GNU AWK
>  GNU AWK is needed if you want kernel builds to generate address range data for
>  builtin modules (CONFIG_BUILTIN_MODULE_RANGES).
>  
> +pigz (optional)
> +---------------
> +
> +pigz is a parallel implementation of gzip. If it is installed the compressed
> +kernel image is produced with it rather than with gzip, see ``KPGZIP`` in
> +Documentation/kbuild/kbuild.rst.
> +
>  System utilities
>  ****************
>  
> diff --git a/Makefile b/Makefile
> ...


  parent reply	other threads:[~2026-09-18 14:28 UTC|newest]

Thread overview: 58+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-09-17 16:06 [PATCH v3 00/20] kbuild: significantly speed up kernel builds Lorenzo Stoakes (ARM)
2026-09-17 16:06 ` [PATCH v3 01/20] kbuild: do not allocate .modinfo in vmlinux Lorenzo Stoakes (ARM)
2026-09-17 16:52   ` Kees Cook
2026-09-17 17:41     ` Lorenzo Stoakes (ARM)
2026-09-18  0:51     ` Nathan Chancellor
2026-09-18 17:09       ` Nicolas Schier
2026-09-18 21:44         ` Nathan Chancellor
2026-09-17 17:05   ` Kees Cook
2026-09-17 17:39     ` Lorenzo Stoakes (ARM)
2026-09-17 16:06 ` [PATCH v3 02/20] kallsyms: index symbols by token to speed up table compression Lorenzo Stoakes (ARM)
2026-09-17 17:27   ` Kees Cook
2026-09-17 17:45     ` Lorenzo Stoakes (ARM)
2026-09-17 16:06 ` [PATCH v3 03/20] kallsyms: output binary data to speed output and kallsyms assembly Lorenzo Stoakes (ARM)
2026-09-17 17:36   ` Kees Cook
2026-09-17 16:06 ` [PATCH v3 04/20] kbuild: do not sort nm output where the order is irrelevant Lorenzo Stoakes (ARM)
2026-09-17 17:38   ` Kees Cook
2026-09-18 19:14   ` Nicolas Schier
2026-09-17 16:06 ` [PATCH v3 05/20] kbuild: only emit vmlinux relocations when required Lorenzo Stoakes (ARM)
2026-09-17 17:41   ` Kees Cook
2026-09-17 17:48     ` Lorenzo Stoakes (ARM)
2026-09-18 19:27   ` Nicolas Schier
2026-09-17 16:06 ` [PATCH v3 06/20] elf-parse: add section flags, symbol binding and a read-only mapping Lorenzo Stoakes (ARM)
2026-09-17 17:44   ` Kees Cook
2026-09-17 16:06 ` [PATCH v3 07/20] kallsyms: reimplement mksysmap in C Lorenzo Stoakes (ARM)
2026-09-17 18:07   ` Kees Cook
2026-09-17 16:06 ` [PATCH v3 08/20] kbuild: cache list, composite object state per object Lorenzo Stoakes (ARM)
2026-09-17 18:11   ` Kees Cook
2026-09-18 19:51     ` Nicolas Schier
2026-09-17 16:06 ` [PATCH v3 09/20] kbuild: implement and use depcheck to check dependency timestamps Lorenzo Stoakes (ARM)
2026-09-17 18:42   ` Kees Cook
2026-09-17 20:59     ` Kees Cook
2026-09-17 16:06 ` [PATCH v3 10/20] kbuild: move the toolchain checks into init/Kconfig.toolchain Lorenzo Stoakes (ARM)
2026-09-17 18:53   ` Kees Cook
2026-09-18  1:07     ` Nathan Chancellor
2026-09-18 14:35       ` Lorenzo Stoakes (ARM)
2026-09-18 19:54         ` Nicolas Schier
2026-09-18 21:12           ` Nathan Chancellor
2026-09-17 16:06 ` [PATCH v3 11/20] kbuild: avoid re-running compiler and linker probes Lorenzo Stoakes (ARM)
2026-09-17 19:26   ` Kees Cook
2026-09-18  1:21     ` Nathan Chancellor
2026-09-18  4:44       ` Kees Cook
2026-09-18  5:40         ` Nathan Chancellor
2026-09-17 16:06 ` [PATCH v3 12/20] modpost: cache section relocation mismatch state Lorenzo Stoakes (ARM)
2026-09-17 16:06 ` [PATCH v3 13/20] modpost: emit module descriptors as assembly Lorenzo Stoakes (ARM)
2026-09-17 16:06 ` [PATCH v3 14/20] kbuild: batch module finalisation Lorenzo Stoakes (ARM)
2026-09-17 17:01   ` Kees Cook
2026-09-17 16:06 ` [PATCH v3 15/20] objtool: cache relocations, do less work, eliminate relocation hash Lorenzo Stoakes (ARM)
2026-09-17 16:06 ` [PATCH v3 16/20] objtool: size the instruction hash to the text Lorenzo Stoakes (ARM)
2026-09-17 16:06 ` [PATCH v3 17/20] objtool: decode instructions and resolve branch targets in parallel Lorenzo Stoakes (ARM)
2026-09-17 16:06 ` [PATCH v3 18/20] rust: make exports.o depend on the headers generated for it Lorenzo Stoakes (ARM)
2026-09-17 16:06 ` [PATCH v3 19/20] kbuild: build rust crates in parallel with the rest of the build Lorenzo Stoakes (ARM)
2026-09-17 16:06 ` [PATCH v3 20/20] kbuild: compress the kernel with pigz if available Lorenzo Stoakes (ARM)
2026-09-17 16:58   ` Kees Cook
2026-09-18 14:27   ` Manuel Ebner [this message]
2026-09-17 17:15 ` [PATCH v3 00/20] kbuild: significantly speed up kernel builds Linus Torvalds
2026-09-17 17:36   ` Lorenzo Stoakes (ARM)
2026-09-17 19:42     ` Lorenzo Stoakes (ARM)
2026-09-17 20:02       ` Nick Desaulniers

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=3be3dca3bee5695972e5419a1198bfee40a1ede7.camel@mailbox.org \
    --to=manuelebnerli@mailbox.org \
    --cc=a.hindborg@kernel.org \
    --cc=acourbot@nvidia.com \
    --cc=alex@ghiti.fr \
    --cc=aliceryhl@google.com \
    --cc=aou@eecs.berkeley.edu \
    --cc=ardb@kernel.org \
    --cc=arnd@arndb.de \
    --cc=axboe@kernel.dk \
    --cc=bjorn3_gh@protonmail.com \
    --cc=boqun@kernel.org \
    --cc=bp@alien8.de \
    --cc=catalin.marinas@arm.com \
    --cc=corbet@lwn.net \
    --cc=dakr@kernel.org \
    --cc=daniel.almeida@collabora.com \
    --cc=dave.hansen@linux.intel.com \
    --cc=gary@garyguo.net \
    --cc=gustavoars@kernel.org \
    --cc=hpa@zytor.com \
    --cc=ilias.apalodimas@linaro.org \
    --cc=jpoimboe@kernel.org \
    --cc=justinstitt@google.com \
    --cc=kees@kernel.org \
    --cc=legion@kernel.org \
    --cc=linux-arch@vger.kernel.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-efi@vger.kernel.org \
    --cc=linux-hardening@vger.kernel.org \
    --cc=linux-kbuild@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-riscv@lists.infradead.org \
    --cc=ljs@kernel.org \
    --cc=llvm@lists.linux.dev \
    --cc=lossin@kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=masahiroy@kernel.org \
    --cc=mingo@redhat.com \
    --cc=morbo@google.com \
    --cc=nathan@kernel.org \
    --cc=ndesaulniers@google.com \
    --cc=nsc@kernel.org \
    --cc=ojeda@kernel.org \
    --cc=palmer@dabbelt.com \
    --cc=peterz@infradead.org \
    --cc=pjw@kernel.org \
    --cc=rdunlap@infradead.org \
    --cc=rust-for-linux@vger.kernel.org \
    --cc=tamird@kernel.org \
    --cc=tglx@kernel.org \
    --cc=tmgross@umich.edu \
    --cc=torvalds@linux-foundation.org \
    --cc=will@kernel.org \
    --cc=work@onurozkan.dev \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®