* [PATCH v1 0/2] kbuild: Give jobserver-exec recipes the jobserver
@ 2026-10-09 6:27 Kees Cook
2026-10-09 6:27 ` [PATCH v1 1/2] kbuild: Mark the kernel image gzip recipes as recursive Kees Cook
2026-10-09 6:27 ` [PATCH v1 2/2] kbuild: Mark the initcall ordering recipe " Kees Cook
0 siblings, 2 replies; 8+ messages in thread
From: Kees Cook @ 2026-10-09 6:27 UTC (permalink / raw)
To: Nathan Chancellor
Cc: Kees Cook, Nicolas Schier, Lorenzo Stoakes, Manuel Ebner,
Paul E . McKenney, Sami Tolvanen, linux-kbuild, llvm,
linux-kernel, linux-hardening
Hi!
Paul reported a build failure on CentOS Stream 9 with linux-next[1]: the
kernel builds, but scripts/jobserver-exec prints
WARNING: Unable to reopen jobserver read-side pipe: FileNotFoundError(2, 'No such file or directory')
and rcutorture's build check counts the "Error" in that line. CentOS's
make 4.3 carries a backport of the fix for GNU Make bug 58232, so
sub-makes, like the top-level make, give the jobserver pipe only to
recipes marked recursive, while MAKEFLAGS still names it. The new pigz
step (1/2) and the older Clang LTO initcall ordering step (2/2) run
jobserver-exec from recipes without "+", so pigz gets one thread and the
initcall script caps its children at the CPU count instead of the free
job slots. Mark both recipes recursive, as was done for rustc.
Thanks!
-Kees
[1] https://lore.kernel.org/all/0ddb5e5e-f801-444c-aa98-0cfe814a3026@paulmck-laptop/
Kees Cook (2):
kbuild: Mark the kernel image gzip recipes as recursive
kbuild: Mark the initcall ordering recipe as recursive
arch/alpha/boot/Makefile | 4 ++--
arch/arc/boot/Makefile | 2 +-
arch/arm64/boot/Makefile | 2 +-
arch/microblaze/boot/Makefile | 2 +-
arch/mips/boot/Makefile | 2 +-
arch/nios2/boot/Makefile | 2 +-
arch/parisc/boot/compressed/Makefile | 2 +-
arch/riscv/boot/Makefile | 2 +-
arch/s390/boot/Makefile | 2 +-
arch/sh/boot/Makefile | 2 +-
arch/sh/boot/compressed/Makefile | 2 +-
arch/sparc/boot/Makefile | 4 ++--
arch/x86/boot/compressed/Makefile | 2 +-
arch/xtensa/boot/Makefile | 2 +-
scripts/Makefile.vmlinux_o | 2 +-
15 files changed, 17 insertions(+), 17 deletions(-)
--
2.55.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v1 1/2] kbuild: Mark the kernel image gzip recipes as recursive
2026-10-09 6:27 [PATCH v1 0/2] kbuild: Give jobserver-exec recipes the jobserver Kees Cook
@ 2026-10-09 6:27 ` Kees Cook
2026-10-09 9:08 ` Nicolas Schier
` (2 more replies)
2026-10-09 6:27 ` [PATCH v1 2/2] kbuild: Mark the initcall ordering recipe " Kees Cook
1 sibling, 3 replies; 8+ messages in thread
From: Kees Cook @ 2026-10-09 6:27 UTC (permalink / raw)
To: Nathan Chancellor
Cc: Kees Cook, Nicolas Schier, Lorenzo Stoakes, Manuel Ebner,
linux-kbuild, Paul E. McKenney, linux-kernel, linux-hardening
Since commit 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if
available"), cmd_gzip runs pigz through scripts/jobserver-exec, which
takes the job slots make has free. GNU Make passes the jobserver pipe
only to recipes it treats as recursive, but leaves --jobserver-auth in
MAKEFLAGS for all of them. The make 4.3 in CentOS Stream 9 also hides
the pipe in sub-makes (a backport of the fix for GNU Make bug 58232), so
jobserver-exec cannot open it, runs pigz on one thread, and prints:
WARNING: Unable to reopen jobserver read-side pipe: FileNotFoundError(2, 'No such file or directory')
Mark the gzip recipes as recursive with "+", as commit ecab4115c44cc
("kbuild: mark `rustc` (and others) invocations as recursive") did for
rustc. As with those, "make -n" now runs them.
Tested ARCH=x86_64 and ARCH=arm64 defconfig with GNU Make 4.3 plus
CentOS Stream 9's make-4.3-cloexec.patch, GCC 16.2.0, and -j64: without
this change the warning appears and pigz runs with "-p 1"; with it there
is no warning, and pigz runs with "-p 54" and "-p 64".
Reported-by: Paul E. McKenney <paulmck@kernel.org>
Closes: https://lore.kernel.org/all/0ddb5e5e-f801-444c-aa98-0cfe814a3026@paulmck-laptop/
Fixes: 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if available")
Assisted-by: LLM
Signed-off-by: Kees Cook <kees@kernel.org>
---
arch/alpha/boot/Makefile | 4 ++--
arch/arc/boot/Makefile | 2 +-
arch/arm64/boot/Makefile | 2 +-
arch/microblaze/boot/Makefile | 2 +-
arch/mips/boot/Makefile | 2 +-
arch/nios2/boot/Makefile | 2 +-
arch/parisc/boot/compressed/Makefile | 2 +-
arch/riscv/boot/Makefile | 2 +-
arch/s390/boot/Makefile | 2 +-
arch/sh/boot/Makefile | 2 +-
arch/sh/boot/compressed/Makefile | 2 +-
arch/sparc/boot/Makefile | 4 ++--
arch/x86/boot/compressed/Makefile | 2 +-
arch/xtensa/boot/Makefile | 2 +-
14 files changed, 16 insertions(+), 16 deletions(-)
diff --git a/arch/alpha/boot/Makefile b/arch/alpha/boot/Makefile
index d8dba85e606c..b050753e9015 100644
--- a/arch/alpha/boot/Makefile
+++ b/arch/alpha/boot/Makefile
@@ -40,7 +40,7 @@ endif
# Compressed kernel image
$(obj)/vmlinux.gz: $(obj)/vmlinux FORCE
- $(call if_changed,gzip)
+ +$(call if_changed,gzip)
@echo ' Kernel $@ is ready'
$(obj)/main.o: $(obj)/ksize.h
@@ -85,7 +85,7 @@ $(obj)/vmlinux.nh: vmlinux $(OBJSTRIP) FORCE
$(call if_changed,objstrip)
$(obj)/vmlinux.nh.gz: $(obj)/vmlinux.nh FORCE
- $(call if_changed,gzip)
+ +$(call if_changed,gzip)
$(obj)/tools/lxboot: $(obj)/bootloader $(OBJSTRIP) FORCE
$(call if_changed,objstrip)
diff --git a/arch/arc/boot/Makefile b/arch/arc/boot/Makefile
index 5a8550124b73..8a44c3e7ff3e 100644
--- a/arch/arc/boot/Makefile
+++ b/arch/arc/boot/Makefile
@@ -23,7 +23,7 @@ $(obj)/vmlinux.bin: vmlinux FORCE
$(call if_changed,objcopy)
$(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE
- $(call if_changed,gzip)
+ +$(call if_changed,gzip)
$(obj)/vmlinux.bin.lzma: $(obj)/vmlinux.bin FORCE
$(call if_changed,lzma)
diff --git a/arch/arm64/boot/Makefile b/arch/arm64/boot/Makefile
index b5a08333bc57..d42d07667822 100644
--- a/arch/arm64/boot/Makefile
+++ b/arch/arm64/boot/Makefile
@@ -26,7 +26,7 @@ $(obj)/Image.bz2: $(obj)/Image FORCE
$(call if_changed,bzip2)
$(obj)/Image.gz: $(obj)/Image FORCE
- $(call if_changed,gzip)
+ +$(call if_changed,gzip)
$(obj)/Image.lz4: $(obj)/Image FORCE
$(call if_changed,lz4)
diff --git a/arch/microblaze/boot/Makefile b/arch/microblaze/boot/Makefile
index 2b42c370d574..e3df3a64808a 100644
--- a/arch/microblaze/boot/Makefile
+++ b/arch/microblaze/boot/Makefile
@@ -14,7 +14,7 @@ $(obj)/linux.bin.ub: $(obj)/linux.bin FORCE
$(call if_changed,uimage)
$(obj)/linux.bin.gz: $(obj)/linux.bin FORCE
- $(call if_changed,gzip)
+ +$(call if_changed,gzip)
quiet_cmd_strip = STRIP $< $@$2
cmd_strip = $(STRIP) -K microblaze_start -K _end -K __log_buf \
diff --git a/arch/mips/boot/Makefile b/arch/mips/boot/Makefile
index 8473c4671702..996bf41b78de 100644
--- a/arch/mips/boot/Makefile
+++ b/arch/mips/boot/Makefile
@@ -63,7 +63,7 @@ $(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE
$(call if_changed,bzip2)
$(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE
- $(call if_changed,gzip)
+ +$(call if_changed,gzip)
$(obj)/vmlinux.bin.lzma: $(obj)/vmlinux.bin FORCE
$(call if_changed,lzma)
diff --git a/arch/nios2/boot/Makefile b/arch/nios2/boot/Makefile
index 29c11a06b750..1381e6e428a8 100644
--- a/arch/nios2/boot/Makefile
+++ b/arch/nios2/boot/Makefile
@@ -18,7 +18,7 @@ $(obj)/vmlinux.bin: vmlinux FORCE
$(call if_changed,objcopy)
$(obj)/vmlinux.gz: $(obj)/vmlinux.bin FORCE
- $(call if_changed,gzip)
+ +$(call if_changed,gzip)
$(obj)/vmImage: $(obj)/vmlinux.gz FORCE
$(call if_changed,uimage)
diff --git a/arch/parisc/boot/compressed/Makefile b/arch/parisc/boot/compressed/Makefile
index 14eefb5ed5d1..a7fc7d424c00 100644
--- a/arch/parisc/boot/compressed/Makefile
+++ b/arch/parisc/boot/compressed/Makefile
@@ -55,7 +55,7 @@ suffix-$(CONFIG_KERNEL_LZO) := lzo
suffix-$(CONFIG_KERNEL_XZ) := xz
$(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE
- $(call if_changed,gzip)
+ +$(call if_changed,gzip)
$(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE
$(call if_changed,bzip2_with_size)
$(obj)/vmlinux.bin.lz4: $(obj)/vmlinux.bin FORCE
diff --git a/arch/riscv/boot/Makefile b/arch/riscv/boot/Makefile
index 4a2d75b7e42d..b80a077a8e8f 100644
--- a/arch/riscv/boot/Makefile
+++ b/arch/riscv/boot/Makefile
@@ -23,7 +23,7 @@ $(obj)/Image: vmlinux FORCE
$(call if_changed,objcopy)
$(obj)/Image.gz: $(obj)/Image FORCE
- $(call if_changed,gzip)
+ +$(call if_changed,gzip)
$(obj)/loader.o: $(src)/loader.S $(obj)/Image
diff --git a/arch/s390/boot/Makefile b/arch/s390/boot/Makefile
index 10b75e053a6f..e8e43ae80dd5 100644
--- a/arch/s390/boot/Makefile
+++ b/arch/s390/boot/Makefile
@@ -117,7 +117,7 @@ suffix-$(CONFIG_KERNEL_XZ) := .xz
suffix-$(CONFIG_KERNEL_ZSTD) := .zst
$(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE
- $(call if_changed,gzip)
+ +$(call if_changed,gzip)
$(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE
$(call if_changed,bzip2_with_size)
$(obj)/vmlinux.bin.lz4: $(obj)/vmlinux.bin FORCE
diff --git a/arch/sh/boot/Makefile b/arch/sh/boot/Makefile
index 1f5d2df3c7e0..cf63e319b786 100644
--- a/arch/sh/boot/Makefile
+++ b/arch/sh/boot/Makefile
@@ -66,7 +66,7 @@ $(obj)/vmlinux.bin: vmlinux FORCE
$(call if_changed,objcopy)
$(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE
- $(call if_changed,gzip)
+ +$(call if_changed,gzip)
$(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE
$(call if_changed,bzip2)
diff --git a/arch/sh/boot/compressed/Makefile b/arch/sh/boot/compressed/Makefile
index 58df491778b2..b85aa5f0d616 100644
--- a/arch/sh/boot/compressed/Makefile
+++ b/arch/sh/boot/compressed/Makefile
@@ -39,7 +39,7 @@ $(obj)/vmlinux.bin: vmlinux FORCE
$(call if_changed,objcopy)
$(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE
- $(call if_changed,gzip)
+ +$(call if_changed,gzip)
$(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE
$(call if_changed,bzip2_with_size)
$(obj)/vmlinux.bin.lzma: $(obj)/vmlinux.bin FORCE
diff --git a/arch/sparc/boot/Makefile b/arch/sparc/boot/Makefile
index 339c42d35089..b53cf596611e 100644
--- a/arch/sparc/boot/Makefile
+++ b/arch/sparc/boot/Makefile
@@ -23,7 +23,7 @@ ifeq ($(CONFIG_SPARC64),y)
# Actual linking
$(obj)/zImage: $(obj)/image FORCE
- $(call if_changed,gzip)
+ +$(call if_changed,gzip)
@$(kecho) 'Kernel: $@ is ready' '(#'$(or $(KBUILD_BUILD_VERSION),`cat .version`)')'
$(obj)/vmlinux.aout: vmlinux FORCE
@@ -45,7 +45,7 @@ $(obj)/image.bin: $(obj)/image FORCE
$(call if_changed,objcopy)
$(obj)/image.gz: $(obj)/image.bin FORCE
- $(call if_changed,gzip)
+ +$(call if_changed,gzip)
UIMAGE_LOADADDR = $(CONFIG_UBOOT_LOAD_ADDR)
UIMAGE_ENTRYADDR = $(CONFIG_UBOOT_ENTRY_ADDR)
diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile
index 6ec5e031db1d..680ad803cba7 100644
--- a/arch/x86/boot/compressed/Makefile
+++ b/arch/x86/boot/compressed/Makefile
@@ -133,7 +133,7 @@ vmlinux.bin.all-y := $(obj)/vmlinux.bin
vmlinux.bin.all-$(CONFIG_X86_NEED_RELOCS) += $(obj)/vmlinux.relocs
$(obj)/vmlinux.bin.gz: $(vmlinux.bin.all-y) FORCE
- $(call if_changed,gzip)
+ +$(call if_changed,gzip)
$(obj)/vmlinux.bin.bz2: $(vmlinux.bin.all-y) FORCE
$(call if_changed,bzip2_with_size)
$(obj)/vmlinux.bin.lzma: $(vmlinux.bin.all-y) FORCE
diff --git a/arch/xtensa/boot/Makefile b/arch/xtensa/boot/Makefile
index d8b0fadf429a..934f9015b15f 100644
--- a/arch/xtensa/boot/Makefile
+++ b/arch/xtensa/boot/Makefile
@@ -36,7 +36,7 @@ $(obj)/vmlinux.bin: vmlinux FORCE
$(call if_changed,objcopy)
$(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE
- $(call if_changed,gzip)
+ +$(call if_changed,gzip)
boot-elf: $(obj)/vmlinux.bin
boot-redboot: $(obj)/vmlinux.bin.gz
--
2.55.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v1 2/2] kbuild: Mark the initcall ordering recipe as recursive
2026-10-09 6:27 [PATCH v1 0/2] kbuild: Give jobserver-exec recipes the jobserver Kees Cook
2026-10-09 6:27 ` [PATCH v1 1/2] kbuild: Mark the kernel image gzip recipes as recursive Kees Cook
@ 2026-10-09 6:27 ` Kees Cook
2026-10-09 15:20 ` Nicolas Schier
1 sibling, 1 reply; 8+ messages in thread
From: Kees Cook @ 2026-10-09 6:27 UTC (permalink / raw)
To: Nathan Chancellor
Cc: Kees Cook, Nicolas Schier, Sami Tolvanen, linux-kbuild, llvm,
linux-kernel, linux-hardening
generate_initcall_order.pl, which orders initcalls for Clang LTO, forks
a child per input file, up to the job count scripts/jobserver-exec
passes it in PARALLELISM. It used to run from link-vmlinux.sh, which
the top-level Makefile invokes as a recursive recipe, but
commit 5d45950dfbb15 ("kbuild: move vmlinux.o link to
scripts/Makefile.vmlinux_o") moved it into a recipe without "+".
GNU Make passes the jobserver pipe only to recipes it treats as
recursive, and the make 4.3 in CentOS Stream 9 also hides it in
sub-makes (a backport of the fix for GNU Make bug 58232), so
jobserver-exec cannot open it and prints:
WARNING: Unable to reopen jobserver read-side pipe: FileNotFoundError(2, 'No such file or directory')
PARALLELISM is then left unset, and the script caps its children at the
number of online CPUs instead of the free job slots.
Mark the recipe as recursive again.
Tested ARCH=x86_64 defconfig with CONFIG_LTO_CLANG_THIN=y, LLVM=1
(Clang 24.0.0git), and GNU Make 4.3 plus CentOS Stream 9's
make-4.3-cloexec.patch.
Fixes: 5d45950dfbb15 ("kbuild: move vmlinux.o link to scripts/Makefile.vmlinux_o")
Assisted-by: LLM
Signed-off-by: Kees Cook <kees@kernel.org>
---
scripts/Makefile.vmlinux_o | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/Makefile.vmlinux_o b/scripts/Makefile.vmlinux_o
index 24a3a4fd271c..0b37bd5dc3a4 100644
--- a/scripts/Makefile.vmlinux_o
+++ b/scripts/Makefile.vmlinux_o
@@ -19,7 +19,7 @@ quiet_cmd_gen_initcalls_lds = GEN $@
.tmp_initcalls.lds: $(srctree)/scripts/generate_initcall_order.pl \
vmlinux.a $(KBUILD_VMLINUX_LIBS) FORCE
- $(call if_changed,gen_initcalls_lds)
+ +$(call if_changed,gen_initcalls_lds)
targets := .tmp_initcalls.lds
--
2.55.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v1 1/2] kbuild: Mark the kernel image gzip recipes as recursive
2026-10-09 6:27 ` [PATCH v1 1/2] kbuild: Mark the kernel image gzip recipes as recursive Kees Cook
@ 2026-10-09 9:08 ` Nicolas Schier
2026-10-09 12:05 ` Lorenzo Stoakes (ARM)
2026-10-09 15:51 ` Paul E. McKenney
2 siblings, 0 replies; 8+ messages in thread
From: Nicolas Schier @ 2026-10-09 9:08 UTC (permalink / raw)
To: Kees Cook
Cc: Nathan Chancellor, Nicolas Schier, Lorenzo Stoakes, Manuel Ebner,
linux-kbuild, Paul E. McKenney, linux-kernel, linux-hardening
> Since commit 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if
> available"), cmd_gzip runs pigz through scripts/jobserver-exec, which
> takes the job slots make has free. GNU Make passes the jobserver pipe
> only to recipes it treats as recursive, but leaves --jobserver-auth in
> MAKEFLAGS for all of them. The make 4.3 in CentOS Stream 9 also hides
> the pipe in sub-makes (a backport of the fix for GNU Make bug 58232), so
> jobserver-exec cannot open it, runs pigz on one thread, and prints:
>
> WARNING: Unable to reopen jobserver read-side pipe: FileNotFoundError(2, 'No such file or directory')
>
> Mark the gzip recipes as recursive with "+", as commit ecab4115c44cc
> ("kbuild: mark `rustc` (and others) invocations as recursive") did for
> rustc. As with those, "make -n" now runs them.
>
> Tested ARCH=x86_64 and ARCH=arm64 defconfig with GNU Make 4.3 plus
> CentOS Stream 9's make-4.3-cloexec.patch, GCC 16.2.0, and -j64: without
> this change the warning appears and pigz runs with "-p 1"; with it there
> is no warning, and pigz runs with "-p 54" and "-p 64".
>
> Reported-by: Paul E. McKenney <paulmck@kernel.org>
> Closes: https://lore.kernel.org/all/0ddb5e5e-f801-444c-aa98-0cfe814a3026@paulmck-laptop/
> Fixes: 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if available")
> Assisted-by: LLM
> Signed-off-by: Kees Cook <kees@kernel.org>
> ---
> arch/alpha/boot/Makefile | 4 ++--
> arch/arc/boot/Makefile | 2 +-
> arch/arm64/boot/Makefile | 2 +-
> arch/microblaze/boot/Makefile | 2 +-
> arch/mips/boot/Makefile | 2 +-
> arch/nios2/boot/Makefile | 2 +-
> arch/parisc/boot/compressed/Makefile | 2 +-
> arch/riscv/boot/Makefile | 2 +-
> arch/s390/boot/Makefile | 2 +-
> arch/sh/boot/Makefile | 2 +-
> arch/sh/boot/compressed/Makefile | 2 +-
> arch/sparc/boot/Makefile | 4 ++--
> arch/x86/boot/compressed/Makefile | 2 +-
> arch/xtensa/boot/Makefile | 2 +-
> 14 files changed, 16 insertions(+), 16 deletions(-)
Reviewed-by: Nicolas Schier <n.schier@fritz.com>
Before it gets lost, we should also update the documentation bit that
references $(if_changed,gzip):
diff --git a/Documentation/kbuild/makefiles.rst b/Documentation/kbuild/makefiles.rst
index 7521cae7d56f..2760717ab7b7 100644
--- a/Documentation/kbuild/makefiles.rst
+++ b/Documentation/kbuild/makefiles.rst
@@ -1452,13 +1452,14 @@ objcopy
OBJCOPYFLAGS_$@ may be used to set additional options.
gzip
- Compress target. Use maximum compression to compress target.
+ Compress target. Use maximum compression to compress target. Add '+' prefix
+ to allow parallel gzip compression via pigz (if available).
Example::
#arch/x86/boot/compressed/Makefile
$(obj)/vmlinux.bin.gz: $(vmlinux.bin.all-y) FORCE
- $(call if_changed,gzip)
+ +$(call if_changed,gzip)
dtc
Create flattened device tree blob object suitable for linking
--
Nicolas
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v1 1/2] kbuild: Mark the kernel image gzip recipes as recursive
2026-10-09 6:27 ` [PATCH v1 1/2] kbuild: Mark the kernel image gzip recipes as recursive Kees Cook
2026-10-09 9:08 ` Nicolas Schier
@ 2026-10-09 12:05 ` Lorenzo Stoakes (ARM)
2026-10-09 15:52 ` Paul E. McKenney
2026-10-09 15:51 ` Paul E. McKenney
2 siblings, 1 reply; 8+ messages in thread
From: Lorenzo Stoakes (ARM) @ 2026-10-09 12:05 UTC (permalink / raw)
To: Kees Cook
Cc: Nathan Chancellor, Nicolas Schier, Manuel Ebner, linux-kbuild,
Paul E. McKenney, linux-kernel, linux-hardening
On Thu, Oct 08, 2026 at 11:27:30PM -0700, Kees Cook wrote:
> Since commit 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if
> available"), cmd_gzip runs pigz through scripts/jobserver-exec, which
> takes the job slots make has free. GNU Make passes the jobserver pipe
> only to recipes it treats as recursive, but leaves --jobserver-auth in
> MAKEFLAGS for all of them. The make 4.3 in CentOS Stream 9 also hides
> the pipe in sub-makes (a backport of the fix for GNU Make bug 58232), so
> jobserver-exec cannot open it, runs pigz on one thread, and prints:
>
> WARNING: Unable to reopen jobserver read-side pipe: FileNotFoundError(2, 'No such file or directory')
>
> Mark the gzip recipes as recursive with "+", as commit ecab4115c44cc
> ("kbuild: mark `rustc` (and others) invocations as recursive") did for
> rustc. As with those, "make -n" now runs them.
>
> Tested ARCH=x86_64 and ARCH=arm64 defconfig with GNU Make 4.3 plus
> CentOS Stream 9's make-4.3-cloexec.patch, GCC 16.2.0, and -j64: without
> this change the warning appears and pigz runs with "-p 1"; with it there
> is no warning, and pigz runs with "-p 54" and "-p 64".
>
> Reported-by: Paul E. McKenney <paulmck@kernel.org>
> Closes: https://lore.kernel.org/all/0ddb5e5e-f801-444c-aa98-0cfe814a3026@paulmck-laptop/
> Fixes: 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if available")
> Assisted-by: LLM
> Signed-off-by: Kees Cook <kees@kernel.org>
Nice find!
Acked-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Sorry for breaking your build Paul ;)
--
Cheers, Lorenzo
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v1 2/2] kbuild: Mark the initcall ordering recipe as recursive
2026-10-09 6:27 ` [PATCH v1 2/2] kbuild: Mark the initcall ordering recipe " Kees Cook
@ 2026-10-09 15:20 ` Nicolas Schier
0 siblings, 0 replies; 8+ messages in thread
From: Nicolas Schier @ 2026-10-09 15:20 UTC (permalink / raw)
To: Kees Cook
Cc: Nathan Chancellor, Nicolas Schier, Sami Tolvanen, linux-kbuild,
llvm, linux-kernel, linux-hardening
> generate_initcall_order.pl, which orders initcalls for Clang LTO, forks
> a child per input file, up to the job count scripts/jobserver-exec
> passes it in PARALLELISM. It used to run from link-vmlinux.sh, which
> the top-level Makefile invokes as a recursive recipe, but
> commit 5d45950dfbb15 ("kbuild: move vmlinux.o link to
> scripts/Makefile.vmlinux_o") moved it into a recipe without "+".
>
> GNU Make passes the jobserver pipe only to recipes it treats as
> recursive, and the make 4.3 in CentOS Stream 9 also hides it in
> sub-makes (a backport of the fix for GNU Make bug 58232), so
> jobserver-exec cannot open it and prints:
>
> WARNING: Unable to reopen jobserver read-side pipe: FileNotFoundError(2, 'No such file or directory')
>
> PARALLELISM is then left unset, and the script caps its children at the
> number of online CPUs instead of the free job slots.
>
> Mark the recipe as recursive again.
>
> Tested ARCH=x86_64 defconfig with CONFIG_LTO_CLANG_THIN=y, LLVM=1
> (Clang 24.0.0git), and GNU Make 4.3 plus CentOS Stream 9's
> make-4.3-cloexec.patch.
>
> Fixes: 5d45950dfbb15 ("kbuild: move vmlinux.o link to scripts/Makefile.vmlinux_o")
> Assisted-by: LLM
> Signed-off-by: Kees Cook <kees@kernel.org>
> ---
> scripts/Makefile.vmlinux_o | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/scripts/Makefile.vmlinux_o b/scripts/Makefile.vmlinux_o
> index 24a3a4fd271c..0b37bd5dc3a4 100644
> --- a/scripts/Makefile.vmlinux_o
> +++ b/scripts/Makefile.vmlinux_o
> @@ -19,7 +19,7 @@ quiet_cmd_gen_initcalls_lds = GEN $@
>
> .tmp_initcalls.lds: $(srctree)/scripts/generate_initcall_order.pl \
> vmlinux.a $(KBUILD_VMLINUX_LIBS) FORCE
> - $(call if_changed,gen_initcalls_lds)
> + +$(call if_changed,gen_initcalls_lds)
b4 hinted me on [1], where a similar fix was rejected; but as we know
have more arguments to generate_initcall_order.pl
($(KBUILD_VMLINUX_LIBS)), the situation is slightly different.
Thanks!
Reviewed-by: Nicolas Schier <n.schier@fritz.com>
[1]: https://lore.kernel.org/all/CAK7LNARCM=rUm8mA8GRQ7ufeyfneGf4OEvHmESKt=zuxs2KrHw@mail.gmail.com/
--
Nicolas
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v1 1/2] kbuild: Mark the kernel image gzip recipes as recursive
2026-10-09 6:27 ` [PATCH v1 1/2] kbuild: Mark the kernel image gzip recipes as recursive Kees Cook
2026-10-09 9:08 ` Nicolas Schier
2026-10-09 12:05 ` Lorenzo Stoakes (ARM)
@ 2026-10-09 15:51 ` Paul E. McKenney
2 siblings, 0 replies; 8+ messages in thread
From: Paul E. McKenney @ 2026-10-09 15:51 UTC (permalink / raw)
To: Kees Cook
Cc: Nathan Chancellor, Nicolas Schier, Lorenzo Stoakes, Manuel Ebner,
linux-kbuild, linux-kernel, linux-hardening
On Thu, Oct 08, 2026 at 11:27:30PM -0700, Kees Cook wrote:
> Since commit 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if
> available"), cmd_gzip runs pigz through scripts/jobserver-exec, which
> takes the job slots make has free. GNU Make passes the jobserver pipe
> only to recipes it treats as recursive, but leaves --jobserver-auth in
> MAKEFLAGS for all of them. The make 4.3 in CentOS Stream 9 also hides
> the pipe in sub-makes (a backport of the fix for GNU Make bug 58232), so
> jobserver-exec cannot open it, runs pigz on one thread, and prints:
>
> WARNING: Unable to reopen jobserver read-side pipe: FileNotFoundError(2, 'No such file or directory')
>
> Mark the gzip recipes as recursive with "+", as commit ecab4115c44cc
> ("kbuild: mark `rustc` (and others) invocations as recursive") did for
> rustc. As with those, "make -n" now runs them.
>
> Tested ARCH=x86_64 and ARCH=arm64 defconfig with GNU Make 4.3 plus
> CentOS Stream 9's make-4.3-cloexec.patch, GCC 16.2.0, and -j64: without
> this change the warning appears and pigz runs with "-p 1"; with it there
> is no warning, and pigz runs with "-p 54" and "-p 64".
Thank you, Kees!
My initial test of this patch (using rcutorture) still showed the failure.
I am running GCC 11.5.0 and without any patches to "make". I take it
that I need to find and apply CentOS Stream 9's make-4.3-cloexec.patch?
But if I use "make -j" instead of rcutorture, this patch does work fine.
Even with "make -j632".
I put rcutorture in the background using nohup and full redirection,
but "nohup make -j632 > /tmp/Make.out 2>&1 < /dev/null &" works just
fine. And running rcutorture in the foreground still fails.
Very strange.
So I guess I could give a Tested-by for builds using straight-up make,
but not from rcutorture. ;-)
Thanx, Paul
> Reported-by: Paul E. McKenney <paulmck@kernel.org>
> Closes: https://lore.kernel.org/all/0ddb5e5e-f801-444c-aa98-0cfe814a3026@paulmck-laptop/
> Fixes: 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if available")
> Assisted-by: LLM
> Signed-off-by: Kees Cook <kees@kernel.org>
> ---
> arch/alpha/boot/Makefile | 4 ++--
> arch/arc/boot/Makefile | 2 +-
> arch/arm64/boot/Makefile | 2 +-
> arch/microblaze/boot/Makefile | 2 +-
> arch/mips/boot/Makefile | 2 +-
> arch/nios2/boot/Makefile | 2 +-
> arch/parisc/boot/compressed/Makefile | 2 +-
> arch/riscv/boot/Makefile | 2 +-
> arch/s390/boot/Makefile | 2 +-
> arch/sh/boot/Makefile | 2 +-
> arch/sh/boot/compressed/Makefile | 2 +-
> arch/sparc/boot/Makefile | 4 ++--
> arch/x86/boot/compressed/Makefile | 2 +-
> arch/xtensa/boot/Makefile | 2 +-
> 14 files changed, 16 insertions(+), 16 deletions(-)
>
> diff --git a/arch/alpha/boot/Makefile b/arch/alpha/boot/Makefile
> index d8dba85e606c..b050753e9015 100644
> --- a/arch/alpha/boot/Makefile
> +++ b/arch/alpha/boot/Makefile
> @@ -40,7 +40,7 @@ endif
>
> # Compressed kernel image
> $(obj)/vmlinux.gz: $(obj)/vmlinux FORCE
> - $(call if_changed,gzip)
> + +$(call if_changed,gzip)
> @echo ' Kernel $@ is ready'
>
> $(obj)/main.o: $(obj)/ksize.h
> @@ -85,7 +85,7 @@ $(obj)/vmlinux.nh: vmlinux $(OBJSTRIP) FORCE
> $(call if_changed,objstrip)
>
> $(obj)/vmlinux.nh.gz: $(obj)/vmlinux.nh FORCE
> - $(call if_changed,gzip)
> + +$(call if_changed,gzip)
>
> $(obj)/tools/lxboot: $(obj)/bootloader $(OBJSTRIP) FORCE
> $(call if_changed,objstrip)
> diff --git a/arch/arc/boot/Makefile b/arch/arc/boot/Makefile
> index 5a8550124b73..8a44c3e7ff3e 100644
> --- a/arch/arc/boot/Makefile
> +++ b/arch/arc/boot/Makefile
> @@ -23,7 +23,7 @@ $(obj)/vmlinux.bin: vmlinux FORCE
> $(call if_changed,objcopy)
>
> $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE
> - $(call if_changed,gzip)
> + +$(call if_changed,gzip)
>
> $(obj)/vmlinux.bin.lzma: $(obj)/vmlinux.bin FORCE
> $(call if_changed,lzma)
> diff --git a/arch/arm64/boot/Makefile b/arch/arm64/boot/Makefile
> index b5a08333bc57..d42d07667822 100644
> --- a/arch/arm64/boot/Makefile
> +++ b/arch/arm64/boot/Makefile
> @@ -26,7 +26,7 @@ $(obj)/Image.bz2: $(obj)/Image FORCE
> $(call if_changed,bzip2)
>
> $(obj)/Image.gz: $(obj)/Image FORCE
> - $(call if_changed,gzip)
> + +$(call if_changed,gzip)
>
> $(obj)/Image.lz4: $(obj)/Image FORCE
> $(call if_changed,lz4)
> diff --git a/arch/microblaze/boot/Makefile b/arch/microblaze/boot/Makefile
> index 2b42c370d574..e3df3a64808a 100644
> --- a/arch/microblaze/boot/Makefile
> +++ b/arch/microblaze/boot/Makefile
> @@ -14,7 +14,7 @@ $(obj)/linux.bin.ub: $(obj)/linux.bin FORCE
> $(call if_changed,uimage)
>
> $(obj)/linux.bin.gz: $(obj)/linux.bin FORCE
> - $(call if_changed,gzip)
> + +$(call if_changed,gzip)
>
> quiet_cmd_strip = STRIP $< $@$2
> cmd_strip = $(STRIP) -K microblaze_start -K _end -K __log_buf \
> diff --git a/arch/mips/boot/Makefile b/arch/mips/boot/Makefile
> index 8473c4671702..996bf41b78de 100644
> --- a/arch/mips/boot/Makefile
> +++ b/arch/mips/boot/Makefile
> @@ -63,7 +63,7 @@ $(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE
> $(call if_changed,bzip2)
>
> $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE
> - $(call if_changed,gzip)
> + +$(call if_changed,gzip)
>
> $(obj)/vmlinux.bin.lzma: $(obj)/vmlinux.bin FORCE
> $(call if_changed,lzma)
> diff --git a/arch/nios2/boot/Makefile b/arch/nios2/boot/Makefile
> index 29c11a06b750..1381e6e428a8 100644
> --- a/arch/nios2/boot/Makefile
> +++ b/arch/nios2/boot/Makefile
> @@ -18,7 +18,7 @@ $(obj)/vmlinux.bin: vmlinux FORCE
> $(call if_changed,objcopy)
>
> $(obj)/vmlinux.gz: $(obj)/vmlinux.bin FORCE
> - $(call if_changed,gzip)
> + +$(call if_changed,gzip)
>
> $(obj)/vmImage: $(obj)/vmlinux.gz FORCE
> $(call if_changed,uimage)
> diff --git a/arch/parisc/boot/compressed/Makefile b/arch/parisc/boot/compressed/Makefile
> index 14eefb5ed5d1..a7fc7d424c00 100644
> --- a/arch/parisc/boot/compressed/Makefile
> +++ b/arch/parisc/boot/compressed/Makefile
> @@ -55,7 +55,7 @@ suffix-$(CONFIG_KERNEL_LZO) := lzo
> suffix-$(CONFIG_KERNEL_XZ) := xz
>
> $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE
> - $(call if_changed,gzip)
> + +$(call if_changed,gzip)
> $(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE
> $(call if_changed,bzip2_with_size)
> $(obj)/vmlinux.bin.lz4: $(obj)/vmlinux.bin FORCE
> diff --git a/arch/riscv/boot/Makefile b/arch/riscv/boot/Makefile
> index 4a2d75b7e42d..b80a077a8e8f 100644
> --- a/arch/riscv/boot/Makefile
> +++ b/arch/riscv/boot/Makefile
> @@ -23,7 +23,7 @@ $(obj)/Image: vmlinux FORCE
> $(call if_changed,objcopy)
>
> $(obj)/Image.gz: $(obj)/Image FORCE
> - $(call if_changed,gzip)
> + +$(call if_changed,gzip)
>
> $(obj)/loader.o: $(src)/loader.S $(obj)/Image
>
> diff --git a/arch/s390/boot/Makefile b/arch/s390/boot/Makefile
> index 10b75e053a6f..e8e43ae80dd5 100644
> --- a/arch/s390/boot/Makefile
> +++ b/arch/s390/boot/Makefile
> @@ -117,7 +117,7 @@ suffix-$(CONFIG_KERNEL_XZ) := .xz
> suffix-$(CONFIG_KERNEL_ZSTD) := .zst
>
> $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE
> - $(call if_changed,gzip)
> + +$(call if_changed,gzip)
> $(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE
> $(call if_changed,bzip2_with_size)
> $(obj)/vmlinux.bin.lz4: $(obj)/vmlinux.bin FORCE
> diff --git a/arch/sh/boot/Makefile b/arch/sh/boot/Makefile
> index 1f5d2df3c7e0..cf63e319b786 100644
> --- a/arch/sh/boot/Makefile
> +++ b/arch/sh/boot/Makefile
> @@ -66,7 +66,7 @@ $(obj)/vmlinux.bin: vmlinux FORCE
> $(call if_changed,objcopy)
>
> $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE
> - $(call if_changed,gzip)
> + +$(call if_changed,gzip)
>
> $(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE
> $(call if_changed,bzip2)
> diff --git a/arch/sh/boot/compressed/Makefile b/arch/sh/boot/compressed/Makefile
> index 58df491778b2..b85aa5f0d616 100644
> --- a/arch/sh/boot/compressed/Makefile
> +++ b/arch/sh/boot/compressed/Makefile
> @@ -39,7 +39,7 @@ $(obj)/vmlinux.bin: vmlinux FORCE
> $(call if_changed,objcopy)
>
> $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE
> - $(call if_changed,gzip)
> + +$(call if_changed,gzip)
> $(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE
> $(call if_changed,bzip2_with_size)
> $(obj)/vmlinux.bin.lzma: $(obj)/vmlinux.bin FORCE
> diff --git a/arch/sparc/boot/Makefile b/arch/sparc/boot/Makefile
> index 339c42d35089..b53cf596611e 100644
> --- a/arch/sparc/boot/Makefile
> +++ b/arch/sparc/boot/Makefile
> @@ -23,7 +23,7 @@ ifeq ($(CONFIG_SPARC64),y)
> # Actual linking
>
> $(obj)/zImage: $(obj)/image FORCE
> - $(call if_changed,gzip)
> + +$(call if_changed,gzip)
> @$(kecho) 'Kernel: $@ is ready' '(#'$(or $(KBUILD_BUILD_VERSION),`cat .version`)')'
>
> $(obj)/vmlinux.aout: vmlinux FORCE
> @@ -45,7 +45,7 @@ $(obj)/image.bin: $(obj)/image FORCE
> $(call if_changed,objcopy)
>
> $(obj)/image.gz: $(obj)/image.bin FORCE
> - $(call if_changed,gzip)
> + +$(call if_changed,gzip)
>
> UIMAGE_LOADADDR = $(CONFIG_UBOOT_LOAD_ADDR)
> UIMAGE_ENTRYADDR = $(CONFIG_UBOOT_ENTRY_ADDR)
> diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile
> index 6ec5e031db1d..680ad803cba7 100644
> --- a/arch/x86/boot/compressed/Makefile
> +++ b/arch/x86/boot/compressed/Makefile
> @@ -133,7 +133,7 @@ vmlinux.bin.all-y := $(obj)/vmlinux.bin
> vmlinux.bin.all-$(CONFIG_X86_NEED_RELOCS) += $(obj)/vmlinux.relocs
>
> $(obj)/vmlinux.bin.gz: $(vmlinux.bin.all-y) FORCE
> - $(call if_changed,gzip)
> + +$(call if_changed,gzip)
> $(obj)/vmlinux.bin.bz2: $(vmlinux.bin.all-y) FORCE
> $(call if_changed,bzip2_with_size)
> $(obj)/vmlinux.bin.lzma: $(vmlinux.bin.all-y) FORCE
> diff --git a/arch/xtensa/boot/Makefile b/arch/xtensa/boot/Makefile
> index d8b0fadf429a..934f9015b15f 100644
> --- a/arch/xtensa/boot/Makefile
> +++ b/arch/xtensa/boot/Makefile
> @@ -36,7 +36,7 @@ $(obj)/vmlinux.bin: vmlinux FORCE
> $(call if_changed,objcopy)
>
> $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE
> - $(call if_changed,gzip)
> + +$(call if_changed,gzip)
>
> boot-elf: $(obj)/vmlinux.bin
> boot-redboot: $(obj)/vmlinux.bin.gz
> --
> 2.55.0
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v1 1/2] kbuild: Mark the kernel image gzip recipes as recursive
2026-10-09 12:05 ` Lorenzo Stoakes (ARM)
@ 2026-10-09 15:52 ` Paul E. McKenney
0 siblings, 0 replies; 8+ messages in thread
From: Paul E. McKenney @ 2026-10-09 15:52 UTC (permalink / raw)
To: Lorenzo Stoakes (ARM)
Cc: Kees Cook, Nathan Chancellor, Nicolas Schier, Manuel Ebner,
linux-kbuild, linux-kernel, linux-hardening
On Fri, Oct 09, 2026 at 01:05:00PM +0100, Lorenzo Stoakes (ARM) wrote:
> On Thu, Oct 08, 2026 at 11:27:30PM -0700, Kees Cook wrote:
> > Since commit 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if
> > available"), cmd_gzip runs pigz through scripts/jobserver-exec, which
> > takes the job slots make has free. GNU Make passes the jobserver pipe
> > only to recipes it treats as recursive, but leaves --jobserver-auth in
> > MAKEFLAGS for all of them. The make 4.3 in CentOS Stream 9 also hides
> > the pipe in sub-makes (a backport of the fix for GNU Make bug 58232), so
> > jobserver-exec cannot open it, runs pigz on one thread, and prints:
> >
> > WARNING: Unable to reopen jobserver read-side pipe: FileNotFoundError(2, 'No such file or directory')
> >
> > Mark the gzip recipes as recursive with "+", as commit ecab4115c44cc
> > ("kbuild: mark `rustc` (and others) invocations as recursive") did for
> > rustc. As with those, "make -n" now runs them.
> >
> > Tested ARCH=x86_64 and ARCH=arm64 defconfig with GNU Make 4.3 plus
> > CentOS Stream 9's make-4.3-cloexec.patch, GCC 16.2.0, and -j64: without
> > this change the warning appears and pigz runs with "-p 1"; with it there
> > is no warning, and pigz runs with "-p 54" and "-p 64".
> >
> > Reported-by: Paul E. McKenney <paulmck@kernel.org>
> > Closes: https://lore.kernel.org/all/0ddb5e5e-f801-444c-aa98-0cfe814a3026@paulmck-laptop/
> > Fixes: 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if available")
> > Assisted-by: LLM
> > Signed-off-by: Kees Cook <kees@kernel.org>
>
> Nice find!
>
> Acked-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
>
> Sorry for breaking your build Paul ;)
Hey, this was the fastest bisection that I have ever done by far!
Normally it takes hours per step, but this was about 40 seconds per
step. ;-)
Thanx, Paul
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2026-10-09 15:52 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-10-09 6:27 [PATCH v1 0/2] kbuild: Give jobserver-exec recipes the jobserver Kees Cook
2026-10-09 6:27 ` [PATCH v1 1/2] kbuild: Mark the kernel image gzip recipes as recursive Kees Cook
2026-10-09 9:08 ` Nicolas Schier
2026-10-09 12:05 ` Lorenzo Stoakes (ARM)
2026-10-09 15:52 ` Paul E. McKenney
2026-10-09 15:51 ` Paul E. McKenney
2026-10-09 6:27 ` [PATCH v1 2/2] kbuild: Mark the initcall ordering recipe " Kees Cook
2026-10-09 15:20 ` Nicolas Schier
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®