* [PATCH v1 0/2] kbuild: Give jobserver-exec recipes the jobserver @ 2026-10-09 6:27 Kees Cook 2026-10-09 6:27 ` [PATCH v1 1/2] kbuild: Mark the kernel image gzip recipes as recursive Kees Cook 2026-10-09 6:27 ` [PATCH v1 2/2] kbuild: Mark the initcall ordering recipe " Kees Cook 0 siblings, 2 replies; 8+ messages in thread From: Kees Cook @ 2026-10-09 6:27 UTC (permalink / raw) To: Nathan Chancellor Cc: Kees Cook, Nicolas Schier, Lorenzo Stoakes, Manuel Ebner, Paul E . McKenney, Sami Tolvanen, linux-kbuild, llvm, linux-kernel, linux-hardening Hi! Paul reported a build failure on CentOS Stream 9 with linux-next[1]: the kernel builds, but scripts/jobserver-exec prints WARNING: Unable to reopen jobserver read-side pipe: FileNotFoundError(2, 'No such file or directory') and rcutorture's build check counts the "Error" in that line. CentOS's make 4.3 carries a backport of the fix for GNU Make bug 58232, so sub-makes, like the top-level make, give the jobserver pipe only to recipes marked recursive, while MAKEFLAGS still names it. The new pigz step (1/2) and the older Clang LTO initcall ordering step (2/2) run jobserver-exec from recipes without "+", so pigz gets one thread and the initcall script caps its children at the CPU count instead of the free job slots. Mark both recipes recursive, as was done for rustc. Thanks! -Kees [1] https://lore.kernel.org/all/0ddb5e5e-f801-444c-aa98-0cfe814a3026@paulmck-laptop/ Kees Cook (2): kbuild: Mark the kernel image gzip recipes as recursive kbuild: Mark the initcall ordering recipe as recursive arch/alpha/boot/Makefile | 4 ++-- arch/arc/boot/Makefile | 2 +- arch/arm64/boot/Makefile | 2 +- arch/microblaze/boot/Makefile | 2 +- arch/mips/boot/Makefile | 2 +- arch/nios2/boot/Makefile | 2 +- arch/parisc/boot/compressed/Makefile | 2 +- arch/riscv/boot/Makefile | 2 +- arch/s390/boot/Makefile | 2 +- arch/sh/boot/Makefile | 2 +- arch/sh/boot/compressed/Makefile | 2 +- arch/sparc/boot/Makefile | 4 ++-- arch/x86/boot/compressed/Makefile | 2 +- arch/xtensa/boot/Makefile | 2 +- scripts/Makefile.vmlinux_o | 2 +- 15 files changed, 17 insertions(+), 17 deletions(-) -- 2.55.0 ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v1 1/2] kbuild: Mark the kernel image gzip recipes as recursive 2026-10-09 6:27 [PATCH v1 0/2] kbuild: Give jobserver-exec recipes the jobserver Kees Cook @ 2026-10-09 6:27 ` Kees Cook 2026-10-09 9:08 ` Nicolas Schier ` (2 more replies) 2026-10-09 6:27 ` [PATCH v1 2/2] kbuild: Mark the initcall ordering recipe " Kees Cook 1 sibling, 3 replies; 8+ messages in thread From: Kees Cook @ 2026-10-09 6:27 UTC (permalink / raw) To: Nathan Chancellor Cc: Kees Cook, Nicolas Schier, Lorenzo Stoakes, Manuel Ebner, linux-kbuild, Paul E. McKenney, linux-kernel, linux-hardening Since commit 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if available"), cmd_gzip runs pigz through scripts/jobserver-exec, which takes the job slots make has free. GNU Make passes the jobserver pipe only to recipes it treats as recursive, but leaves --jobserver-auth in MAKEFLAGS for all of them. The make 4.3 in CentOS Stream 9 also hides the pipe in sub-makes (a backport of the fix for GNU Make bug 58232), so jobserver-exec cannot open it, runs pigz on one thread, and prints: WARNING: Unable to reopen jobserver read-side pipe: FileNotFoundError(2, 'No such file or directory') Mark the gzip recipes as recursive with "+", as commit ecab4115c44cc ("kbuild: mark `rustc` (and others) invocations as recursive") did for rustc. As with those, "make -n" now runs them. Tested ARCH=x86_64 and ARCH=arm64 defconfig with GNU Make 4.3 plus CentOS Stream 9's make-4.3-cloexec.patch, GCC 16.2.0, and -j64: without this change the warning appears and pigz runs with "-p 1"; with it there is no warning, and pigz runs with "-p 54" and "-p 64". Reported-by: Paul E. McKenney <paulmck@kernel.org> Closes: https://lore.kernel.org/all/0ddb5e5e-f801-444c-aa98-0cfe814a3026@paulmck-laptop/ Fixes: 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if available") Assisted-by: LLM Signed-off-by: Kees Cook <kees@kernel.org> --- arch/alpha/boot/Makefile | 4 ++-- arch/arc/boot/Makefile | 2 +- arch/arm64/boot/Makefile | 2 +- arch/microblaze/boot/Makefile | 2 +- arch/mips/boot/Makefile | 2 +- arch/nios2/boot/Makefile | 2 +- arch/parisc/boot/compressed/Makefile | 2 +- arch/riscv/boot/Makefile | 2 +- arch/s390/boot/Makefile | 2 +- arch/sh/boot/Makefile | 2 +- arch/sh/boot/compressed/Makefile | 2 +- arch/sparc/boot/Makefile | 4 ++-- arch/x86/boot/compressed/Makefile | 2 +- arch/xtensa/boot/Makefile | 2 +- 14 files changed, 16 insertions(+), 16 deletions(-) diff --git a/arch/alpha/boot/Makefile b/arch/alpha/boot/Makefile index d8dba85e606c..b050753e9015 100644 --- a/arch/alpha/boot/Makefile +++ b/arch/alpha/boot/Makefile @@ -40,7 +40,7 @@ endif # Compressed kernel image $(obj)/vmlinux.gz: $(obj)/vmlinux FORCE - $(call if_changed,gzip) + +$(call if_changed,gzip) @echo ' Kernel $@ is ready' $(obj)/main.o: $(obj)/ksize.h @@ -85,7 +85,7 @@ $(obj)/vmlinux.nh: vmlinux $(OBJSTRIP) FORCE $(call if_changed,objstrip) $(obj)/vmlinux.nh.gz: $(obj)/vmlinux.nh FORCE - $(call if_changed,gzip) + +$(call if_changed,gzip) $(obj)/tools/lxboot: $(obj)/bootloader $(OBJSTRIP) FORCE $(call if_changed,objstrip) diff --git a/arch/arc/boot/Makefile b/arch/arc/boot/Makefile index 5a8550124b73..8a44c3e7ff3e 100644 --- a/arch/arc/boot/Makefile +++ b/arch/arc/boot/Makefile @@ -23,7 +23,7 @@ $(obj)/vmlinux.bin: vmlinux FORCE $(call if_changed,objcopy) $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE - $(call if_changed,gzip) + +$(call if_changed,gzip) $(obj)/vmlinux.bin.lzma: $(obj)/vmlinux.bin FORCE $(call if_changed,lzma) diff --git a/arch/arm64/boot/Makefile b/arch/arm64/boot/Makefile index b5a08333bc57..d42d07667822 100644 --- a/arch/arm64/boot/Makefile +++ b/arch/arm64/boot/Makefile @@ -26,7 +26,7 @@ $(obj)/Image.bz2: $(obj)/Image FORCE $(call if_changed,bzip2) $(obj)/Image.gz: $(obj)/Image FORCE - $(call if_changed,gzip) + +$(call if_changed,gzip) $(obj)/Image.lz4: $(obj)/Image FORCE $(call if_changed,lz4) diff --git a/arch/microblaze/boot/Makefile b/arch/microblaze/boot/Makefile index 2b42c370d574..e3df3a64808a 100644 --- a/arch/microblaze/boot/Makefile +++ b/arch/microblaze/boot/Makefile @@ -14,7 +14,7 @@ $(obj)/linux.bin.ub: $(obj)/linux.bin FORCE $(call if_changed,uimage) $(obj)/linux.bin.gz: $(obj)/linux.bin FORCE - $(call if_changed,gzip) + +$(call if_changed,gzip) quiet_cmd_strip = STRIP $< $@$2 cmd_strip = $(STRIP) -K microblaze_start -K _end -K __log_buf \ diff --git a/arch/mips/boot/Makefile b/arch/mips/boot/Makefile index 8473c4671702..996bf41b78de 100644 --- a/arch/mips/boot/Makefile +++ b/arch/mips/boot/Makefile @@ -63,7 +63,7 @@ $(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE $(call if_changed,bzip2) $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE - $(call if_changed,gzip) + +$(call if_changed,gzip) $(obj)/vmlinux.bin.lzma: $(obj)/vmlinux.bin FORCE $(call if_changed,lzma) diff --git a/arch/nios2/boot/Makefile b/arch/nios2/boot/Makefile index 29c11a06b750..1381e6e428a8 100644 --- a/arch/nios2/boot/Makefile +++ b/arch/nios2/boot/Makefile @@ -18,7 +18,7 @@ $(obj)/vmlinux.bin: vmlinux FORCE $(call if_changed,objcopy) $(obj)/vmlinux.gz: $(obj)/vmlinux.bin FORCE - $(call if_changed,gzip) + +$(call if_changed,gzip) $(obj)/vmImage: $(obj)/vmlinux.gz FORCE $(call if_changed,uimage) diff --git a/arch/parisc/boot/compressed/Makefile b/arch/parisc/boot/compressed/Makefile index 14eefb5ed5d1..a7fc7d424c00 100644 --- a/arch/parisc/boot/compressed/Makefile +++ b/arch/parisc/boot/compressed/Makefile @@ -55,7 +55,7 @@ suffix-$(CONFIG_KERNEL_LZO) := lzo suffix-$(CONFIG_KERNEL_XZ) := xz $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE - $(call if_changed,gzip) + +$(call if_changed,gzip) $(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE $(call if_changed,bzip2_with_size) $(obj)/vmlinux.bin.lz4: $(obj)/vmlinux.bin FORCE diff --git a/arch/riscv/boot/Makefile b/arch/riscv/boot/Makefile index 4a2d75b7e42d..b80a077a8e8f 100644 --- a/arch/riscv/boot/Makefile +++ b/arch/riscv/boot/Makefile @@ -23,7 +23,7 @@ $(obj)/Image: vmlinux FORCE $(call if_changed,objcopy) $(obj)/Image.gz: $(obj)/Image FORCE - $(call if_changed,gzip) + +$(call if_changed,gzip) $(obj)/loader.o: $(src)/loader.S $(obj)/Image diff --git a/arch/s390/boot/Makefile b/arch/s390/boot/Makefile index 10b75e053a6f..e8e43ae80dd5 100644 --- a/arch/s390/boot/Makefile +++ b/arch/s390/boot/Makefile @@ -117,7 +117,7 @@ suffix-$(CONFIG_KERNEL_XZ) := .xz suffix-$(CONFIG_KERNEL_ZSTD) := .zst $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE - $(call if_changed,gzip) + +$(call if_changed,gzip) $(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE $(call if_changed,bzip2_with_size) $(obj)/vmlinux.bin.lz4: $(obj)/vmlinux.bin FORCE diff --git a/arch/sh/boot/Makefile b/arch/sh/boot/Makefile index 1f5d2df3c7e0..cf63e319b786 100644 --- a/arch/sh/boot/Makefile +++ b/arch/sh/boot/Makefile @@ -66,7 +66,7 @@ $(obj)/vmlinux.bin: vmlinux FORCE $(call if_changed,objcopy) $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE - $(call if_changed,gzip) + +$(call if_changed,gzip) $(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE $(call if_changed,bzip2) diff --git a/arch/sh/boot/compressed/Makefile b/arch/sh/boot/compressed/Makefile index 58df491778b2..b85aa5f0d616 100644 --- a/arch/sh/boot/compressed/Makefile +++ b/arch/sh/boot/compressed/Makefile @@ -39,7 +39,7 @@ $(obj)/vmlinux.bin: vmlinux FORCE $(call if_changed,objcopy) $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE - $(call if_changed,gzip) + +$(call if_changed,gzip) $(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE $(call if_changed,bzip2_with_size) $(obj)/vmlinux.bin.lzma: $(obj)/vmlinux.bin FORCE diff --git a/arch/sparc/boot/Makefile b/arch/sparc/boot/Makefile index 339c42d35089..b53cf596611e 100644 --- a/arch/sparc/boot/Makefile +++ b/arch/sparc/boot/Makefile @@ -23,7 +23,7 @@ ifeq ($(CONFIG_SPARC64),y) # Actual linking $(obj)/zImage: $(obj)/image FORCE - $(call if_changed,gzip) + +$(call if_changed,gzip) @$(kecho) 'Kernel: $@ is ready' '(#'$(or $(KBUILD_BUILD_VERSION),`cat .version`)')' $(obj)/vmlinux.aout: vmlinux FORCE @@ -45,7 +45,7 @@ $(obj)/image.bin: $(obj)/image FORCE $(call if_changed,objcopy) $(obj)/image.gz: $(obj)/image.bin FORCE - $(call if_changed,gzip) + +$(call if_changed,gzip) UIMAGE_LOADADDR = $(CONFIG_UBOOT_LOAD_ADDR) UIMAGE_ENTRYADDR = $(CONFIG_UBOOT_ENTRY_ADDR) diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile index 6ec5e031db1d..680ad803cba7 100644 --- a/arch/x86/boot/compressed/Makefile +++ b/arch/x86/boot/compressed/Makefile @@ -133,7 +133,7 @@ vmlinux.bin.all-y := $(obj)/vmlinux.bin vmlinux.bin.all-$(CONFIG_X86_NEED_RELOCS) += $(obj)/vmlinux.relocs $(obj)/vmlinux.bin.gz: $(vmlinux.bin.all-y) FORCE - $(call if_changed,gzip) + +$(call if_changed,gzip) $(obj)/vmlinux.bin.bz2: $(vmlinux.bin.all-y) FORCE $(call if_changed,bzip2_with_size) $(obj)/vmlinux.bin.lzma: $(vmlinux.bin.all-y) FORCE diff --git a/arch/xtensa/boot/Makefile b/arch/xtensa/boot/Makefile index d8b0fadf429a..934f9015b15f 100644 --- a/arch/xtensa/boot/Makefile +++ b/arch/xtensa/boot/Makefile @@ -36,7 +36,7 @@ $(obj)/vmlinux.bin: vmlinux FORCE $(call if_changed,objcopy) $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE - $(call if_changed,gzip) + +$(call if_changed,gzip) boot-elf: $(obj)/vmlinux.bin boot-redboot: $(obj)/vmlinux.bin.gz -- 2.55.0 ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v1 1/2] kbuild: Mark the kernel image gzip recipes as recursive 2026-10-09 6:27 ` [PATCH v1 1/2] kbuild: Mark the kernel image gzip recipes as recursive Kees Cook @ 2026-10-09 9:08 ` Nicolas Schier 2026-10-09 12:05 ` Lorenzo Stoakes (ARM) 2026-10-09 15:51 ` Paul E. McKenney 2 siblings, 0 replies; 8+ messages in thread From: Nicolas Schier @ 2026-10-09 9:08 UTC (permalink / raw) To: Kees Cook Cc: Nathan Chancellor, Nicolas Schier, Lorenzo Stoakes, Manuel Ebner, linux-kbuild, Paul E. McKenney, linux-kernel, linux-hardening > Since commit 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if > available"), cmd_gzip runs pigz through scripts/jobserver-exec, which > takes the job slots make has free. GNU Make passes the jobserver pipe > only to recipes it treats as recursive, but leaves --jobserver-auth in > MAKEFLAGS for all of them. The make 4.3 in CentOS Stream 9 also hides > the pipe in sub-makes (a backport of the fix for GNU Make bug 58232), so > jobserver-exec cannot open it, runs pigz on one thread, and prints: > > WARNING: Unable to reopen jobserver read-side pipe: FileNotFoundError(2, 'No such file or directory') > > Mark the gzip recipes as recursive with "+", as commit ecab4115c44cc > ("kbuild: mark `rustc` (and others) invocations as recursive") did for > rustc. As with those, "make -n" now runs them. > > Tested ARCH=x86_64 and ARCH=arm64 defconfig with GNU Make 4.3 plus > CentOS Stream 9's make-4.3-cloexec.patch, GCC 16.2.0, and -j64: without > this change the warning appears and pigz runs with "-p 1"; with it there > is no warning, and pigz runs with "-p 54" and "-p 64". > > Reported-by: Paul E. McKenney <paulmck@kernel.org> > Closes: https://lore.kernel.org/all/0ddb5e5e-f801-444c-aa98-0cfe814a3026@paulmck-laptop/ > Fixes: 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if available") > Assisted-by: LLM > Signed-off-by: Kees Cook <kees@kernel.org> > --- > arch/alpha/boot/Makefile | 4 ++-- > arch/arc/boot/Makefile | 2 +- > arch/arm64/boot/Makefile | 2 +- > arch/microblaze/boot/Makefile | 2 +- > arch/mips/boot/Makefile | 2 +- > arch/nios2/boot/Makefile | 2 +- > arch/parisc/boot/compressed/Makefile | 2 +- > arch/riscv/boot/Makefile | 2 +- > arch/s390/boot/Makefile | 2 +- > arch/sh/boot/Makefile | 2 +- > arch/sh/boot/compressed/Makefile | 2 +- > arch/sparc/boot/Makefile | 4 ++-- > arch/x86/boot/compressed/Makefile | 2 +- > arch/xtensa/boot/Makefile | 2 +- > 14 files changed, 16 insertions(+), 16 deletions(-) Reviewed-by: Nicolas Schier <n.schier@fritz.com> Before it gets lost, we should also update the documentation bit that references $(if_changed,gzip): diff --git a/Documentation/kbuild/makefiles.rst b/Documentation/kbuild/makefiles.rst index 7521cae7d56f..2760717ab7b7 100644 --- a/Documentation/kbuild/makefiles.rst +++ b/Documentation/kbuild/makefiles.rst @@ -1452,13 +1452,14 @@ objcopy OBJCOPYFLAGS_$@ may be used to set additional options. gzip - Compress target. Use maximum compression to compress target. + Compress target. Use maximum compression to compress target. Add '+' prefix + to allow parallel gzip compression via pigz (if available). Example:: #arch/x86/boot/compressed/Makefile $(obj)/vmlinux.bin.gz: $(vmlinux.bin.all-y) FORCE - $(call if_changed,gzip) + +$(call if_changed,gzip) dtc Create flattened device tree blob object suitable for linking -- Nicolas ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v1 1/2] kbuild: Mark the kernel image gzip recipes as recursive 2026-10-09 6:27 ` [PATCH v1 1/2] kbuild: Mark the kernel image gzip recipes as recursive Kees Cook 2026-10-09 9:08 ` Nicolas Schier @ 2026-10-09 12:05 ` Lorenzo Stoakes (ARM) 2026-10-09 15:52 ` Paul E. McKenney 2026-10-09 15:51 ` Paul E. McKenney 2 siblings, 1 reply; 8+ messages in thread From: Lorenzo Stoakes (ARM) @ 2026-10-09 12:05 UTC (permalink / raw) To: Kees Cook Cc: Nathan Chancellor, Nicolas Schier, Manuel Ebner, linux-kbuild, Paul E. McKenney, linux-kernel, linux-hardening On Thu, Oct 08, 2026 at 11:27:30PM -0700, Kees Cook wrote: > Since commit 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if > available"), cmd_gzip runs pigz through scripts/jobserver-exec, which > takes the job slots make has free. GNU Make passes the jobserver pipe > only to recipes it treats as recursive, but leaves --jobserver-auth in > MAKEFLAGS for all of them. The make 4.3 in CentOS Stream 9 also hides > the pipe in sub-makes (a backport of the fix for GNU Make bug 58232), so > jobserver-exec cannot open it, runs pigz on one thread, and prints: > > WARNING: Unable to reopen jobserver read-side pipe: FileNotFoundError(2, 'No such file or directory') > > Mark the gzip recipes as recursive with "+", as commit ecab4115c44cc > ("kbuild: mark `rustc` (and others) invocations as recursive") did for > rustc. As with those, "make -n" now runs them. > > Tested ARCH=x86_64 and ARCH=arm64 defconfig with GNU Make 4.3 plus > CentOS Stream 9's make-4.3-cloexec.patch, GCC 16.2.0, and -j64: without > this change the warning appears and pigz runs with "-p 1"; with it there > is no warning, and pigz runs with "-p 54" and "-p 64". > > Reported-by: Paul E. McKenney <paulmck@kernel.org> > Closes: https://lore.kernel.org/all/0ddb5e5e-f801-444c-aa98-0cfe814a3026@paulmck-laptop/ > Fixes: 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if available") > Assisted-by: LLM > Signed-off-by: Kees Cook <kees@kernel.org> Nice find! Acked-by: Lorenzo Stoakes (ARM) <ljs@kernel.org> Sorry for breaking your build Paul ;) -- Cheers, Lorenzo ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v1 1/2] kbuild: Mark the kernel image gzip recipes as recursive 2026-10-09 12:05 ` Lorenzo Stoakes (ARM) @ 2026-10-09 15:52 ` Paul E. McKenney 0 siblings, 0 replies; 8+ messages in thread From: Paul E. McKenney @ 2026-10-09 15:52 UTC (permalink / raw) To: Lorenzo Stoakes (ARM) Cc: Kees Cook, Nathan Chancellor, Nicolas Schier, Manuel Ebner, linux-kbuild, linux-kernel, linux-hardening On Fri, Oct 09, 2026 at 01:05:00PM +0100, Lorenzo Stoakes (ARM) wrote: > On Thu, Oct 08, 2026 at 11:27:30PM -0700, Kees Cook wrote: > > Since commit 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if > > available"), cmd_gzip runs pigz through scripts/jobserver-exec, which > > takes the job slots make has free. GNU Make passes the jobserver pipe > > only to recipes it treats as recursive, but leaves --jobserver-auth in > > MAKEFLAGS for all of them. The make 4.3 in CentOS Stream 9 also hides > > the pipe in sub-makes (a backport of the fix for GNU Make bug 58232), so > > jobserver-exec cannot open it, runs pigz on one thread, and prints: > > > > WARNING: Unable to reopen jobserver read-side pipe: FileNotFoundError(2, 'No such file or directory') > > > > Mark the gzip recipes as recursive with "+", as commit ecab4115c44cc > > ("kbuild: mark `rustc` (and others) invocations as recursive") did for > > rustc. As with those, "make -n" now runs them. > > > > Tested ARCH=x86_64 and ARCH=arm64 defconfig with GNU Make 4.3 plus > > CentOS Stream 9's make-4.3-cloexec.patch, GCC 16.2.0, and -j64: without > > this change the warning appears and pigz runs with "-p 1"; with it there > > is no warning, and pigz runs with "-p 54" and "-p 64". > > > > Reported-by: Paul E. McKenney <paulmck@kernel.org> > > Closes: https://lore.kernel.org/all/0ddb5e5e-f801-444c-aa98-0cfe814a3026@paulmck-laptop/ > > Fixes: 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if available") > > Assisted-by: LLM > > Signed-off-by: Kees Cook <kees@kernel.org> > > Nice find! > > Acked-by: Lorenzo Stoakes (ARM) <ljs@kernel.org> > > Sorry for breaking your build Paul ;) Hey, this was the fastest bisection that I have ever done by far! Normally it takes hours per step, but this was about 40 seconds per step. ;-) Thanx, Paul ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v1 1/2] kbuild: Mark the kernel image gzip recipes as recursive 2026-10-09 6:27 ` [PATCH v1 1/2] kbuild: Mark the kernel image gzip recipes as recursive Kees Cook 2026-10-09 9:08 ` Nicolas Schier 2026-10-09 12:05 ` Lorenzo Stoakes (ARM) @ 2026-10-09 15:51 ` Paul E. McKenney 2 siblings, 0 replies; 8+ messages in thread From: Paul E. McKenney @ 2026-10-09 15:51 UTC (permalink / raw) To: Kees Cook Cc: Nathan Chancellor, Nicolas Schier, Lorenzo Stoakes, Manuel Ebner, linux-kbuild, linux-kernel, linux-hardening On Thu, Oct 08, 2026 at 11:27:30PM -0700, Kees Cook wrote: > Since commit 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if > available"), cmd_gzip runs pigz through scripts/jobserver-exec, which > takes the job slots make has free. GNU Make passes the jobserver pipe > only to recipes it treats as recursive, but leaves --jobserver-auth in > MAKEFLAGS for all of them. The make 4.3 in CentOS Stream 9 also hides > the pipe in sub-makes (a backport of the fix for GNU Make bug 58232), so > jobserver-exec cannot open it, runs pigz on one thread, and prints: > > WARNING: Unable to reopen jobserver read-side pipe: FileNotFoundError(2, 'No such file or directory') > > Mark the gzip recipes as recursive with "+", as commit ecab4115c44cc > ("kbuild: mark `rustc` (and others) invocations as recursive") did for > rustc. As with those, "make -n" now runs them. > > Tested ARCH=x86_64 and ARCH=arm64 defconfig with GNU Make 4.3 plus > CentOS Stream 9's make-4.3-cloexec.patch, GCC 16.2.0, and -j64: without > this change the warning appears and pigz runs with "-p 1"; with it there > is no warning, and pigz runs with "-p 54" and "-p 64". Thank you, Kees! My initial test of this patch (using rcutorture) still showed the failure. I am running GCC 11.5.0 and without any patches to "make". I take it that I need to find and apply CentOS Stream 9's make-4.3-cloexec.patch? But if I use "make -j" instead of rcutorture, this patch does work fine. Even with "make -j632". I put rcutorture in the background using nohup and full redirection, but "nohup make -j632 > /tmp/Make.out 2>&1 < /dev/null &" works just fine. And running rcutorture in the foreground still fails. Very strange. So I guess I could give a Tested-by for builds using straight-up make, but not from rcutorture. ;-) Thanx, Paul > Reported-by: Paul E. McKenney <paulmck@kernel.org> > Closes: https://lore.kernel.org/all/0ddb5e5e-f801-444c-aa98-0cfe814a3026@paulmck-laptop/ > Fixes: 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if available") > Assisted-by: LLM > Signed-off-by: Kees Cook <kees@kernel.org> > --- > arch/alpha/boot/Makefile | 4 ++-- > arch/arc/boot/Makefile | 2 +- > arch/arm64/boot/Makefile | 2 +- > arch/microblaze/boot/Makefile | 2 +- > arch/mips/boot/Makefile | 2 +- > arch/nios2/boot/Makefile | 2 +- > arch/parisc/boot/compressed/Makefile | 2 +- > arch/riscv/boot/Makefile | 2 +- > arch/s390/boot/Makefile | 2 +- > arch/sh/boot/Makefile | 2 +- > arch/sh/boot/compressed/Makefile | 2 +- > arch/sparc/boot/Makefile | 4 ++-- > arch/x86/boot/compressed/Makefile | 2 +- > arch/xtensa/boot/Makefile | 2 +- > 14 files changed, 16 insertions(+), 16 deletions(-) > > diff --git a/arch/alpha/boot/Makefile b/arch/alpha/boot/Makefile > index d8dba85e606c..b050753e9015 100644 > --- a/arch/alpha/boot/Makefile > +++ b/arch/alpha/boot/Makefile > @@ -40,7 +40,7 @@ endif > > # Compressed kernel image > $(obj)/vmlinux.gz: $(obj)/vmlinux FORCE > - $(call if_changed,gzip) > + +$(call if_changed,gzip) > @echo ' Kernel $@ is ready' > > $(obj)/main.o: $(obj)/ksize.h > @@ -85,7 +85,7 @@ $(obj)/vmlinux.nh: vmlinux $(OBJSTRIP) FORCE > $(call if_changed,objstrip) > > $(obj)/vmlinux.nh.gz: $(obj)/vmlinux.nh FORCE > - $(call if_changed,gzip) > + +$(call if_changed,gzip) > > $(obj)/tools/lxboot: $(obj)/bootloader $(OBJSTRIP) FORCE > $(call if_changed,objstrip) > diff --git a/arch/arc/boot/Makefile b/arch/arc/boot/Makefile > index 5a8550124b73..8a44c3e7ff3e 100644 > --- a/arch/arc/boot/Makefile > +++ b/arch/arc/boot/Makefile > @@ -23,7 +23,7 @@ $(obj)/vmlinux.bin: vmlinux FORCE > $(call if_changed,objcopy) > > $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE > - $(call if_changed,gzip) > + +$(call if_changed,gzip) > > $(obj)/vmlinux.bin.lzma: $(obj)/vmlinux.bin FORCE > $(call if_changed,lzma) > diff --git a/arch/arm64/boot/Makefile b/arch/arm64/boot/Makefile > index b5a08333bc57..d42d07667822 100644 > --- a/arch/arm64/boot/Makefile > +++ b/arch/arm64/boot/Makefile > @@ -26,7 +26,7 @@ $(obj)/Image.bz2: $(obj)/Image FORCE > $(call if_changed,bzip2) > > $(obj)/Image.gz: $(obj)/Image FORCE > - $(call if_changed,gzip) > + +$(call if_changed,gzip) > > $(obj)/Image.lz4: $(obj)/Image FORCE > $(call if_changed,lz4) > diff --git a/arch/microblaze/boot/Makefile b/arch/microblaze/boot/Makefile > index 2b42c370d574..e3df3a64808a 100644 > --- a/arch/microblaze/boot/Makefile > +++ b/arch/microblaze/boot/Makefile > @@ -14,7 +14,7 @@ $(obj)/linux.bin.ub: $(obj)/linux.bin FORCE > $(call if_changed,uimage) > > $(obj)/linux.bin.gz: $(obj)/linux.bin FORCE > - $(call if_changed,gzip) > + +$(call if_changed,gzip) > > quiet_cmd_strip = STRIP $< $@$2 > cmd_strip = $(STRIP) -K microblaze_start -K _end -K __log_buf \ > diff --git a/arch/mips/boot/Makefile b/arch/mips/boot/Makefile > index 8473c4671702..996bf41b78de 100644 > --- a/arch/mips/boot/Makefile > +++ b/arch/mips/boot/Makefile > @@ -63,7 +63,7 @@ $(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE > $(call if_changed,bzip2) > > $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE > - $(call if_changed,gzip) > + +$(call if_changed,gzip) > > $(obj)/vmlinux.bin.lzma: $(obj)/vmlinux.bin FORCE > $(call if_changed,lzma) > diff --git a/arch/nios2/boot/Makefile b/arch/nios2/boot/Makefile > index 29c11a06b750..1381e6e428a8 100644 > --- a/arch/nios2/boot/Makefile > +++ b/arch/nios2/boot/Makefile > @@ -18,7 +18,7 @@ $(obj)/vmlinux.bin: vmlinux FORCE > $(call if_changed,objcopy) > > $(obj)/vmlinux.gz: $(obj)/vmlinux.bin FORCE > - $(call if_changed,gzip) > + +$(call if_changed,gzip) > > $(obj)/vmImage: $(obj)/vmlinux.gz FORCE > $(call if_changed,uimage) > diff --git a/arch/parisc/boot/compressed/Makefile b/arch/parisc/boot/compressed/Makefile > index 14eefb5ed5d1..a7fc7d424c00 100644 > --- a/arch/parisc/boot/compressed/Makefile > +++ b/arch/parisc/boot/compressed/Makefile > @@ -55,7 +55,7 @@ suffix-$(CONFIG_KERNEL_LZO) := lzo > suffix-$(CONFIG_KERNEL_XZ) := xz > > $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE > - $(call if_changed,gzip) > + +$(call if_changed,gzip) > $(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE > $(call if_changed,bzip2_with_size) > $(obj)/vmlinux.bin.lz4: $(obj)/vmlinux.bin FORCE > diff --git a/arch/riscv/boot/Makefile b/arch/riscv/boot/Makefile > index 4a2d75b7e42d..b80a077a8e8f 100644 > --- a/arch/riscv/boot/Makefile > +++ b/arch/riscv/boot/Makefile > @@ -23,7 +23,7 @@ $(obj)/Image: vmlinux FORCE > $(call if_changed,objcopy) > > $(obj)/Image.gz: $(obj)/Image FORCE > - $(call if_changed,gzip) > + +$(call if_changed,gzip) > > $(obj)/loader.o: $(src)/loader.S $(obj)/Image > > diff --git a/arch/s390/boot/Makefile b/arch/s390/boot/Makefile > index 10b75e053a6f..e8e43ae80dd5 100644 > --- a/arch/s390/boot/Makefile > +++ b/arch/s390/boot/Makefile > @@ -117,7 +117,7 @@ suffix-$(CONFIG_KERNEL_XZ) := .xz > suffix-$(CONFIG_KERNEL_ZSTD) := .zst > > $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE > - $(call if_changed,gzip) > + +$(call if_changed,gzip) > $(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE > $(call if_changed,bzip2_with_size) > $(obj)/vmlinux.bin.lz4: $(obj)/vmlinux.bin FORCE > diff --git a/arch/sh/boot/Makefile b/arch/sh/boot/Makefile > index 1f5d2df3c7e0..cf63e319b786 100644 > --- a/arch/sh/boot/Makefile > +++ b/arch/sh/boot/Makefile > @@ -66,7 +66,7 @@ $(obj)/vmlinux.bin: vmlinux FORCE > $(call if_changed,objcopy) > > $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE > - $(call if_changed,gzip) > + +$(call if_changed,gzip) > > $(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE > $(call if_changed,bzip2) > diff --git a/arch/sh/boot/compressed/Makefile b/arch/sh/boot/compressed/Makefile > index 58df491778b2..b85aa5f0d616 100644 > --- a/arch/sh/boot/compressed/Makefile > +++ b/arch/sh/boot/compressed/Makefile > @@ -39,7 +39,7 @@ $(obj)/vmlinux.bin: vmlinux FORCE > $(call if_changed,objcopy) > > $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE > - $(call if_changed,gzip) > + +$(call if_changed,gzip) > $(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE > $(call if_changed,bzip2_with_size) > $(obj)/vmlinux.bin.lzma: $(obj)/vmlinux.bin FORCE > diff --git a/arch/sparc/boot/Makefile b/arch/sparc/boot/Makefile > index 339c42d35089..b53cf596611e 100644 > --- a/arch/sparc/boot/Makefile > +++ b/arch/sparc/boot/Makefile > @@ -23,7 +23,7 @@ ifeq ($(CONFIG_SPARC64),y) > # Actual linking > > $(obj)/zImage: $(obj)/image FORCE > - $(call if_changed,gzip) > + +$(call if_changed,gzip) > @$(kecho) 'Kernel: $@ is ready' '(#'$(or $(KBUILD_BUILD_VERSION),`cat .version`)')' > > $(obj)/vmlinux.aout: vmlinux FORCE > @@ -45,7 +45,7 @@ $(obj)/image.bin: $(obj)/image FORCE > $(call if_changed,objcopy) > > $(obj)/image.gz: $(obj)/image.bin FORCE > - $(call if_changed,gzip) > + +$(call if_changed,gzip) > > UIMAGE_LOADADDR = $(CONFIG_UBOOT_LOAD_ADDR) > UIMAGE_ENTRYADDR = $(CONFIG_UBOOT_ENTRY_ADDR) > diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile > index 6ec5e031db1d..680ad803cba7 100644 > --- a/arch/x86/boot/compressed/Makefile > +++ b/arch/x86/boot/compressed/Makefile > @@ -133,7 +133,7 @@ vmlinux.bin.all-y := $(obj)/vmlinux.bin > vmlinux.bin.all-$(CONFIG_X86_NEED_RELOCS) += $(obj)/vmlinux.relocs > > $(obj)/vmlinux.bin.gz: $(vmlinux.bin.all-y) FORCE > - $(call if_changed,gzip) > + +$(call if_changed,gzip) > $(obj)/vmlinux.bin.bz2: $(vmlinux.bin.all-y) FORCE > $(call if_changed,bzip2_with_size) > $(obj)/vmlinux.bin.lzma: $(vmlinux.bin.all-y) FORCE > diff --git a/arch/xtensa/boot/Makefile b/arch/xtensa/boot/Makefile > index d8b0fadf429a..934f9015b15f 100644 > --- a/arch/xtensa/boot/Makefile > +++ b/arch/xtensa/boot/Makefile > @@ -36,7 +36,7 @@ $(obj)/vmlinux.bin: vmlinux FORCE > $(call if_changed,objcopy) > > $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE > - $(call if_changed,gzip) > + +$(call if_changed,gzip) > > boot-elf: $(obj)/vmlinux.bin > boot-redboot: $(obj)/vmlinux.bin.gz > -- > 2.55.0 > ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v1 2/2] kbuild: Mark the initcall ordering recipe as recursive 2026-10-09 6:27 [PATCH v1 0/2] kbuild: Give jobserver-exec recipes the jobserver Kees Cook 2026-10-09 6:27 ` [PATCH v1 1/2] kbuild: Mark the kernel image gzip recipes as recursive Kees Cook @ 2026-10-09 6:27 ` Kees Cook 2026-10-09 15:20 ` Nicolas Schier 1 sibling, 1 reply; 8+ messages in thread From: Kees Cook @ 2026-10-09 6:27 UTC (permalink / raw) To: Nathan Chancellor Cc: Kees Cook, Nicolas Schier, Sami Tolvanen, linux-kbuild, llvm, linux-kernel, linux-hardening generate_initcall_order.pl, which orders initcalls for Clang LTO, forks a child per input file, up to the job count scripts/jobserver-exec passes it in PARALLELISM. It used to run from link-vmlinux.sh, which the top-level Makefile invokes as a recursive recipe, but commit 5d45950dfbb15 ("kbuild: move vmlinux.o link to scripts/Makefile.vmlinux_o") moved it into a recipe without "+". GNU Make passes the jobserver pipe only to recipes it treats as recursive, and the make 4.3 in CentOS Stream 9 also hides it in sub-makes (a backport of the fix for GNU Make bug 58232), so jobserver-exec cannot open it and prints: WARNING: Unable to reopen jobserver read-side pipe: FileNotFoundError(2, 'No such file or directory') PARALLELISM is then left unset, and the script caps its children at the number of online CPUs instead of the free job slots. Mark the recipe as recursive again. Tested ARCH=x86_64 defconfig with CONFIG_LTO_CLANG_THIN=y, LLVM=1 (Clang 24.0.0git), and GNU Make 4.3 plus CentOS Stream 9's make-4.3-cloexec.patch. Fixes: 5d45950dfbb15 ("kbuild: move vmlinux.o link to scripts/Makefile.vmlinux_o") Assisted-by: LLM Signed-off-by: Kees Cook <kees@kernel.org> --- scripts/Makefile.vmlinux_o | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/Makefile.vmlinux_o b/scripts/Makefile.vmlinux_o index 24a3a4fd271c..0b37bd5dc3a4 100644 --- a/scripts/Makefile.vmlinux_o +++ b/scripts/Makefile.vmlinux_o @@ -19,7 +19,7 @@ quiet_cmd_gen_initcalls_lds = GEN $@ .tmp_initcalls.lds: $(srctree)/scripts/generate_initcall_order.pl \ vmlinux.a $(KBUILD_VMLINUX_LIBS) FORCE - $(call if_changed,gen_initcalls_lds) + +$(call if_changed,gen_initcalls_lds) targets := .tmp_initcalls.lds -- 2.55.0 ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v1 2/2] kbuild: Mark the initcall ordering recipe as recursive 2026-10-09 6:27 ` [PATCH v1 2/2] kbuild: Mark the initcall ordering recipe " Kees Cook @ 2026-10-09 15:20 ` Nicolas Schier 0 siblings, 0 replies; 8+ messages in thread From: Nicolas Schier @ 2026-10-09 15:20 UTC (permalink / raw) To: Kees Cook Cc: Nathan Chancellor, Nicolas Schier, Sami Tolvanen, linux-kbuild, llvm, linux-kernel, linux-hardening > generate_initcall_order.pl, which orders initcalls for Clang LTO, forks > a child per input file, up to the job count scripts/jobserver-exec > passes it in PARALLELISM. It used to run from link-vmlinux.sh, which > the top-level Makefile invokes as a recursive recipe, but > commit 5d45950dfbb15 ("kbuild: move vmlinux.o link to > scripts/Makefile.vmlinux_o") moved it into a recipe without "+". > > GNU Make passes the jobserver pipe only to recipes it treats as > recursive, and the make 4.3 in CentOS Stream 9 also hides it in > sub-makes (a backport of the fix for GNU Make bug 58232), so > jobserver-exec cannot open it and prints: > > WARNING: Unable to reopen jobserver read-side pipe: FileNotFoundError(2, 'No such file or directory') > > PARALLELISM is then left unset, and the script caps its children at the > number of online CPUs instead of the free job slots. > > Mark the recipe as recursive again. > > Tested ARCH=x86_64 defconfig with CONFIG_LTO_CLANG_THIN=y, LLVM=1 > (Clang 24.0.0git), and GNU Make 4.3 plus CentOS Stream 9's > make-4.3-cloexec.patch. > > Fixes: 5d45950dfbb15 ("kbuild: move vmlinux.o link to scripts/Makefile.vmlinux_o") > Assisted-by: LLM > Signed-off-by: Kees Cook <kees@kernel.org> > --- > scripts/Makefile.vmlinux_o | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/scripts/Makefile.vmlinux_o b/scripts/Makefile.vmlinux_o > index 24a3a4fd271c..0b37bd5dc3a4 100644 > --- a/scripts/Makefile.vmlinux_o > +++ b/scripts/Makefile.vmlinux_o > @@ -19,7 +19,7 @@ quiet_cmd_gen_initcalls_lds = GEN $@ > > .tmp_initcalls.lds: $(srctree)/scripts/generate_initcall_order.pl \ > vmlinux.a $(KBUILD_VMLINUX_LIBS) FORCE > - $(call if_changed,gen_initcalls_lds) > + +$(call if_changed,gen_initcalls_lds) b4 hinted me on [1], where a similar fix was rejected; but as we know have more arguments to generate_initcall_order.pl ($(KBUILD_VMLINUX_LIBS)), the situation is slightly different. Thanks! Reviewed-by: Nicolas Schier <n.schier@fritz.com> [1]: https://lore.kernel.org/all/CAK7LNARCM=rUm8mA8GRQ7ufeyfneGf4OEvHmESKt=zuxs2KrHw@mail.gmail.com/ -- Nicolas ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2026-10-09 15:52 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2026-10-09 6:27 [PATCH v1 0/2] kbuild: Give jobserver-exec recipes the jobserver Kees Cook 2026-10-09 6:27 ` [PATCH v1 1/2] kbuild: Mark the kernel image gzip recipes as recursive Kees Cook 2026-10-09 9:08 ` Nicolas Schier 2026-10-09 12:05 ` Lorenzo Stoakes (ARM) 2026-10-09 15:52 ` Paul E. McKenney 2026-10-09 15:51 ` Paul E. McKenney 2026-10-09 6:27 ` [PATCH v1 2/2] kbuild: Mark the initcall ordering recipe " Kees Cook 2026-10-09 15:20 ` Nicolas Schier
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®