From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 00982521229; Wed, 16 Sep 2026 15:27:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789572435; cv=none; b=nZ/AJaPLpyO9/S4WH4SgW76mt+s80JHR63g35/awvW6PP3efFarw5tWUi/56XSWO+7UdnAv/SmZP4xfyUqPgj1Yiy4/2stQz7xJRrAiL5Y0N6gvAfg4Rc7ND5sgTUbd3uhMHWbydhwLTLeZArK/+aT7c3ByRkmNYdacsfQGvyRE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789572435; c=relaxed/simple; bh=fQV4rSiHmdiI0jtkCgcO0ezwkrfUrjLXCKAPjBp4W1k=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=G7RiLh1RDmpsychIuxcEiJsZI0xK5BwxtajXr+sOriCutlB5AA9VIL+zJw7GRmiJI97isR/vIYqsMJQhTYDyGjEM+25F1D2RtaH8TSe9gduw8KjuxuvvVfU2GTapkBCruUSIQXbI6yCjiVefq72Sb9TC/TauaBrwtU52Ggf+D2g= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=K4zyDQK3; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="K4zyDQK3" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 29E4B1F00893; Wed, 16 Sep 2026 15:27:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1789572430; bh=UkRbXLbpw+AP2jr8iP96phPK0XDdfYv/MOHvNo1E1vo=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=K4zyDQK3PDEMTdWFvmeoWoqSY4bftx/jIM47m+e5m/CxuUgT2ZlxOgrpbi8scn6zA QVKN7ZD9SORGKm3IBe/NE6t27UzhPfsBm/WZe+N7lz8+tNzpq09wAFnfd0tXjv81xh 8rvuaPTitkf5DvhUxl5/uIMlCntdrSUBTbc0Nl9wCiBCLt0ysG9RDdTG7UNBtFuv+N vx4N/BEQJLgSIglrxXmmd/2SbmxnmWg7UK0VPlxcFsCJ8072505DSOwvaGsjM7tV+d dNbIGqX8V6mt4oDoL+ZGWrk110WemqKgp5jhPvmlaWG38bbsOuVV/b72clK+giRlzw R8h3G0UoIzEGg== Date: Wed, 16 Sep 2026 08:27:09 -0700 From: Kees Cook To: Arnd Bergmann Cc: Lorenzo Stoakes , Linus Torvalds , Nathan Chancellor , Nicolas Schier , Nick Desaulniers , Bill Wendling , Justin Stitt , Masahiro Yamada , Alexey Gladkov , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Ilias Apalodimas , Josh Poimboeuf , Peter Zijlstra , Miguel Ojeda , Boqun Feng , Gary Guo , =?iso-8859-1?Q?Bj=F6rn?= Roy Baron , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross , Danilo Krummrich , Daniel Almeida , Tamir Duberstein , Alexandre Courbot , Onur =?iso-8859-1?Q?=D6zkan?= , Jonathan Corbet , Randy Dunlap , "Gustavo A. R. Silva" , linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-riscv@lists.infradead.org, Linux-Arch , linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-doc@vger.kernel.org, Jens Axboe , linux-hardening@vger.kernel.org Subject: Re: [PATCH v2 21/21] kbuild: use pigz for gzip compression if available Message-ID: <202609160809.78E97DC34F@keescook> References: <20260914-build-speedup-v2-0-39817ec5db23@kernel.org> <20260914-build-speedup-v2-21-39817ec5db23@kernel.org> <202609151758.BA5732348@keescook> <081b2b60-e295-4cd7-b201-d996a88c5f28@app.fastmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <081b2b60-e295-4cd7-b201-d996a88c5f28@app.fastmail.com> On Wed, Sep 16, 2026 at 08:38:33AM +0200, Arnd Bergmann wrote: > On Wed, Sep 16, 2026, at 03:06, Kees Cook wrote: > > On Mon, Sep 14, 2026 at 10:22:20AM +0100, Lorenzo Stoakes (ARM) wrote: > > > > Makefile: > > > > KPGZIP := $(if $(shell command -v pigz 2>/dev/null),$(PYTHON3) > > $(abs_srctree)/scripts/jobserver-exec > > $(abs_srctree)/scripts/parallel-gzip,gzip) > > > > and the same scripts/parallel-gzip: > > > > #!/bin/sh > > exec pigz -p ${PARALLELISM:-1} "$@" > > > > And then replace the /boot/ KGZIP uses with KPGZIP. Then we don't need > > to change modules, config_data, new users, nor cmd_gzip, etc, which all > > continue to use the regular KGZIP and the "use python" part for the > > KPGZIP case becomes is a 1-time cost? > > Nice, I like the idea and this this will work. I didn't know about > scripts/jobserver-exec, but this does seem like a perfect fit. > > I checked the other compression tools we support, and found that > zstd is another one that has built-in support for parallel > compression, so we probably want to have it there as well. Follow-up idea: Having to make a dedicated wrapper script to use jobserver-exec is a bit clunky, so maybe the ergonomics need to be improved there slightly? Maybe we can add a simple template thingy to tools/lib/python/jobserver.py's "run" to replace, say, '%PARALLELISM%' automatically, then it becomes: KPGZIP := $(if $(shell command -v pigz 2>/dev/null),$(PYTHON3) \ $(abs_srctree)/scripts/jobserver-exec \ pigz -p %PARALLELISM% -Kees -- Kees Cook