mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Kees Cook <kees@kernel.org>
To: "Lorenzo Stoakes (ARM)" <ljs@kernel.org>
Cc: "Linus Torvalds" <torvalds@linux-foundation.org>,
	"Nathan Chancellor" <nathan@kernel.org>,
	"Nicolas Schier" <nsc@kernel.org>,
	"Nick Desaulniers" <ndesaulniers@google.com>,
	"Bill Wendling" <morbo@google.com>,
	"Justin Stitt" <justinstitt@google.com>,
	"Masahiro Yamada" <masahiroy@kernel.org>,
	"Alexey Gladkov" <legion@kernel.org>,
	"Thomas Gleixner" <tglx@kernel.org>,
	"Ingo Molnar" <mingo@redhat.com>,
	"Borislav Petkov" <bp@alien8.de>,
	"Dave Hansen" <dave.hansen@linux.intel.com>,
	x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>,
	"Paul Walmsley" <pjw@kernel.org>,
	"Palmer Dabbelt" <palmer@dabbelt.com>,
	"Albert Ou" <aou@eecs.berkeley.edu>,
	"Alexandre Ghiti" <alex@ghiti.fr>,
	"Arnd Bergmann" <arnd@arndb.de>,
	"Catalin Marinas" <catalin.marinas@arm.com>,
	"Will Deacon" <will@kernel.org>,
	"Mark Rutland" <mark.rutland@arm.com>,
	"Ard Biesheuvel" <ardb@kernel.org>,
	"Ilias Apalodimas" <ilias.apalodimas@linaro.org>,
	"Josh Poimboeuf" <jpoimboe@kernel.org>,
	"Peter Zijlstra" <peterz@infradead.org>,
	"Miguel Ojeda" <ojeda@kernel.org>,
	"Boqun Feng" <boqun@kernel.org>, "Gary Guo" <gary@garyguo.net>,
	"Björn Roy Baron" <bjorn3_gh@protonmail.com>,
	"Benno Lossin" <lossin@kernel.org>,
	"Andreas Hindborg" <a.hindborg@kernel.org>,
	"Alice Ryhl" <aliceryhl@google.com>,
	"Trevor Gross" <tmgross@umich.edu>,
	"Danilo Krummrich" <dakr@kernel.org>,
	"Daniel Almeida" <daniel.almeida@collabora.com>,
	"Tamir Duberstein" <tamird@kernel.org>,
	"Alexandre Courbot" <acourbot@nvidia.com>,
	"Onur Özkan" <work@onurozkan.dev>,
	"Jonathan Corbet" <corbet@lwn.net>,
	"Randy Dunlap" <rdunlap@infradead.org>,
	"Gustavo A. R. Silva" <gustavoars@kernel.org>,
	linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org,
	llvm@lists.linux.dev, linux-riscv@lists.infradead.org,
	linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org,
	linux-efi@vger.kernel.org, rust-for-linux@vger.kernel.org,
	linux-doc@vger.kernel.org, "Jens Axboe" <axboe@kernel.dk>,
	linux-hardening@vger.kernel.org
Subject: Re: [PATCH v3 09/20] kbuild: implement and use depcheck to check dependency timestamps
Date: Thu, 17 Sep 2026 11:42:50 -0700	[thread overview]
Message-ID: <202609171117.35BFDCC13@keescook> (raw)
In-Reply-To: <20260917-build-speedup-v3-9-9ecf4163ff36@kernel.org>

On Thu, Sep 17, 2026 at 05:06:19PM +0100, Lorenzo Stoakes (ARM) wrote:
> This can be done faster in C, so implement scripts/basic/depcheck to do so.
> 
> It does as little work as possible, reading the .cmd files from a
> directory's targets and running stat on each dependency only a single time.

Doesn't this run the risk of missing transitive deps?

C.cmd:	C.o depends on A.c and B.c and B.h
B.cmd:	B.h depends on B.data and B.script

depcheck looks at C.o and B.h's times and is happy so it drops C.cmd file,
but then see B.script has changed compared to B.h, so it keeps B.cmd,
and then the build runs and doesn't have the C.o dep list any more,
and C.o goes unbuilt?

Also, I don't think this handles if_changed commands at all? The kbuild
rebuild condition is timestamps plus if_changed's command-line check.

> diff --git a/scripts/basic/depcheck.c b/scripts/basic/depcheck.c
> new file mode 100644
> index 000000000000..622101c33650
> --- /dev/null
> +++ b/scripts/basic/depcheck.c
> @@ -0,0 +1,441 @@
> [...]
> +/* What fixdep writes, see above. */
> +#define DEPS_PREFIX		"deps_"
> +#define DEPS_RULE_PREFIX	"$(" DEPS_PREFIX
> +#define RULE_SUFFIX		":"
> +#define LINE_CONTINUATION	" \\"
> +#define WILDCARD_OPEN		"$(wildcard "
> +#define WILDCARD_CLOSE		")"
> +#define CMD_SUFFIX		".cmd"
> [...]
> +static bool is_blank(char chr)
> [...]
> +static bool str_ends_with(const char *str, const char *suffix)
> [...]
> +static bool line_starts_with(const struct line *line, const char *prefix)
> [...]
> +static bool line_ends_with(const struct line *line, const char *suffix)
> [...]
> +static bool line_is_blank(const struct line *line)
> [...]
> +static bool line_is_continued(const struct line *line)
> [...]
> +static void line_strip_continuation(struct line *line)
> [...]
> +static void line_trim(struct line *line)
> [...]
> +static bool next_line(const char **pos, const char *end, struct line *line)

There is a lot of sting handling in here. I know you're going for speed,
but it just feels like a python script doing all this would be much more
readable without wrecking speed compared to C much. Yes, C will win out,
but the maintainability of this helper does not fill me with joy. :P

> [...]
> +static const struct dep *lookup_dep(const char *path)
> +{
> +	const unsigned int key = hash_str(path);
> +	struct dep *dep;
> +	struct stat st;
> +
> +	hash_for_each_possible(dep_table, dep, hnode, key) {
> +		if (!strcmp(dep->path, path))
> +			return dep;
> +	}
> +
> +	dep = xmalloc(sizeof(*dep) + strlen(path) + 1);
> +	strcpy(dep->path, path);
> +	dep->exists = !stat(path, &st);
> +	if (dep->exists)
> +		dep->mtime = st.st_mtim;

I realize you only check "mtime" after an "exists" check, but I bristle
at leaving a time-stamp uninitialized. Can we just add an "else
dep->mtime = 0" here?

-Kees

-- 
Kees Cook

  reply	other threads:[~2026-09-17 18:42 UTC|newest]

Thread overview: 47+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-09-17 16:06 [PATCH v3 00/20] kbuild: significantly speed up kernel builds Lorenzo Stoakes (ARM)
2026-09-17 16:06 ` [PATCH v3 01/20] kbuild: do not allocate .modinfo in vmlinux Lorenzo Stoakes (ARM)
2026-09-17 16:52   ` Kees Cook
2026-09-17 17:41     ` Lorenzo Stoakes (ARM)
2026-09-18  0:51     ` Nathan Chancellor
2026-09-17 17:05   ` Kees Cook
2026-09-17 17:39     ` Lorenzo Stoakes (ARM)
2026-09-17 16:06 ` [PATCH v3 02/20] kallsyms: index symbols by token to speed up table compression Lorenzo Stoakes (ARM)
2026-09-17 17:27   ` Kees Cook
2026-09-17 17:45     ` Lorenzo Stoakes (ARM)
2026-09-17 16:06 ` [PATCH v3 03/20] kallsyms: output binary data to speed output and kallsyms assembly Lorenzo Stoakes (ARM)
2026-09-17 17:36   ` Kees Cook
2026-09-17 16:06 ` [PATCH v3 04/20] kbuild: do not sort nm output where the order is irrelevant Lorenzo Stoakes (ARM)
2026-09-17 17:38   ` Kees Cook
2026-09-17 16:06 ` [PATCH v3 05/20] kbuild: only emit vmlinux relocations when required Lorenzo Stoakes (ARM)
2026-09-17 17:41   ` Kees Cook
2026-09-17 17:48     ` Lorenzo Stoakes (ARM)
2026-09-17 16:06 ` [PATCH v3 06/20] elf-parse: add section flags, symbol binding and a read-only mapping Lorenzo Stoakes (ARM)
2026-09-17 17:44   ` Kees Cook
2026-09-17 16:06 ` [PATCH v3 07/20] kallsyms: reimplement mksysmap in C Lorenzo Stoakes (ARM)
2026-09-17 18:07   ` Kees Cook
2026-09-17 16:06 ` [PATCH v3 08/20] kbuild: cache list, composite object state per object Lorenzo Stoakes (ARM)
2026-09-17 18:11   ` Kees Cook
2026-09-17 16:06 ` [PATCH v3 09/20] kbuild: implement and use depcheck to check dependency timestamps Lorenzo Stoakes (ARM)
2026-09-17 18:42   ` Kees Cook [this message]
2026-09-17 20:59     ` Kees Cook
2026-09-17 16:06 ` [PATCH v3 10/20] kbuild: move the toolchain checks into init/Kconfig.toolchain Lorenzo Stoakes (ARM)
2026-09-17 18:53   ` Kees Cook
2026-09-18  1:07     ` Nathan Chancellor
2026-09-17 16:06 ` [PATCH v3 11/20] kbuild: avoid re-running compiler and linker probes Lorenzo Stoakes (ARM)
2026-09-17 19:26   ` Kees Cook
2026-09-18  1:21     ` Nathan Chancellor
2026-09-17 16:06 ` [PATCH v3 12/20] modpost: cache section relocation mismatch state Lorenzo Stoakes (ARM)
2026-09-17 16:06 ` [PATCH v3 13/20] modpost: emit module descriptors as assembly Lorenzo Stoakes (ARM)
2026-09-17 16:06 ` [PATCH v3 14/20] kbuild: batch module finalisation Lorenzo Stoakes (ARM)
2026-09-17 17:01   ` Kees Cook
2026-09-17 16:06 ` [PATCH v3 15/20] objtool: cache relocations, do less work, eliminate relocation hash Lorenzo Stoakes (ARM)
2026-09-17 16:06 ` [PATCH v3 16/20] objtool: size the instruction hash to the text Lorenzo Stoakes (ARM)
2026-09-17 16:06 ` [PATCH v3 17/20] objtool: decode instructions and resolve branch targets in parallel Lorenzo Stoakes (ARM)
2026-09-17 16:06 ` [PATCH v3 18/20] rust: make exports.o depend on the headers generated for it Lorenzo Stoakes (ARM)
2026-09-17 16:06 ` [PATCH v3 19/20] kbuild: build rust crates in parallel with the rest of the build Lorenzo Stoakes (ARM)
2026-09-17 16:06 ` [PATCH v3 20/20] kbuild: compress the kernel with pigz if available Lorenzo Stoakes (ARM)
2026-09-17 16:58   ` Kees Cook
2026-09-17 17:15 ` [PATCH v3 00/20] kbuild: significantly speed up kernel builds Linus Torvalds
2026-09-17 17:36   ` Lorenzo Stoakes (ARM)
2026-09-17 19:42     ` Lorenzo Stoakes (ARM)
2026-09-17 20:02       ` Nick Desaulniers

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=202609171117.35BFDCC13@keescook \
    --to=kees@kernel.org \
    --cc=a.hindborg@kernel.org \
    --cc=acourbot@nvidia.com \
    --cc=alex@ghiti.fr \
    --cc=aliceryhl@google.com \
    --cc=aou@eecs.berkeley.edu \
    --cc=ardb@kernel.org \
    --cc=arnd@arndb.de \
    --cc=axboe@kernel.dk \
    --cc=bjorn3_gh@protonmail.com \
    --cc=boqun@kernel.org \
    --cc=bp@alien8.de \
    --cc=catalin.marinas@arm.com \
    --cc=corbet@lwn.net \
    --cc=dakr@kernel.org \
    --cc=daniel.almeida@collabora.com \
    --cc=dave.hansen@linux.intel.com \
    --cc=gary@garyguo.net \
    --cc=gustavoars@kernel.org \
    --cc=hpa@zytor.com \
    --cc=ilias.apalodimas@linaro.org \
    --cc=jpoimboe@kernel.org \
    --cc=justinstitt@google.com \
    --cc=legion@kernel.org \
    --cc=linux-arch@vger.kernel.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-efi@vger.kernel.org \
    --cc=linux-hardening@vger.kernel.org \
    --cc=linux-kbuild@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-riscv@lists.infradead.org \
    --cc=ljs@kernel.org \
    --cc=llvm@lists.linux.dev \
    --cc=lossin@kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=masahiroy@kernel.org \
    --cc=mingo@redhat.com \
    --cc=morbo@google.com \
    --cc=nathan@kernel.org \
    --cc=ndesaulniers@google.com \
    --cc=nsc@kernel.org \
    --cc=ojeda@kernel.org \
    --cc=palmer@dabbelt.com \
    --cc=peterz@infradead.org \
    --cc=pjw@kernel.org \
    --cc=rdunlap@infradead.org \
    --cc=rust-for-linux@vger.kernel.org \
    --cc=tamird@kernel.org \
    --cc=tglx@kernel.org \
    --cc=tmgross@umich.edu \
    --cc=torvalds@linux-foundation.org \
    --cc=will@kernel.org \
    --cc=work@onurozkan.dev \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®