* [PATCH v3 0/7] kbuild: record cmd-prereqs in .cmd files
@ 2026-10-12 21:33 Luis Augenstein
2026-10-12 21:33 ` [PATCH v3 1/7] scripts/make_fit: limit .cmd file parsing to the savedcmd_ line Luis Augenstein
` (6 more replies)
0 siblings, 7 replies; 8+ messages in thread
From: Luis Augenstein @ 2026-10-12 21:33 UTC (permalink / raw)
To: nathan, nsc
Cc: linux-kbuild, linux-kernel, akpm, gregkh, kstewart,
maximilian.huber, Luis Augenstein
KernelSbom (scripts/sbom/sbom.py) reconstructs the kernel build graph
from the .cmd files written by Kbuild. For most build outputs,
KernelSbom must infer the build inputs from the shell command stored in
savedcmd_*. This requires command-specific parsers, which add
unnecessary complexity and are difficult to maintain.
Kbuild already knows these inputs as $(real-prereqs), the non-phony
prerequisites of the Make rule. This series records them in .cmd files
using a new metadata field:
make_prereqs_<target> := <prerequisites>
KernelSbom is updated to read the new field instead of parsing the
shell command, and the obsolete command-specific parsers are removed.
The issue of manually parsing shell commands was originally mentioned
in https://lore.kernel.org/r/a01233b9-23a2-4666-91ed-f1cf030dcb9f@tngtech.com.
This series is a proposal to address that.
Cmd writers and consumers
=========================
The new field should not affect existing behavior.
Below is a summary of what I checked.
.cmd files are primarily written by cmd_and_savecmd and cmd_and_fixdep.
Both writers add make_prereqs.
The following rules append information to existing .cmd files.
None of these is affected by the new field:
- scripts/Makefile.lib: cmd_gen_objtooldep only appends new rule
target: $(wildcard ./tools/objtool/objtool)
- scripts/Makefile.build: gen_symversions only appends comments
#SYMVER <name> <crc>
- scripts/Makefile.lib: cmd_save_c_flags only appends a variable
saved_c_flags_<target> := <flags>
- rust/Makefile: cmd_gendwarfksyms only appends comments
#SYMVER <name> <crc>
- kernel/Makefile: kheaders_data_dep appends a deps_* block
deps_<target> := <headers>.
This series updates the following .cmd file consumers:
- scripts/sbom/sbom/cmd_graph/cmd_file.py: CmdFile.create() is
rewritten to parse the fields by name, including the new
make_prereqs field.
- scripts/make_fit.py: process_dtb() previously read and split the
complete .cmd file to find the fdtoverlay command. It is changed to
read only the first line, so it keeps working once the new field is
introduced.
The remaining consumers are not affected:
- Kbuild Makefiles: -include reads .cmd files as Makefile fragments. The
new field is an ordinary variable assignment not used in any dependency
rule.
- scripts/Makefile.thinlto: the saved_c_flags_* lookup only reads
saved_c_flags_<target> := <flags>.
- scripts/mod/modpost.c: extract_crcs_for_object() only reads comments
#SYMVER <name> <crc>.
- scripts/mod/sumversion.c: parse_source_files() only reads
source_<target> := <source> and deps_<target> := <dependencies>.
- scripts/clang-tools/gen_compile_commands.py: main() only reads the
first line containing savedcmd_<target> := <command>.
- scripts/generate_builtin_ranges.awk and
scripts/verify_builtin_ranges.awk: get_module_info() only reads the
first line and searches the saved command for DKBUILD_MODFILE or
RUST_MODFILE.
Alternative: extending the deps_* field
=======================================
An alternative to the new make_prereqs field is to merge $(real-prereqs)
into the existing deps_* field for both cmd_and_savecmd and cmd_and_fixdep.
This would avoid a new field and provide one dependency list. However, it
would broaden the meaning of deps_* from dependencies discovered through
generated .d files to include prerequisites already declared to Make.
This increases the risk of undesired side effects.
The writers listed above would need to ensure that each .cmd file
contains only one deps_* block. In particular, kheaders_data_dep would
need to merge its header dependencies into the block written by
cmd_and_savecmd instead of appending a second deps_* block.
cmd_and_fixdep, or fixdep.c depending on the implementation, would
likewise need to merge $(real-prereqs) with the generated dependencies.
Apart from KernelSbom, scripts/mod/sumversion.c is the only consumer
that explicitly reads deps_*. It uses selected dependency files to
calculate a module's srcversion. Extending deps_* could change this
module metadata. Such an unrelated change should probably be avoided.
Kbuild Makefiles also consume deps_* by including .cmd files. This
should not be an issue though, because $(real-prereqs) are already
present on the original target rule, so repeating them through deps_*
should not change rebuild behavior.
SBOM comparison
===============
I compared the SBOMs before and after the series for the same kernel
builds on x86 and arm64 allmodconfig. They are ~99% the same. The
following files are newly included in the SBOM because the manual
parsers did not cover them.
- Linker scripts: *.lds.S sources and the generated *.lds scripts.
The old tool never extracted -T arguments, so the whole
linker-script chain was missing.
- The intermediate vmlinux link steps: vmlinux.o,
.vmlinux.export.c/.o, the realmode pasyms.h. Previously vmlinux.a
was hardcoded as an input of scripts/link-vmlinux.sh. Now the real
chain vmlinux link -> vmlinux.o -> vmlinux.a is included.
- Per-module .mod object lists: the old tool expanded the .mod file
contents in place and recorded the objects directly as link inputs,
without explicitly including the .mod file in the SBOM.
- Generated headers: the drm/xe and msm register tables, the aic7xxx
sequence tables, the x32 syscall header, and module.lds.h. The old
tool found a header only where a .c file actually includes it via
the dep file; the new tool records the full make view, including
headers that are never included at all.
- Config and header archives: kernel/config_data.gz and
kernel/kheaders_data.tar.xz. The old tool missed them because their
generation commands had no matching parser.
- Devicetree overlay sources: the .dtso files are now recorded as
inputs of the dtb/dtbo build steps. The old tool saw only the
temporary files those steps produce, never the sources themselves.
The following files are removed:
- dtc preprocessing scratch files: .dts.tmp, .d.pre.tmp,
.d.dtc.tmp. The old tool recorded these temporary build by-products
as inputs; the new tool records the real .dtso source instead. This
is a cleanup, not a loss.
- Host tooling: the rust toolchain .md docs, the gen_smb1/2_mapping
perl generators, and the dtc host tool's libfdt headers.
Deliberately excluded. The SBOM should track what ends up in the image or
modules, not the tooling that produces it.
Changes in v3:
- split the changes into individual patches
- include the sbom changes directly in this series instead of deferring it
- add the SBOM comparison section to the cover letter
- Assisted-by trailer follows
Documentation/process/coding-assistants.rst
Changes in v2:
- filter depfile entries from make_prereqs in fixdep.c to ensure
make_prereqs is consistent between clean and incremental builds.
Filtering $(deps_$@) from previous builds is still required to not
include deps that got removed in the current depfile.
---
Luis Augenstein (7):
scripts/make_fit: limit .cmd file parsing to the savedcmd_ line
kbuild: refactor fixdep to collect dependencies before printing
scripts/sbom: skip make_prereqs lines when parsing .cmd files
kbuild: record cmd-prereqs in .cmd files
scripts/sbom: parse make_prereqs from .cmd files
scripts/sbom: use make_prereqs to determine build inputs
scripts/sbom: remove the command-specific parsers
Documentation/tools/sbom/sbom.rst | 22 +-
scripts/Kbuild.include | 10 +-
scripts/basic/fixdep.c | 163 +++++-
scripts/make_fit.py | 2 +-
scripts/sbom/sbom/cmd_graph/cmd_file.py | 152 ++----
scripts/sbom/sbom/cmd_graph/cmd_graph_node.py | 5 +-
.../sbom/sbom/cmd_graph/dependency_filter.py | 59 ++
.../sbom/cmd_graph/hardcoded_dependencies.py | 1 -
.../cmd_graph/savedcmd_parser/__init__.py | 6 -
.../command_parser_registry.py | 516 ------------------
.../savedcmd_parser/command_splitter.py | 128 -----
.../savedcmd_parser/savedcmd_parser.py | 67 ---
.../cmd_graph/savedcmd_parser/tokenizer.py | 92 ----
scripts/sbom/sbom/config.py | 14 -
scripts/sbom/sbom/spdx_graph/kernel_file.py | 64 ++-
.../tests/cmd_graph/test_savedcmd_parser.py | 468 ----------------
16 files changed, 296 insertions(+), 1473 deletions(-)
create mode 100644 scripts/sbom/sbom/cmd_graph/dependency_filter.py
delete mode 100644 scripts/sbom/sbom/cmd_graph/savedcmd_parser/__init__.py
delete mode 100644 scripts/sbom/sbom/cmd_graph/savedcmd_parser/command_parser_registry.py
delete mode 100644 scripts/sbom/sbom/cmd_graph/savedcmd_parser/command_splitter.py
delete mode 100644 scripts/sbom/sbom/cmd_graph/savedcmd_parser/savedcmd_parser.py
delete mode 100644 scripts/sbom/sbom/cmd_graph/savedcmd_parser/tokenizer.py
delete mode 100644 scripts/sbom/tests/cmd_graph/test_savedcmd_parser.py
--
2.43.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v3 1/7] scripts/make_fit: limit .cmd file parsing to the savedcmd_ line
2026-10-12 21:33 [PATCH v3 0/7] kbuild: record cmd-prereqs in .cmd files Luis Augenstein
@ 2026-10-12 21:33 ` Luis Augenstein
2026-10-12 21:33 ` [PATCH v3 2/7] kbuild: refactor fixdep to collect dependencies before printing Luis Augenstein
` (5 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Luis Augenstein @ 2026-10-12 21:33 UTC (permalink / raw)
To: nathan, nsc
Cc: linux-kbuild, linux-kernel, akpm, gregkh, kstewart,
maximilian.huber, Luis Augenstein
The command is always stored in the first line of the .cmd file. Read
only the first line, so remaining lines cannot get misparsed once
.cmd files record additional fields.
Assisted-by: LLM
Co-developed-by: Maximilian Huber <maximilian.huber@tngtech.com>
Signed-off-by: Maximilian Huber <maximilian.huber@tngtech.com>
Signed-off-by: Luis Augenstein <luis.augenstein@tngtech.com>
---
scripts/make_fit.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/make_fit.py b/scripts/make_fit.py
index 15ba26974fd..346e8a7ec12 100755
--- a/scripts/make_fit.py
+++ b/scripts/make_fit.py
@@ -288,7 +288,7 @@ def process_dtb(fname, args):
path, basename = os.path.split(fname)
cmd_fname = os.path.join(path, f'.{basename}.cmd')
with open(cmd_fname, 'r', encoding='ascii') as inf:
- cmd = inf.read()
+ cmd = inf.readline()
if 'scripts/dtc/fdtoverlay' in cmd:
# This depends on the structure of the composite DTB command
--
2.43.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v3 2/7] kbuild: refactor fixdep to collect dependencies before printing
2026-10-12 21:33 [PATCH v3 0/7] kbuild: record cmd-prereqs in .cmd files Luis Augenstein
2026-10-12 21:33 ` [PATCH v3 1/7] scripts/make_fit: limit .cmd file parsing to the savedcmd_ line Luis Augenstein
@ 2026-10-12 21:33 ` Luis Augenstein
2026-10-12 21:33 ` [PATCH v3 3/7] scripts/sbom: skip make_prereqs lines when parsing .cmd files Luis Augenstein
` (4 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Luis Augenstein @ 2026-10-12 21:33 UTC (permalink / raw)
To: nathan, nsc
Cc: linux-kbuild, linux-kernel, akpm, gregkh, kstewart,
maximilian.huber, Luis Augenstein
fixdep currently writes the .cmd file content while parsing the .dep
file. Separate the parsing from the printing: parse_dep_file() now
collects the source file and all dependencies in a list, and
print_dep_file() emits the assignments afterwards. No functional
change intended.
Assisted-by: LLM
Co-developed-by: Maximilian Huber <maximilian.huber@tngtech.com>
Signed-off-by: Maximilian Huber <maximilian.huber@tngtech.com>
Signed-off-by: Luis Augenstein <luis.augenstein@tngtech.com>
---
scripts/basic/fixdep.c | 109 ++++++++++++++++++++++++++++++++++-------
1 file changed, 92 insertions(+), 17 deletions(-)
diff --git a/scripts/basic/fixdep.c b/scripts/basic/fixdep.c
index 54063d98044..f514ef606ec 100644
--- a/scripts/basic/fixdep.c
+++ b/scripts/basic/fixdep.c
@@ -114,6 +114,17 @@ struct item {
char name[];
};
+struct dep_item {
+ struct dep_item *next;
+ char name[];
+};
+
+struct dep_info {
+ char *source;
+ struct dep_item *deps;
+ struct dep_item *last_dep;
+};
+
#define HASHSZ 256
static struct item *config_hashtab[HASHSZ], *file_hashtab[HASHSZ];
@@ -163,16 +174,42 @@ static bool in_hashtable(const char *name, int len, struct item *hashtab[])
return false;
}
+/*
+ * Append a dependency or include/config/<SYM> path to the info linked list.
+ */
+static void dep_info_add(struct dep_info *info, const char *name, int len)
+{
+ struct dep_item *dep = xmalloc(sizeof(*dep) + len + 1);
+
+ memcpy(dep->name, name, len);
+ dep->name[len] = '\0';
+ dep->next = NULL;
+
+ if (info->last_dep)
+ info->last_dep->next = dep;
+ else
+ info->deps = dep;
+ info->last_dep = dep;
+}
+
/*
* Record the use of a CONFIG_* word.
*/
-static void use_config(const char *m, int slen)
+static void use_config(const char *m, int slen, struct dep_info *info)
{
if (in_hashtable(m, slen, config_hashtab))
return;
- /* Print out a dependency path from a symbol name. */
- printf(" $(wildcard include/config/%.*s) \\\n", slen, m);
+ /* Build a dependency path from a symbol name. */
+ static const char config_path[] = "include/config/";
+ char *path;
+ int path_len = sizeof(config_path) - 1 + slen;
+
+ path = xmalloc(path_len);
+ memcpy(path, config_path, sizeof(config_path) - 1);
+ memcpy(path + sizeof(config_path) - 1, m, slen);
+ dep_info_add(info, path, path_len);
+ free(path);
}
/* test if s ends in sub */
@@ -186,7 +223,12 @@ static int str_ends_with(const char *s, int slen, const char *sub)
return !memcmp(s + slen - sublen, sub, sublen);
}
-static void parse_config_file(const char *p)
+/*
+ * Scan dependency p for CONFIG_ words, map each to an
+ * include/config/ path and append to the dep_info list if not
+ * already included.
+ */
+static void parse_config_file(const char *p, struct dep_info *info)
{
const char *q, *r;
const char *start = p;
@@ -205,7 +247,7 @@ static void parse_config_file(const char *p)
else
r = q;
if (r > p)
- use_config(p, r - p);
+ use_config(p, r - p, info);
p = q;
}
}
@@ -253,13 +295,9 @@ static int is_no_parse_file(const char *s, int len)
str_ends_with(s, len, ".so");
}
-/*
- * Important: The below generated source_foo.o and deps_foo.o variable
- * assignments are parsed not only by make, but also by the rather simple
- * parser in scripts/mod/sumversion.c.
- */
-static void parse_dep_file(char *p, const char *target)
+static struct dep_info parse_dep_file(char *p)
{
+ struct dep_info info = {};
bool saw_any_target = false;
bool is_target = true;
bool is_source = false;
@@ -375,13 +413,12 @@ static void parse_dep_file(char *p, const char *target)
*/
if (!saw_any_target) {
saw_any_target = true;
- printf("source_%s := %s\n\n", target, p);
- printf("deps_%s := \\\n", target);
+ info.source = xstrdup(p);
need_parse = true;
}
} else if (!is_ignored_file(p, q - p) &&
!in_hashtable(p, q - p, file_hashtab)) {
- printf(" %s \\\n", p);
+ dep_info_add(&info, p, q - p);
need_parse = true;
}
@@ -389,7 +426,7 @@ static void parse_dep_file(char *p, const char *target)
void *buf;
buf = read_file(p);
- parse_config_file(buf);
+ parse_config_file(buf, &info);
free(buf);
}
@@ -403,6 +440,40 @@ static void parse_dep_file(char *p, const char *target)
exit(1);
}
+ return info;
+}
+
+static void free_dep_info(struct dep_info *info)
+{
+ struct dep_item *dep, *next;
+
+ for (dep = info->deps; dep; dep = next) {
+ next = dep->next;
+ free(dep);
+ }
+ free(info->source);
+}
+
+/*
+ * Important: The below generated source_foo.o and deps_foo.o variable
+ * assignments are parsed not only by make, but also by the rather simple
+ * parser in scripts/mod/sumversion.c.
+ */
+static void print_dep_file(const char *target, const struct dep_info *info)
+{
+ static const char config_path[] = "include/config/";
+ const struct dep_item *dep;
+
+ printf("source_%s := %s\n\n", target, info->source);
+ printf("deps_%s := \\\n", target);
+
+ for (dep = info->deps; dep; dep = dep->next) {
+ if (!strncmp(dep->name, config_path, sizeof(config_path) - 1))
+ printf(" $(wildcard %s) \\\n", dep->name);
+ else
+ printf(" %s \\\n", dep->name);
+ }
+
printf("\n%s: $(deps_%s)\n\n", target, target);
printf("$(deps_%s):\n", target);
}
@@ -410,6 +481,7 @@ static void parse_dep_file(char *p, const char *target)
int main(int argc, char *argv[])
{
const char *depfile, *target, *cmdline;
+ struct dep_info info;
void *buf;
if (argc != 4)
@@ -419,10 +491,13 @@ int main(int argc, char *argv[])
target = argv[2];
cmdline = argv[3];
+ buf = read_file(depfile);
+ info = parse_dep_file(buf);
+
printf("savedcmd_%s := %s\n\n", target, cmdline);
+ print_dep_file(target, &info);
- buf = read_file(depfile);
- parse_dep_file(buf, target);
+ free_dep_info(&info);
free(buf);
fflush(stdout);
--
2.43.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v3 3/7] scripts/sbom: skip make_prereqs lines when parsing .cmd files
2026-10-12 21:33 [PATCH v3 0/7] kbuild: record cmd-prereqs in .cmd files Luis Augenstein
2026-10-12 21:33 ` [PATCH v3 1/7] scripts/make_fit: limit .cmd file parsing to the savedcmd_ line Luis Augenstein
2026-10-12 21:33 ` [PATCH v3 2/7] kbuild: refactor fixdep to collect dependencies before printing Luis Augenstein
@ 2026-10-12 21:33 ` Luis Augenstein
2026-10-12 21:33 ` [PATCH v3 4/7] kbuild: record cmd-prereqs in " Luis Augenstein
` (3 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Luis Augenstein @ 2026-10-12 21:33 UTC (permalink / raw)
To: nathan, nsc
Cc: linux-kbuild, linux-kernel, akpm, gregkh, kstewart,
maximilian.huber, Luis Augenstein
kbuild will record the prerequisites of a target in a new
make_prereqs_<target> field in .cmd files. Skip such lines so the
parser keeps working once the field is introduced.
Assisted-by: LLM
Co-developed-by: Maximilian Huber <maximilian.huber@tngtech.com>
Signed-off-by: Maximilian Huber <maximilian.huber@tngtech.com>
Signed-off-by: Luis Augenstein <luis.augenstein@tngtech.com>
---
scripts/sbom/sbom/cmd_graph/cmd_file.py | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/scripts/sbom/sbom/cmd_graph/cmd_file.py b/scripts/sbom/sbom/cmd_graph/cmd_file.py
index dcd63e284a3..08819b4d117 100644
--- a/scripts/sbom/sbom/cmd_graph/cmd_file.py
+++ b/scripts/sbom/sbom/cmd_graph/cmd_file.py
@@ -50,6 +50,10 @@ class CmdFile:
with open(cmd_file_path, "rt", encoding="utf-8") as f:
lines = [line.strip() for line in f.readlines() if line.strip() != "" and not line.startswith("#")]
+ # make_prereqs_* is recorded for future use. Ignore it for now to
+ # preserve the existing parser behavior.
+ lines = [line for line in lines if not line.startswith("make_prereqs_")]
+
# savedcmd
match = SAVEDCMD_PATTERN.match(lines[0] if lines else "")
if match is None:
--
2.43.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v3 4/7] kbuild: record cmd-prereqs in .cmd files
2026-10-12 21:33 [PATCH v3 0/7] kbuild: record cmd-prereqs in .cmd files Luis Augenstein
` (2 preceding siblings ...)
2026-10-12 21:33 ` [PATCH v3 3/7] scripts/sbom: skip make_prereqs lines when parsing .cmd files Luis Augenstein
@ 2026-10-12 21:33 ` Luis Augenstein
2026-10-12 21:33 ` [PATCH v3 5/7] scripts/sbom: parse make_prereqs from " Luis Augenstein
` (2 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Luis Augenstein @ 2026-10-12 21:33 UTC (permalink / raw)
To: nathan, nsc
Cc: linux-kbuild, linux-kernel, akpm, gregkh, kstewart,
maximilian.huber, Luis Augenstein
Record $(cmd-prereqs), i.e. $(real-prereqs) with the deps_*
dependencies filtered out, in a new .cmd file metadata field:
make_prereqs_<target> := <prerequisites>
On an incremental build, deps_<target> holds the dependencies
recorded from the depfile of the previous build. Those are filtered
out from $(real-prereqs), even if they no longer appear in the
current depfile. fixdep additionally filters the dependencies
collected from the current depfile, so that they are not recorded in
both deps_<target> and make_prereqs_<target>. This keeps the field
consistent between clean and incremental builds.
Link: https://lore.kernel.org/r/a01233b9-23a2-4666-91ed-f1cf030dcb9f@tngtech.com
Assisted-by: LLM
Co-developed-by: Maximilian Huber <maximilian.huber@tngtech.com>
Signed-off-by: Maximilian Huber <maximilian.huber@tngtech.com>
Signed-off-by: Luis Augenstein <luis.augenstein@tngtech.com>
---
scripts/Kbuild.include | 10 ++++++--
scripts/basic/fixdep.c | 54 +++++++++++++++++++++++++++++++++++++-----
2 files changed, 56 insertions(+), 8 deletions(-)
diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include
index 8c311b997e2..7790519f92f 100644
--- a/scripts/Kbuild.include
+++ b/scripts/Kbuild.include
@@ -181,6 +181,9 @@ endif
# (needed for the shell)
make-cmd = $(call escsq,$(subst $(pound),$$(pound),$(subst $$,$$$$,$(cmd_$(1)))))
+# prerequisites to record in .cmd files, excluding those covered in deps_*
+cmd-prereqs = $(call escsq,$(filter-out $(deps_$@), $(real-prereqs)))
+
# Find any prerequisites that are newer than target or that do not exist.
# PHONY targets skipped in both cases.
# If there is no prerequisite other than phony targets, $(newer-prereqs) becomes
@@ -198,14 +201,17 @@ if_changed = $(if $(if-changed-cond),$(cmd_and_savecmd),@:)
cmd_and_savecmd = \
$(cmd); \
- printf '%s\n' 'savedcmd_$@ := $(make-cmd)' > $(dot-target).cmd
+ printf '%s\n' \
+ 'savedcmd_$@ := $(make-cmd)' '' \
+ 'make_prereqs_$@ := $(cmd-prereqs)' > $(dot-target).cmd
# Execute the command and also postprocess generated .d dependencies file.
if_changed_dep = $(if $(if-changed-cond),$(cmd_and_fixdep),@:)
cmd_and_fixdep = \
$(cmd); \
- $(objtree)/scripts/basic/fixdep $(depfile) $@ '$(make-cmd)' > $(dot-target).cmd;\
+ $(objtree)/scripts/basic/fixdep $(depfile) $@ '$(make-cmd)' \
+ '$(cmd-prereqs)' > $(dot-target).cmd; \
rm -f $(depfile)
# Usage: $(call if_changed_rule,foo)
diff --git a/scripts/basic/fixdep.c b/scripts/basic/fixdep.c
index f514ef606ec..005c1c3df49 100644
--- a/scripts/basic/fixdep.c
+++ b/scripts/basic/fixdep.c
@@ -62,15 +62,17 @@
*
* It is invoked as
*
- * fixdep <depfile> <target> <cmdline>
+ * fixdep <depfile> <target> <cmdline> <prereqs>
*
* and will read the dependency file <depfile>
*
* The transformed dependency snipped is written to stdout.
*
- * It first generates a line
+ * It first generates the lines
*
- * savedcmd_<target> = <cmdline>
+ * savedcmd_<target> := <cmdline>
+ *
+ * make_prereqs_<target> := <prereqs_without_deps>
*
* and then basically copies the .<target>.d file to stdout, in the
* process filtering out the dependency on autoconf.h and adding
@@ -103,7 +105,7 @@
static void usage(void)
{
- fprintf(stderr, "Usage: fixdep <depfile> <target> <cmdline>\n");
+ fprintf(stderr, "Usage: fixdep <depfile> <target> <cmdline> <prereqs>\n");
exit(1);
}
@@ -478,23 +480,63 @@ static void print_dep_file(const char *target, const struct dep_info *info)
printf("$(deps_%s):\n", target);
}
+/*
+ * Print make_prereqs_<target> from prereqs, excluding the deps listed
+ * in info.
+ */
+static void print_make_prereqs(const char *target, const char *prereqs,
+ const struct dep_info *info)
+{
+ struct item *seen_prereqs_hashtab[HASHSZ] = {};
+ const struct dep_item *dep;
+ const char *p = prereqs;
+
+ for (dep = info->deps; dep; dep = dep->next)
+ in_hashtable(dep->name, strlen(dep->name),
+ seen_prereqs_hashtab);
+
+ printf("make_prereqs_%s :=", target);
+
+ while (*p) {
+ const char *start;
+ int len;
+
+ while (*p == ' ' || *p == '\t')
+ p++;
+ if (!*p)
+ break;
+
+ start = p;
+ while (*p && *p != ' ' && *p != '\t')
+ p++;
+ len = p - start;
+
+ if (!in_hashtable(start, len, seen_prereqs_hashtab))
+ printf(" %.*s", len, start);
+ }
+
+ printf("\n\n");
+}
+
int main(int argc, char *argv[])
{
- const char *depfile, *target, *cmdline;
+ const char *depfile, *target, *cmdline, *prereqs;
struct dep_info info;
void *buf;
- if (argc != 4)
+ if (argc != 5)
usage();
depfile = argv[1];
target = argv[2];
cmdline = argv[3];
+ prereqs = argv[4];
buf = read_file(depfile);
info = parse_dep_file(buf);
printf("savedcmd_%s := %s\n\n", target, cmdline);
+ print_make_prereqs(target, prereqs, &info);
print_dep_file(target, &info);
free_dep_info(&info);
--
2.43.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v3 5/7] scripts/sbom: parse make_prereqs from .cmd files
2026-10-12 21:33 [PATCH v3 0/7] kbuild: record cmd-prereqs in .cmd files Luis Augenstein
` (3 preceding siblings ...)
2026-10-12 21:33 ` [PATCH v3 4/7] kbuild: record cmd-prereqs in " Luis Augenstein
@ 2026-10-12 21:33 ` Luis Augenstein
2026-10-12 21:33 ` [PATCH v3 6/7] scripts/sbom: use make_prereqs to determine build inputs Luis Augenstein
2026-10-12 21:33 ` [PATCH v3 7/7] scripts/sbom: remove the command-specific parsers Luis Augenstein
6 siblings, 0 replies; 8+ messages in thread
From: Luis Augenstein @ 2026-10-12 21:33 UTC (permalink / raw)
To: nathan, nsc
Cc: linux-kbuild, linux-kernel, akpm, gregkh, kstewart,
maximilian.huber, Luis Augenstein
Add support for parsing the make_prereqs_<target> field recorded by
kbuild in .cmd files.
Refactor CmdFile.create() to collect the required fields by name
instead of deriving the file structure from the number of lines and
the position.
The source_ entry is no longer parsed, because it is always also
included in make_prereqs_<target> and therefore redundant for
KernelSbom.
Assisted-by: LLM
Co-developed-by: Maximilian Huber <maximilian.huber@tngtech.com>
Signed-off-by: Maximilian Huber <maximilian.huber@tngtech.com>
Signed-off-by: Luis Augenstein <luis.augenstein@tngtech.com>
---
scripts/sbom/sbom/cmd_graph/cmd_file.py | 103 +++++++++---------------
1 file changed, 36 insertions(+), 67 deletions(-)
diff --git a/scripts/sbom/sbom/cmd_graph/cmd_file.py b/scripts/sbom/sbom/cmd_graph/cmd_file.py
index 08819b4d117..cc576c7c6be 100644
--- a/scripts/sbom/sbom/cmd_graph/cmd_file.py
+++ b/scripts/sbom/sbom/cmd_graph/cmd_file.py
@@ -10,97 +10,66 @@ import sbom.sbom_logging as sbom_logging
from sbom.path_utils import PathStr
SAVEDCMD_PATTERN = re.compile(r"^(saved)?cmd_.*?:=\s*(?P<full_command>.+)$")
-SOURCE_PATTERN = re.compile(r"^source.*?:=\s*(?P<source_file>.+)$")
+MAKE_PREREQS_PATTERN = re.compile(r"^make_prereqs_.*?:=\s*(?P<prereqs>.+)$")
+DEPS_ASSIGNMENT_PATTERN = re.compile(r"^deps_.*?:=")
@dataclass
class CmdFile:
cmd_file_path: PathStr
savedcmd: str
- source: PathStr | None = None
+ make_prereqs: list[str] = field(default_factory=list)
deps: list[str] = field(default_factory=list)
- make_rules: list[str] = field(default_factory=list)
@classmethod
def create(cls, cmd_file_path: PathStr) -> "CmdFile | None":
- """
- Parses a .cmd file.
- .cmd files are assumed to have one of the following structures:
- 1. Full Cmd File
- (saved)?cmd_<output> := <command>
- source_<output> := <main_input>
- deps_<output> := \
- <dependencies>
- <output> := $(deps_<output>)
- $(deps_<output>):
-
- 2. Command Only Cmd File
- (saved)?cmd_<output> := <command>
-
- 3. Single Dependency Cmd File
- (saved)?cmd_<output> := <command>
- <output> : <dependency>
+ r"""
+ Parse the following fields from a `.cmd` file:
+
+ * `(saved)?cmd_<target> := <shell command>` (required) —
+ the shell command Kbuild ran to produce the build target.
+ * `make_prereqs_<target> := <prerequisites>` (required but can be empty) —
+ the non-phony make prerequisites.
+ * `deps_<target> := \` (optional) —
+ additional dependencies not covered by make prerequisites, e.g., headers.
Args:
- cmd_file_path (Path): absolute Path to a .cmd file
+ cmd_file_path (Path): absolute Path to a `.cmd` file.
Returns:
- cmd_file (CmdFile): Parsed cmd file.
+ cmd_file (CmdFile): Parsed `.cmd` file or `None` if no `savedcmd` line was found.
"""
with open(cmd_file_path, "rt", encoding="utf-8") as f:
lines = [line.strip() for line in f.readlines() if line.strip() != "" and not line.startswith("#")]
- # make_prereqs_* is recorded for future use. Ignore it for now to
- # preserve the existing parser behavior.
- lines = [line for line in lines if not line.startswith("make_prereqs_")]
+ savedcmd: str | None = None
+ make_prereqs: list[str] = []
+ deps: list[str] = []
+
+ for i, line in enumerate(lines):
+ match = SAVEDCMD_PATTERN.match(line)
+ if match is not None:
+ savedcmd = match.group("full_command")
+ continue
+
+ match = MAKE_PREREQS_PATTERN.match(line)
+ if match is not None:
+ make_prereqs = match.group("prereqs").split()
+ continue
- # savedcmd
- match = SAVEDCMD_PATTERN.match(lines[0] if lines else "")
- if match is None:
+ if DEPS_ASSIGNMENT_PATTERN.match(line):
+ j = i + 1
+ while j < len(lines) and lines[j].endswith("\\"):
+ deps.append(lines[j].removesuffix("\\").strip())
+ j += 1
+
+ if savedcmd is None:
sbom_logging.error(
"Skip parsing '{cmd_file_path}' because no 'savedcmd_' command was found.", cmd_file_path=cmd_file_path
)
return None
- savedcmd = match.group("full_command")
-
- # Command Only Cmd File
- if len(lines) == 1:
- return CmdFile(cmd_file_path, savedcmd)
-
- # Single Dependency Cmd File
- if len(lines) == 2:
- parts = lines[1].split(":", 1)
- if len(parts) != 2:
- sbom_logging.error(
- "Skip parsing '{cmd_file_path}'. Expected dependency line '<output>: <dependency>' but got {second_line}", cmd_file_path=cmd_file_path, second_line=lines[1]
- )
- return None
- dep = parts[1].strip()
- return CmdFile(cmd_file_path, savedcmd, deps=[dep])
-
- # Full Cmd File
- # source
- line1 = SOURCE_PATTERN.match(lines[1])
- if line1 is None:
- sbom_logging.error(
- "Skip parsing '{cmd_file_path}' because no 'source_' entry was found.", cmd_file_path=cmd_file_path
- )
- return CmdFile(cmd_file_path, savedcmd)
- source = line1.group("source_file")
-
- # deps
- deps: list[str] = []
- i = 3 # lines[2] includes the variable assignment but no actual dependency, so we need to start at lines[3].
- while i < len(lines):
- if not lines[i].endswith("\\"):
- break
- deps.append(lines[i][:-1].strip())
- i += 1
-
- # make_rules
- make_rules = lines[i:]
- return CmdFile(cmd_file_path, savedcmd, source, deps, make_rules)
+ return CmdFile(cmd_file_path, savedcmd, make_prereqs, deps)
def get_dependencies(
self: "CmdFile", target_path: PathStr, obj_tree: PathStr, fail_on_unknown_build_command: bool
--
2.43.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v3 6/7] scripts/sbom: use make_prereqs to determine build inputs
2026-10-12 21:33 [PATCH v3 0/7] kbuild: record cmd-prereqs in .cmd files Luis Augenstein
` (4 preceding siblings ...)
2026-10-12 21:33 ` [PATCH v3 5/7] scripts/sbom: parse make_prereqs from " Luis Augenstein
@ 2026-10-12 21:33 ` Luis Augenstein
2026-10-12 21:33 ` [PATCH v3 7/7] scripts/sbom: remove the command-specific parsers Luis Augenstein
6 siblings, 0 replies; 8+ messages in thread
From: Luis Augenstein @ 2026-10-12 21:33 UTC (permalink / raw)
To: nathan, nsc
Cc: linux-kbuild, linux-kernel, akpm, gregkh, kstewart,
maximilian.huber, Luis Augenstein
Build the list of file dependencies of a target from the
make_prereqs_<target> field and the deps_<target> entries of the .cmd
file instead of parsing the shell command stored in savedcmd_.
The prerequisites include more than KernelSbom wants to track: files
should be tracked if they end up in the kernel image or loadable
modules, but not each and every host tool or helper script used to
build them. Add a manually maintained exclude list for such
dependencies. It is still manual, but much easier to maintain than
the command-specific parsers that were needed to extract the relevant
inputs from each individual build command.
Assisted-by: LLM
Co-developed-by: Maximilian Huber <maximilian.huber@tngtech.com>
Signed-off-by: Maximilian Huber <maximilian.huber@tngtech.com>
Signed-off-by: Luis Augenstein <luis.augenstein@tngtech.com>
---
scripts/sbom/sbom/cmd_graph/cmd_file.py | 46 ++-----------
.../sbom/sbom/cmd_graph/dependency_filter.py | 59 +++++++++++++++++
scripts/sbom/sbom/spdx_graph/kernel_file.py | 64 ++++++++++++-------
3 files changed, 105 insertions(+), 64 deletions(-)
create mode 100644 scripts/sbom/sbom/cmd_graph/dependency_filter.py
diff --git a/scripts/sbom/sbom/cmd_graph/cmd_file.py b/scripts/sbom/sbom/cmd_graph/cmd_file.py
index cc576c7c6be..ab4b00b252b 100644
--- a/scripts/sbom/sbom/cmd_graph/cmd_file.py
+++ b/scripts/sbom/sbom/cmd_graph/cmd_file.py
@@ -5,7 +5,7 @@ import os
import re
from dataclasses import dataclass, field
from sbom.cmd_graph.deps_parser import parse_cmd_file_deps
-from sbom.cmd_graph.savedcmd_parser import parse_inputs_from_commands
+from sbom.cmd_graph.dependency_filter import should_include_dependency
import sbom.sbom_logging as sbom_logging
from sbom.path_utils import PathStr
@@ -85,51 +85,17 @@ class CmdFile:
Returns:
list[PathStr]: dependency file paths relative to `obj_tree`.
"""
- input_files: list[PathStr] = [
- str(p) for p in parse_inputs_from_commands(self.savedcmd, fail_on_unknown_build_command)
- ]
+ input_files: list[PathStr] = list(self.make_prereqs)
+
if self.deps:
input_files += [str(p) for p in parse_cmd_file_deps(self.deps)]
- input_files = _expand_resolve_files(input_files, obj_tree)
cmd_file_dependencies: list[PathStr] = []
for input_file in input_files:
- # input files are either absolute or relative to the object tree
+ # input files are either absolute or already relative to the object tree
if os.path.isabs(input_file):
input_file = os.path.relpath(input_file, obj_tree)
- if input_file == target_path:
- # Skip target file to prevent cycles. This is necessary because some multi stage commands first create an output and then pass it as input to the next command, e.g., objcopy.
- continue
- cmd_file_dependencies.append(input_file)
+ if should_include_dependency(input_file):
+ cmd_file_dependencies.append(input_file)
unique_cmd_file_dependencies = list(dict.fromkeys(cmd_file_dependencies))
return unique_cmd_file_dependencies
-
-
-def _expand_resolve_files(input_files: list[PathStr], obj_tree: PathStr) -> list[PathStr]:
- """
- Expands resolve files which may reference additional files via '@' notation.
-
- Args:
- input_files (list[PathStr]): List of file paths relative to the object tree, where paths starting with '@' refer to files
- containing further file paths, each on a separate line.
- obj_tree: Absolute path to the root of the object tree.
-
- Returns:
- list[PathStr]: Flattened list of all input file paths, with any nested '@' file references resolved recursively.
- """
- expanded_input_files: list[PathStr] = []
- for input_file in input_files:
- if not input_file.startswith("@"):
- expanded_input_files.append(input_file)
- continue
- resolve_file_path = os.path.join(obj_tree, input_file.removeprefix("@"))
- if not os.path.exists(resolve_file_path):
- sbom_logging.error(
- "Skip resolving '{resolve_file_path}' because the response file does not exist.",
- resolve_file_path=resolve_file_path,
- )
- continue
- with open(resolve_file_path, "rt", encoding="utf-8") as f:
- resolve_file_content = [line_stripped for line in f.readlines() if (line_stripped := line.strip())]
- expanded_input_files += _expand_resolve_files(resolve_file_content, obj_tree)
- return expanded_input_files
diff --git a/scripts/sbom/sbom/cmd_graph/dependency_filter.py b/scripts/sbom/sbom/cmd_graph/dependency_filter.py
new file mode 100644
index 00000000000..f1fdc2cb965
--- /dev/null
+++ b/scripts/sbom/sbom/cmd_graph/dependency_filter.py
@@ -0,0 +1,59 @@
+# SPDX-License-Identifier: GPL-2.0-only OR MIT
+# Copyright (C) 2025 TNG Technology Consulting GmbH
+
+import re
+
+from sbom.path_utils import PathStr
+
+# Regex patterns for dependencies that should not be tracked in the dependency graph.
+_COMPILED_EXCLUDE_PATTERNS: tuple[re.Pattern[str], ...] = (
+ # scripts in source tree (except module-common.c, which is compiled into every module)
+ re.compile(r"^\.\./scripts/(?!module-common\.c$)"),
+ # scripts and tools in obj tree
+ re.compile(r"^scripts/"),
+ re.compile(r"^tools/"),
+ re.compile(r"^arch/[^/]+/tools/"),
+ # Host helper scripts
+ re.compile(r".*\.sh$"),
+ re.compile(r".*\.py$"),
+ re.compile(r".*\.pl$"),
+ # Host-tool executables
+ re.compile(r"^certs/extract-cert$"),
+ re.compile(r"^security/selinux/genheaders$"),
+ re.compile(r"^usr/gen_init_cpio$"),
+ re.compile(r"^drivers/gpu/drm/radeon/mkregtable$"),
+ re.compile(r"^drivers/video/logo/pnmtologo$"),
+ re.compile(r"^lib/raid/raid6/mktables$"),
+ re.compile(r"^\.\./fs/smb/client/gen_smb1_mapping$"),
+ re.compile(r"^\.\./fs/smb/client/gen_smb2_mapping$"),
+ # Other
+ re.compile(r".*\.xsd$"),
+ re.compile(r".*purgatory\.chk$"),
+ re.compile(r".*\.genkey$"),
+ re.compile(r".*\.md$"),
+ re.compile(r"^include/config/"),
+ # x86
+ re.compile(r"^arch/x86/boot/compressed/mkpiggy$"),
+ re.compile(r"^arch/x86/boot/mkcpustr$"),
+ # arm
+ re.compile(r"^arch/arm/vdso/vdsomunge$"),
+ # arm64
+ re.compile(r"^arch/arm64/kvm/hyp/nvhe/gen-hyprel$"),
+ re.compile(r"^arch/arm64/kernel/pi/relacheck$"),
+)
+
+def should_include_dependency(path: PathStr) -> bool:
+ """
+ Return whether a `.cmd` dependency should appear in the tracked dependency graph.
+
+ Track files that are inputs to compiling or linking artifacts that end up in the
+ kernel image or loadable modules (e.g. source → object → archive → image/module).
+ Exclude host tools and helper scripts whose code is not linked into the kernel.
+
+ Args:
+ path: Dependency path relative to the object tree.
+
+ Returns:
+ True if the dependency should be included in the dependency graph.
+ """
+ return not any(pattern.match(path) for pattern in _COMPILED_EXCLUDE_PATTERNS)
diff --git a/scripts/sbom/sbom/spdx_graph/kernel_file.py b/scripts/sbom/sbom/spdx_graph/kernel_file.py
index 505f25f66eb..cea14cb3538 100644
--- a/scripts/sbom/sbom/spdx_graph/kernel_file.py
+++ b/scripts/sbom/sbom/spdx_graph/kernel_file.py
@@ -245,7 +245,7 @@ def _get_primary_purpose(absolute_path: PathStr) -> SoftwarePurpose | None:
return any(segment in absolute_path for segment in path_segments)
# Source code
- if ends_with([".c", ".h", ".S", ".s", ".rs", ".pl", "gen_smb1_mapping", "gen_smb2_mapping"]):
+ if ends_with([".c", ".h", ".S", ".s", ".rs"]):
return "source"
# Libraries
@@ -271,45 +271,61 @@ def _get_primary_purpose(absolute_path: PathStr) -> SoftwarePurpose | None:
# Data files
if ends_with(
[
- ".tbl",
- ".relocs",
- ".rmeta",
- ".in",
+ ".awk",
".dbg",
- ".x509",
- ".pbm",
- ".ppm",
".dtb",
- ".uc",
- ".inc",
+ ".dtbo",
".dts",
".dtsi",
- ".dtbo",
- ".xml",
+ ".dtso",
+ ".in",
+ ".inc",
+ ".json",
+ ".ld",
+ ".lds",
+ ".mod",
+ ".order",
+ ".pbm",
+ ".ppm",
+ ".relocs",
+ ".rmeta",
".ro",
- "initramfs_inc_data",
- "default_cpio_list",
- "x509_certificate_list",
- "utf8data.c_shipped",
+ ".tbl",
+ ".uc",
+ ".x509",
+ ".xml",
"blacklist_hash_list",
- "x509_revocation_list",
"cpucaps",
- "sysreg",
+ "default_cpio_list",
+ "initramfs_inc_data",
+ "kheaders.md5",
+ "kheaders-objlist",
+ "kheaders-srclist",
"mach-types",
+ "sysreg",
+ "utf8data.c_shipped",
+ "x509_certificate_list",
+ "x509_revocation_list",
]
) or includes_path_segments(["drivers/gpu/drm/radeon/reg_srcs/"]):
return "data"
# Configuration files
- if ends_with([".pem", ".key", ".conf", ".config", ".cfg", ".bconf"]):
+ if ends_with([
+ ".pem",
+ ".key",
+ ".conf",
+ ".config",
+ ".cfg",
+ ".bconf",
+ "config_data",
+ ".genkey",
+ "bindgen_parameters",
+ ]):
return "configuration"
- # Documentation
- if ends_with([".md"]):
- return "documentation"
-
# Other / miscellaneous
- if ends_with([".o", ".tmp"]):
+ if ends_with([".o"]):
return "other"
sbom_logging.warning("Could not infer primary purpose for {absolute_path}", absolute_path=absolute_path)
--
2.43.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v3 7/7] scripts/sbom: remove the command-specific parsers
2026-10-12 21:33 [PATCH v3 0/7] kbuild: record cmd-prereqs in .cmd files Luis Augenstein
` (5 preceding siblings ...)
2026-10-12 21:33 ` [PATCH v3 6/7] scripts/sbom: use make_prereqs to determine build inputs Luis Augenstein
@ 2026-10-12 21:33 ` Luis Augenstein
6 siblings, 0 replies; 8+ messages in thread
From: Luis Augenstein @ 2026-10-12 21:33 UTC (permalink / raw)
To: nathan, nsc
Cc: linux-kbuild, linux-kernel, akpm, gregkh, kstewart,
maximilian.huber, Luis Augenstein
With the build inputs taken from make_prereqs_ and deps_, the
command-specific parsers are no longer needed. Remove them, and with
them the --do-not-fail-on-unknown-build-command option, which only
existed to tolerate build commands that the parsers did not support.
Assisted-by: LLM
Co-developed-by: Maximilian Huber <maximilian.huber@tngtech.com>
Signed-off-by: Maximilian Huber <maximilian.huber@tngtech.com>
Signed-off-by: Luis Augenstein <luis.augenstein@tngtech.com>
---
Documentation/tools/sbom/sbom.rst | 22 +-
scripts/sbom/sbom/cmd_graph/cmd_file.py | 5 +-
scripts/sbom/sbom/cmd_graph/cmd_graph_node.py | 5 +-
.../sbom/cmd_graph/hardcoded_dependencies.py | 1 -
.../cmd_graph/savedcmd_parser/__init__.py | 6 -
.../command_parser_registry.py | 516 ------------------
.../savedcmd_parser/command_splitter.py | 128 -----
.../savedcmd_parser/savedcmd_parser.py | 67 ---
.../cmd_graph/savedcmd_parser/tokenizer.py | 92 ----
scripts/sbom/sbom/config.py | 14 -
.../tests/cmd_graph/test_savedcmd_parser.py | 468 ----------------
11 files changed, 5 insertions(+), 1319 deletions(-)
delete mode 100644 scripts/sbom/sbom/cmd_graph/savedcmd_parser/__init__.py
delete mode 100644 scripts/sbom/sbom/cmd_graph/savedcmd_parser/command_parser_registry.py
delete mode 100644 scripts/sbom/sbom/cmd_graph/savedcmd_parser/command_splitter.py
delete mode 100644 scripts/sbom/sbom/cmd_graph/savedcmd_parser/savedcmd_parser.py
delete mode 100644 scripts/sbom/sbom/cmd_graph/savedcmd_parser/tokenizer.py
delete mode 100644 scripts/sbom/tests/cmd_graph/test_savedcmd_parser.py
diff --git a/Documentation/tools/sbom/sbom.rst b/Documentation/tools/sbom/sbom.rst
index 029b08b6ad8..aada5f4875c 100644
--- a/Documentation/tools/sbom/sbom.rst
+++ b/Documentation/tools/sbom/sbom.rst
@@ -121,9 +121,9 @@ from multiple sources:
* **.cmd files**: The primary source is the ``.cmd`` file of the
generated artifact, e.g., ``arch/x86/boot/.bzImage.cmd``. These files
- contain the exact command used to build the artifact and often include
- an explicit list of input dependencies. By parsing the ``.cmd``
- file, the full list of dependencies can be obtained.
+ record the shell command used to build the artifact, the make
+ prerequisites (``make_prereqs_*``), and, for compiler-generated targets,
+ additional dependencies (``deps_*``) like header files.
* **.incbin statements**: The second source are include binary
``.incbin`` statements in ``.S`` assembly files.
@@ -188,19 +188,3 @@ In this scenario, KernelSbom does not produce a dedicated
artifacts are included together in ``sbom-build.spdx.json``, and
``sbom.used-files.txt`` lists all files referenced in the build document.
-Unknown Build Commands
-~~~~~~~~~~~~~~~~~~~~~~
-
-Because the kernel supports a wide range of configurations and versions,
-KernelSbom may encounter build commands in ``.cmd`` files that it does
-not yet support. By default, KernelSbom will fail if an unknown build
-command is encountered.
-
-If you still wish to generate SPDX documents despite unsupported
-commands, you can use the ``--do-not-fail-on-unknown-build-command``
-option. KernelSbom will continue and produce the documents, although
-the resulting SBOM will be incomplete.
-
-This option should only be used when the missing portion of the
-dependency graph is small and an incomplete SBOM is acceptable for
-your use case.
diff --git a/scripts/sbom/sbom/cmd_graph/cmd_file.py b/scripts/sbom/sbom/cmd_graph/cmd_file.py
index ab4b00b252b..1c88854faf5 100644
--- a/scripts/sbom/sbom/cmd_graph/cmd_file.py
+++ b/scripts/sbom/sbom/cmd_graph/cmd_file.py
@@ -71,16 +71,13 @@ class CmdFile:
return CmdFile(cmd_file_path, savedcmd, make_prereqs, deps)
- def get_dependencies(
- self: "CmdFile", target_path: PathStr, obj_tree: PathStr, fail_on_unknown_build_command: bool
- ) -> list[PathStr]:
+ def get_dependencies(self: "CmdFile", target_path: PathStr, obj_tree: PathStr) -> list[PathStr]:
"""
Parses all dependencies required to build a target file from its cmd file.
Args:
target_path: path to the target file relative to `obj_tree`.
obj_tree: absolute path to the object tree.
- fail_on_unknown_build_command: Whether to fail if an unknown build command is encountered.
Returns:
list[PathStr]: dependency file paths relative to `obj_tree`.
diff --git a/scripts/sbom/sbom/cmd_graph/cmd_graph_node.py b/scripts/sbom/sbom/cmd_graph/cmd_graph_node.py
index 61f3a8140ce..55ad5bb027d 100644
--- a/scripts/sbom/sbom/cmd_graph/cmd_graph_node.py
+++ b/scripts/sbom/sbom/cmd_graph/cmd_graph_node.py
@@ -23,7 +23,6 @@ class IncbinDependency:
class CmdGraphNodeConfig(Protocol):
obj_tree: PathStr
src_tree: PathStr
- fail_on_unknown_build_command: bool
@dataclass
@@ -120,9 +119,7 @@ class CmdGraphNode:
if cmd_file is not None:
node.cmd_file_dependencies = [
_build_child_node(cmd_file_dependency_path)
- for cmd_file_dependency_path in cmd_file.get_dependencies(
- target_path, config.obj_tree, config.fail_on_unknown_build_command
- )
+ for cmd_file_dependency_path in cmd_file.get_dependencies(target_path, config.obj_tree)
]
if node.absolute_path.endswith(".S"):
diff --git a/scripts/sbom/sbom/cmd_graph/hardcoded_dependencies.py b/scripts/sbom/sbom/cmd_graph/hardcoded_dependencies.py
index 2eb04d30f4e..f911fc6f23a 100644
--- a/scripts/sbom/sbom/cmd_graph/hardcoded_dependencies.py
+++ b/scripts/sbom/sbom/cmd_graph/hardcoded_dependencies.py
@@ -10,7 +10,6 @@ from sbom.environment import Environment
HARDCODED_DEPENDENCIES: dict[str, list[str]] = {
# defined in linux/Kbuild
"include/generated/rq-offsets.h": ["kernel/sched/rq-offsets.s"],
- "kernel/sched/rq-offsets.s": ["include/generated/asm-offsets.h"],
"include/generated/bounds.h": ["kernel/bounds.s"],
"include/generated/asm-offsets.h": ["arch/{arch}/kernel/asm-offsets.s"],
}
diff --git a/scripts/sbom/sbom/cmd_graph/savedcmd_parser/__init__.py b/scripts/sbom/sbom/cmd_graph/savedcmd_parser/__init__.py
deleted file mode 100644
index d13876af4df..00000000000
--- a/scripts/sbom/sbom/cmd_graph/savedcmd_parser/__init__.py
+++ /dev/null
@@ -1,6 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only OR MIT
-# Copyright (C) 2025 TNG Technology Consulting GmbH
-
-from sbom.cmd_graph.savedcmd_parser.savedcmd_parser import parse_inputs_from_commands
-
-__all__ = ["parse_inputs_from_commands"]
diff --git a/scripts/sbom/sbom/cmd_graph/savedcmd_parser/command_parser_registry.py b/scripts/sbom/sbom/cmd_graph/savedcmd_parser/command_parser_registry.py
deleted file mode 100644
index a48040b2c13..00000000000
--- a/scripts/sbom/sbom/cmd_graph/savedcmd_parser/command_parser_registry.py
+++ /dev/null
@@ -1,516 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only OR MIT
-# Copyright (C) 2025 TNG Technology Consulting GmbH
-
-import re
-import shlex
-from typing import Callable, Iterator
-
-import sbom.sbom_logging as sbom_logging
-from sbom.environment import Environment
-from sbom.cmd_graph.savedcmd_parser.command_splitter import IfBlock, split_commands
-from sbom.cmd_graph.savedcmd_parser.tokenizer import (
- CmdParsingError,
- Option,
- Positional,
- tokenize_single_command,
- tokenize_single_command_positionals_only,
-)
-from sbom.path_utils import PathStr
-
-CommandParser = Callable[[str], list[PathStr]]
-CommandParserRegistryEntry = tuple[re.Pattern[str], CommandParser]
-
-
-def _parse_dd_command(command: str) -> list[PathStr]:
- match = re.match(r"dd.*?if=(\S+)", command)
- if match:
- return [match.group(1)]
- return []
-
-
-def _parse_cat_command(command: str) -> list[PathStr]:
- positionals = tokenize_single_command_positionals_only(command)
- # expect positionals to be ["cat", input1, input2, ...]
- return [p for p in positionals[1:]]
-
-
-def _parse_compound_command(command: str) -> list[PathStr]:
- compound_command_parsers: list[CommandParserRegistryEntry] = [
- (re.compile(r"dd\b"), _parse_dd_command),
- (re.compile(r"cat.*?\|"), lambda c: _parse_cat_command(c.split("|")[0])),
- (re.compile(r"cat\b[^|>]*$"), _parse_cat_command),
- (re.compile(r"echo\b"), _parse_noop),
- (re.compile(r"\S+="), _parse_noop),
- (re.compile(r"printf\b"), _parse_noop),
- (re.compile(r"sed\b"), _parse_sed_command),
- (
- re.compile(r"(.*/)scripts/bin2c\s*<"),
- lambda c: [input] if (input := c.split("<")[1].split(">")[0].strip()) != "/dev/null" else [],
- ),
- (re.compile(r"^:$"), _parse_noop),
- ]
-
- match = re.match(r"\s*[\(\{](.*)[\)\}]\s*>", command, re.DOTALL)
- if match is None:
- raise CmdParsingError("No inner commands found for compound command")
- input_files: list[PathStr] = []
- inner_commands = split_commands(match.group(1))
- for inner_command in inner_commands:
- if isinstance(inner_command, IfBlock):
- sbom_logging.error(
- "Skip parsing inner command {inner_command} of compound command because IfBlock is not supported",
- inner_command=inner_command,
- )
- continue
-
- parser = next((parser for pattern, parser in compound_command_parsers if pattern.match(inner_command)), None)
- if parser is None:
- sbom_logging.error(
- "Skip parsing inner command {inner_command} of compound command because no matching parser was found",
- inner_command=inner_command,
- )
- continue
- try:
- input_files += parser(inner_command)
- except (CmdParsingError, IndexError) as e:
- sbom_logging.error(
- "Skip parsing inner command {inner_command} of compound command because of command parsing error: {error_message}",
- inner_command=inner_command,
- error_message=str(e),
- )
- return input_files
-
-
-def _parse_objcopy_command(command: str) -> list[PathStr]:
- command_parts = tokenize_single_command(command, flag_options=["-S", "-w"])
- positionals = [part.value for part in command_parts if isinstance(part, Positional)]
- # expect positionals to be ['objcopy', input_file] or ['objcopy', input_file, output_file]
- return [positionals[1]]
-
-
-def _parse_link_vmlinux_command(command: str) -> list[PathStr]:
- """
- For simplicity we do not parse the `scripts/link-vmlinux.sh` script.
- Instead the `vmlinux.a` dependency is just hardcoded for now.
- """
- return ["vmlinux.a"]
-
-
-def _parse_cp_command(command: str) -> list[PathStr]:
- positionals = tokenize_single_command_positionals_only(command)
- # expect positionals to be ["cp", input1, ..., destination]
- return positionals[1:-1]
-
-
-def _parse_noop(command: str) -> list[PathStr]:
- """
- No-op parser for commands with no input files (e.g., 'rm', 'true').
- Returns an empty list.
- """
- return []
-
-
-def _parse_ar_command(command: str) -> list[PathStr]:
- positionals = tokenize_single_command_positionals_only(command)
- # expect positionals to be ['ar', flags, output, input1, input2, ...]
- flags = positionals[1]
- if "r" not in flags:
- # 'r' option indicates that new files are added to the archive.
- # If this option is missing we won't find any relevant input files.
- return []
- return positionals[3:]
-
-
-def _parse_ar_piped_xargs_command(command: str) -> list[PathStr]:
- printf_command, _ = command.split("|", 1)
- positionals = tokenize_single_command_positionals_only(printf_command.strip())
- # expect positionals to be ['printf', '{prefix_path}%s ', input1, input2, ...]
- prefix_path = positionals[1].removesuffix("%s ")
- return [f"{prefix_path}{filename}" for filename in positionals[2:]]
-
-
-def _parse_gcc_or_clang_command(command: str) -> list[PathStr]:
- parts = shlex.split(command)
- # compile mode: expect last positional argument ending in a source file extension to be the input file
- for part in reversed(parts):
- if not part.startswith("-") and any(part.endswith(suffix) for suffix in [".c", ".S", ".dts"]):
- return [part]
-
- # linking mode: expect all .o files to be the inputs
- return [p for p in parts if p.endswith(".o")]
-
-
-def _parse_rustc_command(command: str) -> list[PathStr]:
- parts = shlex.split(command)
- # expect last positional argument ending in `.rs` to be the input file
- for part in reversed(parts):
- if not part.startswith("-") and part.endswith(".rs"):
- return [part]
- raise CmdParsingError("Could not find .rs input source file")
-
-
-def _parse_rustdoc_command(command: str) -> list[PathStr]:
- parts = shlex.split(command)
- # expect last positional argument ending in `.rs` to be the input file
- for part in reversed(parts):
- if not part.startswith("-") and part.endswith(".rs"):
- return [part]
- raise CmdParsingError("Could not find .rs input source file")
-
-
-def _parse_syscallhdr_command(command: str) -> list[PathStr]:
- command_parts = tokenize_single_command(command.strip(), flag_options=["--emit-nr"])
- positionals = [p.value for p in command_parts if isinstance(p, Positional)]
- # expect positionals to be ["sh", path/to/syscallhdr.sh, input, output]
- return [positionals[2]]
-
-
-def _parse_syscalltbl_command(command: str) -> list[PathStr]:
- command_parts = tokenize_single_command(command.strip())
- positionals = [p.value for p in command_parts if isinstance(p, Positional)]
- # expect positionals to be ["sh", path/to/syscalltbl.sh, input, output]
- return [positionals[2]]
-
-
-def _parse_mkcapflags_command(command: str) -> list[PathStr]:
- positionals = tokenize_single_command_positionals_only(command)
- # expect positionals to be ["sh", path/to/mkcapflags.sh, output, input1, input2]
- return [positionals[3], positionals[4]]
-
-
-def _parse_orc_hash_command(command: str) -> list[PathStr]:
- positionals = tokenize_single_command_positionals_only(command)
- # expect positionals to be ["sh", path/to/orc_hash.sh, '<', input, '>', output]
- return [positionals[3]]
-
-
-def _parse_xen_hypercalls_command(command: str) -> list[PathStr]:
- positionals = tokenize_single_command_positionals_only(command)
- # expect positionals to be ["sh", path/to/xen-hypercalls.sh, output, input1, input2, ...]
- return positionals[3:]
-
-
-def _parse_gen_initramfs_command(command: str) -> list[PathStr]:
- command_parts = tokenize_single_command(command)
- positionals = [p.value for p in command_parts if isinstance(p, Positional)]
- # expect positionals to be ["sh", path/to/gen_initramfs.sh, input1, input2, ...]
- return positionals[2:]
-
-
-def _parse_vdso2c_command(command: str) -> list[PathStr]:
- positionals = tokenize_single_command_positionals_only(command)
- # expect positionals to be ['vdso2c', raw_input, stripped_input, output]
- return [positionals[1], positionals[2]]
-
-
-def _parse_vdsomunge_command(command: str) -> list[PathStr]:
- positionals = tokenize_single_command_positionals_only(command)
- # expect positionals to be ['vdsomunge', input, output]
- return [positionals[1]]
-
-
-def _parse_ld_command(command: str) -> list[PathStr]:
- command_parts = tokenize_single_command(
- command=command.strip(),
- flag_options=[
- "-shared",
- "--no-undefined",
- "--eh-frame-hdr",
- "-Bsymbolic",
- "-r",
- "--no-ld-generated-unwind-info",
- "--no-dynamic-linker",
- "-pie",
- "--no-dynamic-linker--whole-archive",
- "--whole-archive",
- "--no-whole-archive",
- "--start-group",
- "--end-group",
- ],
- )
- positionals = [p.value for p in command_parts if isinstance(p, Positional)]
- # expect positionals to be ["ld", input1, input2, ...]
- return positionals[1:]
-
-
-def _parse_sed_command(command: str) -> list[PathStr]:
- command_parts = shlex.split(command)
- # expect command parts to be ["sed", *, input]
- input = command_parts[-1]
- if input == "/dev/null":
- return []
- return [input]
-
-
-def _parse_awk(command: str) -> list[PathStr]:
- command_parts = tokenize_single_command(command)
- options = [p for p in command_parts if isinstance(p, Option)]
- positionals = [p.value for p in command_parts if isinstance(p, Positional)]
- has_script_file = any(p.name == "-f" for p in options)
- # With -f option: expect ["awk", input1, input2, ...]
- # Without -f option: expect ["awk", inline_program, input1, input2, ...]
- return positionals[1:] if has_script_file else positionals[2:]
-
-
-def _parse_nm_piped_command(command: str) -> list[PathStr]:
- nm_command, _ = command.split("|", 1)
- command_parts = tokenize_single_command(
- command=nm_command.strip(),
- flag_options=["-p", "--defined-only"],
- )
- positionals = [p.value for p in command_parts if isinstance(p, Positional)]
- # expect positionals to be ["nm", input1, input2, ...]
- return [p for p in positionals[1:]]
-
-
-def _parse_pnm_to_logo_command(command: str) -> list[PathStr]:
- command_parts = shlex.split(command)
- # expect command parts to be ["pnmtologo", <options>, input]
- return [command_parts[-1]]
-
-
-def _parse_relacheck(command: str) -> list[PathStr]:
- positionals = tokenize_single_command_positionals_only(command)
- # expect positionals to be ["relacheck", input, log_reference]
- return [positionals[1]]
-
-
-def _parse_gen_hyprel_command(command: str) -> list[PathStr]:
- gen_hyprel_command, _ = command.split(">", 1)
- command_parts = shlex.split(gen_hyprel_command)
- # expect command_parts to be ["gen-hyprel", input]
- return [command_parts[1]]
-
-
-def _parse_perl_command(command: str) -> list[PathStr]:
- positionals = tokenize_single_command_positionals_only(command.strip())
- # expect positionals to be ["perl", input]
- return [positionals[1]]
-
-
-def _parse_strip_command(command: str) -> list[PathStr]:
- command_parts = tokenize_single_command(command, flag_options=["--strip-debug"])
- positionals = [p.value for p in command_parts if isinstance(p, Positional)]
- # expect positionals to be ["strip", input1, input2, ...]
- return positionals[1:]
-
-
-def _parse_mkpiggy_command(command: str) -> list[PathStr]:
- mkpiggy_command, _ = command.split(">", 1)
- positionals = tokenize_single_command_positionals_only(mkpiggy_command)
- # expect positionals to be ["mkpiggy", input]
- return [positionals[1]]
-
-
-def _parse_relocs_command(command: str) -> list[PathStr]:
- if ">" not in command:
- # Only consider relocs commands that redirect output to a file.
- # If there's no redirection, we assume it produces no output file and therefore has no input we care about.
- return []
- relocs_command, _ = command.split(">", 1)
- command_parts = shlex.split(relocs_command)
- # expect command_parts to be ["relocs", options, input]
- return [command_parts[-1]]
-
-
-def _parse_mk_elfconfig_command(command: str) -> list[PathStr]:
- positionals = tokenize_single_command_positionals_only(command)
- # expect positionals to be ["mk_elfconfig", "<", input, ">", output]
- return [positionals[2]]
-
-
-def _parse_flex_command(command: str) -> list[PathStr]:
- parts = shlex.split(command)
- # expect last positional argument ending in `.l` to be the input file
- for part in reversed(parts):
- if not part.startswith("-") and part.endswith(".l"):
- return [part]
- raise CmdParsingError("Could not find .l input source file in command")
-
-
-def _parse_bison_command(command: str) -> list[PathStr]:
- parts = shlex.split(command)
- # expect last positional argument ending in `.y` to be the input file
- for part in reversed(parts):
- if not part.startswith("-") and part.endswith(".y"):
- return [part]
- raise CmdParsingError("Could not find input .y input source file in command")
-
-
-def _parse_tools_build_command(command: str) -> list[PathStr]:
- positionals = tokenize_single_command_positionals_only(command)
- # expect positionals to be ["tools/build", "input1", "input2", "input3", "output"]
- return positionals[1:-1]
-
-
-def _parse_extract_cert_command(command: str) -> list[PathStr]:
- command_parts = shlex.split(command)
- # expect command parts to be [path/to/extract-cert, input, output]
- input = command_parts[1]
- if not input:
- return []
- return [input]
-
-
-def _parse_dtc_command(command: str) -> list[PathStr]:
- wno_flags = [command_part for command_part in shlex.split(command) if command_part.startswith("-Wno-")]
- command_parts = tokenize_single_command(command, flag_options=wno_flags)
- positionals = [p.value for p in command_parts if isinstance(p, Positional)]
- # expect positionals to be [path/to/dtc, input]
- return [positionals[1]]
-
-
-def _parse_bindgen_command(command: str) -> list[PathStr]:
- command_parts = shlex.split(command)
- header_file_input_paths = [part for part in command_parts if part.endswith(".h")]
- return header_file_input_paths
-
-
-def _parse_gen_header(command: str) -> list[PathStr]:
- command_parts = shlex.split(command)
- # expect command parts to be ["python3", path/to/gen_headers.py, ..., "--xml", input]
- i = next((i for i, token in enumerate(command_parts) if token == "--xml"), None)
- if i is None:
- raise CmdParsingError(f"Expected --xml input file in gen_headers command but got {command}")
- return [command_parts[i + 1]]
-
-def _parse_mkuboot_command(command: str) -> list[PathStr]:
- command_parts = tokenize_single_command(command)
- # mkuboot.sh passes all args to mkimage; -d specifies the data/input image file
- for part in command_parts:
- if isinstance(part, Option) and part.name == "-d" and part.value is not None:
- return [part.value]
- raise CmdParsingError("Could not find -d (data file) option in mkuboot.sh command")
-
-
-def _parse_syscallnr_command(command: str) -> list[PathStr]:
- command_parts = tokenize_single_command(command.strip())
- positionals = [p.value for p in command_parts if isinstance(p, Positional)]
- # expect positionals to be ["sh", path/to/syscallnr.sh, input, output]
- return [positionals[2]]
-
-
-def _parse_gen_kernel_hwcaps_command(command: str) -> list[PathStr]:
- command_parts = tokenize_single_command(command.strip(), flag_options=["-e"])
- positionals = [p.value for p in command_parts if isinstance(p, Positional)]
- # expect positionals to be ["sh", path/to/gen-kernel-hwcaps.sh, input]
- return [positionals[2]]
-
-
-class CommandParserRegistry:
- """
- Registry mapping command patterns to their input-file parsers.
- """
-
- def __init__(self, entries: list[CommandParserRegistryEntry]) -> None:
- self._entries = entries
-
- def __iter__(self) -> Iterator[CommandParserRegistryEntry]:
- return iter(self._entries)
-
- @staticmethod
- def create() -> "CommandParserRegistry":
- def env_or_default_pattern(env_value: str | None, default_pattern: str) -> str:
- if env_value is None or not env_value.strip():
- return default_pattern
- return rf"(?:{re.escape(env_value.strip())}|{default_pattern})"
-
- cc_pattern = env_or_default_pattern(Environment.CC(), r"([^\s]+-)?(gcc|clang)")
- ld_pattern = env_or_default_pattern(Environment.LD(), r"([^\s]+-)?ld")
- ar_pattern = env_or_default_pattern(Environment.AR(), r"([^\s]+-)?ar")
- nm_pattern = env_or_default_pattern(Environment.NM(), r"([^\s]+-)?nm")
- objcopy_pattern = env_or_default_pattern(Environment.OBJCOPY(), r"([^\s]+-)?objcopy")
- strip_pattern = env_or_default_pattern(Environment.STRIP(), r"([^\s]+-)?strip")
-
- entries: list[CommandParserRegistryEntry] = [
- # Compound commands
- (re.compile(r"\(.*?\)\s*>", re.DOTALL), _parse_compound_command),
- (re.compile(r"\{.*?\}\s*>", re.DOTALL), _parse_compound_command),
- # Standard Unix utilities and system tools
- (re.compile(r"^rm\b"), _parse_noop),
- (re.compile(r"^mkdir\b"), _parse_noop),
- (re.compile(r"^touch\b"), _parse_noop),
- (re.compile(r"^cp\b"), _parse_cp_command),
- (re.compile(r"^truncate\b"), _parse_noop),
- (re.compile(r"^cat\b.*?[\|>]"), lambda c: _parse_cat_command(c.split("|")[0].split(">")[0])),
- (re.compile(r"^echo[^|]*$"), _parse_noop),
- (re.compile(r"^sed.*?>"), lambda c: _parse_sed_command(c.split(">")[0])),
- (re.compile(r"^sed\b"), _parse_noop),
- (re.compile(r"^awk.*?<.*?>"), lambda c: [c.split("<")[1].split(">")[0]]),
- (re.compile(r"^awk.*?>"), lambda c: _parse_awk(c.split(">")[0])),
- (re.compile(r"^(/bin/)?true\b"), _parse_noop),
- (re.compile(r"^(/bin/)?false\b"), _parse_noop),
- (re.compile(r"^openssl\s+req.*?-new.*?-keyout"), _parse_noop),
- # Compilers and code generators
- # (C/LLVM toolchain, Rust, Flex/Bison, Bindgen, Perl, etc.)
- (
- re.compile(rf"^{cc_pattern}\b"),
- lambda command: _parse_gcc_or_clang_command(re.sub(rf"^{cc_pattern}\b", "gcc", command, count=1)),
- ),
- (
- re.compile(rf"^{ld_pattern}\b"),
- lambda command: _parse_ld_command(re.sub(rf"^{ld_pattern}\b", "ld", command, count=1)),
- ),
- (
- re.compile(rf"^printf\b.*\| xargs {ar_pattern}\b"),
- lambda command: _parse_ar_piped_xargs_command(
- re.sub(rf"xargs {ar_pattern}\b", "xargs ar", command, count=1)
- ),
- ),
- (
- re.compile(rf"^{ar_pattern}\b"),
- lambda command: _parse_ar_command(re.sub(rf"^{ar_pattern}\b", "ar", command, count=1)),
- ),
- (
- re.compile(rf"^{nm_pattern}\b.*?\|"),
- lambda command: _parse_nm_piped_command(re.sub(rf"^{nm_pattern}\b", "nm", command, count=1)),
- ),
- (
- re.compile(rf"^{objcopy_pattern}\b"),
- lambda command: _parse_objcopy_command(re.sub(rf"^{objcopy_pattern}\b", "objcopy", command, count=1)),
- ),
- (
- re.compile(rf"^{strip_pattern}\b"),
- lambda command: _parse_strip_command(re.sub(rf"^{strip_pattern}\b", "strip", command, count=1)),
- ),
- (re.compile(r".*?rustc\b"), _parse_rustc_command),
- (re.compile(r".*?rustdoc\b"), _parse_rustdoc_command),
- (re.compile(r"^flex\b"), _parse_flex_command),
- (re.compile(r"^bison\b"), _parse_bison_command),
- (re.compile(r"^bindgen\b"), _parse_bindgen_command),
- (re.compile(r"^perl\b"), _parse_perl_command),
- # Kernel-specific build scripts and tools
- (re.compile(r"^(.*/)?link-vmlinux\.sh\b"), _parse_link_vmlinux_command),
- (re.compile(r"sh (.*/)?syscallhdr\.sh\b"), _parse_syscallhdr_command),
- (re.compile(r"sh (.*/)?syscalltbl\.sh\b"), _parse_syscalltbl_command),
- (re.compile(r"sh (.*/)?mkcapflags\.sh\b"), _parse_mkcapflags_command),
- (re.compile(r"sh (.*/)?orc_hash\.sh\b"), _parse_orc_hash_command),
- (re.compile(r"sh (.*/)?xen-hypercalls\.sh\b"), _parse_xen_hypercalls_command),
- (re.compile(r"sh (.*/)?gen_initramfs\.sh\b"), _parse_gen_initramfs_command),
- (re.compile(r"sh (.*/)?checkundef\.sh\b"), _parse_noop),
- (re.compile(r"(bash|sh) (.*/)?mkuboot\.sh\b"), _parse_mkuboot_command),
- (re.compile(r"sh (.*/)?syscallnr\.sh\b"), _parse_syscallnr_command),
- (re.compile(r"(/bin/)?sh (.*/)?gen-kernel-hwcaps\.sh\b"), lambda c: _parse_gen_kernel_hwcaps_command(c.split(">")[0])),
- (re.compile(r"(.*/)?vdso2c\b"), _parse_vdso2c_command),
- (re.compile(r"(.*/)?vdsomunge\b"), _parse_vdsomunge_command),
- (re.compile(r"^(.*/)?mkpiggy.*?>"), _parse_mkpiggy_command),
- (re.compile(r"^(.*/)?relocs\b"), _parse_relocs_command),
- (re.compile(r"^(.*/)?mk_elfconfig.*?<.*?>"), _parse_mk_elfconfig_command),
- (re.compile(r"^(.*/)?tools/build\b"), _parse_tools_build_command),
- (re.compile(r"^(.*/)?certs/extract-cert"), _parse_extract_cert_command),
- (re.compile(r"^(.*/)?scripts/dtc/dtc\b"), _parse_dtc_command),
- (re.compile(r"^(.*/)?pnmtologo\b"), _parse_pnm_to_logo_command),
- (re.compile(r"^(.*/)?kernel/pi/relacheck"), _parse_relacheck),
- (re.compile(r"^(.*/)?gen-hyprel\b"), _parse_gen_hyprel_command),
- (re.compile(r"^drivers/gpu/drm/radeon/mkregtable"), lambda c: [c.split(" ")[1]]),
- (re.compile(r"(.*/)?genheaders\b"), _parse_noop),
- (re.compile(r"^(.*/)?mkcpustr\s+>"), _parse_noop),
- (re.compile(r"^(.*/)polgen\b"), _parse_noop),
- (re.compile(r"make -f .*/arch/x86/Makefile\.postlink"), _parse_noop),
- (re.compile(r"^(.*/)?raid6/mktables\s+>"), _parse_noop),
- (re.compile(r"^(.*/)?objtool\b"), _parse_noop),
- (re.compile(r"^(.*/)?module/gen_test_kallsyms.sh"), _parse_noop),
- (re.compile(r"^(.*/)?gen_header.py"), _parse_gen_header),
- (re.compile(r"^(.*/)?scripts/rustdoc_test_gen"), _parse_noop),
- ]
- return CommandParserRegistry(entries)
diff --git a/scripts/sbom/sbom/cmd_graph/savedcmd_parser/command_splitter.py b/scripts/sbom/sbom/cmd_graph/savedcmd_parser/command_splitter.py
deleted file mode 100644
index 4749f4bd669..00000000000
--- a/scripts/sbom/sbom/cmd_graph/savedcmd_parser/command_splitter.py
+++ /dev/null
@@ -1,128 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only OR MIT
-# Copyright (C) 2025 TNG Technology Consulting GmbH
-
-import re
-from dataclasses import dataclass
-
-
-# If Block pattern to match a simple, single-level if-then-fi block. Nested If blocks are not supported.
-IF_BLOCK_PATTERN = re.compile(
- r"""
- ^if(.*?);\s* # Match 'if <condition>;' (non-greedy)
- then(.*?);\s* # Match 'then <body>;' (non-greedy)
- fi\b # Match 'fi'
- """,
- re.VERBOSE,
-)
-
-
-@dataclass
-class IfBlock:
- condition: str
- then_statement: str
-
-
-def _unwrap_outer_parentheses(s: str) -> str:
- s = s.strip()
- if not (s.startswith("(") and s.endswith(")")):
- return s
-
- count = 0
- for i, char in enumerate(s):
- if char == "(":
- count += 1
- elif char == ")":
- count -= 1
- # If count is 0 before the end, outer parentheses don't match
- if count == 0 and i != len(s) - 1:
- return s
-
- # outer parentheses do match, unwrap once
- return _unwrap_outer_parentheses(s[1:-1])
-
-
-def _find_first_top_level_command_separator(
- commands: str, separators: list[str] = [";", "&&"]
-) -> tuple[int | None, int | None]:
- def is_escaped(index: int) -> bool:
- preceding = commands[:index]
- return (len(preceding) - len(preceding.rstrip("\\"))) % 2 == 1
-
- in_single_quote = False
- in_double_quote = False
- in_curly_braces = 0
- in_braces = 0
- for i, char in enumerate(commands):
- if char == "'" and not in_double_quote and not is_escaped(i):
- # Toggle single quote state (unless inside double quotes or escaped)
- in_single_quote = not in_single_quote
- elif char == '"' and not in_single_quote and not is_escaped(i):
- # Toggle double quote state (unless inside single quotes or escaped)
- in_double_quote = not in_double_quote
-
- if in_single_quote or in_double_quote:
- continue
-
- # Toggle braces state
- if char == "{":
- in_curly_braces += 1
- if char == "}":
- in_curly_braces -= 1
-
- if char == "(":
- in_braces += 1
- if char == ")":
- in_braces -= 1
-
- if in_curly_braces > 0 or in_braces > 0:
- continue
-
- # return found separator position and separator length
- for separator in separators:
- if commands[i : i + len(separator)] == separator:
- return i, len(separator)
-
- return None, None
-
-
-def split_commands(commands: str) -> list[str | IfBlock]:
- """
- Splits a string of command-line commands into individual parts.
-
- This function handles:
- - Top-level command separators (e.g., `;` and `&&`) to split multiple commands.
- - Conditional if-blocks, returning them as `IfBlock` instances.
- - Preserves the order of commands and trims whitespace.
-
- Args:
- commands (str): The raw command string.
-
- Returns:
- list[str | IfBlock]: A list of single commands or `IfBlock` objects.
- """
- single_commands: list[str | IfBlock] = []
- remaining_commands = _unwrap_outer_parentheses(commands)
- while len(remaining_commands) > 0:
- remaining_commands = remaining_commands.strip()
-
- # if block
- matched_if = IF_BLOCK_PATTERN.match(remaining_commands)
- if matched_if:
- condition, then_statement = matched_if.groups()
- single_commands.append(IfBlock(condition.strip(), then_statement.strip()))
- full_matched = matched_if.group(0)
- remaining_commands = remaining_commands.removeprefix(full_matched).lstrip("; \n")
- continue
-
- # command until next separator
- separator_position, separator_length = _find_first_top_level_command_separator(remaining_commands)
- if separator_position is not None and separator_length is not None:
- single_commands.append(remaining_commands[:separator_position].strip())
- remaining_commands = remaining_commands[separator_position + separator_length :].strip()
- continue
-
- # single last command
- single_commands.append(remaining_commands)
- break
-
- return single_commands
diff --git a/scripts/sbom/sbom/cmd_graph/savedcmd_parser/savedcmd_parser.py b/scripts/sbom/sbom/cmd_graph/savedcmd_parser/savedcmd_parser.py
deleted file mode 100644
index d2ca842a784..00000000000
--- a/scripts/sbom/sbom/cmd_graph/savedcmd_parser/savedcmd_parser.py
+++ /dev/null
@@ -1,67 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only OR MIT
-# Copyright (C) 2025 TNG Technology Consulting GmbH
-
-import sbom.sbom_logging as sbom_logging
-from sbom.cmd_graph.savedcmd_parser.command_splitter import IfBlock, split_commands
-from sbom.cmd_graph.savedcmd_parser.command_parser_registry import CommandParserRegistry
-from sbom.cmd_graph.savedcmd_parser.tokenizer import CmdParsingError
-from sbom.path_utils import PathStr
-
-DEFAULT_COMMAND_PARSER_REGISTRY = CommandParserRegistry.create()
-
-
-def parse_inputs_from_commands(
- commands: str,
- fail_on_unknown_build_command: bool,
- registry: CommandParserRegistry | None = None,
-) -> list[PathStr]:
- """
- Extract input files referenced in a set of command-line commands.
-
- Args:
- commands (str): Command line expression to parse.
- fail_on_unknown_build_command (bool): Whether to fail if an unknown build command is encountered. If False, errors are logged as warnings.
- registry (CommandParserRegistry | None): Registry of single command parsers.
-
- Returns:
- list[PathStr]: List of input file paths required by the commands.
- """
-
- def log_error_or_warning(message: str, /, **kwargs: str) -> None:
- if fail_on_unknown_build_command:
- sbom_logging.error(message, **kwargs)
- else:
- sbom_logging.warning(message, **kwargs)
-
- if registry is None:
- registry = DEFAULT_COMMAND_PARSER_REGISTRY
-
- input_files: list[PathStr] = []
- for single_command in split_commands(commands):
- if isinstance(single_command, IfBlock):
- inputs = parse_inputs_from_commands(single_command.then_statement, fail_on_unknown_build_command, registry)
- if inputs:
- log_error_or_warning(
- "Skipped parsing command {then_statement} because input files in IfBlock 'then' statement are not supported",
- then_statement=single_command.then_statement,
- )
- continue
-
- matched_parser = next((parser for pattern, parser in registry if pattern.match(single_command)), None)
- if matched_parser is None:
- log_error_or_warning(
- "Skipped parsing command {single_command} because no matching parser was found",
- single_command=single_command,
- )
- continue
- try:
- inputs = matched_parser(single_command)
- input_files.extend(inputs)
- except (CmdParsingError, IndexError, ValueError) as e:
- log_error_or_warning(
- "Skipped parsing command {single_command} because of command parsing error: {error_message}",
- single_command=single_command,
- error_message=str(e),
- )
-
- return [input.strip().rstrip("/") for input in input_files]
diff --git a/scripts/sbom/sbom/cmd_graph/savedcmd_parser/tokenizer.py b/scripts/sbom/sbom/cmd_graph/savedcmd_parser/tokenizer.py
deleted file mode 100644
index 1bf081f40be..00000000000
--- a/scripts/sbom/sbom/cmd_graph/savedcmd_parser/tokenizer.py
+++ /dev/null
@@ -1,92 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only OR MIT
-# Copyright (C) 2025 TNG Technology Consulting GmbH
-
-import re
-import shlex
-from dataclasses import dataclass
-from typing import Union
-
-
-class CmdParsingError(Exception):
- pass
-
-
-@dataclass
-class Option:
- name: str
- value: str | None = None
-
-
-@dataclass
-class Positional:
- value: str
-
-
-_SUBCOMMAND_PATTERN = re.compile(r"\$\$\(([^()]*)\)")
-"""Pattern to match $$(...) blocks"""
-
-
-def tokenize_single_command(command: str, flag_options: list[str] | None = None) -> list[Union[Option, Positional]]:
- """
- Parse a shell command into a list of Options and Positionals.
- - Positional: the command and any positional arguments.
- - Options: handles flags and options with values provided as space-separated, or equals-sign
- (e.g., '--opt val', '--opt=val', '--flag').
-
- Args:
- command: Command line string.
- flag_options: Options that are flags without values (e.g., '--verbose').
-
- Returns:
- List of `Option` and `Positional` objects in command order.
- """
-
- # Wrap all $$(...) blocks in double quotes to prevent shlex from splitting them.
- command_with_protected_subcommands = _SUBCOMMAND_PATTERN.sub(lambda m: f'"$$({m.group(1)})"', command)
- tokens = shlex.split(command_with_protected_subcommands)
-
- parsed: list[Option | Positional] = []
- i = 0
- while i < len(tokens):
- token = tokens[i]
-
- # Positional
- if not token.startswith("-"):
- parsed.append(Positional(token))
- i += 1
- continue
-
- # Option without value (--flag)
- if (token.startswith("-") and i + 1 < len(tokens) and tokens[i + 1].startswith("-")) or (
- flag_options and token in flag_options
- ):
- parsed.append(Option(name=token))
- i += 1
- continue
-
- # Option with equals sign (--opt=val)
- if "=" in token:
- name, value = token.split("=", 1)
- parsed.append(Option(name=name, value=value))
- i += 1
- continue
-
- # Option with space-separated value (--opt val)
- if i + 1 < len(tokens) and not tokens[i + 1].startswith("-"):
- parsed.append(Option(name=token, value=tokens[i + 1]))
- i += 2
- continue
-
- raise CmdParsingError(f"Unrecognized token: {token} in command {command}")
-
- return parsed
-
-
-def tokenize_single_command_positionals_only(command: str) -> list[str]:
- command_parts = tokenize_single_command(command)
- positionals = [p.value for p in command_parts if isinstance(p, Positional)]
- if len(positionals) != len(command_parts):
- raise CmdParsingError(
- f"Invalid command format: expected positional arguments only but got options in command {command}."
- )
- return positionals
diff --git a/scripts/sbom/sbom/config.py b/scripts/sbom/sbom/config.py
index 6811f782943..08474c2bfc3 100644
--- a/scripts/sbom/sbom/config.py
+++ b/scripts/sbom/sbom/config.py
@@ -47,9 +47,6 @@ class KernelSbomConfig:
debug: bool
"""Whether to enable debug logging."""
- fail_on_unknown_build_command: bool
- """Whether to fail if an unknown build command is encountered in a .cmd file."""
-
write_output_on_error: bool
"""Whether to write output documents even if errors occur."""
@@ -141,15 +138,6 @@ def _parse_cli_arguments(parser: argparse.ArgumentParser) -> dict[str, Any]:
)
# Error handling settings
- parser.add_argument(
- "--do-not-fail-on-unknown-build-command",
- action="store_true",
- default=False,
- help=(
- "Whether to fail if an unknown build command is encountered in a .cmd file.\n"
- "If set to True, errors are logged as warnings instead. (default: False)"
- ),
- )
parser.add_argument(
"--write-output-on-error",
action="store_true",
@@ -245,7 +233,6 @@ def get_config() -> KernelSbomConfig:
output_directory = os.path.realpath(args["output_directory"])
debug = args["debug"]
- fail_on_unknown_build_command = not args["do_not_fail_on_unknown_build_command"]
write_output_on_error = args["write_output_on_error"]
created = datetime.fromtimestamp(
@@ -283,7 +270,6 @@ def get_config() -> KernelSbomConfig:
used_files_file_name=used_files_file_name,
output_directory=output_directory,
debug=debug,
- fail_on_unknown_build_command=fail_on_unknown_build_command,
write_output_on_error=write_output_on_error,
created=created,
spdxId_prefix=spdxId_prefix,
diff --git a/scripts/sbom/tests/cmd_graph/test_savedcmd_parser.py b/scripts/sbom/tests/cmd_graph/test_savedcmd_parser.py
deleted file mode 100644
index d7776f072e0..00000000000
--- a/scripts/sbom/tests/cmd_graph/test_savedcmd_parser.py
+++ /dev/null
@@ -1,468 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only OR MIT
-# Copyright (C) 2025 TNG Technology Consulting GmbH
-
-import os
-import unittest
-from unittest.mock import patch
-
-from sbom.cmd_graph.savedcmd_parser import parse_inputs_from_commands
-from sbom.cmd_graph.savedcmd_parser.command_parser_registry import CommandParserRegistry
-import sbom.sbom_logging as sbom_logging
-
-
-class TestSavedCmdParser(unittest.TestCase):
- def _assert_parsing(self, cmd: str, expected: str, registry: CommandParserRegistry | None = None) -> None:
- sbom_logging.init()
- parsed = parse_inputs_from_commands(cmd, fail_on_unknown_build_command=False, registry=registry)
- target = [] if expected == "" else expected.split(" ")
- self.assertEqual(parsed, target)
- errors = sbom_logging._error_logger._message_counts # type: ignore
- self.assertEqual(errors, {})
-
- # Error handling tests
- def test_malformed_shell_quoting(self):
- command = 'gcc "unterminated'
- with patch.object(sbom_logging, "warning") as warning:
- parsed = parse_inputs_from_commands(command, fail_on_unknown_build_command=False)
-
- self.assertEqual(parsed, [])
- warning.assert_called_once_with(
- "Skipped parsing command {single_command} because of command parsing error: {error_message}",
- single_command=command,
- error_message="No closing quotation",
- )
-
- def test_missing_positional_argument(self):
- command = "objcopy"
- with patch.object(sbom_logging, "warning") as warning:
- parsed = parse_inputs_from_commands(command, fail_on_unknown_build_command=False)
-
- self.assertEqual(parsed, [])
- warning.assert_called_once_with(
- "Skipped parsing command {single_command} because of command parsing error: {error_message}",
- single_command=command,
- error_message="list index out of range",
- )
-
- # Compound command tests
- def test_dd_cat(self):
- cmd = "(dd if=arch/x86/boot/setup.bin bs=4k conv=sync status=none; cat arch/x86/boot/vmlinux.bin) >arch/x86/boot/bzImage"
- expected = "arch/x86/boot/setup.bin arch/x86/boot/vmlinux.bin"
- self._assert_parsing(cmd, expected)
-
- def test_manual_file_creation(self):
- cmd = """{ symbase=__dtbo_overlay_bad_unresolved; echo '$(pound)include <asm-generic/vmlinux.lds.h>'; echo '.section .rodata,"a"'; echo '.balign STRUCT_ALIGNMENT'; echo ".global $${symbase}_begin"; echo "$${symbase}_begin:"; echo '.incbin "drivers/of/unittest-data/overlay_bad_unresolved.dtbo" '; echo ".global $${symbase}_end"; echo "$${symbase}_end:"; echo '.balign STRUCT_ALIGNMENT'; } > drivers/of/unittest-data/overlay_bad_unresolved.dtbo.S"""
- expected = ""
- self._assert_parsing(cmd, expected)
-
- def test_cat_xz_wrap(self):
- cmd = "{ cat arch/x86/boot/compressed/vmlinux.bin | sh ../scripts/xz_wrap.sh; printf \\130\\064\\024\\000; } > arch/x86/boot/compressed/vmlinux.bin.xz"
- expected = "arch/x86/boot/compressed/vmlinux.bin"
- self._assert_parsing(cmd, expected)
-
- def test_printf_sed(self):
- cmd = r"""{ printf 'static char tomoyo_builtin_profile[] __initdata =\n'; sed -e 's/\\/\\\\/g' -e 's/\"/\\"/g' -e 's/\(.*\)/\t"\1\\n"/' -- /dev/null; printf '\t"";\n'; printf 'static char tomoyo_builtin_exception_policy[] __initdata =\n'; sed -e 's/\\/\\\\/g' -e 's/\"/\\"/g' -e 's/\(.*\)/\t"\1\\n"/' -- ../security/tomoyo/policy/exception_policy.conf.default; printf '\t"";\n'; printf 'static char tomoyo_builtin_domain_policy[] __initdata =\n'; sed -e 's/\\/\\\\/g' -e 's/\"/\\"/g' -e 's/\(.*\)/\t"\1\\n"/' -- /dev/null; printf '\t"";\n'; printf 'static char tomoyo_builtin_manager[] __initdata =\n'; sed -e 's/\\/\\\\/g' -e 's/\"/\\"/g' -e 's/\(.*\)/\t"\1\\n"/' -- /dev/null; printf '\t"";\n'; printf 'static char tomoyo_builtin_stat[] __initdata =\n'; sed -e 's/\\/\\\\/g' -e 's/\"/\\"/g' -e 's/\(.*\)/\t"\1\\n"/' -- /dev/null; printf '\t"";\n'; } > security/tomoyo/builtin-policy.h"""
- expected = "../security/tomoyo/policy/exception_policy.conf.default"
- self._assert_parsing(cmd, expected)
-
- def test_bin2c_echo(self):
- cmd = """(echo "static char tomoyo_builtin_profile[] __initdata ="; ./scripts/bin2c </dev/null; echo ";"; echo "static char tomoyo_builtin_exception_policy[] __initdata ="; ./scripts/bin2c <../security/tomoyo/policy/exception_policy.conf.default; echo ";"; echo "static char tomoyo_builtin_domain_policy[] __initdata ="; ./scripts/bin2c </dev/null; echo ";"; echo "static char tomoyo_builtin_manager[] __initdata ="; ./scripts/bin2c </dev/null; echo ";"; echo "static char tomoyo_builtin_stat[] __initdata ="; ./scripts/bin2c </dev/null; echo ";") >security/tomoyo/builtin-policy.h"""
- expected = "../security/tomoyo/policy/exception_policy.conf.default"
- self._assert_parsing(cmd, expected)
-
- def test_cat_colon(self):
- cmd = "{ cat init/modules.order; cat usr/modules.order; cat arch/x86/modules.order; cat arch/x86/boot/startup/modules.order; cat kernel/modules.order; cat certs/modules.order; cat mm/modules.order; cat fs/modules.order; cat ipc/modules.order; cat security/modules.order; cat crypto/modules.order; cat block/modules.order; cat io_uring/modules.order; cat lib/modules.order; cat arch/x86/lib/modules.order; cat drivers/modules.order; cat sound/modules.order; cat samples/modules.order; cat net/modules.order; cat virt/modules.order; cat arch/x86/pci/modules.order; cat arch/x86/power/modules.order; cat arch/x86/video/modules.order; :; } > modules.order"
- expected = "init/modules.order usr/modules.order arch/x86/modules.order arch/x86/boot/startup/modules.order kernel/modules.order certs/modules.order mm/modules.order fs/modules.order ipc/modules.order security/modules.order crypto/modules.order block/modules.order io_uring/modules.order lib/modules.order arch/x86/lib/modules.order drivers/modules.order sound/modules.order samples/modules.order net/modules.order virt/modules.order arch/x86/pci/modules.order arch/x86/power/modules.order arch/x86/video/modules.order"
- self._assert_parsing(cmd, expected)
-
- def test_cat_zstd(self):
- cmd = "{ cat arch/x86/boot/compressed/vmlinux.bin arch/x86/boot/compressed/vmlinux.relocs | zstd -22 --ultra; printf \\340\\362\\066\\003; } > arch/x86/boot/compressed/vmlinux.bin.zst"
- expected = "arch/x86/boot/compressed/vmlinux.bin arch/x86/boot/compressed/vmlinux.relocs"
- self._assert_parsing(cmd, expected)
-
- # cat command tests
- def test_cat_redirect(self):
- cmd = "cat ../fs/unicode/utf8data.c_shipped > fs/unicode/utf8data.c"
- expected = "../fs/unicode/utf8data.c_shipped"
- self._assert_parsing(cmd, expected)
-
- def test_cat_piped(self):
- cmd = "cat arch/x86/boot/compressed/vmlinux.bin arch/x86/boot/compressed/vmlinux.relocs | gzip -n -f -9 > arch/x86/boot/compressed/vmlinux.bin.gz"
- expected = "arch/x86/boot/compressed/vmlinux.bin arch/x86/boot/compressed/vmlinux.relocs"
- self._assert_parsing(cmd, expected)
-
- # sed command tests
- def test_sed(self):
- cmd = "sed -n 's/.*define *BLIST_\\([A-Z0-9_]*\\) *.*/BLIST_FLAG_NAME(\\1),/p' ../include/scsi/scsi_devinfo.h > drivers/scsi/scsi_devinfo_tbl.c"
- expected = "../include/scsi/scsi_devinfo.h"
- self._assert_parsing(cmd, expected)
-
- # awk command tests
- def test_awk(self):
- cmd = "awk -f ../arch/arm64/tools/gen-cpucaps.awk ../arch/arm64/tools/cpucaps > arch/arm64/include/generated/asm/cpucap-defs.h"
- expected = "../arch/arm64/tools/cpucaps"
- self._assert_parsing(cmd, expected)
-
- def test_awk_with_input_redirection(self):
- cmd = "awk -v N=1 -f ../lib/raid6/unroll.awk < ../lib/raid6/int.uc > lib/raid6/int1.c"
- expected = "../lib/raid6/int.uc"
- self._assert_parsing(cmd, expected)
-
- # openssl command tests
- def test_openssl(self):
- cmd = "openssl req -new -nodes -utf8 -sha256 -days 36500 -batch -x509 -config certs/x509.genkey -outform PEM -out certs/signing_key.pem -keyout certs/signing_key.pem 2>&1"
- expected = ""
- self._assert_parsing(cmd, expected)
-
- # gcc/clang command tests
- def test_gcc(self):
- cmd = (
- "gcc -Wp,-MMD,arch/x86/pci/.i386.o.d -nostdinc -I../arch/x86/include -I./arch/x86/include/generated -I../include -I./include -I../arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I../include/uapi -I./include/generated/uapi -include ../include/linux/compiler-version.h -include ../include/linux/kconfig.h -include ../include/linux/compiler_types.h -D__KERNEL__ -fmacro-prefix-map=../= -Werror -std=gnu11 -fshort-wchar -funsigned-char -fno-common -fno-PIE -fno-strict-aliasing -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -fcf-protection=branch -fno-jump-tables -m64 -falign-jumps=1 -falign-loops=1 -mno-80387 -mno-fp-ret-in-387 -mpreferred-stack-boundary=3 -mskip-rax-setup -march=x86-64 -mtune=generic -mno-red-zone -mcmodel=kernel -mstack-protector-guard-reg=gs -mstack-protector-guard-symbol=__ref_stack_chk_guard -Wno-sign-compare -fno-asynchronous-unwind-tables -mindirect-branch=thunk-extern -mindirect-branch-register -mindirect-branch-cs-prefix -mfunction-return=thunk-extern -fno-jump-tables -fpatchable-function-entry=16,16 -fno-delete-null-pointer-checks -O2 -fno-allow-store-data-races -fstack-protector-strong -fomit-frame-pointer -fno-stack-clash-protection -falign-functions=16 -fno-strict-overflow -fno-stack-check -fconserve-stack -fno-builtin-wcslen -Wall -Wextra -Wundef -Werror=implicit-function-declaration -Werror=implicit-int -Werror=return-type -Werror=strict-prototypes -Wno-format-security -Wno-trigraphs -Wno-frame-address -Wno-address-of-packed-member -Wmissing-declarations -Wmissing-prototypes -Wframe-larger-than=2048 -Wno-main -Wvla-larger-than=1 -Wno-pointer-sign -Wcast-function-type -Wno-array-bounds -Wno-stringop-overflow -Wno-alloc-size-larger-than -Wimplicit-fallthrough=5 -Werror=date-time -Werror=incompatible-pointer-types -Werror=designated-init -Wenum-conversion -Wunused -Wno-unused-but-set-variable -Wno-unused-const-variable -Wno-packed-not-aligned -Wno-format-overflow -Wno-format-truncation -Wno-stringop-truncation -Wno-override-init -Wno-missing-field-initializers -Wno-type-limits -Wno-shift-negative-value -Wno-maybe-uninitialized -Wno-sign-compare -Wno-unused-parameter -I../arch/x86/pci -Iarch/x86/pci -DKBUILD_MODFILE="
- "arch/x86/pci/i386"
- " -DKBUILD_BASENAME="
- "i386"
- " -DKBUILD_MODNAME="
- "i386"
- " -D__KBUILD_MODNAME=kmod_i386 -c -o arch/x86/pci/i386.o ../arch/x86/pci/i386.c "
- )
- expected = "../arch/x86/pci/i386.c"
- self._assert_parsing(cmd, expected)
-
- def test_gcc_linking(self):
- cmd = "gcc -o arch/x86/tools/relocs arch/x86/tools/relocs_32.o arch/x86/tools/relocs_64.o arch/x86/tools/relocs_common.o"
- expected = "arch/x86/tools/relocs_32.o arch/x86/tools/relocs_64.o arch/x86/tools/relocs_common.o"
- self._assert_parsing(cmd, expected)
-
- def test_gcc_without_compile_flag(self):
- cmd = "gcc -Wp,-MMD,arch/x86/boot/compressed/.mkpiggy.d -Wall -Wmissing-prototypes -Wstrict-prototypes -O2 -fomit-frame-pointer -std=gnu11 -I ../scripts/include -I../tools/include -I arch/x86/boot/compressed -o arch/x86/boot/compressed/mkpiggy ../arch/x86/boot/compressed/mkpiggy.c"
- expected = "../arch/x86/boot/compressed/mkpiggy.c"
- self._assert_parsing(cmd, expected)
-
- def test_gcc_with_env_override(self):
- with patch.dict(os.environ, {"CC": "ccache gcc"}):
- registry = CommandParserRegistry.create()
- cmd = "gcc -o arch/x86/tools/relocs arch/x86/tools/relocs_32.o arch/x86/tools/relocs_64.o arch/x86/tools/relocs_common.o"
- expected = "arch/x86/tools/relocs_32.o arch/x86/tools/relocs_64.o arch/x86/tools/relocs_common.o"
- self._assert_parsing(cmd, expected, registry)
- self._assert_parsing(f"ccache {cmd}", expected, registry)
-
- def test_gcc_dts_preprocessing(self):
- cmd = "gcc -E -Wp,-MMD,drivers/of/.empty_root.dtb.d.pre.tmp -nostdinc -I ../scripts/dtc/include-prefixes -undef -D__DTS__ -x assembler-with-cpp -o drivers/of/.empty_root.dtb.dts.tmp ../drivers/of/empty_root.dts"
- expected = "../drivers/of/empty_root.dts"
- self._assert_parsing(cmd, expected)
-
- def test_clang(self):
- cmd = """clang -Wp,-MMD,arch/x86/entry/.entry_64_compat.o.d -nostdinc -I../arch/x86/include -I./arch/x86/include/generated -I../include -I./include -I../arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I../include/uapi -I./include/generated/uapi -include ../include/linux/compiler-version.h -include ../include/linux/kconfig.h -D__KERNEL__ --target=x86_64-linux-gnu -fintegrated-as -Werror=unknown-warning-option -Werror=ignored-optimization-argument -Werror=option-ignored -Werror=unused-command-line-argument -fmacro-prefix-map=../= -Werror -D__ASSEMBLY__ -fno-PIE -m64 -I../arch/x86/entry -Iarch/x86/entry -DKBUILD_MODFILE='"arch/x86/entry/entry_64_compat"' -DKBUILD_MODNAME='"entry_64_compat"' -D__KBUILD_MODNAME=kmod_entry_64_compat -c -o arch/x86/entry/entry_64_compat.o ../arch/x86/entry/entry_64_compat.S"""
- expected = "../arch/x86/entry/entry_64_compat.S"
- self._assert_parsing(cmd, expected)
-
- # ld command tests
- def test_ld(self):
- cmd = r'ld -o arch/x86/entry/vdso/vdso64.so.dbg -shared --hash-style=both --build-id=sha1 --no-undefined --eh-frame-hdr -Bsymbolic -z noexecstack -m elf_x86_64 -soname linux-vdso.so.1 -z max-page-size=4096 -T arch/x86/entry/vdso/vdso.lds arch/x86/entry/vdso/vdso-note.o arch/x86/entry/vdso/vclock_gettime.o arch/x86/entry/vdso/vgetcpu.o arch/x86/entry/vdso/vgetrandom.o arch/x86/entry/vdso/vgetrandom-chacha.o; if readelf -rW arch/x86/entry/vdso/vdso64.so.dbg | grep -v _NONE | grep -q " R_\w*_"; then (echo >&2 "arch/x86/entry/vdso/vdso64.so.dbg: dynamic relocations are not supported"; rm -f arch/x86/entry/vdso/vdso64.so.dbg; /bin/false); fi'
- expected = "arch/x86/entry/vdso/vdso-note.o arch/x86/entry/vdso/vclock_gettime.o arch/x86/entry/vdso/vgetcpu.o arch/x86/entry/vdso/vgetrandom.o arch/x86/entry/vdso/vgetrandom-chacha.o"
- self._assert_parsing(cmd, expected)
-
- def test_ld_with_env_override(self):
- with patch.dict(os.environ, {"LD": "some-tool ld"}):
- registry = CommandParserRegistry.create()
- cmd = r'ld -o arch/x86/entry/vdso/vdso64.so.dbg -shared --hash-style=both --build-id=sha1 --no-undefined --eh-frame-hdr -Bsymbolic -z noexecstack -m elf_x86_64 -soname linux-vdso.so.1 -z max-page-size=4096 -T arch/x86/entry/vdso/vdso.lds arch/x86/entry/vdso/vdso-note.o arch/x86/entry/vdso/vclock_gettime.o arch/x86/entry/vdso/vgetcpu.o arch/x86/entry/vdso/vgetrandom.o arch/x86/entry/vdso/vgetrandom-chacha.o; if readelf -rW arch/x86/entry/vdso/vdso64.so.dbg | grep -v _NONE | grep -q " R_\w*_"; then (echo >&2 "arch/x86/entry/vdso/vdso64.so.dbg: dynamic relocations are not supported"; rm -f arch/x86/entry/vdso/vdso64.so.dbg; /bin/false); fi'
- expected = "arch/x86/entry/vdso/vdso-note.o arch/x86/entry/vdso/vclock_gettime.o arch/x86/entry/vdso/vgetcpu.o arch/x86/entry/vdso/vgetrandom.o arch/x86/entry/vdso/vgetrandom-chacha.o"
- self._assert_parsing(cmd, expected, registry)
- self._assert_parsing(f"some-tool {cmd}", expected, registry)
-
- def test_ld_whole_archive(self):
- cmd = "ld -m elf_x86_64 -z noexecstack -r -o vmlinux.o --whole-archive vmlinux.a --no-whole-archive --start-group --end-group"
- expected = "vmlinux.a"
- self._assert_parsing(cmd, expected)
-
- def test_ld_with_at_symbol(self):
- cmd = "ld.lld -m elf_x86_64 -z noexecstack -r -o fs/efivarfs/efivarfs.o @fs/efivarfs/efivarfs.mod ; ./tools/objtool/objtool --hacks=jump_label --hacks=noinstr --hacks=skylake --ibt --orc --retpoline --rethunk --static-call --uaccess --prefix=16 --link --module fs/efivarfs/efivarfs.o"
- expected = "@fs/efivarfs/efivarfs.mod"
- self._assert_parsing(cmd, expected)
-
- def test_ld_if_objdump(self):
- cmd = """ld -o arch/x86/entry/vdso/vdso64.so.dbg -shared --hash-style=both --build-id=sha1 --eh-frame-hdr -Bsymbolic -z noexecstack -m elf_x86_64 -soname linux-vdso.so.1 --no-undefined -z max-page-size=4096 -T arch/x86/entry/vdso/vdso.lds arch/x86/entry/vdso/vdso-note.o arch/x86/entry/vdso/vclock_gettime.o arch/x86/entry/vdso/vgetcpu.o arch/x86/entry/vdso/vsgx.o && sh ./arch/x86/entry/vdso/checkundef.sh 'nm' 'arch/x86/entry/vdso/vdso64.so.dbg'; if objdump -R arch/x86/entry/vdso/vdso64.so.dbg | grep -E -h "R_X86_64_JUMP_SLOT|R_X86_64_GLOB_DAT|R_X86_64_RELATIVE| R_386_GLOB_DAT|R_386_JMP_SLOT|R_386_RELATIVE"; then (echo >&2 "arch/x86/entry/vdso/vdso64.so.dbg: dynamic relocations are not supported"; rm -f arch/x86/entry/vdso/vdso64.so.dbg; /bin/false); fi"""
- expected = "arch/x86/entry/vdso/vdso-note.o arch/x86/entry/vdso/vclock_gettime.o arch/x86/entry/vdso/vgetcpu.o arch/x86/entry/vdso/vsgx.o"
- self._assert_parsing(cmd, expected)
-
- # printf | xargs ar command tests
- def test_ar_printf(self):
- cmd = 'rm -f built-in.a; printf "./%s " init/built-in.a usr/built-in.a arch/x86/built-in.a arch/x86/boot/startup/built-in.a kernel/built-in.a certs/built-in.a mm/built-in.a fs/built-in.a ipc/built-in.a security/built-in.a crypto/built-in.a block/built-in.a io_uring/built-in.a lib/built-in.a arch/x86/lib/built-in.a drivers/built-in.a sound/built-in.a net/built-in.a virt/built-in.a arch/x86/pci/built-in.a arch/x86/power/built-in.a arch/x86/video/built-in.a | xargs ar cDPrST built-in.a'
- expected = "./init/built-in.a ./usr/built-in.a ./arch/x86/built-in.a ./arch/x86/boot/startup/built-in.a ./kernel/built-in.a ./certs/built-in.a ./mm/built-in.a ./fs/built-in.a ./ipc/built-in.a ./security/built-in.a ./crypto/built-in.a ./block/built-in.a ./io_uring/built-in.a ./lib/built-in.a ./arch/x86/lib/built-in.a ./drivers/built-in.a ./sound/built-in.a ./net/built-in.a ./virt/built-in.a ./arch/x86/pci/built-in.a ./arch/x86/power/built-in.a ./arch/x86/video/built-in.a"
- self._assert_parsing(cmd, expected)
-
- def test_ar_printf_nested(self):
- cmd = 'rm -f arch/x86/pci/built-in.a; printf "arch/x86/pci/%s " i386.o init.o mmconfig_64.o direct.o mmconfig-shared.o fixup.o acpi.o legacy.o irq.o common.o early.o bus_numa.o amd_bus.o | xargs ar cDPrST arch/x86/pci/built-in.a'
- expected = "arch/x86/pci/i386.o arch/x86/pci/init.o arch/x86/pci/mmconfig_64.o arch/x86/pci/direct.o arch/x86/pci/mmconfig-shared.o arch/x86/pci/fixup.o arch/x86/pci/acpi.o arch/x86/pci/legacy.o arch/x86/pci/irq.o arch/x86/pci/common.o arch/x86/pci/early.o arch/x86/pci/bus_numa.o arch/x86/pci/amd_bus.o"
- self._assert_parsing(cmd, expected)
-
- # ar command tests
- def test_ar_reordering(self):
- cmd = "rm -f vmlinux.a; ar cDPrST vmlinux.a built-in.a lib/lib.a arch/x86/lib/lib.a; ar mPiT $$(ar t vmlinux.a | sed -n 1p) vmlinux.a $$(ar t vmlinux.a | grep -F -f ../scripts/head-object-list.txt)"
- expected = "built-in.a lib/lib.a arch/x86/lib/lib.a"
- self._assert_parsing(cmd, expected)
-
- def test_ar_default(self):
- cmd = "rm -f lib/lib.a; ar cDPrsT lib/lib.a lib/argv_split.o lib/bug.o lib/buildid.o lib/clz_tab.o lib/cmdline.o lib/cpumask.o lib/ctype.o lib/dec_and_lock.o lib/decompress.o lib/decompress_bunzip2.o lib/decompress_inflate.o lib/decompress_unlz4.o lib/decompress_unlzma.o lib/decompress_unlzo.o lib/decompress_unxz.o lib/decompress_unzstd.o lib/dump_stack.o lib/earlycpio.o lib/extable.o lib/flex_proportions.o lib/idr.o lib/iomem_copy.o lib/irq_regs.o lib/is_single_threaded.o lib/klist.o lib/kobject.o lib/kobject_uevent.o lib/logic_pio.o lib/maple_tree.o lib/memcat_p.o lib/nmi_backtrace.o lib/objpool.o lib/plist.o lib/radix-tree.o lib/ratelimit.o lib/rbtree.o lib/seq_buf.o lib/siphash.o lib/string.o lib/sys_info.o lib/timerqueue.o lib/union_find.o lib/vsprintf.o lib/win_minmax.o lib/xarray.o"
- expected = "lib/argv_split.o lib/bug.o lib/buildid.o lib/clz_tab.o lib/cmdline.o lib/cpumask.o lib/ctype.o lib/dec_and_lock.o lib/decompress.o lib/decompress_bunzip2.o lib/decompress_inflate.o lib/decompress_unlz4.o lib/decompress_unlzma.o lib/decompress_unlzo.o lib/decompress_unxz.o lib/decompress_unzstd.o lib/dump_stack.o lib/earlycpio.o lib/extable.o lib/flex_proportions.o lib/idr.o lib/iomem_copy.o lib/irq_regs.o lib/is_single_threaded.o lib/klist.o lib/kobject.o lib/kobject_uevent.o lib/logic_pio.o lib/maple_tree.o lib/memcat_p.o lib/nmi_backtrace.o lib/objpool.o lib/plist.o lib/radix-tree.o lib/ratelimit.o lib/rbtree.o lib/seq_buf.o lib/siphash.o lib/string.o lib/sys_info.o lib/timerqueue.o lib/union_find.o lib/vsprintf.o lib/win_minmax.o lib/xarray.o"
- self._assert_parsing(cmd, expected)
-
- def test_ar_llvm(self):
- cmd = "llvm-ar mPiT $$(llvm-ar t vmlinux.a | sed -n 1p) vmlinux.a $$(llvm-ar t vmlinux.a | grep -F -f ../scripts/head-object-list.txt)"
- expected = ""
- self._assert_parsing(cmd, expected)
-
- # nm command tests
- def test_nm(self):
- cmd = """llvm-nm -p --defined-only rust/core.o | awk '$$2~/(T|R|D|B)/ && $$3!~/__(pfx|cfi|odr_asan)/ { printf "EXPORT_SYMBOL_RUST_GPL(%s);\n",$$3 }' > rust/exports_core_generated.h"""
- expected = "rust/core.o"
- self._assert_parsing(cmd, expected)
-
- def test_nm_vmlinux(self):
- cmd = r"nm vmlinux | sed -n -e 's/^\([0-9a-fA-F]*\) [ABbCDGRSTtVW] \(_text\|__start_rodata\|__bss_start\|_end\)$/#define VO_\2 _AC(0x\1,UL)/p' > arch/x86/boot/voffset.h"
- expected = "vmlinux"
- self._assert_parsing(cmd, expected)
-
- # objcopy command tests
- def test_objcopy(self):
- cmd = "objcopy --remove-section='.rel*' --remove-section=!'.rel*.dyn' vmlinux.unstripped vmlinux"
- expected = "vmlinux.unstripped"
- self._assert_parsing(cmd, expected)
-
- def test_objcopy_llvm(self):
- cmd = "llvm-objcopy --remove-section='.rel*' --remove-section=!'.rel*.dyn' vmlinux.unstripped vmlinux"
- expected = "vmlinux.unstripped"
- self._assert_parsing(cmd, expected)
-
- # strip command tests
- def test_strip(self):
- cmd = "strip --strip-debug -o drivers/firmware/efi/libstub/mem.stub.o drivers/firmware/efi/libstub/mem.o"
- expected = "drivers/firmware/efi/libstub/mem.o"
- self._assert_parsing(cmd, expected)
-
- # cp command tests
- def test_cp_truncate(self):
- cmd = "cp arch/arm64/boot/Image arch/arm64/boot/vmlinux.bin; truncate -s $$(hexdump -s16 -n4 -e '\"%u\"' arch/arm64/boot/Image) arch/arm64/boot/vmlinux.bin"
- expected = "arch/arm64/boot/Image"
- self._assert_parsing(cmd, expected)
-
- # rustc command tests
- def test_rustc(self):
- cmd = """OBJTREE=/workspace/linux/kernel_build rustc -Zbinary_dep_depinfo=y -Astable_features -Dnon_ascii_idents -Dunsafe_op_in_unsafe_fn -Wmissing_docs -Wrust_2018_idioms -Wclippy::all -Wclippy::as_ptr_cast_mut -Wclippy::as_underscore -Wclippy::cast_lossless -Wclippy::ignored_unit_patterns -Wclippy::mut_mut -Wclippy::needless_bitwise_bool -Aclippy::needless_lifetimes -Wclippy::no_mangle_with_rust_abi -Wclippy::ptr_as_ptr -Wclippy::ptr_cast_constness -Wclippy::ref_as_ptr -Wclippy::undocumented_unsafe_blocks -Wclippy::unnecessary_safety_comment -Wclippy::unnecessary_safety_doc -Wrustdoc::missing_crate_level_docs -Wrustdoc::unescaped_backticks -Cpanic=abort -Cembed-bitcode=n -Clto=n -Cforce-unwind-tables=n -Ccodegen-units=1 -Csymbol-mangling-version=v0 -Crelocation-model=static -Zfunction-sections=n -Wclippy::float_arithmetic --target=./scripts/target.json -Ctarget-feature=-sse,-sse2,-sse3,-ssse3,-sse4.1,-sse4.2,-avx,-avx2 -Zcf-protection=branch -Zno-jump-tables -Ctarget-cpu=x86-64 -Ztune-cpu=generic -Cno-redzone=y -Ccode-model=kernel -Zfunction-return=thunk-extern -Zpatchable-function-entry=16,16 -Copt-level=2 -Cdebug-assertions=n -Coverflow-checks=y -Dwarnings @./include/generated/rustc_cfg --edition=2021 --cfg no_fp_fmt_parse --emit=dep-info=rust/.core.o.d --emit=obj=rust/core.o --emit=metadata=rust/libcore.rmeta --crate-type rlib -L./rust --crate-name core /usr/lib/rust-1.84/lib/rustlib/src/rust/library/core/src/lib.rs --sysroot=/dev/null ;llvm-objcopy --redefine-sym __addsf3=__rust__addsf3 --redefine-sym __eqsf2=__rust__eqsf2 --redefine-sym __extendsfdf2=__rust__extendsfdf2 --redefine-sym __gesf2=__rust__gesf2 --redefine-sym __lesf2=__rust__lesf2 --redefine-sym __ltsf2=__rust__ltsf2 --redefine-sym __mulsf3=__rust__mulsf3 --redefine-sym __nesf2=__rust__nesf2 --redefine-sym __truncdfsf2=__rust__truncdfsf2 --redefine-sym __unordsf2=__rust__unordsf2 --redefine-sym __adddf3=__rust__adddf3 --redefine-sym __eqdf2=__rust__eqdf2 --redefine-sym __ledf2=__rust__ledf2 --redefine-sym __ltdf2=__rust__ltdf2 --redefine-sym __muldf3=__rust__muldf3 --redefine-sym __unorddf2=__rust__unorddf2 --redefine-sym __muloti4=__rust__muloti4 --redefine-sym __multi3=__rust__multi3 --redefine-sym __udivmodti4=__rust__udivmodti4 --redefine-sym __udivti3=__rust__udivti3 --redefine-sym __umodti3=__rust__umodti3 rust/core.o"""
- expected = "/usr/lib/rust-1.84/lib/rustlib/src/rust/library/core/src/lib.rs rust/core.o"
- self._assert_parsing(cmd, expected)
-
- # rustdoc command tests
- def test_rustdoc(self):
- cmd = """OBJTREE=/workspace/linux/kernel_build rustdoc --test --edition=2021 -Zbinary_dep_depinfo=y -Astable_features -Dnon_ascii_idents -Dunsafe_op_in_unsafe_fn -Wmissing_docs -Wrust_2018_idioms -Wunreachable_pub -Wclippy::all -Wclippy::as_ptr_cast_mut -Wclippy::as_underscore -Wclippy::cast_lossless -Wclippy::ignored_unit_patterns -Wclippy::mut_mut -Wclippy::needless_bitwise_bool -Aclippy::needless_lifetimes -Wclippy::no_mangle_with_rust_abi -Wclippy::ptr_as_ptr -Wclippy::ptr_cast_constness -Wclippy::ref_as_ptr -Wclippy::undocumented_unsafe_blocks -Wclippy::unnecessary_safety_comment -Wclippy::unnecessary_safety_doc -Wrustdoc::missing_crate_level_docs -Wrustdoc::unescaped_backticks -Cpanic=abort -Cembed-bitcode=n -Clto=n -Cforce-unwind-tables=n -Ccodegen-units=1 -Csymbol-mangling-version=v0 -Crelocation-model=static -Zfunction-sections=n -Wclippy::float_arithmetic --target=aarch64-unknown-none -Ctarget-feature="-neon" -Cforce-unwind-tables=n -Zbranch-protection=pac-ret -Copt-level=2 -Cdebug-assertions=y -Coverflow-checks=y -Dwarnings -Cforce-frame-pointers=y -Zsanitizer=kernel-address -Zsanitizer-recover=kernel-address -Cllvm-args=-asan-mapping-offset=0xdfff800000000000 -Cpasses=sancov-module -Cllvm-args=-sanitizer-coverage-level=3 -Cllvm-args=-sanitizer-coverage-trace-pc -Cllvm-args=-sanitizer-coverage-trace-compares @./include/generated/rustc_cfg -L./rust --extern ffi --extern pin_init --extern kernel --extern build_error --extern macros --extern bindings --extern uapi --no-run --crate-name kernel -Zunstable-options --sysroot=/dev/null --test-builder ./scripts/rustdoc_test_builder ../rust/kernel/lib.rs >/dev/null"""
- expected = "../rust/kernel/lib.rs"
- self._assert_parsing(cmd, expected)
-
- def test_rustdoc_test_gen(self):
- cmd = "./scripts/rustdoc_test_gen"
- expected = ""
- self._assert_parsing(cmd, expected)
-
- # flex command tests
- def test_flex(self):
- cmd = "flex -oscripts/kconfig/lexer.lex.c -L ../scripts/kconfig/lexer.l"
- expected = "../scripts/kconfig/lexer.l"
- self._assert_parsing(cmd, expected)
-
- # bison command tests
- def test_bison(self):
- cmd = "bison -o scripts/kconfig/parser.tab.c --defines=scripts/kconfig/parser.tab.h -t -l ../scripts/kconfig/parser.y"
- expected = "../scripts/kconfig/parser.y"
- self._assert_parsing(cmd, expected)
-
- # bindgen command tests
- def test_bindgen(self):
- cmd = (
- "bindgen ../rust/bindings/bindings_helper.h "
- "--blocklist-type __kernel_s?size_t --blocklist-type __kernel_ptrdiff_t "
- "--opaque-type xregs_state --opaque-type desc_struct --no-doc-comments "
- "--rust-target 1.68 --use-core --with-derive-default -o rust/bindings/bindings_generated.rs "
- "-- -Wp,-MMD,rust/bindings/.bindings_generated.rs.d -nostdinc -I../arch/x86/include "
- "-include ../include/linux/compiler-version.h -D__KERNEL__ -fintegrated-as -fno-builtin -DMODULE; "
- "sed -Ei 's/pub const RUST_CONST_HELPER_([a-zA-Z0-9_]*)/pub const \\1/g' rust/bindings/bindings_generated.rs"
- )
- expected = "../rust/bindings/bindings_helper.h ../include/linux/compiler-version.h"
- self._assert_parsing(cmd, expected)
-
- # perl command tests
- def test_perl(self):
- cmd = "perl ../lib/crypto/x86/poly1305-x86_64-cryptogams.pl > lib/crypto/x86/poly1305-x86_64-cryptogams.S"
- expected = "../lib/crypto/x86/poly1305-x86_64-cryptogams.pl"
- self._assert_parsing(cmd, expected)
-
- # link-vmlinux.sh command tests
- def test_link_vmlinux(self):
- cmd = '../scripts/link-vmlinux.sh "ld" "-m elf_x86_64 -z noexecstack" "-z max-page-size=0x200000 --build-id=sha1 --orphan-handling=error --emit-relocs --discard-none" "vmlinux.unstripped"; true'
- expected = "vmlinux.a"
- self._assert_parsing(cmd, expected)
-
- def test_link_vmlinux_postlink(self):
- cmd = '../scripts/link-vmlinux.sh "ld" "-m elf_x86_64 -z noexecstack --no-warn-rwx-segments" "--emit-relocs --discard-none -z max-page-size=0x200000 --build-id=sha1 -X --orphan-handling=error"; make -f ../arch/x86/Makefile.postlink vmlinux'
- expected = "vmlinux.a"
- self._assert_parsing(cmd, expected)
-
- # syscallhdr.sh command tests
- def test_syscallhdr(self):
- cmd = "sh ../scripts/syscallhdr.sh --abis common,64 --emit-nr ../arch/x86/entry/syscalls/syscall_64.tbl arch/x86/include/generated/uapi/asm/unistd_64.h"
- expected = "../arch/x86/entry/syscalls/syscall_64.tbl"
- self._assert_parsing(cmd, expected)
-
- # syscalltbl.sh command tests
- def test_syscalltbl(self):
- cmd = "sh ../scripts/syscalltbl.sh --abis common,64 ../arch/x86/entry/syscalls/syscall_64.tbl arch/x86/include/generated/asm/syscalls_64.h"
- expected = "../arch/x86/entry/syscalls/syscall_64.tbl"
- self._assert_parsing(cmd, expected)
-
- # mkcapflags.sh command tests
- def test_mkcapflags(self):
- cmd = "sh ../arch/x86/kernel/cpu/mkcapflags.sh arch/x86/kernel/cpu/capflags.c ../arch/x86/kernel/cpu/../../include/asm/cpufeatures.h ../arch/x86/kernel/cpu/../../include/asm/vmxfeatures.h ../arch/x86/kernel/cpu/mkcapflags.sh FORCE"
- expected = "../arch/x86/kernel/cpu/../../include/asm/cpufeatures.h ../arch/x86/kernel/cpu/../../include/asm/vmxfeatures.h"
- self._assert_parsing(cmd, expected)
-
- # orc_hash.sh command tests
- def test_orc_hash(self):
- cmd = "mkdir -p arch/x86/include/generated/asm/; sh ../scripts/orc_hash.sh < ../arch/x86/include/asm/orc_types.h > arch/x86/include/generated/asm/orc_hash.h"
- expected = "../arch/x86/include/asm/orc_types.h"
- self._assert_parsing(cmd, expected)
-
- # xen-hypercalls.sh command tests
- def test_xen_hypercalls(self):
- cmd = "sh '../scripts/xen-hypercalls.sh' arch/x86/include/generated/asm/xen-hypercalls.h ../include/xen/interface/xen-mca.h ../include/xen/interface/xen.h ../include/xen/interface/xenpmu.h"
- expected = "../include/xen/interface/xen-mca.h ../include/xen/interface/xen.h ../include/xen/interface/xenpmu.h"
- self._assert_parsing(cmd, expected)
-
- # gen_initramfs.sh command tests
- def test_gen_initramfs(self):
- cmd = "sh ../usr/gen_initramfs.sh -o usr/initramfs_data.cpio -l usr/.initramfs_data.cpio.d ../usr/default_cpio_list"
- expected = "../usr/default_cpio_list"
- self._assert_parsing(cmd, expected)
-
- # mkuboot.sh command tests
- def test_mkuboot(self):
- cmd = "bash ../scripts/mkuboot.sh -A arm -O linux -C none -T kernel -a 0x8000 -e 0x8000 -n 'Linux-6.15.0' -d arch/arm/boot/zImage arch/arm/boot/uImage"
- expected = "arch/arm/boot/zImage"
- self._assert_parsing(cmd, expected)
-
- # syscallnr.sh command tests
- def test_syscallnr(self):
- cmd = "sh ../arch/arm/tools/syscallnr.sh ../arch/arm/tools/syscall.tbl arch/arm/include/generated/asm/unistd-nr.h"
- expected = "../arch/arm/tools/syscall.tbl"
- self._assert_parsing(cmd, expected)
-
- # gen-kernel-hwcaps.sh command tests
- def test_gen_kernel_hwcaps(self):
- cmd = "/bin/sh -e ../arch/arm64/tools/gen-kernel-hwcaps.sh ../arch/arm64/include/uapi/asm/hwcap.h > arch/arm64/include/generated/asm/kernel-hwcap.h"
- expected = "../arch/arm64/include/uapi/asm/hwcap.h"
- self._assert_parsing(cmd, expected)
-
- # vdso2c command tests
- def test_vdso2c(self):
- cmd = "arch/x86/entry/vdso/vdso2c arch/x86/entry/vdso/vdso64.so.dbg arch/x86/entry/vdso/vdso64.so arch/x86/entry/vdso/vdso-image-64.c"
- expected = "arch/x86/entry/vdso/vdso64.so.dbg arch/x86/entry/vdso/vdso64.so"
- self._assert_parsing(cmd, expected)
-
- # vdsomunge command tests
- def test_vdsomunge(self):
- cmd = "arch/arm64/kernel/vdso32/../../../arm/vdso/vdsomunge arch/arm64/kernel/vdso32/vdso.so.raw arch/arm64/kernel/vdso32/vdso32.so.dbg"
- expected = "arch/arm64/kernel/vdso32/vdso.so.raw"
- self._assert_parsing(cmd, expected)
-
- # mkpiggy command tests
- def test_mkpiggy(self):
- cmd = "arch/x86/boot/compressed/mkpiggy arch/x86/boot/compressed/vmlinux.bin.gz > arch/x86/boot/compressed/piggy.S"
- expected = "arch/x86/boot/compressed/vmlinux.bin.gz"
- self._assert_parsing(cmd, expected)
-
- # relocs command tests
- def test_relocs(self):
- cmd = "arch/x86/tools/relocs vmlinux.unstripped > arch/x86/boot/compressed/vmlinux.relocs;arch/x86/tools/relocs --abs-relocs vmlinux.unstripped"
- expected = "vmlinux.unstripped"
- self._assert_parsing(cmd, expected)
-
- def test_relocs_with_realmode(self):
- cmd = (
- "arch/x86/tools/relocs --realmode arch/x86/realmode/rm/realmode.elf > arch/x86/realmode/rm/realmode.relocs"
- )
- expected = "arch/x86/realmode/rm/realmode.elf"
- self._assert_parsing(cmd, expected)
-
- # mk_elfconfig command tests
- def test_mk_elfconfig(self):
- cmd = "scripts/mod/mk_elfconfig < scripts/mod/empty.o > scripts/mod/elfconfig.h"
- expected = "scripts/mod/empty.o"
- self._assert_parsing(cmd, expected)
-
- # tools/build command tests
- def test_build(self):
- cmd = "arch/x86/boot/tools/build arch/x86/boot/setup.bin arch/x86/boot/vmlinux.bin arch/x86/boot/zoffset.h arch/x86/boot/bzImage"
- expected = "arch/x86/boot/setup.bin arch/x86/boot/vmlinux.bin arch/x86/boot/zoffset.h"
- self._assert_parsing(cmd, expected)
-
- # extract-cert command tests
- def test_extract_cert(self):
- cmd = 'certs/extract-cert "" certs/signing_key.x509'
- expected = ""
- self._assert_parsing(cmd, expected)
-
- # dtc command tests
- def test_dtc_cat(self):
- cmd = "./scripts/dtc/dtc -o drivers/of/empty_root.dtb -b 0 -i../drivers/of/ -i../scripts/dtc/include-prefixes -Wno-unique_unit_address -Wno-unit_address_vs_reg -Wno-avoid_unnecessary_addr_size -Wno-alias_paths -Wno-graph_child_address -Wno-simple_bus_reg -d drivers/of/.empty_root.dtb.d.dtc.tmp drivers/of/.empty_root.dtb.dts.tmp ; cat drivers/of/.empty_root.dtb.d.pre.tmp drivers/of/.empty_root.dtb.d.dtc.tmp > drivers/of/.empty_root.dtb.d"
- expected = "drivers/of/.empty_root.dtb.dts.tmp drivers/of/.empty_root.dtb.d.pre.tmp drivers/of/.empty_root.dtb.d.dtc.tmp"
- self._assert_parsing(cmd, expected)
-
- # pnmtologo command tests
- def test_pnmtologo(self):
- cmd = "drivers/video/logo/pnmtologo -t clut224 -n logo_linux_clut224 -o drivers/video/logo/logo_linux_clut224.c ../drivers/video/logo/logo_linux_clut224.ppm"
- expected = "../drivers/video/logo/logo_linux_clut224.ppm"
- self._assert_parsing(cmd, expected)
-
- # relacheck command tests
- def test_relacheck(self):
- cmd = "arch/arm64/kernel/pi/relacheck arch/arm64/kernel/pi/idreg-override.pi.o arch/arm64/kernel/pi/idreg-override.o"
- expected = "arch/arm64/kernel/pi/idreg-override.pi.o"
- self._assert_parsing(cmd, expected)
-
- # gen-hyprel command tests
- def test_gen_hyprel(self):
- cmd = "arch/arm64/kvm/hyp/nvhe/gen-hyprel arch/arm64/kvm/hyp/nvhe/kvm_nvhe.tmp.o > arch/arm64/kvm/hyp/nvhe/hyp-reloc.S"
- expected = "arch/arm64/kvm/hyp/nvhe/kvm_nvhe.tmp.o"
- self._assert_parsing(cmd, expected)
-
- # mkregtable command tests
- def test_mkregtable(self):
- cmd = "drivers/gpu/drm/radeon/mkregtable ../drivers/gpu/drm/radeon/reg_srcs/r100 > drivers/gpu/drm/radeon/r100_reg_safe.h"
- expected = "../drivers/gpu/drm/radeon/reg_srcs/r100"
- self._assert_parsing(cmd, expected)
-
- # genheaders command tests
- def test_genheaders(self):
- cmd = "security/selinux/genheaders security/selinux/flask.h security/selinux/av_permissions.h"
- expected = ""
- self._assert_parsing(cmd, expected)
-
- # mkcpustr command tests
- def test_mkcpustr(self):
- cmd = "arch/x86/boot/mkcpustr > arch/x86/boot/cpustr.h"
- expected = ""
- self._assert_parsing(cmd, expected)
-
- # polgen command tests
- def test_polgen(self):
- cmd = "scripts/ipe/polgen/polgen security/ipe/boot_policy.c"
- expected = ""
- self._assert_parsing(cmd, expected)
-
- # gen_header.py command tests
- def test_gen_header(self):
- cmd = "mkdir -p drivers/gpu/drm/msm/generated && python3 ../drivers/gpu/drm/msm/registers/gen_header.py --no-validate --rnn ../drivers/gpu/drm/msm/registers --xml ../drivers/gpu/drm/msm/registers/adreno/a2xx.xml c-defines > drivers/gpu/drm/msm/generated/a2xx.xml.h"
- expected = "../drivers/gpu/drm/msm/registers/adreno/a2xx.xml"
- self._assert_parsing(cmd, expected)
-
-
-if __name__ == "__main__":
- unittest.main()
--
2.43.0
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2026-10-12 21:34 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-10-12 21:33 [PATCH v3 0/7] kbuild: record cmd-prereqs in .cmd files Luis Augenstein
2026-10-12 21:33 ` [PATCH v3 1/7] scripts/make_fit: limit .cmd file parsing to the savedcmd_ line Luis Augenstein
2026-10-12 21:33 ` [PATCH v3 2/7] kbuild: refactor fixdep to collect dependencies before printing Luis Augenstein
2026-10-12 21:33 ` [PATCH v3 3/7] scripts/sbom: skip make_prereqs lines when parsing .cmd files Luis Augenstein
2026-10-12 21:33 ` [PATCH v3 4/7] kbuild: record cmd-prereqs in " Luis Augenstein
2026-10-12 21:33 ` [PATCH v3 5/7] scripts/sbom: parse make_prereqs from " Luis Augenstein
2026-10-12 21:33 ` [PATCH v3 6/7] scripts/sbom: use make_prereqs to determine build inputs Luis Augenstein
2026-10-12 21:33 ` [PATCH v3 7/7] scripts/sbom: remove the command-specific parsers Luis Augenstein
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®