[PATCH] kbuild: prevent building single targets twice

[PATCH] kbuild: prevent building single targets twice

[Mathias Krause]

When building individual targets via 'make path/to/target.o', target.o
may be build multiple times concurrently, leading to build errors like
below:

$ make O=~/O/vanilla-7.1/ -j$(nproc) arch/x86/pci/intel_mid.o
make[1]: Entering directory '/scratch/minipli/obj/vanilla-7.1'
  DESCEND objtool
  DESCEND bpf/resolve_btfids
  CC      arch/x86/pci/intel_mid.o
  CC      arch/x86/pci/intel_mid.o
fixdep: error opening file: arch/x86/pci/.intel_mid.o.d: No such file or directory
make[4]: *** [/scratch/minipli/src/linux/scripts/Makefile.build:290: arch/x86/pci/intel_mid.o] Error 2
make[4]: *** Deleting file 'arch/x86/pci/intel_mid.o'
make[3]: *** [/scratch/minipli/src/linux/scripts/Makefile.build:551: arch/x86] Error 2
make[3]: *** Waiting for unfinished jobs....
make[2]: *** [/scratch/minipli/src/linux/Makefile:2229: .] Error 2
make[1]: *** [/scratch/minipli/src/linux/Makefile:248: __sub-make] Error 2
make[1]: Leaving directory '/scratch/minipli/obj/vanilla-7.1'
make: *** [Makefile:248: __sub-make] Error 2

Please note that arch/x86/pci/intel_mid.o gets output twice and, in
fact, tried to get built twice, causing the two parallel builds to
interfere with each other.

The reason for the double build attempt is that the target can be
reached twice, via arch/x86/'s Makefile and via arch/x86/pci/'s which
both end up in '$(single-subdirs)' as both fit the prefix match.

Only the latter should be used as it contains a cflags override for
CONFIG_PCI_DEBUG=y which would be skipped when using arch/x86/Makefile
to build arch/x86/pci/intel_mid.o.

Fix the multi-match issue by using only the deepest match instead of all
possible ones, which requires two changes:
1. Limiting 'single-subdirs' to a single match per target and
2. filtering out deeper targets in the recursive make invocation.

Also de-duplicate the list of target subdirectories to simplify the
use of '$(single-subdirs)' in filter expressions for multiple targets in
the same directory.

Fixes: cc306abd19e8 ("kbuild: fix and refactor single target build")
Assisted-by: LLM
Signed-off-by: Mathias Krause <minipli@grsecurity.net>
---
I ran into this quite a few times in the past and only now gave the LLM
a try to dig into it. The bug only affects certain targets and only when
mentioned on the command line, so regular 'make -j$(nproc)' isn't
affected.

 scripts/Makefile.build | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 0f4fdc119e58..d5f58513d02a 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -534,7 +534,10 @@ endif
 # Single targets
 # ---------------------------------------------------------------------------
 
-single-subdirs := $(foreach d, $(subdir-ym), $(if $(filter $d/%, $(MAKECMDGOALS)), $d))
+# A single target may match nested entries in subdir-ym.  Descend through only
+# the deepest match so that two recursive makes do not build the same target.
+single-subdir = $(lastword $(sort $(foreach d, $(subdir-ym), $(if $(filter $d/%, $(1)), $d))))
+single-subdirs := $(sort $(foreach g, $(MAKECMDGOALS), $(call single-subdir, $g)))
 single-subdir-goals := $(filter $(addsuffix /%, $(single-subdirs)), $(MAKECMDGOALS))
 
 $(single-subdir-goals): $(single-subdirs)
@@ -548,7 +551,8 @@ $(subdir-ym):
 	$(Q)$(MAKE) $(build)=$@ \
 	need-builtin=$(if $(filter $@/built-in.a, $(subdir-builtin)),1) \
 	need-modorder=$(if $(filter $@/modules.order, $(subdir-modorder)),1) \
-	$(filter $@/%, $(single-subdir-goals))
+	$(filter-out $(addsuffix /%, $(filter $@/%, $(single-subdirs))), \
+		$(filter $@/%, $(single-subdir-goals)))
 
 # Add FORCE to the prerequisites of a target to force it to be always rebuilt.
 # ---------------------------------------------------------------------------
-- 
2.47.3

Re: [PATCH] kbuild: prevent building single targets twice

[Mathias Krause]

On 23.09.26 14:39, Mathias Krause wrote:
> When building individual targets via 'make path/to/target.o', target.o
> may be build multiple times concurrently, leading to build errors like
> below:
> 
> $ make O=~/O/vanilla-7.1/ -j$(nproc) arch/x86/pci/intel_mid.o
> make[1]: Entering directory '/scratch/minipli/obj/vanilla-7.1'
>   DESCEND objtool
>   DESCEND bpf/resolve_btfids
>   CC      arch/x86/pci/intel_mid.o
>   CC      arch/x86/pci/intel_mid.o
> fixdep: error opening file: arch/x86/pci/.intel_mid.o.d: No such file or directory
> make[4]: *** [/scratch/minipli/src/linux/scripts/Makefile.build:290: arch/x86/pci/intel_mid.o] Error 2
> make[4]: *** Deleting file 'arch/x86/pci/intel_mid.o'
> make[3]: *** [/scratch/minipli/src/linux/scripts/Makefile.build:551: arch/x86] Error 2
> make[3]: *** Waiting for unfinished jobs....
> make[2]: *** [/scratch/minipli/src/linux/Makefile:2229: .] Error 2
> make[1]: *** [/scratch/minipli/src/linux/Makefile:248: __sub-make] Error 2
> make[1]: Leaving directory '/scratch/minipli/obj/vanilla-7.1'
> make: *** [Makefile:248: __sub-make] Error 2
> 
> Please note that arch/x86/pci/intel_mid.o gets output twice and, in
> fact, tried to get built twice, causing the two parallel builds to
> interfere with each other.
> 
> The reason for the double build attempt is that the target can be
> reached twice, via arch/x86/'s Makefile and via arch/x86/pci/'s which
> both end up in '$(single-subdirs)' as both fit the prefix match.
> 
> Only the latter should be used as it contains a cflags override for
> CONFIG_PCI_DEBUG=y which would be skipped when using arch/x86/Makefile
> to build arch/x86/pci/intel_mid.o.
> 
> Fix the multi-match issue by using only the deepest match instead of all
> possible ones, which requires two changes:
> 1. Limiting 'single-subdirs' to a single match per target and
> 2. filtering out deeper targets in the recursive make invocation.
> 
> Also de-duplicate the list of target subdirectories to simplify the
> use of '$(single-subdirs)' in filter expressions for multiple targets in
> the same directory.
> 
> Fixes: cc306abd19e8 ("kbuild: fix and refactor single target build")
> Assisted-by: LLM
> Signed-off-by: Mathias Krause <minipli@grsecurity.net>
> ---
> I ran into this quite a few times in the past and only now gave the LLM
> a try to dig into it. The bug only affects certain targets and only when
> mentioned on the command line, so regular 'make -j$(nproc)' isn't
> affected.
> 
>  scripts/Makefile.build | 8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/scripts/Makefile.build b/scripts/Makefile.build
> index 0f4fdc119e58..d5f58513d02a 100644
> --- a/scripts/Makefile.build
> +++ b/scripts/Makefile.build
> @@ -534,7 +534,10 @@ endif
>  # Single targets
>  # ---------------------------------------------------------------------------
>  
> -single-subdirs := $(foreach d, $(subdir-ym), $(if $(filter $d/%, $(MAKECMDGOALS)), $d))
> +# A single target may match nested entries in subdir-ym.  Descend through only
> +# the deepest match so that two recursive makes do not build the same target.
> +single-subdir = $(lastword $(sort $(foreach d, $(subdir-ym), $(if $(filter $d/%, $(1)), $d))))
> +single-subdirs := $(sort $(foreach g, $(MAKECMDGOALS), $(call single-subdir, $g)))
>  single-subdir-goals := $(filter $(addsuffix /%, $(single-subdirs)), $(MAKECMDGOALS))
>  
>  $(single-subdir-goals): $(single-subdirs)
> @@ -548,7 +551,8 @@ $(subdir-ym):
>  	$(Q)$(MAKE) $(build)=$@ \
>  	need-builtin=$(if $(filter $@/built-in.a, $(subdir-builtin)),1) \
>  	need-modorder=$(if $(filter $@/modules.order, $(subdir-modorder)),1) \
> -	$(filter $@/%, $(single-subdir-goals))
> +	$(filter-out $(addsuffix /%, $(filter $@/%, $(single-subdirs))), \
> +		$(filter $@/%, $(single-subdir-goals)))
>  
>  # Add FORCE to the prerequisites of a target to force it to be always rebuilt.
>  # ---------------------------------------------------------------------------

Ping! I know, it's the merge window but this patch may have fallen
through the cracks as there was no response so far. Just trying to take
care it doesn't get lost.

Thanks,
Mathias