Re: [PATCH v2 17/21] objtool: decode instructions and resolve branch targets in parallel

["Lorenzo Stoakes (ARM)" <ljs@kernel.org>]

On Mon, Sep 14, 2026 at 11:20:37AM -0700, Kees Cook wrote:
> On Mon, Sep 14, 2026 at 10:22:16AM +0100, Lorenzo Stoakes (ARM) wrote:
> > Threads are only created for objects with 8 MiB or more of text, meaning
> > that runs involving smaller objects remain unaffected.
> > [...]
> > objtool on vmlinux.o is on the serial tail of every build that links
> > vmlinux, no-op builds are unchanged.
>
> This isn't limited to vmlinux.o, though? With CONFIG_X86_KERNEL_IBT
> (default y on x86_64), delay-objtool is set in scripts/Makefile.lib, and
> objtool then runs on every multi-object module, in the middle of the
> parallel part of the build. In an x86_64 allmodconfig build here,
> amdgpu.o has 46.2 MiB of executable text and i915.o has 10.2 MiB, so
> each of those objtool runs gets 16 threads on top of the -jN jobs that
> are already running.

Ack, will fix in v3 to limit to only the --link run without --module, which
is how vmlinux.o is processed, leaves modules as they are.

>
> Any parallelism added need to be handled by the make jobserver, not
> hard-coded anyway.

I'm not sure that really gains us anything - at the point of doing the
vmlinux.o processing (which with the above change is all that is being done
now) everything's idle so the jobserver would just hand over the same 16
threads.

It'd need quite a bit of work to interact with it too with added complexity
+ some overhead so would have to check impact on perf here.

You did complain about added complexity also ;) if you insist I can try.

>
> > [...]
> > +/* Only an object this large, e.g. vmlinux.o, is decoded on several threads. */
> > +#define DECODE_THREADED_MIN_TEXT	SZ_8M
> > +/* Only decoding and the branch passes are threaded, so more gains nothing. */
> > +#define DECODE_MAX_THREADS		16
> > +#define DECODE_RANGES_PER_THREAD	4
>
> I just don't think a size-based approach is going to work. And any max
> parallelism needs a rationale.

A rationale was provided:

	Threading is limited to decoding and the jump pass, so the gain
	flattens out at 16 threads and any further threads were found to
	only add overhead.

	When performing an allmodconfig build, the clang invocation of
	objtool when processing vmlinux.o took 5.93s on 1 thread, 4.56s on
	8, 4.51s on 16 and 4.63s on 128.

Let me know what you feel was missing there? These things inevitably have
to be somewhat heuristic.

The size threshold is about paying for too many threads for no real gain,
and now (for v3) the code is limited to a --link operation only, that'll
only really stop a tinyconfig vmlinux.o :)

>
> > +static unsigned int decode_threads(unsigned long text_size)
> > +{
> > +	const long nr_cpus = sysconf(_SC_NPROCESSORS_ONLN);
> > +
> > +	if (text_size < DECODE_THREADED_MIN_TEXT || nr_cpus < 2)
> > +		return 1;
> > +
> > +	return min_t(unsigned int, nr_cpus, DECODE_MAX_THREADS);
> > +}
>
> No, anything using _SC_NPROCESSORS_ONLN internally is wrong, whether its
> pigz or objtool.

Why? We shouldn't cap a 4 CPU box to 4 CPUs?

The idea is that the empirically obtained 16 core limit should naturally be
capped to the number of CPUs.

I guess in a world where make hands jobs off this could be dropped, though,
if the complexity there seems worthwhile.

>
> > [...]
> > +static int add_jump_destinations(struct objtool_file *file)
> > +{
> > +	return run_insn_ranges(file, add_jump_destinations_range);
> > +}
>
> I haven't examined this myself yet, but my LLM doesn't like this,
> saying:
>
>   For an internal sibling call, add_jump_destination() calls
>   add_call_dest(file, insn, dest_sym, true), and add_call_dest() calls
>   annotate_call_site(), which for --hack-noinstr profiling calls, and for
>   __fentry__ calls with --mcount --mnop, does set_reloc_type() and
>   elf_write_insn() on file->elf. The per-range shadow gives each thread
>   its own lists, but the struct elf is shared, and both helpers write to
>   the section and mark it changed without any locking.
>   add_call_destinations() is kept serial because "annotating a call site
>   rewrites instructions the dead end walks read", so it looks like this
>   path needs the same care: a lock, or deferring those writes until the
>   threads have been joined. This is from reading the code; I haven't hit
>   a failure. Was the thread sanitizer run mentioned in the cover letter
>   done with those objtool options enabled?
>
> So, take it with a grain of salt. ;)

I mean sashiko is hallucinating at a rate >50% on this series, so pinches
of salt all round :) I do think frontier models with a good amount of
checking + e.g. Chris Mason's review-prompts are helpful to rein in stuff.

Anyway I had mine check it ;) and I also have a beefy threadripper sitting
around for bounding reality on this stuff and it all came back clean:

	Those writes go to the thread's own instruction bytes and its own
	relocation, the only shared state touched is the section and elf
	"changed" flags, which every writer sets to true, and the mcount
	case for a sibling call is a warning path. The v2 sanitizer run
	used the gcc allmodconfig options, which lack
	--mcount/--mnop/--cfi/--fineibt, so I reran it on the clang
	allmodconfig vmlinux.o with its full option set: zero reports, the
	jump pass included.

>
> -Kees
>
> --
> Kees Cook

--
Cheers, Lorenzo