mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: "Lorenzo Stoakes (ARM)" <ljs@kernel.org>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: "Nathan Chancellor" <nathan@kernel.org>,
	"Nicolas Schier" <nsc@kernel.org>,
	"Nick Desaulniers" <ndesaulniers@google.com>,
	"Bill Wendling" <morbo@google.com>,
	"Justin Stitt" <justinstitt@google.com>,
	"Masahiro Yamada" <masahiroy@kernel.org>,
	"Alexey Gladkov" <legion@kernel.org>,
	"Thomas Gleixner" <tglx@kernel.org>,
	"Ingo Molnar" <mingo@redhat.com>,
	"Borislav Petkov" <bp@alien8.de>,
	"Dave Hansen" <dave.hansen@linux.intel.com>,
	x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>,
	"Paul Walmsley" <pjw@kernel.org>,
	"Palmer Dabbelt" <palmer@dabbelt.com>,
	"Albert Ou" <aou@eecs.berkeley.edu>,
	"Alexandre Ghiti" <alex@ghiti.fr>,
	"Arnd Bergmann" <arnd@arndb.de>,
	"Catalin Marinas" <catalin.marinas@arm.com>,
	"Will Deacon" <will@kernel.org>,
	"Mark Rutland" <mark.rutland@arm.com>,
	"Ard Biesheuvel" <ardb@kernel.org>,
	"Ilias Apalodimas" <ilias.apalodimas@linaro.org>,
	"Josh Poimboeuf" <jpoimboe@kernel.org>,
	"Peter Zijlstra" <peterz@infradead.org>,
	"Miguel Ojeda" <ojeda@kernel.org>,
	"Boqun Feng" <boqun@kernel.org>, "Gary Guo" <gary@garyguo.net>,
	"Björn Roy Baron" <bjorn3_gh@protonmail.com>,
	"Benno Lossin" <lossin@kernel.org>,
	"Andreas Hindborg" <a.hindborg@kernel.org>,
	"Alice Ryhl" <aliceryhl@google.com>,
	"Trevor Gross" <tmgross@umich.edu>,
	"Danilo Krummrich" <dakr@kernel.org>,
	"Daniel Almeida" <daniel.almeida@collabora.com>,
	"Tamir Duberstein" <tamird@kernel.org>,
	"Alexandre Courbot" <acourbot@nvidia.com>,
	"Onur Özkan" <work@onurozkan.dev>,
	"Jonathan Corbet" <corbet@lwn.net>,
	"Randy Dunlap" <rdunlap@infradead.org>,
	linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org,
	llvm@lists.linux.dev, linux-riscv@lists.infradead.org,
	linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org,
	linux-efi@vger.kernel.org, rust-for-linux@vger.kernel.org,
	linux-doc@vger.kernel.org, "Jens Axboe" <axboe@kernel.dk>
Subject: Re: [PATCH 15/23] modpost: emit module descriptors as assembly
Date: Mon, 14 Sep 2026 09:50:15 +0100	[thread overview]
Message-ID: <aqe0WGxQXUyFT_9Q@gremlin> (raw)
In-Reply-To: <CAHk-=wgAWpVHmR74jC7sbSJGQEHyaaN0awNfSz_SjPv3VGSYYA@mail.gmail.com>

On Wed, Sep 09, 2026 at 07:59:55AM -0700, Linus Torvalds wrote:
> On Tue, 8 Sept 2026 at 13:58, Lorenzo Stoakes (ARM) <ljs@kernel.org> wrote:
> >
> > modpost generates a descriptor for every module in the form of a
> > <module>.mod.c file with .modinfo strings, the __this_module descriptor,
> > exported symbol tables and (with CONFIG_MODVERSIONS set), the CRC of
> > imported symbols.
> >
> > These files are compiled like any other kernel C file with all of the
> > -include preamble, as well as including linux/module.h, header dependencies
> > generated by fixdep of a few hundred headers, an objtool run and if LTO is
> > being performed, a link is performed to generate native code.
>
> This kind of triggers the same reaction as the kallsyms code: I think
> not including <linux/module.h> is probably the biggest win in this
> all, and precomputing the offsets of the (few) structures that modpost
> uses then makes the whole end result fairly trivial in raw asm.

Right indeed :)

>
> Fine.
>
> BUT.
>
> You end up with things like this instead:
>
> > diff --git a/scripts/mod/module-offsets.c b/scripts/mod/module-offsets.c
>
> where it generates a header file for those asm scripts so that the asm
> generation can do things like
>
>         buf_printf(buf, "\t.size __this_module, %d\n",
> MOD_SIZEOF_struct_module);
>
> etc.
>
> But I - again - get the feeling that this is all going through AS as a
> silly way to generate an object file, and that there really isn't any
> obvious advantage to that side quest.

Well it seems the biggest advantage in doing assembly over directly
manipulating an object file is that you avoid any arch-specific nonsense,
as with kallsyms.

>
> Becuase no human ever wants to see that asm file, and it's no more
> legible than generating the object file directly would have been.
>
> So I get the feeling that it would again be more natural to just
> generate the object file directly when all those files really end up
> being are a few symbol definitions and a couple of fixed structures
> (is there anything but the magic "__this_module" thing and the modinfo
> unique ID thing?)
>
> And again - I feel like nobody sane would want to write that code. I
> certainly don't. But if this was at least partly automated anyway,
> going that extra step might not be all that horrific.
>
> I'll also be honest: part of why I do these suggestions is that I do
> think it would be faster to skip the intermediate assembler step, but
> a big part of it is that I don't love the random turds our build
> creates. Getting rid of the *.mod.c files would be a cleanup of our
> strange build.

Well the *.mod.c turds are flushed already, but replaced with smaller
*.mod.S ones instead :)

The LLM says the time taken per assembler step is ~7.5ms per .mod.S which
is .65s of wall time on the threadripper so it terms of a build it's into
the noise territory at least for a bigger box (plus obviously you won't go
to _zero_ cost by writing the object file directly).

Overall I think it's not _currently_ worth it as part of this work, but
it's something I can set the LLM on as a follow-up maybe? Could do a
'eliminate build artifacts' pass in general.


>
>              Linus

--
Cheers, Lorenzo

  reply	other threads:[~2026-09-14  8:50 UTC|newest]

Thread overview: 105+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-09-08 20:55 [PATCH 00/23] kbuild: significantly speed up kernel builds Lorenzo Stoakes (ARM)
2026-09-08 20:55 ` [PATCH 01/23] scripts/mksysmap: drop the MODULE_INFO() symbols from kallsyms Lorenzo Stoakes (ARM)
2026-09-09 19:47   ` Nicolas Schier
2026-09-10 11:00     ` Lorenzo Stoakes (ARM)
2026-09-10  4:19   ` Nathan Chancellor
2026-09-10 11:03     ` Lorenzo Stoakes (ARM)
2026-09-10 19:00   ` Nicolas Schier
2026-09-08 20:55 ` [PATCH 02/23] scripts/mksysmap: fix escape of '$' in the __pi_ pattern Lorenzo Stoakes (ARM)
2026-09-09 19:47   ` Nicolas Schier
2026-09-10 11:04     ` Lorenzo Stoakes (ARM)
2026-09-10  4:19   ` Nathan Chancellor
2026-09-10 11:21     ` Lorenzo Stoakes (ARM)
2026-09-10 19:00   ` Nicolas Schier
2026-09-08 20:55 ` [PATCH 03/23] kallsyms: index symbols by token to speed up table compression Lorenzo Stoakes (ARM)
2026-09-08 20:55 ` [PATCH 04/23] kallsyms: output binary data to speed output and kallsyms assembly Lorenzo Stoakes (ARM)
2026-09-09 14:35   ` Linus Torvalds
2026-09-14  8:39     ` Lorenzo Stoakes (ARM)
2026-09-14  8:57       ` Lorenzo Stoakes (ARM)
2026-09-10  9:29   ` David Laight
2026-09-11 11:07     ` Lorenzo Stoakes (ARM)
2026-09-12  6:38     ` [4/23] " Markus Elfring
2026-09-08 20:55 ` [PATCH 05/23] kbuild: do not sort nm output where the order is irrelevant Lorenzo Stoakes (ARM)
2026-09-10  4:19   ` Nathan Chancellor
2026-09-08 20:55 ` [PATCH 06/23] kbuild: only emit vmlinux relocations when required Lorenzo Stoakes (ARM)
2026-09-10  4:19   ` Nathan Chancellor
2026-09-13 19:33     ` Lorenzo Stoakes (ARM)
2026-09-08 20:55 ` [PATCH 07/23] elf-parse: add section flags, symbol binding and a read-only mapping Lorenzo Stoakes (ARM)
2026-09-08 20:55 ` [PATCH 08/23] kallsyms: reimplement mksysmap in C Lorenzo Stoakes (ARM)
2026-09-11 18:52   ` Markus Elfring
2026-09-11 19:15   ` Markus Elfring
2026-09-11 19:42   ` Markus Elfring
2026-09-08 20:55 ` [PATCH 09/23] kbuild: do not allocate .modinfo in vmlinux Lorenzo Stoakes (ARM)
2026-09-10  4:19   ` Nathan Chancellor
2026-09-10 10:59     ` Lorenzo Stoakes (ARM)
2026-09-08 20:55 ` [PATCH 10/23] kbuild: cache list, composite object state per object Lorenzo Stoakes (ARM)
2026-09-08 20:55 ` [PATCH 11/23] kbuild: implement and use depcheck to check dependency timestamps Lorenzo Stoakes (ARM)
2026-09-09 15:26   ` Linus Torvalds
2026-09-14  8:28     ` Lorenzo Stoakes (ARM)
2026-09-08 20:55 ` [PATCH 12/23] kbuild: avoid re-running compiler and linker probes Lorenzo Stoakes (ARM)
2026-09-10  4:19   ` Nathan Chancellor
2026-09-10 15:42     ` Nicolas Schier
2026-09-11 10:30       ` Lorenzo Stoakes (ARM)
2026-09-11 18:10         ` Nicolas Schier
2026-09-11 18:25           ` Lorenzo Stoakes (ARM)
2026-09-11 17:33       ` David Laight
2026-09-11 18:17         ` Nicolas Schier
2026-09-11 18:24           ` Lorenzo Stoakes (ARM)
2026-09-11 19:34             ` Nicolas Schier
2026-09-11 21:01             ` David Laight
2026-09-12  7:11             ` Nathan Chancellor
2026-09-13 20:02               ` Lorenzo Stoakes (ARM)
2026-09-11 10:26     ` Lorenzo Stoakes (ARM)
2026-09-12  6:51       ` Nathan Chancellor
2026-09-12 10:03     ` David Laight
2026-09-08 20:55 ` [PATCH 13/23] modpost: hash module source per-file, not per-byte Lorenzo Stoakes (ARM)
2026-09-10 12:52   ` Petr Pavlu
2026-09-11 10:41     ` Lorenzo Stoakes (ARM)
2026-09-11 11:57       ` Petr Pavlu
2026-09-11 12:21         ` Lorenzo Stoakes (ARM)
2026-09-08 20:55 ` [PATCH 14/23] modpost: cache section relocation mismatch state Lorenzo Stoakes (ARM)
2026-09-08 20:55 ` [PATCH 15/23] modpost: emit module descriptors as assembly Lorenzo Stoakes (ARM)
2026-09-09 14:59   ` Linus Torvalds
2026-09-14  8:50     ` Lorenzo Stoakes (ARM) [this message]
2026-09-08 20:55 ` [PATCH 16/23] kbuild: batch module finalisation Lorenzo Stoakes (ARM)
2026-09-10 15:48   ` Nicolas Schier
2026-09-11 10:23     ` Lorenzo Stoakes (ARM)
2026-09-08 20:55 ` [PATCH 17/23] modpost: perform srcversion hashing in parallel Lorenzo Stoakes (ARM)
2026-09-10 10:32   ` David Laight
2026-09-11 10:49     ` Lorenzo Stoakes (ARM)
2026-09-12 11:50   ` Yann Droneaud
2026-09-13 16:36     ` Lorenzo Stoakes (ARM)
2026-09-08 20:55 ` [PATCH 18/23] objtool: cache relocations and function dead end state, do less work Lorenzo Stoakes (ARM)
2026-09-12 22:47   ` Josh Poimboeuf
2026-09-13 20:28     ` Lorenzo Stoakes (ARM)
2026-09-08 20:55 ` [PATCH 19/23] objtool: decode instructions and resolve branch targets in parallel Lorenzo Stoakes (ARM)
2026-09-12 23:39   ` Josh Poimboeuf
2026-09-13 16:23     ` Lorenzo Stoakes (ARM)
2026-09-08 20:55 ` [PATCH 20/23] kbuild: rust: parallelise rustc front end Lorenzo Stoakes (ARM)
2026-09-08 21:13   ` Miguel Ojeda
2026-09-09 14:22     ` Lorenzo Stoakes (ARM)
2026-09-09 10:22   ` Björn Baron
2026-09-09 12:59     ` Miguel Ojeda
2026-09-09 14:26       ` Lorenzo Stoakes (ARM)
2026-09-10 12:25     ` Nicolas Schier (FRITZ!)
2026-09-11 19:00   ` Nicolas Schier
2026-09-13 21:30     ` Lorenzo Stoakes (ARM)
2026-09-08 20:55 ` [PATCH 21/23] rust: make exports.o depend on the headers generated for it Lorenzo Stoakes (ARM)
2026-09-08 20:55 ` [PATCH 22/23] kbuild: build rust crates in parallel with the rest of the build Lorenzo Stoakes (ARM)
2026-09-08 20:55 ` [PATCH 23/23] kbuild: use pigz for gzip compression if available Lorenzo Stoakes (ARM)
2026-09-10  4:19   ` Nathan Chancellor
2026-09-11 11:03     ` Lorenzo Stoakes (ARM)
2026-09-12  6:38       ` Nathan Chancellor
2026-09-13 19:31         ` Lorenzo Stoakes (ARM)
2026-09-08 21:06 ` [PATCH 00/23] kbuild: significantly speed up kernel builds Nick Desaulniers
2026-09-09 14:17   ` Lorenzo Stoakes (ARM)
2026-09-09 22:09     ` Nick Desaulniers
2026-09-11 11:25       ` Lorenzo Stoakes (ARM)
2026-09-09 15:37 ` Linus Torvalds
2026-09-09 16:30   ` Lorenzo Stoakes (ARM)
2026-09-09 21:58 ` Florian Fainelli
2026-09-11 11:28   ` Lorenzo Stoakes (ARM)
2026-09-12  6:44     ` Nathan Chancellor
2026-09-13 16:27       ` Lorenzo Stoakes (ARM)
2026-09-10  4:19 ` Nathan Chancellor
2026-09-11 11:13   ` Lorenzo Stoakes (ARM)

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=aqe0WGxQXUyFT_9Q@gremlin \
    --to=ljs@kernel.org \
    --cc=a.hindborg@kernel.org \
    --cc=acourbot@nvidia.com \
    --cc=alex@ghiti.fr \
    --cc=aliceryhl@google.com \
    --cc=aou@eecs.berkeley.edu \
    --cc=ardb@kernel.org \
    --cc=arnd@arndb.de \
    --cc=axboe@kernel.dk \
    --cc=bjorn3_gh@protonmail.com \
    --cc=boqun@kernel.org \
    --cc=bp@alien8.de \
    --cc=catalin.marinas@arm.com \
    --cc=corbet@lwn.net \
    --cc=dakr@kernel.org \
    --cc=daniel.almeida@collabora.com \
    --cc=dave.hansen@linux.intel.com \
    --cc=gary@garyguo.net \
    --cc=hpa@zytor.com \
    --cc=ilias.apalodimas@linaro.org \
    --cc=jpoimboe@kernel.org \
    --cc=justinstitt@google.com \
    --cc=legion@kernel.org \
    --cc=linux-arch@vger.kernel.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-efi@vger.kernel.org \
    --cc=linux-kbuild@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-riscv@lists.infradead.org \
    --cc=llvm@lists.linux.dev \
    --cc=lossin@kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=masahiroy@kernel.org \
    --cc=mingo@redhat.com \
    --cc=morbo@google.com \
    --cc=nathan@kernel.org \
    --cc=ndesaulniers@google.com \
    --cc=nsc@kernel.org \
    --cc=ojeda@kernel.org \
    --cc=palmer@dabbelt.com \
    --cc=peterz@infradead.org \
    --cc=pjw@kernel.org \
    --cc=rdunlap@infradead.org \
    --cc=rust-for-linux@vger.kernel.org \
    --cc=tamird@kernel.org \
    --cc=tglx@kernel.org \
    --cc=tmgross@umich.edu \
    --cc=torvalds@linux-foundation.org \
    --cc=will@kernel.org \
    --cc=work@onurozkan.dev \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®