Re: [PATCH 00/23] kbuild: significantly speed up kernel builds

[Daniel Campos Ramos <capitain_jack@yahoo.com>]

Hi Lorenzo, Linus,

A follow-up to my 20 October note, with controlled numbers.

The principle is the compiler's, applied to comments: prepare once what the machine reads, and keep what people read beside it.
The web already does this: minifiers remove 48 to 70% of JavaScript bytes in one study, and source maps lead back to the original.
Debian does it for binaries: stripped in the package, debug symbols in -dbgsym.
Lean Code does it for the text a system reads: headers, scripts, settings.

Every changed file has three forms:
1. The authored source, unchanged, in the source package.
2. The machine form, installed: comment-only lines removed; blank lines, directives and code untouched.
3. The human form, the complete authored text, in a <package>-lc-hr companion at the same version.
The human form costs nothing at build or run time, so documentation can grow there, better grounded, paid for once by the person who reads it.
Line numbers shift on purpose; a tool finds a machine-form line by its content in the authored source (dpkg-stereo --resolve PATH:LINE).
An earlier form inserted #line directives: it cost about 21% more preprocessing and silently disabled -Wmisleading-indentation. It is gone.

Measured, per file type (instructions with perf):
- Kernel headers, linux-headers 7.3.1, 50.7 to 46.9 MB (-7.5%): preprocessing -0.78%, compiling -0.52%.
- libstdc++, 864 headers (-23.6% bytes): preprocessing -2.7 to -3.0% instructions and -3.9 to -4.3% cycles; -O2 objects byte-identical; one compile of bits/stdc++.h reads 1.9 MB less.

Measured, the whole system: a 2016 AMD A10 laptop, 28,500 installed files lean, then restored to the originals, then lean again:
- DKMS module builds against the kernel headers: 6 to 9% faster warm, 1 to 6% cold. That is more than the instruction counts, which points at reading and caching; the two are not separated yet.
- Programs use 3 to 6% less memory; boot time is unchanged; installed package files take 140 MB less disk.
- The cost: 0.4 to 0.8 s more per dpkg run for the integrity check (about 64 ms in the current version), and a full verification takes 22 s instead of 11.
- Modules built lean are identical to each other; against the original headers, modules whose macros carry header line numbers differ in those numbers only.

Scaled, as estimates rather than measurements; I found no reliable count of Linux machines, so per million, to multiply by any fleet:
- A developer building 1 hour a day against kernel headers saves 15 to 22 hours a year; per million such machines, 15 to 22 million hours.
- Per million machines: 140 TB less disk, and 3.8 TB less page cache where the kernel headers are kept.
- A desktop user who never compiles comes out about even.

For scale: GCC 16 builds the kernel about 2.6% slower than GCC 15 in one report; Ingo Molnar's first header attempts gave barely 4% before Fast Kernel Headers reached 50 to 80% with over 2,000 commits; your series reports allmodconfig up to 36% faster.
This stacks with all of them, with no source change.

I am not proposing anything for the kernel tree.
If a defconfig or allmodconfig comparison on the kernel's own include/, with the objects checked, would be useful, I can run it.
Methods and raw data:
https://github.com/Sparky-OS/sparky-stereo-os/blob/main/docs/proofs/lean-code-a10-aba.md
https://github.com/Sparky-OS/sparky-stereo-os/blob/main/docs/proofs/lean-code-compile-cycles.md

Daniel Campos Ramos