Hi,

following an IRC discussion with many parties...

On 24. 09. 22, 20:19, Masahiro Yamada wrote:
> The objects placed at the head of vmlinux need special treatments:
> 
>   - arch/$(SRCARCH)/Makefile adds them to head-y in order to place
>     them before other archives in the linker command line.
> 
>   - arch/$(SRCARCH)/kernel/Makefile adds them to extra-y instead of
>     obj-y to avoid them going into built-in.a.
> 
> This commit gets rid of the latter.
> 
> Create vmlinux.a to collect all the objects that are unconditionally
> linked to vmlinux. The objects listed in head-y are moved to the head
> of vmlinux.a by using 'ar m'.
> 
> With this, arch/$(SRCARCH)/kernel/Makefile can consistently use obj-y
> for builtin objects.
> 
> There is no *.o that is directly linked to vmlinux. Drop unneeded code
> in scripts/clang-tools/gen_compile_commands.py.
> 
> $(AR) mPi needs 'T' to workaround the llvm-ar bug. The fix was suggested
> by Nathan Chancellor [1].
> 
> [1]: https://lore.kernel.org/llvm/YyjjT5gQ2hGMH0ni@dev-arch.thelio-3990X/
> 
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---
...
> --- a/scripts/Makefile.vmlinux_o
> +++ b/scripts/Makefile.vmlinux_o
> @@ -18,7 +18,7 @@ quiet_cmd_gen_initcalls_lds = GEN     $@
>   	$(PERL) $(real-prereqs) > $@
>   
>   .tmp_initcalls.lds: $(srctree)/scripts/generate_initcall_order.pl \
> -		$(KBUILD_VMLINUX_OBJS) $(KBUILD_VMLINUX_LIBS) FORCE
> +		vmlinux.a $(KBUILD_VMLINUX_LIBS) FORCE

There is a slight problem with this. The kernel built with gcc-LTO does 
not boot. But as I understand it, it's not limited to gcc-LTO only.

On x86, startup_64() is supposed to be at offset >zero< of the image 
(see .Lrelocated()). It was ensured by putting head64.o to the beginning 
of vmlinux (by KBUILD_VMLINUX_OBJS on the LD command-line above). The 
patch above instead packs head64.o into vmlinux.a and then moves it 
using "ar -m" to the beginning (it's in 7/7 of the series IIRC).

The problem is that .o files listed on the LD command line explicitly 
are taken as spelled. But unpacking .a inside LD gives no guarantees on 
the order of packed objects. To quote: "that it happens to work 
sometimes is pure luck." (Correct me guys, if I misunderstood you.)

For x86, the most ideal fix seems to be to fix it in the linker script. 
By putting startup_64() to a different section and handle it in the ld 
script specially -- see the attachment. It should always have been put 
this way, the command line order is only a workaround. But this might 
need more fixes on other archs too -- I haven't take a look.

Ideas, comments? I'll send the attachment as a PATCH later (if there are 
no better suggestions).

thanks,
-- 
js
suse labs