From: Fangrui Song <maskray@google.com>
To: Masahiro Yamada <masahiroy@kernel.org>
Cc: Sedat Dilek <sedat.dilek@gmail.com>, Jiri Slaby <jslaby@suse.cz>,
Linux Kbuild mailing list <linux-kbuild@vger.kernel.org>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
Michal Marek <michal.lkml@markovi.net>,
Nick Desaulniers <ndesaulniers@google.com>,
Nathan Chancellor <nathan@kernel.org>,
Sami Tolvanen <samitolvanen@google.com>,
clang-built-linux <llvm@lists.linux.dev>
Subject: Re: [PATCH] kbuild: pass jobserver to cmd_ld_vmlinux.o
Date: Fri, 17 Jun 2022 13:05:53 -0700 [thread overview]
Message-ID: <20220617200553.kg7jmkvwdp7yqfkm@google.com> (raw)
In-Reply-To: <CAK7LNATHY88PbJ_=A6g7v8NMQnBcQ9g06k1+SCe+NM+xd5dLwA@mail.gmail.com>
On 2022-06-18, Masahiro Yamada wrote:
>(+LLVM list, Fangrui Song)
Thanks for tagging me. I'll clarify some stuff.
>On Fri, Jun 17, 2022 at 7:41 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>>
>> On Fri, Jun 17, 2022 at 12:35 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>> >
>> > On Fri, Jun 17, 2022 at 12:53 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>> > >
>> > > On Thu, Jun 16, 2022 at 4:09 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>> > > >
>> > > > On Thu, Jun 16, 2022 at 12:45 PM Jiri Slaby <jslaby@suse.cz> wrote:
>> > > > >
>> > > > > Until the link-vmlinux.sh split (cf. the commit below), the linker was
>> > > > > run with jobserver set in MAKEFLAGS. After the split, the command in
>> > > > > Makefile.vmlinux_o is not prefixed by "+" anymore, so this information
>> > > > > is lost.
>> > > > >
>> > > > > Restore it as linkers working in parallel (esp. the LTO ones) make a use
>> > > > > of i
>
>Hi Jiri,
>
>Please let me clarify first.
>
>Here, is it OK to assume you are talking about Clang LTO
>instead of GCC LTO because the latter is not upstreamed ?
>
>
>
>
>
>I tested this patch but I did not see any performance change for Clang LTO.
>
>
>[1] CONFIG_CLANG_LTO_FULL
>
> lld always runs sequential.
> It never runs in parallel even if you pass -j option to Make
"lld always runs sequential" is not accurate. There are a number of
parallel linker passes. ld.lld --threads= defaults to
llvm::hardware_concurrency (similar to
https://en.cppreference.com/w/cpp/thread/thread/hardware_concurrency,
but uses sched_getaffinity to compute the number of available cores).
"lld always runs sequential" is only correct only when --threads=1 is
specified or the system only provides one thread to the lld process.
I think people may be more interested in LTO parallelism here. Regular
LTO (sometimes called full LTO when there is mixed-thin-and-regular LTO)
supports limited parallelism which applies to code generation, but not
IR-level optimization. (IR-level optimization has many interprocedural
optimizations passes. Splitting will make LTO less effective. Code
generation is per function, so parallelism does not regress
optimization.)
>
>[2] CONFIG_CLANG_LTO_THIN
>
> lld always runs in parallel even if you do not pass -j option
>
> In my machine, lld always allocated 12 threads.
> This is irrespective of the Make parallelisms.
>
>
>
>
>One more thing, if a program wants to participate in
>Make's jobserver, it must parse MAKEFLAGS, and extract
>file descriptors to be used to communicate to the jobserver.
>
>As a code example in the kernel tree,
>scripts/jobserver-exec parses "MAKEFLAGS" and "--jobserver".
>
>
>I grepped the lld source code, but it does not contain
>"MAKEFLAGS" or "jobserver".
>masahiro@oscar:~/ref/lld$ git remote show origin
>* remote origin
> Fetch URL: https://github.com/llvm-mirror/lld.git
> Push URL: https://github.com/llvm-mirror/lld.git
> HEAD branch: master
> Remote branches:
> master tracked
> release_36 tracked
> release_37 tracked
> release_38 tracked
> release_39 tracked
> release_40 tracked
> release_50 tracked
> release_60 tracked
> release_70 tracked
> release_80 tracked
> release_90 tracked
> Local branch configured for 'git pull':
> master merges with remote master
> Local ref configured for 'git push':
> master pushes to master (up to date)
>masahiro@oscar:~/ref/lld$ git grep MAKEFLAGS
>masahiro@oscar:~/ref/lld$ git grep jobserver
>
>
>So, in my research, LLD does not seem to support the jobserver.
Correct. lld does not support GNU make's jobserver. On the other hand,
I don't think the jobserver implementation supports flexible "give this
target N hardware concurrency". A heavy link target does not necessarily
get more resources than a quick target.
If a make target knows how many hardware concurrency it gets, we can
pass --threads= to lld. LTO easily takes 95+% link time, so LTO
parallelism may needs a dedicated setting. lld has --thinlto-jobs=.
>
>
>
>If you are talking about GCC LTO, yes, the code
>tries to parse "--jobserver-auth=" from the MAKEFLAGS
>environment variable. [1]
>
>[1]: https://github.com/gcc-mirror/gcc/blob/releases/gcc-12.1.0/gcc/lto-wrapper.cc#L1341
>
>
>But, as you may know, GCC LTO works in a different way,
>at least, we cannot do it before modpost.
>
>
>--
>Best Regards
>Masahiro Yamada
>
next prev parent reply other threads:[~2022-06-17 20:06 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-06-16 10:45 Jiri Slaby
2022-06-16 14:09 ` Sedat Dilek
2022-06-16 22:53 ` Sedat Dilek
2022-06-17 10:35 ` Sedat Dilek
2022-06-17 10:41 ` Sedat Dilek
2022-06-17 15:21 ` Masahiro Yamada
2022-06-17 20:05 ` Fangrui Song [this message]
2022-06-18 6:13 ` Sedat Dilek
2022-06-17 16:32 ` Nick Desaulniers
2022-06-17 17:50 ` Sedat Dilek
2022-06-17 19:34 ` Nathan Chancellor
2022-06-18 5:57 ` Sedat Dilek
2022-06-21 7:30 ` Jiri Slaby
2022-06-21 8:41 ` Masahiro Yamada
2022-09-20 8:43 ` Jiri Slaby
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220617200553.kg7jmkvwdp7yqfkm@google.com \
--to=maskray@google.com \
--cc=jslaby@suse.cz \
--cc=linux-kbuild@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=llvm@lists.linux.dev \
--cc=masahiroy@kernel.org \
--cc=michal.lkml@markovi.net \
--cc=nathan@kernel.org \
--cc=ndesaulniers@google.com \
--cc=samitolvanen@google.com \
--cc=sedat.dilek@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®