* [PATCH v2] kbuild: fix UNUSED_KSYMS_WHITELIST for Clang LTO
@ 2021-02-26 6:25 Masahiro Yamada
2021-02-26 9:25 ` Sedat Dilek
0 siblings, 1 reply; 5+ messages in thread
From: Masahiro Yamada @ 2021-02-26 6:25 UTC (permalink / raw)
To: linux-kbuild
Cc: Sami Tolvanen, linux-kernel, Arnd Bergmann, Masahiro Yamada,
Kees Cook, Nathan Chancellor, Nick Desaulniers,
clang-built-linux
Commit fbe078d397b4 ("kbuild: lto: add a default list of used symbols")
does not work as expected if the .config file has already specified
CONFIG_UNUSED_KSYMS_WHITELIST="my/own/white/list" before enabling
CONFIG_LTO_CLANG.
So, the user-supplied whitelist and LTO-specific white list must be
independent of each other.
I refactored the shell script so CONFIG_MODVERSIONS and CONFIG_CLANG_LTO
handle whitelists in the same way.
Fixes: fbe078d397b4 ("kbuild: lto: add a default list of used symbols")
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---
Changes in v2:
- Rebase on top of Arnd's patch:
https://lore.kernel.org/lkml/20210225143456.3829513-1-arnd@kernel.org/
init/Kconfig | 1 -
scripts/gen_autoksyms.sh | 35 ++++++++++++++++++++++++---------
scripts/lto-used-symbollist.txt | 6 ------
3 files changed, 26 insertions(+), 16 deletions(-)
delete mode 100644 scripts/lto-used-symbollist.txt
diff --git a/init/Kconfig b/init/Kconfig
index 719871f8727c..64c32300d1b4 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -2283,7 +2283,6 @@ config TRIM_UNUSED_KSYMS
config UNUSED_KSYMS_WHITELIST
string "Whitelist of symbols to keep in ksymtab"
depends on TRIM_UNUSED_KSYMS
- default "scripts/lto-used-symbollist.txt" if LTO_CLANG
help
By default, all unused exported symbols will be un-exported from the
build when TRIM_UNUSED_KSYMS is selected.
diff --git a/scripts/gen_autoksyms.sh b/scripts/gen_autoksyms.sh
index d54dfba15bf2..da320151e7c3 100755
--- a/scripts/gen_autoksyms.sh
+++ b/scripts/gen_autoksyms.sh
@@ -19,7 +19,26 @@ esac
# We need access to CONFIG_ symbols
. include/config/auto.conf
-ksym_wl=/dev/null
+needed_symbols=
+
+# Special case for modversions (see modpost.c)
+if [ -n "$CONFIG_MODVERSIONS" ]; then
+ needed_symbols="$needed_symbols module_layout"
+fi
+
+# With CONFIG_LTO_CLANG, LLVM bitcode has not yet been compiled into a binary
+# when the .mod files are generated, which means they don't yet contain
+# references to certain symbols that will be present in the final binaries.
+if [ -n "$CONFIG_LTO_CLANG" ]; then
+ # intrinsic functions
+ needed_symbols="$needed_symbols memcpy memmove memset"
+ # ftrace
+ needed_symbols="$needed_symbols _mcount"
+ # stack protector symbols
+ needed_symbols="$needed_symbols __stack_chk_fail __stack_chk_guard"
+fi
+
+ksym_wl=
if [ -n "$CONFIG_UNUSED_KSYMS_WHITELIST" ]; then
# Use 'eval' to expand the whitelist path and check if it is relative
eval ksym_wl="$CONFIG_UNUSED_KSYMS_WHITELIST"
@@ -40,16 +59,14 @@ cat > "$output_file" << EOT
EOT
[ -f modules.order ] && modlist=modules.order || modlist=/dev/null
-sed 's/ko$/mod/' $modlist |
-xargs -n1 sed -n -e '2{s/ /\n/g;/^$/!p;}' -- |
-cat - "$ksym_wl" |
+
+{
+ sed 's/ko$/mod/' $modlist | xargs -n1 sed -n -e '2p'
+ echo "$needed_symbols"
+ [ -n "$ksym_wl" ] && cat "$ksym_wl"
+} | sed -e 's/ /\n/g' | sed -n -e '/^$/!p' |
# Remove the dot prefix for ppc64; symbol names with a dot (.) hold entry
# point addresses.
sed -e 's/^\.//' |
sort -u |
sed -e 's/\(.*\)/#define __KSYM_\1 1/' >> "$output_file"
-
-# Special case for modversions (see modpost.c)
-if [ -n "$CONFIG_MODVERSIONS" ]; then
- echo "#define __KSYM_module_layout 1" >> "$output_file"
-fi
diff --git a/scripts/lto-used-symbollist.txt b/scripts/lto-used-symbollist.txt
deleted file mode 100644
index 406ada65e926..000000000000
--- a/scripts/lto-used-symbollist.txt
+++ /dev/null
@@ -1,6 +0,0 @@
-memcpy
-memmove
-memset
-_mcount
-__stack_chk_fail
-__stack_chk_guard
--
2.27.0
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [PATCH v2] kbuild: fix UNUSED_KSYMS_WHITELIST for Clang LTO
2021-02-26 6:25 [PATCH v2] kbuild: fix UNUSED_KSYMS_WHITELIST for Clang LTO Masahiro Yamada
@ 2021-02-26 9:25 ` Sedat Dilek
2021-02-27 6:54 ` Masahiro Yamada
0 siblings, 1 reply; 5+ messages in thread
From: Sedat Dilek @ 2021-02-26 9:25 UTC (permalink / raw)
To: Masahiro Yamada
Cc: linux-kbuild, Sami Tolvanen, linux-kernel, Arnd Bergmann,
Kees Cook, Nathan Chancellor, Nick Desaulniers,
Clang-Built-Linux ML
On Fri, Feb 26, 2021 at 7:26 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> Commit fbe078d397b4 ("kbuild: lto: add a default list of used symbols")
> does not work as expected if the .config file has already specified
> CONFIG_UNUSED_KSYMS_WHITELIST="my/own/white/list" before enabling
> CONFIG_LTO_CLANG.
>
> So, the user-supplied whitelist and LTO-specific white list must be
> independent of each other.
>
> I refactored the shell script so CONFIG_MODVERSIONS and CONFIG_CLANG_LTO
> handle whitelists in the same way.
>
> Fixes: fbe078d397b4 ("kbuild: lto: add a default list of used symbols")
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---
>
> Changes in v2:
> - Rebase on top of Arnd's patch:
> https://lore.kernel.org/lkml/20210225143456.3829513-1-arnd@kernel.org/
>
> init/Kconfig | 1 -
> scripts/gen_autoksyms.sh | 35 ++++++++++++++++++++++++---------
> scripts/lto-used-symbollist.txt | 6 ------
People who want to use their own "white-listed" (allow-listed)
symbollist-file can do that via
CONFIG_UNUSED_KSYMS_WHITELIST="my/own/white/list".
Correct?
Why do you delete the default "scripts/lto-used-symbollist.txt" file
and remove the default in the appropriate Kconfig for people who want
to enable Clang-(Thin)LTO?
These people should now use
CONFIG_UNUSED_KSYMS_WHITELIST="scripts/lto-used-symbollist.txt"?
But again - the file was deleted with your patch.
Do I miss something?
- Sedat -
> 3 files changed, 26 insertions(+), 16 deletions(-)
> delete mode 100644 scripts/lto-used-symbollist.txt
>
> diff --git a/init/Kconfig b/init/Kconfig
> index 719871f8727c..64c32300d1b4 100644
> --- a/init/Kconfig
> +++ b/init/Kconfig
> @@ -2283,7 +2283,6 @@ config TRIM_UNUSED_KSYMS
> config UNUSED_KSYMS_WHITELIST
> string "Whitelist of symbols to keep in ksymtab"
> depends on TRIM_UNUSED_KSYMS
> - default "scripts/lto-used-symbollist.txt" if LTO_CLANG
> help
> By default, all unused exported symbols will be un-exported from the
> build when TRIM_UNUSED_KSYMS is selected.
> diff --git a/scripts/gen_autoksyms.sh b/scripts/gen_autoksyms.sh
> index d54dfba15bf2..da320151e7c3 100755
> --- a/scripts/gen_autoksyms.sh
> +++ b/scripts/gen_autoksyms.sh
> @@ -19,7 +19,26 @@ esac
> # We need access to CONFIG_ symbols
> . include/config/auto.conf
>
> -ksym_wl=/dev/null
> +needed_symbols=
> +
> +# Special case for modversions (see modpost.c)
> +if [ -n "$CONFIG_MODVERSIONS" ]; then
> + needed_symbols="$needed_symbols module_layout"
> +fi
> +
> +# With CONFIG_LTO_CLANG, LLVM bitcode has not yet been compiled into a binary
> +# when the .mod files are generated, which means they don't yet contain
> +# references to certain symbols that will be present in the final binaries.
> +if [ -n "$CONFIG_LTO_CLANG" ]; then
> + # intrinsic functions
> + needed_symbols="$needed_symbols memcpy memmove memset"
> + # ftrace
> + needed_symbols="$needed_symbols _mcount"
> + # stack protector symbols
> + needed_symbols="$needed_symbols __stack_chk_fail __stack_chk_guard"
> +fi
> +
> +ksym_wl=
> if [ -n "$CONFIG_UNUSED_KSYMS_WHITELIST" ]; then
> # Use 'eval' to expand the whitelist path and check if it is relative
> eval ksym_wl="$CONFIG_UNUSED_KSYMS_WHITELIST"
> @@ -40,16 +59,14 @@ cat > "$output_file" << EOT
> EOT
>
> [ -f modules.order ] && modlist=modules.order || modlist=/dev/null
> -sed 's/ko$/mod/' $modlist |
> -xargs -n1 sed -n -e '2{s/ /\n/g;/^$/!p;}' -- |
> -cat - "$ksym_wl" |
> +
> +{
> + sed 's/ko$/mod/' $modlist | xargs -n1 sed -n -e '2p'
> + echo "$needed_symbols"
> + [ -n "$ksym_wl" ] && cat "$ksym_wl"
> +} | sed -e 's/ /\n/g' | sed -n -e '/^$/!p' |
> # Remove the dot prefix for ppc64; symbol names with a dot (.) hold entry
> # point addresses.
> sed -e 's/^\.//' |
> sort -u |
> sed -e 's/\(.*\)/#define __KSYM_\1 1/' >> "$output_file"
> -
> -# Special case for modversions (see modpost.c)
> -if [ -n "$CONFIG_MODVERSIONS" ]; then
> - echo "#define __KSYM_module_layout 1" >> "$output_file"
> -fi
> diff --git a/scripts/lto-used-symbollist.txt b/scripts/lto-used-symbollist.txt
> deleted file mode 100644
> index 406ada65e926..000000000000
> --- a/scripts/lto-used-symbollist.txt
> +++ /dev/null
> @@ -1,6 +0,0 @@
> -memcpy
> -memmove
> -memset
> -_mcount
> -__stack_chk_fail
> -__stack_chk_guard
> --
> 2.27.0
>
> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/20210226062548.3334081-1-masahiroy%40kernel.org.
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [PATCH v2] kbuild: fix UNUSED_KSYMS_WHITELIST for Clang LTO
2021-02-26 9:25 ` Sedat Dilek
@ 2021-02-27 6:54 ` Masahiro Yamada
2021-02-27 7:08 ` Sedat Dilek
0 siblings, 1 reply; 5+ messages in thread
From: Masahiro Yamada @ 2021-02-27 6:54 UTC (permalink / raw)
To: Sedat Dilek
Cc: Linux Kbuild mailing list, Sami Tolvanen,
Linux Kernel Mailing List, Arnd Bergmann, Kees Cook,
Nathan Chancellor, Nick Desaulniers, Clang-Built-Linux ML
On Fri, Feb 26, 2021 at 6:26 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>
> On Fri, Feb 26, 2021 at 7:26 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > Commit fbe078d397b4 ("kbuild: lto: add a default list of used symbols")
> > does not work as expected if the .config file has already specified
> > CONFIG_UNUSED_KSYMS_WHITELIST="my/own/white/list" before enabling
> > CONFIG_LTO_CLANG.
> >
> > So, the user-supplied whitelist and LTO-specific white list must be
> > independent of each other.
> >
> > I refactored the shell script so CONFIG_MODVERSIONS and CONFIG_CLANG_LTO
> > handle whitelists in the same way.
> >
> > Fixes: fbe078d397b4 ("kbuild: lto: add a default list of used symbols")
> > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> > ---
> >
> > Changes in v2:
> > - Rebase on top of Arnd's patch:
> > https://lore.kernel.org/lkml/20210225143456.3829513-1-arnd@kernel.org/
> >
> > init/Kconfig | 1 -
> > scripts/gen_autoksyms.sh | 35 ++++++++++++++++++++++++---------
> > scripts/lto-used-symbollist.txt | 6 ------
>
> People who want to use their own "white-listed" (allow-listed)
> symbollist-file can do that via
> CONFIG_UNUSED_KSYMS_WHITELIST="my/own/white/list".
> Correct?
>
> Why do you delete the default "scripts/lto-used-symbollist.txt" file
> and remove the default in the appropriate Kconfig for people who want
> to enable Clang-(Thin)LTO?
> These people should now use
> CONFIG_UNUSED_KSYMS_WHITELIST="scripts/lto-used-symbollist.txt"?
> But again - the file was deleted with your patch.
> Do I miss something?
I think so.
I moved those symbols to scripts/gen_autoksyms.sh
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [PATCH v2] kbuild: fix UNUSED_KSYMS_WHITELIST for Clang LTO
2021-02-27 6:54 ` Masahiro Yamada
@ 2021-02-27 7:08 ` Sedat Dilek
2021-02-27 20:05 ` Sedat Dilek
0 siblings, 1 reply; 5+ messages in thread
From: Sedat Dilek @ 2021-02-27 7:08 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Linux Kbuild mailing list, Sami Tolvanen,
Linux Kernel Mailing List, Arnd Bergmann, Kees Cook,
Nathan Chancellor, Nick Desaulniers, Clang-Built-Linux ML
On Sat, Feb 27, 2021 at 7:55 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> On Fri, Feb 26, 2021 at 6:26 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> >
> > On Fri, Feb 26, 2021 at 7:26 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > >
> > > Commit fbe078d397b4 ("kbuild: lto: add a default list of used symbols")
> > > does not work as expected if the .config file has already specified
> > > CONFIG_UNUSED_KSYMS_WHITELIST="my/own/white/list" before enabling
> > > CONFIG_LTO_CLANG.
> > >
> > > So, the user-supplied whitelist and LTO-specific white list must be
> > > independent of each other.
> > >
> > > I refactored the shell script so CONFIG_MODVERSIONS and CONFIG_CLANG_LTO
> > > handle whitelists in the same way.
> > >
> > > Fixes: fbe078d397b4 ("kbuild: lto: add a default list of used symbols")
> > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> > > ---
> > >
> > > Changes in v2:
> > > - Rebase on top of Arnd's patch:
> > > https://lore.kernel.org/lkml/20210225143456.3829513-1-arnd@kernel.org/
> > >
> > > init/Kconfig | 1 -
> > > scripts/gen_autoksyms.sh | 35 ++++++++++++++++++++++++---------
> > > scripts/lto-used-symbollist.txt | 6 ------
> >
> > People who want to use their own "white-listed" (allow-listed)
> > symbollist-file can do that via
> > CONFIG_UNUSED_KSYMS_WHITELIST="my/own/white/list".
> > Correct?
> >
> > Why do you delete the default "scripts/lto-used-symbollist.txt" file
> > and remove the default in the appropriate Kconfig for people who want
> > to enable Clang-(Thin)LTO?
> > These people should now use
> > CONFIG_UNUSED_KSYMS_WHITELIST="scripts/lto-used-symbollist.txt"?
> > But again - the file was deleted with your patch.
> > Do I miss something?
>
> I think so.
>
> I moved those symbols to scripts/gen_autoksyms.sh
>
OK, I have overseen hat.
- Sedat -
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [PATCH v2] kbuild: fix UNUSED_KSYMS_WHITELIST for Clang LTO
2021-02-27 7:08 ` Sedat Dilek
@ 2021-02-27 20:05 ` Sedat Dilek
0 siblings, 0 replies; 5+ messages in thread
From: Sedat Dilek @ 2021-02-27 20:05 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Linux Kbuild mailing list, Sami Tolvanen,
Linux Kernel Mailing List, Arnd Bergmann, Kees Cook,
Nathan Chancellor, Nick Desaulniers, Clang-Built-Linux ML
On Sat, Feb 27, 2021 at 8:08 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>
> On Sat, Feb 27, 2021 at 7:55 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > On Fri, Feb 26, 2021 at 6:26 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> > >
> > > On Fri, Feb 26, 2021 at 7:26 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > > >
> > > > Commit fbe078d397b4 ("kbuild: lto: add a default list of used symbols")
> > > > does not work as expected if the .config file has already specified
> > > > CONFIG_UNUSED_KSYMS_WHITELIST="my/own/white/list" before enabling
> > > > CONFIG_LTO_CLANG.
> > > >
> > > > So, the user-supplied whitelist and LTO-specific white list must be
> > > > independent of each other.
> > > >
> > > > I refactored the shell script so CONFIG_MODVERSIONS and CONFIG_CLANG_LTO
> > > > handle whitelists in the same way.
> > > >
> > > > Fixes: fbe078d397b4 ("kbuild: lto: add a default list of used symbols")
> > > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
- Sedat -
> > > > ---
> > > >
> > > > Changes in v2:
> > > > - Rebase on top of Arnd's patch:
> > > > https://lore.kernel.org/lkml/20210225143456.3829513-1-arnd@kernel.org/
> > > >
> > > > init/Kconfig | 1 -
> > > > scripts/gen_autoksyms.sh | 35 ++++++++++++++++++++++++---------
> > > > scripts/lto-used-symbollist.txt | 6 ------
> > >
> > > People who want to use their own "white-listed" (allow-listed)
> > > symbollist-file can do that via
> > > CONFIG_UNUSED_KSYMS_WHITELIST="my/own/white/list".
> > > Correct?
> > >
> > > Why do you delete the default "scripts/lto-used-symbollist.txt" file
> > > and remove the default in the appropriate Kconfig for people who want
> > > to enable Clang-(Thin)LTO?
> > > These people should now use
> > > CONFIG_UNUSED_KSYMS_WHITELIST="scripts/lto-used-symbollist.txt"?
> > > But again - the file was deleted with your patch.
> > > Do I miss something?
> >
> > I think so.
> >
> > I moved those symbols to scripts/gen_autoksyms.sh
> >
>
> OK, I have overseen hat.
>
> - Sedat -
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2021-02-27 20:06 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-02-26 6:25 [PATCH v2] kbuild: fix UNUSED_KSYMS_WHITELIST for Clang LTO Masahiro Yamada
2021-02-26 9:25 ` Sedat Dilek
2021-02-27 6:54 ` Masahiro Yamada
2021-02-27 7:08 ` Sedat Dilek
2021-02-27 20:05 ` Sedat Dilek
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®