[PATCH] scripts/tags.sh: improve compiled sources generation

[PATCH] scripts/tags.sh: improve compiled sources generation

[Jialu Xu]

Use grep instead of sed for all compiled sources generation, it is three
times more efficient.

Signed-off-by: Jialu Xu <xujialu@vimux.org>
---
 scripts/tags.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/tags.sh b/scripts/tags.sh
index ea31640b2671..938dba219534 100755
--- a/scripts/tags.sh
+++ b/scripts/tags.sh
@@ -98,7 +98,7 @@ all_compiled_sources()
 	{
 		echo include/generated/autoconf.h
 		find $ignore -name "*.cmd" -exec \
-			sed -n -E 's/^source_.* (.*)/\1/p; s/^  (\S.*) \\/\1/p' {} \+ |
+			grep -Poh '(?<=^  )\S+|(?<== )\S+[^\\](?=$)' {} \+ |
 		awk '!a[$0]++'
 	} | xargs realpath -esq $([ -z "$KBUILD_ABS_SRCTREE" ] && echo --relative-to=.) |
 	sort -u
-- 
2.30.2

Re: [PATCH] scripts/tags.sh: improve compiled sources generation

[Carlos Llamas]

On Thu, Jun 01, 2023 at 09:04:02AM +0800, Jialu Xu wrote:
> Use grep instead of sed for all compiled sources generation, it is three
> times more efficient.

Although I'm not sure how you obtained a 3x efficiency it did make
things faster for me. I suppose there are several factors involved.
I used defconfig for arm64 and got the following results:
	real    1m50.790s
	real    1m27.907s (with patch)

Tested-by: Carlos Llamas <cmllamas@google.com>

> 
> Signed-off-by: Jialu Xu <xujialu@vimux.org>
> ---
>  scripts/tags.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/scripts/tags.sh b/scripts/tags.sh
> index ea31640b2671..938dba219534 100755
> --- a/scripts/tags.sh
> +++ b/scripts/tags.sh
> @@ -98,7 +98,7 @@ all_compiled_sources()
>  	{
>  		echo include/generated/autoconf.h
>  		find $ignore -name "*.cmd" -exec \
> -			sed -n -E 's/^source_.* (.*)/\1/p; s/^  (\S.*) \\/\1/p' {} \+ |
> +			grep -Poh '(?<=^  )\S+|(?<== )\S+[^\\](?=$)' {} \+ |
>  		awk '!a[$0]++'
>  	} | xargs realpath -esq $([ -z "$KBUILD_ABS_SRCTREE" ] && echo --relative-to=.) |
>  	sort -u
> -- 
> 2.30.2
> 

Re: Re: [PATCH] scripts/tags.sh: improve compiled sources generation

[Jialu Xu]

Here is a test log of `time make COMPILED_SOURCE=1 cscope`

  # git show
  commit 71f226c00c8a87de4698eda7d2a25e9b602d08e2 (HEAD -> tags.sh)
  Author: Jialu Xu <xujialu@vimux.org>
  Date:   Sat May 6 16:52:13 2023 +0800

      scripts/tags.sh: improve compiled sources generation
      
      Use grep instead of sed for all compiled sources generation, it is three
      times more efficient.

  diff --git a/scripts/tags.sh b/scripts/tags.sh
  index ea31640b26715..938dba219534a 100755
  --- a/scripts/tags.sh
  +++ b/scripts/tags.sh
  @@ -98,7 +98,7 @@ all_compiled_sources()
          {
                  echo include/generated/autoconf.h
                  find $ignore -name "*.cmd" -exec \
  -                       sed -n -E 's/^source_.* (.*)/\1/p; s/^  (\S.*) \\/\1/p' {} \+ |
  +                       grep -Poh '(?<=^  )\S+|(?<== )\S+[^\\](?=$)' {} \+ |
                  awk '!a[$0]++'
          } | xargs realpath -esq $([ -z "$KBUILD_ABS_SRCTREE" ] && echo --relative-to=.) |
          sort -u
  # time make COMPILED_SOURCE=1 cscope
    GEN     cscope

  real	0m3.869s
  user	0m4.254s
  sys	0m0.389s
  # time make COMPILED_SOURCE=1 cscope
    GEN     cscope

  real	0m3.880s
  user	0m4.202s
  sys	0m0.428s
  # git revert HEAD
  [tags.sh 4dc58acca6240] Revert "scripts/tags.sh: improve compiled sources generation"
   1 file changed, 1 insertion(+), 1 deletion(-)
  # git show
  commit 4dc58acca6240364a2303b9d9a4f55db159d60c4 (HEAD -> tags.sh)
  Author: Jialu Xu <xujialu@vimux.org>
  Date:   Fri Jun 2 08:06:09 2023 +0800

      Revert "scripts/tags.sh: improve compiled sources generation"
      
      This reverts commit 71f226c00c8a87de4698eda7d2a25e9b602d08e2.

  diff --git a/scripts/tags.sh b/scripts/tags.sh
  index 938dba219534a..ea31640b26715 100755
  --- a/scripts/tags.sh
  +++ b/scripts/tags.sh
  @@ -98,7 +98,7 @@ all_compiled_sources()
          {
                  echo include/generated/autoconf.h
                  find $ignore -name "*.cmd" -exec \
  -                       grep -Poh '(?<=^  )\S+|(?<== )\S+[^\\](?=$)' {} \+ |
  +                       sed -n -E 's/^source_.* (.*)/\1/p; s/^  (\S.*) \\/\1/p' {} \+ |
                  awk '!a[$0]++'
          } | xargs realpath -esq $([ -z "$KBUILD_ABS_SRCTREE" ] && echo --relative-to=.) |
          sort -u
  # time make COMPILED_SOURCE=1 cscope
    GEN     cscope

  real	0m15.855s
  user	0m16.308s
  sys	0m0.536s
  # time make COMPILED_SOURCE=1 cscope
    GEN     cscope

  real	0m16.043s
  user	0m16.489s
  sys	0m0.493s