[PATCH] scripts: coccicheck: Change default value for parallelism

[PATCH] scripts: coccicheck: Change default value for parallelism

[Sumera Priyadarsini]

By default, coccicheck utilizes all available threads to implement
parallelisation. However, when hyperthreading is enabled, this leads
to all threads per core being occupied resulting in longer wall-clock
times and higher power consumption. Hence, to improve performance,
modify coccicheck to use only one thread per core atmost.

In the cases where the total number of threads is more than 8 and
hyperthreading is enabled, it was observed that optimum performance
is achieved around one-fourth of the total number of cores.
Modify the script further to accommodate this use case.

Signed-off-by: Sumera Priyadarsini <sylphrenadin@gmail.com>
---
 scripts/coccicheck | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/scripts/coccicheck b/scripts/coccicheck
index e04d328210ac..dd228dcc915e 100755
--- a/scripts/coccicheck
+++ b/scripts/coccicheck
@@ -75,8 +75,17 @@ else
         OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE"
     fi
 
+    # Use only one thread per core by default if hyperthreading is enabled
+    THREADS_PER_CORE=$(lscpu | grep "Thread(s) per core: " | tr -cd [:digit:])
     if [ -z "$J" ]; then
         NPROC=$(getconf _NPROCESSORS_ONLN)
+	if [ $THREADS_PER_CORE -gt 1 -a $NPROC -gt 2 ] ; then
+		if [ $NPROC -gt 8 ] ; then
+			NPROC=$((NPROC/4))
+		else
+			NPROC=$((NPROC/2))
+		fi
+	fi
     else
         NPROC="$J"
     fi
-- 
2.17.1

Re: [Cocci] [PATCH] scripts: coccicheck: Change default value for parallelism

[Julia Lawall]

On Wed, 12 Aug 2020, Sumera Priyadarsini wrote:

> By default, coccicheck utilizes all available threads to implement
> parallelisation. However, when hyperthreading is enabled, this leads
> to all threads per core being occupied resulting in longer wall-clock
> times and higher power consumption.

I have the feeling that the above sentence is not quite optimal.
Actually, using all of the available hardware threads would not be a bad
thing, if it was giving a benefit.  The point is that it doesn't.  It
makes the performance worse instead.

> Hence, to improve performance,
> modify coccicheck to use only one thread per core atmost.

"atmost" is not a word.  It would be clearer to say "to use at most one
thread per core".

> In the cases where the total number of threads is more than 8 and
> hyperthreading is enabled, it was observed that optimum performance
> is achieved around one-fourth of the total number of cores.
> Modify the script further to accommodate this use case.

It would be nice to give some performance numbers and some information
about the machine used.

thanks,
julia

>
> Signed-off-by: Sumera Priyadarsini <sylphrenadin@gmail.com>
> ---
>  scripts/coccicheck | 9 +++++++++
>  1 file changed, 9 insertions(+)
>
> diff --git a/scripts/coccicheck b/scripts/coccicheck
> index e04d328210ac..dd228dcc915e 100755
> --- a/scripts/coccicheck
> +++ b/scripts/coccicheck
> @@ -75,8 +75,17 @@ else
>          OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE"
>      fi
>
> +    # Use only one thread per core by default if hyperthreading is enabled
> +    THREADS_PER_CORE=$(lscpu | grep "Thread(s) per core: " | tr -cd [:digit:])
>      if [ -z "$J" ]; then
>          NPROC=$(getconf _NPROCESSORS_ONLN)
> +	if [ $THREADS_PER_CORE -gt 1 -a $NPROC -gt 2 ] ; then
> +		if [ $NPROC -gt 8 ] ; then
> +			NPROC=$((NPROC/4))
> +		else
> +			NPROC=$((NPROC/2))
> +		fi
> +	fi
>      else
>          NPROC="$J"
>      fi
> --
> 2.17.1
>
> _______________________________________________
> Cocci mailing list
> Cocci@systeme.lip6.fr
> https://systeme.lip6.fr/mailman/listinfo/cocci
>

Re: [Cocci] [PATCH] scripts: coccicheck: Change default value for parallelism

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 3130 bytes --]



On Thu, 13 Aug 2020, Sumera Priyadarsini wrote:

>
>
> On Thu, 13 Aug, 2020, 1:45 AM Julia Lawall, <julia.lawall@inria.fr> wrote:
>
>
>       On Wed, 12 Aug 2020, Sumera Priyadarsini wrote:
>
>       > By default, coccicheck utilizes all available threads to implement
>       > parallelisation. However, when hyperthreading is enabled, this leads
>       > to all threads per core being occupied resulting in longer wall-clock
>       > times and higher power consumption.
>
>       I have the feeling that the above sentence is not quite optimal.
>       Actually, using all of the available hardware threads would not be a bad
>       thing, if it was giving a benefit.  The point is that it doesn't.  It
>       makes the performance worse instead.
>
>
> How does this sound? 
>  
> However, when all available threads are used, a decrease in performance was noted. The elapsed time was minimum when at most one thread per core was used. 

This looks better, thanks.

julia

>
>
>
>
>       > Hence, to improve performance,
>       > modify coccicheck to use only one thread per core atmost.
>
>       "atmost" is not a word.  It would be clearer to say "to use at most one
>       thread per core".
>
>       > In the cases where the total number of threads is more than 8 and
>       > hyperthreading is enabled, it was observed that optimum performance
>       > is achieved around one-fourth of the total number of cores.
>       > Modify the script further to accommodate this use case.
>
>       It would be nice to give some performance numbers and some information
>       about the machine used.
>
>
> Alright, will add this. 
>
>
>       thanks,
>       julia
>
>       >
>       > Signed-off-by: Sumera Priyadarsini <sylphrenadin@gmail.com>
>       > ---
>       >  scripts/coccicheck | 9 +++++++++
>       >  1 file changed, 9 insertions(+)
>       >
>       > diff --git a/scripts/coccicheck b/scripts/coccicheck
>       > index e04d328210ac..dd228dcc915e 100755
>       > --- a/scripts/coccicheck
>       > +++ b/scripts/coccicheck
>       > @@ -75,8 +75,17 @@ else
>       >          OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE"
>       >      fi
>       >
>       > +    # Use only one thread per core by default if hyperthreading is enabled
>       > +    THREADS_PER_CORE=$(lscpu | grep "Thread(s) per core: " | tr -cd [:digit:])
>       >      if [ -z "$J" ]; then
>       >          NPROC=$(getconf _NPROCESSORS_ONLN)
>       > +     if [ $THREADS_PER_CORE -gt 1 -a $NPROC -gt 2 ] ; then
>       > +             if [ $NPROC -gt 8 ] ; then
>       > +                     NPROC=$((NPROC/4))
>       > +             else
>       > +                     NPROC=$((NPROC/2))
>       > +             fi
>       > +     fi
>       >      else
>       >          NPROC="$J"
>       >      fi
>       > --
>       > 2.17.1
>       >
>       > _______________________________________________
>       > Cocci mailing list
>       > Cocci@systeme.lip6.fr
>       > https://systeme.lip6.fr/mailman/listinfo/cocci
>       >
>
>
>