[PATCH] staging: iio: tsl2x7x_core: Fix standard deviation calculation

[PATCH] staging: iio: tsl2x7x_core: Fix standard deviation calculation

[Eva Rachel Retuya]

Standard deviation is calculated as the square root of the variance
where variance is the mean of sample_sum and length. Correct the
computation of statP->stddev in accordance to the proper calculation.

Fixes: 3c97c08b5735 ("staging: iio: add TAOS tsl2x7x driver")
Reported-by: Abhiram Balasubramanian <abhiram@cs.utah.edu>
Signed-off-by: Eva Rachel Retuya <eraretuya@gmail.com>
---
 drivers/staging/iio/light/tsl2x7x_core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/staging/iio/light/tsl2x7x_core.c b/drivers/staging/iio/light/tsl2x7x_core.c
index ea15bc1..197201a 100644
--- a/drivers/staging/iio/light/tsl2x7x_core.c
+++ b/drivers/staging/iio/light/tsl2x7x_core.c
@@ -854,7 +854,7 @@ void tsl2x7x_prox_calculate(int *data, int length,
 		tmp = data[i] - statP->mean;
 		sample_sum += tmp * tmp;
 	}
-	statP->stddev = int_sqrt((long)sample_sum) / length;
+	statP->stddev = int_sqrt((long)sample_sum / length);
 }
 
 /**
-- 
2.7.4

Re: [PATCH] staging: iio: tsl2x7x_core: Fix standard deviation calculation

[Jonathan Cameron]

On 20/03/17 11:27, Eva Rachel Retuya wrote:
> Standard deviation is calculated as the square root of the variance
> where variance is the mean of sample_sum and length. Correct the
> computation of statP->stddev in accordance to the proper calculation.
> 
> Fixes: 3c97c08b5735 ("staging: iio: add TAOS tsl2x7x driver")
> Reported-by: Abhiram Balasubramanian <abhiram@cs.utah.edu>
> Signed-off-by: Eva Rachel Retuya <eraretuya@gmail.com>
I've applied this to the togreg branch of iio.git rather than sending
as a quick fix for the simple reason that nothing currently uses the
value once it has been computed that I can immediately see!

That whole chunk of code probably just wants to be ripped out in
the longer term.

Jonathan
> ---
>  drivers/staging/iio/light/tsl2x7x_core.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/staging/iio/light/tsl2x7x_core.c b/drivers/staging/iio/light/tsl2x7x_core.c
> index ea15bc1..197201a 100644
> --- a/drivers/staging/iio/light/tsl2x7x_core.c
> +++ b/drivers/staging/iio/light/tsl2x7x_core.c
> @@ -854,7 +854,7 @@ void tsl2x7x_prox_calculate(int *data, int length,
>  		tmp = data[i] - statP->mean;
>  		sample_sum += tmp * tmp;
>  	}
> -	statP->stddev = int_sqrt((long)sample_sum) / length;
> +	statP->stddev = int_sqrt((long)sample_sum / length);
>  }
>  
>  /**
>