[PATCH] kselftest/alsa: Increase kselftest timeout

[PATCH] kselftest/alsa: Increase kselftest timeout

[Nícolas F. R. A. Prado]

The default timeout for kselftests is 45 seconds, but pcm-test can take
longer than that to run depending on the number of PCMs present on a
device.

As a data point, running pcm-test on mt8192-asurada-spherion takes about
1m15s.

Set the timeout to 10 minutes, which should give enough slack to run the
test even on devices with many PCMs.

Signed-off-by: Nícolas F. R. A. Prado <nfraprado@collabora.com>

---

 tools/testing/selftests/alsa/settings | 1 +
 1 file changed, 1 insertion(+)
 create mode 100644 tools/testing/selftests/alsa/settings

diff --git a/tools/testing/selftests/alsa/settings b/tools/testing/selftests/alsa/settings
new file mode 100644
index 000000000000..a62d2fa1275c
--- /dev/null
+++ b/tools/testing/selftests/alsa/settings
@@ -0,0 +1 @@
+timeout=600
-- 
2.39.0

Re: [PATCH] kselftest/alsa: Increase kselftest timeout

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 364 bytes --]

On Tue, Dec 13, 2022 at 03:32:42PM -0300, Nícolas F. R. A. Prado wrote:
> The default timeout for kselftests is 45 seconds, but pcm-test can take
> longer than that to run depending on the number of PCMs present on a
> device.

Reviewed-by: Mark Brown <broonie@kernel.org>

This is also useful for mixer-test with slower control buses and fancier
CODECs.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH] kselftest/alsa: Increase kselftest timeout

[Shuah Khan]

On 12/13/22 11:32, Nícolas F. R. A. Prado wrote:
> The default timeout for kselftests is 45 seconds, but pcm-test can take
> longer than that to run depending on the number of PCMs present on a
> device.
> 
> As a data point, running pcm-test on mt8192-asurada-spherion takes about
> 1m15s.
> 
> Set the timeout to 10 minutes, which should give enough slack to run the
> test even on devices with many PCMs.
> 

10 minutes is way too long.

> Signed-off-by: Nícolas F. R. A. Prado <nfraprado@collabora.com>
> 
> ---
> 
>   tools/testing/selftests/alsa/settings | 1 +
>   1 file changed, 1 insertion(+)
>   create mode 100644 tools/testing/selftests/alsa/settings
> 
> diff --git a/tools/testing/selftests/alsa/settings b/tools/testing/selftests/alsa/settings
> new file mode 100644
> index 000000000000..a62d2fa1275c
> --- /dev/null
> +++ b/tools/testing/selftests/alsa/settings
> @@ -0,0 +1 @@
> +timeout=600

Adding timeouts like this especially 10 minutes will increase the time
it takes to run tests. We run the risk of people not wanting to run tests
anymore.

thanks,
-- Shuah

Re: [PATCH] kselftest/alsa: Increase kselftest timeout

[Nícolas F. R. A. Prado]

On Tue, Dec 13, 2022 at 11:37:56AM -0700, Shuah Khan wrote:
> On 12/13/22 11:32, Nícolas F. R. A. Prado wrote:
> > The default timeout for kselftests is 45 seconds, but pcm-test can take
> > longer than that to run depending on the number of PCMs present on a
> > device.
> > 
> > As a data point, running pcm-test on mt8192-asurada-spherion takes about
> > 1m15s.
> > 
> > Set the timeout to 10 minutes, which should give enough slack to run the
> > test even on devices with many PCMs.
> > 
> 
> 10 minutes is way too long.
> 
> > Signed-off-by: Nícolas F. R. A. Prado <nfraprado@collabora.com>
> > 
> > ---
> > 
> >   tools/testing/selftests/alsa/settings | 1 +
> >   1 file changed, 1 insertion(+)
> >   create mode 100644 tools/testing/selftests/alsa/settings
> > 
> > diff --git a/tools/testing/selftests/alsa/settings b/tools/testing/selftests/alsa/settings
> > new file mode 100644
> > index 000000000000..a62d2fa1275c
> > --- /dev/null
> > +++ b/tools/testing/selftests/alsa/settings
> > @@ -0,0 +1 @@
> > +timeout=600
> 
> Adding timeouts like this especially 10 minutes will increase the time
> it takes to run tests. We run the risk of people not wanting to run tests
> anymore.

I set it high because I suspect the time it takes to run pcm-test might be a lot
higher in some machines (even on mt8192-asurada-spherion, it takes 1m15s, but
only ~10% of the tests are actually running, since many of the PCMs are
disabled).

But I can lower it to, say, 3 minutes, and we can increase the timeout as
needed.

Thanks,
Nícolas

Re: [PATCH] kselftest/alsa: Increase kselftest timeout

[Ricardo Cañuelo]

Hi Shuah,

Bumping an old thread, this is still an issue for this test [1] and it
could end up affecting many others.

On mar 13-12-2022 11:37:56, Shuah Khan wrote:
> On 12/13/22 11:32, Nícolas F. R. A. Prado wrote:
> > The default timeout for kselftests is 45 seconds, but pcm-test can take
> > longer than that to run depending on the number of PCMs present on a
> > device.
> > 
> > As a data point, running pcm-test on mt8192-asurada-spherion takes about
> > 1m15s.
> > 
> > Set the timeout to 10 minutes, which should give enough slack to run the
> > test even on devices with many PCMs.
> > 
> 
> 10 minutes is way too long.

Is there a downside to that? There'll be some tests that take more time,
I don't think that's a problem with the tests or a reason to let them
timeout. IMO it's the test framework which should adapt to the needs of
different types of tests, and the solution provided by this patch is
good enough, it addresses the problem for this particular test suite.

If the solution is still unacceptable, do you have an alternative
proposal in mind that we can try to implement?

Thanks,
Ricardo

[1] https://linux.kernelci.org/test/case/id/646127cf62438996ba2e8648/

Re: [PATCH] kselftest/alsa: Increase kselftest timeout

[Ricardo Cañuelo]

Hi Shuah,

Gentle ping for this

On lun, may 15 2023 at 11:43:10, Ricardo Cañuelo <ricardo.canuelo@collabora.com> wrote:
> Is there a downside to that? There'll be some tests that take more time,
> I don't think that's a problem with the tests or a reason to let them
> timeout. IMO it's the test framework which should adapt to the needs of
> different types of tests, and the solution provided by this patch is
> good enough, it addresses the problem for this particular test suite.
>
> If the solution is still unacceptable, do you have an alternative
> proposal in mind that we can try to implement?

There are some tests failing because of this and we're trying to address
these problems to clean up the regression results.

Thanks,
Ricardo