Re: [PATCH v2 0/2] selftests: cgroup: Report failed checks in test_memcontrol

["Michal Koutný" <mkoutny@suse.com>]

[-- Attachment #1: Type: text/plain, Size: 3096 bytes --]

Hi Sebastian.

On Sat, Sep 26, 2026 at 07:51:46AM +0200, Sebastian Chlad <sebastianchlad@gmail.com> wrote:
> test_memcontrol fails intermittently, but most of its checks fail
> silently, so a "not ok" line gives no indication of how far off the
> measured value was. This series makes those failures print the observed
> and expected values. It does not change any threshold or pass/fail
> behaviour.
> 
> Patch 1 converts the terminal tolerance checks to values_close_report(),
> as was done for test_cpu in commit 4cdde87d723a ("selftests: cgroup: Use
> values_close_report in test_cpu"). values_close() is kept where it is a
> loop or retry condition, where a mismatch is expected.
> 
> Patch 2 reports the "memory.current < 50M" check in
> alloc_pagecache_50M_check(). Because test_memcg_high() calls the same
> helper expecting it to fail, it gets a quiet variant. An alternative
> would be to pass the flag through the currently unused void *arg of
> the cg_run() callback; I am happy to switch to that if preferred.
> 
> With patch 1 applied, we observe at times failures like these:
> 
>   not ok 3 test_memcg_min
>   [FAIL] actual=18722816 expected=30408704 | diff=11685888 | limit=7369725 | tolerance=15% | actual_error=23.78%
> 
>   not ok 4 test_memcg_low
>   [FAIL] actual=20852736 expected=30408704 | diff=9555968 | limit=7689210 | tolerance=15% | actual_error=18.64%
> 
>   not ok 4 test_memcg_low
>   [FAIL] actual=14364672 expected=22020096 | diff=7655424 | limit=7276940 | tolerance=20% | actual_error=21.04%
> 
> These are the child memory.current checks in test_memcg_protection()
> (expected ~29M and ~21M). Their tolerances were already raised in commit
> d2def68ae06a ("selftests: memcg: increase error tolerance of child
> memory.current check in test_memcg_protection()"), and the values above
> still fall outside them. In the first case the child is about 38% below
> the expected 29M. This series only makes such failures visible; whether
> the expectations or the tolerances should change is left for a separate
> discussion.
> 
> Changes in v2:
> - Patch 1: pass the measured value first in the alloc_anon_50M_check()
>   values_close_report() call, so that a failure prints actual and
>   expected the right way round (reported by Sashiko AI review).
> 
> v1: https://lore.kernel.org/all/20260926034923.31718-1-sebastian.chlad@suse.com/
> 
> Sebastian Chlad (2):
>   selftests: cgroup: Use values_close_report in test_memcontrol
>   selftests: cgroup: Report memory.current shortfall in
>     alloc_pagecache_50M_check
> 
>  .../selftests/cgroup/test_memcontrol.c        | 50 +++++++++++++------
>  1 file changed, 36 insertions(+), 14 deletions(-)

AFAIU, the test is catching two types of errors:
a) running under global pressure (preparations already fail),
b) mismatches in expected values of protection tests (after
   preparation).

I like the reporting of the latter -- with (only) that all consequent
silent failures would imply^H strongly hint the error of type a), no?
:-)

Thanks,
Michal

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 265 bytes --]