From: Robin Murphy <robin.murphy@arm.com>
To: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Mostafa Saleh <smostafa@google.com>,
iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org, will@kernel.org, joro@8bytes.org,
praan@google.com
Subject: Re: [PATCH v5 4/4] iommu/io-pgtable-arm-selftests: Use KUnit
Date: Thu, 16 Oct 2025 18:17:35 +0100 [thread overview]
Message-ID: <73a1d5d0-8077-450c-a38f-c1b027088258@arm.com> (raw)
In-Reply-To: <20251015151002.GH3938986@ziepe.ca>
On 2025-10-15 4:10 pm, Jason Gunthorpe wrote:
> On Wed, Oct 15, 2025 at 02:51:15PM +0100, Robin Murphy wrote:
>> On 2025-10-15 10:53 am, Mostafa Saleh wrote:
>>>> Conversely, this is infrastructure, not an actual test of expected
>>>> io-pgtable behaviour, so I think just:
>>>>
>>>> cfg.iommu_dev = kunit_device_register(test, "io-pgtable-test");
>>>> if (IS_ERR(cfg.iommu_dev))
>>>> return;
>>>>
>>>> (it doesn't return NULLs either)
>>>>
>>>
>>> Yes, I was not sure about this one, when checking the code base, every test
>>> handles kunit_device_register() failure differently, this seemed the
>>> most strict one so I used it, I will update that in the next version.
>>
>> Yeah, my impression is that those have likely been copied from the
>> lib/kunit/ code where it is actually testing its own API. I've now sent a
>> patch for the example in the docs...
>
> I think any failure to run the test should be reported, either with an
> err or a skip. Tests that didn't do anything and silently report
> success are not a good design.
Sure, a test that hasn't completed hasn't succeeded, but it hasn't
failed either, in the sense of there being no fault found in the actual
code being tested. Hence it seems intuitive to me that reporting a
failure in the test infrastructure itself as if it were incorrect
behaviour of the target code is not right.
In this case AFAICS kunit_device_register() can only fail due to OOM or
something unexpectedly messed up in the kobject/sysfs hierarchy, all of
which should already scream (and represent the system being sufficiently
hosed that any actual test results probably no longer matter anyway) -
otherwise I would have suggested a kunit_err() message too.
> Looking at the existing users I see alot are in init functions, so
> they propogate an error code and fail the init.
Indeed I think this one ultimately wants to end up in an init function
too once everything is unpicked, but for now...
> And the rest inside tests do something like this:
>
> dev = kunit_device_register(test, dev_name);
> KUNIT_ASSERT_FALSE_MSG(test, IS_ERR(dev);
> "Cannot register test device\n");
The confusion is that most of those KUNIT_ASSERTs are where the KUnit
API *is* the thing under test, or have clearly copy-pasted the same
example soon after, and some are in init functions which could fail
gracefully, so it's far from clear how much considered intent they
actually represent. In fact AFAICS it's only the two cases of that exact
pattern in overflow and fortify that hint at being an older notion.
However, on further inspection, I see that there also about as many of
examples of kunit_skip() being used when failing to allocate per-test
resources, so I guess that's the pattern I was after, hurrah! I wouldn't
consider the difference between "test skipped because it unexpectedly
couldn't run" and "test skipped because it knowingly wouldn't have been
meaningful or possible to run" significant enough to be worth trying to
split further.
Thanks,
Robin.
next prev parent reply other threads:[~2025-10-16 17:17 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-09-29 15:49 [PATCH v5 0/4] Move io-pgtable-arm selftest to KUnit Mostafa Saleh
2025-09-29 15:49 ` [PATCH v5 1/4] iommu/io-pgtable-arm: Simplify error prints for selftests Mostafa Saleh
2025-10-14 11:09 ` Robin Murphy
2025-10-15 9:41 ` Mostafa Saleh
2025-09-29 15:49 ` [PATCH v5 2/4] iommu/io-pgtable-arm: Move selftests to a separate file Mostafa Saleh
2025-09-29 15:49 ` [PATCH v5 3/4] iommu/io-pgtable-arm-selftests: Modularize the test Mostafa Saleh
2025-09-29 15:49 ` [PATCH v5 4/4] iommu/io-pgtable-arm-selftests: Use KUnit Mostafa Saleh
2025-10-14 13:43 ` Robin Murphy
2025-10-15 9:53 ` Mostafa Saleh
2025-10-15 13:51 ` Robin Murphy
2025-10-15 15:10 ` Jason Gunthorpe
2025-10-16 17:17 ` Robin Murphy [this message]
2025-10-16 17:25 ` Jason Gunthorpe
2025-10-17 13:04 ` Mostafa Saleh
2025-10-17 14:13 ` Jason Gunthorpe
2025-10-17 17:59 ` Mostafa Saleh
2025-10-13 9:32 ` [PATCH v5 0/4] Move io-pgtable-arm selftest to KUnit Mostafa Saleh
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=73a1d5d0-8077-450c-a38f-c1b027088258@arm.com \
--to=robin.murphy@arm.com \
--cc=iommu@lists.linux.dev \
--cc=jgg@ziepe.ca \
--cc=joro@8bytes.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=praan@google.com \
--cc=smostafa@google.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®