mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Robin Murphy <robin.murphy@arm.com>
To: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Mostafa Saleh <smostafa@google.com>,
	iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org,
	linux-kernel@vger.kernel.org, will@kernel.org, joro@8bytes.org,
	praan@google.com
Subject: Re: [PATCH v5 4/4] iommu/io-pgtable-arm-selftests: Use KUnit
Date: Thu, 16 Oct 2025 18:17:35 +0100	[thread overview]
Message-ID: <73a1d5d0-8077-450c-a38f-c1b027088258@arm.com> (raw)
In-Reply-To: <20251015151002.GH3938986@ziepe.ca>

On 2025-10-15 4:10 pm, Jason Gunthorpe wrote:
> On Wed, Oct 15, 2025 at 02:51:15PM +0100, Robin Murphy wrote:
>> On 2025-10-15 10:53 am, Mostafa Saleh wrote:
>>>> Conversely, this is infrastructure, not an actual test of expected
>>>> io-pgtable behaviour, so I think just:
>>>>
>>>> 	cfg.iommu_dev = kunit_device_register(test, "io-pgtable-test");
>>>> 	if (IS_ERR(cfg.iommu_dev))
>>>> 		return;
>>>>
>>>> (it doesn't return NULLs either)
>>>>
>>>
>>> Yes, I was not sure about this one, when checking the code base, every test
>>> handles kunit_device_register() failure differently, this seemed the
>>> most strict one so I used it, I will update that in the next version.
>>
>> Yeah, my impression is that those have likely been copied from the
>> lib/kunit/ code where it is actually testing its own API. I've now sent a
>> patch for the example in the docs...
> 
> I think any failure to run the test should be reported, either with an
> err or a skip. Tests that didn't do anything and silently report
> success are not a good design.

Sure, a test that hasn't completed hasn't succeeded, but it hasn't 
failed either, in the sense of there being no fault found in the actual 
code being tested. Hence it seems intuitive to me that reporting a 
failure in the test infrastructure itself as if it were incorrect 
behaviour of the target code is not right.

In this case AFAICS kunit_device_register() can only fail due to OOM or 
something unexpectedly messed up in the kobject/sysfs hierarchy, all of 
which should already scream (and represent the system being sufficiently 
hosed that any actual test results probably no longer matter anyway) - 
otherwise I would have suggested a kunit_err() message too.

> Looking at the existing users I see alot are in init functions, so
> they propogate an error code and fail the init.

Indeed I think this one ultimately wants to end up in an init function 
too once everything is unpicked, but for now...
> And the rest inside tests do something like this:
> 
> 	dev = kunit_device_register(test, dev_name);			
> 	KUNIT_ASSERT_FALSE_MSG(test, IS_ERR(dev);
> 			       "Cannot register test device\n");

The confusion is that most of those KUNIT_ASSERTs are where the KUnit 
API *is* the thing under test, or have clearly copy-pasted the same 
example soon after, and some are in init functions which could fail 
gracefully, so it's far from clear how much considered intent they 
actually represent. In fact AFAICS it's only the two cases of that exact 
pattern in overflow and fortify that hint at being an older notion.

However, on further inspection, I see that there also about as many of 
examples of kunit_skip() being used when failing to allocate per-test 
resources, so I guess that's the pattern I was after, hurrah! I wouldn't 
consider the difference between "test skipped because it unexpectedly 
couldn't run" and "test skipped because it knowingly wouldn't have been 
meaningful or possible to run" significant enough to be worth trying to 
split further.

Thanks,
Robin.

  reply	other threads:[~2025-10-16 17:17 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-09-29 15:49 [PATCH v5 0/4] Move io-pgtable-arm selftest to KUnit Mostafa Saleh
2025-09-29 15:49 ` [PATCH v5 1/4] iommu/io-pgtable-arm: Simplify error prints for selftests Mostafa Saleh
2025-10-14 11:09   ` Robin Murphy
2025-10-15  9:41     ` Mostafa Saleh
2025-09-29 15:49 ` [PATCH v5 2/4] iommu/io-pgtable-arm: Move selftests to a separate file Mostafa Saleh
2025-09-29 15:49 ` [PATCH v5 3/4] iommu/io-pgtable-arm-selftests: Modularize the test Mostafa Saleh
2025-09-29 15:49 ` [PATCH v5 4/4] iommu/io-pgtable-arm-selftests: Use KUnit Mostafa Saleh
2025-10-14 13:43   ` Robin Murphy
2025-10-15  9:53     ` Mostafa Saleh
2025-10-15 13:51       ` Robin Murphy
2025-10-15 15:10         ` Jason Gunthorpe
2025-10-16 17:17           ` Robin Murphy [this message]
2025-10-16 17:25             ` Jason Gunthorpe
2025-10-17 13:04               ` Mostafa Saleh
2025-10-17 14:13                 ` Jason Gunthorpe
2025-10-17 17:59                   ` Mostafa Saleh
2025-10-13  9:32 ` [PATCH v5 0/4] Move io-pgtable-arm selftest to KUnit Mostafa Saleh

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=73a1d5d0-8077-450c-a38f-c1b027088258@arm.com \
    --to=robin.murphy@arm.com \
    --cc=iommu@lists.linux.dev \
    --cc=jgg@ziepe.ca \
    --cc=joro@8bytes.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=praan@google.com \
    --cc=smostafa@google.com \
    --cc=will@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®