mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Mark Brown <broonie@kernel.org>
To: "Shuah Khan" <shuah@kernel.org>,
	"Nathan Chancellor" <nathan@kernel.org>,
	"Nick Desaulniers" <nick.desaulniers+lkml@gmail.com>,
	"Bill Wendling" <morbo@google.com>,
	"Justin Stitt" <justinstitt@google.com>,
	"Ricardo B. Marlière" <rbm@suse.com>
Cc: Shuah Khan <skhan@linuxfoundation.org>,
	linux-kselftest@vger.kernel.org,  linux-kernel@vger.kernel.org,
	llvm@lists.linux.dev,  Mark Brown <broonie@kernel.org>
Subject: [PATCH] selftests: Deescalate error reporting
Date: Thu, 16 Apr 2026 14:19:24 +0100	[thread overview]
Message-ID: <20260416-selftests-deescalate-error-reporting-v1-1-38e7c0536227@kernel.org> (raw)

Commit 7e47389142b8 ("selftests: Preserve subtarget failures in
all/install") updated the propagation of errors from indivdual kselftest
targets to be similar to that seen with FORCE_TARGETS.  While it would
be really nice to be in a position to do this currently it is premature
to do this as the default behaviour.

At present we default to trying to build all selftests but a combination
of code quality issues and build dependencies mean that it is almost
certain that at least one of them will fail to build (for example,
several depend on clang so don't work in a GCC container) and a top
level failure in the kselftest build reported.  Further, the resulting
failures mean that the install target does not run at all so any build
problem is escallated to a complete failure to produce a kselftest
tarball so CI systems that run into issues loose all selftests coverage.
This has been causing disruption to a range of CI systems including
KernelCI, mine and Arm's internal one.

Revert the commit, users who need this behaviour should be able to use
FORCE_TARGETS for the time being.  At present users that do this (such
as linux-next) are most likely building a subset of targets known to
succeed in their environments.

Fixes: 7e47389142b8 ("selftests: Preserve subtarget failures in all/install")
Signed-off-by: Mark Brown <broonie@kernel.org>
---
 tools/testing/selftests/Makefile | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
index 0949f370ad78..450f13ba4cca 100644
--- a/tools/testing/selftests/Makefile
+++ b/tools/testing/selftests/Makefile
@@ -209,14 +209,14 @@ export KHDR_INCLUDES
 .DEFAULT_GOAL := all
 
 all:
-	@ret=0;							\
+	@ret=1;							\
 	for TARGET in $(TARGETS) $(INSTALL_DEP_TARGETS); do	\
 		BUILD_TARGET=$$BUILD/$$TARGET;			\
 		mkdir $$BUILD_TARGET  -p;			\
 		$(MAKE) OUTPUT=$$BUILD_TARGET -C $$TARGET	\
 				O=$(abs_objtree)		\
 				$(if $(FORCE_TARGETS),|| exit);	\
-		[ $$? -eq 0 ] || ret=1;			\
+		ret=$$((ret * $$?));				\
 	done; exit $$ret;
 
 run_tests: all
@@ -274,7 +274,7 @@ ifdef INSTALL_PATH
 	install -m 744 kselftest/ksft.py $(INSTALL_PATH)/kselftest/
 	install -m 744 run_kselftest.sh $(INSTALL_PATH)/
 	rm -f $(TEST_LIST)
-	@ret=0;	\
+	@ret=1;	\
 	for TARGET in $(TARGETS) $(INSTALL_DEP_TARGETS); do \
 		BUILD_TARGET=$$BUILD/$$TARGET;	\
 		$(MAKE) OUTPUT=$$BUILD_TARGET -C $$TARGET install \
@@ -283,7 +283,7 @@ ifdef INSTALL_PATH
 				OBJ_PATH=$(INSTALL_PATH) \
 				O=$(abs_objtree)		\
 				$(if $(FORCE_TARGETS),|| exit);	\
-		[ $$? -eq 0 ] || ret=1;		\
+		ret=$$((ret * $$?));		\
 	done; exit $$ret;
 
 

---
base-commit: 7e47389142b8ada66280be71e01a3238751086f0
change-id: 20260416-selftests-deescalate-error-reporting-4a492571bb27

Best regards,
--  
Mark Brown <broonie@kernel.org>


             reply	other threads:[~2026-04-16 13:21 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-04-16 13:19 Mark Brown [this message]
2026-04-16 21:14 ` Shuah Khan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260416-selftests-deescalate-error-reporting-v1-1-38e7c0536227@kernel.org \
    --to=broonie@kernel.org \
    --cc=justinstitt@google.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=llvm@lists.linux.dev \
    --cc=morbo@google.com \
    --cc=nathan@kernel.org \
    --cc=nick.desaulniers+lkml@gmail.com \
    --cc=rbm@suse.com \
    --cc=shuah@kernel.org \
    --cc=skhan@linuxfoundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®