Re: [PATCH] selftests: use $SHELL to exec selftests

["Luis R. Rodriguez" <mcgrof@kernel.org>]

On Thu, Aug 03, 2017 at 10:27:33AM -0700, Kees Cook wrote:
> On Thu, Aug 3, 2017 at 9:59 AM, Luis R. Rodriguez <mcgrof@kernel.org> wrote:
> > Executing selftests is fragile as if someone forgot to set a secript
> > as executable the test will fail. Setting scripts as executable is
> > desirable to enable folks to execute tests as independent units,
> > however, we can avoid the fragile errors of forgetting to set the
> > script as executable by just invoking the $SHELL for running each
> > script.
> >
> > Suggsted-by: Andrew Morton <akpm@linux-foundation.org>
> > Signed-off-by: Luis R. Rodriguez <mcgrof@kernel.org>
> > ---
> >
> > Shuah, while the last two patches could be queued in for 4.13-final,
> > this one I think is more appropriate for v4.14-rc1 only.
> >
> >  tools/testing/selftests/lib.mk | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk
> > index 959273c3a52e..2d6abb8037be 100644
> > --- a/tools/testing/selftests/lib.mk
> > +++ b/tools/testing/selftests/lib.mk
> > @@ -14,7 +14,7 @@ all: $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) $(TEST_GEN_FILES)
> >  define RUN_TESTS
> >         @for TEST in $(TEST_GEN_PROGS) $(TEST_PROGS); do \
> >                 BASENAME_TEST=`basename $$TEST`;        \
> > -               cd `dirname $$TEST`; (./$$BASENAME_TEST && echo "selftests: $$BASENAME_TEST [PASS]") || echo "selftests:  $$BASENAME_TEST [FAIL]"; cd -;\
> > +               cd `dirname $$TEST`; ($$SHELL ./$$BASENAME_TEST && echo "selftests: $$BASENAME_TEST [PASS]") || echo "selftests:  $$BASENAME_TEST [FAIL]"; cd -;\
> >         done;
> >  endef
> 
> Is BASENAME_TEST always a script? Can't it be a built binary too?

True, this should only be attempted then if the file is *not* a shell script.
Furthermore any interpreter could be used, so using a shell would not work
for all anyway. So this would be wrong too:

diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk
index 959273c3a52e..e963ee375d77 100644
--- a/tools/testing/selftests/lib.mk
+++ b/tools/testing/selftests/lib.mk
@@ -14,7 +14,12 @@ all: $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) $(TEST_GEN_FILES)
 define RUN_TESTS
 	@for TEST in $(TEST_GEN_PROGS) $(TEST_PROGS); do \
 		BASENAME_TEST=`basename $$TEST`;	\
-		cd `dirname $$TEST`; (./$$BASENAME_TEST && echo "selftests: $$BASENAME_TEST [PASS]") || echo "selftests:  $$BASENAME_TEST [FAIL]"; cd -;\
+		USE_SHELL="";				\
+		if [ ! -x $$BASENAME_TEST ]; then	\
+			USE_SHELL="$$SHELL";		\
+			echo "Warning: file $$BASENAME_TEST is not executable, correct this. Assuming we can run it as a shell script";\
+		fi;					\
+		cd `dirname $$TEST`; ($$USE_SHELL ./$$BASENAME_TEST && echo "selftests: $$BASENAME_TEST [PASS]") || echo "selftests:  $$BASENAME_TEST [FAIL]"; cd -;\
 	done;
 endef
 
Just distinguishing between an ELF binary and an interpreted script seems
rather hacky here too -- even if say we use head -c 2 and grep for "#!" -- this
all just seems convoluted for a Makefile. As such for now I think its best we
just inform the user that the reason why a test failed was the lack of the
executable bit.

Will send patch follow up as v2.

  Luis