[PATCH v4 0/4] selftests/livepatch: build and run the tests outside the running kernel

[PATCH v4 0/4] selftests/livepatch: build and run the tests outside the running kernel

[Sasha Levin]

The livepatch selftests build their test modules against the running
kernel and skip every test without its devel files, so they do not
work when built for one kernel and run on another, as CI systems do.
In KernelCI, 36 of 69 kselftest builds of mainline fail to compile the
modules because the kernel lacks CONFIG_LIVEPATCH, and where the
modules do build, the installed tests skip themselves.

- Patch 1 skips the modules and tests when livepatch is not enabled.
- Patch 2 lets "make clean" remove the modules without a kernel build
  directory.
- Patch 3 builds the modules against the kernel in O= or KBUILD_OUTPUT.
- Patch 4 runs the tests with the installed modules, without the devel
  files.

Tested on x86_64 against kernels built with O= and in place, a kernel
without livepatch and the Debian headers. The modules build and clean
in every case, and the installed tests pass 8/8 in a VM without devel
files, where they were all skipped before.

Changes since v3:
- Split into four patches, rewrote the commit messages and added
  comments. (Petr)
- Moved the CONFIG_LIVEPATCH check into test_modules/Makefile, which
  drops the extra clean target (Petr), and read it from
  include/config/auto.conf.
- Patch 4 still skips when neither the modules nor KDIR are available.
  (Petr)

v3: https://lore.kernel.org/all/20261103222706.1642224-1-sashal@kernel.org/

Sasha Levin (4):
  selftests/livepatch: skip the tests on kernels without livepatch
  selftests/livepatch: clean the test modules without a kernel build
    directory
  selftests/livepatch: build the test modules against the kernel in O=
  selftests/livepatch: run the tests with prebuilt test modules

 tools/testing/selftests/livepatch/Makefile    | 26 +++++++++++++++++
 .../testing/selftests/livepatch/functions.sh  | 13 ++++++---
 .../selftests/livepatch/test_modules/Makefile | 29 ++++++++++++++-----
 3 files changed, 57 insertions(+), 11 deletions(-)


base-commit: b3dfbde62cf1981c4638939c159dfd087b7a72e9
-- 
2.53.0

[PATCH v4 1/4] selftests/livepatch: skip the tests on kernels without livepatch

[Sasha Levin]

The test modules can only be built against a kernel configured with
CONFIG_LIVEPATCH, but the selftests try to build them for any kernel.

KernelCI builds the selftests for many kernel configurations, most of
which do not enable livepatch. On mainline from November 8, 36 of its
69 kselftest builds fail to compile the test modules, for example:

  test_klp_callbacks_demo2.c:23:16: error: invalid use of undefined type 'struct klp_object'

These are arm and riscv kernels, which do not support livepatch, and
x86, arm64, powerpc and loongarch configurations that do not enable it.

Do not build the test modules when the kernel in KDIR was configured
without CONFIG_LIVEPATCH, and skip the tests when the running kernel has
no /sys/kernel/livepatch.

Fixes: c4bbe83d27c2 ("livepatch: Move tests from lib/livepatch to selftests/livepatch")
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 tools/testing/selftests/livepatch/functions.sh     |  1 +
 .../selftests/livepatch/test_modules/Makefile      | 14 ++++++++++++--
 2 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/livepatch/functions.sh b/tools/testing/selftests/livepatch/functions.sh
index 429fce1f31431..8b26b70f92d8c 100644
--- a/tools/testing/selftests/livepatch/functions.sh
+++ b/tools/testing/selftests/livepatch/functions.sh
@@ -150,6 +150,7 @@ function cleanup() {
 #		 the ftrace_enabled sysctl.
 function setup_config() {
 	is_root
+	[[ -d "$SYSFS_KLP_DIR" ]] || skip "livepatch is not available"
 	has_kdir
 	push_config
 	set_dynamic_debug
diff --git a/tools/testing/selftests/livepatch/test_modules/Makefile b/tools/testing/selftests/livepatch/test_modules/Makefile
index a13d398585dcb..eddfb420db320 100644
--- a/tools/testing/selftests/livepatch/test_modules/Makefile
+++ b/tools/testing/selftests/livepatch/test_modules/Makefile
@@ -16,9 +16,19 @@ obj-m += test_klp_atomic_replace.o \
 	test_klp_state3.o \
 	test_klp_syscall.o
 
-# Ensure that KDIR exists, otherwise skip the compilation
-modules:
+# Build the modules only if KDIR exists and its kernel was configured with
+# CONFIG_LIVEPATCH, as recorded in include/config/auto.conf. If that cannot
+# be read, still try the build, so that a KDIR that is not a kernel build
+# directory fails the build instead of being skipped.
+KDIR_CONFIG := $(KDIR)/include/config/auto.conf
 ifneq ("$(wildcard $(KDIR))", "")
+ifneq ($(shell [ -r "$(KDIR_CONFIG)" ] && ! grep -qx 'CONFIG_LIVEPATCH=y' "$(KDIR_CONFIG)" && echo n),n)
+BUILD_TEST_MODS := y
+endif
+endif
+
+modules:
+ifdef BUILD_TEST_MODS
 	$(Q)$(MAKE) -C $(KDIR) modules KBUILD_EXTMOD=$(TESTMODS_DIR)
 endif
 
-- 
2.53.0

[PATCH v4 2/4] selftests/livepatch: clean the test modules without a kernel build directory

[Sasha Levin]

"make clean" removes the test modules only when KDIR exists, and KDIR
defaults to the build directory of the running kernel. Modules built
for another kernel are therefore left in the source tree by a plain
"make clean" on a machine without the running kernel's devel files,
for example after building them as KernelCI does:

  make -C tools/testing/selftests O=<kernel> KDIR=<kernel> TARGETS=livepatch
  make -C tools/testing/selftests TARGETS=livepatch clean

Removing external modules does not need a configured kernel. Clean
them through the kernel source tree that contains the selftests, and
do not pass on O= or KBUILD_OUTPUT, which the kernel Makefile would
take as its build directory.

Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 .../testing/selftests/livepatch/test_modules/Makefile | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/tools/testing/selftests/livepatch/test_modules/Makefile b/tools/testing/selftests/livepatch/test_modules/Makefile
index eddfb420db320..01eaf2e1755c9 100644
--- a/tools/testing/selftests/livepatch/test_modules/Makefile
+++ b/tools/testing/selftests/livepatch/test_modules/Makefile
@@ -32,8 +32,11 @@ ifdef BUILD_TEST_MODS
 	$(Q)$(MAKE) -C $(KDIR) modules KBUILD_EXTMOD=$(TESTMODS_DIR)
 endif
 
-# Ensure that KDIR exists, otherwise skip the clean target
+# Cleaning external modules needs no configured kernel, so clean them
+# through the kernel source tree that contains this directory. The kernel
+# Makefile would take an O= or KBUILD_OUTPUT passed on from the selftests
+# build as its build directory, so drop them.
+clean: MAKEOVERRIDES := $(filter-out O=% KBUILD_OUTPUT=%,$(MAKEOVERRIDES))
 clean:
-ifneq ("$(wildcard $(KDIR))", "")
-	$(Q)$(MAKE) -C $(KDIR) clean KBUILD_EXTMOD=$(TESTMODS_DIR)
-endif
+	$(Q)env -u KBUILD_OUTPUT $(MAKE) -C $(TESTMODS_DIR)/../../../../.. \
+		clean KBUILD_EXTMOD=$(TESTMODS_DIR)
-- 
2.53.0

[PATCH v4 3/4] selftests/livepatch: build the test modules against the kernel in O=

[Sasha Levin]

Documentation/dev-tools/kselftest.rst describes building the selftests
for a kernel built in a separate output directory:

  make O=/tmp/kselftest TARGETS=livepatch kselftest

The livepatch test modules are not built against the kernel in
/tmp/kselftest, but against the running kernel, using its devel files
in /lib/modules/$(uname -r)/build. When those are not installed, the
modules are silently left out. Only an explicit KDIR=/tmp/kselftest
builds them for the kernel in /tmp/kselftest.

When KDIR is not set, use the output directory given with O= or
KBUILD_OUTPUT if a kernel was built there, then the kernel source tree
if the kernel was built in it, and only then the running kernel.

The selftests also pass their O= on to the module build, and the kernel
Makefile takes it as the kernel build directory instead of KDIR. The
top-level selftests Makefile passes O= even when it is empty, so on a
Debian machine with the devel files of the running kernel installed,

  make -C tools/testing/selftests TARGETS=livepatch

uses the source part of the devel files as the build directory and
fails:

  /usr/src/linux-headers-6.19.13+deb14-common/Makefile:798: /usr/src/linux-headers-6.19.13+deb14-common/include/config/auto.conf: No such file or directory

Do not pass O= and KBUILD_OUTPUT on to the module build, since KDIR
already names the kernel build directory.

Fixes: c4bbe83d27c2 ("livepatch: Move tests from lib/livepatch to selftests/livepatch")
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 tools/testing/selftests/livepatch/Makefile    | 26 +++++++++++++++++++
 .../selftests/livepatch/test_modules/Makefile | 12 +++++----
 2 files changed, 33 insertions(+), 5 deletions(-)

diff --git a/tools/testing/selftests/livepatch/Makefile b/tools/testing/selftests/livepatch/Makefile
index a080eb54a215d..97478ef90945c 100644
--- a/tools/testing/selftests/livepatch/Makefile
+++ b/tools/testing/selftests/livepatch/Makefile
@@ -1,5 +1,31 @@
 # SPDX-License-Identifier: GPL-2.0
 
+# The test modules are built against the kernel in KDIR. When KDIR is not
+# set, use the output directory given with O= or KBUILD_OUTPUT if a kernel
+# was built there, as with
+#   make O=/tmp/kselftest TARGETS=livepatch kselftest
+# otherwise the kernel source tree if the kernel was built in it, as with
+#   cd tools/testing/selftests/livepatch; make
+# and otherwise the build directory of the running kernel, which its
+# devel files provide. A kernel was built in a directory if it has a
+# Module.symvers.
+ifndef KDIR
+KBUILD_DIR := $(firstword $(O) $(KBUILD_OUTPUT))
+ifneq ($(and $(KBUILD_DIR),$(wildcard $(KBUILD_DIR)/Module.symvers)),)
+KDIR := $(KBUILD_DIR)
+else ifneq ($(wildcard ../../../../Module.symvers),)
+KDIR := ../../../..
+else
+KDIR := /lib/modules/$(shell uname -r)/build
+endif
+endif
+
+# test_modules is built from its own directory, so pass KDIR on as an
+# absolute path, also when it was given on the command line.
+override KDIR := $(abspath $(KDIR))
+export KDIR
+MAKEOVERRIDES := $(filter-out KDIR=%,$(MAKEOVERRIDES))
+
 TEST_GEN_FILES := test_klp-call_getpid
 TEST_GEN_MODS_DIR := test_modules
 TEST_PROGS_EXTENDED := functions.sh
diff --git a/tools/testing/selftests/livepatch/test_modules/Makefile b/tools/testing/selftests/livepatch/test_modules/Makefile
index 01eaf2e1755c9..82f0119f6cf0d 100644
--- a/tools/testing/selftests/livepatch/test_modules/Makefile
+++ b/tools/testing/selftests/livepatch/test_modules/Makefile
@@ -16,6 +16,11 @@ obj-m += test_klp_atomic_replace.o \
 	test_klp_state3.o \
 	test_klp_syscall.o
 
+# The kernel Makefile would take an O= or KBUILD_OUTPUT passed on from the
+# selftests build as its build directory, instead of KDIR or, for clean,
+# the source tree. Drop them.
+modules clean: MAKEOVERRIDES := $(filter-out O=% KBUILD_OUTPUT=%,$(MAKEOVERRIDES))
+
 # Build the modules only if KDIR exists and its kernel was configured with
 # CONFIG_LIVEPATCH, as recorded in include/config/auto.conf. If that cannot
 # be read, still try the build, so that a KDIR that is not a kernel build
@@ -29,14 +34,11 @@ endif
 
 modules:
 ifdef BUILD_TEST_MODS
-	$(Q)$(MAKE) -C $(KDIR) modules KBUILD_EXTMOD=$(TESTMODS_DIR)
+	$(Q)env -u KBUILD_OUTPUT $(MAKE) -C $(KDIR) modules KBUILD_EXTMOD=$(TESTMODS_DIR)
 endif
 
 # Cleaning external modules needs no configured kernel, so clean them
-# through the kernel source tree that contains this directory. The kernel
-# Makefile would take an O= or KBUILD_OUTPUT passed on from the selftests
-# build as its build directory, so drop them.
-clean: MAKEOVERRIDES := $(filter-out O=% KBUILD_OUTPUT=%,$(MAKEOVERRIDES))
+# through the kernel source tree that contains this directory.
 clean:
 	$(Q)env -u KBUILD_OUTPUT $(MAKE) -C $(TESTMODS_DIR)/../../../../.. \
 		clean KBUILD_EXTMOD=$(TESTMODS_DIR)
-- 
2.53.0

[PATCH v4 4/4] selftests/livepatch: run the tests with prebuilt test modules

[Sasha Levin]

The tests are skipped unless the machine running them has a kernel
build directory, in $KDIR or /lib/modules/$(uname -r)/build. They do
not need one when the test modules were built with the selftests,
because the scripts load the modules from test_modules/.

As a result the tests are skipped on a machine running an installed
selftests tree without the kernel devel files, for example one
installed with

  make O=/tmp/kselftest TARGETS=livepatch kselftest-install

although the modules are there.

Run the tests when the test modules are available, so that they can
run without the kernel devel files. Without the modules, keep skipping
the tests when there is no kernel build directory to build them with.

Fixes: 54ee3526796f ("selftests: livepatch: Avoid running the tests if kernel-devel is missing")
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 tools/testing/selftests/livepatch/functions.sh | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/tools/testing/selftests/livepatch/functions.sh b/tools/testing/selftests/livepatch/functions.sh
index 8b26b70f92d8c..eebd36b0e0eaf 100644
--- a/tools/testing/selftests/livepatch/functions.sh
+++ b/tools/testing/selftests/livepatch/functions.sh
@@ -44,14 +44,18 @@ function is_root() {
 	fi
 }
 
-# Check if we can compile the modules before loading them
-function has_kdir() {
+# Check that the test modules were built, or that we can compile them
+function has_test_modules() {
+	if compgen -G "test_modules/*.ko" > /dev/null; then
+		return
+	fi
+
 	if [ -z "$KDIR" ]; then
 		KDIR="/lib/modules/$(uname -r)/build"
 	fi
 
 	if [ ! -d "$KDIR" ]; then
-		echo "skip all tests: KDIR ($KDIR) not available to compile modules."
+		echo "skip all tests: test modules are not available and KDIR ($KDIR) is not available to compile them."
 		exit $ksft_skip
 	fi
 }
@@ -151,7 +155,7 @@ function cleanup() {
 function setup_config() {
 	is_root
 	[[ -d "$SYSFS_KLP_DIR" ]] || skip "livepatch is not available"
-	has_kdir
+	has_test_modules
 	push_config
 	set_dynamic_debug
 	set_ftrace_enabled 1
-- 
2.53.0