[PATCH v3 0/5] vfio: selftests: Automatically detect the IOMMU mode

[PATCH v3 0/5] vfio: selftests: Automatically detect the IOMMU mode

[David Matlack]

This series makes the VFIO selftests library automatically detect which
IOMMU mode to use, rather than hardcoding iommufd as the default, and
makes tests skip, rather than fail, when the IOMMU mode they need is not
available. This was suggested by Sean while reviewing the KVM selftest
irq_test [1].

iommufd is an optional feature, and VFIO's iommufd support additionally
requires CONFIG_VFIO_DEVICE_CDEV. On otherwise functional kernels that
only support the legacy VFIO container, the VFIO selftests currently
fail with an unhelpful assert:

  tools/testing/selftests/vfio/lib/iommu.c:448: Assertion Failure

  Expression: iommu->iommufd > 0
  Observed: 0xffffffffffffffff > 0
  [errno: 2 - No such file or directory]

irq_test avoids this by probing for the IOMMU mode itself, but its probe
picks iommufd whenever /dev/iommu exists, even if VFIO device cdevs are
not available.

Patch 1 makes iommu_init(NULL) pick the most preferred IOMMU mode that
is available on the current system, and drops the global
default_iommu_mode.

Patch 2 adds iommu_mode_is_available(), iommu_mode_unavailable_msg(),
and SKIP_IF_IOMMU_MODE_UNAVAILABLE(), and uses them to skip tests when
the IOMMU mode they need is not available.

Patch 3 makes the VFIO selftests library print the names of all known
IOMMU modes when passed an unrecognized mode, and the names of the
available modes when a test skips because the mode it requested is not
available. It also adds iommu_mode_names_all() and
iommu_mode_names_available() so that tests can print both lists.

Patch 4 converts irq_test to use the VFIO selftests library to pick the
IOMMU mode, and drops its probe_iommu_type().

Patch 5 lists the known and available IOMMU modes in irq_test's help
output.

This series is based on vfio/next (vfio-v7.4-rc1).

Testing
-------

Tested in a QEMU VM with an emulated igb device behind an emulated Intel
IOMMU, bound to vfio-pci, on a kernel with CONFIG_IOMMUFD=y,
CONFIG_VFIO_DEVICE_CDEV=y, and CONFIG_VFIO_CONTAINER=y. To simulate
kernels with fewer IOMMU modes available, /dev/iommu and /dev/vfio/vfio
were deleted in the VM:

 - All IOMMU modes available: All VFIO selftests pass. irq_test -d
   passes using iommufd. irq_test -h lists all five IOMMU modes as
   available, and irq_test -d -t bogus fails and lists the known modes.

 - Only the legacy VFIO container available (/dev/iommu deleted): All
   VFIO selftests pass. The iommufd and iommufd_compat variants and
   vfio_iommufd_setup_test skip instead of failing. irq_test -d passes
   using vfio_type1v2_iommu. irq_test -d -t iommufd skips and lists the
   available modes, as does irq_test -h.

 - No IOMMU mode available (/dev/iommu and /dev/vfio/vfio deleted): All
   VFIO selftests and irq_test -d skip with a message explaining why,
   and with well-formed TAP output. irq_test -h lists no available
   modes.

Also tested with CONFIG_VFIO_GROUP=n (cdev-only VFIO):

 - All VFIO selftests pass, with the iommufd variants running and the
   vfio_type1* and iommufd_compat variants skipped. Without the VFIO
   group check added in v3, the iommufd_compat variants fail instead.
   irq_test -d passes using iommufd, and irq_test -h lists only iommufd
   as available.

Changelog
---------

v3:
 - Require VFIO group support (CONFIG_VFIO_GROUP) for container modes,
   including the iommufd_compat modes (patch 1) [Sashiko [2]]
 - List the available IOMMU modes in irq_test's help output, alongside
   all known modes, and when the requested mode is not available
   (patches 3 and 5) [Sean]
 - Simplify the -t help text in irq_test (patch 5) [Sean]
 - Make the unrecognized IOMMU mode error easier to read (patch 3)
   [Sean]
 - Rename iommu_type to iommu_mode in irq_test (patches 4 and 5) [Sean]
 - Skip vfio_iommufd_setup_test when iommufd is not available (patch 2)

v2: https://lore.kernel.org/kvm/20261103002205.3348519-1-dmatlack@google.com/
v1: https://lore.kernel.org/kvm/20261029221710.1871112-1-dmatlack@google.com/

[1] https://lore.kernel.org/kvm/aiIG2L_KdPwuxYA1@google.com/
[2] https://lore.kernel.org/kvm/sashiko-outbox-194380@kernel.org/

David Matlack (5):
  vfio: selftests: Automatically detect the IOMMU mode in iommu_init()
  vfio: selftests: Skip tests when the IOMMU mode is not available
  vfio: selftests: Print the IOMMU modes when a requested mode can't be
    used
  KVM: selftests: Use VFIO selftests library to pick IOMMU mode in
    irq_test
  KVM: selftests: List the IOMMU modes in irq_test's help output

 tools/testing/selftests/kvm/irq_test.c        |  32 ++--
 .../vfio/lib/include/libvfio/iommu.h          |  43 +++++-
 tools/testing/selftests/vfio/lib/iommu.c      | 139 ++++++++++++++++--
 .../vfio/vfio_dma_mapping_mmio_test.c         |   2 +
 .../selftests/vfio/vfio_dma_mapping_test.c    |   4 +
 .../selftests/vfio/vfio_iommufd_setup_test.c  |   2 +
 .../vfio/vfio_pci_device_init_perf_test.c     |   2 +
 .../selftests/vfio/vfio_pci_device_test.c     |   8 +-
 .../selftests/vfio/vfio_pci_driver_test.c     |   9 +-
 .../selftests/vfio/vfio_pci_sriov_uapi_test.c |   4 +
 10 files changed, 205 insertions(+), 40 deletions(-)


base-commit: 076d7e989cb69f6ded7d60f2a080eb909930061b
-- 
2.56.0.506.g506a7eabc5-goog

[PATCH v3 1/5] vfio: selftests: Automatically detect the IOMMU mode in iommu_init()

[David Matlack]

Pick the most preferred IOMMU mode that is available on the current
system when iommu_init() is passed a NULL mode, rather than always using
iommufd. Drop the global default_iommu_mode and convert its users to
pass NULL to iommu_init().

iommufd is an optional feature, and VFIO's iommufd support additionally
requires CONFIG_VFIO_DEVICE_CDEV. Hardcoding iommufd as the default
causes tests to fail with an unhelpful assert on otherwise functional
kernels that only support the legacy VFIO container:

  tools/testing/selftests/vfio/lib/iommu.c:448: Assertion Failure

  Expression: iommu->iommufd > 0
  Observed: 0xffffffffffffffff > 0
  [errno: 2 - No such file or directory]

Order iommu_modes[] by preference: iommufd, then the VFIO container,
then iommufd's VFIO container compatibility modes, with type1v2 ahead of
type1, i.e. prefer iommufd, the native VFIO container over iommufd's
emulation of it, and type1v2 over the older type1. Consider a mode
available if its container (or /dev/iommu) can be opened and supports
the mode's IOMMU type. Also require VFIO device cdevs for iommufd mode
and VFIO groups for container modes, since that is how the selftests
access devices in those modes. If no mode is available, fail with a
message suggesting that VFIO or IOMMUFD may not be loaded and enabled.

Suggested-by: Sean Christopherson <seanjc@google.com>
Link: https://lore.kernel.org/kvm/aiIG2L_KdPwuxYA1@google.com/
Signed-off-by: David Matlack <dmatlack@google.com>
---
 .../vfio/lib/include/libvfio/iommu.h          |  6 +-
 tools/testing/selftests/vfio/lib/iommu.c      | 83 ++++++++++++++++---
 .../selftests/vfio/vfio_pci_device_test.c     |  4 +-
 .../selftests/vfio/vfio_pci_driver_test.c     |  2 +-
 4 files changed, 79 insertions(+), 16 deletions(-)

diff --git a/tools/testing/selftests/vfio/lib/include/libvfio/iommu.h b/tools/testing/selftests/vfio/lib/include/libvfio/iommu.h
index e9a3386a4719..53e38130c334 100644
--- a/tools/testing/selftests/vfio/lib/include/libvfio/iommu.h
+++ b/tools/testing/selftests/vfio/lib/include/libvfio/iommu.h
@@ -15,8 +15,6 @@ struct iommu_mode {
 	unsigned long iommu_type;
 };
 
-extern const char *default_iommu_mode;
-
 struct dma_region {
 	struct list_head link;
 	void *vaddr;
@@ -32,6 +30,10 @@ struct iommu {
 	struct list_head dma_regions;
 };
 
+/*
+ * Pass NULL for iommu_mode to use the most preferred IOMMU mode that is
+ * available on the current system.
+ */
 struct iommu *iommu_init(const char *iommu_mode);
 void iommu_cleanup(struct iommu *iommu);
 
diff --git a/tools/testing/selftests/vfio/lib/iommu.c b/tools/testing/selftests/vfio/lib/iommu.c
index b6f3c5c84e01..ecdfbb62ac5e 100644
--- a/tools/testing/selftests/vfio/lib/iommu.c
+++ b/tools/testing/selftests/vfio/lib/iommu.c
@@ -20,14 +20,18 @@
 #include "../../../kselftest.h"
 #include <libvfio.h>
 
-const char *default_iommu_mode = MODE_IOMMUFD;
+#define NO_IOMMU_MODE_AVAILABLE_MSG \
+	"No IOMMU mode is available. Is VFIO (or IOMMUFD) loaded and enabled?"
 
-/* Reminder: Keep in sync with FIXTURE_VARIANT_ADD_ALL_IOMMU_MODES(). */
+/*
+ * Ordered by preference, i.e. iommu_init(NULL) picks the first mode in this
+ * list that is available on the current system.
+ *
+ * Reminder: Keep in sync with FIXTURE_VARIANT_ADD_ALL_IOMMU_MODES().
+ */
 static const struct iommu_mode iommu_modes[] = {
 	{
-		.name = MODE_VFIO_TYPE1_IOMMU,
-		.container_path = "/dev/vfio/vfio",
-		.iommu_type = VFIO_TYPE1_IOMMU,
+		.name = MODE_IOMMUFD,
 	},
 	{
 		.name = MODE_VFIO_TYPE1V2_IOMMU,
@@ -35,8 +39,8 @@ static const struct iommu_mode iommu_modes[] = {
 		.iommu_type = VFIO_TYPE1v2_IOMMU,
 	},
 	{
-		.name = MODE_IOMMUFD_COMPAT_TYPE1,
-		.container_path = "/dev/iommu",
+		.name = MODE_VFIO_TYPE1_IOMMU,
+		.container_path = "/dev/vfio/vfio",
 		.iommu_type = VFIO_TYPE1_IOMMU,
 	},
 	{
@@ -45,16 +49,72 @@ static const struct iommu_mode iommu_modes[] = {
 		.iommu_type = VFIO_TYPE1v2_IOMMU,
 	},
 	{
-		.name = MODE_IOMMUFD,
+		.name = MODE_IOMMUFD_COMPAT_TYPE1,
+		.container_path = "/dev/iommu",
+		.iommu_type = VFIO_TYPE1_IOMMU,
 	},
 };
 
-static const struct iommu_mode *lookup_iommu_mode(const char *iommu_mode)
+static bool iommufd_probe(void)
+{
+	/* iommufd mode requires VFIO device cdevs (CONFIG_VFIO_DEVICE_CDEV). */
+	return !access("/dev/vfio/devices", F_OK) &&
+	       !access("/dev/iommu", R_OK | W_OK);
+}
+
+static bool vfio_container_probe(const struct iommu_mode *mode)
+{
+	bool available;
+	int fd;
+
+	/*
+	 * Container modes, including the iommufd_compat modes, access devices
+	 * through VFIO groups (CONFIG_VFIO_GROUP). Check for them explicitly,
+	 * since iommufd provides the compat container even without VFIO groups.
+	 */
+	if (access("/sys/class/vfio", F_OK))
+		return false;
+
+	fd = open(mode->container_path, O_RDWR);
+	if (fd < 0)
+		return false;
+
+	available = ioctl(fd, VFIO_CHECK_EXTENSION, mode->iommu_type) > 0;
+	close(fd);
+	return available;
+}
+
+static bool iommu_mode_probe(const struct iommu_mode *mode)
+{
+	if (mode->container_path)
+		return vfio_container_probe(mode);
+
+	return iommufd_probe();
+}
+
+/* Returns the most preferred IOMMU mode that is available, or NULL if none. */
+static const struct iommu_mode *iommu_mode_detect(void)
+{
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(iommu_modes); i++) {
+		if (iommu_mode_probe(&iommu_modes[i]))
+			return &iommu_modes[i];
+	}
+
+	return NULL;
+}
+
+/*
+ * Returns the IOMMU mode with the given name, or if iommu_mode is NULL, the
+ * most preferred IOMMU mode that is available (or NULL if none).
+ */
+static const struct iommu_mode *iommu_mode_lookup(const char *iommu_mode)
 {
 	int i;
 
 	if (!iommu_mode)
-		iommu_mode = default_iommu_mode;
+		return iommu_mode_detect();
 
 	for (i = 0; i < ARRAY_SIZE(iommu_modes); i++) {
 		if (strcmp(iommu_mode, iommu_modes[i].name))
@@ -425,7 +485,8 @@ struct iommu *iommu_init(const char *iommu_mode)
 
 	INIT_LIST_HEAD(&iommu->dma_regions);
 
-	iommu->mode = lookup_iommu_mode(iommu_mode);
+	iommu->mode = iommu_mode_lookup(iommu_mode);
+	VFIO_ASSERT_NOT_NULL(iommu->mode, NO_IOMMU_MODE_AVAILABLE_MSG);
 
 	container_path = iommu->mode->container_path;
 	if (container_path) {
diff --git a/tools/testing/selftests/vfio/vfio_pci_device_test.c b/tools/testing/selftests/vfio/vfio_pci_device_test.c
index 93c11fd5e081..59a7c1f71804 100644
--- a/tools/testing/selftests/vfio/vfio_pci_device_test.c
+++ b/tools/testing/selftests/vfio/vfio_pci_device_test.c
@@ -29,7 +29,7 @@ FIXTURE(vfio_pci_device_test) {
 
 FIXTURE_SETUP(vfio_pci_device_test)
 {
-	self->iommu = iommu_init(default_iommu_mode);
+	self->iommu = iommu_init(NULL);
 	self->device = vfio_pci_device_init(device_bdf, self->iommu);
 }
 
@@ -121,7 +121,7 @@ FIXTURE_VARIANT_ADD(vfio_pci_irq_test, msix) {
 
 FIXTURE_SETUP(vfio_pci_irq_test)
 {
-	self->iommu = iommu_init(default_iommu_mode);
+	self->iommu = iommu_init(NULL);
 	self->device = vfio_pci_device_init(device_bdf, self->iommu);
 }
 
diff --git a/tools/testing/selftests/vfio/vfio_pci_driver_test.c b/tools/testing/selftests/vfio/vfio_pci_driver_test.c
index 9538a3a436a6..f8f99ee6bb1a 100644
--- a/tools/testing/selftests/vfio/vfio_pci_driver_test.c
+++ b/tools/testing/selftests/vfio/vfio_pci_driver_test.c
@@ -255,7 +255,7 @@ static bool device_has_selftests_driver(const char *bdf)
 	struct iommu *iommu;
 	bool has_driver;
 
-	iommu = iommu_init(default_iommu_mode);
+	iommu = iommu_init(NULL);
 	device = vfio_pci_device_init(device_bdf, iommu);
 
 	has_driver = !!device->driver.ops;
-- 
2.56.0.506.g506a7eabc5-goog

[PATCH v3 2/5] vfio: selftests: Skip tests when the IOMMU mode is not available

[David Matlack]

Skip tests when the IOMMU mode they need is not available on the current
system, instead of failing. Add iommu_mode_is_available() to check
whether a given IOMMU mode (or, if NULL, any IOMMU mode) is available,
iommu_mode_unavailable_msg() to explain why a test is skipping, and
SKIP_IF_IOMMU_MODE_UNAVAILABLE() to skip a kselftest harness test if the
mode is not available.

On a kernel without iommufd, for example, all of the iommufd and
iommufd_compat test variants currently fail on an assert in
iommu_init(). Skip them instead, with a message that explains why:

  ok 33 vfio_pci_driver_test.iommufd.send_msi # SKIP IOMMU mode 'iommufd' is not available

Use SKIP_IF_IOMMU_MODE_UNAVAILABLE() before every call to iommu_init()
in the kselftest harness tests, and in vfio_iommufd_setup_test, which
uses iommufd directly. Also skip vfio_pci_driver_test, which calls
iommu_init() from main() to probe the device, if no IOMMU mode is
available.

Skip via the harness's SKIP() rather than calling ksft_exit_skip() from
iommu_init(). ksft_exit_skip() is not aware of the harness, and would
emit a bogus "1..0 # SKIP" plan line in the middle of the TAP output.

Signed-off-by: David Matlack <dmatlack@google.com>
---
 .../vfio/lib/include/libvfio/iommu.h          | 30 ++++++++++++++++++-
 tools/testing/selftests/vfio/lib/iommu.c      | 20 +++++++++++++
 .../vfio/vfio_dma_mapping_mmio_test.c         |  2 ++
 .../selftests/vfio/vfio_dma_mapping_test.c    |  4 +++
 .../selftests/vfio/vfio_iommufd_setup_test.c  |  2 ++
 .../vfio/vfio_pci_device_init_perf_test.c     |  2 ++
 .../selftests/vfio/vfio_pci_device_test.c     |  4 +++
 .../selftests/vfio/vfio_pci_driver_test.c     |  7 +++++
 .../selftests/vfio/vfio_pci_sriov_uapi_test.c |  4 +++
 9 files changed, 74 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/vfio/lib/include/libvfio/iommu.h b/tools/testing/selftests/vfio/lib/include/libvfio/iommu.h
index 53e38130c334..8306ade5f069 100644
--- a/tools/testing/selftests/vfio/lib/include/libvfio/iommu.h
+++ b/tools/testing/selftests/vfio/lib/include/libvfio/iommu.h
@@ -32,11 +32,26 @@ struct iommu {
 
 /*
  * Pass NULL for iommu_mode to use the most preferred IOMMU mode that is
- * available on the current system.
+ * available on the current system. iommu_init() fails the test if the mode
+ * is not available, so check iommu_mode_is_available() first to skip instead.
  */
 struct iommu *iommu_init(const char *iommu_mode);
 void iommu_cleanup(struct iommu *iommu);
 
+/*
+ * Returns true if the given IOMMU mode is available on the current system,
+ * or, if iommu_mode is NULL, if any IOMMU mode is available.
+ */
+bool iommu_mode_is_available(const char *iommu_mode);
+
+/*
+ * Returns a message explaining that the given IOMMU mode is not available,
+ * or, if iommu_mode is NULL, that no IOMMU mode is available, e.g. for tests
+ * to print when skipping. The returned string may be overwritten by the next
+ * call.
+ */
+const char *iommu_mode_unavailable_msg(const char *iommu_mode);
+
 int __iommu_map(struct iommu *iommu, struct dma_region *region);
 
 static inline void iommu_map(struct iommu *iommu, struct dma_region *region)
@@ -81,4 +96,17 @@ FIXTURE_VARIANT_ADD_IOMMU_MODE(iommufd_compat_type1, ##__VA_ARGS__); \
 FIXTURE_VARIANT_ADD_IOMMU_MODE(iommufd_compat_type1v2, ##__VA_ARGS__); \
 FIXTURE_VARIANT_ADD_IOMMU_MODE(iommufd, ##__VA_ARGS__)
 
+/*
+ * Skip the current kselftest harness test if _iommu_mode is not available,
+ * or, if _iommu_mode is NULL, if no IOMMU mode is available. The statement
+ * argument is passed to SKIP(), e.g. return.
+ */
+#define SKIP_IF_IOMMU_MODE_UNAVAILABLE(statement, _iommu_mode) do {		\
+	const char *__iommu_mode = (_iommu_mode);				\
+										\
+	if (!iommu_mode_is_available(__iommu_mode))				\
+		SKIP(statement, "%s",						\
+		     iommu_mode_unavailable_msg(__iommu_mode));			\
+} while (0)
+
 #endif /* SELFTESTS_VFIO_LIB_INCLUDE_LIBVFIO_IOMMU_H */
diff --git a/tools/testing/selftests/vfio/lib/iommu.c b/tools/testing/selftests/vfio/lib/iommu.c
index ecdfbb62ac5e..56696bd5c30d 100644
--- a/tools/testing/selftests/vfio/lib/iommu.c
+++ b/tools/testing/selftests/vfio/lib/iommu.c
@@ -126,6 +126,26 @@ static const struct iommu_mode *iommu_mode_lookup(const char *iommu_mode)
 	VFIO_FAIL("Unrecognized IOMMU mode: %s\n", iommu_mode);
 }
 
+bool iommu_mode_is_available(const char *iommu_mode)
+{
+	if (!iommu_mode)
+		return iommu_mode_detect();
+
+	return iommu_mode_probe(iommu_mode_lookup(iommu_mode));
+}
+
+const char *iommu_mode_unavailable_msg(const char *iommu_mode)
+{
+	static char msg[128];
+
+	if (!iommu_mode)
+		return NO_IOMMU_MODE_AVAILABLE_MSG;
+
+	snprintf(msg, sizeof(msg), "IOMMU mode '%s' is not available",
+		 iommu_mode);
+	return msg;
+}
+
 int __iommu_hva2iova(struct iommu *iommu, void *vaddr, iova_t *iova)
 {
 	struct dma_region *region;
diff --git a/tools/testing/selftests/vfio/vfio_dma_mapping_mmio_test.c b/tools/testing/selftests/vfio/vfio_dma_mapping_mmio_test.c
index d7f25ef77671..add7d3881930 100644
--- a/tools/testing/selftests/vfio/vfio_dma_mapping_mmio_test.c
+++ b/tools/testing/selftests/vfio/vfio_dma_mapping_mmio_test.c
@@ -64,6 +64,8 @@ FIXTURE_VARIANT_ADD_ALL_IOMMU_MODES();
 
 FIXTURE_SETUP(vfio_dma_mapping_mmio_test)
 {
+	SKIP_IF_IOMMU_MODE_UNAVAILABLE(return, variant->iommu_mode);
+
 	self->iommu = iommu_init(variant->iommu_mode);
 	self->device = vfio_pci_device_init(device_bdf, self->iommu);
 	self->iova_allocator = iova_allocator_init(self->iommu);
diff --git a/tools/testing/selftests/vfio/vfio_dma_mapping_test.c b/tools/testing/selftests/vfio/vfio_dma_mapping_test.c
index 7d0de8c79de1..e42170aecf6e 100644
--- a/tools/testing/selftests/vfio/vfio_dma_mapping_test.c
+++ b/tools/testing/selftests/vfio/vfio_dma_mapping_test.c
@@ -119,6 +119,8 @@ FIXTURE_VARIANT_ADD_ALL_IOMMU_MODES(anonymous_hugetlb_1gb, SZ_1G, MAP_HUGETLB |
 
 FIXTURE_SETUP(vfio_dma_mapping_test)
 {
+	SKIP_IF_IOMMU_MODE_UNAVAILABLE(return, variant->iommu_mode);
+
 	self->iommu = iommu_init(variant->iommu_mode);
 	self->device = vfio_pci_device_init(device_bdf, self->iommu);
 	self->iova_allocator = iova_allocator_init(self->iommu);
@@ -228,6 +230,8 @@ FIXTURE_SETUP(vfio_dma_map_limit_test)
 	iova_t last_iova;
 	u32 nranges;
 
+	SKIP_IF_IOMMU_MODE_UNAVAILABLE(return, variant->iommu_mode);
+
 	/*
 	 * Over-allocate mmap by double the size to provide enough backing vaddr
 	 * for overflow tests
diff --git a/tools/testing/selftests/vfio/vfio_iommufd_setup_test.c b/tools/testing/selftests/vfio/vfio_iommufd_setup_test.c
index ec1e5633e080..f4792899fd79 100644
--- a/tools/testing/selftests/vfio/vfio_iommufd_setup_test.c
+++ b/tools/testing/selftests/vfio/vfio_iommufd_setup_test.c
@@ -67,6 +67,8 @@ FIXTURE(vfio_cdev) {
 
 FIXTURE_SETUP(vfio_cdev)
 {
+	SKIP_IF_IOMMU_MODE_UNAVAILABLE(return, MODE_IOMMUFD);
+
 	ASSERT_LE(0, (self->cdev_fd = open(cdev_path, O_RDWR, 0)));
 	ASSERT_LE(0, (self->iommufd = open(iommu_dev_path, O_RDWR, 0)));
 }
diff --git a/tools/testing/selftests/vfio/vfio_pci_device_init_perf_test.c b/tools/testing/selftests/vfio/vfio_pci_device_init_perf_test.c
index e1a54e153cd3..8a98c86096be 100644
--- a/tools/testing/selftests/vfio/vfio_pci_device_init_perf_test.c
+++ b/tools/testing/selftests/vfio/vfio_pci_device_init_perf_test.c
@@ -44,6 +44,8 @@ FIXTURE_SETUP(vfio_pci_device_init_perf_test)
 {
 	int i;
 
+	SKIP_IF_IOMMU_MODE_UNAVAILABLE(return, variant->iommu_mode);
+
 	self->iommu = iommu_init(variant->iommu_mode);
 	self->threads = calloc_assert(nr_devices, sizeof(self->threads[0]));
 	self->thread_args = calloc_assert(nr_devices, sizeof(self->thread_args[0]));
diff --git a/tools/testing/selftests/vfio/vfio_pci_device_test.c b/tools/testing/selftests/vfio/vfio_pci_device_test.c
index 59a7c1f71804..1b1baa1e1c85 100644
--- a/tools/testing/selftests/vfio/vfio_pci_device_test.c
+++ b/tools/testing/selftests/vfio/vfio_pci_device_test.c
@@ -29,6 +29,8 @@ FIXTURE(vfio_pci_device_test) {
 
 FIXTURE_SETUP(vfio_pci_device_test)
 {
+	SKIP_IF_IOMMU_MODE_UNAVAILABLE(return, NULL);
+
 	self->iommu = iommu_init(NULL);
 	self->device = vfio_pci_device_init(device_bdf, self->iommu);
 }
@@ -121,6 +123,8 @@ FIXTURE_VARIANT_ADD(vfio_pci_irq_test, msix) {
 
 FIXTURE_SETUP(vfio_pci_irq_test)
 {
+	SKIP_IF_IOMMU_MODE_UNAVAILABLE(return, NULL);
+
 	self->iommu = iommu_init(NULL);
 	self->device = vfio_pci_device_init(device_bdf, self->iommu);
 }
diff --git a/tools/testing/selftests/vfio/vfio_pci_driver_test.c b/tools/testing/selftests/vfio/vfio_pci_driver_test.c
index f8f99ee6bb1a..56178c25c2fd 100644
--- a/tools/testing/selftests/vfio/vfio_pci_driver_test.c
+++ b/tools/testing/selftests/vfio/vfio_pci_driver_test.c
@@ -80,6 +80,8 @@ FIXTURE_SETUP(vfio_pci_driver_test)
 {
 	struct vfio_pci_driver *driver;
 
+	SKIP_IF_IOMMU_MODE_UNAVAILABLE(return, variant->iommu_mode);
+
 	self->iommu = iommu_init(variant->iommu_mode);
 	self->device = vfio_pci_device_init(device_bdf, self->iommu);
 	self->iova_allocator = iova_allocator_init(self->iommu);
@@ -270,6 +272,11 @@ int main(int argc, char *argv[])
 {
 	device_bdf = vfio_selftests_get_bdf(&argc, argv);
 
+	if (!iommu_mode_is_available(NULL)) {
+		fprintf(stderr, "%s\n", iommu_mode_unavailable_msg(NULL));
+		return KSFT_SKIP;
+	}
+
 	if (!device_has_selftests_driver(device_bdf)) {
 		fprintf(stderr, "No driver found for device %s\n", device_bdf);
 		return KSFT_SKIP;
diff --git a/tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c b/tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c
index f3a018b8699d..32a1e651c815 100644
--- a/tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c
+++ b/tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c
@@ -90,6 +90,8 @@ FIXTURE_VARIANT_ADD_ALL_IOMMU_MODES(null_uuid, NULL);
 
 FIXTURE_SETUP(vfio_pci_sriov_uapi_test)
 {
+	SKIP_IF_IOMMU_MODE_UNAVAILABLE(return, variant->iommu_mode);
+
 	self->iommu = iommu_init(variant->iommu_mode);
 
 	self->pf_token = UUID_1;
@@ -163,6 +165,8 @@ TEST(failed_second_open_does_not_clobber_token)
 	struct iommu *iommu;
 	int ret;
 
+	SKIP_IF_IOMMU_MODE_UNAVAILABLE(return, MODE_IOMMUFD);
+
 	iommu = iommu_init(MODE_IOMMUFD);
 
 	/* Create and bind PF using UUID_1 */
-- 
2.56.0.506.g506a7eabc5-goog

[PATCH v3 3/5] vfio: selftests: Print the IOMMU modes when a requested mode can't be used

[David Matlack]

Print the names of all known IOMMU modes when passed an unrecognized
IOMMU mode, and the names of the available IOMMU modes when a test skips
because the IOMMU mode it requested is not available. If no IOMMU mode
is available at all, print the generic "No IOMMU mode is available"
message instead. Expose both lists via iommu_mode_names_all() and
iommu_mode_names_available() so that tests can also print them, e.g. in
their help output.

irq_test lets users choose the IOMMU mode with -t, so a typo or
unavailable mode should tell them which modes are valid.

Suggested-by: Sean Christopherson <seanjc@google.com>
Link: https://lore.kernel.org/kvm/auit0kVkAaWcNUvk@google.com/
Signed-off-by: David Matlack <dmatlack@google.com>
---
 .../vfio/lib/include/libvfio/iommu.h          |  9 ++++
 tools/testing/selftests/vfio/lib/iommu.c      | 44 ++++++++++++++++---
 2 files changed, 48 insertions(+), 5 deletions(-)

diff --git a/tools/testing/selftests/vfio/lib/include/libvfio/iommu.h b/tools/testing/selftests/vfio/lib/include/libvfio/iommu.h
index 8306ade5f069..309b231eb0d8 100644
--- a/tools/testing/selftests/vfio/lib/include/libvfio/iommu.h
+++ b/tools/testing/selftests/vfio/lib/include/libvfio/iommu.h
@@ -52,6 +52,15 @@ bool iommu_mode_is_available(const char *iommu_mode);
  */
 const char *iommu_mode_unavailable_msg(const char *iommu_mode);
 
+/* Returns a comma-separated list of the names of all IOMMU modes. */
+const char *iommu_mode_names_all(void);
+
+/*
+ * Returns a comma-separated list of the names of the IOMMU modes that are
+ * available on the current system, or "none" if no IOMMU mode is available.
+ */
+const char *iommu_mode_names_available(void);
+
 int __iommu_map(struct iommu *iommu, struct dma_region *region);
 
 static inline void iommu_map(struct iommu *iommu, struct dma_region *region)
diff --git a/tools/testing/selftests/vfio/lib/iommu.c b/tools/testing/selftests/vfio/lib/iommu.c
index 56696bd5c30d..bde53bd55208 100644
--- a/tools/testing/selftests/vfio/lib/iommu.c
+++ b/tools/testing/selftests/vfio/lib/iommu.c
@@ -105,6 +105,38 @@ static const struct iommu_mode *iommu_mode_detect(void)
 	return NULL;
 }
 
+static const char *iommu_mode_names(bool include_unavailable)
+{
+	static char all_names[256];
+	static char avl_names[sizeof(all_names)];
+	char *names = include_unavailable ? all_names : avl_names;
+	int i, len = 0;
+
+	if (names[0])
+		return names;
+
+	for (i = 0; i < ARRAY_SIZE(iommu_modes); i++) {
+		if (!include_unavailable && !iommu_mode_probe(&iommu_modes[i]))
+			continue;
+
+		len += snprintf(names + len, sizeof(all_names) - len, "%s%s",
+				names[0] ? ", " : "", iommu_modes[i].name);
+		VFIO_ASSERT_LT(len, (int)sizeof(all_names));
+	}
+
+	return names[0] ? names : "none";
+}
+
+const char *iommu_mode_names_all(void)
+{
+	return iommu_mode_names(true);
+}
+
+const char *iommu_mode_names_available(void)
+{
+	return iommu_mode_names(false);
+}
+
 /*
  * Returns the IOMMU mode with the given name, or if iommu_mode is NULL, the
  * most preferred IOMMU mode that is available (or NULL if none).
@@ -123,7 +155,8 @@ static const struct iommu_mode *iommu_mode_lookup(const char *iommu_mode)
 		return &iommu_modes[i];
 	}
 
-	VFIO_FAIL("Unrecognized IOMMU mode: %s\n", iommu_mode);
+	VFIO_FAIL("Unrecognized IOMMU mode '%s'.  Known modes: %s.\n",
+		  iommu_mode, iommu_mode_names_all());
 }
 
 bool iommu_mode_is_available(const char *iommu_mode)
@@ -136,13 +169,14 @@ bool iommu_mode_is_available(const char *iommu_mode)
 
 const char *iommu_mode_unavailable_msg(const char *iommu_mode)
 {
-	static char msg[128];
+	static char msg[512];
 
-	if (!iommu_mode)
+	if (!iommu_mode || !iommu_mode_detect())
 		return NO_IOMMU_MODE_AVAILABLE_MSG;
 
-	snprintf(msg, sizeof(msg), "IOMMU mode '%s' is not available",
-		 iommu_mode);
+	snprintf(msg, sizeof(msg),
+		 "IOMMU mode '%s' is not available.  Available modes: %s.",
+		 iommu_mode, iommu_mode_names_available());
 	return msg;
 }
 
-- 
2.56.0.506.g506a7eabc5-goog

[PATCH v3 4/5] KVM: selftests: Use VFIO selftests library to pick IOMMU mode in irq_test

[David Matlack]

Use the VFIO selftests library to pick the IOMMU mode in irq_test and
drop probe_iommu_type(). Pass NULL to iommu_init() when the user does
not specify a mode with -t, and skip the test if the requested mode is
not available, or if no mode is available at all.

probe_iommu_type() picks iommufd whenever /dev/iommu can be opened, even
if VFIO device cdevs are not available (CONFIG_VFIO_DEVICE_CDEV=n), in
which case the test fails rather than using the VFIO container. The VFIO
selftests library accounts for this when picking a mode, so there is no
need for irq_test to probe for a mode itself.

Note, this changes irq_test's default container mode from
vfio_type1_iommu to vfio_type1v2_iommu, which the VFIO selftests library
prefers. This only matters when iommufd is not available.

Opportunistically rename iommu_type to iommu_mode to match the
terminology used by the VFIO selftests library.

Suggested-by: Sean Christopherson <seanjc@google.com>
Link: https://lore.kernel.org/kvm/aiIG2L_KdPwuxYA1@google.com/
Signed-off-by: David Matlack <dmatlack@google.com>
---
 tools/testing/selftests/kvm/irq_test.c | 26 +++++---------------------
 1 file changed, 5 insertions(+), 21 deletions(-)

diff --git a/tools/testing/selftests/kvm/irq_test.c b/tools/testing/selftests/kvm/irq_test.c
index 791e576c334c..0d6ca1a052c4 100644
--- a/tools/testing/selftests/kvm/irq_test.c
+++ b/tools/testing/selftests/kvm/irq_test.c
@@ -148,22 +148,6 @@ static void kvm_set_empty_gsi_routing(struct kvm_vm *vm)
 	vm_ioctl(vm, KVM_SET_GSI_ROUTING, &routing);
 }
 
-static const char *probe_iommu_type(void)
-{
-	int io_fd;
-
-	io_fd = open("/dev/iommu", O_RDONLY);
-	if (io_fd >= 0) {
-		close(io_fd);
-		return MODE_IOMMUFD;
-	}
-
-	io_fd = __open_path_or_exit("/dev/vfio/vfio", O_RDONLY,
-				    "Is VFIO (or IOMMUFD) loaded and enabled?");
-	close(io_fd);
-	return MODE_VFIO_TYPE1_IOMMU;
-}
-
 static void help(const char *name)
 {
 	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-m] [-n] [-t iommu_type] [-v nr_vcpus] [-x]\n", name);
@@ -205,7 +189,7 @@ int main(int argc, char **argv)
 	int nr_irqs = 1000, nr_vcpus = 1;
 	bool set_empty_routing = false;
 	const char *device_bdf = NULL;
-	const char *iommu_type = NULL;
+	const char *iommu_mode = NULL;
 	int i, j, c, msix, eventfd;
 	bool migrate_vcpus = false;
 	cpu_set_t available_cpus;
@@ -235,7 +219,7 @@ int main(int argc, char **argv)
 			use_nmi = true;
 			break;
 		case 't':
-			iommu_type = optarg;
+			iommu_mode = optarg;
 			break;
 		case 'v':
 			nr_vcpus = atoi_positive("Number of vCPUS", optarg);
@@ -266,9 +250,9 @@ int main(int argc, char **argv)
 	}
 
 	if (device_bdf) {
-		if (!iommu_type)
-			iommu_type = probe_iommu_type();
-		iommu = iommu_init(iommu_type);
+		__TEST_REQUIRE(iommu_mode_is_available(iommu_mode), "%s",
+			       iommu_mode_unavailable_msg(iommu_mode));
+		iommu = iommu_init(iommu_mode);
 		device = vfio_pci_device_init(device_bdf, iommu);
 		msix = vfio_setup_msi(device);
 		irq = vfio_msix_to_host_irq(device_bdf, msix);
-- 
2.56.0.506.g506a7eabc5-goog

[PATCH v3 5/5] KVM: selftests: List the IOMMU modes in irq_test's help output

[David Matlack]

List the names of all known IOMMU modes, and of the IOMMU modes that are
available on the current system, in the help output for irq_test's -t
option.

The help output currently lists only vfio_type1_iommu and iommufd, even
though the VFIO selftests library supports five IOMMU modes, and the
list would go stale whenever a mode is added. List the available modes
separately so that users can see which modes work on the current system.

Opportunistically refer to the -t option's argument as iommu_mode rather
than iommu_type in the usage line, to match the help text.

Signed-off-by: David Matlack <dmatlack@google.com>
---
 tools/testing/selftests/kvm/irq_test.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/kvm/irq_test.c b/tools/testing/selftests/kvm/irq_test.c
index 0d6ca1a052c4..9076271cc0dd 100644
--- a/tools/testing/selftests/kvm/irq_test.c
+++ b/tools/testing/selftests/kvm/irq_test.c
@@ -150,7 +150,7 @@ static void kvm_set_empty_gsi_routing(struct kvm_vm *vm)
 
 static void help(const char *name)
 {
-	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-m] [-n] [-t iommu_type] [-v nr_vcpus] [-x]\n", name);
+	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-m] [-n] [-t iommu_mode] [-v nr_vcpus] [-x]\n", name);
 	printf("\n");
 	printf("Tests KVM interrupt routing and delivery via irqfd.\n");
 	printf("-a	Affine the device's host IRQ to a random physical CPU\n");
@@ -159,7 +159,9 @@ static void help(const char *name)
 	printf("-i	The number of IRQs to generate during the test\n");
 	printf("-m	Pin target vCPU to random physical CPU before triggering interrupt\n");
 	printf("-n	Deliver 50 percent of IRQs as non-maskable interrupts\n");
-	printf("-t	Override the IOMMU type to use (vfio_type1_iommu or iommufd)\n");
+	printf("-t	Override the IOMMU mode to use\n");
+	printf("	Options: %s\n", iommu_mode_names_all());
+	printf("	Available: %s\n", iommu_mode_names_available());
 	printf("-v	Number of vCPUS to run\n");
 	printf("-x	Use xAPIC mode instead of x2APIC mode in the guest\n");
 	printf("\n");
-- 
2.56.0.506.g506a7eabc5-goog