From: David Matlack <dmatlack@google.com>
To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-kselftest@vger.kernel.org, stable@vger.kernel.org
Cc: Abhishek Sahu <abhsahu@nvidia.com>,
Alex Williamson <alex@shazbot.org>,
David Matlack <dmatlack@google.com>,
Raghavendra Rao Ananta <rananta@google.com>,
Shuah Khan <shuah@kernel.org>, Vipin Sharma <vipinsh@google.com>
Subject: [PATCH v2 0/8] vfio/pci: Fix power state race with SR-IOV enablement
Date: Thu, 19 Nov 2026 00:01:42 +0000 [thread overview]
Message-ID: <20261119000151.442543-1-dmatlack@google.com> (raw)
This series can be found on GitHub:
https://github.com/dmatlack/linux/tree/vfio/set_power_state/v2
This series fixes a race between closing a vfio-pci device and enabling
SR-IOV on it, both of which can transition the device from D3hot to D0
without being serialized. On devices with NoSoftRst-, this can restore
the PF's BARs after VFs have been enabled (triggering a WARN in
pci_iov_update_resource()), or double-free vdev->pm_save.
This series also adds a selftest that exercises VFIO's handling of
power state transitions requested by userspace through the PM
capability, including a test case that reproduces the race.
This series applies on top of vfio/next (vfio-v7.4-rc1).
Series Overview
---------------
This series implements the following:
1. Hold memory_lock for all calls to vfio_pci_set_power_state() and
assert that it is held, which fixes the race (patch 1).
2. Annotate the intentional lockless accesses to vdev->sriov_active
with READ_ONCE() and WRITE_ONCE() (patch 2).
3. Skip vfio_pci_sriov_uapi_test instead of failing when SR-IOV
cannot be configured on the PF, e.g. because vfio-pci's
enable_sriov module parameter is not set (patches 3-4).
4. Add library helpers to find a PCI capability, read
/sys/kernel/warn_count, and print kernel warnings from /dev/kmsg
(patches 5-7).
5. Add vfio_pci_power_state_test, which covers closing a device in
D3hot and reopening it, enabling SR-IOV while the PF is in D3hot,
and racing a close in D3hot against enabling SR-IOV. The test fails
if /sys/kernel/warn_count changes during a test case, and prints
the warnings logged to /dev/kmsg to help debug the failure
(patch 8).
Testing
-------
All patches were build-tested individually. The series was tested on
x86_64 in a QEMU VM with an emulated Intel 82576 NIC (igb), which has
NoSoftRst- and supports SR-IOV, bound to vfio-pci:
- Without patch 1 (KASAN kernel), close_in_d3hot_races_sriov_enable
fails, hitting the pci_iov_update_resource() WARN on 9 or 10 of the
test's 10 iterations.
- With the full series (KASAN and lockdep kernels), all VFIO
selftests pass with no warnings, KASAN reports, or lockdep splats.
To run the new test (the SR-IOV test cases are skipped unless
vfio_pci.enable_sriov=1):
# tools/testing/selftests/vfio/scripts/setup.sh 0000:01:00.0
# tools/testing/selftests/vfio/scripts/run.sh \
tools/testing/selftests/vfio/vfio_pci_power_state_test
Changelog
---------
v2:
- Rename close_in_d3hot to reopen_after_close_in_d3hot and clarify that
it does not verify the D0 transition during close, since that is not
reliably observable from userspace (Alex)
- Print the kernel warnings logged to /dev/kmsg when the warn_count
check fails, so the failure can be debugged from the test log (Alex)
- Drop the claim that memory_lock protects the sriov_active check from
the comment in vfio_pci_set_power_state(), since it is cleared
without memory_lock
- Skip the SR-IOV tests, including the existing
vfio_pci_sriov_uapi_test, when SR-IOV cannot be configured on the PF
(e.g. vfio_pci.enable_sriov=N) instead of failing
- Use kernel-doc for the new library helpers
- Clean up commit messages, comments, and the cover letter
v1: https://lore.kernel.org/kvm/20261029203324.1637772-1-dmatlack@google.com/
David Matlack (8):
vfio/pci: Hold memory_lock for all calls to vfio_pci_set_power_state()
vfio/pci: Annotate lockless accesses to vdev->sriov_active
vfio: selftests: Add __sysfs_sriov_numvfs_set()
vfio: selftests: Skip vfio_pci_sriov_uapi_test if SR-IOV can't be
enabled
vfio: selftests: Add vfio_pci_find_capability()
vfio: selftests: Add sysfs_kernel_warn_count_get()
vfio: selftests: Add helpers to print kernel warnings from /dev/kmsg
vfio: selftests: Add test for device power state transitions
drivers/vfio/pci/vfio_pci_core.c | 16 +-
tools/testing/selftests/vfio/Makefile | 1 +
.../selftests/vfio/lib/include/libvfio.h | 1 +
.../selftests/vfio/lib/include/libvfio/kmsg.h | 8 +
.../vfio/lib/include/libvfio/sysfs.h | 2 +
.../lib/include/libvfio/vfio_pci_device.h | 2 +
tools/testing/selftests/vfio/lib/kmsg.c | 134 +++++++++
tools/testing/selftests/vfio/lib/libvfio.mk | 1 +
tools/testing/selftests/vfio/lib/sysfs.c | 66 ++++-
.../selftests/vfio/lib/vfio_pci_device.c | 36 +++
.../vfio/vfio_pci_power_state_test.c | 256 ++++++++++++++++++
.../selftests/vfio/vfio_pci_sriov_uapi_test.c | 8 +-
12 files changed, 512 insertions(+), 19 deletions(-)
create mode 100644 tools/testing/selftests/vfio/lib/include/libvfio/kmsg.h
create mode 100644 tools/testing/selftests/vfio/lib/kmsg.c
create mode 100644 tools/testing/selftests/vfio/vfio_pci_power_state_test.c
base-commit: 076d7e989cb69f6ded7d60f2a080eb909930061b
--
2.56.1.708.g77570a6a1d-goog
next reply other threads:[~2026-11-19 0:01 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-19 0:01 David Matlack [this message]
2026-11-19 0:01 ` [PATCH v2 1/8] vfio/pci: Hold memory_lock for all calls to vfio_pci_set_power_state() David Matlack
2026-11-19 0:01 ` [PATCH v2 2/8] vfio/pci: Annotate lockless accesses to vdev->sriov_active David Matlack
2026-11-19 0:01 ` [PATCH v2 3/8] vfio: selftests: Add __sysfs_sriov_numvfs_set() David Matlack
2026-11-19 0:01 ` [PATCH v2 4/8] vfio: selftests: Skip vfio_pci_sriov_uapi_test if SR-IOV can't be enabled David Matlack
2026-11-19 0:01 ` [PATCH v2 5/8] vfio: selftests: Add vfio_pci_find_capability() David Matlack
2026-11-19 0:01 ` [PATCH v2 6/8] vfio: selftests: Add sysfs_kernel_warn_count_get() David Matlack
2026-11-19 0:01 ` [PATCH v2 7/8] vfio: selftests: Add helpers to print kernel warnings from /dev/kmsg David Matlack
2026-11-19 0:01 ` [PATCH v2 8/8] vfio: selftests: Add test for device power state transitions David Matlack
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261119000151.442543-1-dmatlack@google.com \
--to=dmatlack@google.com \
--cc=abhsahu@nvidia.com \
--cc=alex@shazbot.org \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=rananta@google.com \
--cc=shuah@kernel.org \
--cc=stable@vger.kernel.org \
--cc=vipinsh@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®