[PATCH v6 00/11] Add Rust PCI SR-IOV support

[PATCH v6 00/11] Add Rust PCI SR-IOV support

[Simon Song]

Rust PCI drivers need to configure SR-IOV and let VF drivers borrow
selected PF-owned data. This series provides those operations and a
registration that keeps the shared data available until VF removal
completes.

This builds on Peter Colberg's Rust PCI SR-IOV series [1] and Danilo
Krummrich's typed PF registration design [2]. The PF/VF sample demonstrates
the interface using QEMU's 82576 emulation.

Drivers implement separate sriov_enable() and sriov_disable() callbacks,
each receiving pinned driver data and a token for the requested operation.
Enabling VFs returns a guard that disables them if later setup fails.
Returning the guard successfully leaves the VFs enabled, with
VfRegistration owning their final teardown. Drivers that do not share
data with VFs can register ().

Changes since v5 [3]:
- Stub the SR-IOV API when CONFIG_PCI_IOV is disabled instead of
  cfg-gating it, so drivers need no configuration guards, following
  Danilo's suggested diff [4].
- Add SriovEnable::enable_all() (patch 9), following Danilo's
  suggestion [4].
- Split the drvdata_borrow() safety documentation fix into its own
  patch (patch 5), and explain why driver data is dropped before
  remove() returns, following Danilo's and Alexandre's review.
- Use NonZero<u16> for the requested VF count, const_assert!() and
  #[optional] for the paired SR-IOV callbacks, and document the errors
  returned by SriovEnable::enable(), following Alexandre's review.
- Build the sample as a single module with the PF and VF drivers in
  separate files, and simplify it, following Danilo's and Alexandre's
  review.
- Collect Alexandre's Reviewed-by (patches 1-4, 8 and 10) and Bjorn's
  Acked-by for include/linux/pci.h (patch 7).
- Dropped Alexandre's Reviewed-by from patch 7 after adding
  CONFIG_PCI_IOV=n support; the revised patch needs renewed review.

[1]
https://lore.kernel.org/rust-for-linux/20260303-rust-pci-sriov-v3-0-4443c35f0c88@redhat.com/
[2]
https://lore.kernel.org/nova-gpu/DLCRZLO06SIO.LS7TWQXIPZSQ@kernel.org/
[3]
https://lore.kernel.org/all/20261030171230.196717-1-xinmengs@nvidia.com/T/
[4]
https://lore.kernel.org/nova-gpu/DMRNF7IZQ1YI.1MRPH47QJC9XC@kernel.org/

John Hubbard (1):
  rust: pci: add is_virtfn(), to check for VFs

Peter Colberg (5):
  rust: pci: add internal SR-IOV enable and disable helpers
  rust: pci: add vtable attribute to pci::Driver trait
  rust: pci: add num_vf(), to return number of VFs
  rust: pci: add SR-IOV enable and disable callbacks
  samples: rust: add a Rust SR-IOV PF and VF driver sample

Simon Song (1):
  rust: pci: add SriovEnable::enable_all(), to enable all requested VFs

Zhi Wang (4):
  rust: device: clarify the drvdata_borrow() safety requirement
  rust: pci: drop driver data before remove returns
  rust: pci: add typed SR-IOV PF registration data
  rust: pci: add SR-IOV enable and disable tokens

 MAINTAINERS                           |   1 +
 drivers/gpu/nova-core/driver.rs       |   1 +
 include/linux/pci.h                   |   7 +
 rust/helpers/pci.c                    |  16 +
 rust/kernel/device.rs                 |   4 +-
 rust/kernel/pci.rs                    | 116 +++++++
 rust/kernel/pci/iov.rs                | 468 ++++++++++++++++++++++++++
 samples/rust/Kconfig                  |  11 +
 samples/rust/Makefile                 |   1 +
 samples/rust/rust_dma.rs              |   1 +
 samples/rust/rust_driver_auxiliary.rs |   1 +
 samples/rust/rust_driver_pci.rs       |   1 +
 samples/rust/rust_driver_sriov.rs     | 178 ++++++++++
 samples/rust/rust_driver_sriov_vf.rs  |  49 +++
 14 files changed, 853 insertions(+), 2 deletions(-)
 create mode 100644 rust/kernel/pci/iov.rs
 create mode 100644 samples/rust/rust_driver_sriov.rs
 create mode 100644 samples/rust/rust_driver_sriov_vf.rs


base-commit: 6e557330b874b0e1fdd03e8fe4c438f36c93eefa
-- 
2.43.7

[PATCH v6 01/11] rust: pci: add internal SR-IOV enable and disable helpers

[Simon Song]

From: Peter Colberg <peter@colberg.org>

Wrap pci_enable_sriov() and pci_disable_sriov() on Device<CoreInternal>
for use by the SR-IOV callback tokens and their rollback guard.
Keep these operations internal so drivers can only enable or disable
VFs through the callback-scoped API introduced later in this series.

Stub both helpers when CONFIG_PCI_IOV is disabled, so the SR-IOV API
builds regardless and drivers need no configuration guards.

Suggested-by: Danilo Krummrich <dakr@kernel.org>
Signed-off-by: Peter Colberg <peter@colberg.org>
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Simon Song <xinmengs@nvidia.com>
---
 rust/helpers/pci.c     | 11 ++++++++++
 rust/kernel/pci.rs     |  1 +
 rust/kernel/pci/iov.rs | 48 ++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 60 insertions(+)
 create mode 100644 rust/kernel/pci/iov.rs

diff --git a/rust/helpers/pci.c b/rust/helpers/pci.c
index a8540f90b13c..a87b5a77efef 100644
--- a/rust/helpers/pci.c
+++ b/rust/helpers/pci.c
@@ -41,6 +41,17 @@ __rust_helper u32 rust_helper_pci_ext_cap_next(u32 header)
 }
 
 #ifndef CONFIG_PCI_IOV
+__rust_helper int rust_helper_pci_enable_sriov(struct pci_dev *pdev,
+					       int nr_virtfn)
+{
+	return pci_enable_sriov(pdev, nr_virtfn);
+}
+
+__rust_helper void rust_helper_pci_disable_sriov(struct pci_dev *pdev)
+{
+	pci_disable_sriov(pdev);
+}
+
 __rust_helper unsigned int
 rust_helper_pci_sriov_get_totalvfs(struct pci_dev *pdev)
 {
diff --git a/rust/kernel/pci.rs b/rust/kernel/pci.rs
index 769d01d23429..8a4af261d1d4 100644
--- a/rust/kernel/pci.rs
+++ b/rust/kernel/pci.rs
@@ -36,6 +36,7 @@
 mod cap;
 mod id;
 mod io;
+mod iov;
 mod irq;
 
 pub use self::cap::{
diff --git a/rust/kernel/pci/iov.rs b/rust/kernel/pci/iov.rs
new file mode 100644
index 000000000000..18e42b2d3965
--- /dev/null
+++ b/rust/kernel/pci/iov.rs
@@ -0,0 +1,48 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Abstractions for PCI Single Root I/O Virtualization (SR-IOV) drivers.
+
+use super::Device;
+
+use crate::{
+    bindings,
+    device,
+    error::to_result,
+    prelude::*, //
+};
+
+impl Device<device::CoreInternal<'_>> {
+    /// Enable the Single Root I/O Virtualization (SR-IOV) capability for this device,
+    /// where `nr_virtfn` is number of Virtual Functions (VF) to enable.
+    #[inline]
+    #[expect(dead_code)]
+    pub(crate) fn enable_sriov(&self, nr_virtfn: u16) -> Result {
+        // SAFETY:
+        // - `self.as_raw` returns a valid pointer to a `struct pci_dev`.
+        //
+        // - `pci_enable_sriov()` checks that the enable operation is valid:
+        //   - the device is a Physical Function (PF),
+        //   - SR-IOV is currently disabled, and
+        //   - `nr_virtfn` does not exceed the total number of supported VFs.
+        //
+        // - The `CoreInternal` device context inherits from the `Bound` device context,
+        //   which guarantees that the PF device is bound to a driver.
+        to_result(unsafe { bindings::pci_enable_sriov(self.as_raw(), nr_virtfn.into()) })
+    }
+
+    /// Disable the Single Root I/O Virtualization (SR-IOV) capability for this device.
+    #[inline]
+    #[expect(dead_code)]
+    pub(crate) fn disable_sriov(&self) {
+        // SAFETY:
+        // - `self.as_raw` returns a valid pointer to a `struct pci_dev`.
+        //
+        // - `pci_disable_sriov()` checks that the disable operation is valid:
+        //   - the device is a Physical Function (PF), and
+        //   - SR-IOV is currently enabled.
+        //
+        // - The `CoreInternal` device context inherits from the `Bound` device context,
+        //   which guarantees that the PF device is bound to a driver.
+        unsafe { bindings::pci_disable_sriov(self.as_raw()) };
+    }
+}
-- 
2.43.7

[PATCH v6 02/11] rust: pci: add vtable attribute to pci::Driver trait

[Simon Song]

From: Peter Colberg <peter@colberg.org>

Add the #[vtable] attribute to pci::Driver and its implementations to
support the optional sriov_enable() and sriov_disable() callbacks
introduced later in this series.

Suggested-by: Danilo Krummrich <dakr@kernel.org>
Signed-off-by: Peter Colberg <peter@colberg.org>
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Simon Song <xinmengs@nvidia.com>
---
 drivers/gpu/nova-core/driver.rs       | 1 +
 rust/kernel/pci.rs                    | 2 ++
 samples/rust/rust_dma.rs              | 1 +
 samples/rust/rust_driver_auxiliary.rs | 1 +
 samples/rust/rust_driver_pci.rs       | 1 +
 5 files changed, 6 insertions(+)

diff --git a/drivers/gpu/nova-core/driver.rs b/drivers/gpu/nova-core/driver.rs
index 560a70c4f5a7..65897c303e7f 100644
--- a/drivers/gpu/nova-core/driver.rs
+++ b/drivers/gpu/nova-core/driver.rs
@@ -91,6 +91,7 @@ pub(crate) fn bar1_resource_index(pdev: &pci::Device<Bound>) -> Result<u32> {
     ]
 );
 
+#[vtable]
 impl pci::Driver for NovaCoreDriver {
     type IdInfo = ();
     type Data<'bound> = NovaCore<'bound>;
diff --git a/rust/kernel/pci.rs b/rust/kernel/pci.rs
index 8a4af261d1d4..4d32b55779fd 100644
--- a/rust/kernel/pci.rs
+++ b/rust/kernel/pci.rs
@@ -282,6 +282,7 @@ macro_rules! pci_device_table {
 ///     ]
 /// );
 ///
+/// #[vtable]
 /// impl pci::Driver for MyDriver {
 ///     type IdInfo = ();
 ///     type Data<'bound> = Self;
@@ -297,6 +298,7 @@ macro_rules! pci_device_table {
 ///```
 /// Drivers must implement this trait in order to get a PCI driver registered. Please refer to the
 /// `Adapter` documentation for an example.
+#[vtable]
 pub trait Driver {
     /// The type holding information about each device id supported by the driver.
     // TODO: Use `associated_type_defaults` once stabilized:
diff --git a/samples/rust/rust_dma.rs b/samples/rust/rust_dma.rs
index ffb693544673..804d55a63005 100644
--- a/samples/rust/rust_dma.rs
+++ b/samples/rust/rust_dma.rs
@@ -69,6 +69,7 @@ unsafe impl kernel::transmute::FromBytes for MyStruct {}
     [(pci::DeviceId::from_id(pci::Vendor::REDHAT, 0x5), ())]
 );
 
+#[vtable]
 impl pci::Driver for DmaSampleDriver {
     type IdInfo = ();
     type Data<'bound> = DmaSampleData<'bound>;
diff --git a/samples/rust/rust_driver_auxiliary.rs b/samples/rust/rust_driver_auxiliary.rs
index 0bee16faecc6..6cff9bf8c80c 100644
--- a/samples/rust/rust_driver_auxiliary.rs
+++ b/samples/rust/rust_driver_auxiliary.rs
@@ -90,6 +90,7 @@ struct ParentData<'bound> {
     [(pci::DeviceId::from_id(pci::Vendor::REDHAT, 0x5), ())]
 );
 
+#[vtable]
 impl pci::Driver for ParentDriver {
     type IdInfo = ();
     type Data<'bound> = ParentData<'bound>;
diff --git a/samples/rust/rust_driver_pci.rs b/samples/rust/rust_driver_pci.rs
index 13b035a95756..50c8709739c6 100644
--- a/samples/rust/rust_driver_pci.rs
+++ b/samples/rust/rust_driver_pci.rs
@@ -139,6 +139,7 @@ fn config_space(pdev: &pci::Device<Bound>) {
     }
 }
 
+#[vtable]
 impl pci::Driver for SampleDriver {
     type IdInfo = TestIndex;
     type Data<'bound> = SampleDriverData<'bound>;
-- 
2.43.7

[PATCH v6 03/11] rust: pci: add is_virtfn(), to check for VFs

[Simon Song]

From: John Hubbard <jhubbard@nvidia.com>

Add a method to check if a PCI device is a Virtual Function (VF) created
through Single Root I/O Virtualization (SR-IOV).

Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Alistair Popple <apopple@nvidia.com>
Reviewed-by: Joel Fernandes <joelagnelf@nvidia.com>
Signed-off-by: Peter Colberg <peter@colberg.org>
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Simon Song <xinmengs@nvidia.com>
---
 rust/kernel/pci.rs | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/rust/kernel/pci.rs b/rust/kernel/pci.rs
index 4d32b55779fd..9e31b9bb4d0d 100644
--- a/rust/kernel/pci.rs
+++ b/rust/kernel/pci.rs
@@ -362,6 +362,13 @@ fn as_raw(&self) -> *mut bindings::pci_dev {
 }
 
 impl Device {
+    /// Returns `true` if this device is a Virtual Function (VF).
+    #[inline]
+    pub fn is_virtfn(&self) -> bool {
+        // SAFETY: `self.as_raw` is a valid pointer to a `struct pci_dev`.
+        unsafe { (*self.as_raw()).is_virtfn() != 0 }
+    }
+
     /// Returns the PCI vendor ID as [`Vendor`].
     ///
     /// # Examples
-- 
2.43.7

[PATCH v6 04/11] rust: pci: add num_vf(), to return number of VFs

[Simon Song]

From: Peter Colberg <peter@colberg.org>

Add a method to return the number of Virtual Functions (VF) enabled for
a Physical Function (PF).

Expose the query in the PCI core callback context and return u16,
matching the type of the VF count stored by the PCI core.

Signed-off-by: Peter Colberg <peter@colberg.org>
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Simon Song <xinmengs@nvidia.com>
---
 rust/helpers/pci.c     |  5 +++++
 rust/kernel/pci/iov.rs | 13 +++++++++++++
 2 files changed, 18 insertions(+)

diff --git a/rust/helpers/pci.c b/rust/helpers/pci.c
index a87b5a77efef..57c28ddeeb9b 100644
--- a/rust/helpers/pci.c
+++ b/rust/helpers/pci.c
@@ -52,6 +52,11 @@ __rust_helper void rust_helper_pci_disable_sriov(struct pci_dev *pdev)
 	pci_disable_sriov(pdev);
 }
 
+__rust_helper int rust_helper_pci_num_vf(struct pci_dev *pdev)
+{
+	return pci_num_vf(pdev);
+}
+
 __rust_helper unsigned int
 rust_helper_pci_sriov_get_totalvfs(struct pci_dev *pdev)
 {
diff --git a/rust/kernel/pci/iov.rs b/rust/kernel/pci/iov.rs
index 18e42b2d3965..78e279169be1 100644
--- a/rust/kernel/pci/iov.rs
+++ b/rust/kernel/pci/iov.rs
@@ -11,6 +11,19 @@
     prelude::*, //
 };
 
+impl Device<device::Core<'_>> {
+    /// Returns the number of Virtual Functions (VF) enabled for a Physical Function (PF).
+    ///
+    /// Returns zero when `CONFIG_PCI_IOV` is disabled.
+    #[inline]
+    pub fn num_vf(&self) -> u16 {
+        // SAFETY: `self.as_raw()` is valid and this call runs in the PCI core callback context.
+        unsafe { bindings::pci_num_vf(self.as_raw()) }
+            .try_into()
+            .expect("pdev->sriov->num_VFs is u16")
+    }
+}
+
 impl Device<device::CoreInternal<'_>> {
     /// Enable the Single Root I/O Virtualization (SR-IOV) capability for this device,
     /// where `nr_virtfn` is number of Virtual Functions (VF) to enable.
-- 
2.43.7

[PATCH v6 05/11] rust: device: clarify the drvdata_borrow() safety requirement

[Simon Song]

From: Zhi Wang <zhiw@nvidia.com>

The safety requirement of drvdata_borrow() says it must only be called
after Device::set_drvdata() and before the device is fully unbound.
This describes when the stored data is expected to be present, rather
than the requirement itself.

State the requirement directly: the data stored by
Device::set_drvdata() must still be installed and remain valid for the
returned borrow.

Signed-off-by: Zhi Wang <zhiw@nvidia.com>
Signed-off-by: Simon Song <xinmengs@nvidia.com>
---
 rust/kernel/device.rs | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/rust/kernel/device.rs b/rust/kernel/device.rs
index 2291d85b6849..1f6bb58396e2 100644
--- a/rust/kernel/device.rs
+++ b/rust/kernel/device.rs
@@ -243,8 +243,8 @@ impl<Ctx: InternalBoundContext> Device<Ctx> {
     ///
     /// # Safety
     ///
-    /// - Must only be called after a preceding call to [`Device::set_drvdata`] and before the
-    ///   device is fully unbound.
+    /// - The data stored by [`Device::set_drvdata`] must still be installed and remain valid
+    ///   for the returned borrow.
     /// - The type `T` must match the type of the `ForeignOwnable` previously stored by
     ///   [`Device::set_drvdata`].
     pub unsafe fn drvdata_borrow<T>(&self) -> Pin<&T> {
-- 
2.43.7

[PATCH v6 06/11] rust: pci: drop driver data before remove returns

[Simon Song]

From: Zhi Wang <zhiw@nvidia.com>

A Rust PCI driver's private data is currently dropped by
driver::Registration::post_unbind_callback(). The driver core calls it
from device_unbind_cleanup(), after the PCI bus remove callback,
pci_device_remove(), has returned.

VF drivers will store a vfio::pci::Registration in their private data,
which calls vfio_pci_core_register_device() when created and
vfio_pci_core_unregister_device() when destroyed.

vfio_pci_core_unregister_device() expects to be called from
pci_device_remove(), and hence can't be deferred beyond it. This also
makes it incompatible with devres, i.e. a
devm_vfio_pci_core_register_device() helper wouldn't work either, since
devres_release_all() also runs from device_unbind_cleanup().

Hence, drop the private data in pci::Adapter::remove_callback(), so it
is dropped before pci_device_remove() returns.

This is a special case for VFIO-PCI. In general, a driver's private
data must not rely on being dropped from the bus remove callback; any
point after the driver's remove callback and before devres_release_all()
in device_unbind_cleanup() is sufficient.

Suggested-by: Danilo Krummrich <dakr@kernel.org>
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
Signed-off-by: Simon Song <xinmengs@nvidia.com>
---
 rust/kernel/pci.rs | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/rust/kernel/pci.rs b/rust/kernel/pci.rs
index 9e31b9bb4d0d..75d764975a4e 100644
--- a/rust/kernel/pci.rs
+++ b/rust/kernel/pci.rs
@@ -149,6 +149,19 @@ extern "C" fn remove_callback(pdev: *mut bindings::pci_dev) {
         let data = unsafe { pdev.as_ref().drvdata_borrow::<T::Data<'_>>() };
 
         T::unbind(pdev, data);
+
+        // SAFETY: The driver's unbind callback has returned, and no callbacks retain a borrow.
+        let data = unsafe { pdev.as_ref().drvdata_obtain::<T::Data<'_>>() };
+
+        // Drop the private data here, ahead of `driver::Registration::post_unbind_callback()`,
+        // which would otherwise drop it after `pci_device_remove()` returns. `drvdata_obtain()`
+        // has cleared the driver data pointer, so that callback finds nothing left to drop.
+        //
+        // This is a special case for VFIO-PCI: `vfio_pci_core_unregister_device()` must be
+        // called from `pci_device_remove()`. In general, private data must not rely on being
+        // dropped from the bus remove callback; any point after the driver's remove callback and
+        // before `devres_release_all()` in `device_unbind_cleanup()` is sufficient.
+        drop(data);
     }
 }
 
-- 
2.43.7

[PATCH v6 07/11] rust: pci: add typed SR-IOV PF registration data

[Simon Song]

From: Zhi Wang <zhiw@nvidia.com>

Rust PF and VF drivers bind to separate PCI devices and may reside
in different modules. A VF driver that calls PF operations needs
typed access to the data exposed by the PF driver. This data must
remain valid until VF driver removal completes.

Add VfRegistration to register a pinned object in the PF's private
data. VF drivers can borrow it after checking the requested Rust
type, using a higher-ranked closure or a covariant lifetime encoding.

Keep VfRegistration usable when CONFIG_PCI_IOV is disabled so PF
drivers need no configuration guards around their registration field
or its initialization.

Capture the core callback context in the initializer. Reject active
VFs and an occupied registration slot before initializing the payload,
then recheck the slot before publication in case the payload initializer
created another registration. Publish only after all fallible work.

During registration teardown, disable SR-IOV and wait for VF remove
callbacks to finish before clearing the pointer and dropping the object.

Co-developed-by: Danilo Krummrich <dakr@kernel.org>
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com> # include/linux/pci.h
Signed-off-by: Simon Song <xinmengs@nvidia.com>
---
 include/linux/pci.h    |   7 ++
 rust/kernel/pci.rs     |   1 +
 rust/kernel/pci/iov.rs | 258 ++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 265 insertions(+), 1 deletion(-)

diff --git a/include/linux/pci.h b/include/linux/pci.h
index aa3765a20a96..01b82f6561ce 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -552,6 +552,13 @@ struct pci_dev {
 	u16		ats_cap;	/* ATS Capability offset */
 	u8		ats_stu;	/* ATS Smallest Translation Unit */
 #endif
+#if defined(CONFIG_PCI_IOV) && defined(CONFIG_RUST)
+	/*
+	 * Private data owned by the PF's Rust driver, readable by VF drivers
+	 * through the PCI VF registration data Rust abstraction.
+	 */
+	void		*vf_registration_data_rust;
+#endif
 #ifdef CONFIG_PCI_PRI
 	u16		pri_cap;	/* PRI Capability offset */
 	u32		pri_reqs_alloc; /* Number of PRI requests allocated */
diff --git a/rust/kernel/pci.rs b/rust/kernel/pci.rs
index 75d764975a4e..033c49363d1e 100644
--- a/rust/kernel/pci.rs
+++ b/rust/kernel/pci.rs
@@ -59,6 +59,7 @@
     Extended,
     Normal, //
 };
+pub use self::iov::VfRegistration;
 pub use self::irq::{
     IrqType,
     IrqTypes,
diff --git a/rust/kernel/pci/iov.rs b/rust/kernel/pci/iov.rs
index 78e279169be1..3f4414aa7d36 100644
--- a/rust/kernel/pci/iov.rs
+++ b/rust/kernel/pci/iov.rs
@@ -8,8 +8,51 @@
     bindings,
     device,
     error::to_result,
-    prelude::*, //
+    prelude::*,
+    types::{
+        CovariantForLt,
+        ForLt, //
+    }, //
 };
+use core::{
+    any::TypeId,
+    marker::PhantomPinned, //
+};
+
+/// Returns the PF's registration pointer, or null when `CONFIG_PCI_IOV` is disabled.
+///
+/// # Safety
+///
+/// `pdev` must point to a live PCI device. The caller must exclude concurrent changes to its
+/// registration pointer.
+unsafe fn vf_registration_data_ptr(pdev: *mut bindings::pci_dev) -> *mut c_void {
+    #[cfg(CONFIG_PCI_IOV)]
+    {
+        // SAFETY: The caller guarantees the device's validity and excludes concurrent changes.
+        unsafe { (*pdev).vf_registration_data_rust }
+    }
+    #[cfg(not(CONFIG_PCI_IOV))]
+    {
+        let _ = pdev;
+        core::ptr::null_mut()
+    }
+}
+
+/// Updates the PF's registration pointer, doing nothing when `CONFIG_PCI_IOV` is disabled.
+///
+/// # Safety
+///
+/// `pdev` must point to a live PCI device. The caller must exclude concurrent access to its
+/// registration pointer and uphold the invariants of any published registration data.
+unsafe fn set_vf_registration_data_ptr(pdev: *mut bindings::pci_dev, data: *mut c_void) {
+    #[cfg(CONFIG_PCI_IOV)]
+    // SAFETY: The caller guarantees the device's validity and exclusive access to its slot.
+    unsafe {
+        (*pdev).vf_registration_data_rust = data;
+    }
+    #[cfg(not(CONFIG_PCI_IOV))]
+    let _ = (pdev, data);
+}
 
 impl Device<device::Core<'_>> {
     /// Returns the number of Virtual Functions (VF) enabled for a Physical Function (PF).
@@ -59,3 +102,216 @@ pub(crate) fn disable_sriov(&self) {
         unsafe { bindings::pci_disable_sriov(self.as_raw()) };
     }
 }
+
+/// Wrapper for VF registration data stored inside a [`VfRegistration`].
+///
+/// Stores a [`TypeId`] header (derived from `F`) followed by the pinned data,
+/// so that [`Device::vf_registration_data_with()`] can verify the type at
+/// runtime.
+#[repr(C)]
+#[pin_data]
+struct VfRegistrationData<'a, F: ForLt + 'static> {
+    type_id: TypeId,
+    #[pin]
+    data: F::Of<'a>,
+}
+
+static_assert!(
+    core::mem::offset_of!(VfRegistrationData<'static, CovariantForLt!(())>, type_id) == 0
+);
+
+impl<'a, F: ForLt + 'static> VfRegistrationData<'a, F> {
+    /// Pin-initializer for the registration data.
+    #[inline]
+    fn new<E>(data: impl PinInit<F::Of<'a>, E>) -> impl PinInit<Self, E> {
+        try_pin_init!(Self {
+            type_id: TypeId::of::<F>(),
+            data <- data,
+        }? E)
+    }
+}
+
+/// SR-IOV VF registration on a PF device.
+///
+/// Owns the registration data that VF drivers access via
+/// [`Device::vf_registration_data_with()`] and [`Device::vf_registration_data()`].
+///
+/// Drop disables SR-IOV before clearing the registration pointer and releasing the data.
+///
+/// When `CONFIG_PCI_IOV` is disabled, the registration still initializes, pins and owns its data,
+/// but does not publish it to VF drivers.
+#[pin_data(PinnedDrop)]
+pub struct VfRegistration<'a, F: ForLt + 'static> {
+    pdev: &'a Device<device::Bound>,
+    #[pin]
+    inner: VfRegistrationData<'a, F>,
+    #[pin]
+    _pin: PhantomPinned,
+}
+
+impl<'a, F: ForLt + 'static> VfRegistration<'a, F>
+where
+    for<'b> F::Of<'b>: Send + Sync,
+{
+    /// Create a new VF registration.
+    ///
+    /// Returns a pin-initializer so the registration can be embedded directly
+    /// in the PF driver's bus device private data.
+    ///
+    /// # Safety
+    ///
+    /// The returned registration must be embedded in the driver's bus device private data.
+    pub unsafe fn new<'core, I>(
+        pdev: &'a Device<device::Core<'core>>,
+        data: I,
+    ) -> impl PinInit<Self, Error> + 'a + use<'a, 'core, F, I>
+    where
+        I: PinInit<F::Of<'a>, Error> + 'a,
+    {
+        try_pin_init!(Self {
+            _: {
+                if pdev.is_virtfn() {
+                    return Err(ENODEV);
+                }
+                if pdev.num_vf() != 0 {
+                    return Err(EBUSY);
+                }
+                // SAFETY: The core callback serializes changes to this device's registration.
+                if !unsafe { vf_registration_data_ptr(pdev.as_raw()) }.is_null() {
+                    return Err(EBUSY);
+                }
+            },
+            pdev,
+            inner <- VfRegistrationData::new(data),
+            _pin: PhantomPinned,
+            _: {
+                // The data initializer may have registered another object on this device.
+                // SAFETY: The captured core context still serializes registration changes.
+                if !unsafe { vf_registration_data_ptr(pdev.as_raw()) }.is_null() {
+                    return Err(EBUSY);
+                }
+
+                // SAFETY: The data is initialized and pinned, the slot is unoccupied, and
+                // no VF can access it yet. No fallible work follows publication.
+                unsafe {
+                    set_vf_registration_data_ptr(
+                        pdev.as_raw(),
+                        core::ptr::from_ref(inner.as_ref().get_ref()).cast_mut().cast(),
+                    );
+                }
+            },
+        })
+    }
+}
+
+#[pinned_drop]
+impl<F: ForLt + 'static> PinnedDrop for VfRegistration<'_, F> {
+    fn drop(self: Pin<&mut Self>) {
+        // SAFETY: `pci_disable_sriov()` is safe to call on any `pci_dev`; it
+        // is a no-op if the device has no VFs enabled. When VFs are enabled,
+        // this blocks until all VF `remove()` callbacks complete.
+        unsafe { bindings::pci_disable_sriov(self.pdev.as_raw()) };
+
+        // SAFETY: The device is valid and all VF remove callbacks have completed, so no VF
+        // can access the registration pointer. The data remains alive until this returns.
+        unsafe { set_vf_registration_data_ptr(self.pdev.as_raw(), core::ptr::null_mut()) };
+    }
+}
+
+// SAFETY: The inner data is `Send` (enforced by the bound), and `&Device` is `Send + Sync`.
+unsafe impl<F: ForLt> Send for VfRegistration<'_, F> where for<'a> F::Of<'a>: Send {}
+
+// SAFETY: The inner data is `Send + Sync`. `VfRegistration` doesn't expose mutable access;
+// VF drivers only read the data through an immutable pinned reference.
+unsafe impl<F: ForLt> Sync for VfRegistration<'_, F> where for<'a> F::Of<'a>: Send + Sync {}
+
+impl Device<device::Bound> {
+    /// Internal helper: reads the `vf_registration_data_rust` pointer from the
+    /// PF, checks the `TypeId`, and returns a pinned reference.
+    ///
+    /// # Safety
+    ///
+    /// The data lifetime must be hidden behind a higher-ranked closure independently of the
+    /// reference lifetime, or `F` must be covariant in its encoded lifetime.
+    #[cfg(CONFIG_PCI_IOV)]
+    unsafe fn vf_registration_data_pinned<F: ForLt + 'static>(&self) -> Result<Pin<&F::Of<'_>>> {
+        if !self.is_virtfn() {
+            return Err(ENODEV);
+        }
+
+        // SAFETY: This bound VF uses the `physfn` union field. PCI retains its PF until
+        // VF removal completes, and the PF keeps the registration installed until then.
+        let ptr = unsafe {
+            let pf = (*self.as_raw()).__bindgen_anon_1.physfn;
+            vf_registration_data_ptr(pf)
+        };
+
+        if ptr.is_null() {
+            return Err(ENODEV);
+        }
+
+        // SAFETY: The registration keeps its data installed until VF removal completes,
+        // including when probe initialization rolls back.
+        // `ptr` points to a `VfRegistrationData` whose first field is a `TypeId`.
+        let type_id = unsafe { ptr.cast::<TypeId>().read() };
+        if type_id != TypeId::of::<F>() {
+            return Err(EINVAL);
+        }
+
+        // SAFETY: TypeId check confirms the stored type matches `F`. The data
+        // is pinned inside the PF's driver data struct. Lifetime shortening
+        // from the PF's binding scope to `'_` is layout-compatible.
+        let data_ptr = unsafe {
+            let vfrd = ptr.cast::<VfRegistrationData<'_, F>>();
+            &raw const (*vfrd).data
+        };
+
+        // SAFETY: `data` is structurally pinned inside `VfRegistrationData`.
+        Ok(unsafe { Pin::new_unchecked(&*data_ptr) })
+    }
+
+    /// Returns [`ENODEV`] because no VF registration can be published.
+    ///
+    /// # Safety
+    ///
+    /// The data lifetime must be hidden behind a higher-ranked closure independently of the
+    /// reference lifetime, or `F` must be covariant in its encoded lifetime.
+    #[cfg(not(CONFIG_PCI_IOV))]
+    unsafe fn vf_registration_data_pinned<F: ForLt + 'static>(&self) -> Result<Pin<&F::Of<'_>>> {
+        Err(ENODEV)
+    }
+
+    /// Access the VF registration data through a closure with an HRTB lifetime.
+    ///
+    /// `F` is the [`ForLt`](trait@ForLt) encoding of the data type. Returns
+    /// [`ENODEV`] if this is not a VF or no data was registered,
+    /// or [`EINVAL`] if `F` does not match the type registered by the PF.
+    /// Returns [`ENODEV`] when `CONFIG_PCI_IOV` is disabled.
+    ///
+    /// The reference is borrowed from this VF, while the registration data's lifetime remains
+    /// abstract so the closure cannot store shorter-lived references in invariant data.
+    #[inline]
+    pub fn vf_registration_data_with<'this, F: ForLt + 'static, R>(
+        &'this self,
+        f: impl for<'a> FnOnce(Pin<&'this F::Of<'a>>) -> R,
+    ) -> Result<R> {
+        // SAFETY: The closure is higher-ranked over the data lifetime, independently of `'this`.
+        // It cannot insert shorter-lived references, and the outer borrow cannot outlive this VF.
+        let pinned = unsafe { self.vf_registration_data_pinned::<F>()? };
+        Ok(f(pinned))
+    }
+
+    /// Returns a pinned reference to the VF registration data.
+    ///
+    /// Available only when `F` implements [`CovariantForLt`](trait@crate::types::CovariantForLt),
+    /// guaranteeing that shortening the PF data lifetime is sound.
+    ///
+    /// For non-covariant types, use [`Self::vf_registration_data_with()`].
+    ///
+    /// It returns the same errors as [`Self::vf_registration_data_with()`].
+    #[inline]
+    pub fn vf_registration_data<F: CovariantForLt + 'static>(&self) -> Result<Pin<&F::Of<'_>>> {
+        // SAFETY: `CovariantForLt` permits shortening the encoded lifetime to this borrow.
+        unsafe { self.vf_registration_data_pinned::<F>() }
+    }
+}
-- 
2.43.7

[PATCH v6 08/11] rust: pci: add SR-IOV enable and disable tokens

[Simon Song]

From: Zhi Wang <zhiw@nvidia.com>

Enabling VFs can succeed before the driver's remaining setup fails.
Use a callback-scoped token to return an enabled guard that disables
SR-IOV when dropped, rolling back those VFs on a later error.

Add SriovEnable, SriovEnabled and SriovDisable for the split callbacks
introduced later in this series. Tie each token to its PF and callback
lifetime, and limit the enabled count to the user's request. The PCI
adapter disarms the guard when accepting a successful enable callback.

Require a VfRegistration to own final VF teardown after that handoff.
Drivers that do not share data with VFs can register (). A disable
token permits explicit shutdown without forcing it on a rejected
request.

Suggested-by: Danilo Krummrich <dakr@kernel.org>
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Simon Song <xinmengs@nvidia.com>
---
 rust/kernel/pci.rs     |   7 ++-
 rust/kernel/pci/iov.rs | 102 +++++++++++++++++++++++++++++++++++++++--
 2 files changed, 105 insertions(+), 4 deletions(-)

diff --git a/rust/kernel/pci.rs b/rust/kernel/pci.rs
index 033c49363d1e..077fa80e7425 100644
--- a/rust/kernel/pci.rs
+++ b/rust/kernel/pci.rs
@@ -59,7 +59,12 @@
     Extended,
     Normal, //
 };
-pub use self::iov::VfRegistration;
+pub use self::iov::{
+    SriovDisable,
+    SriovEnable,
+    SriovEnabled,
+    VfRegistration, //
+};
 pub use self::irq::{
     IrqType,
     IrqTypes,
diff --git a/rust/kernel/pci/iov.rs b/rust/kernel/pci/iov.rs
index 3f4414aa7d36..bda2bffffcda 100644
--- a/rust/kernel/pci/iov.rs
+++ b/rust/kernel/pci/iov.rs
@@ -16,7 +16,9 @@
 };
 use core::{
     any::TypeId,
-    marker::PhantomPinned, //
+    marker::PhantomPinned,
+    mem::ManuallyDrop,
+    num::NonZero, //
 };
 
 /// Returns the PF's registration pointer, or null when `CONFIG_PCI_IOV` is disabled.
@@ -71,7 +73,6 @@ impl Device<device::CoreInternal<'_>> {
     /// Enable the Single Root I/O Virtualization (SR-IOV) capability for this device,
     /// where `nr_virtfn` is number of Virtual Functions (VF) to enable.
     #[inline]
-    #[expect(dead_code)]
     pub(crate) fn enable_sriov(&self, nr_virtfn: u16) -> Result {
         // SAFETY:
         // - `self.as_raw` returns a valid pointer to a `struct pci_dev`.
@@ -88,7 +89,6 @@ pub(crate) fn enable_sriov(&self, nr_virtfn: u16) -> Result {
 
     /// Disable the Single Root I/O Virtualization (SR-IOV) capability for this device.
     #[inline]
-    #[expect(dead_code)]
     pub(crate) fn disable_sriov(&self) {
         // SAFETY:
         // - `self.as_raw` returns a valid pointer to a `struct pci_dev`.
@@ -103,6 +103,102 @@ pub(crate) fn disable_sriov(&self) {
     }
 }
 
+/// Permission to enable VFs during a driver's `sriov_enable()` callback.
+///
+/// The PCI adapter creates this token for the PF being configured. It cannot be cloned or sent
+/// to another thread, and its lifetime is restricted to the callback. Enabling VFs consumes it.
+///
+/// The callback lifetime cannot be extended:
+///
+/// ```ignore,compile_fail
+/// use kernel::pci::SriovEnable;
+///
+/// fn escape(token: SriovEnable<'_>) -> SriovEnable<'static> {
+///     token
+/// }
+/// ```
+pub struct SriovEnable<'a> {
+    pdev: &'a Device<device::CoreInternal<'a>>,
+    num_vfs: NonZero<u16>,
+}
+
+impl<'a> SriovEnable<'a> {
+    /// Returns the number of VFs requested for this callback.
+    #[inline]
+    pub fn num_vfs(&self) -> NonZero<u16> {
+        self.num_vfs
+    }
+
+    /// Enables between one and the requested number of VFs.
+    ///
+    /// VF drivers can probe before this method returns, so the PF resources they access must
+    /// already be initialized. The returned guard disables the VFs if subsequent setup fails.
+    /// Return it from `sriov_enable()` to leave the VFs enabled on callback success.
+    ///
+    /// # Errors
+    ///
+    /// - [`EINVAL`]: `num_vfs` is greater than [`Self::num_vfs()`].
+    /// - [`ENODEV`]: the PF driver has not registered a [`VfRegistration`].
+    /// - Any error returned by `pci_enable_sriov()`.
+    pub fn enable(self, num_vfs: NonZero<u16>) -> Result<SriovEnabled<'a>> {
+        if num_vfs > self.num_vfs {
+            return Err(EINVAL);
+        }
+        let num_vfs = num_vfs.get();
+
+        // SAFETY: The PF's driver data and registration remain installed throughout this callback.
+        // The registration owns final VF teardown after the enable guard is disarmed.
+        if unsafe { vf_registration_data_ptr(self.pdev.as_raw()) }.is_null() {
+            return Err(ENODEV);
+        }
+
+        self.pdev.enable_sriov(num_vfs)?;
+        Ok(SriovEnabled {
+            pdev: self.pdev,
+            num_vfs,
+        })
+    }
+}
+
+/// Enabled VFs awaiting successful completion of `sriov_enable()`.
+pub struct SriovEnabled<'a> {
+    pdev: &'a Device<device::CoreInternal<'a>>,
+    num_vfs: u16,
+}
+
+impl SriovEnabled<'_> {
+    #[inline]
+    #[expect(dead_code)]
+    fn disarm(self) -> u16 {
+        ManuallyDrop::new(self).num_vfs
+    }
+}
+
+impl Drop for SriovEnabled<'_> {
+    fn drop(&mut self) {
+        self.pdev.disable_sriov();
+    }
+}
+
+/// Permission to disable VFs during a driver's `sriov_disable()` callback.
+///
+/// The PCI adapter creates this token for the PF being configured. It cannot be cloned or sent
+/// to another thread, and its lifetime is restricted to the callback. Dropping the token does
+/// not disable VFs, allowing the callback to reject a disable request.
+pub struct SriovDisable<'a> {
+    pdev: &'a Device<device::CoreInternal<'a>>,
+}
+
+impl SriovDisable<'_> {
+    /// Disables all VFs and waits for their drivers to unbind.
+    ///
+    /// The PF resources used by VF drivers must remain available until this method returns.
+    #[inline]
+    pub fn disable(self) {
+        self.pdev.disable_sriov();
+    }
+}
+
 /// Wrapper for VF registration data stored inside a [`VfRegistration`].
 ///
 /// Stores a [`TypeId`] header (derived from `F`) followed by the pinned data,
-- 
2.43.7

[PATCH v6 09/11] rust: pci: add SriovEnable::enable_all(), to enable all requested VFs

[Simon Song]

PF drivers typically enable all VFs that userspace requests through
sriov_numvfs. With the SriovEnable token this takes two steps:
enable() consumes the token, so the requested count has to be read
from it first:

	let num_vfs = token.num_vfs();
	token.enable(num_vfs)

Add enable_all() for this common case. It passes the requested number
of VFs to enable(), so callers do not need to handle the count.

Suggested-by: Danilo Krummrich <dakr@kernel.org>
Signed-off-by: Simon Song <xinmengs@nvidia.com>
---
 rust/kernel/pci/iov.rs | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/rust/kernel/pci/iov.rs b/rust/kernel/pci/iov.rs
index bda2bffffcda..a3d1c4c567f7 100644
--- a/rust/kernel/pci/iov.rs
+++ b/rust/kernel/pci/iov.rs
@@ -158,6 +158,20 @@ pub fn enable(self, num_vfs: NonZero<u16>) -> Result<SriovEnabled<'a>> {
             num_vfs,
         })
     }
+
+    /// Enables all VFs requested for this callback.
+    ///
+    /// This is equivalent to calling [`Self::enable()`] with [`Self::num_vfs()`].
+    ///
+    /// # Errors
+    ///
+    /// - [`ENODEV`]: the PF driver has not registered a [`VfRegistration`].
+    /// - Any error returned by `pci_enable_sriov()`.
+    #[inline]
+    pub fn enable_all(self) -> Result<SriovEnabled<'a>> {
+        let num_vfs = self.num_vfs;
+        self.enable(num_vfs)
+    }
 }
 
 /// Enabled VFs awaiting successful completion of `sriov_enable()`.
-- 
2.43.7

[PATCH v6 10/11] rust: pci: add SR-IOV enable and disable callbacks

[Simon Song]

From: Peter Colberg <peter@colberg.org>

Allow Rust PCI drivers to configure SR-IOV through sriov_numvfs.
Split the C configure callback into sriov_enable() and sriov_disable(),
passing pinned driver data and a token for the requested operation.
Require drivers to implement both callbacks together.

Return the enabled guard's VF count on success and disarm its rollback.
On a disable request, verify that the driver has removed all VFs before
reporting success. VfRegistration handles VF removal during PF teardown;
these callbacks only handle explicit configuration requests.

The PCI core invokes sriov_configure only through the PF-only
sriov_numvfs sysfs attribute, so no extra PF role check is needed.

Suggested-by: Danilo Krummrich <dakr@kernel.org>
Signed-off-by: Peter Colberg <peter@colberg.org>
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Simon Song <xinmengs@nvidia.com>
---
 rust/kernel/pci.rs     | 87 ++++++++++++++++++++++++++++++++++++++++++
 rust/kernel/pci/iov.rs | 55 ++++++++++++++++++++++----
 2 files changed, 135 insertions(+), 7 deletions(-)

diff --git a/rust/kernel/pci.rs b/rust/kernel/pci.rs
index 077fa80e7425..4887052d31b1 100644
--- a/rust/kernel/pci.rs
+++ b/rust/kernel/pci.rs
@@ -94,12 +94,20 @@ unsafe fn register(
         name: &'static CStr,
         module: &'static ThisModule,
     ) -> Result {
+        const_assert!(
+            T::HAS_SRIOV_ENABLE == T::HAS_SRIOV_DISABLE,
+            "PCI drivers must implement both sriov_enable and sriov_disable"
+        );
+
         // SAFETY: It's safe to set the fields of `struct pci_driver` on initialization.
         unsafe {
             (*pdrv.get()).name = name.as_char_ptr();
             (*pdrv.get()).probe = Some(Self::probe_callback);
             (*pdrv.get()).remove = Some(Self::remove_callback);
             (*pdrv.get()).id_table = T::ID_TABLE.as_ptr();
+            if T::HAS_SRIOV_ENABLE {
+                (*pdrv.get()).sriov_configure = Some(Self::sriov_configure_callback);
+            }
         }
 
         // SAFETY: `pdrv` is guaranteed to be a valid `DriverType`.
@@ -355,6 +363,85 @@ fn probe<'bound>(
     fn unbind<'bound>(dev: &'bound Device<device::Core<'_>>, this: Pin<&Self::Data<'bound>>) {
         let _ = (dev, this);
     }
+
+    /// Enables Single Root I/O Virtualization (SR-IOV) for a PF.
+    ///
+    /// This callback is only invoked when `CONFIG_PCI_IOV` is enabled.
+    ///
+    /// Called when userspace writes a nonzero VF count to `sriov_numvfs`. The token carries the
+    /// requested count and permission to enable VFs on this PF. Return the guard obtained from
+    /// [`SriovEnable::enable()`] after completing setup; dropping it rolls back the enable.
+    ///
+    /// The PF must own a [`VfRegistration`] before enabling VFs. That registration disables
+    /// SR-IOV when the driver data is destroyed, while resources needed by VF drivers remain live.
+    /// Implement this callback and [`Self::sriov_disable()`] together, or implement neither.
+    ///
+    /// See [PCI Express I/O Virtualization].
+    ///
+    /// [PCI Express I/O Virtualization]: https://docs.kernel.org/PCI/pci-iov-howto.html
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// # use kernel::{
+    /// #     device::Core,
+    /// #     pci,
+    /// #     prelude::*, //
+    /// # };
+    /// # struct Data;
+    /// fn sriov_enable<'a>(
+    ///     _dev: &'a pci::Device<Core<'_>>,
+    ///     _data: Pin<&Data>,
+    ///     token: pci::SriovEnable<'a>,
+    /// ) -> Result<pci::SriovEnabled<'a>> {
+    ///     let num_vfs = token.num_vfs();
+    ///     let enabled = token.enable(num_vfs)?;
+    ///     // Complete any additional setup before returning the guard.
+    ///     Ok(enabled)
+    /// }
+    /// ```
+    #[optional]
+    fn sriov_enable<'a>(
+        _dev: &'a Device<device::Core<'_>>,
+        _data: Pin<&Self::Data<'a>>,
+        _token: SriovEnable<'a>,
+    ) -> Result<SriovEnabled<'a>>;
+
+    /// Disables all VFs of a PF in response to a userspace request.
+    ///
+    /// This callback is only invoked when `CONFIG_PCI_IOV` is enabled.
+    ///
+    /// Called when userspace writes zero to `sriov_numvfs`. Call [`SriovDisable::disable()`]
+    /// while resources used by VF drivers are still available. Returning an error before calling
+    /// it leaves the VFs enabled. Returning success requires that all VFs have been disabled.
+    ///
+    /// This callback is not invoked during PF unbind; [`VfRegistration`] owns that teardown.
+    /// Implement this callback and [`Self::sriov_enable()`] together, or implement neither.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// # use kernel::{
+    /// #     device::Core,
+    /// #     pci,
+    /// #     prelude::*, //
+    /// # };
+    /// # struct Data;
+    /// fn sriov_disable<'a>(
+    ///     _dev: &'a pci::Device<Core<'_>>,
+    ///     _data: Pin<&Data>,
+    ///     token: pci::SriovDisable<'a>,
+    /// ) -> Result {
+    ///     token.disable();
+    ///     Ok(())
+    /// }
+    /// ```
+    #[optional]
+    fn sriov_disable<'a>(
+        _dev: &'a Device<device::Core<'_>>,
+        _data: Pin<&Self::Data<'a>>,
+        _token: SriovDisable<'a>,
+    ) -> Result;
 }
 
 /// The PCI device representation.
diff --git a/rust/kernel/pci/iov.rs b/rust/kernel/pci/iov.rs
index a3d1c4c567f7..21c034c34758 100644
--- a/rust/kernel/pci/iov.rs
+++ b/rust/kernel/pci/iov.rs
@@ -2,12 +2,19 @@
 
 //! Abstractions for PCI Single Root I/O Virtualization (SR-IOV) drivers.
 
-use super::Device;
+use super::{
+    Adapter,
+    Device,
+    Driver, //
+};
 
 use crate::{
     bindings,
     device,
-    error::to_result,
+    error::{
+        from_result,
+        to_result, //
+    },
     prelude::*,
     types::{
         CovariantForLt,
@@ -103,7 +110,7 @@ pub(crate) fn disable_sriov(&self) {
     }
 }
 
-/// Permission to enable VFs during a driver's `sriov_enable()` callback.
+/// Permission to enable VFs during a driver's [`Driver::sriov_enable()`] callback.
 ///
 /// The PCI adapter creates this token for the PF being configured. It cannot be cloned or sent
 /// to another thread, and its lifetime is restricted to the callback. Enabling VFs consumes it.
@@ -133,7 +140,7 @@ pub fn num_vfs(&self) -> NonZero<u16> {
     ///
     /// VF drivers can probe before this method returns, so the PF resources they access must
     /// already be initialized. The returned guard disables the VFs if subsequent setup fails.
-    /// Return it from `sriov_enable()` to leave the VFs enabled on callback success.
+    /// Return it from [`Driver::sriov_enable()`] to leave the VFs enabled on callback success.
     ///
     /// # Errors
     ///
@@ -174,7 +181,7 @@ pub fn enable_all(self) -> Result<SriovEnabled<'a>> {
     }
 }
 
-/// Enabled VFs awaiting successful completion of `sriov_enable()`.
+/// Enabled VFs awaiting successful completion of [`Driver::sriov_enable()`].
 pub struct SriovEnabled<'a> {
     pdev: &'a Device<device::CoreInternal<'a>>,
     num_vfs: u16,
@@ -182,7 +189,6 @@ pub struct SriovEnabled<'a> {
 
 impl SriovEnabled<'_> {
     #[inline]
-    #[expect(dead_code)]
     fn disarm(self) -> u16 {
         ManuallyDrop::new(self).num_vfs
     }
@@ -194,7 +200,7 @@ fn drop(&mut self) {
     }
 }
 
-/// Permission to disable VFs during a driver's `sriov_disable()` callback.
+/// Permission to disable VFs during a driver's [`Driver::sriov_disable()`] callback.
 ///
 /// The PCI adapter creates this token for the PF being configured. It cannot be cloned or sent
 /// to another thread, and its lifetime is restricted to the callback. Dropping the token does
@@ -213,6 +219,41 @@ pub fn disable(self) {
     }
 }
 
+impl<T: Driver> Adapter<T> {
+    pub(super) extern "C" fn sriov_configure_callback(
+        pdev: *mut bindings::pci_dev,
+        nr_virtfn: c_int,
+    ) -> c_int {
+        // SAFETY: The PCI core calls this through the PF-only `sriov_numvfs` sysfs attribute,
+        // with a valid `struct pci_dev` and while holding the device lock.
+        //
+        // INVARIANT: `pdev` is valid for the duration of `sriov_configure_callback()`.
+        let pdev = unsafe { &*pdev.cast::<Device<device::CoreInternal<'_>>>() };
+
+        // SAFETY: `sriov_configure` is called only after a successful probe and before unbind, so
+        // the stored pointer has type `T::Data<'_>` and remains valid throughout this callback.
+        let data = unsafe { pdev.as_ref().drvdata_borrow::<T::Data<'_>>() };
+
+        from_result(|| {
+            if nr_virtfn == 0 {
+                T::sriov_disable(pdev, data, SriovDisable { pdev })?;
+                // The driver may return `Ok` without calling `SriovDisable::disable()`, since
+                // dropping the token does not disable the VFs. Do not report success while VFs
+                // remain enabled.
+                if pdev.num_vf() != 0 {
+                    return Err(EBUSY);
+                }
+                Ok(0)
+            } else {
+                let num_vfs =
+                    NonZero::new(u16::try_from(nr_virtfn).map_err(|_| EINVAL)?).ok_or(EINVAL)?;
+                let enabled = T::sriov_enable(pdev, data, SriovEnable { pdev, num_vfs })?;
+                Ok(enabled.disarm().into())
+            }
+        })
+    }
+}
+
 /// Wrapper for VF registration data stored inside a [`VfRegistration`].
 ///
 /// Stores a [`TypeId`] header (derived from `F`) followed by the pinned data,
-- 
2.43.7

[PATCH v6 11/11] samples: rust: add a Rust SR-IOV PF and VF driver sample

[Simon Song]

From: Peter Colberg <peter@colberg.org>

Add a sample module with a PF and a VF driver that demonstrates SR-IOV
using QEMU's 82576 (igb) emulation.

The PF publishes a pinned API object in its private data and provides a
submit_request() helper. The VF calls this helper with its Bound device
borrow. The helper performs the typed registration lookup, so the VF
driver does not use the API object or its lifetime encoding directly.

Keep the PF and VF drivers in separate files, but build them into a
single module that registers both drivers. Separate modules would need
cross-crate Rust symbol exports, which can be added once the build
system supports them.

Unbinding the PF disables SR-IOV and removes its VFs before releasing
the registered data.

Link: https://www.qemu.org/docs/master/system/devices/igb.html
Signed-off-by: Peter Colberg <peter@colberg.org>
Co-developed-by: Zhi Wang <zhiw@nvidia.com>
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
Co-developed-by: Simon Song <xinmengs@nvidia.com>
Signed-off-by: Simon Song <xinmengs@nvidia.com>
---
 MAINTAINERS                          |   1 +
 samples/rust/Kconfig                 |  11 ++
 samples/rust/Makefile                |   1 +
 samples/rust/rust_driver_sriov.rs    | 178 +++++++++++++++++++++++++++
 samples/rust/rust_driver_sriov_vf.rs |  49 ++++++++
 5 files changed, 240 insertions(+)
 create mode 100644 samples/rust/rust_driver_sriov.rs
 create mode 100644 samples/rust/rust_driver_sriov_vf.rs

diff --git a/MAINTAINERS b/MAINTAINERS
index 93abbb315065..4ea281037109 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -21634,6 +21634,7 @@ F:	rust/helpers/pci.c
 F:	rust/kernel/pci.rs
 F:	rust/kernel/pci/
 F:	samples/rust/rust_driver_pci.rs
+F:	samples/rust/rust_driver_sriov*
 
 PCIE BANDWIDTH CONTROLLER
 M:	Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
diff --git a/samples/rust/Kconfig b/samples/rust/Kconfig
index 31d62533ef25..b88473558b9d 100644
--- a/samples/rust/Kconfig
+++ b/samples/rust/Kconfig
@@ -128,6 +128,17 @@ config SAMPLE_RUST_DRIVER_PLATFORM
 
 	  If unsure, say N.
 
+config SAMPLE_RUST_DRIVER_SRIOV
+	tristate "SR-IOV PF and VF Drivers"
+	depends on PCI_IOV
+	help
+	  This option builds the Rust SR-IOV PF and VF driver sample.
+
+	  To compile this as a module, choose M here:
+	  the module will be called rust_driver_sriov.
+
+	  If unsure, say N.
+
 config SAMPLE_RUST_DRIVER_USB
 	tristate "USB Driver"
 	depends on USB = y
diff --git a/samples/rust/Makefile b/samples/rust/Makefile
index b986b681cde5..238a11d5ec39 100644
--- a/samples/rust/Makefile
+++ b/samples/rust/Makefile
@@ -11,6 +11,7 @@ obj-$(CONFIG_SAMPLE_RUST_DRIVER_I2C)		+= rust_driver_i2c.o
 obj-$(CONFIG_SAMPLE_RUST_I2C_CLIENT)		+= rust_i2c_client.o
 obj-$(CONFIG_SAMPLE_RUST_DRIVER_PCI)		+= rust_driver_pci.o
 obj-$(CONFIG_SAMPLE_RUST_DRIVER_PLATFORM)	+= rust_driver_platform.o
+obj-$(CONFIG_SAMPLE_RUST_DRIVER_SRIOV)		+= rust_driver_sriov.o
 obj-$(CONFIG_SAMPLE_RUST_DRIVER_USB)		+= rust_driver_usb.o
 obj-$(CONFIG_SAMPLE_RUST_DRIVER_FAUX)		+= rust_driver_faux.o
 obj-$(CONFIG_SAMPLE_RUST_DRIVER_AUXILIARY)	+= rust_driver_auxiliary.o
diff --git a/samples/rust/rust_driver_sriov.rs b/samples/rust/rust_driver_sriov.rs
new file mode 100644
index 000000000000..35e45297b21d
--- /dev/null
+++ b/samples/rust/rust_driver_sriov.rs
@@ -0,0 +1,178 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Rust SR-IOV driver sample based on QEMU's 82576 ([igb]) emulation.
+//!
+//! To make these drivers probe, QEMU must be run with `-device igb`.
+//!
+//! Further, enable [vIOMMU] with interrupt remapping using, e.g.,
+//!
+//! `-M q35,accel=kvm,kernel-irqchip=split -device intel-iommu,intremap=on,caching-mode=on`
+//!
+//! and append `intel_iommu=on` to the guest kernel arguments.
+//!
+//! This module registers both the PF driver and the VF driver.
+//!
+//! [igb]: https://www.qemu.org/docs/master/system/devices/igb.html
+//! [vIOMMU]: https://wiki.qemu.org/Features/VT-d
+
+use kernel::{
+    device::{
+        Bound,
+        Core, //
+    },
+    driver,
+    pci,
+    prelude::*,
+    sync::atomic::{
+        Atomic,
+        Relaxed, //
+    },
+    types::CovariantForLt,
+    InPlaceModule, //
+};
+
+#[path = "rust_driver_sriov_vf.rs"]
+mod vf;
+
+struct SamplePfDriver;
+
+#[pin_data]
+struct PfApi<'bound> {
+    pdev: &'bound pci::Device<Bound>,
+    next_request: Atomic<u64>,
+}
+
+type PfApiForLt = CovariantForLt!(PfApi<'_>);
+
+impl PfApi<'_> {
+    fn submit(self: Pin<&Self>, vf: &pci::Device<Bound>) -> u64 {
+        let request = self.next_request.fetch_add(1, Relaxed);
+
+        dev_info!(
+            self.pdev,
+            "Handle PF request {} from VF devfn {:#x}.\n",
+            request,
+            vf.dev_id()
+        );
+
+        request
+    }
+}
+
+/// Submits a request to the PF associated with a bound VF.
+fn submit_request(vf: &pci::Device<Bound>) -> Result<u64> {
+    Ok(vf.vf_registration_data::<PfApiForLt>()?.submit(vf))
+}
+
+#[pin_data]
+struct PfDriverData<'bound> {
+    #[pin]
+    _registration: pci::VfRegistration<'bound, PfApiForLt>,
+    pdev: &'bound pci::Device<Bound>,
+}
+
+kernel::pci_device_table!(
+    PF_TABLE,
+    <SamplePfDriver as pci::Driver>::IdInfo,
+    [(
+        // E1000_DEV_ID_82576
+        pci::DeviceId::from_id(pci::Vendor::INTEL, 0x10c9),
+        ()
+    )]
+);
+
+#[vtable]
+impl pci::Driver for SamplePfDriver {
+    type IdInfo = ();
+    type Data<'bound> = PfDriverData<'bound>;
+
+    const ID_TABLE: pci::IdTable<Self::IdInfo> = &PF_TABLE;
+
+    fn probe<'bound>(
+        pdev: &'bound pci::Device<Core<'_>>,
+        _info: Option<&'bound Self::IdInfo>,
+    ) -> impl PinInit<Self::Data<'bound>, Error> + 'bound {
+        try_pin_init!(PfDriverData {
+            _: {
+                dev_info!(
+                    pdev,
+                    "Probe Rust SR-IOV PF sample (PCI ID: {}, 0x{:x}).\n",
+                    pdev.vendor_id(),
+                    pdev.device_id()
+                );
+            },
+            // SAFETY: The registration is embedded in the PF driver's private data.
+            _registration <- unsafe {
+                pci::VfRegistration::new(
+                    pdev,
+                    try_pin_init!(PfApi {
+                        pdev,
+                        next_request: Atomic::new(1),
+                    }),
+                )
+            },
+            _: {
+                pdev.enable_device_mem()?;
+                pdev.set_master();
+            },
+            pdev,
+        })
+    }
+
+    fn sriov_enable<'a>(
+        _dev: &'a pci::Device<Core<'_>>,
+        data: Pin<&Self::Data<'a>>,
+        token: pci::SriovEnable<'a>,
+    ) -> Result<pci::SriovEnabled<'a>> {
+        dev_info!(
+            data.pdev,
+            "Enable SR-IOV (PCI ID: {}, 0x{:x}).\n",
+            data.pdev.vendor_id(),
+            data.pdev.device_id()
+        );
+
+        token.enable_all()
+    }
+
+    fn sriov_disable<'a>(
+        _dev: &'a pci::Device<Core<'_>>,
+        data: Pin<&Self::Data<'a>>,
+        token: pci::SriovDisable<'a>,
+    ) -> Result {
+        dev_info!(
+            data.pdev,
+            "Disable SR-IOV (PCI ID: {}, 0x{:x}).\n",
+            data.pdev.vendor_id(),
+            data.pdev.device_id()
+        );
+
+        token.disable();
+        Ok(())
+    }
+}
+
+#[pin_data]
+struct SampleModule {
+    #[pin]
+    _pf_driver: driver::Registration<pci::Adapter<SamplePfDriver>>,
+    #[pin]
+    _vf_driver: driver::Registration<pci::Adapter<vf::SampleVfDriver>>,
+}
+
+impl InPlaceModule for SampleModule {
+    fn init(module: &'static kernel::ThisModule) -> impl PinInit<Self, Error> {
+        // Both drivers are on the PCI bus, so they need distinct driver names.
+        try_pin_init!(Self {
+            _pf_driver <- driver::Registration::new(c"rust_driver_sriov_pf", module),
+            _vf_driver <- driver::Registration::new(c"rust_driver_sriov_vf", module),
+        })
+    }
+}
+
+module! {
+    type: SampleModule,
+    name: "rust_driver_sriov",
+    authors: ["Peter Colberg"],
+    description: "Rust SR-IOV PF and VF driver sample",
+    license: "GPL v2",
+}
diff --git a/samples/rust/rust_driver_sriov_vf.rs b/samples/rust/rust_driver_sriov_vf.rs
new file mode 100644
index 000000000000..e03ff05bb910
--- /dev/null
+++ b/samples/rust/rust_driver_sriov_vf.rs
@@ -0,0 +1,49 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Virtual Function driver for the Rust SR-IOV sample.
+
+use kernel::{
+    device::Core,
+    pci,
+    prelude::*, //
+};
+
+pub(crate) struct SampleVfDriver;
+
+kernel::pci_device_table!(
+    VF_TABLE,
+    <SampleVfDriver as pci::Driver>::IdInfo,
+    [(
+        // E1000_DEV_ID_82576_VF
+        pci::DeviceId::from_id(pci::Vendor::INTEL, 0x10ca),
+        ()
+    )]
+);
+
+#[vtable]
+impl pci::Driver for SampleVfDriver {
+    type IdInfo = ();
+    type Data<'bound> = Self;
+
+    const ID_TABLE: pci::IdTable<Self::IdInfo> = &VF_TABLE;
+
+    fn probe<'bound>(
+        pdev: &'bound pci::Device<Core<'_>>,
+        _info: Option<&'bound Self::IdInfo>,
+    ) -> impl PinInit<Self, Error> + 'bound {
+        dev_info!(
+            pdev,
+            "Probe Rust SR-IOV VF sample (PCI ID: {}, 0x{:x}).\n",
+            pdev.vendor_id(),
+            pdev.device_id()
+        );
+
+        let request = super::submit_request(pdev)?;
+        dev_info!(pdev, "Submitted request {} through PF data.\n", request);
+
+        pdev.enable_device_mem()?;
+        pdev.set_master();
+
+        Ok(Self)
+    }
+}
-- 
2.43.7