[PATCH v3 0/4] workqueue: OwnedQueue and ScopedWork

[PATCH v3 0/4] workqueue: OwnedQueue and ScopedWork

[Alice Ryhl]

This series extends the Rust workqueue bindings with the ability to create
custom workqueues (OwnedQueue), and introduces ScopedWork, a work item
type that cancels synchronously on drop, allowing drivers to use
workqueues with lifetime-bound data.

This is a continuation of Danilo's v2 [4], which picked up my earlier
series [1] and Onur's ScopedQueue work [2].

Patch 1/6 of v2, "rust: workqueue: replace deprecated system_wq with
system_{percpu,dfl}_wq", is a fix and has been sent separately [3]. This
series no longer depends on it.

[1] https://lore.kernel.org/all/20260312-create-workqueue-v4-0-ea39c351c38f@google.com
[2] https://lore.kernel.org/all/20260617144645.253444-1-work@onurozkan.dev
[3] https://lore.kernel.org/all/20261014-rust-workqueue-system-fix-v2-1-c018b8f870f9@google.com
[4] https://lore.kernel.org/all/20260807165252.3849875-1-dakr@kernel.org

Changes in v3:
  - Rebase on v7.3-rc7
  - Split out the system_wq patch and send it separately as a fix [3];
    this series no longer depends on it
  - Drop ScopedQueue: with the ScopedWork changes below it is no longer
    needed to enqueue non-'static work items, and it has no other user
  - ScopedWork: make ScopedWork::new() unsafe instead of
    Queue::enqueue_scoped(), which is now a safe method accepting
    &ScopedWork<T> on any queue
  - ScopedWork: drop the WorkItem/WorkItemPointer/HasWork impls and the
    public ScopedWorkRef type; embed the work_struct directly with a
    dedicated callback, and keep the Opaque-wrapped payload private
  - ScopedWork: drop T on drop (Opaque does not run destructors)
  - ScopedWork: propagate the initializer's error type from new()
  - Builder: new_power_efficient() uses WQ_PERCPU | WQ_POWER_EFFICIENT;
    remove the now pointless percpu() modifier
  - Pick up Reviewed-by tags from Daniel
  - Link to v2: https://lore.kernel.org/all/20260807165252.3849875-1-dakr@kernel.org

Changes in v2:
  - Merge scoped_work.rs and scoped_queue.rs into scoped.rs
  - Introduce ScopedWorkRef<T> and wrap it in Opaque inside ScopedWork to avoid
    &mut aliasing in drop
  - Pass &ScopedWorkRef<Self> to ScopedWorkItem::run to enable self-re-enqueue
  - Add Deref<Target = Queue> for ScopedQueue; remove duplicate enqueue_scoped()
  - Add Send + Sync for OwnedQueue
  - Fix stale workqueue::system() references in docs
  - Drop Work::cancel_work_sync()
  - Make sysfs() available on ordered workqueues
  - Add #[inline] annotations throughout

Signed-off-by: Alice Ryhl <aliceryhl@google.com>
---
Alice Ryhl (3):
      rust: workqueue: restrict delayed work to global wqs
      rust: workqueue: create workqueue subdirectory
      rust: workqueue: add creation of workqueues

Danilo Krummrich (1):
      rust: workqueue: add ScopedWork for non-'static work items

 MAINTAINERS                                    |   1 +
 rust/helpers/workqueue.c                       |   7 +
 rust/kernel/workqueue/builder.rs               | 370 +++++++++++++++++++++++++
 rust/kernel/{workqueue.rs => workqueue/mod.rs} | 108 +++++++-
 rust/kernel/workqueue/scoped.rs                | 342 +++++++++++++++++++++++
 5 files changed, 823 insertions(+), 5 deletions(-)
---
base-commit: 24a19b953e2a3d0ea5a8c4951a0ab8f32720c907
change-id: 20261012-rust-workqueue-658cd0fbe770

Best regards,
-- 
Alice Ryhl <aliceryhl@google.com>

[PATCH v3 1/4] rust: workqueue: restrict delayed work to global wqs

[Alice Ryhl]

When a workqueue is shut down, delayed work that is pending but not
scheduled does not get properly cleaned up, so it's not safe to use
`enqueue_delayed` on a workqueue that might be destroyed. To fix this,
restricted `enqueue_delayed` to static queues.

This may be fixed in the future by an approach along the lines of [1].

Cc: stable@vger.kernel.org
Fixes: 7c098cd5eaae ("workqueue: rust: add delayed work items")
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Danilo Krummrich <dakr@kernel.org>
Reviewed-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Andreas Hindborg <a.hindborg@kernel.org>
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Link: https://lore.kernel.org/r/20250423-destroy-workqueue-flush-v1-1-3d74820780a5@google.com [1]
Signed-off-by: Alice Ryhl <aliceryhl@google.com>
---
 rust/kernel/workqueue.rs | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/rust/kernel/workqueue.rs b/rust/kernel/workqueue.rs
index 7e253b6f299c..aa8a472f4547 100644
--- a/rust/kernel/workqueue.rs
+++ b/rust/kernel/workqueue.rs
@@ -302,8 +302,15 @@ pub fn enqueue<W, const ID: u64>(&self, w: W) -> W::EnqueueOutput
     ///
     /// This may fail if the work item is already enqueued in a workqueue.
     ///
+    /// This is only valid for global workqueues (with static lifetimes) because those are the only
+    /// ones that outlive all possible delayed work items.
+    ///
     /// The work item will be submitted using `WORK_CPU_UNBOUND`.
-    pub fn enqueue_delayed<W, const ID: u64>(&self, w: W, delay: Jiffies) -> W::EnqueueOutput
+    pub fn enqueue_delayed<W, const ID: u64>(
+        &'static self,
+        w: W,
+        delay: Jiffies,
+    ) -> W::EnqueueOutput
     where
         W: RawDelayedWorkItem<ID> + Send + 'static,
     {

-- 
2.56.0.212.g129ede20d9-goog

[PATCH v3 2/4] rust: workqueue: create workqueue subdirectory

[Alice Ryhl]

The following patch will implement a workqueue builder in a separate
file. To prepare for that, create a rust/kernel/workqueue subdirectory
and move the existing file.

Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Signed-off-by: Alice Ryhl <aliceryhl@google.com>
---
 MAINTAINERS                                    | 1 +
 rust/kernel/{workqueue.rs => workqueue/mod.rs} | 0
 2 files changed, 1 insertion(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index f53047aa8d3c..685ace2e27d1 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -29515,6 +29515,7 @@ F:	Documentation/core-api/workqueue.rst
 F:	include/linux/workqueue.h
 F:	kernel/workqueue.c
 F:	kernel/workqueue_internal.h
+F:	rust/kernel/workqueue/
 
 WWAN DRIVERS
 M:	Loic Poulain <loic.poulain@oss.qualcomm.com>
diff --git a/rust/kernel/workqueue.rs b/rust/kernel/workqueue/mod.rs
similarity index 100%
rename from rust/kernel/workqueue.rs
rename to rust/kernel/workqueue/mod.rs

-- 
2.56.0.212.g129ede20d9-goog

[PATCH v3 3/4] rust: workqueue: add creation of workqueues

[Alice Ryhl]

Creating workqueues is needed by various GPU drivers. Not only does it
give you better control over execution, it also allows devices to ensure
that all tasks have exited before the device is unbound (or similar) by
running the workqueue destructor.

Reviewed-by: Andreas Hindborg <a.hindborg@kernel.org>
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Co-developed-by: Danilo Krummrich <dakr@kernel.org>
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Signed-off-by: Alice Ryhl <aliceryhl@google.com>
---
 rust/helpers/workqueue.c         |   7 +
 rust/kernel/workqueue/builder.rs | 370 +++++++++++++++++++++++++++++++++++++++
 rust/kernel/workqueue/mod.rs     |  51 +++++-
 3 files changed, 425 insertions(+), 3 deletions(-)

diff --git a/rust/helpers/workqueue.c b/rust/helpers/workqueue.c
index ce1c3a5b2150..e4b9d1b3d6bf 100644
--- a/rust/helpers/workqueue.c
+++ b/rust/helpers/workqueue.c
@@ -14,3 +14,10 @@ __rust_helper void rust_helper_init_work_with_key(struct work_struct *work,
 	INIT_LIST_HEAD(&work->entry);
 	work->func = func;
 }
+
+__rust_helper
+struct workqueue_struct *rust_helper_alloc_workqueue(const char *fmt, unsigned int flags,
+						     int max_active, const void *data)
+{
+	return alloc_workqueue(fmt, flags, max_active, data);
+}
diff --git a/rust/kernel/workqueue/builder.rs b/rust/kernel/workqueue/builder.rs
new file mode 100644
index 000000000000..d253c2f8ed4a
--- /dev/null
+++ b/rust/kernel/workqueue/builder.rs
@@ -0,0 +1,370 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Workqueue builders.
+
+use kernel::{
+    alloc::AllocError,
+    prelude::*,
+    workqueue::{
+        OwnedQueue, //
+        Queue,
+    }, //
+};
+
+use core::{
+    marker::PhantomData,
+    ptr::{
+        self,
+        NonNull, //
+    },
+};
+
+/// Workqueue builder.
+///
+/// A valid combination of workqueue flags contains one of the base flags (`WQ_UNBOUND`, `WQ_BH`,
+/// or `WQ_PERCPU`) and a combination of modifier flags that are compatible with the selected base
+/// flag.
+///
+/// For details, please refer to `Documentation/core-api/workqueue.rst`.
+pub struct Builder<Kind> {
+    flags: bindings::wq_flags,
+    max_active: i32,
+    _kind: PhantomData<Kind>,
+}
+
+pub enum TypeUnbound {}
+pub enum TypePercpu {}
+pub enum TypePowerEfficient {}
+pub enum TypeBH {}
+pub enum TypeOrdered {}
+
+/// Entry-points to the builder API.
+impl Queue {
+    /// Build a workqueue whose work may execute on any cpu.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_unbound().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from unbound wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "WQ_UNBOUND")]
+    pub fn new_unbound() -> Builder<TypeUnbound> {
+        Builder {
+            flags: bindings::wq_flags_WQ_UNBOUND,
+            max_active: 0,
+            _kind: PhantomData,
+        }
+    }
+
+    /// Build a workqueue whose work items are bound to the CPU they are queued on.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_percpu().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from percpu wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "WQ_PERCPU")]
+    pub fn new_percpu() -> Builder<TypePercpu> {
+        Builder {
+            flags: bindings::wq_flags_WQ_PERCPU,
+            max_active: 0,
+            _kind: PhantomData,
+        }
+    }
+
+    /// Build a power-efficient workqueue.
+    ///
+    /// A power-efficient workqueue is per-CPU by default, but becomes unbound when the
+    /// `workqueue.power_efficient` kernel parameter is set.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_power_efficient().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from power-efficient wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "WQ_POWER_EFFICIENT")]
+    pub fn new_power_efficient() -> Builder<TypePowerEfficient> {
+        Builder {
+            flags: bindings::wq_flags_WQ_PERCPU | bindings::wq_flags_WQ_POWER_EFFICIENT,
+            max_active: 0,
+            _kind: PhantomData,
+        }
+    }
+
+    /// Build a single-threaded workqueue that executes jobs in order.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_ordered().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from ordered wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "alloc_ordered_workqueue")]
+    #[doc(alias = "__WQ_ORDERED")]
+    pub fn new_ordered() -> Builder<TypeOrdered> {
+        Builder {
+            flags: bindings::wq_flags_WQ_UNBOUND | bindings::wq_flags___WQ_ORDERED,
+            max_active: 1,
+            _kind: PhantomData,
+        }
+    }
+
+    /// Build a workqueue that executes in bottom-half (softirq) context.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_bh().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from BH wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "WQ_BH")]
+    pub fn new_bh() -> Builder<TypeBH> {
+        Builder {
+            flags: bindings::wq_flags_WQ_PERCPU | bindings::wq_flags_WQ_BH,
+            max_active: 0,
+            _kind: PhantomData,
+        }
+    }
+}
+
+/// Options that may be used with all workqueue types.
+impl<Kind> Builder<Kind> {
+    /// Mark this workqueue high priority.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_unbound().highpri().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from highpri wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "WQ_HIGHPRI")]
+    pub fn highpri(mut self) -> Self {
+        self.flags |= bindings::wq_flags_WQ_HIGHPRI;
+        self
+    }
+
+    /// Creates the workqueue.
+    ///
+    /// The provided name is used verbatim as the workqueue name.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// // create an unbound workqueue registered with sysfs
+    /// let wq = Queue::new_unbound().sysfs().build(c"my-wq")?;
+    ///
+    /// // spawn a work item on it
+    /// wq.try_spawn(
+    ///     GFP_KERNEL,
+    ///     || pr_warn!("Printing from my-wq"),
+    /// )?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "alloc_workqueue")]
+    pub fn build(self, name: &CStr) -> Result<OwnedQueue, AllocError> {
+        // SAFETY:
+        // * c"%s" is compatible with passing the name as a c-string.
+        // * the builder only permits valid flag combinations
+        let ptr = unsafe {
+            bindings::alloc_workqueue(
+                c"%s".as_char_ptr(),
+                self.flags,
+                self.max_active,
+                name.as_char_ptr().cast::<c_void>(),
+            )
+        };
+
+        // INVARIANT: We successfully created the workqueue, so we can return ownership to the
+        // caller.
+        Ok(OwnedQueue {
+            queue: NonNull::new(ptr).ok_or(AllocError)?.cast(),
+        })
+    }
+
+    /// Creates the workqueue.
+    ///
+    /// # Examples
+    ///
+    /// This example shows how to pass a Rust string formatter to the workqueue name, creating
+    /// workqueues with names such as `my-wq-1` and `my-wq-2`.
+    ///
+    /// ```
+    /// use kernel::workqueue::{Queue, OwnedQueue};
+    ///
+    /// fn my_wq(num: u32) -> Result<OwnedQueue> {
+    ///     // create a percpu workqueue called my-wq-{num}
+    ///     let wq = Queue::new_percpu().build_fmt(fmt!("my-wq-{num}"))?;
+    ///     Ok(wq)
+    /// }
+    /// ```
+    #[inline]
+    pub fn build_fmt(self, name: kernel::fmt::Arguments<'_>) -> Result<OwnedQueue, AllocError> {
+        // SAFETY:
+        // * c"%pA" is compatible with passing an `Arguments` pointer.
+        // * the builder only permits valid flag combinations
+        let ptr = unsafe {
+            bindings::alloc_workqueue(
+                c"%pA".as_char_ptr(),
+                self.flags,
+                self.max_active,
+                ptr::from_ref(&name).cast::<c_void>(),
+            )
+        };
+
+        // INVARIANT: We successfully created the workqueue, so we can return ownership to the
+        // caller.
+        Ok(OwnedQueue {
+            queue: NonNull::new(ptr).ok_or(AllocError)?.cast(),
+        })
+    }
+}
+
+/// Indicates that this workqueue is threaded.
+pub trait TypeThreaded {}
+impl TypeThreaded for TypeUnbound {}
+impl TypeThreaded for TypePercpu {}
+impl TypeThreaded for TypePowerEfficient {}
+
+/// Options that are not available on BH or ordered workqueues.
+impl<Kind: TypeThreaded> Builder<Kind> {
+    /// Set the maximum number of concurrently executing work items.
+    ///
+    /// For percpu workqueues this is per-CPU. If not set, a default value of
+    /// `WQ_DFL_ACTIVE` is used. The maximum value is `WQ_MAX_ACTIVE`.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_unbound().max_active(16).build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from wq with max_active=16"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    pub fn max_active(mut self, max_active: u32) -> Self {
+        // If provided `max_active` is greater than `i32::MAX`, then we need to trigger the C-side
+        // comparison with `WQ_MAX_ACTIVE`, which we can do by clamping to `i32::MAX`.
+        self.max_active = i32::try_from(max_active).unwrap_or(i32::MAX);
+        self
+    }
+
+    /// Mark this workqueue as cpu intensive.
+    ///
+    /// Work items will not contribute to the concurrency level, preventing
+    /// them from stalling other work items in the same per-CPU worker pool.
+    /// This is meaningless for unbound workqueues.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_unbound().cpu_intensive().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from cpu-intensive wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "WQ_CPU_INTENSIVE")]
+    pub fn cpu_intensive(mut self) -> Self {
+        self.flags |= bindings::wq_flags_WQ_CPU_INTENSIVE;
+        self
+    }
+}
+
+/// Indicates that this workqueue runs in a normal context (as opposed to softirq context).
+pub trait TypeNormal {}
+impl TypeNormal for TypeUnbound {}
+impl TypeNormal for TypePercpu {}
+impl TypeNormal for TypePowerEfficient {}
+impl TypeNormal for TypeOrdered {}
+
+/// Options that are not available on BH workqueues.
+impl<Kind: TypeNormal> Builder<Kind> {
+    /// Make this workqueue visible in sysfs.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_unbound().sysfs().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from sysfs wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "WQ_SYSFS")]
+    pub fn sysfs(mut self) -> Self {
+        self.flags |= bindings::wq_flags_WQ_SYSFS;
+        self
+    }
+
+    /// Allow this workqueue to be frozen during suspend.
+    ///
+    /// Work items on the workqueue are drained and no new work items start
+    /// execution until thawed.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_unbound().freezable().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from freezable wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "WQ_FREEZABLE")]
+    pub fn freezable(mut self) -> Self {
+        self.flags |= bindings::wq_flags_WQ_FREEZABLE;
+        self
+    }
+
+    /// This workqueue may be used during memory reclaim.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_unbound().mem_reclaim().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from mem_reclaim wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "WQ_MEM_RECLAIM")]
+    pub fn mem_reclaim(mut self) -> Self {
+        self.flags |= bindings::wq_flags_WQ_MEM_RECLAIM;
+        self
+    }
+}
diff --git a/rust/kernel/workqueue/mod.rs b/rust/kernel/workqueue/mod.rs
index aa8a472f4547..78e6a7bb960a 100644
--- a/rust/kernel/workqueue/mod.rs
+++ b/rust/kernel/workqueue/mod.rs
@@ -186,7 +186,10 @@
 //! C header: [`include/linux/workqueue.h`](srctree/include/linux/workqueue.h)
 
 use crate::{
-    alloc::{AllocError, Flags},
+    alloc::{
+        self,
+        AllocError, //
+    },
     container_of,
     prelude::*,
     sync::{
@@ -200,7 +203,14 @@
     time::Jiffies,
     types::Opaque,
 };
-use core::{marker::PhantomData, ptr::NonNull};
+use core::{
+    marker::PhantomData,
+    ops::Deref,
+    ptr::NonNull, //
+};
+
+mod builder;
+pub use self::builder::Builder;
 
 /// Creates a [`Work`] initialiser with the given name and a newly-created lock class.
 #[macro_export]
@@ -346,7 +356,7 @@ pub fn enqueue_delayed<W, const ID: u64>(
     /// This method can fail because it allocates memory to store the work item.
     pub fn try_spawn<T: 'static + Send + FnOnce()>(
         &self,
-        flags: Flags,
+        flags: alloc::Flags,
         func: T,
     ) -> Result<(), AllocError> {
         let init = pin_init!(ClosureWork {
@@ -359,6 +369,41 @@ pub fn try_spawn<T: 'static + Send + FnOnce()>(
     }
 }
 
+/// An owned kernel work queue.
+///
+/// Dropping a workqueue blocks on all pending work.
+///
+/// # Invariants
+///
+/// `queue` points at a valid workqueue that is owned by this `OwnedQueue`.
+pub struct OwnedQueue {
+    queue: NonNull<Queue>,
+}
+
+// SAFETY: `OwnedQueue` uniquely owns a valid `Queue`, which is `Send + Sync`.
+unsafe impl Send for OwnedQueue {}
+// SAFETY: `&OwnedQueue` only provides `&Queue` (via `Deref`), which is safe to share.
+unsafe impl Sync for OwnedQueue {}
+
+impl Deref for OwnedQueue {
+    type Target = Queue;
+    #[inline]
+    fn deref(&self) -> &Queue {
+        // SAFETY: By the type invariants, this pointer references a valid queue.
+        unsafe { &*self.queue.as_ptr() }
+    }
+}
+
+impl Drop for OwnedQueue {
+    #[inline]
+    fn drop(&mut self) {
+        // SAFETY: This `OwnedQueue` owns a valid workqueue, so we can destroy it. There is no
+        // delayed work scheduled on this queue that may attempt to use it after this call, as
+        // scheduling delayed work requires a 'static reference.
+        unsafe { bindings::destroy_workqueue(self.queue.as_ptr().cast()) }
+    }
+}
+
 /// A helper type used in [`try_spawn`].
 ///
 /// [`try_spawn`]: Queue::try_spawn

-- 
2.56.0.212.g129ede20d9-goog

[PATCH v3 4/4] rust: workqueue: add ScopedWork for non-'static work items

[Alice Ryhl]

From: Danilo Krummrich <dakr@kernel.org>

Add ScopedWork<T>, a work item wrapper whose destructor calls
cancel_work_sync(), allowing T to carry non-'static lifetimes. Ownership
of the data is not transferred to the workqueue; instead, the
synchronous cancellation on drop guarantees the work function is not
running when the data is freed.

Because that guarantee only holds if the destructor actually runs,
ScopedWork::new() (and thus new_scoped_work!) is unsafe and requires
that the resulting value is never forgotten. In return,
Queue::enqueue_scoped() is a safe method that accepts a &ScopedWork<T>
on any queue, including the system queues.

ScopedWork does not use the Work/WorkItem machinery: it embeds the
work_struct directly and registers its own C callback, which recovers
the &ScopedWork<T> via container_of!() and passes it to
ScopedWorkItem::run(). The work_struct and the user data live in a
private Inner<T> that is wrapped in Opaque as a whole, so that the &mut
Self held by the destructor while cancel_work_sync() waits does not
claim uniqueness over memory (including padding) that a worker thread
may still be reading through &ScopedWork<T>.

Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Co-developed-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Alice Ryhl <aliceryhl@google.com>
---
 rust/kernel/workqueue/mod.rs    |  48 +++++-
 rust/kernel/workqueue/scoped.rs | 342 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 389 insertions(+), 1 deletion(-)

diff --git a/rust/kernel/workqueue/mod.rs b/rust/kernel/workqueue/mod.rs
index 78e6a7bb960a..73fa0bea4220 100644
--- a/rust/kernel/workqueue/mod.rs
+++ b/rust/kernel/workqueue/mod.rs
@@ -212,6 +212,13 @@
 mod builder;
 pub use self::builder::Builder;
 
+mod scoped;
+pub use self::scoped::{
+    new_scoped_work,
+    ScopedWork,
+    ScopedWorkItem, //
+};
+
 /// Creates a [`Work`] initialiser with the given name and a newly-created lock class.
 #[macro_export]
 macro_rules! new_work {
@@ -283,11 +290,50 @@ pub unsafe fn from_raw<'a>(ptr: *const bindings::workqueue_struct) -> &'a Queue
     pub fn enqueue<W, const ID: u64>(&self, w: W) -> W::EnqueueOutput
     where
         W: RawWorkItem<ID> + Send + 'static,
+    {
+        // SAFETY: `W: 'static`, so everything borrowed by `W` remains valid forever.
+        unsafe { self.enqueue_raw(w) }
+    }
+
+    /// Enqueues a [`ScopedWork`] item.
+    ///
+    /// Unlike [`Queue::enqueue`], this does not require the work item to be `'static`. This is
+    /// sound because a [`ScopedWork`] cancels itself synchronously when it is dropped, and
+    /// [`ScopedWork::new`] requires that this destructor runs before any lifetime in `T` expires.
+    ///
+    /// This may fail if the work item is already enqueued in a workqueue.
+    ///
+    /// The work item will be submitted using `WORK_CPU_UNBOUND`.
+    #[inline]
+    pub fn enqueue_scoped<T>(&self, w: &ScopedWork<T>) -> bool
+    where
+        T: ScopedWorkItem + Sync,
+    {
+        // SAFETY: The `ScopedWork` destructor calls `cancel_work_sync()`, and `ScopedWork::new`
+        // guarantees that this destructor runs before any lifetime in `T` expires. Hence everything
+        // borrowed by `T` remains valid until the work item has run to completion or has been
+        // cancelled.
+        unsafe { self.enqueue_raw(w) }
+    }
+
+    /// Enqueues a work item without requiring it to be `'static`.
+    ///
+    /// # Safety
+    ///
+    /// Everything borrowed by `W` must remain valid until the work item has run to completion or
+    /// has been cancelled.
+    #[inline]
+    unsafe fn enqueue_raw<W, const ID: u64>(&self, w: W) -> W::EnqueueOutput
+    where
+        W: RawWorkItem<ID> + Send,
     {
         let queue_ptr = self.0.get();
 
         // SAFETY: We only return `false` if the `work_struct` is already in a workqueue. The other
-        // `__enqueue` requirements are not relevant since `W` is `Send` and static.
+        // `__enqueue` requirements are not relevant since `W` is `Send` and the caller guarantees
+        // that everything borrowed by `W` remains valid until the work item has run to completion
+        // or has been cancelled, so the function pointer is never called after a lifetime in `W`
+        // expires.
         //
         // The call to `bindings::queue_work_on` will dereference the provided raw pointer, which
         // is ok because `__enqueue` guarantees that the pointer is valid for the duration of this
diff --git a/rust/kernel/workqueue/scoped.rs b/rust/kernel/workqueue/scoped.rs
new file mode 100644
index 000000000000..e490ac752423
--- /dev/null
+++ b/rust/kernel/workqueue/scoped.rs
@@ -0,0 +1,342 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Scoped work items.
+//!
+//! Provides [`ScopedWork`] for work items that may borrow data with some non-`'static` lifetime.
+//!
+//! Unlike [`Work`]-based items, ownership of a [`ScopedWork`] is not transferred to
+//! the workqueue. Instead, its destructor calls `cancel_work_sync()`, which guarantees that the
+//! work function is not running (and will not run) once the destructor returns. This is what
+//! allows the inner data to carry non-`'static` lifetimes.
+//!
+//! Creating a [`ScopedWork`] is `unsafe`, because the above guarantee only holds if the destructor
+//! actually runs: the value must not be forgotten. Once created, a [`ScopedWork`] can be enqueued
+//! on any queue using the safe [`Queue::enqueue_scoped`].
+//!
+//! # Examples
+//!
+//! ```
+//! use kernel::workqueue::{
+//!     new_scoped_work,
+//!     Queue,
+//!     ScopedWork,
+//!     ScopedWorkItem,
+//! };
+//!
+//! struct MyWork {
+//!     value: u32,
+//! }
+//!
+//! impl ScopedWorkItem for MyWork {
+//!     fn run(work: &ScopedWork<Self>) {
+//!         pr_info!("value = {}\n", work.value);
+//!     }
+//! }
+//!
+//! let work = KBox::pin_init(
+//!     // SAFETY: `work` is not forgotten.
+//!     unsafe { new_scoped_work!("MyWork", MyWork { value: 42 }) },
+//!     GFP_KERNEL,
+//! )?;
+//!
+//! let queue = Queue::new_unbound().build(c"my-wq")?;
+//! queue.enqueue_scoped(&work);
+//!
+//! // Dropping `queue` first drains it, so the work item is guaranteed to have run by the time
+//! // `work` is dropped.
+//! drop(queue);
+//! # Ok::<(), Error>(())
+//! ```
+
+use crate::{
+    bindings,
+    container_of,
+    prelude::*,
+    sync::LockClassKey,
+    types::Opaque,
+    workqueue::RawWorkItem, //
+};
+
+#[cfg(doc)]
+use super::{
+    Queue,
+    Work, //
+};
+
+use pin_init::Wrapper;
+
+use core::ops::Deref;
+
+/// Trait for types that can be used as scoped work items.
+///
+/// Implementers define the work function that executes when the item is dequeued by a workqueue
+/// thread. The callback receives a reference to the containing [`ScopedWork`], which provides
+/// access to the inner data via [`Deref`] and can be passed to [`Queue::enqueue_scoped`] to
+/// re-enqueue the work item from within the work function.
+pub trait ScopedWorkItem: Sized {
+    /// Called when the work item is executed.
+    fn run(work: &ScopedWork<Self>);
+}
+
+/// The payload of a [`ScopedWork`].
+///
+/// # Invariants
+///
+/// `work` is an initialized `work_struct` whose work function is [`ScopedWork::<T>::run`]. An
+/// `Inner<T>` only ever exists inside a [`ScopedWork<T>`].
+#[pin_data]
+struct Inner<T: ScopedWorkItem> {
+    #[pin]
+    work: Opaque<bindings::work_struct>,
+    #[pin]
+    data: T,
+}
+
+/// A scoped work item that cancels synchronously on drop.
+///
+/// `ScopedWork<T>` contains a `work_struct` and the user data `T`. Its destructor calls
+/// `cancel_work_sync()`, guaranteeing the work function is not running when the data is dropped.
+///
+/// This allows `T` to carry non-`'static` lifetimes.
+///
+/// Construct via [`new_scoped_work!`], which returns an `impl PinInit` suitable for embedding
+/// in-place inside other pinned structs. Because the soundness of [`Queue::enqueue_scoped`]
+/// relies on the destructor running, [`ScopedWork::new`] is `unsafe` and requires that the
+/// resulting value is never forgotten.
+///
+/// # Examples
+///
+/// Self-re-enqueueing from within the work function:
+///
+/// ```
+/// # use kernel::sync::atomic::{Atomic, Relaxed};
+/// # use kernel::time::{Delta, delay::fsleep};
+/// use kernel::workqueue::{
+///     self,
+///     new_scoped_work,
+///     Queue,
+///     ScopedWork,
+///     ScopedWorkItem,
+/// };
+///
+/// struct RequeueWork<'a> {
+///     counter: Atomic<u32>,
+///     queue: &'a Queue,
+/// }
+///
+/// impl ScopedWorkItem for RequeueWork<'_> {
+///     fn run(work: &ScopedWork<Self>) {
+///         if work.counter.fetch_add(1u32, Relaxed) < 2 {
+///             work.queue.enqueue_scoped(work);
+///         }
+///     }
+/// }
+///
+/// let queue = Queue::new_ordered().build(c"requeue_wq")?;
+///
+/// let work = KBox::pin_init(
+///     // SAFETY: `work` is not forgotten.
+///     unsafe {
+///         new_scoped_work!(
+///             "RequeueWork",
+///             RequeueWork {
+///                 counter: Atomic::new(0u32),
+///                 queue: &queue,
+///             }
+///         )
+///     },
+///     GFP_KERNEL,
+/// )?;
+///
+/// queue.enqueue_scoped(&work);
+/// # fsleep(Delta::from_millis(300));
+///
+/// assert_eq!(work.counter.load(Relaxed), 3);
+/// # Ok::<(), Error>(())
+/// ```
+#[pin_data(PinnedDrop)]
+#[repr(transparent)]
+pub struct ScopedWork<T: ScopedWorkItem> {
+    /// The whole payload is wrapped in [`Opaque`] on purpose: while the destructor waits in
+    /// `cancel_work_sync()`, a worker thread may still be executing [`ScopedWorkItem::run`] with a
+    /// shared reference to this value, concurrently with the `&mut Self` held by `drop`. [`Opaque`]
+    /// opts out of the uniqueness guarantee of that `&mut`. Wrapping individual fields would not be
+    /// enough, since the `&mut Self` would still claim exclusive access to the padding bytes.
+    ///
+    /// # Invariants
+    ///
+    /// `inner` is always initialized.
+    #[pin]
+    inner: Opaque<Inner<T>>,
+}
+
+// SAFETY: A `&ScopedWork<T>` only hands out `&T` (via `Deref`) and a pointer to the `work_struct`,
+// which is only ever accessed through the C workqueue API. Hence sharing it is safe when `T: Sync`.
+unsafe impl<T: ScopedWorkItem + Sync> Sync for ScopedWork<T> {}
+
+// SAFETY: The `work_struct` is only ever accessed through the C workqueue API, which is safe to
+// use from any thread. Sending the contained `T` requires `T: Send`.
+unsafe impl<T: ScopedWorkItem + Send> Send for ScopedWork<T> {}
+
+impl<T: ScopedWorkItem> Deref for ScopedWork<T> {
+    type Target = T;
+
+    #[inline]
+    fn deref(&self) -> &T {
+        // SAFETY: `inner` is always initialized (type invariant), and `data` is only ever accessed
+        // through shared references.
+        unsafe { &(*self.inner.get()).data }
+    }
+}
+
+impl<T: ScopedWorkItem> ScopedWork<T> {
+    /// Creates a pin-initializer for a new scoped work item.
+    ///
+    /// Use [`new_scoped_work!`] to automatically provide the lock class key.
+    ///
+    /// # Safety
+    ///
+    /// The caller must ensure that the destructor of the initialized `ScopedWork<T>` runs before
+    /// the memory it lives in is reused or any lifetime in `T` expires. In particular, the value
+    /// must not be leaked, e.g. via [`core::mem::forget`] or by leaking the allocation that
+    /// contains it.
+    #[inline]
+    pub unsafe fn new<E>(
+        name: &'static CStr,
+        key: Pin<&'static LockClassKey>,
+        init: impl PinInit<T, E>,
+    ) -> impl PinInit<Self, E> {
+        try_pin_init!(Self {
+            inner <- Opaque::pin_init(try_pin_init!(Inner::<T> {
+                work <- Opaque::try_ffi_init(|slot| {
+                    // SAFETY: `slot` is a valid pointer to uninitialized memory for a
+                    // `work_struct`, and `Self::run` is a valid work function for a `work_struct`
+                    // embedded in an `Inner<T>` that is embedded in a `ScopedWork<T>`.
+                    unsafe {
+                        bindings::init_work_with_key(
+                            slot,
+                            Some(Self::run),
+                            false,
+                            name.as_char_ptr(),
+                            key.as_ptr(),
+                        )
+                    };
+                    Ok::<(), E>(())
+                }),
+                data <- init,
+            }? E)),
+        }? E)
+    }
+
+    /// Returns a raw pointer to the `work_struct`.
+    #[inline]
+    fn raw_work(&self) -> *mut bindings::work_struct {
+        // SAFETY: `inner` is always initialized (type invariant).
+        unsafe { Opaque::cast_into(&raw const (*self.inner.get()).work) }
+    }
+
+    /// The work function registered with the C workqueue.
+    ///
+    /// # Safety
+    ///
+    /// `ptr` must point to the `work` field of the `Inner<T>` of a live `ScopedWork<T>`.
+    unsafe extern "C" fn run(ptr: *mut bindings::work_struct) {
+        // SAFETY: `Opaque<T>` is `#[repr(transparent)]`, and by the safety requirements `ptr`
+        // points to the `work` field of an `Inner<T>`.
+        let inner =
+            unsafe { container_of!(ptr.cast::<Opaque<bindings::work_struct>>(), Inner<T>, work) };
+
+        // SAFETY: `Inner<T>` only ever exists inside a `ScopedWork<T>` (type invariant), and both
+        // `Opaque` and `ScopedWork` are `#[repr(transparent)]`, so `inner` is also a valid pointer
+        // to the containing `ScopedWork<T>`. By the safety requirements it is live; its destructor
+        // calls `cancel_work_sync()` before the memory is released, so it stays live for the
+        // duration of this call.
+        T::run(unsafe { &*inner.cast::<Self>() });
+    }
+}
+
+// SAFETY: `&ScopedWork<T>` points to a `work_struct` whose work function is
+// `ScopedWork::<T>::run` (type invariant), so passing it to `queue_work_on` is sound. The pointer
+// stays valid until the work function has run or the item is cancelled, because the `ScopedWork`
+// destructor calls `cancel_work_sync()` and `ScopedWork::new` requires that destructor to run.
+unsafe impl<'a, T: ScopedWorkItem + Sync> RawWorkItem<0> for &'a ScopedWork<T> {
+    type EnqueueOutput = bool;
+
+    unsafe fn __enqueue<F>(self, queue_work_on: F) -> Self::EnqueueOutput
+    where
+        F: FnOnce(*mut bindings::work_struct) -> bool,
+    {
+        queue_work_on(self.raw_work())
+    }
+}
+
+#[pinned_drop]
+impl<T: ScopedWorkItem> PinnedDrop for ScopedWork<T> {
+    #[inline]
+    fn drop(self: Pin<&mut Self>) {
+        // SAFETY: `raw_work()` returns a valid `work_struct`. After `cancel_work_sync()` returns,
+        // the work function is guaranteed to not be running.
+        unsafe { bindings::cancel_work_sync(self.raw_work()) };
+
+        // SAFETY: `inner` is initialized (type invariant), no longer accessed by the work
+        // function, and never accessed again after this point. `Opaque` does not drop its
+        // contents, so we have to do it here.
+        unsafe { core::ptr::drop_in_place(self.inner.get()) };
+    }
+}
+
+/// Creates a [`ScopedWork`] pin-initializer with a new lock class.
+///
+/// This macro expands to a call to [`ScopedWork::new`], which is `unsafe`, so the invocation must
+/// be wrapped in an `unsafe` block.
+///
+/// # Safety
+///
+/// The caller must uphold the safety requirements of [`ScopedWork::new`]: the destructor of the
+/// initialized `ScopedWork` must run before the memory it lives in is reused or any lifetime in
+/// the inner type expires.
+///
+/// # Examples
+///
+/// ```
+/// use kernel::workqueue::{
+///     new_scoped_work,
+///     ScopedWork,
+///     ScopedWorkItem,
+/// };
+///
+/// struct MyWork {
+///     value: u32,
+/// }
+///
+/// impl ScopedWorkItem for MyWork {
+///     fn run(work: &ScopedWork<Self>) {
+///         pr_info!("value = {}\n", work.value);
+///     }
+/// }
+///
+/// #[pin_data]
+/// struct MyData {
+///     #[pin]
+///     work: ScopedWork<MyWork>,
+/// }
+///
+/// fn init_data() -> impl PinInit<MyData> {
+///     pin_init!(MyData {
+///         // SAFETY: `MyData` is only ever pin-initialized, so its `work` field is dropped
+///         // whenever `MyData` is.
+///         work <- unsafe { new_scoped_work!("MyWork", MyWork { value: 7 }) },
+///     })
+/// }
+/// ```
+#[macro_export]
+macro_rules! new_scoped_work {
+    ($name:literal, $init:expr) => {
+        $crate::workqueue::ScopedWork::new(
+            $crate::c_str!($name),
+            $crate::static_lock_class!(),
+            $init,
+        )
+    };
+}
+pub use new_scoped_work;

-- 
2.56.0.212.g129ede20d9-goog