[RFC/WIP PATCH 0/4] hazptr: add shared scan path and lockdep use case

[RFC/WIP PATCH 0/4] hazptr: add shared scan path and lockdep use case

[Kunwu Chan]

Hi all,

This RFC/WIP extends the current v3 hazptr implementation [1] and
adapts the lockdep use case from Boqun Feng's earlier hazptr series [2]
to the current hazptr API.

[1] https://lore.kernel.org/all/20260919000056.3132131-26-paulmck@kernel.org/
[2] https://lore.kernel.org/lkml/20250625031101.12555-1-boqun.feng@gmail.com/

The lockdep conversion replaces the expedited RCU wait in
lockdep_unregister_key() with hazptr_synchronize() for dynamic-key
lookups.

The series also adds a shared-scan kthread for concurrent
hazptr_synchronize() callers, rcuscale support, and an LKMM test
for the acquire-before-scan ordering.

I tested the lockdep path on a 96-CPU ARM64 KVM guest with Boqun's
original mq workload [2]. With a multiqueue virtio-net device
(`-device virtio-net-pci,mq=on,vectors=6`), tc mq add/del x100
completed in 850 ms with hazptr and 860 ms with expedited RCU.
The rmmod workload similarly showed no measurable difference
(1360 ms vs. 1380 ms over 10 runs). There were no crashes or hangs
in these tests.

This is still RFC/WIP. I would appreciate feedback on the shared-scan
design and the lockdep conversion.
 
Kunwu Chan (4):
  hazptr: add shared-scan kthread
  locking/lockdep: use hazptr to wait for dynamic key lookups
  rcuscale: add hazptr scale type
  Documentation/litmus-tests: add hazptr acquire-before-scan test

 .../hazptr/hazptr-acquire-before-scan.litmus  |  49 ++++
 kernel/hazptr.c                               | 274 ++++++++++++++++++
 kernel/locking/lockdep.c                      |  30 +-
 kernel/rcu/rcuscale.c                         |  65 ++++-
 4 files changed, 407 insertions(+), 11 deletions(-)
 create mode 100644 Documentation/litmus-tests/hazptr/hazptr-acquire-before-scan.litmus

-- 
2.43.0

[RFC/WIP PATCH 1/4] hazptr: add shared-scan kthread

[Kunwu Chan]

Batch concurrent hazptr_synchronize() callers into a shared scan
cycle, avoiding redundant scans of the per-CPU slots.

Queue waiters to a kthread and let each scan cycle make one pass
over all CPUs.  Each waiter tracks per-CPU progress for both
wildcard generations, allowing multiple waiters to share the same
scan.

Flip the wildcard before scanning.  New acquires then use the new
generation, so the old-generation mask makes forward progress even
under a steady stream of readers.  Waiters that remain blocked are
retried after a short delay.

Fall back to the existing direct two-phase scan if the scan kthread
is unavailable or waiter state cannot be allocated.

Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
---
 kernel/hazptr.c | 274 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 274 insertions(+)

diff --git a/kernel/hazptr.c b/kernel/hazptr.c
index d3d1050d92cf..ce553a61b119 100644
--- a/kernel/hazptr.c
+++ b/kernel/hazptr.c
@@ -12,6 +12,10 @@
 #include <linux/mutex.h>
 #include <linux/list.h>
 #include <linux/export.h>
+#include <linux/completion.h>
+#include <linux/kthread.h>
+#include <linux/slab.h>
+#include <linux/swait.h>
 
 /*
  * The current hazard pointer wildcard. Flips between 1UL and 2UL to guarantee
@@ -209,12 +213,251 @@ void hazptr_scan_period(void *addr, void *scan_wildcard)
 	}
 }
 
+/*
+ * Batch hazptr_synchronize() callers through a shared scan kthread.
+ */
+
+struct hazptr_waiter {
+	struct list_head node;
+	void *addr;
+	struct completion done;
+	/*
+	 * Per-wildcard-generation progress masks.  A CPU bit is
+	 * cleared when the scan observes neither @addr nor that
+	 * generation's wildcard on the CPU.
+	 */
+	unsigned long *cpu_mask;	/* 2 * BITS_TO_LONGS(nr_cpu_ids) */
+};
+
+/* Return waiter @w's progress mask for wildcard generation @gen. */
+static unsigned long *hazptr_waiter_mask(struct hazptr_waiter *w, int gen)
+{
+	return w->cpu_mask + gen * BITS_TO_LONGS(nr_cpu_ids);
+}
+
+struct hazptr_scan_state {
+	struct task_struct *kthread;
+	struct swait_queue_head wq;
+	bool wakeup;
+	struct mutex lock;
+	struct list_head pending;
+	struct list_head scanning;	/* kthread only */
+};
+static struct hazptr_scan_state hazptr_scan;
+
+/*
+ * Check a CPU's overflow lists.  A backup slot can hold a wildcard
+ * because __hazptr_acquire() writes the wildcard to any slot,
+ * including backup slots from hazptr_chain_backup_slot().
+ *
+ * @addr:     address the waiter is waiting on
+ * @old_wc:   wildcard value of the pre-flip generation
+ * @new_wc:   wildcard value of the post-flip generation
+ * @has_old:  set if any overflow slot holds @old_wc
+ * @has_new:  set if any overflow slot holds @new_wc
+ *
+ * Returns true if @addr is present.
+ */
+static bool hazptr_ovf_list_blocked(int cpu, void *addr,
+				    void *old_wc, void *new_wc,
+				    bool *has_old, bool *has_new)
+{
+	struct hazptr_overflow_list_flip *ovf = per_cpu_ptr(&percpu_overflow_list_flip, cpu);
+	bool found_addr = false;
+	int i;
+
+	for (i = 0; i < 2; i++) {
+		struct hazptr_overflow_list *list = &ovf->array[i];
+		struct hazptr_backup_slot *b;
+		unsigned long flags;
+
+		raw_spin_lock_irqsave(&list->lock, flags);
+		hlist_for_each_entry(b, &list->head, overflow_node) {
+			/* Pairs with smp_store_release in hazptr_release(). */
+			void *val = smp_load_acquire(&b->slot.addr);
+
+			if (val == addr)
+				found_addr = true;
+			else if (val == old_wc)
+				*has_old = true;
+			else if (val == new_wc)
+				*has_new = true;
+		}
+		raw_spin_unlock_irqrestore(&list->lock, flags);
+	}
+	return found_addr;
+}
+
+/*
+ * Move pending waiters to ->scanning, flip the wildcard, then make
+ * one pass over all CPUs.  Clear per-waiter bits for CPUs that no
+ * longer hold the waiter address or the corresponding wildcard.
+ *
+ * After the flip, new acquires use the new wildcard.  The old
+ * generation therefore makes forward progress and is fully cleared
+ * after enough scan cycles.
+ */
+static void hazptr_scan_do_cycle(void)
+{
+	void *old_wc, *new_wc;
+	unsigned int old_idx, new_idx;
+	int cpu;
+	struct hazptr_waiter *w, *n;
+	LIST_HEAD(done);
+
+	mutex_lock(&hazptr_wildcard_lock);
+
+	mutex_lock(&hazptr_scan.lock);
+	list_splice_tail_init(&hazptr_scan.pending, &hazptr_scan.scanning);
+	mutex_unlock(&hazptr_scan.lock);
+
+	if (list_empty(&hazptr_scan.scanning)) {
+		mutex_unlock(&hazptr_wildcard_lock);
+		return;
+	}
+
+	old_wc = READ_ONCE(hazptr_wildcard);
+	new_wc = flip_wildcard(old_wc);
+	WRITE_ONCE(hazptr_wildcard, new_wc);
+	old_idx = (unsigned long)old_wc - 1;
+	new_idx = 1 - old_idx;
+
+	/*
+	 * One pass over all CPUs for the per-CPU slots, checking
+	 * overflow lists for the remaining waiters.
+	 */
+	for_each_possible_cpu(cpu) {
+		struct hazptr_percpu_slots *slots = per_cpu_ptr(&hazptr_percpu_slots, cpu);
+		void *vals[NR_HAZPTR_PERCPU_SLOTS];
+		bool has_old = false, has_new = false;
+		unsigned int idx;
+
+		for (idx = 0; idx < NR_HAZPTR_PERCPU_SLOTS; idx++) {
+			/* Pairs with smp_store_release in hazptr_release(). */
+			vals[idx] = smp_load_acquire(&slots->items[idx].slot.addr);
+			if (vals[idx] == old_wc)
+				has_old = true;
+			else if (vals[idx] == new_wc)
+				has_new = true;
+		}
+
+		list_for_each_entry(w, &hazptr_scan.scanning, node) {
+			bool has_addr = false;
+
+			if (!test_bit(cpu, hazptr_waiter_mask(w, old_idx)) &&
+			    !test_bit(cpu, hazptr_waiter_mask(w, new_idx)))
+				continue;	/* Both bits already clear. */
+			for (idx = 0; idx < NR_HAZPTR_PERCPU_SLOTS; idx++) {
+				if (vals[idx] == w->addr) {
+					has_addr = true;
+					break;
+				}
+			}
+			if (!has_addr)
+				has_addr = hazptr_ovf_list_blocked(cpu, w->addr,
+					old_wc, new_wc, &has_old, &has_new);
+			if (has_addr)
+				continue;
+			if (!has_old)
+				__clear_bit(cpu, hazptr_waiter_mask(w, old_idx));
+			if (!has_new)
+				__clear_bit(cpu, hazptr_waiter_mask(w, new_idx));
+		}
+	}
+
+	mutex_unlock(&hazptr_wildcard_lock);
+
+	/* Complete waiters whose masks are both empty. */
+	list_for_each_entry_safe(w, n, &hazptr_scan.scanning, node) {
+		if (bitmap_empty(hazptr_waiter_mask(w, 0), nr_cpu_ids) &&
+		    bitmap_empty(hazptr_waiter_mask(w, 1), nr_cpu_ids))
+			list_move(&w->node, &done);
+	}
+
+	list_for_each_entry_safe(w, n, &done, node) {
+		list_del_init(&w->node);
+		complete(&w->done);
+	}
+}
+
+/*
+ * Shared scan kthread for hazptr_synchronize() waiters.
+ */
+static int hazptr_scan_kthread(void *unused)
+{
+	for (;;) {
+		bool idle;
+
+		swait_event_idle_exclusive(hazptr_scan.wq,
+					   READ_ONCE(hazptr_scan.wakeup));
+
+		hazptr_scan_do_cycle();
+
+		mutex_lock(&hazptr_scan.lock);
+		idle = list_empty(&hazptr_scan.pending) &&
+		      list_empty(&hazptr_scan.scanning);
+		if (idle)
+			WRITE_ONCE(hazptr_scan.wakeup, false);
+		mutex_unlock(&hazptr_scan.lock);
+
+		if (idle)
+			continue;
+		/* Waiters still blocked: retry after a polling delay. */
+		schedule_timeout_idle(1);
+	}
+	return 0;
+}
+
+/*
+ * Queue @addr for scan-thread processing, then sleep until the scan
+ * thread observes that @addr is no longer held by any hazard pointer.
+ * Returns false if the waiter masks cannot be allocated, in which
+ * case the caller falls back to the direct scan.
+ */
+static bool hazptr_synchronize_queued(void *addr)
+{
+	struct hazptr_waiter waiter = {
+		.addr = addr,
+	};
+	unsigned long *masks;
+	unsigned int mask_longs = BITS_TO_LONGS(nr_cpu_ids);
+
+	masks = kcalloc(2, mask_longs * sizeof(unsigned long), GFP_KERNEL);
+	if (!masks)
+		return false;
+	bitmap_fill(masks, nr_cpu_ids);
+	bitmap_fill(masks + mask_longs, nr_cpu_ids);
+	waiter.cpu_mask = masks;
+
+	init_completion(&waiter.done);
+	INIT_LIST_HEAD(&waiter.node);
+
+	/* Enqueue and wake the scan kthread. */
+	mutex_lock(&hazptr_scan.lock);
+	list_add_tail(&waiter.node, &hazptr_scan.pending);
+	if (!READ_ONCE(hazptr_scan.wakeup)) {
+		WRITE_ONCE(hazptr_scan.wakeup, true);
+		swake_up_one(&hazptr_scan.wq);
+	}
+	mutex_unlock(&hazptr_scan.lock);
+
+	/* Sleep until the scan thread completes this waiter. */
+	wait_for_completion(&waiter.done);
+	kfree(masks);
+	return true;
+}
+
 /*
  * hazptr_synchronize: Wait until @addr is released from all slots.
  *
  * Wait to observe that each slot contains a value that differs from
  * @addr before returning.
  * Should be called from preemptible context.
+ *
+ * If the scan kthread is running, the caller is queued and the scan
+ * thread performs the work, allowing multiple concurrent callers to
+ * share a single scan cycle.  Otherwise, the existing direct
+ * two-phase scan is used as a fallback.
  */
 void hazptr_synchronize(void *addr)
 {
@@ -235,6 +478,13 @@ void hazptr_synchronize(void *addr)
 	/* Memory ordering: Store A before Load B. */
 	smp_mb();
 
+	/* Use the scan thread if available. */
+	/* Pairs with smp_store_release in hazptr_scan_init(). */
+	if (smp_load_acquire(&hazptr_scan.kthread) &&
+	    hazptr_synchronize_queued(addr))
+		return;
+
+	/* Fallback: direct two-phase wildcard scan. */
 	guard(mutex)(&hazptr_wildcard_lock);
 	scan_wildcard = flip_wildcard(hazptr_wildcard);
 	hazptr_scan_period(addr, scan_wildcard);
@@ -282,3 +532,27 @@ void __init hazptr_init(void)
 		}
 	}
 }
+
+/*
+ * Initialize the scan kthread.  On failure falls back to the direct
+ * scan (busy-wait) path at synchronize time.
+ * core_initcall ensures the scheduler is ready before kthread_run.
+ */
+static int __init hazptr_scan_init(void)
+{
+	struct task_struct *t;
+
+	init_swait_queue_head(&hazptr_scan.wq);
+	mutex_init(&hazptr_scan.lock);
+	INIT_LIST_HEAD(&hazptr_scan.pending);
+	INIT_LIST_HEAD(&hazptr_scan.scanning);
+
+	t = kthread_run(hazptr_scan_kthread, NULL, "hazptr_scan");
+	if (!IS_ERR(t))
+		/* Pairs with smp_load_acquire in hazptr_synchronize(). */
+		smp_store_release(&hazptr_scan.kthread, t);
+	else
+		pr_warn("hazptr: scan thread failed, using direct scan\n");
+	return 0;
+}
+core_initcall(hazptr_scan_init);
-- 
2.43.0

Re: [RFC/WIP PATCH 1/4] hazptr: add shared-scan kthread

[Boqun Feng]

On Tue, Sep 22, 2026 at 03:09:47PM +0800, Kunwu Chan wrote:
> Batch concurrent hazptr_synchronize() callers into a shared scan
> cycle, avoiding redundant scans of the per-CPU slots.
> 
> Queue waiters to a kthread and let each scan cycle make one pass
> over all CPUs.  Each waiter tracks per-CPU progress for both
> wildcard generations, allowing multiple waiters to share the same
> scan.
> 
> Flip the wildcard before scanning.  New acquires then use the new
> generation, so the old-generation mask makes forward progress even
> under a steady stream of readers.  Waiters that remain blocked are
> retried after a short delay.
> 
> Fall back to the existing direct two-phase scan if the scan kthread
> is unavailable or waiter state cannot be allocated.
> 
> Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
> ---
>  kernel/hazptr.c | 274 ++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 274 insertions(+)
> 
> diff --git a/kernel/hazptr.c b/kernel/hazptr.c
> index d3d1050d92cf..ce553a61b119 100644
> --- a/kernel/hazptr.c
> +++ b/kernel/hazptr.c
> @@ -12,6 +12,10 @@
>  #include <linux/mutex.h>
>  #include <linux/list.h>
>  #include <linux/export.h>
> +#include <linux/completion.h>
> +#include <linux/kthread.h>
> +#include <linux/slab.h>
> +#include <linux/swait.h>
>  
>  /*
>   * The current hazard pointer wildcard. Flips between 1UL and 2UL to guarantee
> @@ -209,12 +213,251 @@ void hazptr_scan_period(void *addr, void *scan_wildcard)
>  	}
>  }
>  
> +/*
> + * Batch hazptr_synchronize() callers through a shared scan kthread.
> + */
> +
> +struct hazptr_waiter {
> +	struct list_head node;
> +	void *addr;
> +	struct completion done;
> +	/*
> +	 * Per-wildcard-generation progress masks.  A CPU bit is
> +	 * cleared when the scan observes neither @addr nor that
> +	 * generation's wildcard on the CPU.
> +	 */
> +	unsigned long *cpu_mask;	/* 2 * BITS_TO_LONGS(nr_cpu_ids) */

This would requires allocation during hazptr_synchronize() and I would
like to avoid that (it's going to introduce a "allocating memory to free
memory" case).

Mathieu brought up a useful data structure for the scan: A Bloom filter:

	https://en.wikipedia.org/wiki/Bloom_filter

, which is basically a bitmap set + k hash functions. Let's say we have
a struct bloom_filter (you can still with a page as the bitmap and k=3)
and put it in hazptr_scan_state. Then the scan would become:

	bloom_filter_clear(); // <- reset the bloom filer.

	for_each_possible_cpu()
          hlist_for_each_entry(b, &list->head, overflow_node) {
	    bloom_filter_set(*b->slot.addr);
	    // ^ add the hazptr_acquire() adress into the bloom filter
	  }
	
	list_for_each_entry(w, &hazptr_scan.scanning, node) {
	  if (!bloom_filter_contains(w->addr)) {
	    list_move(&w->node, &done);
	  }
	}

Of course, there are some additional handling or optimizaiton we can do
with the per-CPU slot and wildcard, but this is the idea. It also makes 
a potential call_hazptr() work.

Willing to give it a try?

Regards,
Boqun

> +};
> +
> +/* Return waiter @w's progress mask for wildcard generation @gen. */
> +static unsigned long *hazptr_waiter_mask(struct hazptr_waiter *w, int gen)
> +{
> +	return w->cpu_mask + gen * BITS_TO_LONGS(nr_cpu_ids);
> +}
> +
> +struct hazptr_scan_state {
> +	struct task_struct *kthread;
> +	struct swait_queue_head wq;
> +	bool wakeup;
> +	struct mutex lock;
> +	struct list_head pending;
> +	struct list_head scanning;	/* kthread only */
> +};
> +static struct hazptr_scan_state hazptr_scan;
> +
> +/*
> + * Check a CPU's overflow lists.  A backup slot can hold a wildcard
> + * because __hazptr_acquire() writes the wildcard to any slot,
> + * including backup slots from hazptr_chain_backup_slot().
> + *
> + * @addr:     address the waiter is waiting on
> + * @old_wc:   wildcard value of the pre-flip generation
> + * @new_wc:   wildcard value of the post-flip generation
> + * @has_old:  set if any overflow slot holds @old_wc
> + * @has_new:  set if any overflow slot holds @new_wc
> + *
> + * Returns true if @addr is present.
> + */
> +static bool hazptr_ovf_list_blocked(int cpu, void *addr,
> +				    void *old_wc, void *new_wc,
> +				    bool *has_old, bool *has_new)
> +{
> +	struct hazptr_overflow_list_flip *ovf = per_cpu_ptr(&percpu_overflow_list_flip, cpu);
> +	bool found_addr = false;
> +	int i;
> +
> +	for (i = 0; i < 2; i++) {
> +		struct hazptr_overflow_list *list = &ovf->array[i];
> +		struct hazptr_backup_slot *b;
> +		unsigned long flags;
> +
> +		raw_spin_lock_irqsave(&list->lock, flags);
> +		hlist_for_each_entry(b, &list->head, overflow_node) {
> +			/* Pairs with smp_store_release in hazptr_release(). */
> +			void *val = smp_load_acquire(&b->slot.addr);
> +
> +			if (val == addr)
> +				found_addr = true;
> +			else if (val == old_wc)
> +				*has_old = true;
> +			else if (val == new_wc)
> +				*has_new = true;
> +		}
> +		raw_spin_unlock_irqrestore(&list->lock, flags);
> +	}
> +	return found_addr;
> +}
> +
> +/*
> + * Move pending waiters to ->scanning, flip the wildcard, then make
> + * one pass over all CPUs.  Clear per-waiter bits for CPUs that no
> + * longer hold the waiter address or the corresponding wildcard.
> + *
> + * After the flip, new acquires use the new wildcard.  The old
> + * generation therefore makes forward progress and is fully cleared
> + * after enough scan cycles.
> + */
> +static void hazptr_scan_do_cycle(void)
> +{
> +	void *old_wc, *new_wc;
> +	unsigned int old_idx, new_idx;
> +	int cpu;
> +	struct hazptr_waiter *w, *n;
> +	LIST_HEAD(done);
> +
> +	mutex_lock(&hazptr_wildcard_lock);
> +
> +	mutex_lock(&hazptr_scan.lock);
> +	list_splice_tail_init(&hazptr_scan.pending, &hazptr_scan.scanning);
> +	mutex_unlock(&hazptr_scan.lock);
> +
> +	if (list_empty(&hazptr_scan.scanning)) {
> +		mutex_unlock(&hazptr_wildcard_lock);
> +		return;
> +	}
> +
> +	old_wc = READ_ONCE(hazptr_wildcard);
> +	new_wc = flip_wildcard(old_wc);
> +	WRITE_ONCE(hazptr_wildcard, new_wc);
> +	old_idx = (unsigned long)old_wc - 1;
> +	new_idx = 1 - old_idx;
> +
> +	/*
> +	 * One pass over all CPUs for the per-CPU slots, checking
> +	 * overflow lists for the remaining waiters.
> +	 */
> +	for_each_possible_cpu(cpu) {
> +		struct hazptr_percpu_slots *slots = per_cpu_ptr(&hazptr_percpu_slots, cpu);
> +		void *vals[NR_HAZPTR_PERCPU_SLOTS];
> +		bool has_old = false, has_new = false;
> +		unsigned int idx;
> +
> +		for (idx = 0; idx < NR_HAZPTR_PERCPU_SLOTS; idx++) {
> +			/* Pairs with smp_store_release in hazptr_release(). */
> +			vals[idx] = smp_load_acquire(&slots->items[idx].slot.addr);
> +			if (vals[idx] == old_wc)
> +				has_old = true;
> +			else if (vals[idx] == new_wc)
> +				has_new = true;
> +		}
> +
> +		list_for_each_entry(w, &hazptr_scan.scanning, node) {
> +			bool has_addr = false;
> +
> +			if (!test_bit(cpu, hazptr_waiter_mask(w, old_idx)) &&
> +			    !test_bit(cpu, hazptr_waiter_mask(w, new_idx)))
> +				continue;	/* Both bits already clear. */
> +			for (idx = 0; idx < NR_HAZPTR_PERCPU_SLOTS; idx++) {
> +				if (vals[idx] == w->addr) {
> +					has_addr = true;
> +					break;
> +				}
> +			}
> +			if (!has_addr)
> +				has_addr = hazptr_ovf_list_blocked(cpu, w->addr,
> +					old_wc, new_wc, &has_old, &has_new);
> +			if (has_addr)
> +				continue;
> +			if (!has_old)
> +				__clear_bit(cpu, hazptr_waiter_mask(w, old_idx));
> +			if (!has_new)
> +				__clear_bit(cpu, hazptr_waiter_mask(w, new_idx));
> +		}
> +	}
> +
> +	mutex_unlock(&hazptr_wildcard_lock);
> +
> +	/* Complete waiters whose masks are both empty. */
> +	list_for_each_entry_safe(w, n, &hazptr_scan.scanning, node) {
> +		if (bitmap_empty(hazptr_waiter_mask(w, 0), nr_cpu_ids) &&
> +		    bitmap_empty(hazptr_waiter_mask(w, 1), nr_cpu_ids))
> +			list_move(&w->node, &done);
> +	}
> +
> +	list_for_each_entry_safe(w, n, &done, node) {
> +		list_del_init(&w->node);
> +		complete(&w->done);
> +	}
> +}
> +
> +/*
> + * Shared scan kthread for hazptr_synchronize() waiters.
> + */
> +static int hazptr_scan_kthread(void *unused)
> +{
> +	for (;;) {
> +		bool idle;
> +
> +		swait_event_idle_exclusive(hazptr_scan.wq,
> +					   READ_ONCE(hazptr_scan.wakeup));
> +
> +		hazptr_scan_do_cycle();
> +
> +		mutex_lock(&hazptr_scan.lock);
> +		idle = list_empty(&hazptr_scan.pending) &&
> +		      list_empty(&hazptr_scan.scanning);
> +		if (idle)
> +			WRITE_ONCE(hazptr_scan.wakeup, false);
> +		mutex_unlock(&hazptr_scan.lock);
> +
> +		if (idle)
> +			continue;
> +		/* Waiters still blocked: retry after a polling delay. */
> +		schedule_timeout_idle(1);
> +	}
> +	return 0;
> +}
> +
> +/*
> + * Queue @addr for scan-thread processing, then sleep until the scan
> + * thread observes that @addr is no longer held by any hazard pointer.
> + * Returns false if the waiter masks cannot be allocated, in which
> + * case the caller falls back to the direct scan.
> + */
> +static bool hazptr_synchronize_queued(void *addr)
> +{
> +	struct hazptr_waiter waiter = {
> +		.addr = addr,
> +	};
> +	unsigned long *masks;
> +	unsigned int mask_longs = BITS_TO_LONGS(nr_cpu_ids);
> +
> +	masks = kcalloc(2, mask_longs * sizeof(unsigned long), GFP_KERNEL);
> +	if (!masks)
> +		return false;
> +	bitmap_fill(masks, nr_cpu_ids);
> +	bitmap_fill(masks + mask_longs, nr_cpu_ids);
> +	waiter.cpu_mask = masks;
> +
> +	init_completion(&waiter.done);
> +	INIT_LIST_HEAD(&waiter.node);
> +
> +	/* Enqueue and wake the scan kthread. */
> +	mutex_lock(&hazptr_scan.lock);
> +	list_add_tail(&waiter.node, &hazptr_scan.pending);
> +	if (!READ_ONCE(hazptr_scan.wakeup)) {
> +		WRITE_ONCE(hazptr_scan.wakeup, true);
> +		swake_up_one(&hazptr_scan.wq);
> +	}
> +	mutex_unlock(&hazptr_scan.lock);
> +
> +	/* Sleep until the scan thread completes this waiter. */
> +	wait_for_completion(&waiter.done);
> +	kfree(masks);
> +	return true;
> +}
> +
>  /*
>   * hazptr_synchronize: Wait until @addr is released from all slots.
>   *
>   * Wait to observe that each slot contains a value that differs from
>   * @addr before returning.
>   * Should be called from preemptible context.
> + *
> + * If the scan kthread is running, the caller is queued and the scan
> + * thread performs the work, allowing multiple concurrent callers to
> + * share a single scan cycle.  Otherwise, the existing direct
> + * two-phase scan is used as a fallback.
>   */
>  void hazptr_synchronize(void *addr)
>  {
> @@ -235,6 +478,13 @@ void hazptr_synchronize(void *addr)
>  	/* Memory ordering: Store A before Load B. */
>  	smp_mb();
>  
> +	/* Use the scan thread if available. */
> +	/* Pairs with smp_store_release in hazptr_scan_init(). */
> +	if (smp_load_acquire(&hazptr_scan.kthread) &&
> +	    hazptr_synchronize_queued(addr))
> +		return;
> +
> +	/* Fallback: direct two-phase wildcard scan. */
>  	guard(mutex)(&hazptr_wildcard_lock);
>  	scan_wildcard = flip_wildcard(hazptr_wildcard);
>  	hazptr_scan_period(addr, scan_wildcard);
> @@ -282,3 +532,27 @@ void __init hazptr_init(void)
>  		}
>  	}
>  }
> +
> +/*
> + * Initialize the scan kthread.  On failure falls back to the direct
> + * scan (busy-wait) path at synchronize time.
> + * core_initcall ensures the scheduler is ready before kthread_run.
> + */
> +static int __init hazptr_scan_init(void)
> +{
> +	struct task_struct *t;
> +
> +	init_swait_queue_head(&hazptr_scan.wq);
> +	mutex_init(&hazptr_scan.lock);
> +	INIT_LIST_HEAD(&hazptr_scan.pending);
> +	INIT_LIST_HEAD(&hazptr_scan.scanning);
> +
> +	t = kthread_run(hazptr_scan_kthread, NULL, "hazptr_scan");
> +	if (!IS_ERR(t))
> +		/* Pairs with smp_load_acquire in hazptr_synchronize(). */
> +		smp_store_release(&hazptr_scan.kthread, t);
> +	else
> +		pr_warn("hazptr: scan thread failed, using direct scan\n");
> +	return 0;
> +}
> +core_initcall(hazptr_scan_init);
> -- 
> 2.43.0
> 

Re: [RFC/WIP PATCH 1/4] hazptr: add shared-scan kthread

[Lian Wang]

Hi Boqun,

Thanks for the suggestion. Kunwu and I will work together on the next
iteration of the shared-scan design, including your idea of avoiding
per-synchronize allocations with a Bloom filter. We will also run the
rcuscale comparisons and collect IPI counts for the lockdep workloads
you suggested, then share the results with the thread.

Thanks,
Lian

On Tue, 22 Sep 2026 10:28:24 +0200 Boqun Feng <boqun@kernel.org> wrote:

> This would requires allocation during hazptr_synchronize() and I would
> like to avoid that (it's going to introduce a "allocating memory to free
> memory" case).
>
> Mathieu brought up a useful data structure for the scan: A Bloom filter:
>
> Willing to give it a try?

[RFC/WIP PATCH 2/4] locking/lockdep: use hazptr to wait for dynamic key lookups

[Kunwu Chan]

lockdep_unregister_key() waits for is_dynamic_key() callers with
synchronize_rcu_expedited(), which sends IPIs to every online CPU.
Have is_dynamic_key() mark the hash bucket with a hazard pointer
and use hazptr_synchronize() to wait specifically for those
traversals.

The hash bucket address from keyhashentry() is stable, making it a
suitable hazptr synchronize target.  The rest of the key hashlist
lifetime (hlist_del_rcu/call_rcu) remains RCU-based.

This adapts the lockdep use case from Boqun Feng's hazard-pointer
series to the current hazptr API.

Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
---
 kernel/locking/lockdep.c | 30 ++++++++++++++++++++----------
 1 file changed, 20 insertions(+), 10 deletions(-)

diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c
index c56a7f91d72e..f67d847f9abf 100644
--- a/kernel/locking/lockdep.c
+++ b/kernel/locking/lockdep.c
@@ -58,6 +58,7 @@
 #include <linux/context_tracking.h>
 #include <linux/console.h>
 #include <linux/kasan.h>
+#include <linux/hazptr.h>
 
 #include <asm/sections.h>
 
@@ -1280,14 +1281,24 @@ static bool is_dynamic_key(const struct lock_class_key *key)
 
 	hash_head = keyhashentry(key);
 
-	rcu_read_lock();
-	hlist_for_each_entry_rcu(k, hash_head, hash_entry) {
-		if (k == key) {
-			found = true;
-			break;
+	/*
+	 * The traversal is protected by a hazard pointer rather
+	 * than an RCU read-side critical section.
+	 */
+	{
+		struct hazptr_ctx ctx;
+		void *bucket = hash_head;
+		void *addr;
+
+		addr = hazptr_acquire(&ctx, &bucket);
+		hlist_for_each_entry_rcu(k, hash_head, hash_entry, 1) {
+			if (k == key) {
+				found = true;
+				break;
+			}
 		}
+		hazptr_release(&ctx, addr);
 	}
-	rcu_read_unlock();
 
 	return found;
 }
@@ -6683,11 +6694,10 @@ void lockdep_unregister_key(struct lock_class_key *key)
 	 *
 	 * Some operations like __qdisc_destroy() will call this in a debug
 	 * kernel, and the network traffic is disabled while waiting, hence
-	 * the delay of the wait matters in debugging cases. Currently use a
-	 * synchronize_rcu_expedited() to speed up the wait at the cost of
-	 * system IPIs. TODO: Replace RCU with hazptr for this.
+	 * the delay of the wait matters in debugging cases. Replace the
+	 * expedited RCU wait with hazptr_synchronize().
 	 */
-	synchronize_rcu_expedited();
+	hazptr_synchronize(keyhashentry(key));
 }
 EXPORT_SYMBOL_GPL(lockdep_unregister_key);
 
-- 
2.43.0

Re: [RFC/WIP PATCH 2/4] locking/lockdep: use hazptr to wait for dynamic key lookups

[Boqun Feng]

On Tue, Sep 22, 2026 at 03:09:48PM +0800, Kunwu Chan wrote:
> lockdep_unregister_key() waits for is_dynamic_key() callers with
> synchronize_rcu_expedited(), which sends IPIs to every online CPU.
> Have is_dynamic_key() mark the hash bucket with a hazard pointer
> and use hazptr_synchronize() to wait specifically for those
> traversals.
> 
> The hash bucket address from keyhashentry() is stable, making it a
> suitable hazptr synchronize target.  The rest of the key hashlist
> lifetime (hlist_del_rcu/call_rcu) remains RCU-based.
> 
> This adapts the lockdep use case from Boqun Feng's hazard-pointer
> series to the current hazptr API.
> 
> Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
> ---
>  kernel/locking/lockdep.c | 30 ++++++++++++++++++++----------
>  1 file changed, 20 insertions(+), 10 deletions(-)
> 
> diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c
> index c56a7f91d72e..f67d847f9abf 100644
> --- a/kernel/locking/lockdep.c
> +++ b/kernel/locking/lockdep.c
> @@ -58,6 +58,7 @@
>  #include <linux/context_tracking.h>
>  #include <linux/console.h>
>  #include <linux/kasan.h>
> +#include <linux/hazptr.h>
>  
>  #include <asm/sections.h>
>  
> @@ -1280,14 +1281,24 @@ static bool is_dynamic_key(const struct lock_class_key *key)
>  
>  	hash_head = keyhashentry(key);
>  
> -	rcu_read_lock();
> -	hlist_for_each_entry_rcu(k, hash_head, hash_entry) {
> -		if (k == key) {
> -			found = true;
> -			break;
> +	/*
> +	 * The traversal is protected by a hazard pointer rather
> +	 * than an RCU read-side critical section.
> +	 */
> +	{
> +		struct hazptr_ctx ctx;
> +		void *bucket = hash_head;
> +		void *addr;
> +
> +		addr = hazptr_acquire(&ctx, &bucket);
> +		hlist_for_each_entry_rcu(k, hash_head, hash_entry, 1) {
> +			if (k == key) {
> +				found = true;
> +				break;
> +			}
>  		}
> +		hazptr_release(&ctx, addr);

This looks good to me. However, we probably want to use scoped_guard()
here, that means cleanup.h support for hazptr.

The other thing that could be added is a debug option that force we
skip the fast path, so we always go into the show path in
hazptr_acquire(). This occurs to me because the slow path here means
acquiring a lock inside lockdep code, it should work, but I just want to
be careful. So something like:

	void *hazptr_acquire(..)
	{
		...
		if (IS_ENABLED(CONFIG_HAZPTR_ACQUIRE_FORCE_SLOWPATH) ||
		    unlikely(slot->addr))
			return __hazptr_acquire(ctx, addr_p);
		...
	}

Thoughts?

Regards,
Boqun

>  	}
> -	rcu_read_unlock();
>  
>  	return found;
>  }
> @@ -6683,11 +6694,10 @@ void lockdep_unregister_key(struct lock_class_key *key)
>  	 *
>  	 * Some operations like __qdisc_destroy() will call this in a debug
>  	 * kernel, and the network traffic is disabled while waiting, hence
> -	 * the delay of the wait matters in debugging cases. Currently use a
> -	 * synchronize_rcu_expedited() to speed up the wait at the cost of
> -	 * system IPIs. TODO: Replace RCU with hazptr for this.
> +	 * the delay of the wait matters in debugging cases. Replace the
> +	 * expedited RCU wait with hazptr_synchronize().
>  	 */
> -	synchronize_rcu_expedited();
> +	hazptr_synchronize(keyhashentry(key));
>  }
>  EXPORT_SYMBOL_GPL(lockdep_unregister_key);
>  
> -- 
> 2.43.0
> 

[RFC/WIP PATCH 3/4] rcuscale: add hazptr scale type

[Kunwu Chan]

Add hazptr reader and synchronize operations so that synchronize
latency can be measured alongside RCU and SRCU.

The read side acquires the hazard pointer in readlock() and holds
it until readunlock(), matching the RCU/SRCU reader model.  The
address of a static object serves as the synchronize target, which
is stable and never reclaimed.  Both normal and expedited sync map
to hazptr_synchronize(), since hazptr has no expedited concept.

Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
---
 kernel/rcu/rcuscale.c | 65 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 64 insertions(+), 1 deletion(-)

diff --git a/kernel/rcu/rcuscale.c b/kernel/rcu/rcuscale.c
index 1097ec15879c..072ddf9526c3 100644
--- a/kernel/rcu/rcuscale.c
+++ b/kernel/rcu/rcuscale.c
@@ -39,6 +39,7 @@
 #include <linux/torture.h>
 #include <linux/vmalloc.h>
 #include <linux/rcupdate_trace.h>
+#include <linux/hazptr.h>
 #include <linux/sched/debug.h>
 
 #include "rcu.h"
@@ -418,6 +419,66 @@ static struct rcu_scale_ops tasks_tracing_ops = {
 
 #endif // #else // #ifdef CONFIG_TASKS_TRACE_RCU
 
+#if IS_ENABLED(CONFIG_HAZPTR_TORTURE_TEST)
+
+static int hazptr_scale_obj;	/* Stable, non-NULL, never reclaimed. */
+static void *hazptr_scale_ptr = &hazptr_scale_obj;
+
+struct hazptr_scale_state {
+	struct hazptr_ctx ctx;
+	void *addr;
+};
+static DEFINE_PER_CPU(struct hazptr_scale_state, hazptr_scale_state);
+
+static int hazptr_scale_read_lock(void)
+{
+	struct hazptr_scale_state *state = this_cpu_ptr(&hazptr_scale_state);
+
+	preempt_disable();
+	state->addr = hazptr_acquire(&state->ctx, &hazptr_scale_ptr);
+	return 0;
+}
+
+static void hazptr_scale_read_unlock(int idx)
+{
+	struct hazptr_scale_state *state = this_cpu_ptr(&hazptr_scale_state);
+
+	udelay(10);
+	hazptr_release(&state->ctx, state->addr);
+	preempt_enable();
+}
+
+static unsigned long hazptr_scale_completed(void)
+{
+	return 0;
+}
+
+static void hazptr_scale_sync(void)
+{
+	hazptr_synchronize(hazptr_scale_ptr);
+}
+
+static void hazptr_scale_sync_exp(void)
+{
+	hazptr_synchronize(hazptr_scale_ptr);
+}
+
+static struct rcu_scale_ops hazptr_scale_ops = {
+	.ptype		= 0,
+	.readlock	= hazptr_scale_read_lock,
+	.readunlock	= hazptr_scale_read_unlock,
+	.get_gp_seq	= hazptr_scale_completed,
+	.gp_diff	= NULL,
+	.sync		= hazptr_scale_sync,
+	.exp_sync	= hazptr_scale_sync_exp,
+	.name		= "hazptr",
+};
+
+#define HAZPTR_SCALE_OPS &hazptr_scale_ops,
+#else
+#define HAZPTR_SCALE_OPS
+#endif
+
 static unsigned long rcuscale_seq_diff(unsigned long new, unsigned long old)
 {
 	if (!cur_ops->gp_diff)
@@ -1110,7 +1171,9 @@ rcu_scale_init(void)
 	long i;
 	long j;
 	static struct rcu_scale_ops *scale_ops[] = {
-		&rcu_ops, &srcu_ops, &srcud_ops, TASKS_OPS TASKS_RUDE_OPS TASKS_TRACING_OPS
+		&rcu_ops, &srcu_ops, &srcud_ops,
+		TASKS_OPS TASKS_RUDE_OPS TASKS_TRACING_OPS
+		HAZPTR_SCALE_OPS
 	};
 
 	if (!torture_init_begin(scale_type, verbose))
-- 
2.43.0

Re: [RFC/WIP PATCH 3/4] rcuscale: add hazptr scale type

[Boqun Feng]

On Tue, Sep 22, 2026 at 03:09:49PM +0800, Kunwu Chan wrote:
> Add hazptr reader and synchronize operations so that synchronize
> latency can be measured alongside RCU and SRCU.
> 
> The read side acquires the hazard pointer in readlock() and holds
> it until readunlock(), matching the RCU/SRCU reader model.  The
> address of a static object serves as the synchronize target, which
> is stable and never reclaimed.  Both normal and expedited sync map
> to hazptr_synchronize(), since hazptr has no expedited concept.
> 
> Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
> ---
>  kernel/rcu/rcuscale.c | 65 ++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 64 insertions(+), 1 deletion(-)
> 
> diff --git a/kernel/rcu/rcuscale.c b/kernel/rcu/rcuscale.c
> index 1097ec15879c..072ddf9526c3 100644
> --- a/kernel/rcu/rcuscale.c
> +++ b/kernel/rcu/rcuscale.c
> @@ -39,6 +39,7 @@
>  #include <linux/torture.h>
>  #include <linux/vmalloc.h>
>  #include <linux/rcupdate_trace.h>
> +#include <linux/hazptr.h>
>  #include <linux/sched/debug.h>
>  
>  #include "rcu.h"
> @@ -418,6 +419,66 @@ static struct rcu_scale_ops tasks_tracing_ops = {
>  
>  #endif // #else // #ifdef CONFIG_TASKS_TRACE_RCU
>  
> +#if IS_ENABLED(CONFIG_HAZPTR_TORTURE_TEST)
> +
> +static int hazptr_scale_obj;	/* Stable, non-NULL, never reclaimed. */

You can probaly make hazptr_scale_obj an arrary, and let an updater
either randomly or round-robin select an object to wait, it'll reflect
better to a real world workload.

> +static void *hazptr_scale_ptr = &hazptr_scale_obj;
> +
> +struct hazptr_scale_state {
> +	struct hazptr_ctx ctx;
> +	void *addr;
> +};
> +static DEFINE_PER_CPU(struct hazptr_scale_state, hazptr_scale_state);
> +
> +static int hazptr_scale_read_lock(void)
> +{
> +	struct hazptr_scale_state *state = this_cpu_ptr(&hazptr_scale_state);
> +
> +	preempt_disable();

I think you can drop the preempt_disable() and preempt_enable() below,
since the new hazptr_acquire()/hazptr_release() work without them (i.e.
the hazptr acquisition no longer requires preemption disable).

Regards,
Boqun

> +	state->addr = hazptr_acquire(&state->ctx, &hazptr_scale_ptr);
> +	return 0;
> +}
> +
> +static void hazptr_scale_read_unlock(int idx)
> +{
> +	struct hazptr_scale_state *state = this_cpu_ptr(&hazptr_scale_state);
> +
> +	udelay(10);
> +	hazptr_release(&state->ctx, state->addr);
> +	preempt_enable();
> +}
> +
> +static unsigned long hazptr_scale_completed(void)
> +{
> +	return 0;
> +}
> +
> +static void hazptr_scale_sync(void)
> +{
> +	hazptr_synchronize(hazptr_scale_ptr);
> +}
> +
> +static void hazptr_scale_sync_exp(void)
> +{
> +	hazptr_synchronize(hazptr_scale_ptr);
> +}
> +
> +static struct rcu_scale_ops hazptr_scale_ops = {
> +	.ptype		= 0,
> +	.readlock	= hazptr_scale_read_lock,
> +	.readunlock	= hazptr_scale_read_unlock,
> +	.get_gp_seq	= hazptr_scale_completed,
> +	.gp_diff	= NULL,
> +	.sync		= hazptr_scale_sync,
> +	.exp_sync	= hazptr_scale_sync_exp,
> +	.name		= "hazptr",
> +};
> +
> +#define HAZPTR_SCALE_OPS &hazptr_scale_ops,
> +#else
> +#define HAZPTR_SCALE_OPS
> +#endif
> +
>  static unsigned long rcuscale_seq_diff(unsigned long new, unsigned long old)
>  {
>  	if (!cur_ops->gp_diff)
> @@ -1110,7 +1171,9 @@ rcu_scale_init(void)
>  	long i;
>  	long j;
>  	static struct rcu_scale_ops *scale_ops[] = {
> -		&rcu_ops, &srcu_ops, &srcud_ops, TASKS_OPS TASKS_RUDE_OPS TASKS_TRACING_OPS
> +		&rcu_ops, &srcu_ops, &srcud_ops,
> +		TASKS_OPS TASKS_RUDE_OPS TASKS_TRACING_OPS
> +		HAZPTR_SCALE_OPS
>  	};
>  
>  	if (!torture_init_begin(scale_type, verbose))
> -- 
> 2.43.0
> 

[RFC/WIP PATCH 4/4] Documentation/litmus-tests: add hazptr acquire-before-scan test

[Kunwu Chan]

Add an LKMM test for the hazard-pointer publication protocol used
by the lockdep conversion.

The test checks that the reclaimer cannot miss a reader hazard
pointer publication while the reader still observes the
pre-unpublish pointer.  The smp_mb() pair provides the required
ordering.

Models the resolved-publication case; the in-flight wildcard window
is covered by the two-phase wildcard scan of the v3 implementation.

Verified with herd7: Never 0 6.

Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
---
 .../hazptr/hazptr-acquire-before-scan.litmus  | 49 +++++++++++++++++++
 1 file changed, 49 insertions(+)
 create mode 100644 Documentation/litmus-tests/hazptr/hazptr-acquire-before-scan.litmus

diff --git a/Documentation/litmus-tests/hazptr/hazptr-acquire-before-scan.litmus b/Documentation/litmus-tests/hazptr/hazptr-acquire-before-scan.litmus
new file mode 100644
index 000000000000..19df97f1c3b2
--- /dev/null
+++ b/Documentation/litmus-tests/hazptr/hazptr-acquire-before-scan.litmus
@@ -0,0 +1,49 @@
+C hazptr-acquire-before-scan
+
+(*
+ * Result: Never
+ *
+ * The reclaimer unpublishes the pointer, executes smp_mb(), then
+ * scans the hazard-pointer slot.  The reader publishes the
+ * protected address, executes smp_mb(), then loads the pointer.
+ *
+ * The smp_mb() pair forbids the reclaimer from missing the
+ * publication while the reader still observes the pre-unpublish
+ * pointer.
+ *
+ * This is the publication protocol used by the lockdep
+ * is_dynamic_key()/lockdep_unregister_key() conversion.
+ *
+ * Models the resolved-publication case; the in-flight wildcard
+ * window is covered by the two-phase wildcard scan of the v3
+ * implementation.
+ *)
+
+{
+int ptr = 1;	(* 1: points to the object, 0: unpublished. *)
+int hp = 0;	(* Hazard-pointer slot: 0: empty, 1: holds the addr. *)
+int data = 1;	(* Object payload; 0: reclaimed. *)
+}
+
+P0(int *ptr, int *hp, int *data)
+{
+	int r0;
+
+	WRITE_ONCE(*ptr, 0);	/* Store A: unpublish. */
+	smp_mb();
+	r0 = READ_ONCE(*hp);	/* Load B: scan. */
+	WRITE_ONCE(*data, 0);	/* Reclaim. */
+}
+
+P1(int *ptr, int *hp, int *data)
+{
+	int r0;
+	int r1;
+
+	WRITE_ONCE(*hp, 1);	/* Store B: publish. */
+	smp_mb();
+	r0 = READ_ONCE(*ptr);	/* Load A: load pointer. */
+	r1 = READ_ONCE(*data);	/* Access object. */
+}
+
+exists (0:r0=0 /\ 1:r0=1)
-- 
2.43.0

Re: [RFC/WIP PATCH 0/4] hazptr: add shared scan path and lockdep use case

[Boqun Feng]

On Tue, Sep 22, 2026 at 03:09:46PM +0800, Kunwu Chan wrote:
> Hi all,
> 

Hello Kunwu,

> This RFC/WIP extends the current v3 hazptr implementation [1] and
> adapts the lockdep use case from Boqun Feng's earlier hazptr series [2]
> to the current hazptr API.
> 
> [1] https://lore.kernel.org/all/20260919000056.3132131-26-paulmck@kernel.org/
> [2] https://lore.kernel.org/lkml/20250625031101.12555-1-boqun.feng@gmail.com/
> 
> The lockdep conversion replaces the expedited RCU wait in
> lockdep_unregister_key() with hazptr_synchronize() for dynamic-key
> lookups.
> 
> The series also adds a shared-scan kthread for concurrent
> hazptr_synchronize() callers, rcuscale support, and an LKMM test
> for the acquire-before-scan ordering.
> 

Thanks a lot for picking up this, much appreciated.

Do you happen to play a bit with rcuscale and see any performance
difference between hazptr_synchronize() vs synchronize_{s}rcu() on
various configs?

> I tested the lockdep path on a 96-CPU ARM64 KVM guest with Boqun's
> original mq workload [2]. With a multiqueue virtio-net device
> (`-device virtio-net-pci,mq=on,vectors=6`), tc mq add/del x100
> completed in 850 ms with hazptr and 860 ms with expedited RCU.
> The rmmod workload similarly showed no measurable difference
> (1360 ms vs. 1380 ms over 10 runs). There were no crashes or hangs
> in these tests.
> 

You could also use /proc/interrupts to measure the IPI counts during the
whole operation, and you should be able to see the difference there.

Regards,
Boqun

> This is still RFC/WIP. I would appreciate feedback on the shared-scan
> design and the lockdep conversion.
>  
> Kunwu Chan (4):
>   hazptr: add shared-scan kthread
>   locking/lockdep: use hazptr to wait for dynamic key lookups
>   rcuscale: add hazptr scale type
>   Documentation/litmus-tests: add hazptr acquire-before-scan test
> 
>  .../hazptr/hazptr-acquire-before-scan.litmus  |  49 ++++
>  kernel/hazptr.c                               | 274 ++++++++++++++++++
>  kernel/locking/lockdep.c                      |  30 +-
>  kernel/rcu/rcuscale.c                         |  65 ++++-
>  4 files changed, 407 insertions(+), 11 deletions(-)
>  create mode 100644 Documentation/litmus-tests/hazptr/hazptr-acquire-before-scan.litmus
> 
> -- 
> 2.43.0
> 

Re: [RFC/WIP PATCH 0/4] hazptr: add shared scan path and lockdep use case

[KunWu Chan]

On Tue, Sep 22, 2026 at 3:48 PM Boqun Feng <boqun@kernel.org> wrote:
>
> On Tue, Sep 22, 2026 at 03:09:46PM +0800, Kunwu Chan wrote:
> > Hi all,
> >
>
> Hello Kunwu,
>
> > This RFC/WIP extends the current v3 hazptr implementation [1] and
> > adapts the lockdep use case from Boqun Feng's earlier hazptr series [2]
> > to the current hazptr API.
> >
> > [1] https://lore.kernel.org/all/20260919000056.3132131-26-paulmck@kernel.org/
> > [2] https://lore.kernel.org/lkml/20250625031101.12555-1-boqun.feng@gmail.com/
> >
> > The lockdep conversion replaces the expedited RCU wait in
> > lockdep_unregister_key() with hazptr_synchronize() for dynamic-key
> > lookups.
> >
> > The series also adds a shared-scan kthread for concurrent
> > hazptr_synchronize() callers, rcuscale support, and an LKMM test
> > for the acquire-before-scan ordering.
> >
>
> Thanks a lot for picking up this, much appreciated.
>
> Do you happen to play a bit with rcuscale and see any performance
> difference between hazptr_synchronize() vs synchronize_{s}rcu() on
> various configs?
>
> > I tested the lockdep path on a 96-CPU ARM64 KVM guest with Boqun's
> > original mq workload [2]. With a multiqueue virtio-net device
> > (`-device virtio-net-pci,mq=on,vectors=6`), tc mq add/del x100
> > completed in 850 ms with hazptr and 860 ms with expedited RCU.
> > The rmmod workload similarly showed no measurable difference
> > (1360 ms vs. 1380 ms over 10 runs). There were no crashes or hangs
> > in these tests.
> >
>
> You could also use /proc/interrupts to measure the IPI counts during the
> whole operation, and you should be able to see the difference there.

Hi Boqun,

Thanks for the suggestions.

I collected both the rcuscale synchronization latency and the
/proc/interrupts IPI counts from the current implementation.

On a 96-CPU ARM64 KVM guest, the rcuscale results with
nreaders=0 are:

                avg       p50       p90       p99
nwriters=1:
hazptr          98 us     96 us     101 us    110 us
rcu             8.4 ms    8.0 ms     8.0 ms   16.0 ms
srcu            8.0 ms   15.9 ms    16.0 ms   16.0 ms

nwriters=16:
hazptr          8.0 ms    8.0 ms     8.0 ms    8.1 ms
rcu            14.9 ms   16.0 ms    16.0 ms   24.0 ms

With a single writer, hazptr completes in ~100 us. With 16
concurrent synchronizers, the latency approaches ~8 ms, while
RCU rises from ~8 ms to ~15 ms. The shared-scan batching
appears to be effective here, while the scan-kthread polling
mechanism has a significant impact on the completion latency
once multiple waiters are present. This also motivates the
Bloom-filter redesign you suggested.

For the lockdep workload (tc qdisc mq x100):

                wall-clock    IPI/op
hazptr          ~810 ms        5.1
exp RCU         ~790 ms       12.3

The hazptr path shows substantially lower IPI activity on this
workload (~41% of expedited RCU), though the wall-clock time is
similar. The lower IPI activity does not translate into a
measurable wall-clock improvement in this workload.

I'll update the shared-scan design based on your Bloom-filter
suggestion, make the rcuscale target cover multiple objects,
and add the forced-slowpath debug option. I'll then rerun
the measurements with the updated implementation.

Thanks,
Kunwu

>
> Regards,
> Boqun
>
> > This is still RFC/WIP. I would appreciate feedback on the shared-scan
> > design and the lockdep conversion.
> >
> > Kunwu Chan (4):
> >   hazptr: add shared-scan kthread
> >   locking/lockdep: use hazptr to wait for dynamic key lookups
> >   rcuscale: add hazptr scale type
> >   Documentation/litmus-tests: add hazptr acquire-before-scan test
> >
> >  .../hazptr/hazptr-acquire-before-scan.litmus  |  49 ++++
> >  kernel/hazptr.c                               | 274 ++++++++++++++++++
> >  kernel/locking/lockdep.c                      |  30 +-
> >  kernel/rcu/rcuscale.c                         |  65 ++++-
> >  4 files changed, 407 insertions(+), 11 deletions(-)
> >  create mode 100644 Documentation/litmus-tests/hazptr/hazptr-acquire-before-scan.litmus
> >
> > --
> > 2.43.0
> >