[RFC PATCH v5 0/2] Hazard Pointers

[RFC PATCH v5 0/2] Hazard Pointers

[Mathieu Desnoyers]

Here is a v5 of my Hazard Pointers series. I know Boqun has a use case
for it for lockdep.

This new version has those significant changes since v4:

- Fold scheduler integration.
- Actually set ctx slot to backup slot on context switch.
- Remove CONFIG_PREEMPT_HAZPTR config option.
- Use per-cpu ctx pointers for context switch slot tracking
  rather than per-task lists. This accelerates the hazptr
  acquire/release fast-path.
- Guarantee scan forward progress with two-lists scheme.
- Reimplement the hazptr acquire with a temporary wildcard
  to eliminate a dependency on the addr_p load, likely to
  cause a pipeline stall due to the needed memory barrier.
  This simplifies the algorithm, removes the need for pointer
  re-load + comparison, and is expected to be faster on some
  architectures.
- Reduce number of percpu slots to 4, introduce a hazptr_slot_item
  struct to contain both the slot and ctx pointers. Reducing number
  of slots to 4 makes sure all the slot and ctx pointers fit in a
  single cache line.
- Rebased on v7.0-rc1.

Feedback is welcome,

Thanks,

Mathieu

Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: "Paul E. McKenney" <paulmck@kernel.org>
Cc: Will Deacon <will@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Boqun Feng <boqun@kernel.org>
Cc: Alan Stern <stern@rowland.harvard.edu>
Cc: John Stultz <jstultz@google.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Frederic Weisbecker <frederic@kernel.org>
Cc: Joel Fernandes <joel@joelfernandes.org>
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Uladzislau Rezki <urezki@gmail.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Zqiang <qiang.zhang1211@gmail.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Waiman Long <longman@redhat.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: maged.michael@gmail.com
Cc: Mateusz Guzik <mjguzik@gmail.com>
Cc: Jonas Oberhauser <jonas.oberhauser@huaweicloud.com>
Cc: rcu@vger.kernel.org
Cc: linux-mm@kvack.org
Cc: lkmm@lists.linux.dev

Mathieu Desnoyers (2):
  hazptr: Implement Hazard Pointers
  hazptr: Add refscale test

 include/linux/hazptr.h | 197 +++++++++++++++++++++++++++++++++
 init/main.c            |   2 +
 kernel/Makefile        |   2 +-
 kernel/hazptr.c        | 242 +++++++++++++++++++++++++++++++++++++++++
 kernel/rcu/refscale.c  |  43 ++++++++
 kernel/sched/core.c    |   2 +
 6 files changed, 487 insertions(+), 1 deletion(-)
 create mode 100644 include/linux/hazptr.h
 create mode 100644 kernel/hazptr.c

-- 
2.39.5

[RFC PATCH v5 1/2] hazptr: Implement Hazard Pointers

[Mathieu Desnoyers]

This API provides existence guarantees of objects through Hazard
Pointers [1] (hazptr).

Its main benefit over RCU is that it allows fast reclaim of
HP-protected pointers without needing to wait for a grace period.

This implementation has 4 statically allocated hazard pointer slots per
cpu for the fast path, and relies on a on-stack backup slot allocated by
the hazard pointer user as fallback in case no per-cpu slot is
available.

It integrates with the scheduler to migrate per-CPU slots to the backup
slot on context switch. This ensures that the per-CPU slots won't be
used by blocked or preempted tasks holding on hazard pointers for a long
time.

References:

[1]: M. M. Michael, "Hazard pointers: safe memory reclamation for
     lock-free objects," in IEEE Transactions on Parallel and
     Distributed Systems, vol. 15, no. 6, pp. 491-504, June 2004

Link: https://lpc.events/event/19/contributions/2082/
Link: https://lore.kernel.org/lkml/j3scdl5iymjlxavomgc6u5ndg3svhab6ga23dr36o4f5mt333w@7xslvq6b6hmv/
Link: https://lpc.events/event/18/contributions/1731/
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: "Paul E. McKenney" <paulmck@kernel.org>
Cc: Will Deacon <will@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Boqun Feng <boqun@kernel.org>
Cc: Alan Stern <stern@rowland.harvard.edu>
Cc: John Stultz <jstultz@google.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Frederic Weisbecker <frederic@kernel.org>
Cc: Joel Fernandes <joel@joelfernandes.org>
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Uladzislau Rezki <urezki@gmail.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Zqiang <qiang.zhang1211@gmail.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Waiman Long <longman@redhat.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: maged.michael@gmail.com
Cc: Mateusz Guzik <mjguzik@gmail.com>
Cc: Jonas Oberhauser <jonas.oberhauser@huaweicloud.com>
Cc: rcu@vger.kernel.org
Cc: linux-mm@kvack.org
Cc: lkmm@lists.linux.dev
---
Changes since v4:
- Fold scheduler integration.
- Actually set ctx slot to backup slot on context switch.
- Remove CONFIG_PREEMPT_HAZPTR config option.
- Use per-cpu ctx pointers for context switch slot tracking
  rather than per-task lists. This accelerates the hazptr
  acquire/release fast-path.
- Guarantee scan forward progress with two-lists scheme.
- Reimplement the hazptr acquire with a temporary wildcard
  to eliminate a dependency on the addr_p load, likely to
  cause a pipeline stall due to the needed memory barrier.
  This simplifies the algorithm, removes the need for pointer
  re-load + comparison, and is expected to be faster on some
  architectures.
- Reduce number of percpu slots to 4, introduce a hazptr_slot_item
  struct to contain both the slot and ctx pointers. Reducing number
  of slots to 4 makes sure all the slot and ctx pointers fit in a
  single cache line.
- Rebased on v7.0-rc1.

Changes since v3:
- Rename hazptr_retire to hazptr_release.
- Remove domains.
- Introduce "backup_slot" within hazptr context structure (on stack)
  to handle slot overflow.
- Rename hazptr_try_protect to hazptr_acquire.
- Preallocate 8 per-CPU slots, and rely on caller-provided backup
  slots (typically on stack) for out-of-slots situations.

Changes since v2:
- Address Peter Zijlstra's comments.
- Address Paul E. McKenney's comments.

Changes since v0:
- Remove slot variable from hp_dereference_allocate().
---
 include/linux/hazptr.h | 197 +++++++++++++++++++++++++++++++++
 init/main.c            |   2 +
 kernel/Makefile        |   2 +-
 kernel/hazptr.c        | 242 +++++++++++++++++++++++++++++++++++++++++
 kernel/sched/core.c    |   2 +
 5 files changed, 444 insertions(+), 1 deletion(-)
 create mode 100644 include/linux/hazptr.h
 create mode 100644 kernel/hazptr.c

diff --git a/include/linux/hazptr.h b/include/linux/hazptr.h
new file mode 100644
index 000000000000..461f481a480b
--- /dev/null
+++ b/include/linux/hazptr.h
@@ -0,0 +1,197 @@
+// SPDX-FileCopyrightText: 2024 Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
+//
+// SPDX-License-Identifier: LGPL-2.1-or-later
+
+#ifndef _LINUX_HAZPTR_H
+#define _LINUX_HAZPTR_H
+
+/*
+ * hazptr: Hazard Pointers
+ *
+ * This API provides existence guarantees of objects through hazard
+ * pointers.
+ *
+ * Its main benefit over RCU is that it allows fast reclaim of
+ * HP-protected pointers without needing to wait for a grace period.
+ *
+ * References:
+ *
+ * [1]: M. M. Michael, "Hazard pointers: safe memory reclamation for
+ *      lock-free objects," in IEEE Transactions on Parallel and
+ *      Distributed Systems, vol. 15, no. 6, pp. 491-504, June 2004
+ */
+
+#include <linux/percpu.h>
+#include <linux/types.h>
+#include <linux/cleanup.h>
+#include <linux/sched.h>
+
+/* 4 slots (each sizeof(hazptr_slot_item)) fit in a single 64-byte cache line. */
+#define NR_HAZPTR_PERCPU_SLOTS	4
+#define HAZPTR_WILDCARD		((void *) 0x1UL)
+
+/*
+ * Hazard pointer slot.
+ */
+struct hazptr_slot {
+	void *addr;
+};
+
+struct hazptr_overflow_list;
+
+struct hazptr_backup_slot {
+	struct hlist_node overflow_node;
+	struct hazptr_slot slot;
+	/* Overflow list where the backup slot is added. */
+	struct hazptr_overflow_list *overflow_list;
+};
+
+struct hazptr_ctx {
+	struct hazptr_slot *slot;
+	/* Backup slot in case all per-CPU slots are used. */
+	struct hazptr_backup_slot backup_slot;
+	struct hlist_node preempt_node;
+};
+
+struct hazptr_slot_ctx {
+	struct hazptr_ctx *ctx;
+};
+
+struct hazptr_slot_item {
+	struct hazptr_slot slot;
+	struct hazptr_slot_ctx ctx;
+};
+
+struct hazptr_percpu_slots {
+	struct hazptr_slot_item items[NR_HAZPTR_PERCPU_SLOTS];
+} ____cacheline_aligned;
+
+DECLARE_PER_CPU(struct hazptr_percpu_slots, hazptr_percpu_slots);
+
+void *__hazptr_acquire(struct hazptr_ctx *ctx, void * const * addr_p);
+
+/*
+ * hazptr_synchronize: Wait until @addr is released from all slots.
+ *
+ * Wait to observe that each slot contains a value that differs from
+ * @addr before returning.
+ * Should be called from preemptible context.
+ */
+void hazptr_synchronize(void *addr);
+
+/*
+ * hazptr_chain_backup_slot: Chain backup slot into overflow list.
+ *
+ * Set backup slot address to @addr, and chain it into the overflow
+ * list.
+ */
+struct hazptr_slot *hazptr_chain_backup_slot(struct hazptr_ctx *ctx);
+
+/*
+ * hazptr_unchain_backup_slot: Unchain backup slot from overflow list.
+ */
+void hazptr_unchain_backup_slot(struct hazptr_ctx *ctx);
+
+static inline
+bool hazptr_slot_is_backup(struct hazptr_ctx *ctx, struct hazptr_slot *slot)
+{
+	return slot == &ctx->backup_slot.slot;
+}
+
+static inline
+void hazptr_note_context_switch(void)
+{
+	struct hazptr_percpu_slots *percpu_slots = this_cpu_ptr(&hazptr_percpu_slots);
+	unsigned int idx;
+
+	for (idx = 0; idx < NR_HAZPTR_PERCPU_SLOTS; idx++) {
+		struct hazptr_slot_item *item = &percpu_slots->items[idx];
+		struct hazptr_slot *slot = &item->slot, *backup_slot;
+		struct hazptr_ctx *ctx;
+
+		if (!slot->addr)
+			continue;
+		ctx = item->ctx.ctx;
+		backup_slot = hazptr_chain_backup_slot(ctx);
+		/*
+		 * Move hazard pointer from the per-CPU slot to the
+		 * backup slot. This requires hazard pointer
+		 * synchronize to iterate on per-CPU slots with
+		 * load-acquire before iterating on the overflow list.
+		 */
+		WRITE_ONCE(backup_slot->addr, slot->addr);
+		/*
+		 * store-release orders store to backup slot addr before
+		 * store to per-CPU slot addr.
+		 */
+		smp_store_release(&slot->addr, NULL);
+		/* Use the backup slot for context. */
+		ctx->slot = backup_slot;
+	}
+}
+
+/*
+ * hazptr_acquire: Load pointer at address and protect with hazard pointer.
+ *
+ * Load @addr_p, and protect the loaded pointer with hazard pointer.
+ * When using hazptr_acquire from interrupt handlers, the acquired slots
+ * need to be released before returning from the interrupt handler.
+ *
+ * Returns a non-NULL protected address if the loaded pointer is non-NULL.
+ * Returns NULL if the loaded pointer is NULL.
+ *
+ * On success the protected hazptr slot is stored in @ctx->slot.
+ */
+static inline
+void *hazptr_acquire(struct hazptr_ctx *ctx, void * const *addr_p)
+{
+	struct hazptr_percpu_slots *percpu_slots;
+	struct hazptr_slot_item *slot_item;
+	struct hazptr_slot *slot;
+	void *addr;
+
+	guard(preempt)();
+	percpu_slots = this_cpu_ptr(&hazptr_percpu_slots);
+	slot_item = &percpu_slots->items[0];
+	slot = &slot_item->slot;
+	if (unlikely(slot->addr))
+		return __hazptr_acquire(ctx, addr_p);
+	WRITE_ONCE(slot->addr, HAZPTR_WILDCARD);	/* Store B */
+
+	/* Memory ordering: Store B before Load A. */
+	smp_mb();
+
+	/*
+	 * Load @addr_p after storing wildcard to the hazard pointer slot.
+	 */
+	addr = READ_ONCE(*addr_p);	/* Load A */
+
+	/*
+	 * We don't care about ordering of Store C. It will simply
+	 * replace the wildcard by a more specific address. If addr is
+	 * NULL, we simply store NULL into the slot.
+	 */
+	WRITE_ONCE(slot->addr, addr);	/* Store C */
+	slot_item->ctx.ctx = ctx;
+	ctx->slot = slot;
+	return addr;
+}
+
+/* Release the protected hazard pointer from @slot. */
+static inline
+void hazptr_release(struct hazptr_ctx *ctx, void *addr)
+{
+	struct hazptr_slot *slot;
+
+	if (!addr)
+		return;
+	guard(preempt)();
+	slot = ctx->slot;
+	smp_store_release(&slot->addr, NULL);
+	if (unlikely(hazptr_slot_is_backup(ctx, slot)))
+		hazptr_unchain_backup_slot(ctx);
+}
+
+void hazptr_init(void);
+
+#endif /* _LINUX_HAZPTR_H */
diff --git a/init/main.c b/init/main.c
index 1cb395dd94e4..b66017629935 100644
--- a/init/main.c
+++ b/init/main.c
@@ -105,6 +105,7 @@
 #include <linux/ptdump.h>
 #include <linux/time_namespace.h>
 #include <linux/unaligned.h>
+#include <linux/hazptr.h>
 #include <net/net_namespace.h>
 
 #include <asm/io.h>
@@ -1101,6 +1102,7 @@ void start_kernel(void)
 	workqueue_init_early();
 
 	rcu_init();
+	hazptr_init();
 	kvfree_rcu_init();
 
 	/* Trace events are available after this */
diff --git a/kernel/Makefile b/kernel/Makefile
index 6785982013dc..b7cef6e23038 100644
--- a/kernel/Makefile
+++ b/kernel/Makefile
@@ -7,7 +7,7 @@ obj-y     = fork.o exec_domain.o panic.o \
 	    cpu.o exit.o softirq.o resource.o \
 	    sysctl.o capability.o ptrace.o user.o \
 	    signal.o sys.o umh.o workqueue.o pid.o task_work.o \
-	    extable.o params.o \
+	    extable.o params.o hazptr.o \
 	    kthread.o sys_ni.o nsproxy.o nstree.o nscommon.o \
 	    notifier.o ksysfs.o cred.o reboot.o \
 	    async.o range.o smpboot.o ucount.o regset.o ksyms_common.o
diff --git a/kernel/hazptr.c b/kernel/hazptr.c
new file mode 100644
index 000000000000..a63ac681cb85
--- /dev/null
+++ b/kernel/hazptr.c
@@ -0,0 +1,242 @@
+// SPDX-FileCopyrightText: 2024 Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
+//
+// SPDX-License-Identifier: LGPL-2.1-or-later
+
+/*
+ * hazptr: Hazard Pointers
+ */
+
+#include <linux/hazptr.h>
+#include <linux/percpu.h>
+#include <linux/spinlock.h>
+#include <linux/mutex.h>
+#include <linux/list.h>
+#include <linux/export.h>
+
+struct hazptr_overflow_list {
+	raw_spinlock_t lock;		/* Lock protecting overflow list and list generation. */
+	struct hlist_head head;		/* Overflow list head. */
+	uint64_t gen;			/* Overflow list generation. */
+};
+
+/*
+ * Flip between two lists to guarantee list scan forward progress even
+ * with frequent generation counter increments. The list additions are
+ * always done on a different list than the one used for scan. The scan
+ * successively iterates on both lists. Therefore, only list removals
+ * can cause the iteration to retry, and the number of removals is
+ * limited to the number of list elements.
+ */
+struct hazptr_overflow_list_flip {
+	struct mutex lock;		/* Mutex protecting add_idx from concurrent updates. */
+	unsigned int add_idx;		/* Index of current flip-list to add to. */
+	struct hazptr_overflow_list array[2];
+};
+
+static DEFINE_PER_CPU(struct hazptr_overflow_list_flip, percpu_overflow_list_flip);
+
+DEFINE_PER_CPU(struct hazptr_percpu_slots, hazptr_percpu_slots);
+EXPORT_PER_CPU_SYMBOL_GPL(hazptr_percpu_slots);
+
+static
+struct hazptr_slot *hazptr_get_free_percpu_slot(struct hazptr_ctx *ctx)
+{
+	struct hazptr_percpu_slots *percpu_slots = this_cpu_ptr(&hazptr_percpu_slots);
+	unsigned int idx;
+
+	for (idx = 0; idx < NR_HAZPTR_PERCPU_SLOTS; idx++) {
+		struct hazptr_slot_item *item = &percpu_slots->items[idx];
+		struct hazptr_slot *slot = &item->slot;
+
+		if (!slot->addr) {
+			item->ctx.ctx = ctx;
+			return slot;
+		}
+	}
+	/* All slots are in use. */
+	return NULL;
+}
+
+/*
+ * Hazard pointer acquire slow path.
+ * Called with preemption disabled.
+ */
+void *__hazptr_acquire(struct hazptr_ctx *ctx, void * const *addr_p)
+{
+	struct hazptr_slot *slot = hazptr_get_free_percpu_slot(ctx);
+	void *addr;
+
+	/*
+	 * If all the per-CPU slots are already in use, fallback
+	 * to the backup slot.
+	 */
+	if (unlikely(!slot))
+		slot = hazptr_chain_backup_slot(ctx);
+	WRITE_ONCE(slot->addr, HAZPTR_WILDCARD);	/* Store B */
+
+	/* Memory ordering: Store B before Load A. */
+	smp_mb();
+
+	/*
+	 * Load @addr_p after storing wildcard to the hazard pointer slot.
+	 */
+	addr = READ_ONCE(*addr_p);	/* Load A */
+
+	/*
+	 * We don't care about ordering of Store C. It will simply
+	 * replace the wildcard by a more specific address. If addr is
+	 * NULL, we simply store NULL into the slot.
+	 */
+	WRITE_ONCE(slot->addr, addr);	/* Store C */
+	ctx->slot = slot;
+	if (!addr && hazptr_slot_is_backup(ctx, slot))
+		hazptr_unchain_backup_slot(ctx);
+	return addr;
+}
+EXPORT_SYMBOL_GPL(__hazptr_acquire);
+
+/*
+ * Perform piecewise iteration on overflow list waiting until "addr" is
+ * not present. Raw spinlock is released and taken between each list
+ * item and busy loop iteration. The overflow list generation is checked
+ * each time the lock is taken to validate that the list has not changed
+ * before resuming iteration or busy wait. If the generation has
+ * changed, retry the entire list traversal.
+ */
+static
+void hazptr_synchronize_overflow_list(struct hazptr_overflow_list *overflow_list, void *addr)
+{
+	struct hazptr_backup_slot *backup_slot;
+	uint64_t snapshot_gen;
+	unsigned long flags;
+
+	raw_spin_lock_irqsave(&overflow_list->lock, flags);
+retry:
+	snapshot_gen = overflow_list->gen;
+	hlist_for_each_entry(backup_slot, &overflow_list->head, overflow_node) {
+		/* Busy-wait if node is found. */
+		for (;;) {
+			void *load_addr = smp_load_acquire(&backup_slot->slot.addr);	/* Load B */
+
+			if (load_addr != addr && load_addr != HAZPTR_WILDCARD)
+				break;
+			raw_spin_unlock_irqrestore(&overflow_list->lock, flags);
+			cpu_relax();
+			raw_spin_lock_irqsave(&overflow_list->lock, flags);
+			if (overflow_list->gen != snapshot_gen)
+				goto retry;
+		}
+		raw_spin_unlock_irqrestore(&overflow_list->lock, flags);
+		/*
+		 * Release raw spinlock, validate generation after
+		 * re-acquiring the lock.
+		 */
+		raw_spin_lock_irqsave(&overflow_list->lock, flags);
+		if (overflow_list->gen != snapshot_gen)
+			goto retry;
+	}
+	raw_spin_unlock_irqrestore(&overflow_list->lock, flags);
+}
+
+static
+void hazptr_synchronize_cpu_slots(int cpu, void *addr)
+{
+	struct hazptr_percpu_slots *percpu_slots = per_cpu_ptr(&hazptr_percpu_slots, cpu);
+	unsigned int idx;
+
+	for (idx = 0; idx < NR_HAZPTR_PERCPU_SLOTS; idx++) {
+		struct hazptr_slot_item *item = &percpu_slots->items[idx];
+
+		/* Busy-wait if node is found. */
+		smp_cond_load_acquire(&item->slot.addr, VAL != addr && VAL != HAZPTR_WILDCARD); /* Load B */
+	}
+}
+
+/*
+ * hazptr_synchronize: Wait until @addr is released from all slots.
+ *
+ * Wait to observe that each slot contains a value that differs from
+ * @addr before returning.
+ * Should be called from preemptible context.
+ */
+void hazptr_synchronize(void *addr)
+{
+	int cpu;
+
+	/*
+	 * Busy-wait should only be done from preemptible context.
+	 */
+	lockdep_assert_preemption_enabled();
+
+	/*
+	 * Store A precedes hazptr_scan(): it unpublishes addr (sets it to
+	 * NULL or to a different value), and thus hides it from hazard
+	 * pointer readers.
+	 */
+	if (!addr)
+		return;
+	/* Memory ordering: Store A before Load B. */
+	smp_mb();
+	/* Scan all CPUs slots. */
+	for_each_possible_cpu(cpu) {
+		struct hazptr_overflow_list_flip *overflow_list_flip = per_cpu_ptr(&percpu_overflow_list_flip, cpu);
+		unsigned int scan_idx;
+
+		/* Scan CPU slots. */
+		hazptr_synchronize_cpu_slots(cpu, addr);
+
+		/*
+		 * Scan backup slots in percpu overflow lists.
+		 * Forward progress is guaranteed by scanning one list
+		 * while new elements are added into the other list.
+		 */
+		guard(mutex)(&overflow_list_flip->lock);
+		scan_idx = overflow_list_flip->add_idx ^ 1;
+		hazptr_synchronize_overflow_list(&overflow_list_flip->array[scan_idx], addr);
+		/* Flip current list. */
+		WRITE_ONCE(overflow_list_flip->add_idx, scan_idx);
+		hazptr_synchronize_overflow_list(&overflow_list_flip->array[scan_idx ^ 1], addr);
+	}
+}
+EXPORT_SYMBOL_GPL(hazptr_synchronize);
+
+struct hazptr_slot *hazptr_chain_backup_slot(struct hazptr_ctx *ctx)
+{
+	struct hazptr_overflow_list_flip *overflow_list_flip = this_cpu_ptr(&percpu_overflow_list_flip);
+	unsigned int list_idx = READ_ONCE(overflow_list_flip->add_idx);
+	struct hazptr_overflow_list *overflow_list = &overflow_list_flip->array[list_idx];
+	struct hazptr_slot *slot = &ctx->backup_slot.slot;
+
+	slot->addr = NULL;
+	guard(raw_spinlock_irqsave)(&overflow_list->lock);
+	overflow_list->gen++;
+	hlist_add_head(&ctx->backup_slot.overflow_node, &overflow_list->head);
+	ctx->backup_slot.overflow_list = overflow_list;
+	return slot;
+}
+EXPORT_SYMBOL_GPL(hazptr_chain_backup_slot);
+
+void hazptr_unchain_backup_slot(struct hazptr_ctx *ctx)
+{
+	struct hazptr_overflow_list *overflow_list = ctx->backup_slot.overflow_list;
+
+	guard(raw_spinlock_irqsave)(&overflow_list->lock);
+	overflow_list->gen++;
+	hlist_del(&ctx->backup_slot.overflow_node);
+}
+EXPORT_SYMBOL_GPL(hazptr_unchain_backup_slot);
+
+void __init hazptr_init(void)
+{
+	int cpu;
+
+	for_each_possible_cpu(cpu) {
+		struct hazptr_overflow_list_flip *overflow_list_flip = per_cpu_ptr(&percpu_overflow_list_flip, cpu);
+
+		mutex_init(&overflow_list_flip->lock);
+		for (int i = 0; i < 2; i++) {
+			raw_spin_lock_init(&overflow_list_flip->array[i].lock);
+			INIT_HLIST_HEAD(&overflow_list_flip->array[i].head);
+		}
+	}
+}
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 759777694c78..b3e10be20329 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -60,6 +60,7 @@
 #include <linux/profile.h>
 #include <linux/psi.h>
 #include <linux/rcuwait_api.h>
+#include <linux/hazptr.h>
 #include <linux/rseq.h>
 #include <linux/sched/wake_q.h>
 #include <linux/scs.h>
@@ -6790,6 +6791,7 @@ static void __sched notrace __schedule(int sched_mode)
 	local_irq_disable();
 	rcu_note_context_switch(preempt);
 	migrate_disable_switch(rq, prev);
+	hazptr_note_context_switch();
 
 	/*
 	 * Make sure that signal_pending_state()->signal_pending() below
-- 
2.39.5

[RFC PATCH v5 2/2] hazptr: Add refscale test

[Mathieu Desnoyers]

Add the refscale test for hazptr to measure the reader side
performance.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Co-developed-by: Boqun Feng <boqun@kernel.org>
---
 kernel/rcu/refscale.c | 43 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/kernel/rcu/refscale.c b/kernel/rcu/refscale.c
index c158b6a947cd..7d64dfe78327 100644
--- a/kernel/rcu/refscale.c
+++ b/kernel/rcu/refscale.c
@@ -29,6 +29,7 @@
 #include <linux/reboot.h>
 #include <linux/sched.h>
 #include <linux/seq_buf.h>
+#include <linux/hazptr.h>
 #include <linux/spinlock.h>
 #include <linux/smp.h>
 #include <linux/stat.h>
@@ -1210,6 +1211,47 @@ static const struct ref_scale_ops typesafe_seqlock_ops = {
 	.name		= "typesafe_seqlock"
 };
 
+static void ref_hazptr_read_section(const int nloops)
+{
+	static void *ref_hazptr_read_section_ptr = ref_hazptr_read_section;
+	int i;
+
+	for (i = nloops; i >= 0; i--) {
+		struct hazptr_ctx ctx;
+		void *addr;
+
+		addr = hazptr_acquire(&ctx, &ref_hazptr_read_section_ptr);
+		hazptr_release(&ctx, addr);
+	}
+}
+
+static void ref_hazptr_delay_section(const int nloops, const int udl, const int ndl)
+{
+	static void *ref_hazptr_delay_section_ptr = ref_hazptr_delay_section;
+	int i;
+
+	for (i = nloops; i >= 0; i--) {
+		struct hazptr_ctx ctx;
+		void *addr;
+
+		addr = hazptr_acquire(&ctx, &ref_hazptr_delay_section_ptr);
+		un_delay(udl, ndl);
+		hazptr_release(&ctx, addr);
+	}
+}
+
+static bool ref_hazptr_init(void)
+{
+	return true;
+}
+
+static const struct ref_scale_ops hazptr_ops = {
+	.init		= ref_hazptr_init,
+	.readsection	= ref_hazptr_read_section,
+	.delaysection	= ref_hazptr_delay_section,
+	.name		= "hazptr"
+};
+
 static void rcu_scale_one_reader(void)
 {
 	if (readdelay <= 0)
@@ -1524,6 +1566,7 @@ ref_scale_init(void)
 		&sched_clock_ops, &clock_ops, &jiffies_ops,
 		&preempt_ops, &bh_ops, &irq_ops, &irqsave_ops,
 		&typesafe_ref_ops, &typesafe_lock_ops, &typesafe_seqlock_ops,
+		&hazptr_ops,
 	};
 
 	if (!torture_init_begin(scale_type, verbose))
-- 
2.39.5

Re: [RFC PATCH v5 1/2] hazptr: Implement Hazard Pointers

[Paul E. McKenney]

[ Trimming CC. ]

On Mon, Feb 23, 2026 at 03:44:17PM -0500, Mathieu Desnoyers wrote:
> This API provides existence guarantees of objects through Hazard
> Pointers [1] (hazptr).
> 
> Its main benefit over RCU is that it allows fast reclaim of
> HP-protected pointers without needing to wait for a grace period.
> 
> This implementation has 4 statically allocated hazard pointer slots per
> cpu for the fast path, and relies on a on-stack backup slot allocated by
> the hazard pointer user as fallback in case no per-cpu slot is
> available.
> 
> It integrates with the scheduler to migrate per-CPU slots to the backup
> slot on context switch. This ensures that the per-CPU slots won't be
> used by blocked or preempted tasks holding on hazard pointers for a long
> time.
> 
> References:
> 
> [1]: M. M. Michael, "Hazard pointers: safe memory reclamation for
>      lock-free objects," in IEEE Transactions on Parallel and
>      Distributed Systems, vol. 15, no. 6, pp. 491-504, June 2004

I (finally) put together a stress test for this, which is the following
commits on -rcu:

bfb239e036d616 torture: Add a hazptrtorture.c torture test
adda38c3cffcd6 hazptrtorture: Add testing of on-stack hazptr_ctx structures
311e199e482873 hazptrtorture: Add microsecond-scale sleep in readers
21ccb41f79a45b hazptrtorture: Enable system-independent CPU overcommit
82e8c141265c81 torture: Add a stutter_will_wait() function
9ad8169bd3a367 hazptrtorture: Use mnemonic local variables for context information
cd541d5eb7347f hazptrtorture: Split hazptr_torture_reader_tail() from hazptr_torture_reader()
0de8bda6b00051 hazptrtorture: Add kthread to release deferred hazard pointers
3deefef00acb22 hazptrtorture: Defer release of hazard pointers
65b2cf3c6ea34e hazptrtorture: Add irq_acquire to acquire hazptr from irq
98d334c04b58fe hazptrtorture: Use task_state_to_char() for task-state reporting
1e89aae50a3fde hazptrtorture: Pass hazptr_pending to hazptr_torture_reader_tail()
4b1a177190e2e1 hazptrtorture: Add the ability to disable the writer kthread
2c966f58358329 hazptrtorture: Add irq_release to release hazptr from irq
16258269b41ed3 hazptrtorture: Accumulate operation statistics
c090e2659abcb3 doc: Add hazptrtorture module parameters

Initial runs suggest that hazard pointers cannot be acquired by one
task and released by another.  Is that expected behavior, or is my test
improperly passing the hazard pointers?  If the latter, we should of
course document the proper hazard-pointer-passing mechanism.

						Thanx, Paul

> Link: https://lpc.events/event/19/contributions/2082/
> Link: https://lore.kernel.org/lkml/j3scdl5iymjlxavomgc6u5ndg3svhab6ga23dr36o4f5mt333w@7xslvq6b6hmv/
> Link: https://lpc.events/event/18/contributions/1731/
> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> Cc: Nicholas Piggin <npiggin@gmail.com>
> Cc: Michael Ellerman <mpe@ellerman.id.au>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> Cc: "Paul E. McKenney" <paulmck@kernel.org>
> Cc: Will Deacon <will@kernel.org>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Boqun Feng <boqun@kernel.org>
> Cc: Alan Stern <stern@rowland.harvard.edu>
> Cc: John Stultz <jstultz@google.com>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Frederic Weisbecker <frederic@kernel.org>
> Cc: Joel Fernandes <joel@joelfernandes.org>
> Cc: Josh Triplett <josh@joshtriplett.org>
> Cc: Uladzislau Rezki <urezki@gmail.com>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Lai Jiangshan <jiangshanlai@gmail.com>
> Cc: Zqiang <qiang.zhang1211@gmail.com>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Waiman Long <longman@redhat.com>
> Cc: Mark Rutland <mark.rutland@arm.com>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Vlastimil Babka <vbabka@suse.cz>
> Cc: maged.michael@gmail.com
> Cc: Mateusz Guzik <mjguzik@gmail.com>
> Cc: Jonas Oberhauser <jonas.oberhauser@huaweicloud.com>
> Cc: rcu@vger.kernel.org
> Cc: linux-mm@kvack.org
> Cc: lkmm@lists.linux.dev
> ---
> Changes since v4:
> - Fold scheduler integration.
> - Actually set ctx slot to backup slot on context switch.
> - Remove CONFIG_PREEMPT_HAZPTR config option.
> - Use per-cpu ctx pointers for context switch slot tracking
>   rather than per-task lists. This accelerates the hazptr
>   acquire/release fast-path.
> - Guarantee scan forward progress with two-lists scheme.
> - Reimplement the hazptr acquire with a temporary wildcard
>   to eliminate a dependency on the addr_p load, likely to
>   cause a pipeline stall due to the needed memory barrier.
>   This simplifies the algorithm, removes the need for pointer
>   re-load + comparison, and is expected to be faster on some
>   architectures.
> - Reduce number of percpu slots to 4, introduce a hazptr_slot_item
>   struct to contain both the slot and ctx pointers. Reducing number
>   of slots to 4 makes sure all the slot and ctx pointers fit in a
>   single cache line.
> - Rebased on v7.0-rc1.
> 
> Changes since v3:
> - Rename hazptr_retire to hazptr_release.
> - Remove domains.
> - Introduce "backup_slot" within hazptr context structure (on stack)
>   to handle slot overflow.
> - Rename hazptr_try_protect to hazptr_acquire.
> - Preallocate 8 per-CPU slots, and rely on caller-provided backup
>   slots (typically on stack) for out-of-slots situations.
> 
> Changes since v2:
> - Address Peter Zijlstra's comments.
> - Address Paul E. McKenney's comments.
> 
> Changes since v0:
> - Remove slot variable from hp_dereference_allocate().
> ---
>  include/linux/hazptr.h | 197 +++++++++++++++++++++++++++++++++
>  init/main.c            |   2 +
>  kernel/Makefile        |   2 +-
>  kernel/hazptr.c        | 242 +++++++++++++++++++++++++++++++++++++++++
>  kernel/sched/core.c    |   2 +
>  5 files changed, 444 insertions(+), 1 deletion(-)
>  create mode 100644 include/linux/hazptr.h
>  create mode 100644 kernel/hazptr.c
> 
> diff --git a/include/linux/hazptr.h b/include/linux/hazptr.h
> new file mode 100644
> index 000000000000..461f481a480b
> --- /dev/null
> +++ b/include/linux/hazptr.h
> @@ -0,0 +1,197 @@
> +// SPDX-FileCopyrightText: 2024 Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> +//
> +// SPDX-License-Identifier: LGPL-2.1-or-later
> +
> +#ifndef _LINUX_HAZPTR_H
> +#define _LINUX_HAZPTR_H
> +
> +/*
> + * hazptr: Hazard Pointers
> + *
> + * This API provides existence guarantees of objects through hazard
> + * pointers.
> + *
> + * Its main benefit over RCU is that it allows fast reclaim of
> + * HP-protected pointers without needing to wait for a grace period.
> + *
> + * References:
> + *
> + * [1]: M. M. Michael, "Hazard pointers: safe memory reclamation for
> + *      lock-free objects," in IEEE Transactions on Parallel and
> + *      Distributed Systems, vol. 15, no. 6, pp. 491-504, June 2004
> + */
> +
> +#include <linux/percpu.h>
> +#include <linux/types.h>
> +#include <linux/cleanup.h>
> +#include <linux/sched.h>
> +
> +/* 4 slots (each sizeof(hazptr_slot_item)) fit in a single 64-byte cache line. */
> +#define NR_HAZPTR_PERCPU_SLOTS	4
> +#define HAZPTR_WILDCARD		((void *) 0x1UL)
> +
> +/*
> + * Hazard pointer slot.
> + */
> +struct hazptr_slot {
> +	void *addr;
> +};
> +
> +struct hazptr_overflow_list;
> +
> +struct hazptr_backup_slot {
> +	struct hlist_node overflow_node;
> +	struct hazptr_slot slot;
> +	/* Overflow list where the backup slot is added. */
> +	struct hazptr_overflow_list *overflow_list;
> +};
> +
> +struct hazptr_ctx {
> +	struct hazptr_slot *slot;
> +	/* Backup slot in case all per-CPU slots are used. */
> +	struct hazptr_backup_slot backup_slot;
> +	struct hlist_node preempt_node;
> +};
> +
> +struct hazptr_slot_ctx {
> +	struct hazptr_ctx *ctx;
> +};
> +
> +struct hazptr_slot_item {
> +	struct hazptr_slot slot;
> +	struct hazptr_slot_ctx ctx;
> +};
> +
> +struct hazptr_percpu_slots {
> +	struct hazptr_slot_item items[NR_HAZPTR_PERCPU_SLOTS];
> +} ____cacheline_aligned;
> +
> +DECLARE_PER_CPU(struct hazptr_percpu_slots, hazptr_percpu_slots);
> +
> +void *__hazptr_acquire(struct hazptr_ctx *ctx, void * const * addr_p);
> +
> +/*
> + * hazptr_synchronize: Wait until @addr is released from all slots.
> + *
> + * Wait to observe that each slot contains a value that differs from
> + * @addr before returning.
> + * Should be called from preemptible context.
> + */
> +void hazptr_synchronize(void *addr);
> +
> +/*
> + * hazptr_chain_backup_slot: Chain backup slot into overflow list.
> + *
> + * Set backup slot address to @addr, and chain it into the overflow
> + * list.
> + */
> +struct hazptr_slot *hazptr_chain_backup_slot(struct hazptr_ctx *ctx);
> +
> +/*
> + * hazptr_unchain_backup_slot: Unchain backup slot from overflow list.
> + */
> +void hazptr_unchain_backup_slot(struct hazptr_ctx *ctx);
> +
> +static inline
> +bool hazptr_slot_is_backup(struct hazptr_ctx *ctx, struct hazptr_slot *slot)
> +{
> +	return slot == &ctx->backup_slot.slot;
> +}
> +
> +static inline
> +void hazptr_note_context_switch(void)
> +{
> +	struct hazptr_percpu_slots *percpu_slots = this_cpu_ptr(&hazptr_percpu_slots);
> +	unsigned int idx;
> +
> +	for (idx = 0; idx < NR_HAZPTR_PERCPU_SLOTS; idx++) {
> +		struct hazptr_slot_item *item = &percpu_slots->items[idx];
> +		struct hazptr_slot *slot = &item->slot, *backup_slot;
> +		struct hazptr_ctx *ctx;
> +
> +		if (!slot->addr)
> +			continue;
> +		ctx = item->ctx.ctx;
> +		backup_slot = hazptr_chain_backup_slot(ctx);
> +		/*
> +		 * Move hazard pointer from the per-CPU slot to the
> +		 * backup slot. This requires hazard pointer
> +		 * synchronize to iterate on per-CPU slots with
> +		 * load-acquire before iterating on the overflow list.
> +		 */
> +		WRITE_ONCE(backup_slot->addr, slot->addr);
> +		/*
> +		 * store-release orders store to backup slot addr before
> +		 * store to per-CPU slot addr.
> +		 */
> +		smp_store_release(&slot->addr, NULL);
> +		/* Use the backup slot for context. */
> +		ctx->slot = backup_slot;
> +	}
> +}
> +
> +/*
> + * hazptr_acquire: Load pointer at address and protect with hazard pointer.
> + *
> + * Load @addr_p, and protect the loaded pointer with hazard pointer.
> + * When using hazptr_acquire from interrupt handlers, the acquired slots
> + * need to be released before returning from the interrupt handler.
> + *
> + * Returns a non-NULL protected address if the loaded pointer is non-NULL.
> + * Returns NULL if the loaded pointer is NULL.
> + *
> + * On success the protected hazptr slot is stored in @ctx->slot.
> + */
> +static inline
> +void *hazptr_acquire(struct hazptr_ctx *ctx, void * const *addr_p)
> +{
> +	struct hazptr_percpu_slots *percpu_slots;
> +	struct hazptr_slot_item *slot_item;
> +	struct hazptr_slot *slot;
> +	void *addr;
> +
> +	guard(preempt)();
> +	percpu_slots = this_cpu_ptr(&hazptr_percpu_slots);
> +	slot_item = &percpu_slots->items[0];
> +	slot = &slot_item->slot;
> +	if (unlikely(slot->addr))
> +		return __hazptr_acquire(ctx, addr_p);
> +	WRITE_ONCE(slot->addr, HAZPTR_WILDCARD);	/* Store B */
> +
> +	/* Memory ordering: Store B before Load A. */
> +	smp_mb();
> +
> +	/*
> +	 * Load @addr_p after storing wildcard to the hazard pointer slot.
> +	 */
> +	addr = READ_ONCE(*addr_p);	/* Load A */
> +
> +	/*
> +	 * We don't care about ordering of Store C. It will simply
> +	 * replace the wildcard by a more specific address. If addr is
> +	 * NULL, we simply store NULL into the slot.
> +	 */
> +	WRITE_ONCE(slot->addr, addr);	/* Store C */
> +	slot_item->ctx.ctx = ctx;
> +	ctx->slot = slot;
> +	return addr;
> +}
> +
> +/* Release the protected hazard pointer from @slot. */
> +static inline
> +void hazptr_release(struct hazptr_ctx *ctx, void *addr)
> +{
> +	struct hazptr_slot *slot;
> +
> +	if (!addr)
> +		return;
> +	guard(preempt)();
> +	slot = ctx->slot;
> +	smp_store_release(&slot->addr, NULL);
> +	if (unlikely(hazptr_slot_is_backup(ctx, slot)))
> +		hazptr_unchain_backup_slot(ctx);
> +}
> +
> +void hazptr_init(void);
> +
> +#endif /* _LINUX_HAZPTR_H */
> diff --git a/init/main.c b/init/main.c
> index 1cb395dd94e4..b66017629935 100644
> --- a/init/main.c
> +++ b/init/main.c
> @@ -105,6 +105,7 @@
>  #include <linux/ptdump.h>
>  #include <linux/time_namespace.h>
>  #include <linux/unaligned.h>
> +#include <linux/hazptr.h>
>  #include <net/net_namespace.h>
>  
>  #include <asm/io.h>
> @@ -1101,6 +1102,7 @@ void start_kernel(void)
>  	workqueue_init_early();
>  
>  	rcu_init();
> +	hazptr_init();
>  	kvfree_rcu_init();
>  
>  	/* Trace events are available after this */
> diff --git a/kernel/Makefile b/kernel/Makefile
> index 6785982013dc..b7cef6e23038 100644
> --- a/kernel/Makefile
> +++ b/kernel/Makefile
> @@ -7,7 +7,7 @@ obj-y     = fork.o exec_domain.o panic.o \
>  	    cpu.o exit.o softirq.o resource.o \
>  	    sysctl.o capability.o ptrace.o user.o \
>  	    signal.o sys.o umh.o workqueue.o pid.o task_work.o \
> -	    extable.o params.o \
> +	    extable.o params.o hazptr.o \
>  	    kthread.o sys_ni.o nsproxy.o nstree.o nscommon.o \
>  	    notifier.o ksysfs.o cred.o reboot.o \
>  	    async.o range.o smpboot.o ucount.o regset.o ksyms_common.o
> diff --git a/kernel/hazptr.c b/kernel/hazptr.c
> new file mode 100644
> index 000000000000..a63ac681cb85
> --- /dev/null
> +++ b/kernel/hazptr.c
> @@ -0,0 +1,242 @@
> +// SPDX-FileCopyrightText: 2024 Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> +//
> +// SPDX-License-Identifier: LGPL-2.1-or-later
> +
> +/*
> + * hazptr: Hazard Pointers
> + */
> +
> +#include <linux/hazptr.h>
> +#include <linux/percpu.h>
> +#include <linux/spinlock.h>
> +#include <linux/mutex.h>
> +#include <linux/list.h>
> +#include <linux/export.h>
> +
> +struct hazptr_overflow_list {
> +	raw_spinlock_t lock;		/* Lock protecting overflow list and list generation. */
> +	struct hlist_head head;		/* Overflow list head. */
> +	uint64_t gen;			/* Overflow list generation. */
> +};
> +
> +/*
> + * Flip between two lists to guarantee list scan forward progress even
> + * with frequent generation counter increments. The list additions are
> + * always done on a different list than the one used for scan. The scan
> + * successively iterates on both lists. Therefore, only list removals
> + * can cause the iteration to retry, and the number of removals is
> + * limited to the number of list elements.
> + */
> +struct hazptr_overflow_list_flip {
> +	struct mutex lock;		/* Mutex protecting add_idx from concurrent updates. */
> +	unsigned int add_idx;		/* Index of current flip-list to add to. */
> +	struct hazptr_overflow_list array[2];
> +};
> +
> +static DEFINE_PER_CPU(struct hazptr_overflow_list_flip, percpu_overflow_list_flip);
> +
> +DEFINE_PER_CPU(struct hazptr_percpu_slots, hazptr_percpu_slots);
> +EXPORT_PER_CPU_SYMBOL_GPL(hazptr_percpu_slots);
> +
> +static
> +struct hazptr_slot *hazptr_get_free_percpu_slot(struct hazptr_ctx *ctx)
> +{
> +	struct hazptr_percpu_slots *percpu_slots = this_cpu_ptr(&hazptr_percpu_slots);
> +	unsigned int idx;
> +
> +	for (idx = 0; idx < NR_HAZPTR_PERCPU_SLOTS; idx++) {
> +		struct hazptr_slot_item *item = &percpu_slots->items[idx];
> +		struct hazptr_slot *slot = &item->slot;
> +
> +		if (!slot->addr) {
> +			item->ctx.ctx = ctx;
> +			return slot;
> +		}
> +	}
> +	/* All slots are in use. */
> +	return NULL;
> +}
> +
> +/*
> + * Hazard pointer acquire slow path.
> + * Called with preemption disabled.
> + */
> +void *__hazptr_acquire(struct hazptr_ctx *ctx, void * const *addr_p)
> +{
> +	struct hazptr_slot *slot = hazptr_get_free_percpu_slot(ctx);
> +	void *addr;
> +
> +	/*
> +	 * If all the per-CPU slots are already in use, fallback
> +	 * to the backup slot.
> +	 */
> +	if (unlikely(!slot))
> +		slot = hazptr_chain_backup_slot(ctx);
> +	WRITE_ONCE(slot->addr, HAZPTR_WILDCARD);	/* Store B */
> +
> +	/* Memory ordering: Store B before Load A. */
> +	smp_mb();
> +
> +	/*
> +	 * Load @addr_p after storing wildcard to the hazard pointer slot.
> +	 */
> +	addr = READ_ONCE(*addr_p);	/* Load A */
> +
> +	/*
> +	 * We don't care about ordering of Store C. It will simply
> +	 * replace the wildcard by a more specific address. If addr is
> +	 * NULL, we simply store NULL into the slot.
> +	 */
> +	WRITE_ONCE(slot->addr, addr);	/* Store C */
> +	ctx->slot = slot;
> +	if (!addr && hazptr_slot_is_backup(ctx, slot))
> +		hazptr_unchain_backup_slot(ctx);
> +	return addr;
> +}
> +EXPORT_SYMBOL_GPL(__hazptr_acquire);
> +
> +/*
> + * Perform piecewise iteration on overflow list waiting until "addr" is
> + * not present. Raw spinlock is released and taken between each list
> + * item and busy loop iteration. The overflow list generation is checked
> + * each time the lock is taken to validate that the list has not changed
> + * before resuming iteration or busy wait. If the generation has
> + * changed, retry the entire list traversal.
> + */
> +static
> +void hazptr_synchronize_overflow_list(struct hazptr_overflow_list *overflow_list, void *addr)
> +{
> +	struct hazptr_backup_slot *backup_slot;
> +	uint64_t snapshot_gen;
> +	unsigned long flags;
> +
> +	raw_spin_lock_irqsave(&overflow_list->lock, flags);
> +retry:
> +	snapshot_gen = overflow_list->gen;
> +	hlist_for_each_entry(backup_slot, &overflow_list->head, overflow_node) {
> +		/* Busy-wait if node is found. */
> +		for (;;) {
> +			void *load_addr = smp_load_acquire(&backup_slot->slot.addr);	/* Load B */
> +
> +			if (load_addr != addr && load_addr != HAZPTR_WILDCARD)
> +				break;
> +			raw_spin_unlock_irqrestore(&overflow_list->lock, flags);
> +			cpu_relax();
> +			raw_spin_lock_irqsave(&overflow_list->lock, flags);
> +			if (overflow_list->gen != snapshot_gen)
> +				goto retry;
> +		}
> +		raw_spin_unlock_irqrestore(&overflow_list->lock, flags);
> +		/*
> +		 * Release raw spinlock, validate generation after
> +		 * re-acquiring the lock.
> +		 */
> +		raw_spin_lock_irqsave(&overflow_list->lock, flags);
> +		if (overflow_list->gen != snapshot_gen)
> +			goto retry;
> +	}
> +	raw_spin_unlock_irqrestore(&overflow_list->lock, flags);
> +}
> +
> +static
> +void hazptr_synchronize_cpu_slots(int cpu, void *addr)
> +{
> +	struct hazptr_percpu_slots *percpu_slots = per_cpu_ptr(&hazptr_percpu_slots, cpu);
> +	unsigned int idx;
> +
> +	for (idx = 0; idx < NR_HAZPTR_PERCPU_SLOTS; idx++) {
> +		struct hazptr_slot_item *item = &percpu_slots->items[idx];
> +
> +		/* Busy-wait if node is found. */
> +		smp_cond_load_acquire(&item->slot.addr, VAL != addr && VAL != HAZPTR_WILDCARD); /* Load B */
> +	}
> +}
> +
> +/*
> + * hazptr_synchronize: Wait until @addr is released from all slots.
> + *
> + * Wait to observe that each slot contains a value that differs from
> + * @addr before returning.
> + * Should be called from preemptible context.
> + */
> +void hazptr_synchronize(void *addr)
> +{
> +	int cpu;
> +
> +	/*
> +	 * Busy-wait should only be done from preemptible context.
> +	 */
> +	lockdep_assert_preemption_enabled();
> +
> +	/*
> +	 * Store A precedes hazptr_scan(): it unpublishes addr (sets it to
> +	 * NULL or to a different value), and thus hides it from hazard
> +	 * pointer readers.
> +	 */
> +	if (!addr)
> +		return;
> +	/* Memory ordering: Store A before Load B. */
> +	smp_mb();
> +	/* Scan all CPUs slots. */
> +	for_each_possible_cpu(cpu) {
> +		struct hazptr_overflow_list_flip *overflow_list_flip = per_cpu_ptr(&percpu_overflow_list_flip, cpu);
> +		unsigned int scan_idx;
> +
> +		/* Scan CPU slots. */
> +		hazptr_synchronize_cpu_slots(cpu, addr);
> +
> +		/*
> +		 * Scan backup slots in percpu overflow lists.
> +		 * Forward progress is guaranteed by scanning one list
> +		 * while new elements are added into the other list.
> +		 */
> +		guard(mutex)(&overflow_list_flip->lock);
> +		scan_idx = overflow_list_flip->add_idx ^ 1;
> +		hazptr_synchronize_overflow_list(&overflow_list_flip->array[scan_idx], addr);
> +		/* Flip current list. */
> +		WRITE_ONCE(overflow_list_flip->add_idx, scan_idx);
> +		hazptr_synchronize_overflow_list(&overflow_list_flip->array[scan_idx ^ 1], addr);
> +	}
> +}
> +EXPORT_SYMBOL_GPL(hazptr_synchronize);
> +
> +struct hazptr_slot *hazptr_chain_backup_slot(struct hazptr_ctx *ctx)
> +{
> +	struct hazptr_overflow_list_flip *overflow_list_flip = this_cpu_ptr(&percpu_overflow_list_flip);
> +	unsigned int list_idx = READ_ONCE(overflow_list_flip->add_idx);
> +	struct hazptr_overflow_list *overflow_list = &overflow_list_flip->array[list_idx];
> +	struct hazptr_slot *slot = &ctx->backup_slot.slot;
> +
> +	slot->addr = NULL;
> +	guard(raw_spinlock_irqsave)(&overflow_list->lock);
> +	overflow_list->gen++;
> +	hlist_add_head(&ctx->backup_slot.overflow_node, &overflow_list->head);
> +	ctx->backup_slot.overflow_list = overflow_list;
> +	return slot;
> +}
> +EXPORT_SYMBOL_GPL(hazptr_chain_backup_slot);
> +
> +void hazptr_unchain_backup_slot(struct hazptr_ctx *ctx)
> +{
> +	struct hazptr_overflow_list *overflow_list = ctx->backup_slot.overflow_list;
> +
> +	guard(raw_spinlock_irqsave)(&overflow_list->lock);
> +	overflow_list->gen++;
> +	hlist_del(&ctx->backup_slot.overflow_node);
> +}
> +EXPORT_SYMBOL_GPL(hazptr_unchain_backup_slot);
> +
> +void __init hazptr_init(void)
> +{
> +	int cpu;
> +
> +	for_each_possible_cpu(cpu) {
> +		struct hazptr_overflow_list_flip *overflow_list_flip = per_cpu_ptr(&percpu_overflow_list_flip, cpu);
> +
> +		mutex_init(&overflow_list_flip->lock);
> +		for (int i = 0; i < 2; i++) {
> +			raw_spin_lock_init(&overflow_list_flip->array[i].lock);
> +			INIT_HLIST_HEAD(&overflow_list_flip->array[i].head);
> +		}
> +	}
> +}
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 759777694c78..b3e10be20329 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -60,6 +60,7 @@
>  #include <linux/profile.h>
>  #include <linux/psi.h>
>  #include <linux/rcuwait_api.h>
> +#include <linux/hazptr.h>
>  #include <linux/rseq.h>
>  #include <linux/sched/wake_q.h>
>  #include <linux/scs.h>
> @@ -6790,6 +6791,7 @@ static void __sched notrace __schedule(int sched_mode)
>  	local_irq_disable();
>  	rcu_note_context_switch(preempt);
>  	migrate_disable_switch(rq, prev);
> +	hazptr_note_context_switch();
>  
>  	/*
>  	 * Make sure that signal_pending_state()->signal_pending() below
> -- 
> 2.39.5
> 

Re: [RFC PATCH v5 1/2] hazptr: Implement Hazard Pointers

[Mathieu Desnoyers]

On 2026-06-26 19:06, Paul E. McKenney wrote:
> [ Trimming CC. ]
> 
> On Mon, Feb 23, 2026 at 03:44:17PM -0500, Mathieu Desnoyers wrote:
>> This API provides existence guarantees of objects through Hazard
>> Pointers [1] (hazptr).
>>
>> Its main benefit over RCU is that it allows fast reclaim of
>> HP-protected pointers without needing to wait for a grace period.
>>
>> This implementation has 4 statically allocated hazard pointer slots per
>> cpu for the fast path, and relies on a on-stack backup slot allocated by
>> the hazard pointer user as fallback in case no per-cpu slot is
>> available.
>>
>> It integrates with the scheduler to migrate per-CPU slots to the backup
>> slot on context switch. This ensures that the per-CPU slots won't be
>> used by blocked or preempted tasks holding on hazard pointers for a long
>> time.
>>
>> References:
>>
>> [1]: M. M. Michael, "Hazard pointers: safe memory reclamation for
>>       lock-free objects," in IEEE Transactions on Parallel and
>>       Distributed Systems, vol. 15, no. 6, pp. 491-504, June 2004
> 
> I (finally) put together a stress test for this, which is the following
> commits on -rcu:
> 
> bfb239e036d616 torture: Add a hazptrtorture.c torture test
> adda38c3cffcd6 hazptrtorture: Add testing of on-stack hazptr_ctx structures
> 311e199e482873 hazptrtorture: Add microsecond-scale sleep in readers
> 21ccb41f79a45b hazptrtorture: Enable system-independent CPU overcommit
> 82e8c141265c81 torture: Add a stutter_will_wait() function
> 9ad8169bd3a367 hazptrtorture: Use mnemonic local variables for context information
> cd541d5eb7347f hazptrtorture: Split hazptr_torture_reader_tail() from hazptr_torture_reader()
> 0de8bda6b00051 hazptrtorture: Add kthread to release deferred hazard pointers
> 3deefef00acb22 hazptrtorture: Defer release of hazard pointers
> 65b2cf3c6ea34e hazptrtorture: Add irq_acquire to acquire hazptr from irq
> 98d334c04b58fe hazptrtorture: Use task_state_to_char() for task-state reporting
> 1e89aae50a3fde hazptrtorture: Pass hazptr_pending to hazptr_torture_reader_tail()
> 4b1a177190e2e1 hazptrtorture: Add the ability to disable the writer kthread
> 2c966f58358329 hazptrtorture: Add irq_release to release hazptr from irq
> 16258269b41ed3 hazptrtorture: Accumulate operation statistics
> c090e2659abcb3 doc: Add hazptrtorture module parameters
> 
> Initial runs suggest that hazard pointers cannot be acquired by one
> task and released by another.  Is that expected behavior, or is my test
> improperly passing the hazard pointers?  If the latter, we should of
> course document the proper hazard-pointer-passing mechanism.

This is unexpected. I've looked at your test code quickly and could
not figure out what's wrong though. But it's late. How do you observe
that it fails ?

Thanks,

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com

Re: [RFC PATCH v5 1/2] hazptr: Implement Hazard Pointers

[Paul E. McKenney]

On Fri, Jun 26, 2026 at 11:56:13PM -0400, Mathieu Desnoyers wrote:
> On 2026-06-26 19:06, Paul E. McKenney wrote:
> > [ Trimming CC. ]
> > 
> > On Mon, Feb 23, 2026 at 03:44:17PM -0500, Mathieu Desnoyers wrote:
> > > This API provides existence guarantees of objects through Hazard
> > > Pointers [1] (hazptr).
> > > 
> > > Its main benefit over RCU is that it allows fast reclaim of
> > > HP-protected pointers without needing to wait for a grace period.
> > > 
> > > This implementation has 4 statically allocated hazard pointer slots per
> > > cpu for the fast path, and relies on a on-stack backup slot allocated by
> > > the hazard pointer user as fallback in case no per-cpu slot is
> > > available.
> > > 
> > > It integrates with the scheduler to migrate per-CPU slots to the backup
> > > slot on context switch. This ensures that the per-CPU slots won't be
> > > used by blocked or preempted tasks holding on hazard pointers for a long
> > > time.
> > > 
> > > References:
> > > 
> > > [1]: M. M. Michael, "Hazard pointers: safe memory reclamation for
> > >       lock-free objects," in IEEE Transactions on Parallel and
> > >       Distributed Systems, vol. 15, no. 6, pp. 491-504, June 2004
> > 
> > I (finally) put together a stress test for this, which is the following
> > commits on -rcu:
> > 
> > bfb239e036d616 torture: Add a hazptrtorture.c torture test
> > adda38c3cffcd6 hazptrtorture: Add testing of on-stack hazptr_ctx structures
> > 311e199e482873 hazptrtorture: Add microsecond-scale sleep in readers
> > 21ccb41f79a45b hazptrtorture: Enable system-independent CPU overcommit
> > 82e8c141265c81 torture: Add a stutter_will_wait() function
> > 9ad8169bd3a367 hazptrtorture: Use mnemonic local variables for context information
> > cd541d5eb7347f hazptrtorture: Split hazptr_torture_reader_tail() from hazptr_torture_reader()
> > 0de8bda6b00051 hazptrtorture: Add kthread to release deferred hazard pointers
> > 3deefef00acb22 hazptrtorture: Defer release of hazard pointers
> > 65b2cf3c6ea34e hazptrtorture: Add irq_acquire to acquire hazptr from irq
> > 98d334c04b58fe hazptrtorture: Use task_state_to_char() for task-state reporting
> > 1e89aae50a3fde hazptrtorture: Pass hazptr_pending to hazptr_torture_reader_tail()
> > 4b1a177190e2e1 hazptrtorture: Add the ability to disable the writer kthread
> > 2c966f58358329 hazptrtorture: Add irq_release to release hazptr from irq
> > 16258269b41ed3 hazptrtorture: Accumulate operation statistics
> > c090e2659abcb3 doc: Add hazptrtorture module parameters
> > 
> > Initial runs suggest that hazard pointers cannot be acquired by one
> > task and released by another.  Is that expected behavior, or is my test
> > improperly passing the hazard pointers?  If the latter, we should of
> > course document the proper hazard-pointer-passing mechanism.
> 
> This is unexpected. I've looked at your test code quickly and could
> not figure out what's wrong though. But it's late. How do you observe
> that it fails ?

Not an emergency, so your schedule is my schedule.  Especially given
that I am mostly taking this coming week off.  And that it took me such
a long time to get this torture test done.  ;-)

I run either scenario (PREEMPT or NOPREEMPT) with:

--bootarg "hazptrtorture.onoff_interval=3333 hazptrtorture.stat_interval=15"

I get the following on the console, and things go downhill from there.
I get the same thing when I turn off CPU hotplug, for whatever that is
worth.

							Thanx, Paul

[    6.872932] Oops: general protection fault, probably for non-canonical address 0xdead000000000110: 0000 [#1] SMP PTI
[    6.874626] CPU: 4 UID: 0 PID: 136 Comm: hazptr_torture_ Not tainted 7.1.0-rc4-00070-g8b866e80f23c #1484 PREEMPT(full) 
[    6.876268] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.15.0-1 04/01/2014
[    6.877411] RIP: 0010:hazptr_synchronize_overflow_list+0x3e/0x90
[    6.878287] Code: 08 48 89 c6 48 85 db 74 54 4d 8b 65 10 eb 1b e8 d8 11 fa 00 f3 90 4c 89 ef e8 de 13 fa 00 48 89 c6 49 8b 45 10 4c 39 e0 75 41 <48> 8b 43 10 4c 89 ef 48 39 e8 74 d9 48 83 f8 01 74 d3 e8 ab 11 fa
[    6.881146] RSP: 0000:ffffbaf7c04ffe60 EFLAGS: 00010086
[    6.881938] RAX: 0000000000000276 RBX: dead000000000100 RCX: 0000000000000003
[    6.883028] RDX: 0000000000000001 RSI: 0000000000000286 RDI: ffffa084df4980e0
[    6.884119] RBP: ffffffffb72031c0 R08: 00000000002547d1 R09: 0000000000000004
[    6.885153] R10: 0000000000000001 R11: ffffa084df31c328 R12: 0000000000000276
[    6.886202] R13: ffffa084df4980e0 R14: ffffffffb72031c0 R15: ffffa084df4980c0
[    6.887213] FS:  0000000000000000(0000) GS:ffffa0852818f000(0000) knlGS:0000000000000000
[    6.888301] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    6.889061] CR2: 0000000000000000 CR3: 0000000007430000 CR4: 00000000000006f0
[    6.890041] Call Trace:
[    6.890408]  <TASK>
[    6.890725]  hazptr_synchronize+0xd1/0x130
[    6.891303]  hazptr_torture_writer+0x180/0x3f0
[    6.891921]  ? __pfx_hazptr_torture_writer+0x10/0x10
[    6.892613]  kthread+0xe5/0x120
[    6.893077]  ? __pfx_kthread+0x10/0x10
[    6.893612]  ret_from_fork+0x1bd/0x220
[    6.894155]  ? __pfx_kthread+0x10/0x10
[    6.894690]  ret_from_fork_asm+0x1a/0x30
[    6.895226]  </TASK>
[    6.895562] Modules linked in:
[    6.895958] ---[ end trace 0000000000000000 ]---

Re: [RFC PATCH v5 1/2] hazptr: Implement Hazard Pointers

[Mathieu Desnoyers]

On 2026-06-27 00:14, Paul E. McKenney wrote:
> On Fri, Jun 26, 2026 at 11:56:13PM -0400, Mathieu Desnoyers wrote:
>> On 2026-06-26 19:06, Paul E. McKenney wrote:
[...]
>>> Initial runs suggest that hazard pointers cannot be acquired by one
>>> task and released by another.  Is that expected behavior, or is my test
>>> improperly passing the hazard pointers?  If the latter, we should of
>>> course document the proper hazard-pointer-passing mechanism.
>>
>> This is unexpected. I've looked at your test code quickly and could
>> not figure out what's wrong though. But it's late. How do you observe
>> that it fails ?
> 
> Not an emergency, so your schedule is my schedule.  Especially given
> that I am mostly taking this coming week off.  And that it took me such
> a long time to get this torture test done.  ;-)
> 
> I run either scenario (PREEMPT or NOPREEMPT) with:
> 
> --bootarg "hazptrtorture.onoff_interval=3333 hazptrtorture.stat_interval=15"
> 
> I get the following on the console, and things go downhill from there.
> I get the same thing when I turn off CPU hotplug, for whatever that is
> worth.
That OOPS is very instructive. I think I found the root cause.

I need to add an API to hazptr so users wishing to explicitly transfer
ownership of the hazptr ctx to another thread can do so.

Currently the scheduler hook is responsible for moving the hazptr
ctx from per-cpu to the backup slots, which makes it entirely movable
afterwards.

But this assumes the ctx is thread-local. Once you explicitly move
ownership of that ctx to another thread, you race with the scheduler of
the original thread.

Let me see what I can do to fix that.

Thanks a lot for the test-case !

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com

Re: [RFC PATCH v5 1/2] hazptr: Implement Hazard Pointers

[Mathieu Desnoyers]

[-- Attachment #1: Type: text/plain, Size: 1919 bytes --]

On 2026-06-27 08:56, Mathieu Desnoyers wrote:
> On 2026-06-27 00:14, Paul E. McKenney wrote:
>> On Fri, Jun 26, 2026 at 11:56:13PM -0400, Mathieu Desnoyers wrote:
>>> On 2026-06-26 19:06, Paul E. McKenney wrote:
> [...]
>>>> Initial runs suggest that hazard pointers cannot be acquired by one
>>>> task and released by another.  Is that expected behavior, or is my test
>>>> improperly passing the hazard pointers?  If the latter, we should of
>>>> course document the proper hazard-pointer-passing mechanism.
>>>
>>> This is unexpected. I've looked at your test code quickly and could
>>> not figure out what's wrong though. But it's late. How do you observe
>>> that it fails ?
>>
>> Not an emergency, so your schedule is my schedule.  Especially given
>> that I am mostly taking this coming week off.  And that it took me such
>> a long time to get this torture test done.  ;-)
>>
>> I run either scenario (PREEMPT or NOPREEMPT) with:
>>
>> --bootarg "hazptrtorture.onoff_interval=3333 
>> hazptrtorture.stat_interval=15"
>>
>> I get the following on the console, and things go downhill from there.
>> I get the same thing when I turn off CPU hotplug, for whatever that is
>> worth.
> That OOPS is very instructive. I think I found the root cause.
> 
> I need to add an API to hazptr so users wishing to explicitly transfer
> ownership of the hazptr ctx to another thread can do so.
> 
> Currently the scheduler hook is responsible for moving the hazptr
> ctx from per-cpu to the backup slots, which makes it entirely movable
> afterwards.
> 
> But this assumes the ctx is thread-local. Once you explicitly move
> ownership of that ctx to another thread, you race with the scheduler of
> the original thread.
> 
> Let me see what I can do to fix that.
> 
> Thanks a lot for the test-case !

Can you try with the attached POC diff ?

Thanks,

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com

[-- Attachment #2: hazptr-fix-handoff.patch --]
[-- Type: text/x-patch, Size: 2776 bytes --]

diff --git a/include/linux/hazptr.h b/include/linux/hazptr.h
index 461f481a480b..d96414575739 100644
--- a/include/linux/hazptr.h
+++ b/include/linux/hazptr.h
@@ -98,6 +98,41 @@ bool hazptr_slot_is_backup(struct hazptr_ctx *ctx, struct hazptr_slot *slot)
 	return slot == &ctx->backup_slot.slot;
 }
 
+/* Internal helper. */
+static inline
+void hazptr_promote_to_backup_slot(struct hazptr_ctx *ctx, struct hazptr_slot *slot)
+{
+	struct hazptr_slot *backup_slot;
+
+	backup_slot = hazptr_chain_backup_slot(ctx);
+	/*
+	 * Move hazard pointer from the per-CPU slot to the
+	 * backup slot. This requires hazard pointer
+	 * synchronize to iterate on per-CPU slots with
+	 * load-acquire before iterating on the overflow list.
+	 */
+	WRITE_ONCE(backup_slot->addr, slot->addr);
+	/*
+	 * store-release orders store to backup slot addr before
+	 * store to per-CPU slot addr.
+	 */
+	smp_store_release(&slot->addr, NULL);
+	/* Use the backup slot for context. */
+	ctx->slot = backup_slot;
+}
+
+static inline
+void hazptr_detach_from_task(struct hazptr_ctx *ctx)
+{
+	struct hazptr_slot *slot;
+
+	guard(preempt)();
+	slot = ctx->slot;
+	if (unlikely(hazptr_slot_is_backup(ctx, slot)))
+		return;
+	hazptr_promote_to_backup_slot(ctx, slot);
+}
+
 static inline
 void hazptr_note_context_switch(void)
 {
@@ -106,27 +141,11 @@ void hazptr_note_context_switch(void)
 
 	for (idx = 0; idx < NR_HAZPTR_PERCPU_SLOTS; idx++) {
 		struct hazptr_slot_item *item = &percpu_slots->items[idx];
-		struct hazptr_slot *slot = &item->slot, *backup_slot;
-		struct hazptr_ctx *ctx;
+		struct hazptr_slot *slot = &item->slot;
 
 		if (!slot->addr)
 			continue;
-		ctx = item->ctx.ctx;
-		backup_slot = hazptr_chain_backup_slot(ctx);
-		/*
-		 * Move hazard pointer from the per-CPU slot to the
-		 * backup slot. This requires hazard pointer
-		 * synchronize to iterate on per-CPU slots with
-		 * load-acquire before iterating on the overflow list.
-		 */
-		WRITE_ONCE(backup_slot->addr, slot->addr);
-		/*
-		 * store-release orders store to backup slot addr before
-		 * store to per-CPU slot addr.
-		 */
-		smp_store_release(&slot->addr, NULL);
-		/* Use the backup slot for context. */
-		ctx->slot = backup_slot;
+		hazptr_promote_to_backup_slot(item->ctx.ctx, slot);
 	}
 }
 
diff --git a/kernel/rcu/hazptrtorture.c b/kernel/rcu/hazptrtorture.c
index 311b387d6ae7..93b383887fc5 100644
--- a/kernel/rcu/hazptrtorture.c
+++ b/kernel/rcu/hazptrtorture.c
@@ -427,6 +427,7 @@ static void hazptr_torture_defer(struct hazptr_pending *hppp, struct torture_ran
 	int cpu = torture_random(trsp) % nr_cpu_ids;
 	struct llist_head *llhp;
 
+	hazptr_detach_from_task(&hppp->hpp_hc);
 	guard(preempt)();
 	cpu = cpumask_next_wrap(cpu, cpu_online_mask);
 	llhp = per_cpu_ptr(&hazptr_pending, cpu);

Re: [RFC PATCH v5 1/2] hazptr: Implement Hazard Pointers

[Paul E. McKenney]

On Sat, Jun 27, 2026 at 11:21:28AM -0400, Mathieu Desnoyers wrote:
> On 2026-06-27 08:56, Mathieu Desnoyers wrote:
> > On 2026-06-27 00:14, Paul E. McKenney wrote:
> > > On Fri, Jun 26, 2026 at 11:56:13PM -0400, Mathieu Desnoyers wrote:
> > > > On 2026-06-26 19:06, Paul E. McKenney wrote:
> > [...]
> > > > > Initial runs suggest that hazard pointers cannot be acquired by one
> > > > > task and released by another.  Is that expected behavior, or is my test
> > > > > improperly passing the hazard pointers?  If the latter, we should of
> > > > > course document the proper hazard-pointer-passing mechanism.
> > > > 
> > > > This is unexpected. I've looked at your test code quickly and could
> > > > not figure out what's wrong though. But it's late. How do you observe
> > > > that it fails ?
> > > 
> > > Not an emergency, so your schedule is my schedule.  Especially given
> > > that I am mostly taking this coming week off.  And that it took me such
> > > a long time to get this torture test done.  ;-)
> > > 
> > > I run either scenario (PREEMPT or NOPREEMPT) with:
> > > 
> > > --bootarg "hazptrtorture.onoff_interval=3333
> > > hazptrtorture.stat_interval=15"
> > > 
> > > I get the following on the console, and things go downhill from there.
> > > I get the same thing when I turn off CPU hotplug, for whatever that is
> > > worth.
> > That OOPS is very instructive. I think I found the root cause.
> > 
> > I need to add an API to hazptr so users wishing to explicitly transfer
> > ownership of the hazptr ctx to another thread can do so.
> > 
> > Currently the scheduler hook is responsible for moving the hazptr
> > ctx from per-cpu to the backup slots, which makes it entirely movable
> > afterwards.
> > 
> > But this assumes the ctx is thread-local. Once you explicitly move
> > ownership of that ctx to another thread, you race with the scheduler of
> > the original thread.
> > 
> > Let me see what I can do to fix that.
> > 
> > Thanks a lot for the test-case !

Glad it helped?  ;-)

> Can you try with the attached POC diff ?

This increases the kernel's life significantly, but still fails as
shown below.  This is the NOPREEMPT scenario as follows:

tools/testing/selftests/rcutorture/bin/kvm.sh --torture hazptr --allcpus --duration 5m --bootargs "hazptrtorture.onoff_interval=3333 hazptrtorture.nwriters=1 hazptrtorture.stat_interval=15" --trust-make

The PREEMPT failure gets similar failures, and the above command runs
both of these scenarios.

							Thanx, Paul

> Thanks,
> 
> Mathieu
> 
> -- 
> Mathieu Desnoyers
> EfficiOS Inc.
> https://www.efficios.com

> diff --git a/include/linux/hazptr.h b/include/linux/hazptr.h
> index 461f481a480b..d96414575739 100644
> --- a/include/linux/hazptr.h
> +++ b/include/linux/hazptr.h
> @@ -98,6 +98,41 @@ bool hazptr_slot_is_backup(struct hazptr_ctx *ctx, struct hazptr_slot *slot)
>  	return slot == &ctx->backup_slot.slot;
>  }
>  
> +/* Internal helper. */
> +static inline
> +void hazptr_promote_to_backup_slot(struct hazptr_ctx *ctx, struct hazptr_slot *slot)
> +{
> +	struct hazptr_slot *backup_slot;
> +
> +	backup_slot = hazptr_chain_backup_slot(ctx);
> +	/*
> +	 * Move hazard pointer from the per-CPU slot to the
> +	 * backup slot. This requires hazard pointer
> +	 * synchronize to iterate on per-CPU slots with
> +	 * load-acquire before iterating on the overflow list.
> +	 */
> +	WRITE_ONCE(backup_slot->addr, slot->addr);
> +	/*
> +	 * store-release orders store to backup slot addr before
> +	 * store to per-CPU slot addr.
> +	 */
> +	smp_store_release(&slot->addr, NULL);
> +	/* Use the backup slot for context. */
> +	ctx->slot = backup_slot;
> +}
> +
> +static inline
> +void hazptr_detach_from_task(struct hazptr_ctx *ctx)
> +{
> +	struct hazptr_slot *slot;
> +
> +	guard(preempt)();
> +	slot = ctx->slot;
> +	if (unlikely(hazptr_slot_is_backup(ctx, slot)))
> +		return;
> +	hazptr_promote_to_backup_slot(ctx, slot);
> +}
> +
>  static inline
>  void hazptr_note_context_switch(void)
>  {
> @@ -106,27 +141,11 @@ void hazptr_note_context_switch(void)
>  
>  	for (idx = 0; idx < NR_HAZPTR_PERCPU_SLOTS; idx++) {
>  		struct hazptr_slot_item *item = &percpu_slots->items[idx];
> -		struct hazptr_slot *slot = &item->slot, *backup_slot;
> -		struct hazptr_ctx *ctx;
> +		struct hazptr_slot *slot = &item->slot;
>  
>  		if (!slot->addr)
>  			continue;
> -		ctx = item->ctx.ctx;
> -		backup_slot = hazptr_chain_backup_slot(ctx);
> -		/*
> -		 * Move hazard pointer from the per-CPU slot to the
> -		 * backup slot. This requires hazard pointer
> -		 * synchronize to iterate on per-CPU slots with
> -		 * load-acquire before iterating on the overflow list.
> -		 */
> -		WRITE_ONCE(backup_slot->addr, slot->addr);
> -		/*
> -		 * store-release orders store to backup slot addr before
> -		 * store to per-CPU slot addr.
> -		 */
> -		smp_store_release(&slot->addr, NULL);
> -		/* Use the backup slot for context. */
> -		ctx->slot = backup_slot;
> +		hazptr_promote_to_backup_slot(item->ctx.ctx, slot);
>  	}
>  }
>  
> diff --git a/kernel/rcu/hazptrtorture.c b/kernel/rcu/hazptrtorture.c
> index 311b387d6ae7..93b383887fc5 100644
> --- a/kernel/rcu/hazptrtorture.c
> +++ b/kernel/rcu/hazptrtorture.c
> @@ -427,6 +427,7 @@ static void hazptr_torture_defer(struct hazptr_pending *hppp, struct torture_ran
>  	int cpu = torture_random(trsp) % nr_cpu_ids;
>  	struct llist_head *llhp;
>  
> +	hazptr_detach_from_task(&hppp->hpp_hc);
>  	guard(preempt)();
>  	cpu = cpumask_next_wrap(cpu, cpu_online_mask);
>  	llhp = per_cpu_ptr(&hazptr_pending, cpu);

------------------------------------------------------------------------

[   16.477672] hazptr-stack-torture: rtc: 00000000927f2a0a ver: 273 HTWS_SYNC tfle: 0 rta: 274 rtaf: 0 rtf: 264 onoff: 1/1:4/4 7,7:10,14 7:44 (HZ=1000) acq: 268127149 rel: 268125688 acqirq: 15502 relirq: 16773 reldefer: 0 relundefer 0
[   16.480858] hazptr-stack-torture: Reader Pipe:  268140258 1798 0 0 0 0 0 0 0 0 0
[   16.482093] hazptr-stack-torture: Reader Batch:  0 0 0 0 0 0 0 0 0 0 0
[   16.483149] hazptr-stack-torture: Free-Block Circulation:  273 272 271 270 269 268 267 266 265 264 0
[   18.133721] clocksource: Watchdog remote CPU 8 read timed out
[   18.652293] Oops: general protection fault, probably for non-canonical address 0xdead000000000110: 0000 [#1] SMP PTI
[   18.653967] CPU: 13 UID: 0 PID: 132 Comm: hazptr_torture_ Not tainted 7.1.0-rc4-00071-g136a43a847a7-dirty #1494 PREEMPTLAZY 
[   18.655726] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.15.0-1 04/01/2014
[   18.656999] RIP: 0010:hazptr_synchronize_overflow_list+0x3e/0x90
[   18.657968] Code: 08 48 89 c6 48 85 db 74 54 4d 8b 65 10 eb 1b e8 78 2d e0 00 f3 90 4c 89 ef e8 7e 2f e0 00 48 89 c6 49 8b 45 10 4c 39 e0 75 41 <48> 8b 43 10 4c 89 ef 48 39 e8 74 d9 48 83 f8 01 74 d3 e8 4b 2d e0
[   18.660803] RSP: 0000:ffffb255004dfe60 EFLAGS: 00010086
[   18.661592] RAX: 00000000000013b2 RBX: dead000000000100 RCX: 0000000000000003
[   18.662706] RDX: 0000000000000001 RSI: 0000000000000286 RDI: ffff9cdc5f3d8060
[   18.663788] RBP: ffffffffa21a7f00 R08: 000000000006d924 R09: 000000000000000d
[   18.664864] R10: 0000000000000001 R11: 0000000000000000 R12: 00000000000013b2
[   18.665975] R13: ffff9cdc5f3d8060 R14: ffffffffa21a7f00 R15: ffff9cdc5f3d8040
[   18.667095] FS:  0000000000000000(0000) GS:ffff9cdcbd421000(0000) knlGS:0000000000000000
[   18.668332] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   18.669228] CR2: 0000000000000000 CR3: 0000000018030000 CR4: 00000000000006f0
[   18.670328] Call Trace:
[   18.670735]  <TASK>
[   18.671077]  hazptr_synchronize+0xd1/0x130
[   18.671744]  hazptr_torture_writer+0x180/0x3c0
[   18.672449]  ? __pfx_hazptr_torture_writer+0x10/0x10
[   18.673228]  kthread+0xd7/0x110
[   18.673719]  ? __pfx_kthread+0x10/0x10
[   18.674281]  ret_from_fork+0x1bd/0x220
[   18.674867]  ? __pfx_kthread+0x10/0x10
[   18.675462]  ret_from_fork_asm+0x1a/0x30
[   18.676058]  </TASK>
[   18.676424] Modules linked in:
[   18.676900] ---[ end trace 0000000000000000 ]---
[   18.677596] RIP: 0010:hazptr_synchronize_overflow_list+0x3e/0x90
[   18.678504] Code: 08 48 89 c6 48 85 db 74 54 4d 8b 65 10 eb 1b e8 78 2d e0 00 f3 90 4c 89 ef e8 7e 2f e0 00 48 89 c6 49 8b 45 10 4c 39 e0 75 41 <48> 8b 43 10 4c 89 ef 48 39 e8 74 d9 48 83 f8 01 74 d3 e8 4b 2d e0
[   18.681215] RSP: 0000:ffffb255004dfe60 EFLAGS: 00010086
[   18.681816] RAX: 00000000000013b2 RBX: dead000000000100 RCX: 0000000000000003
[   18.682950] RDX: 0000000000000001 RSI: 0000000000000286 RDI: ffff9cdc5f3d8060
[   18.683736] RBP: ffffffffa21a7f00 R08: 000000000006d924 R09: 000000000000000d
[   18.684812] R10: 0000000000000001 R11: 0000000000000000 R12: 00000000000013b2
[   18.685886] R13: ffff9cdc5f3d8060 R14: ffffffffa21a7f00 R15: ffff9cdc5f3d8040
[   18.686652] FS:  0000000000000000(0000) GS:ffff9cdcbd421000(0000) knlGS:0000000000000000
[   18.687509] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   18.688134] CR2: 0000000000000000 CR3: 0000000018030000 CR4: 00000000000006f0
[   18.689205] note: hazptr_torture_[132] exited with irqs disabled
[   18.689861] note: hazptr_torture_[132] exited with preempt_count 1
[   18.690519] hazptr_torture_ (132) used greatest stack depth: 13704 bytes left
[   18.695774] smpboot: CPU 2 is now offline
[   22.039232] smpboot: CPU 5 is now offline
[   25.376816] smpboot: Booting Node 0 Processor 2 APIC 0x2
[   31.837655] hazptr-stack-torture: rtc: 0000000073942ea9 ver: 359 HTWS_SYNC tfle: 0 rta: 360 rtaf: 0 rtf: 350 onoff: 2/2:6/6 7,7:10,50 14:104 (HZ=1000) acq: 727372071 rel: 727368302 acqirq: 42089 relirq: 45682 reldefer: 0 relundefer 0
[   31.840069] hazptr-stack-torture: Reader Pipe:  727411641 2343 0 0 0 0 0 0 0 0 0
[   31.840922] hazptr-stack-torture: Reader Batch:  0 0 0 0 0 0 0 0 0 0 0
[   31.841674] hazptr-stack-torture: Free-Block Circulation:  359 358 357 356 355 354 353 352 351 350 0
[   32.057111] smpboot: CPU 7 is now offline
[   35.401073] smpboot: CPU 15 is now offline
[   38.741634] smpboot: Booting Node 0 Processor 1 APIC 0x1
[   42.089276] smpboot: CPU 3 is now offline
[   45.426763] smpboot: Booting Node 0 Processor 15 APIC 0xf
[   47.199600] hazptr-stack-torture: rtc: 0000000073942ea9 ver: 359 HTWS_SYNC tfle: 0 rta: 360 rtaf: 0 rtf: 350 onoff: 4/4:9/9 7,10:10,50 31:135 (HZ=1000) acq: 977618948 rel: 977613883 acqirq: 56384 relirq: 61259 reldefer: 0 relundefer 0
[   47.203087] hazptr-stack-torture: Reader Pipe:  977672190 2343 0 0 0 0 0 0 0 0 0
[   47.204315] hazptr-stack-torture: Reader Batch:  0 0 0 0 0 0 0 0 0 0 0
[   47.205430] hazptr-stack-torture: Free-Block Circulation:  359 358 357 356 355 354 353 352 351 350 0
[   47.206996] ??? Writer stall state HTWS_SYNC(3) g0 f0x0 ->state X cpu 13
[   48.781598] smpboot: CPU 11 is now offline
[   52.125175] smpboot: CPU 1 is now offline
[   55.468194] smpboot: CPU 8 is now offline
[   58.816598] smpboot: CPU 9 is now offline
[   62.160238] smpboot: CPU 10 is now offline
[   62.208650] hazptr-stack-torture: rtc: 0000000073942ea9 ver: 359 HTWS_SYNC tfle: 0 rta: 360 rtaf: 0 rtf: 350 onoff: 4/4:14/14 7,10:10,50 31:197 (HZ=1000) acq: 1336717896 rel: 1336710172 acqirq: 76179 relirq: 83727 reldefer: 0 relundefer 0
[   62.212039] hazptr-stack-torture: Reader Pipe:  1336791556 2343 0 0 0 0 0 0 0 0 0
[   62.213106] hazptr-stack-torture: Reader Batch:  0 0 0 0 0 0 0 0 0 0 0
[   62.214028] hazptr-stack-torture: Free-Block Circulation:  359 358 357 356 355 354 353 352 351 350 0
[   62.215326] ??? Writer stall state HTWS_SYNC(3) g0 f0x0 ->state X cpu 13
[   65.504487] smpboot: CPU 4 is now offline
[   72.176750] smpboot: Booting Node 0 Processor 5 APIC 0x5
[   75.517839] smpboot: Booting Node 0 Processor 11 APIC 0xb
[   77.406600] hazptr-stack-torture: rtc: 0000000073942ea9 ver: 359 HTWS_SYNC tfle: 0 rta: 360 rtaf: 0 rtf: 350 onoff: 6/6:15/15 7,10:10,50 47:209 (HZ=1000) acq: 1513030623 rel: 1513021030 acqirq: 85225 relirq: 94635 reldefer: 0 relundefer 0
[   77.409704] hazptr-stack-torture: Reader Pipe:  1513113120 2343 0 0 0 0 0 0 0 0 0
[   77.410838] hazptr-stack-torture: Reader Batch:  0 0 0 0 0 0 0 0 0 0 0
[   77.411832] hazptr-stack-torture: Free-Block Circulation:  359 358 357 356 355 354 353 352 351 350 0
[   77.413108] ??? Writer stall state HTWS_SYNC(3) g0 f0x0 ->state X cpu 13
[   78.860630] smpboot: Booting Node 0 Processor 10 APIC 0xa
[   82.209063] smpboot: CPU 10 is now offline
[   85.546766] smpboot: Booting Node 0 Processor 3 APIC 0x3
[   88.899717] smpboot: CPU 13 is now offline
[   92.237759] smpboot: Booting Node 0 Processor 1 APIC 0x1
[   92.765706] hazptr-stack-torture: rtc: 0000000073942ea9 ver: 359 HTWS_SYNC tfle: 0 rta: 360 rtaf: 0 rtf: 350 onoff: 9/9:17/17 7,10:10,50 70:235 (HZ=1000) acq: 1815314543 rel: 1815302193 acqirq: 101451 relirq: 113625 reldefer: 0 relundefer 0
[   92.769757] hazptr-stack-torture: Reader Pipe:  1815413475 2343 0 0 0 0 0 0 0 0 0
[   92.771206] hazptr-stack-torture: Reader Batch:  0 0 0 0 0 0 0 0 0 0 0
[   92.772480] hazptr-stack-torture: Free-Block Circulation:  359 358 357 356 355 354 353 352 351 350 0
[   92.774229] ??? Writer stall state HTWS_SYNC(3) g0 f0x0 ->state X cpu 13
[   95.577713] smpboot: Booting Node 0 Processor 6 APIC 0x6
[   98.920645] smpboot: Booting Node 0 Processor 4 APIC 0x4
[  102.273055] smpboot: CPU 3 is now offline
[  105.616678] smpboot: CPU 11 is now offline
[  108.126606] hazptr-stack-torture: rtc: 0000000073942ea9 ver: 359 HTWS_SYNC tfle: 0 rta: 360 rtaf: 0 rtf: 350 onoff: 11/11:19/19 7,15:10,50 93:256 (HZ=1000) acq: 2100515525 rel: 2100501173 acqirq: 117324 relirq: 131491 reldefer: 0 relundefer 0
[  108.129407] hazptr-stack-torture: Reader Pipe:  2100629725 2343 0 0 0 0 0 0 0 0 0
[  108.130408] hazptr-stack-torture: Reader Batch:  0 0 0 0 0 0 0 0 0 0 0
[  108.131460] hazptr-stack-torture: Free-Block Circulation:  359 358 357 356 355 354 353 352 351 350 0
[  108.132682] ??? Writer stall state HTWS_SYNC(3) g0 f0x0 ->state X cpu 13
[  108.964599] smpboot: CPU 4 is now offline
[  112.308116] smpboot: CPU 6 is now offline
[  115.645725] smpboot: Booting Node 0 Processor 10 APIC 0xa
[  118.987152] smpboot: Booting Node 0 Processor 7 APIC 0x7
[  122.334115] smpboot: CPU 5 is now offline
[  123.485674] hazptr-stack-torture: rtc: 0000000073942ea9 ver: 359 HTWS_SYNC tfle: 0 rta: 360 rtaf: 0 rtf: 350 onoff: 13/13:22/22 7,15:10,50 108:289 (HZ=1000) acq: 2385152588 rel: 2385135762 acqirq: 132665 relirq: 149315 reldefer: 0 relundefer 0
[  123.488311] hazptr-stack-torture: Reader Pipe:  2385282734 2343 0 0 0 0 0 0 0 0 0
[  123.489243] hazptr-stack-torture: Reader Batch:  0 0 0 0 0 0 0 0 0 0 0
[  123.490188] hazptr-stack-torture: Free-Block Circulation:  359 358 357 356 355 354 353 352 351 350 0
[  123.491340] ??? Writer stall state HTWS_SYNC(3) g0 f0x0 ->state X cpu 13
[  125.677361] smpboot: CPU 12 is now offline
[  129.017162] smpboot: Booting Node 0 Processor 4 APIC 0x4
[  132.358758] smpboot: Booting Node 0 Processor 5 APIC 0x5
[  135.705206] smpboot: CPU 7 is now offline
[  138.846403] hazptr-stack-torture: rtc: 0000000073942ea9 ver: 359 HTWS_SYNC tfle: 0 rta: 360 rtaf: 0 rtf: 350 onoff: 15/15:24/24 7,15:10,50 125:309 (HZ=1000) acq: 2601900628 rel: 2601881685 acqirq: 144045 relirq: 162805 reldefer: 0 relundefer 0
[  138.849806] hazptr-stack-torture: Reader Pipe:  2602041602 2343 0 0 0 0 0 0 0 0 0
[  138.850887] hazptr-stack-torture: Reader Batch:  0 0 0 0 0 0 0 0 0 0 0
[  138.851839] hazptr-stack-torture: Free-Block Circulation:  359 358 357 356 355 354 353 352 351 350 0
[  138.853198] ??? Writer stall state HTWS_SYNC(3) g0 f0x0 ->state X cpu 13
[  139.044658] smpboot: Booting Node 0 Processor 3 APIC 0x3
[  145.721755] smpboot: Booting Node 0 Processor 6 APIC 0x6
[  149.062641] smpboot: Booting Node 0 Processor 12 APIC 0xc
[  152.404795] smpboot: Booting Node 0 Processor 13 APIC 0xd
[  154.205637] hazptr-stack-torture: rtc: 0000000073942ea9 ver: 359 HTWS_SYNC tfle: 0 rta: 360 rtaf: 0 rtf: 350 onoff: 19/19:24/24 7,15:10,50 158:309 (HZ=1000) acq: 2891489001 rel: 2891468129 acqirq: 160165 relirq: 180861 reldefer: 0 relundefer 0
[  154.208700] hazptr-stack-torture: Reader Pipe:  2891646647 2343 0 0 0 0 0 0 0 0 0
[  154.209779] hazptr-stack-torture: Reader Batch:  0 0 0 0 0 0 0 0 0 0 0
[  154.210707] hazptr-stack-torture: Free-Block Circulation:  359 358 357 356 355 354 353 352 351 350 0
[  154.211966] ??? Writer stall state HTWS_SYNC(3) g0 f0x0 ->state X cpu 13
[  155.745783] smpboot: Booting Node 0 Processor 14 APIC 0xe
[  159.090655] smpboot: Booting Node 0 Processor 8 APIC 0x8
[  162.439216] smpboot: CPU 4 is now offline
[  165.782180] smpboot: CPU 8 is now offline
[  169.133489] smpboot: CPU 14 is now offline
[  169.567402] hazptr-stack-torture: rtc: 0000000073942ea9 ver: 359 HTWS_SYNC tfle: 0 rta: 360 rtaf: 0 rtf: 350 onoff: 21/21:27/27 7,15:10,50 178:346 (HZ=1000) acq: 3279646888 rel: 3279623874 acqirq: 182381 relirq: 205207 reldefer: 0 relundefer 0
[  169.571174] hazptr-stack-torture: Reader Pipe:  3279826075 2343 0 0 0 0 0 0 0 0 0
[  169.572554] hazptr-stack-torture: Reader Batch:  0 0 0 0 0 0 0 0 0 0 0
[  169.573771] hazptr-stack-torture: Free-Block Circulation:  359 358 357 356 355 354 353 352 351 350 0
[  169.575421] ??? Writer stall state HTWS_SYNC(3) g0 f0x0 ->state X cpu 13
[  172.477225] smpboot: CPU 13 is now offline
[  175.815697] smpboot: Booting Node 0 Processor 13 APIC 0xd
[  179.158957] smpboot: Booting Node 0 Processor 7 APIC 0x7
[  182.508261] smpboot: CPU 1 is now offline
[  184.605659] hazptr-stack-torture: rtc: 0000000073942ea9 ver: 359 HTWS_SYNC tfle: 0 rta: 360 rtaf: 0 rtf: 350 onoff: 23/23:29/29 7,15:10,50 197:366 (HZ=1000) acq: 3558237081 rel: 3558212141 acqirq: 197989 relirq: 222753 reldefer: 0 relundefer 0
[  184.608653] hazptr-stack-torture: Reader Pipe:  3558432551 2343 0 0 0 0 0 0 0 0 0
[  184.609616] hazptr-stack-torture: Reader Batch:  0 0 0 0 0 0 0 0 0 0 0
[  184.610722] hazptr-stack-torture: Free-Block Circulation:  359 358 357 356 355 354 353 352 351 350 0
[  184.612241] ??? Writer stall state HTWS_SYNC(3) g0 f0x0 ->state X cpu 13
[  185.852919] smpboot: CPU 5 is now offline
[  189.195273] smpboot: Booting Node 0 Processor 9 APIC 0x9
[  192.544150] smpboot: CPU 7 is now offline
[  195.887234] smpboot: CPU 6 is now offline
[  199.226674] smpboot: Booting Node 0 Processor 14 APIC 0xe
[  199.774416] hazptr-stack-torture: rtc: 0000000073942ea9 ver: 359 HTWS_SYNC tfle: 0 rta: 360 rtaf: 0 rtf: 350 onoff: 25/25:32/32 7,15:10,50 221:398 (HZ=1000) acq: 3878923397 rel: 3878895983 acqirq: 215530 relirq: 242758 reldefer: 0 relundefer 0
[  199.778265] hazptr-stack-torture: Reader Pipe:  3879136232 2343 0 0 0 0 0 0 0 0 0
[  199.779338] hazptr-stack-torture: Reader Batch:  0 0 0 0 0 0 0 0 0 0 0
[  199.780277] hazptr-stack-torture: Free-Block Circulation:  359 358 357 356 355 354 353 352 351 350 0
[  199.781617] ??? Writer stall state HTWS_SYNC(3) g0 f0x0 ->state X cpu 13
[  202.570760] smpboot: Booting Node 0 Processor 7 APIC 0x7
[  205.917183] smpboot: CPU 10 is now offline
[  209.268602] smpboot: CPU 13 is now offline
[  212.606780] smpboot: Booting Node 0 Processor 10 APIC 0xa
[  215.133662] hazptr-stack-torture: rtc: 0000000073942ea9 ver: 359 HTWS_SYNC tfle: 0 rta: 360 rtaf: 0 rtf: 350 onoff: 27/27:34/34 7,15:10,50 235:425 (HZ=1000) acq: 4111862347 rel: 4111833331 acqirq: 228399 relirq: 257239 reldefer: 0 relundefer 0
[  215.136064] hazptr-stack-torture: Reader Pipe:  4112088227 2343 0 0 0 0 0 0 0 0 0
[  215.136819] hazptr-stack-torture: Reader Batch:  0 0 0 0 0 0 0 0 0 0 0
[  215.137467] hazptr-stack-torture: Free-Block Circulation:  359 358 357 356 355 354 353 352 351 350 0
[  215.138386] ??? Writer stall state HTWS_SYNC(3) g0 f0x0 ->state X cpu 13
[  215.947665] smpboot: Booting Node 0 Processor 8 APIC 0x8
[  219.301118] smpboot: CPU 14 is now offline
[  222.645272] smpboot: CPU 7 is now offline
[  225.988823] smpboot: CPU 8 is now offline
[  229.337095] smpboot: CPU 10 is now offline
[  230.494608] hazptr-stack-torture: rtc: 0000000073942ea9 ver: 359 HTWS_SYNC tfle: 0 rta: 360 rtaf: 0 rtf: 350 onoff: 28/28:38/38 7,15:10,50 242:477 (HZ=1000) acq: 4451936851 rel: 4451905359 acqirq: 247133 relirq: 278442 reldefer: 0 relundefer 0
[  230.499098] hazptr-stack-torture: Reader Pipe:  4452181088 2343 0 0 0 0 0 0 0 0 0
[  230.500667] hazptr-stack-torture: Reader Batch:  0 0 0 0 0 0 0 0 0 0 0
[  230.502040] hazptr-stack-torture: Free-Block Circulation:  359 358 357 356 355 354 353 352 351 350 0
[  230.504007] ??? Writer stall state HTWS_SYNC(3) g0 f0x0 ->state X cpu 13
[  232.675700] smpboot: Booting Node 0 Processor 6 APIC 0x6
[  236.021288] smpboot: CPU 2 is now offline
[  239.358045] smpboot: Booting Node 0 Processor 8 APIC 0x8
[  242.704108] smpboot: CPU 15 is now offline
[  245.853640] hazptr-stack-torture: rtc: 0000000073942ea9 ver: 359 HTWS_SYNC tfle: 0 rta: 360 rtaf: 0 rtf: 350 onoff: 30/30:40/40 7,15:10,50 257:497 (HZ=1000) acq: 4631094358 rel: 4631061240 acqirq: 256471 relirq: 289413 reldefer: 0 relundefer 0
[  245.857674] hazptr-stack-torture: Reader Pipe:  4631348310 2343 0 0 0 0 0 0 0 0 0
[  245.859076] hazptr-stack-torture: Reader Batch:  0 0 0 0 0 0 0 0 0 0 0
[  245.860302] hazptr-stack-torture: Free-Block Circulation:  359 358 357 356 355 354 353 352 351 350 0
[  245.862004] ??? Writer stall state HTWS_SYNC(3) g0 f0x0 ->state X cpu 13
[  246.042747] smpboot: Booting Node 0 Processor 15 APIC 0xf
[  252.717782] smpboot: Booting Node 0 Processor 10 APIC 0xa
[  256.065089] smpboot: CPU 6 is now offline
[  259.405180] smpboot: Booting Node 0 Processor 11 APIC 0xb
[  261.213631] hazptr-stack-torture: rtc: 0000000073942ea9 ver: 359 HTWS_SYNC tfle: 0 rta: 360 rtaf: 0 rtf: 350 onoff: 33/33:41/41 7,15:10,50 282:507 (HZ=1000) acq: 4971990752 rel: 4971954460 acqirq: 274559 relirq: 310673 reldefer: 0 relundefer 0
[  261.216128] hazptr-stack-torture: Reader Pipe:  4972262790 2343 0 0 0 0 0 0 0 0 0
[  261.217093] hazptr-stack-torture: Reader Batch:  0 0 0 0 0 0 0 0 0 0 0
[  261.217860] hazptr-stack-torture: Free-Block Circulation:  359 358 357 356 355 354 353 352 351 350 0
[  261.218909] ??? Writer stall state HTWS_SYNC(3) g0 f0x0 ->state X cpu 13
[  266.085175] smpboot: CPU 15 is now offline
[  269.424203] smpboot: Booting Node 0 Processor 15 APIC 0xf
[  272.775068] smpboot: CPU 8 is now offline
[  276.120771] smpboot: CPU 3 is now offline
[  276.223722] hazptr-stack-torture: rtc: 0000000073942ea9 ver: 359 HTWS_SYNC tfle: 0 rta: 360 rtaf: 0 rtf: 350 onoff: 34/34:44/44 7,15:10,50 290:542 (HZ=1000) acq: 5139645571 rel: 5139607698 acqirq: 283686 relirq: 321376 reldefer: 0 relundefer 0
[  276.226721] hazptr-stack-torture: Reader Pipe:  5139926527 2343 0 0 0 0 0 0 0 0 0
[  276.227780] hazptr-stack-torture: Reader Batch:  0 0 0 0 0 0 0 0 0 0 0
[  276.228708] hazptr-stack-torture: Free-Block Circulation:  359 358 357 356 355 354 353 352 351 350 0
[  276.229992] ??? Writer stall state HTWS_SYNC(3) g0 f0x0 ->state X cpu 13
[  279.461630] smpboot: Booting Node 0 Processor 6 APIC 0x6
[  282.809228] smpboot: CPU 11 is now offline
[  286.153098] smpboot: CPU 9 is now offline
[  289.491678] smpboot: Booting Node 0 Processor 7 APIC 0x7
[  291.421660] hazptr-stack-torture: rtc: 0000000073942ea9 ver: 359 HTWS_SYNC tfle: 0 rta: 360 rtaf: 0 rtf: 350 onoff: 36/36:46/46 7,15:10,50 311:563 (HZ=1000) acq: 5422489753 rel: 5422448772 acqirq: 298192 relirq: 338997 reldefer: 0 relundefer 0
[  291.424169] hazptr-stack-torture: Reader Pipe:  5422785426 2343 0 0 0 0 0 0 0 0 0
[  291.424922] hazptr-stack-torture: Reader Batch:  0 0 0 0 0 0 0 0 0 0 0
[  291.425589] hazptr-stack-torture: Free-Block Circulation:  359 358 357 356 355 354 353 352 351 350 0
[  291.426514] ??? Writer stall state HTWS_SYNC(3) g0 f0x0 ->state X cpu 13
[  292.840168] smpboot: CPU 15 is now offline
[  296.179186] smpboot: Booting Node 0 Processor 9 APIC 0x9
[  299.521672] smpboot: Booting Node 0 Processor 11 APIC 0xb
[  301.097957] hazptr-stack-torture: torture_shutdown task shutting down system
[  301.098815] hazptr-stack-torture: Stopping torture_shuffle task
[  301.099526] hazptr-stack-torture: torture_shuffle is stopping
[  301.100361] hazptr-stack-torture: Stopping torture_stutter task
[  301.101078] hazptr-stack-torture: torture_stutter is stopping
[  301.101861] hazptr-stack-torture: Stopping torture_onoff task
[  301.102557] hazptr-stack-torture: torture_onoff is stopping
[  301.108685] smpboot: Booting Node 0 Processor 1 APIC 0x1
[  301.112089] smpboot: Booting Node 0 Processor 2 APIC 0x2
[  301.115250] smpboot: Booting Node 0 Processor 3 APIC 0x3
[  301.118468] smpboot: Booting Node 0 Processor 4 APIC 0x4
[  301.121784] smpboot: Booting Node 0 Processor 5 APIC 0x5
[  301.125039] smpboot: Booting Node 0 Processor 8 APIC 0x8
[  301.128491] smpboot: Booting Node 0 Processor 13 APIC 0xd
[  301.132060] smpboot: Booting Node 0 Processor 14 APIC 0xe
[  301.135836] smpboot: Booting Node 0 Processor 15 APIC 0xf
[  301.139564] hazptr-stack-torture: Stopping hazptr_torture_writer task
[  301.140338] ------------[ cut here ]------------
[  301.140885] refcount_t: addition on 0; use-after-free.
[  301.141484] WARNING: lib/refcount.c:25 at refcount_warn_saturate+0x48/0x90, CPU#9: torture_shutdow/137
[  301.142559] Modules linked in:
[  301.142934] CPU: 9 UID: 0 PID: 137 Comm: torture_shutdow Tainted: G      D             7.1.0-rc4-00071-g136a43a847a7-dirty #1494 PREEMPTLAZY 
[  301.144376] Tainted: [D]=DIE
[  301.144727] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.15.0-1 04/01/2014
[  301.145651] RIP: 0010:refcount_warn_saturate+0x48/0x90
[  301.146257] Code: ce 02 4d 01 67 48 0f b9 3a c3 cc cc cc cc 85 f6 74 44 48 8d 3d c9 02 4d 01 67 48 0f b9 3a e9 4f aa 9d 00 48 8d 3d c8 02 4d 01 <67> 48 0f b9 3a e9 3e aa 9d 00 48 8d 3d c7 02 4d 01 67 48 0f b9 3a
[  301.148379] RSP: 0018:ffffb25500507e90 EFLAGS: 00010246
[  301.148986] RAX: 0000000000000000 RBX: ffff9cdc42221d80 RCX: 3fffffffffffdfff
[  301.149811] RDX: 0000000000000000 RSI: 0000000000000002 RDI: ffffffffa19a9a20
[  301.150640] RBP: 431bde82d7b634db R08: ffffffffa1933da8 R09: 3fffffffffffdfff
[  301.151452] R10: ffffffffa1853dc0 R11: 00000000000007ed R12: ffff9cdc42221da8
[  301.152273] R13: ffffb25500013d70 R14: 0000000000000000 R15: ffff9cdc420cc9c0
[  301.153090] FS:  0000000000000000(0000) GS:ffff9cdcbd321000(0000) knlGS:0000000000000000
[  301.154008] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  301.154669] CR2: 0000000000000000 CR3: 0000000018030000 CR4: 00000000000006f0
[  301.155482] Call Trace:
[  301.155789]  <TASK>
[  301.156052]  kthread_stop+0xeb/0xf0
[  301.156467]  _torture_stop_kthread+0x24/0x90
[  301.156972]  hazptr_torture_cleanup+0x59/0x130
[  301.157496]  torture_shutdown+0xdf/0x2a0
[  301.157957]  ? __pfx_torture_shutdown+0x10/0x10
[  301.158487]  kthread+0xd7/0x110
[  301.158878]  ? __pfx_kthread+0x10/0x10
[  301.159323]  ret_from_fork+0x1bd/0x220
[  301.159765]  ? __pfx_kthread+0x10/0x10
[  301.160208]  ret_from_fork_asm+0x1a/0x30
[  301.160670]  </TASK>
[  301.160934] ---[ end trace 0000000000000000 ]---
[  301.161485] BUG: kernel NULL pointer dereference, address: 0000000000000000
[  301.162275] #PF: supervisor write access in kernel mode
[  301.162868] #PF: error_code(0x0002) - not-present page
[  301.163457] PGD 0 P4D 0 
[  301.163761] Oops: Oops: 0002 [#2] SMP PTI
[  301.164229] CPU: 9 UID: 0 PID: 137 Comm: torture_shutdow Tainted: G      D W           7.1.0-rc4-00071-g136a43a847a7-dirty #1494 PREEMPTLAZY 
[  301.165589] Tainted: [D]=DIE, [W]=WARN
[  301.165966] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.15.0-1 04/01/2014
[  301.166754] RIP: 0010:kthread_stop+0x3b/0xf0
[  301.167192] Code: 55 53 48 89 fb f0 0f c1 47 28 85 c0 0f 84 bb 00 00 00 8d 50 01 09 c2 78 79 f6 43 2e 20 0f 84 86 00 00 00 48 8b ab 60 06 00 00 <f0> 80 4d 00 02 48 89 df e8 28 ff ff ff f0 80 0b 04 48 89 df e8 3c
[  301.168991] RSP: 0018:ffffb25500507e98 EFLAGS: 00010202
[  301.169512] RAX: 0000000000000000 RBX: ffff9cdc42221d80 RCX: 3fffffffffffdfff
[  301.170235] RDX: 0000000000000000 RSI: 0000000000000002 RDI: ffffffffa19a9a20
[  301.170927] RBP: 0000000000000000 R08: ffffffffa1933da8 R09: 3fffffffffffdfff
[  301.171629] R10: ffffffffa1853dc0 R11: 00000000000007ed R12: ffff9cdc42221da8
[  301.172331] R13: ffffb25500013d70 R14: 0000000000000000 R15: ffff9cdc420cc9c0
[  301.173037] FS:  0000000000000000(0000) GS:ffff9cdcbd321000(0000) knlGS:0000000000000000
[  301.173819] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  301.174389] CR2: 0000000000000000 CR3: 0000000018030000 CR4: 00000000000006f0
[  301.175090] Call Trace:
[  301.175343]  <TASK>
[  301.175564]  _torture_stop_kthread+0x24/0x90
[  301.175998]  hazptr_torture_cleanup+0x59/0x130
[  301.176445]  torture_shutdown+0xdf/0x2a0
[  301.176840]  ? __pfx_torture_shutdown+0x10/0x10
[  301.177299]  kthread+0xd7/0x110
[  301.177621]  ? __pfx_kthread+0x10/0x10
[  301.178003]  ret_from_fork+0x1bd/0x220
[  301.178382]  ? __pfx_kthread+0x10/0x10
[  301.178759]  ret_from_fork_asm+0x1a/0x30
[  301.179157]  </TASK>
[  301.179405] Modules linked in:
[  301.179718] CR2: 0000000000000000
[  301.180060] ---[ end trace 0000000000000000 ]---
[  301.180517] RIP: 0010:hazptr_synchronize_overflow_list+0x3e/0x90
[  301.181114] Code: 08 48 89 c6 48 85 db 74 54 4d 8b 65 10 eb 1b e8 78 2d e0 00 f3 90 4c 89 ef e8 7e 2f e0 00 48 89 c6 49 8b 45 10 4c 39 e0 75 41 <48> 8b 43 10 4c 89 ef 48 39 e8 74 d9 48 83 f8 01 74 d3 e8 4b 2d e0
[  301.182909] RSP: 0000:ffffb255004dfe60 EFLAGS: 00010086
[  301.183430] RAX: 00000000000013b2 RBX: dead000000000100 RCX: 0000000000000003
[  301.184136] RDX: 0000000000000001 RSI: 0000000000000286 RDI: ffff9cdc5f3d8060
[  301.184831] RBP: ffffffffa21a7f00 R08: 000000000006d924 R09: 000000000000000d
[  301.185533] R10: 0000000000000001 R11: 0000000000000000 R12: 00000000000013b2
[  301.186249] R13: ffff9cdc5f3d8060 R14: ffffffffa21a7f00 R15: ffff9cdc5f3d8040
[  301.186943] FS:  0000000000000000(0000) GS:ffff9cdcbd321000(0000) knlGS:0000000000000000
[  301.187734] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  301.188304] CR2: 0000000000000000 CR3: 0000000018030000 CR4: 00000000000006f0
[  301.189003] note: torture_shutdow[137] exited with irqs disabled
[  301.189621] torture_shutdow (137) used greatest stack depth: 13680 bytes left
[  304.469828] kworker/u65:2 (153) used greatest stack depth: 12344 bytes left
[  306.084634] hazptr-stack-torture: hazptr_torture_reader is stopping
[  306.084635] hazptr-stack-torture: hazptr_torture_reader is stopping
[  306.084648] hazptr-stack-torture: hazptr_torture_reader is stopping
[  306.084650] hazptr-stack-torture: hazptr_torture_reader is stopping
[  306.084652] hazptr-stack-torture: hazptr_torture_reader is stopping
[  306.084652] hazptr-stack-torture: hazptr_torture_reader is stopping
[  306.084653] hazptr-stack-torture: hazptr_torture_reader is stopping
[  306.084656] hazptr-stack-torture: hazptr_torture_reader is stopping
[  306.084666] hazptr-stack-torture: hazptr_torture_reader is stopping
[  306.084667] hazptr-stack-torture: hazptr_torture_reader is stopping
[  306.084667] hazptr-stack-torture: hazptr_torture_reader is stopping
[  306.084667] hazptr-stack-torture: hazptr_torture_reader is stopping
[  306.084667] hazptr-stack-torture: hazptr_torture_reader is stopping
[  306.084671] hazptr-stack-torture: hazptr_torture_reader is stopping
[  306.084686] hazptr-stack-torture: hazptr_torture_reader is stopping
[  306.084688] hazptr-stack-torture: hazptr_torture_reader is stopping
[  306.781635] hazptr-stack-torture: rtc: 0000000073942ea9 ver: 359 HTWS_SYNC tfle: 0 rta: 360 rtaf: 0 rtf: 350 onoff: 38/38:47/47 7,15:10,50 329:573 (HZ=1000) acq: 5572447965 rel: 5572405682 acqirq: 306205 relirq: 348312 reldefer: 0 relundefer 0
[  306.784165] hazptr-stack-torture: Reader Pipe:  5572751651 2343 0 0 0 0 0 0 0 0 0
[  306.785055] hazptr-stack-torture: Reader Batch:  0 0 0 0 0 0 0 0 0 0 0
[  306.785817] hazptr-stack-torture: Free-Block Circulation:  359 358 357 356 355 354 353 352 351 350 0
[  306.786875] ??? Writer stall state HTWS_SYNC(3) g0 f0x0 ->state X cpu 13
[  306.787662] hazptr-stack-torture: hazptr_torture_stats is stopping

Re: [RFC PATCH v5 1/2] hazptr: Implement Hazard Pointers

[Mathieu Desnoyers]

On 2026-06-27 13:36, Paul E. McKenney wrote:
> On Sat, Jun 27, 2026 at 11:21:28AM -0400, Mathieu Desnoyers wrote:
>> On 2026-06-27 08:56, Mathieu Desnoyers wrote:
>>> On 2026-06-27 00:14, Paul E. McKenney wrote:
>>>> On Fri, Jun 26, 2026 at 11:56:13PM -0400, Mathieu Desnoyers wrote:
>>>>> On 2026-06-26 19:06, Paul E. McKenney wrote:
>>> [...]
>>>>>> Initial runs suggest that hazard pointers cannot be acquired by one
>>>>>> task and released by another.  Is that expected behavior, or is my test
>>>>>> improperly passing the hazard pointers?  If the latter, we should of
>>>>>> course document the proper hazard-pointer-passing mechanism.
>>>>>
>>>>> This is unexpected. I've looked at your test code quickly and could
>>>>> not figure out what's wrong though. But it's late. How do you observe
>>>>> that it fails ?
>>>>
>>>> Not an emergency, so your schedule is my schedule.  Especially given
>>>> that I am mostly taking this coming week off.  And that it took me such
>>>> a long time to get this torture test done.  ;-)
>>>>
>>>> I run either scenario (PREEMPT or NOPREEMPT) with:
>>>>
>>>> --bootarg "hazptrtorture.onoff_interval=3333
>>>> hazptrtorture.stat_interval=15"
>>>>
>>>> I get the following on the console, and things go downhill from there.
>>>> I get the same thing when I turn off CPU hotplug, for whatever that is
>>>> worth.
>>> That OOPS is very instructive. I think I found the root cause.
>>>
>>> I need to add an API to hazptr so users wishing to explicitly transfer
>>> ownership of the hazptr ctx to another thread can do so.
>>>
>>> Currently the scheduler hook is responsible for moving the hazptr
>>> ctx from per-cpu to the backup slots, which makes it entirely movable
>>> afterwards.
>>>
>>> But this assumes the ctx is thread-local. Once you explicitly move
>>> ownership of that ctx to another thread, you race with the scheduler of
>>> the original thread.
>>>
>>> Let me see what I can do to fix that.
>>>
>>> Thanks a lot for the test-case !
> 
> Glad it helped?  ;-)
> 
>> Can you try with the attached POC diff ?
> 
> This increases the kernel's life significantly, but still fails as
> shown below.  This is the NOPREEMPT scenario as follows:
> 
> tools/testing/selftests/rcutorture/bin/kvm.sh --torture hazptr --allcpus --duration 5m --bootargs "hazptrtorture.onoff_interval=3333 hazptrtorture.nwriters=1 hazptrtorture.stat_interval=15" --trust-make
> 
> The PREEMPT failure gets similar failures, and the above command runs
> both of these scenarios.
> 

Re-reviewing kernel/rcu/hazptrtorture.c, I found the following issues:

In my patch provided earlier in this thread, I was missing a "detach
from task" call within hazptr_torture_reader_tail, because it calls
an IPI to do the hazptr release on a remote CPU. e.g.:

diff --git a/kernel/rcu/hazptrtorture.c b/kernel/rcu/hazptrtorture.c
index 311b387d6ae7..ae9ccd89ff87 100644
--- a/kernel/rcu/hazptrtorture.c
+++ b/kernel/rcu/hazptrtorture.c
@@ -411,6 +411,7 @@ hazptr_torture_reader_tail(struct hazptr_pending *hppp, struct torture_random_st
         preempt_enable();
         if (irq_release && !(torture_random(trsp) % irq_release)) {
                 guard(preempt)();
+               hazptr_detach_from_task(&hppp->hpp_hc);
                 cpu = cpumask_next_wrap(smp_processor_id(), cpu_online_mask);
                 smp_call_function_single(cpu, hazptr_torture_release, hppp, 1);
         } else {
@@ -428,6 +429,7 @@ static void hazptr_torture_defer(struct hazptr_pending *hppp, struct torture_ran
         struct llist_head *llhp;
  
         guard(preempt)();
+       hazptr_detach_from_task(&hppp->hpp_hc);
         cpu = cpumask_next_wrap(cpu, cpu_online_mask);
         llhp = per_cpu_ptr(&hazptr_pending, cpu);
         llist_add(&hppp->hpp_node, llhp);

Moreover, I think the whole ownership model of hazptr_torture_do_pending
is broken wrt hazptr requirements. We would need to "detach from task" each
of the hazptr before handing them off to such a remote drain, otherwise
the remote drain iteration on all cpus would basically have to hold the
runqueue locks for each of those cpus to prevent the scheduler from
racing against either hazptr_detach_from_task or hazptr release.

Thanks,

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com

Re: [RFC PATCH v5 1/2] hazptr: Implement Hazard Pointers

[Paul E. McKenney]

On Tue, Jul 07, 2026 at 03:40:41PM -0400, Mathieu Desnoyers wrote:
> On 2026-06-27 13:36, Paul E. McKenney wrote:
> > On Sat, Jun 27, 2026 at 11:21:28AM -0400, Mathieu Desnoyers wrote:
> > > On 2026-06-27 08:56, Mathieu Desnoyers wrote:
> > > > On 2026-06-27 00:14, Paul E. McKenney wrote:
> > > > > On Fri, Jun 26, 2026 at 11:56:13PM -0400, Mathieu Desnoyers wrote:
> > > > > > On 2026-06-26 19:06, Paul E. McKenney wrote:
> > > > [...]
> > > > > > > Initial runs suggest that hazard pointers cannot be acquired by one
> > > > > > > task and released by another.  Is that expected behavior, or is my test
> > > > > > > improperly passing the hazard pointers?  If the latter, we should of
> > > > > > > course document the proper hazard-pointer-passing mechanism.
> > > > > > 
> > > > > > This is unexpected. I've looked at your test code quickly and could
> > > > > > not figure out what's wrong though. But it's late. How do you observe
> > > > > > that it fails ?
> > > > > 
> > > > > Not an emergency, so your schedule is my schedule.  Especially given
> > > > > that I am mostly taking this coming week off.  And that it took me such
> > > > > a long time to get this torture test done.  ;-)
> > > > > 
> > > > > I run either scenario (PREEMPT or NOPREEMPT) with:
> > > > > 
> > > > > --bootarg "hazptrtorture.onoff_interval=3333
> > > > > hazptrtorture.stat_interval=15"
> > > > > 
> > > > > I get the following on the console, and things go downhill from there.
> > > > > I get the same thing when I turn off CPU hotplug, for whatever that is
> > > > > worth.
> > > > That OOPS is very instructive. I think I found the root cause.
> > > > 
> > > > I need to add an API to hazptr so users wishing to explicitly transfer
> > > > ownership of the hazptr ctx to another thread can do so.
> > > > 
> > > > Currently the scheduler hook is responsible for moving the hazptr
> > > > ctx from per-cpu to the backup slots, which makes it entirely movable
> > > > afterwards.
> > > > 
> > > > But this assumes the ctx is thread-local. Once you explicitly move
> > > > ownership of that ctx to another thread, you race with the scheduler of
> > > > the original thread.
> > > > 
> > > > Let me see what I can do to fix that.
> > > > 
> > > > Thanks a lot for the test-case !
> > 
> > Glad it helped?  ;-)
> > 
> > > Can you try with the attached POC diff ?
> > 
> > This increases the kernel's life significantly, but still fails as
> > shown below.  This is the NOPREEMPT scenario as follows:
> > 
> > tools/testing/selftests/rcutorture/bin/kvm.sh --torture hazptr --allcpus --duration 5m --bootargs "hazptrtorture.onoff_interval=3333 hazptrtorture.nwriters=1 hazptrtorture.stat_interval=15" --trust-make
> > 
> > The PREEMPT failure gets similar failures, and the above command runs
> > both of these scenarios.
> > 
> 
> Re-reviewing kernel/rcu/hazptrtorture.c, I found the following issues:
> 
> In my patch provided earlier in this thread, I was missing a "detach
> from task" call within hazptr_torture_reader_tail, because it calls
> an IPI to do the hazptr release on a remote CPU. e.g.:
> 
> diff --git a/kernel/rcu/hazptrtorture.c b/kernel/rcu/hazptrtorture.c
> index 311b387d6ae7..ae9ccd89ff87 100644
> --- a/kernel/rcu/hazptrtorture.c
> +++ b/kernel/rcu/hazptrtorture.c
> @@ -411,6 +411,7 @@ hazptr_torture_reader_tail(struct hazptr_pending *hppp, struct torture_random_st
>         preempt_enable();
>         if (irq_release && !(torture_random(trsp) % irq_release)) {
>                 guard(preempt)();
> +               hazptr_detach_from_task(&hppp->hpp_hc);
>                 cpu = cpumask_next_wrap(smp_processor_id(), cpu_online_mask);
>                 smp_call_function_single(cpu, hazptr_torture_release, hppp, 1);
>         } else {
> @@ -428,6 +429,7 @@ static void hazptr_torture_defer(struct hazptr_pending *hppp, struct torture_ran
>         struct llist_head *llhp;
>         guard(preempt)();
> +       hazptr_detach_from_task(&hppp->hpp_hc);
>         cpu = cpumask_next_wrap(cpu, cpu_online_mask);
>         llhp = per_cpu_ptr(&hazptr_pending, cpu);
>         llist_add(&hppp->hpp_node, llhp);
> 
> Moreover, I think the whole ownership model of hazptr_torture_do_pending
> is broken wrt hazptr requirements. We would need to "detach from task" each
> of the hazptr before handing them off to such a remote drain, otherwise
> the remote drain iteration on all cpus would basically have to hold the
> runqueue locks for each of those cpus to prevent the scheduler from
> racing against either hazptr_detach_from_task or hazptr release.

I applied your earlier patch and this one on top of my -rcu "dev" branch.
Please check in case I messed something up.

This provided an additional lease on kernel life, but still resulted in
issues as shown below.  I agree that it would be good to avoid needing
to explicitly mark hazard pointers that are to be released elsewhere,
if that is what you are getting at.  At the same time, it would also be
severely sub-optimal to acquire multiple CPUs' runqueue locks!

							Thanx, Paul

------------------------------------------------------------------------

[   21.169870] Oops: general protection fault, probably for non-canonical address 0xdead000000000110: 0000 [#1] SMP PTI
[   21.171423] CPU: 6 UID: 0 PID: 130 Comm: hazptr_torture_ Not tainted 7.1.0-rc4-00076-g548241132e40-dirty #1500 PREEMPTLAZY 
[   21.172859] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.15.0-1 04/01/2014
[   21.173906] RIP: 0010:hazptr_synchronize_overflow_list+0x3e/0x90
[   21.174670] Code: 08 48 89 c6 48 85 db 74 54 4d 8b 65 10 eb 1b e8 38 2e e0 00 f3 90 4c 89 ef e8 3e 30 e0 00 48 89 c6 49 8b 45 10 4c 39 e0 75 41 <48> 8b 43 10 4c 89 ef 48 39 e8 74 d9 48 83 f8 01 74 d3 e8 0b 2e e0
[   21.176968] RSP: 0018:ffffa33ac04cfe60 EFLAGS: 00010086
[   21.177621] RAX: 0000000000001ed7 RBX: dead000000000100 RCX: 0000000000000000
[   21.178538] RDX: 0000000000000001 RSI: 0000000000000286 RDI: ffffa230df398078
[   21.179462] RBP: ffffffff977a83a0 R08: 0000000000105ea8 R09: 0000000000000006
[   21.180374] R10: 0000000000000001 R11: ffffa230c1bc2c00 R12: 0000000000001ed7
[   21.181289] R13: ffffa230df398078 R14: ffffffff977a83a0 R15: ffffa230df398040
[   21.182208] FS:  0000000000000000(0000) GS:ffffa23147c61000(0000) knlGS:0000000000000000
[   21.183195] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   21.183881] CR2: 0000000000000000 CR3: 0000000006230000 CR4: 00000000000006f0
[   21.184763] Call Trace:
[   21.185113]  <TASK>
[   21.185395]  hazptr_synchronize+0xd1/0x130
[   21.185917]  hazptr_torture_writer+0x180/0x3c0
[   21.186491]  ? __pfx_hazptr_torture_writer+0x10/0x10
[   21.187121]  kthread+0xd7/0x110
[   21.187529]  ? __pfx_kthread+0x10/0x10
[   21.188014]  ret_from_fork+0x1bd/0x220
[   21.188499]  ? __pfx_kthread+0x10/0x10
[   21.189137]  ret_from_fork_asm+0x1a/0x30
[   21.189627]  </TASK>
[   21.189920] Modules linked in:
[   21.190340] ---[ end trace 0000000000000000 ]---

Re: [RFC PATCH v5 1/2] hazptr: Implement Hazard Pointers

[Paul E. McKenney]

On Tue, Jul 07, 2026 at 02:42:39PM -0700, Paul E. McKenney wrote:
> On Tue, Jul 07, 2026 at 03:40:41PM -0400, Mathieu Desnoyers wrote:
> > On 2026-06-27 13:36, Paul E. McKenney wrote:
> > > On Sat, Jun 27, 2026 at 11:21:28AM -0400, Mathieu Desnoyers wrote:
> > > > On 2026-06-27 08:56, Mathieu Desnoyers wrote:
> > > > > On 2026-06-27 00:14, Paul E. McKenney wrote:
> > > > > > On Fri, Jun 26, 2026 at 11:56:13PM -0400, Mathieu Desnoyers wrote:
> > > > > > > On 2026-06-26 19:06, Paul E. McKenney wrote:
> > > > > [...]
> > > > > > > > Initial runs suggest that hazard pointers cannot be acquired by one
> > > > > > > > task and released by another.  Is that expected behavior, or is my test
> > > > > > > > improperly passing the hazard pointers?  If the latter, we should of
> > > > > > > > course document the proper hazard-pointer-passing mechanism.
> > > > > > > 
> > > > > > > This is unexpected. I've looked at your test code quickly and could
> > > > > > > not figure out what's wrong though. But it's late. How do you observe
> > > > > > > that it fails ?
> > > > > > 
> > > > > > Not an emergency, so your schedule is my schedule.  Especially given
> > > > > > that I am mostly taking this coming week off.  And that it took me such
> > > > > > a long time to get this torture test done.  ;-)
> > > > > > 
> > > > > > I run either scenario (PREEMPT or NOPREEMPT) with:
> > > > > > 
> > > > > > --bootarg "hazptrtorture.onoff_interval=3333
> > > > > > hazptrtorture.stat_interval=15"
> > > > > > 
> > > > > > I get the following on the console, and things go downhill from there.
> > > > > > I get the same thing when I turn off CPU hotplug, for whatever that is
> > > > > > worth.
> > > > > That OOPS is very instructive. I think I found the root cause.
> > > > > 
> > > > > I need to add an API to hazptr so users wishing to explicitly transfer
> > > > > ownership of the hazptr ctx to another thread can do so.
> > > > > 
> > > > > Currently the scheduler hook is responsible for moving the hazptr
> > > > > ctx from per-cpu to the backup slots, which makes it entirely movable
> > > > > afterwards.
> > > > > 
> > > > > But this assumes the ctx is thread-local. Once you explicitly move
> > > > > ownership of that ctx to another thread, you race with the scheduler of
> > > > > the original thread.
> > > > > 
> > > > > Let me see what I can do to fix that.
> > > > > 
> > > > > Thanks a lot for the test-case !
> > > 
> > > Glad it helped?  ;-)
> > > 
> > > > Can you try with the attached POC diff ?
> > > 
> > > This increases the kernel's life significantly, but still fails as
> > > shown below.  This is the NOPREEMPT scenario as follows:
> > > 
> > > tools/testing/selftests/rcutorture/bin/kvm.sh --torture hazptr --allcpus --duration 5m --bootargs "hazptrtorture.onoff_interval=3333 hazptrtorture.nwriters=1 hazptrtorture.stat_interval=15" --trust-make
> > > 
> > > The PREEMPT failure gets similar failures, and the above command runs
> > > both of these scenarios.
> > > 
> > 
> > Re-reviewing kernel/rcu/hazptrtorture.c, I found the following issues:
> > 
> > In my patch provided earlier in this thread, I was missing a "detach
> > from task" call within hazptr_torture_reader_tail, because it calls
> > an IPI to do the hazptr release on a remote CPU. e.g.:
> > 
> > diff --git a/kernel/rcu/hazptrtorture.c b/kernel/rcu/hazptrtorture.c
> > index 311b387d6ae7..ae9ccd89ff87 100644
> > --- a/kernel/rcu/hazptrtorture.c
> > +++ b/kernel/rcu/hazptrtorture.c
> > @@ -411,6 +411,7 @@ hazptr_torture_reader_tail(struct hazptr_pending *hppp, struct torture_random_st
> >         preempt_enable();
> >         if (irq_release && !(torture_random(trsp) % irq_release)) {
> >                 guard(preempt)();
> > +               hazptr_detach_from_task(&hppp->hpp_hc);
> >                 cpu = cpumask_next_wrap(smp_processor_id(), cpu_online_mask);
> >                 smp_call_function_single(cpu, hazptr_torture_release, hppp, 1);
> >         } else {
> > @@ -428,6 +429,7 @@ static void hazptr_torture_defer(struct hazptr_pending *hppp, struct torture_ran
> >         struct llist_head *llhp;
> >         guard(preempt)();
> > +       hazptr_detach_from_task(&hppp->hpp_hc);
> >         cpu = cpumask_next_wrap(cpu, cpu_online_mask);
> >         llhp = per_cpu_ptr(&hazptr_pending, cpu);
> >         llist_add(&hppp->hpp_node, llhp);
> > 
> > Moreover, I think the whole ownership model of hazptr_torture_do_pending
> > is broken wrt hazptr requirements. We would need to "detach from task" each
> > of the hazptr before handing them off to such a remote drain, otherwise
> > the remote drain iteration on all cpus would basically have to hold the
> > runqueue locks for each of those cpus to prevent the scheduler from
> > racing against either hazptr_detach_from_task or hazptr release.
> 
> I applied your earlier patch and this one on top of my -rcu "dev" branch.
> Please check in case I messed something up.
> 
> This provided an additional lease on kernel life, but still resulted in
> issues as shown below.  I agree that it would be good to avoid needing
> to explicitly mark hazard pointers that are to be released elsewhere,
> if that is what you are getting at.  At the same time, it would also be
> severely sub-optimal to acquire multiple CPUs' runqueue locks!

OK, I did check, and all of the hazptr_torture_do_pending() function's
hazard pointers have been passed to hazptr_detach_from_task().

But sometimes a given hazard pointer might be passed to
hazptr_detach_from_task() twice, once just before that llist_add(),
and again if hazptr_torture_reader_tail() decides to release that hazard
pointer via smp_call_function_single().

Do I instead need to be careful to avoid detaching a given hazard pointer
more than once?

							Thanx, Paul

> ------------------------------------------------------------------------
> 
> [   21.169870] Oops: general protection fault, probably for non-canonical address 0xdead000000000110: 0000 [#1] SMP PTI
> [   21.171423] CPU: 6 UID: 0 PID: 130 Comm: hazptr_torture_ Not tainted 7.1.0-rc4-00076-g548241132e40-dirty #1500 PREEMPTLAZY 
> [   21.172859] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.15.0-1 04/01/2014
> [   21.173906] RIP: 0010:hazptr_synchronize_overflow_list+0x3e/0x90
> [   21.174670] Code: 08 48 89 c6 48 85 db 74 54 4d 8b 65 10 eb 1b e8 38 2e e0 00 f3 90 4c 89 ef e8 3e 30 e0 00 48 89 c6 49 8b 45 10 4c 39 e0 75 41 <48> 8b 43 10 4c 89 ef 48 39 e8 74 d9 48 83 f8 01 74 d3 e8 0b 2e e0
> [   21.176968] RSP: 0018:ffffa33ac04cfe60 EFLAGS: 00010086
> [   21.177621] RAX: 0000000000001ed7 RBX: dead000000000100 RCX: 0000000000000000
> [   21.178538] RDX: 0000000000000001 RSI: 0000000000000286 RDI: ffffa230df398078
> [   21.179462] RBP: ffffffff977a83a0 R08: 0000000000105ea8 R09: 0000000000000006
> [   21.180374] R10: 0000000000000001 R11: ffffa230c1bc2c00 R12: 0000000000001ed7
> [   21.181289] R13: ffffa230df398078 R14: ffffffff977a83a0 R15: ffffa230df398040
> [   21.182208] FS:  0000000000000000(0000) GS:ffffa23147c61000(0000) knlGS:0000000000000000
> [   21.183195] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [   21.183881] CR2: 0000000000000000 CR3: 0000000006230000 CR4: 00000000000006f0
> [   21.184763] Call Trace:
> [   21.185113]  <TASK>
> [   21.185395]  hazptr_synchronize+0xd1/0x130
> [   21.185917]  hazptr_torture_writer+0x180/0x3c0
> [   21.186491]  ? __pfx_hazptr_torture_writer+0x10/0x10
> [   21.187121]  kthread+0xd7/0x110
> [   21.187529]  ? __pfx_kthread+0x10/0x10
> [   21.188014]  ret_from_fork+0x1bd/0x220
> [   21.188499]  ? __pfx_kthread+0x10/0x10
> [   21.189137]  ret_from_fork_asm+0x1a/0x30
> [   21.189627]  </TASK>
> [   21.189920] Modules linked in:
> [   21.190340] ---[ end trace 0000000000000000 ]---

Re: [RFC PATCH v5 1/2] hazptr: Implement Hazard Pointers

[Mathieu Desnoyers]

On 2026-07-08 18:55, Paul E. McKenney wrote:
[...]
> OK, I did check, and all of the hazptr_torture_do_pending() function's
> hazard pointers have been passed to hazptr_detach_from_task().
> 
> But sometimes a given hazard pointer might be passed to
> hazptr_detach_from_task() twice, once just before that llist_add(),
> and again if hazptr_torture_reader_tail() decides to release that hazard
> pointer via smp_call_function_single().
> 
> Do I instead need to be careful to avoid detaching a given hazard pointer
> more than once?
That should be OK to call the "detach" more than once, because it checks
if the hazptr is already detached with:

+       if (unlikely(hazptr_slot_is_backup(ctx, slot)))
+               return;

(while under preempt off)

My concern is about this worker thread:

static int hazptr_torture_do_pending(void *arg)
{
         int cpu = 0;
         DEFINE_TORTURE_RANDOM(rand);

         VERBOSE_TOROUT_STRING("hazptr_torture_do_pending task started");
         do {
                 if (stutter_will_wait()) {
                         for_each_possible_cpu(cpu)
                                 hazptr_torture_do_one_pending(cpu, &rand);
                 } else {
                         cpu = cpumask_next_wrap(cpu, cpu_possible_mask);
                         hazptr_torture_do_one_pending(cpu, &rand);
                 }
                 if (torture_must_stop())
                         torture_hrtimeout_ms(kthread_do_pending_ms, USEC_PER_MSEC, &rand);
                 // Omit stutter_wait() because this function needs to do cleanup.
         } while (!torture_must_stop());
         torture_kthread_stopping("hazptr_torture_do_pending");
         return 0;
}

which as you will notice happily invoke "do_one_pending" on either
all possible cpus or on a next cpu (non-local), which does:

         llhp = per_cpu_ptr(&hazptr_pending, cpu);
         llnp = llist_del_all(llhp);
         if (!llnp)
                 return;
         llist_for_each_entry_safe(hppp, hppp1, llnp, hpp_node) {
                 hazptr_torture_reader_tail(hppp, trsp);
                 atomic_long_inc(per_cpu_ptr(&hazptr_torture_releases_undefer,
                                             raw_smp_processor_id()));
                 kfree(hppp);
         }

And here the reader_tail calls cur_ops->readunlock(hcp, htp); which
does a hazptr_release from the wrong CPU without previously detaching
the hazptr from its associated thread as it should.

Does this line of thinking make sense ?

Thanks,

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com

Re: [RFC PATCH v5 1/2] hazptr: Implement Hazard Pointers

[Paul E. McKenney]

On Wed, Jul 08, 2026 at 10:45:23PM -0400, Mathieu Desnoyers wrote:
> On 2026-07-08 18:55, Paul E. McKenney wrote:
> [...]
> > OK, I did check, and all of the hazptr_torture_do_pending() function's
> > hazard pointers have been passed to hazptr_detach_from_task().
> > 
> > But sometimes a given hazard pointer might be passed to
> > hazptr_detach_from_task() twice, once just before that llist_add(),
> > and again if hazptr_torture_reader_tail() decides to release that hazard
> > pointer via smp_call_function_single().
> > 
> > Do I instead need to be careful to avoid detaching a given hazard pointer
> > more than once?
> That should be OK to call the "detach" more than once, because it checks
> if the hazptr is already detached with:
> 
> +       if (unlikely(hazptr_slot_is_backup(ctx, slot)))
> +               return;
> 
> (while under preempt off)
> 
> My concern is about this worker thread:
> 
> static int hazptr_torture_do_pending(void *arg)
> {
>         int cpu = 0;
>         DEFINE_TORTURE_RANDOM(rand);
> 
>         VERBOSE_TOROUT_STRING("hazptr_torture_do_pending task started");
>         do {
>                 if (stutter_will_wait()) {
>                         for_each_possible_cpu(cpu)
>                                 hazptr_torture_do_one_pending(cpu, &rand);
>                 } else {
>                         cpu = cpumask_next_wrap(cpu, cpu_possible_mask);
>                         hazptr_torture_do_one_pending(cpu, &rand);
>                 }
>                 if (torture_must_stop())
>                         torture_hrtimeout_ms(kthread_do_pending_ms, USEC_PER_MSEC, &rand);
>                 // Omit stutter_wait() because this function needs to do cleanup.
>         } while (!torture_must_stop());
>         torture_kthread_stopping("hazptr_torture_do_pending");
>         return 0;
> }
> 
> which as you will notice happily invoke "do_one_pending" on either
> all possible cpus or on a next cpu (non-local), which does:
> 
>         llhp = per_cpu_ptr(&hazptr_pending, cpu);
>         llnp = llist_del_all(llhp);

Except that every item on every CPU's llist has already been detached,
courtesy of the call to hazptr_detach_from_task() from within
hazptr_torture_defer() prior to the llist_add().

>         if (!llnp)
>                 return;
>         llist_for_each_entry_safe(hppp, hppp1, llnp, hpp_node) {
>                 hazptr_torture_reader_tail(hppp, trsp);
>                 atomic_long_inc(per_cpu_ptr(&hazptr_torture_releases_undefer,
>                                             raw_smp_processor_id()));
>                 kfree(hppp);
>         }
> 
> And here the reader_tail calls cur_ops->readunlock(hcp, htp); which
> does a hazptr_release from the wrong CPU without previously detaching
> the hazptr from its associated thread as it should.
> 
> Does this line of thinking make sense ?

Huh.

There are actually two calls to hazptr_detach_from_task() within
hazptr_torture_defer(), one outside of the guard(preempt) and the other
within, as in I somehow misapplied your patch.

But that would certainly not explain failures in the NOPREEMPT case.

Still, time to ditch the call preceding the guard(preempt) and see
what happens.  Like this:

static void hazptr_torture_defer(struct hazptr_pending *hppp, struct torture_random_state *trsp)
{
	int cpu = torture_random(trsp) % nr_cpu_ids;
	struct llist_head *llhp;

	guard(preempt)();
	hazptr_detach_from_task(&hppp->hpp_hc);
	cpu = cpumask_next_wrap(cpu, cpu_online_mask);
	llhp = per_cpu_ptr(&hazptr_pending, cpu);
	llist_add(&hppp->hpp_node, llhp);
	atomic_long_inc(per_cpu_ptr(&hazptr_torture_releases_defer, raw_smp_processor_id()));
}

Except that the same thing happens:

[    8.042111] Oops: general protection fault, probably for non-canonical address 0xdead000000000110: 0000 [#1] SMP PTI
[    8.043367] CPU: 8 UID: 0 PID: 135 Comm: hazptr_torture_ Not tainted 7.1.0-rc4-00080-g3d59e48086a0-dirty #1519 PREEMPT(full) 
[    8.044685] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.15.0-1 04/01/2014
[    8.045626] RIP: 0010:hazptr_synchronize_overflow_list+0x3e/0x90
[    8.046356] Code: 08 48 89 c6 48 85 db 74 54 4d 8b 65 10 eb 1b e8 48 12 fa 00 f3 90 4c 89 ef e8 4e 14 fa 00 48 89 c6 49 8b 45 10 4c 39 e0 75 41 <48> 8b 43 10 4c 89 ef 48 39 e8 74 d9 48 83 f8 01 74 d3 e8 1b 12 fa
[    8.048592] RSP: 0000:ffffb289004f7e60 EFLAGS: 00010086
[    8.049231] RAX: 0000000000000801 RBX: dead000000000100 RCX: 0000000000000003
[    8.050136] RDX: 0000000000000001 RSI: 0000000000000286 RDI: ffff9006df3d80e0
[    8.050993] RBP: ffffffff9b002940 R08: 0000000000122a09 R09: 0000000000000008
[    8.051836] R10: 0000000000000001 R11: ffff9006c1d1b200 R12: 0000000000000801
[    8.052889] R13: ffff9006df3d80e0 R14: ffffffff9b002940 R15: ffff9006df3d80c0
[    8.053746] FS:  0000000000000000(0000) GS:ffff90074448f000(0000) knlGS:0000000000000000
[    8.054641] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    8.055286] CR2: 0000000000000000 CR3: 0000000017830000 CR4: 00000000000006f0
[    8.056086] Call Trace:
[    8.056378]  <TASK>
[    8.056648]  hazptr_synchronize+0xd1/0x130
[    8.057118]  hazptr_torture_writer+0x180/0x3f0
[    8.057642]  ? __pfx_hazptr_torture_writer+0x10/0x10
[    8.058241]  kthread+0xe5/0x120
[    8.058629]  ? __pfx_kthread+0x10/0x10
[    8.059090]  ret_from_fork+0x1bd/0x220
[    8.059529]  ? __pfx_kthread+0x10/0x10
[    8.059954]  ret_from_fork_asm+0x1a/0x30
[    8.060417]  </TASK>
[    8.060685] Modules linked in:
[    8.061041] ---[ end trace 0000000000000000 ]---

So what am I missing here?

							Thanx, Paul

Re: [RFC PATCH v5 1/2] hazptr: Implement Hazard Pointers

[Paul E. McKenney]

On Wed, Jul 08, 2026 at 08:28:29PM -0700, Paul E. McKenney wrote:
> On Wed, Jul 08, 2026 at 10:45:23PM -0400, Mathieu Desnoyers wrote:
> > On 2026-07-08 18:55, Paul E. McKenney wrote:
> > [...]
> > > OK, I did check, and all of the hazptr_torture_do_pending() function's
> > > hazard pointers have been passed to hazptr_detach_from_task().
> > > 
> > > But sometimes a given hazard pointer might be passed to
> > > hazptr_detach_from_task() twice, once just before that llist_add(),
> > > and again if hazptr_torture_reader_tail() decides to release that hazard
> > > pointer via smp_call_function_single().
> > > 
> > > Do I instead need to be careful to avoid detaching a given hazard pointer
> > > more than once?
> > That should be OK to call the "detach" more than once, because it checks
> > if the hazptr is already detached with:
> > 
> > +       if (unlikely(hazptr_slot_is_backup(ctx, slot)))
> > +               return;
> > 
> > (while under preempt off)
> > 
> > My concern is about this worker thread:
> > 
> > static int hazptr_torture_do_pending(void *arg)
> > {
> >         int cpu = 0;
> >         DEFINE_TORTURE_RANDOM(rand);
> > 
> >         VERBOSE_TOROUT_STRING("hazptr_torture_do_pending task started");
> >         do {
> >                 if (stutter_will_wait()) {
> >                         for_each_possible_cpu(cpu)
> >                                 hazptr_torture_do_one_pending(cpu, &rand);
> >                 } else {
> >                         cpu = cpumask_next_wrap(cpu, cpu_possible_mask);
> >                         hazptr_torture_do_one_pending(cpu, &rand);
> >                 }
> >                 if (torture_must_stop())
> >                         torture_hrtimeout_ms(kthread_do_pending_ms, USEC_PER_MSEC, &rand);
> >                 // Omit stutter_wait() because this function needs to do cleanup.
> >         } while (!torture_must_stop());
> >         torture_kthread_stopping("hazptr_torture_do_pending");
> >         return 0;
> > }
> > 
> > which as you will notice happily invoke "do_one_pending" on either
> > all possible cpus or on a next cpu (non-local), which does:
> > 
> >         llhp = per_cpu_ptr(&hazptr_pending, cpu);
> >         llnp = llist_del_all(llhp);
> 
> Except that every item on every CPU's llist has already been detached,
> courtesy of the call to hazptr_detach_from_task() from within
> hazptr_torture_defer() prior to the llist_add().
> 
> >         if (!llnp)
> >                 return;
> >         llist_for_each_entry_safe(hppp, hppp1, llnp, hpp_node) {
> >                 hazptr_torture_reader_tail(hppp, trsp);
> >                 atomic_long_inc(per_cpu_ptr(&hazptr_torture_releases_undefer,
> >                                             raw_smp_processor_id()));
> >                 kfree(hppp);
> >         }
> > 
> > And here the reader_tail calls cur_ops->readunlock(hcp, htp); which
> > does a hazptr_release from the wrong CPU without previously detaching
> > the hazptr from its associated thread as it should.
> > 
> > Does this line of thinking make sense ?
> 
> Huh.
> 
> There are actually two calls to hazptr_detach_from_task() within
> hazptr_torture_defer(), one outside of the guard(preempt) and the other
> within, as in I somehow misapplied your patch.
> 
> But that would certainly not explain failures in the NOPREEMPT case.
> 
> Still, time to ditch the call preceding the guard(preempt) and see
> what happens.  Like this:
> 
> static void hazptr_torture_defer(struct hazptr_pending *hppp, struct torture_random_state *trsp)
> {
> 	int cpu = torture_random(trsp) % nr_cpu_ids;
> 	struct llist_head *llhp;
> 
> 	guard(preempt)();
> 	hazptr_detach_from_task(&hppp->hpp_hc);
> 	cpu = cpumask_next_wrap(cpu, cpu_online_mask);
> 	llhp = per_cpu_ptr(&hazptr_pending, cpu);
> 	llist_add(&hppp->hpp_node, llhp);
> 	atomic_long_inc(per_cpu_ptr(&hazptr_torture_releases_defer, raw_smp_processor_id()));
> }
> 
> Except that the same thing happens:
> 
> [    8.042111] Oops: general protection fault, probably for non-canonical address 0xdead000000000110: 0000 [#1] SMP PTI
> [    8.043367] CPU: 8 UID: 0 PID: 135 Comm: hazptr_torture_ Not tainted 7.1.0-rc4-00080-g3d59e48086a0-dirty #1519 PREEMPT(full) 
> [    8.044685] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.15.0-1 04/01/2014
> [    8.045626] RIP: 0010:hazptr_synchronize_overflow_list+0x3e/0x90
> [    8.046356] Code: 08 48 89 c6 48 85 db 74 54 4d 8b 65 10 eb 1b e8 48 12 fa 00 f3 90 4c 89 ef e8 4e 14 fa 00 48 89 c6 49 8b 45 10 4c 39 e0 75 41 <48> 8b 43 10 4c 89 ef 48 39 e8 74 d9 48 83 f8 01 74 d3 e8 1b 12 fa
> [    8.048592] RSP: 0000:ffffb289004f7e60 EFLAGS: 00010086
> [    8.049231] RAX: 0000000000000801 RBX: dead000000000100 RCX: 0000000000000003
> [    8.050136] RDX: 0000000000000001 RSI: 0000000000000286 RDI: ffff9006df3d80e0
> [    8.050993] RBP: ffffffff9b002940 R08: 0000000000122a09 R09: 0000000000000008
> [    8.051836] R10: 0000000000000001 R11: ffff9006c1d1b200 R12: 0000000000000801
> [    8.052889] R13: ffff9006df3d80e0 R14: ffffffff9b002940 R15: ffff9006df3d80c0
> [    8.053746] FS:  0000000000000000(0000) GS:ffff90074448f000(0000) knlGS:0000000000000000
> [    8.054641] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [    8.055286] CR2: 0000000000000000 CR3: 0000000017830000 CR4: 00000000000006f0
> [    8.056086] Call Trace:
> [    8.056378]  <TASK>
> [    8.056648]  hazptr_synchronize+0xd1/0x130
> [    8.057118]  hazptr_torture_writer+0x180/0x3f0
> [    8.057642]  ? __pfx_hazptr_torture_writer+0x10/0x10
> [    8.058241]  kthread+0xe5/0x120
> [    8.058629]  ? __pfx_kthread+0x10/0x10
> [    8.059090]  ret_from_fork+0x1bd/0x220
> [    8.059529]  ? __pfx_kthread+0x10/0x10
> [    8.059954]  ret_from_fork_asm+0x1a/0x30
> [    8.060417]  </TASK>
> [    8.060685] Modules linked in:
> [    8.061041] ---[ end trace 0000000000000000 ]---
> 
> So what am I missing here?

Am I perhaps just overflowing the backup slots?  Or is there some sort
of memory allocation behind the scenes that I am missing?

							Thanx, Paul

Re: [RFC PATCH v5 1/2] hazptr: Implement Hazard Pointers

[Mathieu Desnoyers]

On 2026-07-09 12:02, Paul E. McKenney wrote:
[...]
> Am I perhaps just overflowing the backup slots?  Or is there some sort
> of memory allocation behind the scenes that I am missing?

I found the root cause, fix coming shortly.

Thanks,

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com

Re: [RFC PATCH v5 1/2] hazptr: Implement Hazard Pointers

[Paul E. McKenney]

On Thu, Jul 09, 2026 at 12:03:49PM -0400, Mathieu Desnoyers wrote:
> On 2026-07-09 12:02, Paul E. McKenney wrote:
> [...]
> > Am I perhaps just overflowing the backup slots?  Or is there some sort
> > of memory allocation behind the scenes that I am missing?
> 
> I found the root cause, fix coming shortly.

Even better, thank you, and looking forward to it!

							Thanx, Paul