mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Andrea Parri <parri.andrea@gmail.com>
To: "Paul E. McKenney" <paulmck@kernel.org>,
	Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Andrea Parri <parri.andrea@gmail.com>,
	Frederic Weisbecker <frederic@kernel.org>,
	Neeraj Upadhyay <neeraj.upadhyay@kernel.org>,
	Joel Fernandes <joelagnelf@nvidia.com>,
	Josh Triplett <josh@joshtriplett.org>,
	Boqun Feng <boqun@kernel.org>,
	Uladzislau Rezki <urezki@gmail.com>,
	Steven Rostedt <rostedt@goodmis.org>,
	Lai Jiangshan <jiangshanlai@gmail.com>,
	Zqiang <qiang.zhang@linux.dev>, Gary Guo <gary@garyguo.net>,
	Bradley Morgan <brads@mainlining.org>,
	Kunwu Chan <kunwu.chan@gmail.com>,
	rcu@vger.kernel.org, lkmm@lists.linux.dev,
	linux-kernel@vger.kernel.org
Subject: [PATCH -rcu/dev 0/4] hazptrtorture: Fix shutdown hang and catch slot owner race
Date: Tue, 27 Oct 2026 17:38:02 +0100	[thread overview]
Message-ID: <20261027163811.2470569-1-parri.andrea@gmail.com> (raw)

Patch 1 fixes a shutdown hang in hazptrtorture with deferral enabled, as
in the PREEMPT scenario: hazptr_torture_cleanup() stops the kthread that
releases deferred hazard pointers before the writer, which can then
wait in ->sync() forever.  It now stops the readers before that
kthread, which releases what is left once the readers have stopped, and
that kthread before the writer.

Patches 2 and 4 make hazptrtorture catch the race fixed by "hazptr:
Record per-CPU slot owner after claiming the slot" (now on -rcu/dev)
should it come back, as Paul asked [1] and agreed to [2].  Patch 2
adds two CONFIG_HAZPTR_DEBUG checks of the recorded slot owner, in
hazptr_release_debug() and hazptr_note_context_switch().  Patch 4 adds
the nest_acquire module parameter, which makes readers take a second,
nested hazard pointer that goes through __hazptr_acquire() while the
first is in slot 0.

Patch 3 fixes a false positive of an existing CONFIG_HAZPTR_DEBUG
check, which the nesting of patch 4 exposes with CPU hotplug: a hazard
pointer that __hazptr_acquire() puts directly in the backup slot warns
if the task migrates before releasing it.  It comes before patch 4,
which would otherwise make runs with CPU hotplug warn.

The series applies on -rcu/dev.  30-minute hazptrtorture runs of the
PREEMPT and NOPREEMPT scenarios pass on x86_64 with 8 CPUs, as do a
30-minute NOPREEMPT run with torture_type=hazptr and three 10-minute
PREEMPT runs: one with irq_acquire=100 and nest_acquire=1, one with
CONFIG_HAZPTR_DEBUG=n, and one with torture_type=hazptr-stack.  The
30-minute PREEMPT run ends with as many hazard pointers released from
the pending lists as were deferred (reldefer: 650385 relundefer
650385).  With kthread_do_pending_ms=10000, two 3-minute PREEMPT runs
shut down in 100 and 130ms, well within one period.

Without patch 4, hazptrtorture passes even without the slot-owner fix:
its readers hold one hazard pointer each, so they reach the slow path
only when the ptr_eq() check fails or the pointer is NULL.  With
patches 2 and 4 but without the fix, PREEMPT and NOPREEMPT report the
race 6.3 and 6.0 seconds after boot, and with nest_acquire=0 both run
10 minutes without a report.

The nesting costs about a third of the outer reader iterations at the
default nest_acquire=2.  If that is too much for every run, it could
default to zero and be enabled from the scenario files instead.

Patches 2 and 4 stand in for the KUnit reproducer mentioned in the RFC
notes of the fix, which needs a test-only hook in
__hazptr_try_acquire().  The checks can miss one case that the KUnit
test covers: an interrupt handler that acquires and releases the slot
instead of detaching it, then a context switch before the task's
release, when no other hazard pointer claims the slot in between.
hazptrtorture does not produce that case either, as
hazptr_torture_acquire() always detaches.

Patch 2's release check also warns about a second hazptr_release() of
a ctx whose per-CPU slot another hazard pointer has claimed since.  The
hazptr_release() kernel-doc permits that second release, but it clears
the other hazard pointer, and a second release of a backup slot oopses
even without this series.  Nothing in-tree releases a hazard pointer
twice.  I will follow up separately on the kernel-doc.

An LLM found the hang fixed by patch 1, the crash shown in patch 2 and
the false positive fixed by patch 3.  It also helped write all four
patches and their changelogs, and ran the tests above.

---
Changes since RFC:
- Patch 1: stop the readers, then do_pending, then the writer, and
  release all deferred hazard pointers after torture_kthread_stopping(),
  instead of keeping do_pending in its loop until it is stopped (Paul).
  Paul suggested changing the "while (!torture_must_stop())" condition
  to "while (!kthread_should_stop())"; this keeps it, for the reason in
  [3].  The changelog also says when kthread_stop() can still delay the
  exit of do_pending by a full kthread_do_pending_ms sleep (Paul).
  Bradley's Acked-by is not carried over, as the patch changed.
- Post patch 1 in a series (Bradley), now with patches 2 and 4, so
  that hazptrtorture catches the race fixed by the slot-owner RFC patch
  (Paul).
- Patch 3: new, for the false positive that patch 4 exposes.
- Do not repost the slot-owner RFC patch, which Paul has queued.
- Drop the Assisted-by tags, following mainline commit 1e035297ffda
  ("docs: remove Assisted-by references"), and describe the LLM use
  in the changelogs and above instead.

[1] https://lore.kernel.org/r/fa16d2b1-594e-494b-8cfc-9b8c13f2c223@paulmck-laptop/
[2] https://lore.kernel.org/r/e0145c14-9d6a-4c91-816c-57dabd98879b@paulmck-laptop/
[3] https://lore.kernel.org/r/20261023120755.2463384-1-parri.andrea@gmail.com/

RFC patch 1: https://lore.kernel.org/r/20261019162005.12888-1-parri.andrea@gmail.com/
RFC slot-owner fix: https://lore.kernel.org/r/20261019161852.12783-1-parri.andrea@gmail.com/

Andrea Parri (4):
  hazptrtorture: Release all deferred hazard pointers before stopping
  hazptr: Check per-CPU slot owner with CONFIG_HAZPTR_DEBUG
  hazptr: Do not warn when a backup slot is released on another CPU
  hazptrtorture: Nest a second hazard pointer in readers

 .../admin-guide/kernel-parameters.txt         |  6 +++
 include/linux/hazptr.h                        | 13 ++++++
 kernel/hazptr.c                               |  4 ++
 kernel/rcu/hazptrtorture.c                    | 40 ++++++++++++++++++-
 4 files changed, 61 insertions(+), 2 deletions(-)


base-commit: 633f91e3408cc65a5237752a3265b031c389e43e
-- 
2.53.0


             reply	other threads:[~2026-10-27 16:38 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-10-27 16:38 Andrea Parri [this message]
2026-10-27 16:38 ` [PATCH -rcu/dev 1/4] hazptrtorture: Release all deferred hazard pointers before stopping Andrea Parri
2026-10-27 16:38 ` [PATCH -rcu/dev 2/4] hazptr: Check per-CPU slot owner with CONFIG_HAZPTR_DEBUG Andrea Parri
2026-10-27 16:38 ` [PATCH -rcu/dev 3/4] hazptr: Do not warn when a backup slot is released on another CPU Andrea Parri
2026-10-27 16:38 ` [PATCH -rcu/dev 4/4] hazptrtorture: Nest a second hazard pointer in readers Andrea Parri

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261027163811.2470569-1-parri.andrea@gmail.com \
    --to=parri.andrea@gmail.com \
    --cc=boqun@kernel.org \
    --cc=brads@mainlining.org \
    --cc=frederic@kernel.org \
    --cc=gary@garyguo.net \
    --cc=jiangshanlai@gmail.com \
    --cc=joelagnelf@nvidia.com \
    --cc=josh@joshtriplett.org \
    --cc=kunwu.chan@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lkmm@lists.linux.dev \
    --cc=mathieu.desnoyers@efficios.com \
    --cc=neeraj.upadhyay@kernel.org \
    --cc=paulmck@kernel.org \
    --cc=qiang.zhang@linux.dev \
    --cc=rcu@vger.kernel.org \
    --cc=rostedt@goodmis.org \
    --cc=urezki@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®