[PATCH 00/13] srcu: Round out atomic SRCU support

[Kunwu Chan <kunwu.chan@gmail.com>]

From: Kunwu Chan <kunwu.chan@gmail.com>

This series rounds out atomic SRCU support with a reader-free fastpath,
some atomic-context fixes, and additional test coverage.

The core change is a reader-free fastpath for synchronize_srcu_atomic().
Atomic SRCU's read-side critical sections disable preemption, so when
the summed lock and unlock counts match for both ranks,
synchronize_srcu_atomic() can end the grace period immediately, without
the index flip or the srcu_advance_state() scans.  Correctness depends
on the grace-period anchor written by srcu_gp_start() preceding the
lock-count scan; two LKMM litmus tests (patches 1-2) verify that
ordering.

The series is:

 1-2  LKMM litmus tests for the reader-free fastpath ordering.
   3  srcutree: add the reader-free fastpath.
   4  rcutorture: add --do-atomic-srcu to torture.sh.
 5-7  srcutree: honor and propagate is_atomic so atomic SRCU never
      transitions to big.
   8  srcutree: forbid srcu_expedite_current() on atomic SRCU.
   9  rcutorture: disable srcu_expedite_current() for atomic SRCU.
  10  srcutree: skip callback scheduling for atomic SRCU grace periods.
  11  srcutree: drop the srcu_barrier() sleep for atomic SRCU.
  12  srcutree: remove leftover debug pr_alert()s.
  13  srcu: restrict the non_block annotation to task context, fixing a
      KCSAN data race.

The KCSAN race in patch 13 is a data race on the interrupted task's
->non_block_count when a timer callback runs atomic-SRCU readers in
inline softirq context.

Tested with herd7 7.58 (litmus tests), torture.sh --do-atomic-srcu, and
torture.sh --do-atomic-srcu --do-kcsan (KCSAN+PROVE_LOCKING).

Kunwu Chan (13):
  litmus: Add SRCU fastpath anchor-before-scan test
  litmus: Add SRCU fastpath scan-before-anchor test
  srcutree: Add reader-free fastpath to synchronize_srcu_atomic()
  rcutorture: Add atomic-SRCU support to torture.sh
  srcutree: Honor is_atomic in check_init_srcu_struct()
  srcutree: Make init_srcu_struct_atomic() prevent transition to big
  srcutree: Don't transition atomic SRCU to big in srcu_gp_end()
  srcutree: Forbid srcu_expedite_current() on atomic SRCU
  rcutorture: Disable srcu_expedite_current() for atomic SRCU
  srcutree: Skip callback scheduling for atomic SRCU grace periods
  srcutree: Remove srcu_barrier() sleep for atomic SRCU
  srcutree: Remove debug pr_alert()s
  srcu: Restrict atomic-SRCU non_block annotation to task context

 include/linux/srcu.h                          | 10 +-
 kernel/rcu/rcutorture.c                       |  1 +
 kernel/rcu/srcutree.c                         | 95 ++++++++++++++-----
 tools/memory-model/litmus-tests/README        | 16 ++++
 .../SRCU-fastpath-anchor-before-scan.litmus   | 56 +++++++++++
 .../SRCU-fastpath-scan-before-anchor.litmus   | 53 +++++++++++
 .../selftests/rcutorture/bin/torture.sh       | 24 +++++
 7 files changed, 229 insertions(+), 26 deletions(-)
 create mode 100644 tools/memory-model/litmus-tests/SRCU-fastpath-anchor-before-scan.litmus
 create mode 100644 tools/memory-model/litmus-tests/SRCU-fastpath-scan-before-anchor.litmus


base-commit: ef15070f96c5f07de3a9b593c36f6e83bc9e3d58
-- 
2.43.0