[PATCH 0/8] md/raid5: scalability and rebuild-path improvements

[PATCH 0/8] md/raid5: scalability and rebuild-path improvements

[Hiroshi Nishida]

This series collects small, individually low-risk md/raid5 changes for
large, many-core, many-disk arrays.  Their common theme is reducing
per-stripe and stripe-cache contention, so the benefit appears mainly
when the raid5 stripe-handling worker threads are in use
(group_thread_cnt > 0); at the default group_thread_cnt = 0 (a single
handling thread) the series is essentially neutral.

 - patches 1-3 remove signed arithmetic from a hot-path divisor, lift an
   arbitrary stripe-cache size cap, and widen a badblock length argument
   that currently truncates large ranges;
 - patch 4 raises NR_STRIPE_HASH_LOCKS (8 -> 32) to spread stripe-hash
   contention on high core-count systems;
 - patches 5 and 8 reduce per-stripe overhead in the resync/recovery
   path and bound the share of the stripe cache a rebuild may hold while
   user I/O is competing;
 - patch 6 allocates each worker group's array on its own NUMA node;
 - patch 7 raises MAX_STRIPE_BATCH (8 -> 32).

Measured effect, treatment vs baseline, % change in mean IOPS (N=3),
swept over group_thread_cnt (RAID6 4+2, 22-core host, ramdisk members):

  workload                       gtc=0   gtc=2   gtc=4   gtc=8
  random 4K write (RMW)          +4.2%   +8.1%  +17.4%   +6.5%
  DB mixed 75/25 8K              +0.4%   +4.2%  +10.3%   +4.7%
  high-concurrency 70/30 4K      +3.9%   +1.2%  +10.0%   +0.2%
  OLTP 70/30 16K                 -0.3%   +4.7%  +10.1%   +9.3%
  partial-stripe write 8K        +1.1%   +4.8%  +11.2%  +14.2%

At the default single handling thread (group_thread_cnt = 0) the series is
neutral (no regression).  As worker threads are added the gain grows,
peaking broadly around group_thread_cnt = 4 at roughly +10-17% across the
whole mix; at gtc = 8 the write-heavy workloads keep gaining while the
read-heavy high-concurrency case has saturated.  (Per-run cv was <1%
except the random-write test, ~5-9%, from a cold first run.)

These numbers are on a ramdisk, which removes device latency and so
overstates the CPU-side contention effect relative to a real device;
they show the direction and the group_thread_cnt dependence, not an
absolute speedup.  The stripe-hash/batch patches (4, 7) and the cache cap
(2) drive this; patch 6 only matters on multi-socket systems (not
exercised above) and patches 5/8 act on the resync/recovery path rather
than this steady-state workload.

Reproduction (stock mdadm + fio):
  mdadm --create /dev/md0 --level=6 --raid-devices=6 --chunk=512 \
        --assume-clean <6 members>
  echo 16384 > /sys/block/md0/md/stripe_cache_size
  echo N     > /sys/block/md0/md/group_thread_cnt      # N = 0,2,4,8
  fio --filename=/dev/md0 --direct=1 --ioengine=libaio --group_reporting \
      --time_based --runtime=15 --name=w <per-workload opts>:
    random write   : --rw=randwrite              --bs=4k  --numjobs=4  --iodepth=32
    DB mixed       : --rw=randrw --rwmixread=75  --bs=8k  --numjobs=8  --iodepth=16
    high-concur.   : --rw=randrw --rwmixread=70  --bs=4k  --numjobs=16 --iodepth=8
    OLTP           : --rw=randrw --rwmixread=70  --bs=16k --numjobs=6  --iodepth=16
    partial-stripe : --rw=randwrite              --bs=8k  --numjobs=4  --iodepth=32

Each patch stands on its own; I am happy to drop or defer any that is not
justified on its own merit.

Functional testing on RAID5 and RAID6: create, fail a member, rebuild
onto a spare / re-add, full data read-back verified, and scrub
("check") reporting mismatch_cnt == 0.  The series was also exercised
with KASAN and lockdep enabled -- including heavy group_thread_cnt
churn on a multi-node setup to stress the per-NUMA-node worker
allocation and the raid5_quiesce hash-lock-all path -- with no reports.

Hiroshi Nishida (8):
  md: change chunk_sectors and stripe cache counts to unsigned int
  md/raid5: raise stripe cache limit from 32768 to 262144
  md: widen badblock sectors param from int to sector_t
  md/raid5: raise NR_STRIPE_HASH_LOCKS from 8 to 32
  md/raid5: submit a window of stripes during resync/recovery
  md/raid5: allocate worker groups per NUMA node
  md/raid5: raise MAX_STRIPE_BATCH from 8 to 32
  md/raid5: reserve stripe cache for user I/O during rebuild

 drivers/md/md.c    |   4 +-
 drivers/md/md.h    |  10 ++--
 drivers/md/raid5.c | 129 ++++++++++++++++++++++++++++++++-------------
 drivers/md/raid5.h |  33 ++++++++----
 4 files changed, 121 insertions(+), 55 deletions(-)

base-commit: 55b77337bdd088c77461588e5ec094421b89911b

-- 
2.43.0

[PATCH 1/8] md: change chunk_sectors and stripe cache counts to unsigned int

[Hiroshi Nishida]

chunk_sectors, new_chunk_sectors, prev_chunk_sectors, max_nr_stripes,
and min_nr_stripes are never negative. Using signed int is semantically
wrong and prevents the compiler from optimizing division/modulo by
power-of-two chunk sizes to right shifts in the hot I/O path.

Change all struct fields and derived local variables to unsigned int:
  mddev->chunk_sectors
  mddev->new_chunk_sectors
  r5conf->chunk_sectors
  r5conf->prev_chunk_sectors
  r5conf->max_nr_stripes
  r5conf->min_nr_stripes
  Local: sectors_per_chunk, new_chunk, chunk_sectors

The min() in r5c_check_cached_full_stripe() required both operands to
match signedness; this is now satisfied with max_nr_stripes unsigned.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Hiroshi Nishida <nishidafmly@gmail.com>
---
 drivers/md/md.h    |  4 ++--
 drivers/md/raid5.c | 14 +++++++-------
 drivers/md/raid5.h |  8 ++++----
 3 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/drivers/md/md.h b/drivers/md/md.h
index d8daf0f75cbb..b9ad26844799 100644
--- a/drivers/md/md.h
+++ b/drivers/md/md.h
@@ -437,7 +437,7 @@ struct mddev {
 	int				external;	/* metadata is
 							 * managed externally */
 	char				metadata_type[17]; /* externally set*/
-	int				chunk_sectors;
+	unsigned int			chunk_sectors;
 	time64_t			ctime, utime;
 	int				level, layout;
 	char				clevel[16];
@@ -466,7 +466,7 @@ struct mddev {
 	 */
 	sector_t			reshape_position;
 	int				delta_disks, new_level, new_layout;
-	int				new_chunk_sectors;
+	unsigned int			new_chunk_sectors;
 	int				reshape_backwards;
 
 	struct md_thread __rcu		*thread;	/* management thread */
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index 0c5c9fb0606e..28828e083c2b 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -2970,7 +2970,7 @@ sector_t raid5_compute_sector(struct r5conf *conf, sector_t r_sector,
 	sector_t new_sector;
 	int algorithm = previous ? conf->prev_algo
 				 : conf->algorithm;
-	int sectors_per_chunk = previous ? conf->prev_chunk_sectors
+	unsigned int sectors_per_chunk = previous ? conf->prev_chunk_sectors
 					 : conf->chunk_sectors;
 	int raid_disks = previous ? conf->previous_raid_disks
 				  : conf->raid_disks;
@@ -3166,7 +3166,7 @@ sector_t raid5_compute_blocknr(struct stripe_head *sh, int i, int previous)
 	int raid_disks = sh->disks;
 	int data_disks = raid_disks - conf->max_degraded;
 	sector_t new_sector = sh->sector, check;
-	int sectors_per_chunk = previous ? conf->prev_chunk_sectors
+	unsigned int sectors_per_chunk = previous ? conf->prev_chunk_sectors
 					 : conf->chunk_sectors;
 	int algorithm = previous ? conf->prev_algo
 				 : conf->algorithm;
@@ -3584,7 +3584,7 @@ static void end_reshape(struct r5conf *conf);
 static void stripe_set_idx(sector_t stripe, struct r5conf *conf, int previous,
 			    struct stripe_head *sh)
 {
-	int sectors_per_chunk =
+	unsigned int sectors_per_chunk =
 		previous ? conf->prev_chunk_sectors : conf->chunk_sectors;
 	int dd_idx;
 	int chunk_offset = sector_div(stripe, sectors_per_chunk);
@@ -6103,7 +6103,7 @@ static enum stripe_result make_stripe_request(struct mddev *mddev,
 static sector_t raid5_bio_lowest_chunk_sector(struct r5conf *conf,
 					      struct bio *bi)
 {
-	int sectors_per_chunk = conf->chunk_sectors;
+	unsigned int sectors_per_chunk = conf->chunk_sectors;
 	int raid_disks = conf->raid_disks;
 	int dd_idx;
 	struct stripe_head sh;
@@ -7930,7 +7930,7 @@ static int raid5_run(struct mddev *mddev)
 		sector_t here_new, here_old;
 		int old_disks;
 		int max_degraded = (mddev->level == 6 ? 2 : 1);
-		int chunk_sectors;
+		unsigned int chunk_sectors;
 		int new_data_disks;
 
 		if (journal_dev) {
@@ -8832,7 +8832,7 @@ static int raid5_check_reshape(struct mddev *mddev)
 	 * to be used by a reshape pass.
 	 */
 	struct r5conf *conf = mddev->private;
-	int new_chunk = mddev->new_chunk_sectors;
+	unsigned int new_chunk = mddev->new_chunk_sectors;
 
 	if (mddev->new_layout >= 0 && !algorithm_valid_raid5(mddev->new_layout))
 		return -EINVAL;
@@ -8866,7 +8866,7 @@ static int raid5_check_reshape(struct mddev *mddev)
 
 static int raid6_check_reshape(struct mddev *mddev)
 {
-	int new_chunk = mddev->new_chunk_sectors;
+	unsigned int new_chunk = mddev->new_chunk_sectors;
 
 	if (mddev->new_layout >= 0 && !algorithm_valid_raid6(mddev->new_layout))
 		return -EINVAL;
diff --git a/drivers/md/raid5.h b/drivers/md/raid5.h
index cb5feae04db2..5cd9d0f36b6e 100644
--- a/drivers/md/raid5.h
+++ b/drivers/md/raid5.h
@@ -572,12 +572,12 @@ struct r5conf {
 	/* only protect corresponding hash list and inactive_list */
 	spinlock_t		hash_locks[NR_STRIPE_HASH_LOCKS];
 	struct mddev		*mddev;
-	int			chunk_sectors;
+	unsigned int		chunk_sectors;
 	int			level, algorithm, rmw_level;
 	int			max_degraded;
 	int			raid_disks;
-	int			max_nr_stripes;
-	int			min_nr_stripes;
+	unsigned int		max_nr_stripes;
+	unsigned int		min_nr_stripes;
 #if PAGE_SIZE != DEFAULT_STRIPE_SIZE
 	unsigned long	stripe_size;
 	unsigned int	stripe_shift;
@@ -595,7 +595,7 @@ struct r5conf {
 	 */
 	sector_t		reshape_safe;
 	int			previous_raid_disks;
-	int			prev_chunk_sectors;
+	unsigned int		prev_chunk_sectors;
 	int			prev_algo;
 	short			generation; /* increments with every reshape */
 	seqcount_spinlock_t	gen_lock;	/* lock against generation changes */
-- 
2.43.0

[PATCH 2/8] md/raid5: raise stripe cache limit from 32768 to 262144

[Hiroshi Nishida]

Stripe cache memory is approximately max_nr_stripes * num_disks *
stripe_size, where stripe_size is 4KB (PAGE_SIZE).  The old hardcoded
ceiling of 32768 stripes therefore limits a 12-disk array to ~1.5GB of
stripe cache.  On servers with 256GB+ RAM backing wide arrays, a larger
cache reduces stalls on stripe-cache misses under heavy write workloads.

Define RAID5_MAX_NR_STRIPES = 262144 (256K) in raid5.h and use it in
raid5_set_cache_size() instead of the magic 32768 literal.  This allows
up to ~12GB of stripe cache on a 12-disk array.  The default stripe count
(NR_STRIPES = 256) is unchanged, so there is no memory impact unless the
administrator raises stripe_cache_size explicitly.

Also fix two local variables in raid5_cache_count() that were declared
as int but read from unsigned int fields; change to unsigned int to
avoid implicit sign-extension in the subtraction.

Tested: echo 262144 > /sys/block/md0/md/stripe_cache_size accepted;
262145 correctly returns EINVAL.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Hiroshi Nishida <nishidafmly@gmail.com>
---
 drivers/md/raid5.c | 6 +++---
 drivers/md/raid5.h | 6 ++++++
 2 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index 28828e083c2b..9cb4ed3bd85c 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -6923,7 +6923,7 @@ raid5_set_cache_size(struct mddev *mddev, int size)
 	int result = 0;
 	struct r5conf *conf = mddev->private;
 
-	if (size <= 16 || size > 32768)
+	if (size <= 16 || size > RAID5_MAX_NR_STRIPES)
 		return -EINVAL;
 
 	WRITE_ONCE(conf->min_nr_stripes, size);
@@ -7505,8 +7505,8 @@ static unsigned long raid5_cache_count(struct shrinker *shrink,
 				       struct shrink_control *sc)
 {
 	struct r5conf *conf = shrink->private_data;
-	int max_stripes = READ_ONCE(conf->max_nr_stripes);
-	int min_stripes = READ_ONCE(conf->min_nr_stripes);
+	unsigned int max_stripes = READ_ONCE(conf->max_nr_stripes);
+	unsigned int min_stripes = READ_ONCE(conf->min_nr_stripes);
 
 	if (max_stripes < min_stripes)
 		/* unlikely, but not impossible */
diff --git a/drivers/md/raid5.h b/drivers/md/raid5.h
index 5cd9d0f36b6e..57349737d393 100644
--- a/drivers/md/raid5.h
+++ b/drivers/md/raid5.h
@@ -479,6 +479,12 @@ struct disk_info {
  */
 
 #define NR_STRIPES		256
+/* Maximum user-settable stripe cache size, in stripes.  Stripe cache memory is
+ * roughly max_nr_stripes * num_disks * stripe_size (stripe_size = 4KB), so the
+ * old cap of 32768 limited a 12-disk array to ~1.5GB.  Raise to 262144 to allow
+ * larger caches on big-RAM systems; the default (NR_STRIPES) is unchanged.
+ */
+#define RAID5_MAX_NR_STRIPES	262144U
 
 #if PAGE_SIZE == DEFAULT_STRIPE_SIZE
 #define STRIPE_SIZE		PAGE_SIZE
-- 
2.43.0

[PATCH 3/8] md: widen badblock sectors param from int to sector_t

[Hiroshi Nishida]

The badblocks core API -- badblocks_set(), badblocks_clear() and
badblocks_check() -- and the is_badblock() helper all take the range
length as sector_t.  The md wrappers rdev_set_badblocks(),
rdev_clear_badblocks() and rdev_has_badblock(), however, declared the
same length as int, narrowing sector_t to int and back again in the
middle of an otherwise 64-bit clean path.

Change the sectors parameter to sector_t in these three wrappers so it
matches the core API and is_badblock().  No functional change: current
callers pass per-I/O or per-resync-chunk lengths well within int range.
This just removes a gratuitous truncation point and keeps the type
consistent end to end.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Hiroshi Nishida <nishidafmly@gmail.com>
---
 drivers/md/md.c | 4 ++--
 drivers/md/md.h | 6 +++---
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/drivers/md/md.c b/drivers/md/md.c
index d1465bcd86c8..61f40fa41e78 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -10553,7 +10553,7 @@ EXPORT_SYMBOL(md_finish_reshape);
 /* Bad block management */
 
 /* Returns true on success, false on failure */
-bool rdev_set_badblocks(struct md_rdev *rdev, sector_t s, int sectors,
+bool rdev_set_badblocks(struct md_rdev *rdev, sector_t s, sector_t sectors,
 			int is_new)
 {
 	struct mddev *mddev = rdev->mddev;
@@ -10593,7 +10593,7 @@ bool rdev_set_badblocks(struct md_rdev *rdev, sector_t s, int sectors,
 }
 EXPORT_SYMBOL_GPL(rdev_set_badblocks);
 
-void rdev_clear_badblocks(struct md_rdev *rdev, sector_t s, int sectors,
+void rdev_clear_badblocks(struct md_rdev *rdev, sector_t s, sector_t sectors,
 			  int is_new)
 {
 	if (is_new)
diff --git a/drivers/md/md.h b/drivers/md/md.h
index b9ad26844799..95835a3286aa 100644
--- a/drivers/md/md.h
+++ b/drivers/md/md.h
@@ -311,7 +311,7 @@ static inline int is_badblock(struct md_rdev *rdev, sector_t s, sector_t sectors
 }
 
 static inline int rdev_has_badblock(struct md_rdev *rdev, sector_t s,
-				    int sectors)
+				    sector_t sectors)
 {
 	sector_t first_bad;
 	sector_t bad_sectors;
@@ -319,9 +319,9 @@ static inline int rdev_has_badblock(struct md_rdev *rdev, sector_t s,
 	return is_badblock(rdev, s, sectors, &first_bad, &bad_sectors);
 }
 
-extern bool rdev_set_badblocks(struct md_rdev *rdev, sector_t s, int sectors,
+extern bool rdev_set_badblocks(struct md_rdev *rdev, sector_t s, sector_t sectors,
 			       int is_new);
-extern void rdev_clear_badblocks(struct md_rdev *rdev, sector_t s, int sectors,
+extern void rdev_clear_badblocks(struct md_rdev *rdev, sector_t s, sector_t sectors,
 				 int is_new);
 struct md_cluster_info;
 struct md_cluster_operations;
-- 
2.43.0

[PATCH 4/8] md/raid5: raise NR_STRIPE_HASH_LOCKS from 8 to 32

[Hiroshi Nishida]

The stripe cache hash is striped across NR_STRIPE_HASH_LOCKS spinlocks
(see stripe_hash_locks_hash()).  The value has been 8 since the per-hash
locking was introduced, which is small for modern many-core servers:
with only 8 buckets, stripe cache lookup and allocation contend on the
same few locks once the CPU count greatly exceeds the bucket count.

Raise it to 32.  Two constraints bound the choice:

  - STRIPE_HASH_LOCKS_MASK is (NR_STRIPE_HASH_LOCKS - 1) and is used as
    a bitmask in stripe_hash_locks_hash(), so the value must be a power
    of two.

  - raid5_quiesce() acquires every hash lock plus device_lock at once
    via lock_all_device_hash_locks_irq().  That holds
    NR_STRIPE_HASH_LOCKS + 1 locks simultaneously, which must stay below
    MAX_LOCK_DEPTH (48) so the held-lock array does not overflow when
    lockdep is enabled.

32 is the largest power of two that satisfies both: 32 + 1 leaves
headroom under 48, whereas 64 would exceed it.  (The pre-existing
"must remain below 64" comment understates this; MAX_LOCK_DEPTH is the
real ceiling.)  STRIPE_HASH_LOCKS_MASK and all NR_STRIPE_HASH_LOCKS-
sized arrays scale automatically.

This is purely a lock-striping change; it does not affect stripe cache
correctness.  The benefit appears on many-core systems, while on small
systems the extra buckets are harmless.

Tested: loop-device RAID-5 create, write/verify, fail a disk, rebuild
onto a spare and scrub all complete cleanly.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Hiroshi Nishida <nishidafmly@gmail.com>
---
 drivers/md/raid5.h | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/drivers/md/raid5.h b/drivers/md/raid5.h
index 57349737d393..3efab71ebef7 100644
--- a/drivers/md/raid5.h
+++ b/drivers/md/raid5.h
@@ -498,12 +498,14 @@ struct disk_info {
 #define HASH_MASK		(NR_HASH - 1)
 #define MAX_STRIPE_BATCH	8
 
-/* NOTE NR_STRIPE_HASH_LOCKS must remain below 64.
- * This is because we sometimes take all the spinlocks
- * and creating that much locking depth can cause
- * problems.
+/* NR_STRIPE_HASH_LOCKS must be a power of two, since
+ * STRIPE_HASH_LOCKS_MASK masks with (NR_STRIPE_HASH_LOCKS - 1).
+ * It must also be small enough that taking all of them at once in
+ * lock_all_device_hash_locks_irq(), plus device_lock, keeps the held
+ * lock count below MAX_LOCK_DEPTH (48) with lockdep enabled.  32 is the
+ * largest power of two that satisfies both constraints.
  */
-#define NR_STRIPE_HASH_LOCKS 8
+#define NR_STRIPE_HASH_LOCKS 32
 #define STRIPE_HASH_LOCKS_MASK (NR_STRIPE_HASH_LOCKS - 1)
 
 struct r5worker {
-- 
2.43.0

[PATCH 5/8] md/raid5: submit a window of stripes during resync/recovery

[Hiroshi Nishida]

raid5_sync_request() dispatches one stripe per call: it fetches a single
stripe head, marks it for sync, and returns one stripe's worth of
sectors.  When the stripe cache is full the NOBLOCK fetch fails and it
re-enters a one-jiffy throttle sleep (schedule_timeout_uninterruptible(1))
before retrying.  Because that sleep is taken per stripe, sustained cache
pressure bounds sync progress to roughly HZ stripes/second regardless of
how fast the member devices are.

Dispatch up to RAID5_SYNC_WINDOW (32) stripes per call instead.  Only the
first stripe of the window keeps the original behaviour (block, then the
one-jiffy throttle if the cache was full); the remaining stripes are
requested with R5_GAS_NOBLOCK and the loop stops as soon as the cache is
full.  So at most one throttle sleep is taken per window rather than per
stripe, and when the cache has free slots a single call can queue a batch
instead of one stripe at a time.

With a warm cache the window stays near full: counting raid5_sync_request()
invocations across a rebuild showed it averaging ~30 of the 32 stripes per
call, i.e. roughly 30x fewer calls into the sync path for the same resync.

The return value reports the number of stripes actually submitted, so
md_do_sync()'s recovery_active accounting stays balanced, and the window
is bounded by both the end of the sync region (max_sector) and
mddev->resync_max, so a user- or cluster-imposed sync ceiling is not
overshot.

This does not change which data is read or written during resync or
recovery.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Hiroshi Nishida <nishidafmly@gmail.com>
---
 drivers/md/raid5.c | 47 +++++++++++++++++++++++++++++++++-------------
 drivers/md/raid5.h |  1 +
 2 files changed, 35 insertions(+), 13 deletions(-)

diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index 9cb4ed3bd85c..8e9edaaca667 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -6563,7 +6563,8 @@ static inline sector_t raid5_sync_request(struct mddev *mddev, sector_t sector_n
 	struct stripe_head *sh;
 	sector_t sync_blocks;
 	bool still_degraded = false;
-	int i;
+	int i, submitted;
+	sector_t win_sector;
 
 	if (sector_nr >= max_sector) {
 		/* just being told to finish up .. nothing much to do */
@@ -6620,16 +6621,7 @@ static inline sector_t raid5_sync_request(struct mddev *mddev, sector_t sector_n
 	if (md_bitmap_enabled(mddev, false))
 		mddev->bitmap_ops->cond_end_sync(mddev, sector_nr, false);
 
-	sh = raid5_get_active_stripe(conf, NULL, sector_nr,
-				     R5_GAS_NOBLOCK);
-	if (sh == NULL) {
-		sh = raid5_get_active_stripe(conf, NULL, sector_nr, 0);
-		/* make sure we don't swamp the stripe cache if someone else
-		 * is trying to get access
-		 */
-		schedule_timeout_uninterruptible(1);
-	}
-	/* Need to check if array will still be degraded after recovery/resync
+	/* Check once whether array will still be degraded after recovery/resync.
 	 * Note in case of > 1 drive failures it's possible we're rebuilding
 	 * one drive while leaving another faulty drive in array.
 	 */
@@ -6640,13 +6632,42 @@ static inline sector_t raid5_sync_request(struct mddev *mddev, sector_t sector_n
 			still_degraded = true;
 	}
 
+	/* First stripe: block if stripe cache is full, then throttle. */
+	sh = raid5_get_active_stripe(conf, NULL, sector_nr, R5_GAS_NOBLOCK);
+	if (sh == NULL) {
+		sh = raid5_get_active_stripe(conf, NULL, sector_nr, 0);
+		/* make sure we don't swamp the stripe cache if someone else
+		 * is trying to get access
+		 */
+		schedule_timeout_uninterruptible(1);
+	}
 	md_bitmap_start_sync(mddev, sector_nr, &sync_blocks, still_degraded);
 	set_bit(STRIPE_SYNC_REQUESTED, &sh->state);
 	set_bit(STRIPE_HANDLE, &sh->state);
-
 	raid5_release_stripe(sh);
 
-	return RAID5_STRIPE_SECTORS(conf);
+	/* Submit remaining stripes in the window non-blocking.  Stop early
+	 * if the stripe cache is full: the disk queue is already saturated.
+	 * Bound by resync_max so a user- or cluster-imposed sync ceiling is
+	 * not overshot.
+	 */
+	win_sector = sector_nr + RAID5_STRIPE_SECTORS(conf);
+	for (submitted = 1;
+	     submitted < RAID5_SYNC_WINDOW && win_sector < max_sector &&
+	     win_sector < mddev->resync_max;
+	     submitted++, win_sector += RAID5_STRIPE_SECTORS(conf)) {
+		sh = raid5_get_active_stripe(conf, NULL, win_sector,
+					     R5_GAS_NOBLOCK);
+		if (!sh)
+			break;
+		md_bitmap_start_sync(mddev, win_sector, &sync_blocks,
+				     still_degraded);
+		set_bit(STRIPE_SYNC_REQUESTED, &sh->state);
+		set_bit(STRIPE_HANDLE, &sh->state);
+		raid5_release_stripe(sh);
+	}
+
+	return submitted * RAID5_STRIPE_SECTORS(conf);
 }
 
 static int  retry_aligned_read(struct r5conf *conf, struct bio *raid_bio,
diff --git a/drivers/md/raid5.h b/drivers/md/raid5.h
index 3efab71ebef7..7aeba1fc7f09 100644
--- a/drivers/md/raid5.h
+++ b/drivers/md/raid5.h
@@ -497,6 +497,7 @@ struct disk_info {
 #define NR_HASH			(PAGE_SIZE / sizeof(struct hlist_head))
 #define HASH_MASK		(NR_HASH - 1)
 #define MAX_STRIPE_BATCH	8
+#define RAID5_SYNC_WINDOW	32	/* stripes to pre-submit per sync_request call */
 
 /* NR_STRIPE_HASH_LOCKS must be a power of two, since
  * STRIPE_HASH_LOCKS_MASK masks with (NR_STRIPE_HASH_LOCKS - 1).
-- 
2.43.0

[PATCH 6/8] md/raid5: allocate worker groups per NUMA node

[Hiroshi Nishida]

alloc_thread_groups() previously allocated all r5worker arrays in a
single kcalloc() block, assigning workers for NUMA node N from node 0
memory.  On multi-socket systems this causes remote memory traffic on
every worker->work and worker->temp_inactive_list access.

Replace the single allocation with kzalloc_node(size, GFP_NOIO, i) per
group so each node's workers live in local memory.  Because the workers
are now separate per-node allocations, both free sites --
free_thread_groups() and the reallocation path in
raid5_store_group_thread_cnt() -- are updated to free each group's
allocation individually instead of only group 0's.

Also fix a latent bug: the original kcalloc() had its nmemb and size
arguments swapped (harmless due to commutativity but semantically wrong).

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Hiroshi Nishida <nishidafmly@gmail.com>
---
 drivers/md/raid5.c | 39 ++++++++++++++++++++++++++-------------
 1 file changed, 26 insertions(+), 13 deletions(-)

diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index 8e9edaaca667..c8787ab7b309 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -7297,8 +7297,12 @@ raid5_store_group_thread_cnt(struct mddev *mddev, const char *page, size_t len)
 			conf->worker_groups = new_groups;
 			spin_unlock_irq(&conf->device_lock);
 
-			if (old_groups)
-				kfree(old_groups[0].workers);
+			if (old_groups) {
+				int node;
+
+				for (node = 0; node < num_possible_nodes(); node++)
+					kfree(old_groups[node].workers);
+			}
 			kfree(old_groups);
 		}
 	}
@@ -7336,7 +7340,6 @@ static int alloc_thread_groups(struct r5conf *conf, int cnt, int *group_cnt,
 {
 	int i, j, k;
 	ssize_t size;
-	struct r5worker *workers;
 
 	if (cnt == 0) {
 		*group_cnt = 0;
@@ -7344,24 +7347,24 @@ static int alloc_thread_groups(struct r5conf *conf, int cnt, int *group_cnt,
 		return 0;
 	}
 	*group_cnt = num_possible_nodes();
-	size = sizeof(struct r5worker) * cnt;
-	workers = kcalloc(size, *group_cnt, GFP_NOIO);
 	*worker_groups = kzalloc_objs(struct r5worker_group, *group_cnt,
 				      GFP_NOIO);
-	if (!*worker_groups || !workers) {
-		kfree(workers);
-		kfree(*worker_groups);
+	if (!*worker_groups)
 		return -ENOMEM;
-	}
 
+	size = sizeof(struct r5worker) * cnt;
 	for (i = 0; i < *group_cnt; i++) {
-		struct r5worker_group *group;
+		struct r5worker_group *group = &(*worker_groups)[i];
+		struct r5worker *workers;
+
+		workers = kzalloc_node(size, GFP_NOIO, i);
+		if (!workers)
+			goto out_free;
 
-		group = &(*worker_groups)[i];
 		INIT_LIST_HEAD(&group->handle_list);
 		INIT_LIST_HEAD(&group->loprio_list);
 		group->conf = conf;
-		group->workers = workers + i * cnt;
+		group->workers = workers;
 
 		for (j = 0; j < cnt; j++) {
 			struct r5worker *worker = group->workers + j;
@@ -7374,12 +7377,22 @@ static int alloc_thread_groups(struct r5conf *conf, int cnt, int *group_cnt,
 	}
 
 	return 0;
+
+out_free:
+	while (--i >= 0)
+		kfree((*worker_groups)[i].workers);
+	kfree(*worker_groups);
+	*worker_groups = NULL;
+	return -ENOMEM;
 }
 
 static void free_thread_groups(struct r5conf *conf)
 {
+	int i;
+
 	if (conf->worker_groups)
-		kfree(conf->worker_groups[0].workers);
+		for (i = 0; i < conf->group_cnt; i++)
+			kfree(conf->worker_groups[i].workers);
 	kfree(conf->worker_groups);
 	conf->worker_groups = NULL;
 }
-- 
2.43.0

[PATCH 7/8] md/raid5: raise MAX_STRIPE_BATCH from 8 to 32

[Hiroshi Nishida]

handle_active_stripes() dequeues up to MAX_STRIPE_BATCH stripes under
device_lock per pass, then processes them after dropping the lock.
Raising the batch from 8 to 32 amortizes each device_lock acquisition
over more stripes, reducing lock cycles per unit of stripe work.

The worker-spawn heuristic in raid5_wakeup_stripe_thread() also divided
by MAX_STRIPE_BATCH; split it into a separate STRIPE_BATCH_WORKERS=8 so
the spawn sensitivity is preserved unchanged while the batch size grows.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Hiroshi Nishida <nishidafmly@gmail.com>
---
 drivers/md/raid5.c | 2 +-
 drivers/md/raid5.h | 3 ++-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index c8787ab7b309..ad6230415af3 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -210,7 +210,7 @@ static void raid5_wakeup_stripe_thread(struct stripe_head *sh)
 	/* at least one worker should run to avoid race */
 	queue_work_on(sh->cpu, raid5_wq, &group->workers[0].work);
 
-	thread_cnt = group->stripes_cnt / MAX_STRIPE_BATCH - 1;
+	thread_cnt = group->stripes_cnt / STRIPE_BATCH_WORKERS - 1;
 	/* wakeup more workers */
 	for (i = 1; i < conf->worker_cnt_per_group && thread_cnt > 0; i++) {
 		if (group->workers[i].working == false) {
diff --git a/drivers/md/raid5.h b/drivers/md/raid5.h
index 7aeba1fc7f09..1f37dabd727b 100644
--- a/drivers/md/raid5.h
+++ b/drivers/md/raid5.h
@@ -496,7 +496,8 @@ struct disk_info {
 #define BYPASS_THRESHOLD	1
 #define NR_HASH			(PAGE_SIZE / sizeof(struct hlist_head))
 #define HASH_MASK		(NR_HASH - 1)
-#define MAX_STRIPE_BATCH	8
+#define MAX_STRIPE_BATCH	32	/* stripes per handle_active_stripes pass */
+#define STRIPE_BATCH_WORKERS	8	/* stripes-per-worker threshold for spawning */
 #define RAID5_SYNC_WINDOW	32	/* stripes to pre-submit per sync_request call */
 
 /* NR_STRIPE_HASH_LOCKS must be a power of two, since
-- 
2.43.0

[PATCH 8/8] md/raid5: reserve stripe cache for user I/O during rebuild

[Hiroshi Nishida]

The resync read-ahead window (RAID5_SYNC_WINDOW) can fill the stripe
cache with rebuild stripes and starve concurrent user I/O, producing a
burst-starvation flip-flop between rebuild and application throughput.

Add two yield points to the window-submission loop:
 - stop the window immediately if any thread is waiting for a stripe
   (waitqueue_active(&conf->wait_for_stripe)); the check is intentionally
   racy -- a waiter appearing just after is serviced by the next
   sync_request call, so no barrier is needed.
 - stop expanding once active_stripes reaches half the cache
   (max_nr_stripes / RAID5_SYNC_HWMARK), but only when
   preread_active_stripes > 0, i.e. user write I/O is actually competing.
   Sync stripes never set STRIPE_PREREAD_ACTIVE, so during a pure rebuild
   the counter stays zero and the window fills freely; rebuild-only
   throughput is unchanged.

This bounds the share of the stripe cache a rebuild may hold while user
I/O is present, so application latency no longer collapses during the
read-ahead bursts, without throttling a rebuild that has the array to
itself.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Hiroshi Nishida <nishidafmly@gmail.com>
---
 drivers/md/raid5.c | 21 +++++++++++++++++++++
 drivers/md/raid5.h |  1 +
 2 files changed, 22 insertions(+)

diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index ad6230415af3..480f3aa069ef 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -6656,6 +6656,27 @@ static inline sector_t raid5_sync_request(struct mddev *mddev, sector_t sector_n
 	     submitted < RAID5_SYNC_WINDOW && win_sector < max_sector &&
 	     win_sector < mddev->resync_max;
 	     submitted++, win_sector += RAID5_STRIPE_SECTORS(conf)) {
+		/*
+		 * Yield to user I/O: stop the read-ahead if anyone is waiting
+		 * for a stripe.  The check is intentionally racy -- a waiter
+		 * appearing just after is serviced by the next sync_request
+		 * call, so no barrier is needed.
+		 */
+		if (waitqueue_active(&conf->wait_for_stripe))
+			break;
+		/*
+		 * Reserve cache for user I/O only when it is actually competing.
+		 * preread_active_stripes counts stripes queued for write I/O
+		 * (including the read phase of RMW); sync stripes never set
+		 * STRIPE_PREREAD_ACTIVE, so during a pure rebuild it stays zero
+		 * and the window fills freely.  Competing user reads do not bump
+		 * the counter but are caught by the waitqueue_active() check
+		 * above.
+		 */
+		if (atomic_read(&conf->preread_active_stripes) > 0 &&
+		    atomic_read(&conf->active_stripes) >=
+		    conf->max_nr_stripes / RAID5_SYNC_HWMARK)
+			break;
 		sh = raid5_get_active_stripe(conf, NULL, win_sector,
 					     R5_GAS_NOBLOCK);
 		if (!sh)
diff --git a/drivers/md/raid5.h b/drivers/md/raid5.h
index 1f37dabd727b..7833cc07597f 100644
--- a/drivers/md/raid5.h
+++ b/drivers/md/raid5.h
@@ -499,6 +499,7 @@ struct disk_info {
 #define MAX_STRIPE_BATCH	32	/* stripes per handle_active_stripes pass */
 #define STRIPE_BATCH_WORKERS	8	/* stripes-per-worker threshold for spawning */
 #define RAID5_SYNC_WINDOW	32	/* stripes to pre-submit per sync_request call */
+#define RAID5_SYNC_HWMARK	2	/* rebuild uses at most 1/N of stripe cache */
 
 /* NR_STRIPE_HASH_LOCKS must be a power of two, since
  * STRIPE_HASH_LOCKS_MASK masks with (NR_STRIPE_HASH_LOCKS - 1).
-- 
2.43.0

Re: [PATCH 0/8] md/raid5: scalability and rebuild-path improvements

[yu kuai]

Hi,

在 2026/6/24 23:54, Hiroshi Nishida 写道:
> This series collects small, individually low-risk md/raid5 changes for
> large, many-core, many-disk arrays.  Their common theme is reducing
> per-stripe and stripe-cache contention, so the benefit appears mainly
> when the raid5 stripe-handling worker threads are in use
> (group_thread_cnt > 0); at the default group_thread_cnt = 0 (a single
> handling thread) the series is essentially neutral.
>
>   - patches 1-3 remove signed arithmetic from a hot-path divisor, lift an
>     arbitrary stripe-cache size cap, and widen a badblock length argument
>     that currently truncates large ranges;
>   - patch 4 raises NR_STRIPE_HASH_LOCKS (8 -> 32) to spread stripe-hash
>     contention on high core-count systems;
>   - patches 5 and 8 reduce per-stripe overhead in the resync/recovery
>     path and bound the share of the stripe cache a rebuild may hold while
>     user I/O is competing;
>   - patch 6 allocates each worker group's array on its own NUMA node;
>   - patch 7 raises MAX_STRIPE_BATCH (8 -> 32).
>
> Measured effect, treatment vs baseline, % change in mean IOPS (N=3),
> swept over group_thread_cnt (RAID6 4+2, 22-core host, ramdisk members):

Testing with ramdisk does serve as a useful reference, but it does not reflect
real world usage.

>
>    workload                       gtc=0   gtc=2   gtc=4   gtc=8
>    random 4K write (RMW)          +4.2%   +8.1%  +17.4%   +6.5%
>    DB mixed 75/25 8K              +0.4%   +4.2%  +10.3%   +4.7%
>    high-concurrency 70/30 4K      +3.9%   +1.2%  +10.0%   +0.2%
>    OLTP 70/30 16K                 -0.3%   +4.7%  +10.1%   +9.3%
>    partial-stripe write 8K        +1.1%   +4.8%  +11.2%  +14.2%

With a quick review I saw many static configurations is changed, I agree
these changes can improve arrays with ssd/nvme and a system with large
memory available. However, we already tested with hdd and about 8G memory
available, these changes will not improve performance at all, with the
extra memory overhead.

I can accept make those values configurable, but not direct modifications.
As validation is required for numerous scenarios. Memory resources are precious
especially for most consumer NAS devices.

>
> At the default single handling thread (group_thread_cnt = 0) the series is
> neutral (no regression).  As worker threads are added the gain grows,
> peaking broadly around group_thread_cnt = 4 at roughly +10-17% across the
> whole mix; at gtc = 8 the write-heavy workloads keep gaining while the
> read-heavy high-concurrency case has saturated.  (Per-run cv was <1%
> except the random-write test, ~5-9%, from a cold first run.)
>
> These numbers are on a ramdisk, which removes device latency and so
> overstates the CPU-side contention effect relative to a real device;
> they show the direction and the group_thread_cnt dependence, not an
> absolute speedup.  The stripe-hash/batch patches (4, 7) and the cache cap
> (2) drive this; patch 6 only matters on multi-socket systems (not
> exercised above) and patches 5/8 act on the resync/recovery path rather
> than this steady-state workload.
>
> Reproduction (stock mdadm + fio):
>    mdadm --create /dev/md0 --level=6 --raid-devices=6 --chunk=512 \
>          --assume-clean <6 members>
>    echo 16384 > /sys/block/md0/md/stripe_cache_size
>    echo N     > /sys/block/md0/md/group_thread_cnt      # N = 0,2,4,8
>    fio --filename=/dev/md0 --direct=1 --ioengine=libaio --group_reporting \
>        --time_based --runtime=15 --name=w <per-workload opts>:
>      random write   : --rw=randwrite              --bs=4k  --numjobs=4  --iodepth=32
>      DB mixed       : --rw=randrw --rwmixread=75  --bs=8k  --numjobs=8  --iodepth=16
>      high-concur.   : --rw=randrw --rwmixread=70  --bs=4k  --numjobs=16 --iodepth=8
>      OLTP           : --rw=randrw --rwmixread=70  --bs=16k --numjobs=6  --iodepth=16
>      partial-stripe : --rw=randwrite              --bs=8k  --numjobs=4  --iodepth=32
>
> Each patch stands on its own; I am happy to drop or defer any that is not
> justified on its own merit.
>
> Functional testing on RAID5 and RAID6: create, fail a member, rebuild
> onto a spare / re-add, full data read-back verified, and scrub
> ("check") reporting mismatch_cnt == 0.  The series was also exercised
> with KASAN and lockdep enabled -- including heavy group_thread_cnt
> churn on a multi-node setup to stress the per-NUMA-node worker
> allocation and the raid5_quiesce hash-lock-all path -- with no reports.
>
> Hiroshi Nishida (8):
>    md: change chunk_sectors and stripe cache counts to unsigned int
>    md/raid5: raise stripe cache limit from 32768 to 262144
>    md: widen badblock sectors param from int to sector_t
>    md/raid5: raise NR_STRIPE_HASH_LOCKS from 8 to 32
>    md/raid5: submit a window of stripes during resync/recovery
>    md/raid5: allocate worker groups per NUMA node
>    md/raid5: raise MAX_STRIPE_BATCH from 8 to 32
>    md/raid5: reserve stripe cache for user I/O during rebuild
>
>   drivers/md/md.c    |   4 +-
>   drivers/md/md.h    |  10 ++--
>   drivers/md/raid5.c | 129 ++++++++++++++++++++++++++++++++-------------
>   drivers/md/raid5.h |  33 ++++++++----
>   4 files changed, 121 insertions(+), 55 deletions(-)
>
> base-commit: 55b77337bdd088c77461588e5ec094421b89911b
>
-- 
Thanks,
Kuai

Re: [PATCH 0/8] md/raid5: scalability and rebuild-path improvements

[Hiroshi Nishida]

Hi Yu Kuai,

Thanks again for the review, and in particular for:

> I can accept make those values configurable, but not direct
> modifications.

I've reworked the tunables along exactly those lines and will send them as
a separate, self-contained series, "md/raid5: size stripe-cache and worker
tuning from the hardware".

Rather than raising any fixed constant, each value now derives a default
from the hardware and stays overridable:

  - the stripe-cache hash lock count (was a fixed 8) is sized from the CPU
    count, clamped to 8..32;
  - the stripe_cache_size ceiling (was a fixed 32768) scales with memory,
    but never drops below 32768;
  - the initial stripe_cache_size (was a fixed 256) scales gently with
    memory, 256..4096;
  - the default group_thread_cnt (was 0, i.e. single-threaded) is derived
    from the CPU count (half the online CPUs per NUMA node, capped);
  - the stripe batch size (was a fixed 8) is exposed as a parameter.

Ahead of those, the series opens with a one-line prerequisite fix:
alloc_thread_groups() sizes the per-node worker_groups[] array by
num_possible_nodes() but indexes it by cpu_to_node(), so a sparse NUMA node
map can index off the end -- reachable once the worker-group default (the
last patch) is on.  It is Fixes:-tagged and can be taken on its own.

The key point for the consumer-NAS concern you raised: each default only
rises on hardware that can back it -- the lock and worker counts scale with
the core count, the cache sizes with RAM -- so a genuinely small system (a
few cores and a few GB) keeps today's values and footprint. And each one is
overridable via a module parameter (group_thread_cnt also via its existing
sysfs attribute), including all the way back to today's behaviour. So nothing is
imposed; the default simply tracks the machine instead of a constant, and
a wide array on a large host no longer needs a recompile or manual
per-array tuning to use the memory and cores it has.

And on performance -- on real NVMe this time, not a ramdisk: on a 16-disk
raid6 array (a 32-vCPU / 16-core host, steady state, interleaved runs) the
hardware-derived defaults run 2.1-3.2x the stock defaults (4K random write
~39k -> ~100k IOPS), essentially all of it from patch 6 turning on the
worker groups.  Patches 2-5 (the cache and lock sizing) are
throughput-neutral on real SSD, as you'd expect -- they earn their place as
configurability and sane defaults, not a speed claim.  And there is no
regression at the small end: a 4-CPU box derives 2 workers and is never
slower on any workload, and a box with two or fewer CPUs derives 0 and is
byte-for-byte unchanged.

It has been through KASAN + lockdep + DEBUG_LIST on RAID5
(create/rebuild/scrub, plus the bitmap add/remove that drives the
lock-all-hash-locks quiesce path), at both the derived defaults and pinned
values including nr_stripe_hash_locks=32.

The two unrelated parts of the original series -- the type-widening /
correctness fixes and the resync/recovery dispatch changes -- I'll send as
their own small series, as discussed.

Thanks,

2026年7月5日(日) 19:56 yu kuai <yukuai@fygo.io>:
>
> Hi,
>
> 在 2026/6/24 23:54, Hiroshi Nishida 写道:
> > This series collects small, individually low-risk md/raid5 changes for
> > large, many-core, many-disk arrays.  Their common theme is reducing
> > per-stripe and stripe-cache contention, so the benefit appears mainly
> > when the raid5 stripe-handling worker threads are in use
> > (group_thread_cnt > 0); at the default group_thread_cnt = 0 (a single
> > handling thread) the series is essentially neutral.
> >
> >   - patches 1-3 remove signed arithmetic from a hot-path divisor, lift an
> >     arbitrary stripe-cache size cap, and widen a badblock length argument
> >     that currently truncates large ranges;
> >   - patch 4 raises NR_STRIPE_HASH_LOCKS (8 -> 32) to spread stripe-hash
> >     contention on high core-count systems;
> >   - patches 5 and 8 reduce per-stripe overhead in the resync/recovery
> >     path and bound the share of the stripe cache a rebuild may hold while
> >     user I/O is competing;
> >   - patch 6 allocates each worker group's array on its own NUMA node;
> >   - patch 7 raises MAX_STRIPE_BATCH (8 -> 32).
> >
> > Measured effect, treatment vs baseline, % change in mean IOPS (N=3),
> > swept over group_thread_cnt (RAID6 4+2, 22-core host, ramdisk members):
>
> Testing with ramdisk does serve as a useful reference, but it does not reflect
> real world usage.
>
> >
> >    workload                       gtc=0   gtc=2   gtc=4   gtc=8
> >    random 4K write (RMW)          +4.2%   +8.1%  +17.4%   +6.5%
> >    DB mixed 75/25 8K              +0.4%   +4.2%  +10.3%   +4.7%
> >    high-concurrency 70/30 4K      +3.9%   +1.2%  +10.0%   +0.2%
> >    OLTP 70/30 16K                 -0.3%   +4.7%  +10.1%   +9.3%
> >    partial-stripe write 8K        +1.1%   +4.8%  +11.2%  +14.2%
>
> With a quick review I saw many static configurations is changed, I agree
> these changes can improve arrays with ssd/nvme and a system with large
> memory available. However, we already tested with hdd and about 8G memory
> available, these changes will not improve performance at all, with the
> extra memory overhead.
>
> I can accept make those values configurable, but not direct modifications.
> As validation is required for numerous scenarios. Memory resources are precious
> especially for most consumer NAS devices.
>
> >
> > At the default single handling thread (group_thread_cnt = 0) the series is
> > neutral (no regression).  As worker threads are added the gain grows,
> > peaking broadly around group_thread_cnt = 4 at roughly +10-17% across the
> > whole mix; at gtc = 8 the write-heavy workloads keep gaining while the
> > read-heavy high-concurrency case has saturated.  (Per-run cv was <1%
> > except the random-write test, ~5-9%, from a cold first run.)
> >
> > These numbers are on a ramdisk, which removes device latency and so
> > overstates the CPU-side contention effect relative to a real device;
> > they show the direction and the group_thread_cnt dependence, not an
> > absolute speedup.  The stripe-hash/batch patches (4, 7) and the cache cap
> > (2) drive this; patch 6 only matters on multi-socket systems (not
> > exercised above) and patches 5/8 act on the resync/recovery path rather
> > than this steady-state workload.
> >
> > Reproduction (stock mdadm + fio):
> >    mdadm --create /dev/md0 --level=6 --raid-devices=6 --chunk=512 \
> >          --assume-clean <6 members>
> >    echo 16384 > /sys/block/md0/md/stripe_cache_size
> >    echo N     > /sys/block/md0/md/group_thread_cnt      # N = 0,2,4,8
> >    fio --filename=/dev/md0 --direct=1 --ioengine=libaio --group_reporting \
> >        --time_based --runtime=15 --name=w <per-workload opts>:
> >      random write   : --rw=randwrite              --bs=4k  --numjobs=4  --iodepth=32
> >      DB mixed       : --rw=randrw --rwmixread=75  --bs=8k  --numjobs=8  --iodepth=16
> >      high-concur.   : --rw=randrw --rwmixread=70  --bs=4k  --numjobs=16 --iodepth=8
> >      OLTP           : --rw=randrw --rwmixread=70  --bs=16k --numjobs=6  --iodepth=16
> >      partial-stripe : --rw=randwrite              --bs=8k  --numjobs=4  --iodepth=32
> >
> > Each patch stands on its own; I am happy to drop or defer any that is not
> > justified on its own merit.
> >
> > Functional testing on RAID5 and RAID6: create, fail a member, rebuild
> > onto a spare / re-add, full data read-back verified, and scrub
> > ("check") reporting mismatch_cnt == 0.  The series was also exercised
> > with KASAN and lockdep enabled -- including heavy group_thread_cnt
> > churn on a multi-node setup to stress the per-NUMA-node worker
> > allocation and the raid5_quiesce hash-lock-all path -- with no reports.
> >
> > Hiroshi Nishida (8):
> >    md: change chunk_sectors and stripe cache counts to unsigned int
> >    md/raid5: raise stripe cache limit from 32768 to 262144
> >    md: widen badblock sectors param from int to sector_t
> >    md/raid5: raise NR_STRIPE_HASH_LOCKS from 8 to 32
> >    md/raid5: submit a window of stripes during resync/recovery
> >    md/raid5: allocate worker groups per NUMA node
> >    md/raid5: raise MAX_STRIPE_BATCH from 8 to 32
> >    md/raid5: reserve stripe cache for user I/O during rebuild
> >
> >   drivers/md/md.c    |   4 +-
> >   drivers/md/md.h    |  10 ++--
> >   drivers/md/raid5.c | 129 ++++++++++++++++++++++++++++++++-------------
> >   drivers/md/raid5.h |  33 ++++++++----
> >   4 files changed, 121 insertions(+), 55 deletions(-)
> >
> > base-commit: 55b77337bdd088c77461588e5ec094421b89911b
> >
> --
> Thanks,
> Kuai



-- 
Hiroshi Nishida
nishidafmly@gmail.com

Re: [PATCH 0/8] md/raid5: scalability and rebuild-path improvements

[Hiroshi Nishida]

The cover quotes the 2.1-3.2x headline; here is the full per-workload data
behind it, in case it helps review.

The same raid456.ko is used in both arms -- only the module parameters
differ at array-create -- so this isolates the *defaults*, not code changes.
Real NVMe SSDs, not a ramdisk.

Setup: GCP n2-standard-32 -- 32 vCPUs = 16 physical cores (a vCPU is one
hyperthread), 2 NUMA nodes, 125 GB RAM, 16x 375 GB local NVMe. Kernel: full
build of the 6-patch series (base 55b77337), non-debug. raid6, 64k chunk,
bitmap=none, --assume-clean. Workloads: a 5-test fio suite (direct=1,
libaio, 30 s each): 4K random write (RMW), DB-mixed 75/25 8K,
high-concurrency 70/30 4K, OLTP 70/30 16K, partial-stripe 8K.

AUTO = all defaults left auto (the hardware-derived values); STOCK = the
historical fixed values (hash 8 / ceiling 32768 / batch 8 / cache 256 /
group_thread_cnt 0). On this host AUTO resolves to group_thread_cnt=8,
stripe_cache_size~3886, 32 hash locks.

1) Wide 16-disk raid6 (14+2), steady state, interleaved N=5 (STOCK and AUTO
   alternate each round so SSD drift cancels; cv 1-8%):

     Workload             STOCK IOPS    AUTO IOPS    AUTO/STOCK
     4K random write          39,147      100,069        2.56x
     DB mixed 75/25 8K        88,040      208,284        2.37x
     High-conc 70/30 4K      122,765      257,952        2.10x
     OLTP 70/30 16K           39,597      100,033        2.53x
     Partial-stripe 8K        20,491       64,511        3.15x

2) Low-end 4-CPU / 8 GB, 6-disk raid6 (4+2), N=3 -- no-regression control.
   4 online CPUs + 8 GB, single node (num_online_cpus()=4 -> gtc=2); the
   cache stays 256 and the hash count stays 8, so AUTO differs from STOCK
   only in group_thread_cnt (2 vs 0):

     Workload             STOCK IOPS    AUTO IOPS    AUTO/STOCK
     4K random write          52,017       81,132        1.56x
     DB mixed                125,043      126,188        1.01x
     High-conc               180,570      217,074        1.20x
     OLTP                     54,308       55,840        1.03x
     Partial-stripe           27,060       42,751        1.58x

   No regression on any workload; modest write gains even here. A
   slower-disk NAS is more device-bound (the gains flatten) but still shows
   no regression, and a box with <=2 online CPUs derives gtc=0 =
   byte-for-byte stock.

The gain is essentially patch 6 (worker groups on by default); patches 2-5
(cache and lock sizing) are throughput-neutral on real SSD and earn their
place as configurability plus sane defaults, not a speed claim. Patch 1 is
the NUMA-sizing prerequisite and does not affect throughput.

Thanks,

2026年7月10日(金) 6:09 Hiroshi Nishida <nishidafmly@gmail.com>:
>
> Hi Yu Kuai,
>
> Thanks again for the review, and in particular for:
>
> > I can accept make those values configurable, but not direct
> > modifications.
>
> I've reworked the tunables along exactly those lines and will send them as
> a separate, self-contained series, "md/raid5: size stripe-cache and worker
> tuning from the hardware".
>
> Rather than raising any fixed constant, each value now derives a default
> from the hardware and stays overridable:
>
>   - the stripe-cache hash lock count (was a fixed 8) is sized from the CPU
>     count, clamped to 8..32;
>   - the stripe_cache_size ceiling (was a fixed 32768) scales with memory,
>     but never drops below 32768;
>   - the initial stripe_cache_size (was a fixed 256) scales gently with
>     memory, 256..4096;
>   - the default group_thread_cnt (was 0, i.e. single-threaded) is derived
>     from the CPU count (half the online CPUs per NUMA node, capped);
>   - the stripe batch size (was a fixed 8) is exposed as a parameter.
>
> Ahead of those, the series opens with a one-line prerequisite fix:
> alloc_thread_groups() sizes the per-node worker_groups[] array by
> num_possible_nodes() but indexes it by cpu_to_node(), so a sparse NUMA node
> map can index off the end -- reachable once the worker-group default (the
> last patch) is on.  It is Fixes:-tagged and can be taken on its own.
>
> The key point for the consumer-NAS concern you raised: each default only
> rises on hardware that can back it -- the lock and worker counts scale with
> the core count, the cache sizes with RAM -- so a genuinely small system (a
> few cores and a few GB) keeps today's values and footprint. And each one is
> overridable via a module parameter (group_thread_cnt also via its existing
> sysfs attribute), including all the way back to today's behaviour. So nothing is
> imposed; the default simply tracks the machine instead of a constant, and
> a wide array on a large host no longer needs a recompile or manual
> per-array tuning to use the memory and cores it has.
>
> And on performance -- on real NVMe this time, not a ramdisk: on a 16-disk
> raid6 array (a 32-vCPU / 16-core host, steady state, interleaved runs) the
> hardware-derived defaults run 2.1-3.2x the stock defaults (4K random write
> ~39k -> ~100k IOPS), essentially all of it from patch 6 turning on the
> worker groups.  Patches 2-5 (the cache and lock sizing) are
> throughput-neutral on real SSD, as you'd expect -- they earn their place as
> configurability and sane defaults, not a speed claim.  And there is no
> regression at the small end: a 4-CPU box derives 2 workers and is never
> slower on any workload, and a box with two or fewer CPUs derives 0 and is
> byte-for-byte unchanged.
>
> It has been through KASAN + lockdep + DEBUG_LIST on RAID5
> (create/rebuild/scrub, plus the bitmap add/remove that drives the
> lock-all-hash-locks quiesce path), at both the derived defaults and pinned
> values including nr_stripe_hash_locks=32.
>
> The two unrelated parts of the original series -- the type-widening /
> correctness fixes and the resync/recovery dispatch changes -- I'll send as
> their own small series, as discussed.
>
> Thanks,
>
> 2026年7月5日(日) 19:56 yu kuai <yukuai@fygo.io>:
> >
> > Hi,
> >
> > 在 2026/6/24 23:54, Hiroshi Nishida 写道:
> > > This series collects small, individually low-risk md/raid5 changes for
> > > large, many-core, many-disk arrays.  Their common theme is reducing
> > > per-stripe and stripe-cache contention, so the benefit appears mainly
> > > when the raid5 stripe-handling worker threads are in use
> > > (group_thread_cnt > 0); at the default group_thread_cnt = 0 (a single
> > > handling thread) the series is essentially neutral.
> > >
> > >   - patches 1-3 remove signed arithmetic from a hot-path divisor, lift an
> > >     arbitrary stripe-cache size cap, and widen a badblock length argument
> > >     that currently truncates large ranges;
> > >   - patch 4 raises NR_STRIPE_HASH_LOCKS (8 -> 32) to spread stripe-hash
> > >     contention on high core-count systems;
> > >   - patches 5 and 8 reduce per-stripe overhead in the resync/recovery
> > >     path and bound the share of the stripe cache a rebuild may hold while
> > >     user I/O is competing;
> > >   - patch 6 allocates each worker group's array on its own NUMA node;
> > >   - patch 7 raises MAX_STRIPE_BATCH (8 -> 32).
> > >
> > > Measured effect, treatment vs baseline, % change in mean IOPS (N=3),
> > > swept over group_thread_cnt (RAID6 4+2, 22-core host, ramdisk members):
> >
> > Testing with ramdisk does serve as a useful reference, but it does not reflect
> > real world usage.
> >
> > >
> > >    workload                       gtc=0   gtc=2   gtc=4   gtc=8
> > >    random 4K write (RMW)          +4.2%   +8.1%  +17.4%   +6.5%
> > >    DB mixed 75/25 8K              +0.4%   +4.2%  +10.3%   +4.7%
> > >    high-concurrency 70/30 4K      +3.9%   +1.2%  +10.0%   +0.2%
> > >    OLTP 70/30 16K                 -0.3%   +4.7%  +10.1%   +9.3%
> > >    partial-stripe write 8K        +1.1%   +4.8%  +11.2%  +14.2%
> >
> > With a quick review I saw many static configurations is changed, I agree
> > these changes can improve arrays with ssd/nvme and a system with large
> > memory available. However, we already tested with hdd and about 8G memory
> > available, these changes will not improve performance at all, with the
> > extra memory overhead.
> >
> > I can accept make those values configurable, but not direct modifications.
> > As validation is required for numerous scenarios. Memory resources are precious
> > especially for most consumer NAS devices.
> >
> > >
> > > At the default single handling thread (group_thread_cnt = 0) the series is
> > > neutral (no regression).  As worker threads are added the gain grows,
> > > peaking broadly around group_thread_cnt = 4 at roughly +10-17% across the
> > > whole mix; at gtc = 8 the write-heavy workloads keep gaining while the
> > > read-heavy high-concurrency case has saturated.  (Per-run cv was <1%
> > > except the random-write test, ~5-9%, from a cold first run.)
> > >
> > > These numbers are on a ramdisk, which removes device latency and so
> > > overstates the CPU-side contention effect relative to a real device;
> > > they show the direction and the group_thread_cnt dependence, not an
> > > absolute speedup.  The stripe-hash/batch patches (4, 7) and the cache cap
> > > (2) drive this; patch 6 only matters on multi-socket systems (not
> > > exercised above) and patches 5/8 act on the resync/recovery path rather
> > > than this steady-state workload.
> > >
> > > Reproduction (stock mdadm + fio):
> > >    mdadm --create /dev/md0 --level=6 --raid-devices=6 --chunk=512 \
> > >          --assume-clean <6 members>
> > >    echo 16384 > /sys/block/md0/md/stripe_cache_size
> > >    echo N     > /sys/block/md0/md/group_thread_cnt      # N = 0,2,4,8
> > >    fio --filename=/dev/md0 --direct=1 --ioengine=libaio --group_reporting \
> > >        --time_based --runtime=15 --name=w <per-workload opts>:
> > >      random write   : --rw=randwrite              --bs=4k  --numjobs=4  --iodepth=32
> > >      DB mixed       : --rw=randrw --rwmixread=75  --bs=8k  --numjobs=8  --iodepth=16
> > >      high-concur.   : --rw=randrw --rwmixread=70  --bs=4k  --numjobs=16 --iodepth=8
> > >      OLTP           : --rw=randrw --rwmixread=70  --bs=16k --numjobs=6  --iodepth=16
> > >      partial-stripe : --rw=randwrite              --bs=8k  --numjobs=4  --iodepth=32
> > >
> > > Each patch stands on its own; I am happy to drop or defer any that is not
> > > justified on its own merit.
> > >
> > > Functional testing on RAID5 and RAID6: create, fail a member, rebuild
> > > onto a spare / re-add, full data read-back verified, and scrub
> > > ("check") reporting mismatch_cnt == 0.  The series was also exercised
> > > with KASAN and lockdep enabled -- including heavy group_thread_cnt
> > > churn on a multi-node setup to stress the per-NUMA-node worker
> > > allocation and the raid5_quiesce hash-lock-all path -- with no reports.
> > >
> > > Hiroshi Nishida (8):
> > >    md: change chunk_sectors and stripe cache counts to unsigned int
> > >    md/raid5: raise stripe cache limit from 32768 to 262144
> > >    md: widen badblock sectors param from int to sector_t
> > >    md/raid5: raise NR_STRIPE_HASH_LOCKS from 8 to 32
> > >    md/raid5: submit a window of stripes during resync/recovery
> > >    md/raid5: allocate worker groups per NUMA node
> > >    md/raid5: raise MAX_STRIPE_BATCH from 8 to 32
> > >    md/raid5: reserve stripe cache for user I/O during rebuild
> > >
> > >   drivers/md/md.c    |   4 +-
> > >   drivers/md/md.h    |  10 ++--
> > >   drivers/md/raid5.c | 129 ++++++++++++++++++++++++++++++++-------------
> > >   drivers/md/raid5.h |  33 ++++++++----
> > >   4 files changed, 121 insertions(+), 55 deletions(-)
> > >
> > > base-commit: 55b77337bdd088c77461588e5ec094421b89911b
> > >
> > --
> > Thanks,
> > Kuai
>
>
>
> --
> Hiroshi Nishida
> nishidafmly@gmail.com



-- 
Hiroshi Nishida
nishidafmly@gmail.com