From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 79D343DBD70 for ; Wed, 24 Jun 2026 15:55:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.179 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782316512; cv=none; b=ppfI2YC5ppTN8/35B4agBgQm1v9SeeVnaARjW3UNAPKqFO3NsiXmbSHx2hJoPwMLLpS9/g27D/qoec/owdZiCB3LshzTu1qzm9n17gaDlXNbiyXnzAywVY1myFMengFZYPkWW6AO4yoHzINOS5kK2yqTe0siazRfl1t43fM1Q2M= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782316512; c=relaxed/simple; bh=Dzy0IoQlIyGAFNK5afExtcTtJOLw+uEl9h4DwaR/O0g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=QOncsGy3MldxcOi1cCmu4JUwf1qIjZzlAA9C5QNQiXUEAtOp8LSfsuPMEWPPvdYMqBqNnEfTAWS/UJ2F/n/d750zEaZjfdF1lLnWAqtZiotwPtH0K0w8L7LQz+Id0Y87S2PFnHtNREkbioHi8lcV0uVVn3INTt6ZKDmadf8UK/w= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=bTPxXeWN; arc=none smtp.client-ip=209.85.210.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="bTPxXeWN" Received: by mail-pf1-f179.google.com with SMTP id d2e1a72fcca58-8423f236418so823521b3a.1 for ; Wed, 24 Jun 2026 08:55:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1782316511; x=1782921311; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=T3Vs0T5e7mY2+mqGN7mTSC6YjD5wSRzVPb5tvSf7gxU=; b=bTPxXeWNZiJ+bcexWT6Sz3LqZMe2EQ44Mt+8dysLonmzRWAqkKWp4wViCnPwkEB/QP lZKMEKA5wFNEl0LzBsluGyM2n75kF+mqUkcSpoxYNs52snaO2vZ220M9GGXMnlJ2JHbJ Mz3gjBCf7CVyM96HkeaioisCPbBbIcgrDVT7GQ9PhNF85ZGfWRQi77zNYIxipjKIzelG +WCFrHucAK946bzqZ+Yi9Jfw4fpv0sGIZqZPXZHpL781GYuvkrI2JE17RIfIevJpPCHz Np0T+aSg01G7h4CY2bCCsZAvQ8UisMdI61mFT71Salvc3uushfYOxjErfce5sMHfNnfX RNiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1782316511; x=1782921311; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=T3Vs0T5e7mY2+mqGN7mTSC6YjD5wSRzVPb5tvSf7gxU=; b=bDaOsoMEqYf2Rc4G5nR6RjVibrQwgYhhANPjJiqklSqdUq1atEufLEoH763furpwwf RDBC1T8o14OE5P3ANjz7mlLi9pkv/5wuQ+7kuWkg2d9MQ+WX2jLiUbtWmAuQaFpXjcBv 4VNk05JxWeq5eiNCx/FiHQSt3zvSTrV1oEHFMUV3bpY4uw8MFB2kzE/ton1RkQ3N5HwI fJ+ecAItY+c2h5U0p/EBKhUaDrtuoqfULoCywSLzG4GL8i0EF7qn0t7hL6NCtnO2Dyw3 JZdZIdvilbxIXCFxZ44vaayiOBl/hYRhFNtFSE1jt53p3JuhmYEpMvcSwkhWqh/L0ex8 qEVg== X-Forwarded-Encrypted: i=1; AFNElJ8ouDuGDL++HZMDV3b4/eYSMptjTcvz9rAJvBiTncINf3uixW1kpC0pCz8gVt5tt0u/iT5f5tC9etBZyT4=@vger.kernel.org X-Gm-Message-State: AOJu0YyRhteW+qxRYHb56DMb0VyXlRSlQCk4Ky/OZPN8f93NrFNUQHlL I3d3oG6F44e/36Tu/9foLlQv1wt5y+nwTteDVuT5ZTnuFzy+5P1X1TI4 X-Gm-Gg: AfdE7cl+fBMdMeQFHgkAsRwIQ0Jdfi4M3NM3pjCiAdIMX0M5syxDo3KWDnWsEJRaxQo kyqndL++9WbUzn3VJ2YEZmNTu00AktAm6D2lIZ/GyQDflw15COupWkjwVpgHO0xNolv66xELrDk OEEGJdHFK6DyjVcTRlPvH2THg2gPu6UBISXuqvDTAO9AWU6Bcb39yUY3K4RbvaXx2+ZueXNp1nP WScJdG55yaEECAiIuOf5H8G8su2lC1zjB3U3c64u6+km1Ox9eD1fVTroFGEkIK8GhLLjFJs2gQJ 15tdwX48OR2ekQU4DsAACxMhPKla2xpS1SDs3M/51jxPI/Kc92Ld6aoL6zq/UbIkQg0De1Y70kz 3O1u07pLVPcYU1Egg2J127bO1/ntgL98ujWYTs9K6qRDaBgllh4v7Z/k/zxvbqAnKqcTt1kyJRi 2Lv7H76nKXvd52g7C99PooIyKmuXJZCx37P8IHmRVqpLLTVKQuQp2XDhtolPTMtjfuLOlZu7DZ7 5hcZxI= X-Received: by 2002:a05:6a00:218f:b0:845:4970:df37 with SMTP id d2e1a72fcca58-845a27d6b2bmr4948474b3a.0.1782316510775; Wed, 24 Jun 2026 08:55:10 -0700 (PDT) Received: from research02.. ([2601:1c1:8700:f5b:fe34:97ff:fea3:c147]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-845a40f55cesm2658387b3a.44.2026.06.24.08.55.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2026 08:55:10 -0700 (PDT) From: Hiroshi Nishida To: Song Liu , Yu Kuai Cc: Li Nan , Xiao Ni , linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org, Hiroshi Nishida Subject: [PATCH 4/8] md/raid5: raise NR_STRIPE_HASH_LOCKS from 8 to 32 Date: Wed, 24 Jun 2026 08:54:48 -0700 Message-ID: <20260624155452.211646-5-nishidafmly@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260624155452.211646-1-nishidafmly@gmail.com> References: <20260624155452.211646-1-nishidafmly@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit The stripe cache hash is striped across NR_STRIPE_HASH_LOCKS spinlocks (see stripe_hash_locks_hash()). The value has been 8 since the per-hash locking was introduced, which is small for modern many-core servers: with only 8 buckets, stripe cache lookup and allocation contend on the same few locks once the CPU count greatly exceeds the bucket count. Raise it to 32. Two constraints bound the choice: - STRIPE_HASH_LOCKS_MASK is (NR_STRIPE_HASH_LOCKS - 1) and is used as a bitmask in stripe_hash_locks_hash(), so the value must be a power of two. - raid5_quiesce() acquires every hash lock plus device_lock at once via lock_all_device_hash_locks_irq(). That holds NR_STRIPE_HASH_LOCKS + 1 locks simultaneously, which must stay below MAX_LOCK_DEPTH (48) so the held-lock array does not overflow when lockdep is enabled. 32 is the largest power of two that satisfies both: 32 + 1 leaves headroom under 48, whereas 64 would exceed it. (The pre-existing "must remain below 64" comment understates this; MAX_LOCK_DEPTH is the real ceiling.) STRIPE_HASH_LOCKS_MASK and all NR_STRIPE_HASH_LOCKS- sized arrays scale automatically. This is purely a lock-striping change; it does not affect stripe cache correctness. The benefit appears on many-core systems, while on small systems the extra buckets are harmless. Tested: loop-device RAID-5 create, write/verify, fail a disk, rebuild onto a spare and scrub all complete cleanly. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Hiroshi Nishida --- drivers/md/raid5.h | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/drivers/md/raid5.h b/drivers/md/raid5.h index 57349737d393..3efab71ebef7 100644 --- a/drivers/md/raid5.h +++ b/drivers/md/raid5.h @@ -498,12 +498,14 @@ struct disk_info { #define HASH_MASK (NR_HASH - 1) #define MAX_STRIPE_BATCH 8 -/* NOTE NR_STRIPE_HASH_LOCKS must remain below 64. - * This is because we sometimes take all the spinlocks - * and creating that much locking depth can cause - * problems. +/* NR_STRIPE_HASH_LOCKS must be a power of two, since + * STRIPE_HASH_LOCKS_MASK masks with (NR_STRIPE_HASH_LOCKS - 1). + * It must also be small enough that taking all of them at once in + * lock_all_device_hash_locks_irq(), plus device_lock, keeps the held + * lock count below MAX_LOCK_DEPTH (48) with lockdep enabled. 32 is the + * largest power of two that satisfies both constraints. */ -#define NR_STRIPE_HASH_LOCKS 8 +#define NR_STRIPE_HASH_LOCKS 32 #define STRIPE_HASH_LOCKS_MASK (NR_STRIPE_HASH_LOCKS - 1) struct r5worker { -- 2.43.0