mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Qiliang Yuan <qiliang.yuan@linux.dev>
To: Song Liu <song@kernel.org>, Yu Kuai <yukuai@fygo.io>,
	 Li Nan <magiclinan@didiglobal.com>, Xiao Ni <xiao@kernel.org>
Cc: linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org,
	 Qiliang Yuan <qiliang.yuan@linux.dev>
Subject: [PATCH] md: wait for a single grace period when stopping an array
Date: Sat, 21 Nov 2026 18:59:13 -0500	[thread overview]
Message-ID: <20261121-feat-md-export-array-gp-v1-1-b1925100be19@linux.dev> (raw)

When an array is stopped, export_array() takes the members off
mddev->disks one by one with md_kick_rdev_from_array(), which unlinks a
member with list_del_rcu() and calls synchronize_rcu() before handing it
to mddev_unlock() for kobject_del().

Each member thus waits for a full grace period of its own, so stopping
an array takes one grace period per member, about 6 ms each in a guest,
with reconfig_mutex held.

Release the serialization resources of all members first, while
mddev_destroy_serial_pool() still sees the members that need the pool,
then unlink all members at once with list_splice_init_rcu() and a single
synchronize_rcu(). Clear rdev->mddev and the sysfs references only after
the grace period, so a reader walking mddev->disks never finds a member
that has already been detached.

Stopping an array made of null_blk devices in a guest with 32 vCPUs,
base and patched kernels alternating over 2 rounds of 3 runs, median of
the 6 runs and range in parentheses:

  level, disks           before (ms)         after (ms)   change
  raid0, 16      132.4 (126.3-147.5)   37.7 (34.5-41.4)     -72%
  raid1, 8         88.5 (86.3-103.1)   43.7 (40.7-45.4)     -51%
  raid5, 8         98.8 (97.6-107.3)   57.7 (55.0-59.1)     -42%
  raid10, 8         87.4 (85.0-89.8)   41.7 (37.8-43.9)     -52%

Signed-off-by: Qiliang Yuan <qiliang.yuan@linux.dev>
---
 drivers/md/md.c | 39 +++++++++++++++++++++++++++++----------
 1 file changed, 29 insertions(+), 10 deletions(-)

diff --git a/drivers/md/md.c b/drivers/md/md.c
index 03e35386a8028..11fa541bce32e 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -2676,14 +2676,9 @@ static void export_rdev(struct md_rdev *rdev)
 	kobject_put(&rdev->kobj);
 }
 
-static void md_kick_rdev_from_array(struct md_rdev *rdev)
+static void md_unbind_rdev(struct md_rdev *rdev)
 {
-	struct mddev *mddev = rdev->mddev;
-
-	bd_unlink_disk_holder(rdev->bdev, rdev->mddev->gendisk);
-	list_del_rcu(&rdev->same_set);
 	pr_debug("md: unbind<%pg>\n", rdev->bdev);
-	mddev_destroy_serial_pool(rdev->mddev, rdev);
 	WRITE_ONCE(rdev->mddev, NULL);
 	sysfs_remove_link(&rdev->kobj, "block");
 	sysfs_put(rdev->sysfs_state);
@@ -2693,6 +2688,16 @@ static void md_kick_rdev_from_array(struct md_rdev *rdev)
 	rdev->sysfs_unack_badblocks = NULL;
 	rdev->sysfs_badblocks = NULL;
 	rdev->badblocks.count = 0;
+}
+
+static void md_kick_rdev_from_array(struct md_rdev *rdev)
+{
+	struct mddev *mddev = rdev->mddev;
+
+	bd_unlink_disk_holder(rdev->bdev, mddev->gendisk);
+	list_del_rcu(&rdev->same_set);
+	mddev_destroy_serial_pool(mddev, rdev);
+	md_unbind_rdev(rdev);
 
 	synchronize_rcu();
 
@@ -2707,12 +2712,26 @@ static void md_kick_rdev_from_array(struct md_rdev *rdev)
 static void export_array(struct mddev *mddev)
 {
 	struct md_rdev *rdev;
+	LIST_HEAD(kicked);
 
-	while (!list_empty(&mddev->disks)) {
-		rdev = list_first_entry(&mddev->disks, struct md_rdev,
-					same_set);
-		md_kick_rdev_from_array(rdev);
+	/*
+	 * Release the serialization resources while every member is still on
+	 * the list, as mddev_destroy_serial_pool() counts the members that
+	 * still need the pool.
+	 */
+	rdev_for_each(rdev, mddev) {
+		bd_unlink_disk_holder(rdev->bdev, mddev->gendisk);
+		mddev_destroy_serial_pool(mddev, rdev);
 	}
+
+	/* Unlink all members at once to wait for a single grace period */
+	list_splice_init_rcu(&mddev->disks, &kicked, synchronize_rcu);
+
+	list_for_each_entry(rdev, &kicked, same_set)
+		md_unbind_rdev(rdev);
+
+	/* Leave kobject_del() to mddev_unlock() */
+	list_splice(&kicked, &mddev->deleting);
 	mddev->raid_disks = 0;
 	mddev->major_version = 0;
 }

---
base-commit: 5f22fb20497f4983094131f00a734362300d22d2
change-id: 20261120-feat-md-export-array-gp-1381456062ec

Best regards,
-- 
Qiliang Yuan <qiliang.yuan@linux.dev>


                 reply	other threads:[~2026-11-21 23:59 UTC|newest]

Thread overview: [no followups] expand[flat|nested]  mbox.gz  Atom feed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261121-feat-md-export-array-gp-v1-1-b1925100be19@linux.dev \
    --to=qiliang.yuan@linux.dev \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-raid@vger.kernel.org \
    --cc=magiclinan@didiglobal.com \
    --cc=song@kernel.org \
    --cc=xiao@kernel.org \
    --cc=yukuai@fygo.io \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®