mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Yu Kuai <yukuai1@huaweicloud.com>
To: Xiao Ni <xni@redhat.com>, Yu Kuai <yukuai1@huaweicloud.com>,
	mpatocka@redhat.com, heinzm@redhat.com,
	blazej.kucman@linux.intel.com, agk@redhat.com,
	snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org,
	jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com,
	akpm@osdl.org
Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org,
	yi.zhang@huawei.com, yangerkun@huawei.com,
	"yukuai (C)" <yukuai3@huawei.com>
Subject: Re: [PATCH v5 05/14] md: don't suspend the array for interrupted reshape
Date: Thu, 29 Feb 2024 10:14:14 +0800	[thread overview]
Message-ID: <0f768b3d-d0a2-92ff-e854-74acc9d0ad7a@huaweicloud.com> (raw)
In-Reply-To: <60d75867-8fb7-4c67-96f7-3e5ba65bdbd9@redhat.com>

Hi,

在 2024/02/29 10:10, Xiao Ni 写道:
> 
> 在 2024/2/1 下午5:25, Yu Kuai 写道:
>> From: Yu Kuai <yukuai3@huawei.com>
>>
>> md_start_sync() will suspend the array if there are spares that can be
>> added or removed from conf, however, if reshape is still in progress,
> 
> 
> Hi Kuai
> 
> Why md_start_sync can run when reshape is still in progress? 
> md_check_recovery should return without queue sync_work, right?
> 
>> this won't happen at all or data will be corrupted(remove_and_add_spares
>> won't be called from md_choose_sync_action for reshape), hence there is
>> no need to suspend the array if reshape is not done yet.
>>
>> Meanwhile, there is a potential deadlock for raid456:
>>
>> 1) reshape is interrupted;
>>
>> 2) set one of the disk WantReplacement, and add a new disk to the array,
>>     however, recovery won't start until the reshape is finished;
>>
>> 3) then issue an IO across reshpae position, this IO will wait for
>>     reshape to make progress;
>>
>> 4) continue to reshape, then md_start_sync() found there is a spare disk
>>     that can be added to conf, mddev_suspend() is called;
> 
> 
> I c. The answer for my above question is reshape is interrupted and then 
> it continues the reshape, right?
> 

Yes, reshape is interrupted and sync_thread is unregister, then
sync_thread can be registered again to continue reshape.

Thanks,
Kuai

> 
> Best Regards
> 
> Xiao
> 
>>
>> Step 4 and step 3 is waiting for each other, deadlock triggered. Noted
>> this problem is found by code review, and it's not reporduced yet.
>>
>> Fix this porblem by don't suspend the array for interrupted reshape,
>> this is safe because conf won't be changed until reshape is done.
>>
>> Fixes: bc08041b32ab ("md: suspend array in md_start_sync() if array 
>> need reconfiguration")
>> Signed-off-by: Yu Kuai <yukuai3@huawei.com>
>> ---
>>   drivers/md/md.c | 13 +++++++++----
>>   1 file changed, 9 insertions(+), 4 deletions(-)
>>
>> diff --git a/drivers/md/md.c b/drivers/md/md.c
>> index 6c5d0a372927..85fde05c37dd 100644
>> --- a/drivers/md/md.c
>> +++ b/drivers/md/md.c
>> @@ -9374,12 +9374,17 @@ static void md_start_sync(struct work_struct *ws)
>>       bool suspend = false;
>>       char *name;
>> -    if (md_spares_need_change(mddev))
>> +    /*
>> +     * If reshape is still in progress, spares won't be added or removed
>> +     * from conf until reshape is done.
>> +     */
>> +    if (mddev->reshape_position == MaxSector &&
>> +        md_spares_need_change(mddev)) {
>>           suspend = true;
>> +        mddev_suspend(mddev, false);
>> +    }
>> -    suspend ? mddev_suspend_and_lock_nointr(mddev) :
>> -          mddev_lock_nointr(mddev);
>> -
>> +    mddev_lock_nointr(mddev);
>>       if (!md_is_rdwr(mddev)) {
>>           /*
>>            * On a read-only array we can:
> 
> .
> 


  reply	other threads:[~2024-02-29  2:14 UTC|newest]

Thread overview: 66+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-02-01  9:25 [PATCH v5 00/14] dm-raid/md/raid: fix v6.7 regressions Yu Kuai
2024-02-01  9:25 ` [PATCH v5 01/14] md: don't ignore suspended array in md_check_recovery() Yu Kuai
2024-02-16  6:58   ` Xiao Ni
2024-02-18  1:14     ` Yu Kuai
2024-02-18  1:33       ` Xiao Ni
2024-02-18  1:46         ` Yu Kuai
2024-02-18  2:27           ` Xiao Ni
2024-02-18  2:34             ` Yu Kuai
2024-02-18  3:15               ` Xiao Ni
2024-02-18  3:24                 ` Yu Kuai
2024-02-18  5:07                   ` Xiao Ni
2024-02-18  6:22                     ` Yu Kuai
2024-02-18  8:07                       ` Xiao Ni
2024-02-18  8:47                         ` Yu Kuai
2024-02-19  7:10                           ` Xiao Ni
2024-02-19  8:19                             ` Yu Kuai
2024-02-01  9:25 ` [PATCH v5 02/14] md: don't ignore read-only " Yu Kuai
2024-02-01  9:25 ` [PATCH v5 03/14] md: make sure md_do_sync() will set MD_RECOVERY_DONE Yu Kuai
2024-02-18  5:56   ` Xiao Ni
2024-02-18  6:51     ` Yu Kuai
2024-02-18  8:41       ` Xiao Ni
2024-02-18  8:59         ` Yu Kuai
2024-02-01  9:25 ` [PATCH v5 04/14] md: don't register sync_thread for reshape directly Yu Kuai
2024-02-28 12:07   ` Xiao Ni
2024-02-28 12:44     ` Yu Kuai
2024-02-28 12:57       ` Xiao Ni
2024-02-01  9:25 ` [PATCH v5 05/14] md: don't suspend the array for interrupted reshape Yu Kuai
2024-02-29  2:10   ` Xiao Ni
2024-02-29  2:14     ` Yu Kuai [this message]
2024-02-01  9:25 ` [PATCH v5 06/14] md: fix missing release of 'active_io' for flush Yu Kuai
2024-02-08  7:47   ` Song Liu
2024-02-01  9:25 ` [PATCH v5 07/14] md: export helpers to stop sync_thread Yu Kuai
2024-02-15 22:27   ` Song Liu
2024-02-18  2:35     ` Yu Kuai
2024-02-01  9:25 ` [PATCH v5 08/14] md: export helper md_is_rdwr() Yu Kuai
2024-02-01  9:25 ` [PATCH v5 09/14] dm-raid: really frozen sync_thread during suspend Yu Kuai
2024-02-18  4:53   ` Xiao Ni
2024-02-18  6:34     ` Yu Kuai
2024-02-19  7:27       ` Xiao Ni
2024-02-19  7:53         ` Yu Kuai
2024-02-19  8:45           ` Xiao Ni
2024-02-01  9:25 ` [PATCH v5 10/14] md/dm-raid: don't call md_reap_sync_thread() directly Yu Kuai
2024-02-01  9:25 ` [PATCH v5 11/14] dm-raid: add a new helper prepare_suspend() in md_personality Yu Kuai
2024-02-01  9:25 ` [PATCH v5 12/14] md/raid456: fix a deadlock for dm-raid456 while io concurrent with reshape Yu Kuai
2024-02-01  9:25 ` [PATCH v5 13/14] dm-raid: fix lockdep waring in "pers->hot_add_disk" Yu Kuai
2024-02-01  9:25 ` [PATCH v5 14/14] dm-raid: remove mddev_suspend/resume() Yu Kuai
2024-02-03  3:19 ` [PATCH v5 00/14] dm-raid/md/raid: fix v6.7 regressions Benjamin Marzinski
2024-02-04  1:35   ` Yu Kuai
2024-02-04  7:00     ` Yu Kuai
2024-02-05 19:35     ` Benjamin Marzinski
2024-02-06  1:36       ` Yu Kuai
2024-02-06  3:57         ` Benjamin Marzinski
2024-02-06  7:03           ` Yu Kuai
2024-02-08  8:04           ` Song Liu
2024-02-08 23:17             ` Benjamin Marzinski
2024-02-09 22:37               ` Song Liu
2024-02-26  7:58                 ` Su Yue
2024-02-12 22:30 ` Song Liu
2024-02-15 22:24 ` Song Liu
2024-02-16  5:46   ` Benjamin Marzinski
2024-02-18  1:24     ` Yu Kuai
2024-02-19 16:05       ` Benjamin Marzinski
2024-02-20  3:09         ` Yu Kuai
2024-02-22  9:00     ` Yu Kuai
2024-02-28  1:19       ` Benjamin Marzinski
2024-02-28  1:35         ` Yu Kuai

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=0f768b3d-d0a2-92ff-e854-74acc9d0ad7a@huaweicloud.com \
    --to=yukuai1@huaweicloud.com \
    --cc=agk@redhat.com \
    --cc=akpm@osdl.org \
    --cc=blazej.kucman@linux.intel.com \
    --cc=dm-devel@lists.linux.dev \
    --cc=heinzm@redhat.com \
    --cc=jbrassow@f14.redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-raid@vger.kernel.org \
    --cc=mpatocka@redhat.com \
    --cc=neilb@suse.de \
    --cc=shli@fb.com \
    --cc=snitzer@kernel.org \
    --cc=song@kernel.org \
    --cc=xni@redhat.com \
    --cc=yangerkun@huawei.com \
    --cc=yi.zhang@huawei.com \
    --cc=yukuai3@huawei.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®