mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: yangxingui <yangxingui@huawei.com>
To: John Garry <john.g.garry@oracle.com>, <yanaijie@huawei.com>,
	<James.Bottomley@HansenPartnership.com>,
	<martin.petersen@oracle.com>
Cc: <linux-scsi@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
	<linuxarm@huawei.com>, <liuyonglong@huawei.com>,
	<kangfenglong@huawei.com>
Subject: Re: [PATCH v2] scsi: libsas: fix HA resume deadlock and hisi_sas disk-wake race
Date: Tue, 14 Jul 2026 10:15:51 +0800	[thread overview]
Message-ID: <e91cdddb-2323-904f-dc63-3d00597b5c8f@huawei.com> (raw)
In-Reply-To: <56d1c5d9-cb3f-4bef-a099-304ef0c49837@oracle.com>

Hi, John

On 2026/7/13 19:26, John Garry wrote:
> On 13/07/2026 04:10, yangxingui wrote:
>> Kindly ping for review...
>>
>> On 2026/7/2 11:32, Xingui Yang wrote:
>>> Commit fbefe22811c3140 ("scsi: libsas: Don't always drain event 
>>> workqueue
>>> for HA resume") introduced sas_resume_ha_no_sync() to avoid a 
>>> deadlock: the
>>> PHYE_RESUME_TIMEOUT handler, running on the HA event workqueue, calls
>>> sas_deform_port() -> sas_destruct_devices(), which removes SCSI 
>>> devices and
>>> waits for the host to become runtime-active. But the host cannot resume
>>> until sas_resume_ha() -> sas_drain_work() returns, and the drain is 
>>> blocked
>>> on that very handler.
>>>
>>> However skipping the drain reintroduces a race: hisi_sas returns from
>>> resume before all PHY UP work and libsas discovery work finish. The
>>> controller may then autosuspend while disks are still waking up. The 
>>> disks
>>> issue IO to a suspended controller, the IO fails, and the disks get
>>> disabled.
>>>
>>> Fix the deadlock at its source by moving the PHYE_RESUME_TIMEOUT
>>> notification to after sas_drain_work(). By then the host resume is 
>>> about to
>>> complete, so device removal through device_link no longer blocks on the
>>> resume and the cycle is broken.
>>>
>>> With the deadlock gone, restore sas_resume_ha() (the draining 
>>> variant) in
>>> hisi_sas and remove sas_resume_ha_no_sync().
>>>
>>> Fixes: fbefe22811c3140 ("scsi: libsas: Don't always drain event 
>>> workqueue for HA resume")
>>> Signed-off-by: Xingui Yang <yangxingui@huawei.com>
> 
> Any idea why this problem has only been discovered after 5 years (from 
> fbefe22811c3140 being merged)?
> 
> Is there some new test case?

Yes, the test environment is connected to more SATA disks. During the 
wake-up process, the controller wakes up first, and then the disks wake 
up one by one. Since the number of SATA disks has increased, and the 
SATA disk enter the EH (Error Handling) process when waking up, the 
wake-up time has become longer than before. It is possible that the 
controller has already entered the next suspend state, while many disks 
are still in the previous wake-up process, leading to a failure in disk 
wake-up and the disks being disabled.

However, we have tested some different kernel versions in between and 
found that the following patch significantly improves this issue, but it 
does not completely resolve it.

bede543d2f8a: ACPI: OSL: Use usleep_range() in acpi_os_sleep()

In addition, if some disks wake up a bit slowly, it may also lead to 
ssubsequent disk wake-up failure.
[  286.932540] hisi_sas_v3_hw 0000:32:04.0: resuming from operating 
state [D0]
[  287.732627] hisi_sas_v3_hw 0000:32:04.0: end of resuming controller
[  293.167893] hisi_sas_v3_hw 0000:32:04.0: entering suspend state
[  341.760789] sas: Enter sas_scsi_recover_host busy: 0 failed: 0
[  341.761006] sas: Executing internal abort failed 5000000000000307 (-22)
[  341.761021] hisi_sas_v3_hw 0000:32:04.0: I_T nexus reset: internal 
abort (-22)
[  341.761028] sas: ata4: end_device-3:3: Unable to reset ata device?
[  341.916134] sas: lldd_execute_task returned: -22
[  341.916161] ata4.00: failed to IDENTIFY (I/O error, err_mask=0x40)
[  341.916165] ata4.00: revalidation failed (errno=-5)
[  346.980523] sas: Executing internal abort failed 5000000000000307 (-22)
[  346.980542] hisi_sas_v3_hw 0000:32:04.0: I_T nexus reset: internal 
abort (-22)
[  346.980552] sas: ata4: end_device-3:3: Unable to reset ata device?
[  347.136137] sas: lldd_execute_task returned: -22
[  347.136163] ata4.00: failed to IDENTIFY (I/O error, err_mask=0x40)
[  347.136169] ata4.00: revalidation failed (errno=-5)
[  352.356924] sas: Executing internal abort failed 5000000000000307 (-22)
[  352.357001] hisi_sas_v3_hw 0000:32:04.0: I_T nexus reset: internal 
abort (-22)
[  352.357011] sas: ata4: end_device-3:3: Unable to reset ata device?
[  352.512174] sas: lldd_execute_task returned: -22
[  352.512255] ata4.00: failed to IDENTIFY (I/O error, err_mask=0x40)
[  352.512263] ata4.00: revalidation failed (errno=-5)
[  352.512269] ata4.00: disable device
[  352.512324] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0 
tries: 1
[  352.512408] sas: sas_resume_sata: for direct-attached device 
5000000000000307 returned -19


Thanks,
Xingui
.


  reply	other threads:[~2026-07-14  2:16 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-07-02  3:32 Xingui Yang
2026-07-13  3:10 ` yangxingui
2026-07-13 11:26   ` John Garry
2026-07-14  2:15     ` yangxingui [this message]
2026-07-14  8:26       ` John Garry
2026-07-14 12:06         ` yangxingui
2026-07-15  8:25           ` John Garry
2026-07-16  6:46             ` yangxingui
2026-07-15  8:23 ` John Garry
2026-07-16  6:50   ` yangxingui

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=e91cdddb-2323-904f-dc63-3d00597b5c8f@huawei.com \
    --to=yangxingui@huawei.com \
    --cc=James.Bottomley@HansenPartnership.com \
    --cc=john.g.garry@oracle.com \
    --cc=kangfenglong@huawei.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-scsi@vger.kernel.org \
    --cc=linuxarm@huawei.com \
    --cc=liuyonglong@huawei.com \
    --cc=martin.petersen@oracle.com \
    --cc=yanaijie@huawei.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox