mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* [PATCH v3 00/11] scsi: mpt3sas: Fix delayed not-responding block deadlock and related bugs
@ 2026-11-18 23:09 Jeff Layton
  2026-11-18 23:09 ` [PATCH v3 01/11] scsi: mpt3sas: Don't leave requeued firmware events pending at teardown Jeff Layton
                   ` (11 more replies)
  0 siblings, 12 replies; 13+ messages in thread
From: Jeff Layton @ 2026-11-18 23:09 UTC (permalink / raw)
  To: Sathya Prakash, Sreekanth Reddy, Suganath Prabu Subramani,
	Ranjan Kumar, James E.J. Bottomley, Martin K. Petersen,
	Nagalakshmi Nandigama, James Bottomley
  Cc: Keith Busch, Sreekanth Reddy, MPT-FusionLinux.pdl, linux-scsi,
	linux-kernel, Jeff Layton

I posted v2 some time ago. Since then, I did a lot of adversarial review
with AI agents. The review found a large number of bugs. Many of these
bugs were preexisting.

This series fixes a deadlock that occurs regularly in Meta's fleet. The
fix unblocks sdevs after a timeout, and this breaks the deadlock.

I don't do much SCSI driver work, but this problem came to me. I
relied heavily on AI agents for this series.

If this series is acceptable, I would like to see it in linux-next
soon.

Thanks,
Jeff

Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
Changes in v3:
- New patches 1-8 and 10 fix pre-existing bugs in the code that this
  series changes:
  - 1-2: fix UAFs and refcounting in the cleanup of the fw event queue.
    Do not leave requeued events pending at teardown.
  - 3: requeue topology events only for busy devices. Do not requeue
    them for devices that the driver cannot add.
  - 4-7: fix these TUR polling bugs:
    - a pcie_device leak
    - a lost unblock retry
    - polling that continues after the scan is cancelled
    - a retry count that is not per device
  - 8: cache the missing delays that the IOC actually uses
  - 10: do not use freed hostdata while the driver polls devices
- Patch 9: also publish sdev->hostdata under ioc->block_lock in
  scsih_sdev_init()
- Patch 11:
  - Use the PCIe missing delay (from PCIe IO Unit Page 1) for NVMe
    devices.
  - Arm dnr_unblock_work for the earliest deadline.
  - Skip the timed unblock during host reset, but not during SCSI EH.
  - Clear the deadline of an sdev under ioc->block_lock before the fw
    event worker polls the sdev.
  - Do not put a timed block on offline sdevs.
  - When a hard reset cancels polling, give new deadlines to the sdevs
    that stay blocked.
  - Give deadlines to the sdevs that a broadcast primitive pass blocks.
    Prevent dnr_unblock_work from unblocking them while the pass runs.
  - A repeat DNR restarts the deadline, at most
    MPT3SAS_DNR_BLOCK_MAX_EXTENDS times.
  - At the next worker unblock, do a TUR check on expired sdevs.
  - Keep the blocks that the driver makes during polling.
  - Use system_dfl_wq instead of system_wq.
- Link to v2: https://lore.kernel.org/r/20261102-sas-abba-v2-0-b3b9ba874b5a@kernel.org

Changes in v2:
- New patch 1: use a new ioc->block_lock to serialize the
  interrupt-time sdev block with the hostdata teardown in
  scsih_sdev_destroy(). This fixes a pre-existing UAF.
- Do the timed unblock under ioc->block_lock. This prevents a race with
  an interrupt-time block.
- Clear block_deadline before the fw event worker polls a returning
  device with TURs. Then dnr_unblock_work cannot unblock the device
  during the poll.
- Link to v1: https://lore.kernel.org/r/20261030-sas-abba-v1-1-b8cdd714a37c@kernel.org

---
Jeff Layton (11):
      scsi: mpt3sas: Don't leave requeued firmware events pending at teardown
      scsi: mpt3sas: Hold event references while cleaning up the event queue
      scsi: mpt3sas: Requeue topology events only for busy devices
      scsi: mpt3sas: Fix pcie_device leak in _scsih_ublock_io_device_wait()
      scsi: mpt3sas: Restore unblock retry in _scsih_ublock_io_device_wait()
      scsi: mpt3sas: Stop polling devices when the unresponding scan is cancelled
      scsi: mpt3sas: Reset the TUR retry count for each device
      scsi: mpt3sas: Cache the missing delays the IOC actually uses
      scsi: mpt3sas: Serialize interrupt-time sdev block with hostdata teardown
      scsi: mpt3sas: Don't use freed hostdata while polling devices
      scsi: mpt3sas: Time out delayed not-responding device blocks

 drivers/scsi/mpt3sas/mpt3sas_base.c  |  62 ++-
 drivers/scsi/mpt3sas/mpt3sas_base.h  |  25 +
 drivers/scsi/mpt3sas/mpt3sas_scsih.c | 869 +++++++++++++++++++++++++++++------
 3 files changed, 810 insertions(+), 146 deletions(-)
---
base-commit: e8017d29067fd1310a65c3494ce1a1b492adbac6
change-id: 20261030-sas-abba-eacfcc45fe36

Best regards,
-- 
Jeff Layton <jlayton@kernel.org>


^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2026-11-18 23:13 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-18 23:09 [PATCH v3 00/11] scsi: mpt3sas: Fix delayed not-responding block deadlock and related bugs Jeff Layton
2026-11-18 23:09 ` [PATCH v3 01/11] scsi: mpt3sas: Don't leave requeued firmware events pending at teardown Jeff Layton
2026-11-18 23:09 ` [PATCH v3 02/11] scsi: mpt3sas: Hold event references while cleaning up the event queue Jeff Layton
2026-11-18 23:10 ` [PATCH v3 03/11] scsi: mpt3sas: Requeue topology events only for busy devices Jeff Layton
2026-11-18 23:10 ` [PATCH v3 04/11] scsi: mpt3sas: Fix pcie_device leak in _scsih_ublock_io_device_wait() Jeff Layton
2026-11-18 23:10 ` [PATCH v3 05/11] scsi: mpt3sas: Restore unblock retry " Jeff Layton
2026-11-18 23:10 ` [PATCH v3 06/11] scsi: mpt3sas: Stop polling devices when the unresponding scan is cancelled Jeff Layton
2026-11-18 23:10 ` [PATCH v3 07/11] scsi: mpt3sas: Reset the TUR retry count for each device Jeff Layton
2026-11-18 23:10 ` [PATCH v3 08/11] scsi: mpt3sas: Cache the missing delays the IOC actually uses Jeff Layton
2026-11-18 23:10 ` [PATCH v3 09/11] scsi: mpt3sas: Serialize interrupt-time sdev block with hostdata teardown Jeff Layton
2026-11-18 23:10 ` [PATCH v3 10/11] scsi: mpt3sas: Don't use freed hostdata while polling devices Jeff Layton
2026-11-18 23:10 ` [PATCH v3 11/11] scsi: mpt3sas: Time out delayed not-responding device blocks Jeff Layton
2026-11-18 23:13 ` [PATCH v3 00/11] scsi: mpt3sas: Fix delayed not-responding block deadlock and related bugs Jeff Layton

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®