From: Jeff Layton <jlayton@kernel.org>
To: Sathya Prakash <sathya.prakash@broadcom.com>,
Sreekanth Reddy <sreekanth.reddy@broadcom.com>,
Suganath Prabu Subramani <suganath-prabu.subramani@broadcom.com>,
Ranjan Kumar <ranjan.kumar@broadcom.com>,
"James E.J. Bottomley" <James.Bottomley@HansenPartnership.com>,
"Martin K. Petersen" <mkp@kernel.org>,
Nagalakshmi Nandigama <Nagalakshmi.Nandigama@lsi.com>,
James Bottomley <JBottomley@Parallels.com>
Cc: Keith Busch <kbusch@kernel.org>,
Sreekanth Reddy <Sreekanth.Reddy@lsi.com>,
MPT-FusionLinux.pdl@broadcom.com, linux-scsi@vger.kernel.org,
linux-kernel@vger.kernel.org, Jeff Layton <jlayton@kernel.org>
Subject: [PATCH v3 01/11] scsi: mpt3sas: Don't leave requeued firmware events pending at teardown
Date: Wed, 18 Nov 2026 15:09:58 -0800 [thread overview]
Message-ID: <20261118-sas-abba-v3-1-5562d690cf61@kernel.org> (raw)
In-Reply-To: <20261118-sas-abba-v3-0-5562d690cf61@kernel.org>
When a topology change event finds a busy device,
_scsih_fw_event_requeue() retries the event. It queues the event's
delayed_work on firmware_event_thread, with a one-second delay.
_scsih_fw_event_cleanup_queue() cancels only the event's work, not its
delayed_work. Thus the timer stays armed. Then scsih_remove() destroys
the workqueue. After the workqueue is freed, the timer queues
_firmware_event_work_delayed() on it:
BUG: KASAN: slab-use-after-free in __queue_work+0xa12/0xb50
Read of size 4 at addr ffff88814fefd1c0 by task swapper/10/0
__queue_work
call_timer_fn
__run_timer_base
...
Allocated by task 11:
__alloc_workqueue
alloc_workqueue_noprof
_scsih_probe [mpt3sas]
Freed by task 0:
kfree
rcu_core
...
workqueue: cannot queue _firmware_event_work_delayed [mpt3sas] on wq fw_event_mpt3sas0
The cleanup does not wait for a running topology change event. If such
an event finds a busy device, it requeues itself. This can occur after
the cleanup passes the event.
Fix this:
- In the cleanup, cancel the delayed_work of a requeued event.
- Do not requeue an event while the cleanup runs or while the driver
removes the host. Do this check under fw_event_lock, which the
cleanup also takes.
- Instead of a requeue, drop the reference that the work held.
Previously, this reference leaked when firmware_event_thread was
NULL.
- Also drop the reference if the delayed_work cannot be queued.
Cleanups can overlap. Thus one cleanup can clear fw_events_cleanup
while another cleanup cancels the delayed_work. The cancel disables
the delayed_work.
- Initialize the delayed_work when the driver allocates the event, not
at the first requeue. Thus a cleanup never cancels the delayed_work
before it is initialized.
Fixes: ad5957193107 ("scsi: mpt3sas: Add firmware event requeue support for busy devices")
Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
drivers/scsi/mpt3sas/mpt3sas_scsih.c | 49 ++++++++++++++++++++++++++++--------
1 file changed, 38 insertions(+), 11 deletions(-)
diff --git a/drivers/scsi/mpt3sas/mpt3sas_scsih.c b/drivers/scsi/mpt3sas/mpt3sas_scsih.c
index 04255286a2a6..545913123d01 100644
--- a/drivers/scsi/mpt3sas/mpt3sas_scsih.c
+++ b/drivers/scsi/mpt3sas/mpt3sas_scsih.c
@@ -320,6 +320,11 @@ static struct fw_event_work *alloc_fw_event_work(int len)
return NULL;
kref_init(&fw_event->refcount);
+ /*
+ * Initialize the delayed_work up front: a cleanup may cancel it
+ * while the event is being requeued.
+ */
+ INIT_DELAYED_WORK(&fw_event->delayed_work, _firmware_event_work_delayed);
return fw_event;
}
@@ -3696,6 +3701,9 @@ _scsih_fw_event_del_from_list(struct MPT3SAS_ADAPTER *ioc, struct fw_event_work
*
* Context: This function will acquire ioc->fw_event_lock.
*
+ * The reference held by the work that ran @fw_event passes to the delayed
+ * work. If the event can't be requeued, that reference is dropped.
+ *
* Return nothing.
*/
static void
@@ -3704,20 +3712,31 @@ _scsih_fw_event_requeue(struct MPT3SAS_ADAPTER *ioc, struct fw_event_work
{
unsigned long flags;
- if (ioc->firmware_event_thread == NULL)
- return;
-
spin_lock_irqsave(&ioc->fw_event_lock, flags);
+ /*
+ * Don't rearm the event while the queue is being flushed or the
+ * host is going away. _scsih_fw_event_cleanup_queue() may already
+ * be past it, and the timer could then fire after the workqueue
+ * has been destroyed.
+ */
+ if (!ioc->firmware_event_thread || ioc->remove_host ||
+ ioc->fw_events_cleanup)
+ goto drop;
+ fw_event->delayed_work_active = 1;
+ /*
+ * Cleanups can overlap, and one can clear fw_events_cleanup while
+ * another is cancelling, and so disabling, the delayed work.
+ */
+ if (!queue_delayed_work(ioc->firmware_event_thread,
+ &fw_event->delayed_work, msecs_to_jiffies(delay)))
+ goto drop;
fw_event_work_get(fw_event);
list_add_tail(&fw_event->list, &ioc->fw_event_list);
- if (!fw_event->delayed_work_active) {
- fw_event->delayed_work_active = 1;
- INIT_DELAYED_WORK(&fw_event->delayed_work,
- _firmware_event_work_delayed);
- }
- queue_delayed_work(ioc->firmware_event_thread, &fw_event->delayed_work,
- msecs_to_jiffies(delay));
spin_unlock_irqrestore(&ioc->fw_event_lock, flags);
+ return;
+drop:
+ spin_unlock_irqrestore(&ioc->fw_event_lock, flags);
+ fw_event_work_put(fw_event);
}
/**
@@ -3869,9 +3888,17 @@ _scsih_fw_event_cleanup_queue(struct MPT3SAS_ADAPTER *ioc)
*
* If it did execute, we wait for it to finish, and the put will
* happen from _firmware_event_work()
+ *
+ * An event requeued by _scsih_fw_event_requeue() is pending
+ * on its delayed_work instead, and its timer must not fire
+ * after the workqueue is destroyed.
*/
- if (cancel_work_sync(&fw_event->work))
+ if (fw_event->delayed_work_active) {
+ if (cancel_delayed_work_sync(&fw_event->delayed_work))
+ fw_event_work_put(fw_event);
+ } else if (cancel_work_sync(&fw_event->work)) {
fw_event_work_put(fw_event);
+ }
}
ioc->fw_events_cleanup = 0;
--
2.56.1
next prev parent reply other threads:[~2026-11-18 23:10 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-18 23:09 [PATCH v3 00/11] scsi: mpt3sas: Fix delayed not-responding block deadlock and related bugs Jeff Layton
2026-11-18 23:09 ` Jeff Layton [this message]
2026-11-18 23:09 ` [PATCH v3 02/11] scsi: mpt3sas: Hold event references while cleaning up the event queue Jeff Layton
2026-11-18 23:10 ` [PATCH v3 03/11] scsi: mpt3sas: Requeue topology events only for busy devices Jeff Layton
2026-11-18 23:10 ` [PATCH v3 04/11] scsi: mpt3sas: Fix pcie_device leak in _scsih_ublock_io_device_wait() Jeff Layton
2026-11-18 23:10 ` [PATCH v3 05/11] scsi: mpt3sas: Restore unblock retry " Jeff Layton
2026-11-18 23:10 ` [PATCH v3 06/11] scsi: mpt3sas: Stop polling devices when the unresponding scan is cancelled Jeff Layton
2026-11-18 23:10 ` [PATCH v3 07/11] scsi: mpt3sas: Reset the TUR retry count for each device Jeff Layton
2026-11-18 23:10 ` [PATCH v3 08/11] scsi: mpt3sas: Cache the missing delays the IOC actually uses Jeff Layton
2026-11-18 23:10 ` [PATCH v3 09/11] scsi: mpt3sas: Serialize interrupt-time sdev block with hostdata teardown Jeff Layton
2026-11-18 23:10 ` [PATCH v3 10/11] scsi: mpt3sas: Don't use freed hostdata while polling devices Jeff Layton
2026-11-18 23:10 ` [PATCH v3 11/11] scsi: mpt3sas: Time out delayed not-responding device blocks Jeff Layton
2026-11-18 23:13 ` [PATCH v3 00/11] scsi: mpt3sas: Fix delayed not-responding block deadlock and related bugs Jeff Layton
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261118-sas-abba-v3-1-5562d690cf61@kernel.org \
--to=jlayton@kernel.org \
--cc=JBottomley@Parallels.com \
--cc=James.Bottomley@HansenPartnership.com \
--cc=MPT-FusionLinux.pdl@broadcom.com \
--cc=Nagalakshmi.Nandigama@lsi.com \
--cc=Sreekanth.Reddy@lsi.com \
--cc=kbusch@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-scsi@vger.kernel.org \
--cc=mkp@kernel.org \
--cc=ranjan.kumar@broadcom.com \
--cc=sathya.prakash@broadcom.com \
--cc=sreekanth.reddy@broadcom.com \
--cc=suganath-prabu.subramani@broadcom.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®