mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Jeff Layton <jlayton@kernel.org>
To: Sathya Prakash <sathya.prakash@broadcom.com>,
	 Sreekanth Reddy <sreekanth.reddy@broadcom.com>,
	 Suganath Prabu Subramani <suganath-prabu.subramani@broadcom.com>,
	 Ranjan Kumar <ranjan.kumar@broadcom.com>,
	 "James E.J. Bottomley" <James.Bottomley@HansenPartnership.com>,
	 "Martin K. Petersen" <mkp@kernel.org>,
	 Nagalakshmi Nandigama <Nagalakshmi.Nandigama@lsi.com>,
	 James Bottomley <JBottomley@Parallels.com>
Cc: Keith Busch <kbusch@kernel.org>,
	 Sreekanth Reddy <Sreekanth.Reddy@lsi.com>,
	MPT-FusionLinux.pdl@broadcom.com,  linux-scsi@vger.kernel.org,
	linux-kernel@vger.kernel.org,  Jeff Layton <jlayton@kernel.org>
Subject: [PATCH v3 01/11] scsi: mpt3sas: Don't leave requeued firmware events pending at teardown
Date: Wed, 18 Nov 2026 15:09:58 -0800	[thread overview]
Message-ID: <20261118-sas-abba-v3-1-5562d690cf61@kernel.org> (raw)
In-Reply-To: <20261118-sas-abba-v3-0-5562d690cf61@kernel.org>

When a topology change event finds a busy device,
_scsih_fw_event_requeue() retries the event. It queues the event's
delayed_work on firmware_event_thread, with a one-second delay.
_scsih_fw_event_cleanup_queue() cancels only the event's work, not its
delayed_work. Thus the timer stays armed. Then scsih_remove() destroys
the workqueue. After the workqueue is freed, the timer queues
_firmware_event_work_delayed() on it:

  BUG: KASAN: slab-use-after-free in __queue_work+0xa12/0xb50
  Read of size 4 at addr ffff88814fefd1c0 by task swapper/10/0
   __queue_work
   call_timer_fn
   __run_timer_base
   ...
  Allocated by task 11:
   __alloc_workqueue
   alloc_workqueue_noprof
   _scsih_probe [mpt3sas]
  Freed by task 0:
   kfree
   rcu_core
  ...
  workqueue: cannot queue _firmware_event_work_delayed [mpt3sas] on wq fw_event_mpt3sas0

The cleanup does not wait for a running topology change event. If such
an event finds a busy device, it requeues itself. This can occur after
the cleanup passes the event.

Fix this:

 - In the cleanup, cancel the delayed_work of a requeued event.

 - Do not requeue an event while the cleanup runs or while the driver
   removes the host. Do this check under fw_event_lock, which the
   cleanup also takes.

 - Instead of a requeue, drop the reference that the work held.
   Previously, this reference leaked when firmware_event_thread was
   NULL.

 - Also drop the reference if the delayed_work cannot be queued.
   Cleanups can overlap. Thus one cleanup can clear fw_events_cleanup
   while another cleanup cancels the delayed_work. The cancel disables
   the delayed_work.

 - Initialize the delayed_work when the driver allocates the event, not
   at the first requeue. Thus a cleanup never cancels the delayed_work
   before it is initialized.

Fixes: ad5957193107 ("scsi: mpt3sas: Add firmware event requeue support for busy devices")
Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
 drivers/scsi/mpt3sas/mpt3sas_scsih.c | 49 ++++++++++++++++++++++++++++--------
 1 file changed, 38 insertions(+), 11 deletions(-)

diff --git a/drivers/scsi/mpt3sas/mpt3sas_scsih.c b/drivers/scsi/mpt3sas/mpt3sas_scsih.c
index 04255286a2a6..545913123d01 100644
--- a/drivers/scsi/mpt3sas/mpt3sas_scsih.c
+++ b/drivers/scsi/mpt3sas/mpt3sas_scsih.c
@@ -320,6 +320,11 @@ static struct fw_event_work *alloc_fw_event_work(int len)
 		return NULL;
 
 	kref_init(&fw_event->refcount);
+	/*
+	 * Initialize the delayed_work up front: a cleanup may cancel it
+	 * while the event is being requeued.
+	 */
+	INIT_DELAYED_WORK(&fw_event->delayed_work, _firmware_event_work_delayed);
 	return fw_event;
 }
 
@@ -3696,6 +3701,9 @@ _scsih_fw_event_del_from_list(struct MPT3SAS_ADAPTER *ioc, struct fw_event_work
  *
  * Context: This function will acquire ioc->fw_event_lock.
  *
+ * The reference held by the work that ran @fw_event passes to the delayed
+ * work.  If the event can't be requeued, that reference is dropped.
+ *
  * Return nothing.
  */
 static void
@@ -3704,20 +3712,31 @@ _scsih_fw_event_requeue(struct MPT3SAS_ADAPTER *ioc, struct fw_event_work
 {
 	unsigned long flags;
 
-	if (ioc->firmware_event_thread == NULL)
-		return;
-
 	spin_lock_irqsave(&ioc->fw_event_lock, flags);
+	/*
+	 * Don't rearm the event while the queue is being flushed or the
+	 * host is going away.  _scsih_fw_event_cleanup_queue() may already
+	 * be past it, and the timer could then fire after the workqueue
+	 * has been destroyed.
+	 */
+	if (!ioc->firmware_event_thread || ioc->remove_host ||
+	    ioc->fw_events_cleanup)
+		goto drop;
+	fw_event->delayed_work_active = 1;
+	/*
+	 * Cleanups can overlap, and one can clear fw_events_cleanup while
+	 * another is cancelling, and so disabling, the delayed work.
+	 */
+	if (!queue_delayed_work(ioc->firmware_event_thread,
+	    &fw_event->delayed_work, msecs_to_jiffies(delay)))
+		goto drop;
 	fw_event_work_get(fw_event);
 	list_add_tail(&fw_event->list, &ioc->fw_event_list);
-	if (!fw_event->delayed_work_active) {
-		fw_event->delayed_work_active = 1;
-		INIT_DELAYED_WORK(&fw_event->delayed_work,
-		    _firmware_event_work_delayed);
-	}
-	queue_delayed_work(ioc->firmware_event_thread, &fw_event->delayed_work,
-	    msecs_to_jiffies(delay));
 	spin_unlock_irqrestore(&ioc->fw_event_lock, flags);
+	return;
+drop:
+	spin_unlock_irqrestore(&ioc->fw_event_lock, flags);
+	fw_event_work_put(fw_event);
 }
 
  /**
@@ -3869,9 +3888,17 @@ _scsih_fw_event_cleanup_queue(struct MPT3SAS_ADAPTER *ioc)
 		 *
 		 * If it did execute, we wait for it to finish, and the put will
 		 * happen from _firmware_event_work()
+		 *
+		 * An event requeued by _scsih_fw_event_requeue() is pending
+		 * on its delayed_work instead, and its timer must not fire
+		 * after the workqueue is destroyed.
 		 */
-		if (cancel_work_sync(&fw_event->work))
+		if (fw_event->delayed_work_active) {
+			if (cancel_delayed_work_sync(&fw_event->delayed_work))
+				fw_event_work_put(fw_event);
+		} else if (cancel_work_sync(&fw_event->work)) {
 			fw_event_work_put(fw_event);
+		}
 
 	}
 	ioc->fw_events_cleanup = 0;

-- 
2.56.1


  reply	other threads:[~2026-11-18 23:10 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-18 23:09 [PATCH v3 00/11] scsi: mpt3sas: Fix delayed not-responding block deadlock and related bugs Jeff Layton
2026-11-18 23:09 ` Jeff Layton [this message]
2026-11-18 23:09 ` [PATCH v3 02/11] scsi: mpt3sas: Hold event references while cleaning up the event queue Jeff Layton
2026-11-18 23:10 ` [PATCH v3 03/11] scsi: mpt3sas: Requeue topology events only for busy devices Jeff Layton
2026-11-18 23:10 ` [PATCH v3 04/11] scsi: mpt3sas: Fix pcie_device leak in _scsih_ublock_io_device_wait() Jeff Layton
2026-11-18 23:10 ` [PATCH v3 05/11] scsi: mpt3sas: Restore unblock retry " Jeff Layton
2026-11-18 23:10 ` [PATCH v3 06/11] scsi: mpt3sas: Stop polling devices when the unresponding scan is cancelled Jeff Layton
2026-11-18 23:10 ` [PATCH v3 07/11] scsi: mpt3sas: Reset the TUR retry count for each device Jeff Layton
2026-11-18 23:10 ` [PATCH v3 08/11] scsi: mpt3sas: Cache the missing delays the IOC actually uses Jeff Layton
2026-11-18 23:10 ` [PATCH v3 09/11] scsi: mpt3sas: Serialize interrupt-time sdev block with hostdata teardown Jeff Layton
2026-11-18 23:10 ` [PATCH v3 10/11] scsi: mpt3sas: Don't use freed hostdata while polling devices Jeff Layton
2026-11-18 23:10 ` [PATCH v3 11/11] scsi: mpt3sas: Time out delayed not-responding device blocks Jeff Layton
2026-11-18 23:13 ` [PATCH v3 00/11] scsi: mpt3sas: Fix delayed not-responding block deadlock and related bugs Jeff Layton

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261118-sas-abba-v3-1-5562d690cf61@kernel.org \
    --to=jlayton@kernel.org \
    --cc=JBottomley@Parallels.com \
    --cc=James.Bottomley@HansenPartnership.com \
    --cc=MPT-FusionLinux.pdl@broadcom.com \
    --cc=Nagalakshmi.Nandigama@lsi.com \
    --cc=Sreekanth.Reddy@lsi.com \
    --cc=kbusch@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-scsi@vger.kernel.org \
    --cc=mkp@kernel.org \
    --cc=ranjan.kumar@broadcom.com \
    --cc=sathya.prakash@broadcom.com \
    --cc=sreekanth.reddy@broadcom.com \
    --cc=suganath-prabu.subramani@broadcom.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®