From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org, stable@vger.kernel.org
Cc: Koby Elbaz <kelbaz@habana.ai>, Oded Gabbay <ogabbay@kernel.org>,
Sasha Levin <sashal@kernel.org>,
Arnd@vger.kernel.org, gregkh@linuxfoundation.org,
obitton@habana.ai, ttayar@habana.ai, oshpigelman@habana.ai,
osharabi@habana.ai, fkassabri@habana.ai, ynudelman@habana.ai,
mhaimovski@habana.ai, amizrahi@habana.ai
Subject: [PATCH AUTOSEL 5.14 03/21] habanalabs: fix race between soft reset and heartbeat
Date: Thu, 16 Sep 2021 22:32:57 -0400 [thread overview]
Message-ID: <20210917023315.816225-3-sashal@kernel.org> (raw)
In-Reply-To: <20210917023315.816225-1-sashal@kernel.org>
From: Koby Elbaz <kelbaz@habana.ai>
[ Upstream commit 8bb8b505761238be0d6a83dc41188867d65e5d4c ]
There is a scenario where an ongoing soft reset would race with an
ongoing heartbeat routine, eventually causing heartbeat to fail and
thus to escalate into a hard reset.
With this fix, soft-reset procedure will disable heartbeat CPU messages
and flush the (ongoing) current one before continuing with reset code.
Signed-off-by: Koby Elbaz <kelbaz@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
drivers/misc/habanalabs/common/device.c | 53 +++++++++++++++-----
drivers/misc/habanalabs/common/firmware_if.c | 18 +++++--
drivers/misc/habanalabs/common/habanalabs.h | 4 +-
drivers/misc/habanalabs/common/hw_queue.c | 30 ++++-------
4 files changed, 67 insertions(+), 38 deletions(-)
diff --git a/drivers/misc/habanalabs/common/device.c b/drivers/misc/habanalabs/common/device.c
index ff4cbde289c0..0a788a13f2c1 100644
--- a/drivers/misc/habanalabs/common/device.c
+++ b/drivers/misc/habanalabs/common/device.c
@@ -682,6 +682,44 @@ int hl_device_set_debug_mode(struct hl_device *hdev, bool enable)
return rc;
}
+static void take_release_locks(struct hl_device *hdev)
+{
+ /* Flush anyone that is inside the critical section of enqueue
+ * jobs to the H/W
+ */
+ hdev->asic_funcs->hw_queues_lock(hdev);
+ hdev->asic_funcs->hw_queues_unlock(hdev);
+
+ /* Flush processes that are sending message to CPU */
+ mutex_lock(&hdev->send_cpu_message_lock);
+ mutex_unlock(&hdev->send_cpu_message_lock);
+
+ /* Flush anyone that is inside device open */
+ mutex_lock(&hdev->fpriv_list_lock);
+ mutex_unlock(&hdev->fpriv_list_lock);
+}
+
+static void cleanup_resources(struct hl_device *hdev, bool hard_reset)
+{
+ if (hard_reset)
+ device_late_fini(hdev);
+
+ /*
+ * Halt the engines and disable interrupts so we won't get any more
+ * completions from H/W and we won't have any accesses from the
+ * H/W to the host machine
+ */
+ hdev->asic_funcs->halt_engines(hdev, hard_reset);
+
+ /* Go over all the queues, release all CS and their jobs */
+ hl_cs_rollback_all(hdev);
+
+ /* Release all pending user interrupts, each pending user interrupt
+ * holds a reference to user context
+ */
+ hl_release_pending_user_interrupts(hdev);
+}
+
/*
* hl_device_suspend - initiate device suspend
*
@@ -707,16 +745,7 @@ int hl_device_suspend(struct hl_device *hdev)
/* This blocks all other stuff that is not blocked by in_reset */
hdev->disabled = true;
- /*
- * Flush anyone that is inside the critical section of enqueue
- * jobs to the H/W
- */
- hdev->asic_funcs->hw_queues_lock(hdev);
- hdev->asic_funcs->hw_queues_unlock(hdev);
-
- /* Flush processes that are sending message to CPU */
- mutex_lock(&hdev->send_cpu_message_lock);
- mutex_unlock(&hdev->send_cpu_message_lock);
+ take_release_locks(hdev);
rc = hdev->asic_funcs->suspend(hdev);
if (rc)
@@ -894,8 +923,8 @@ int hl_device_reset(struct hl_device *hdev, u32 flags)
return 0;
}
- hard_reset = (flags & HL_RESET_HARD) != 0;
- from_hard_reset_thread = (flags & HL_RESET_FROM_RESET_THREAD) != 0;
+ hard_reset = !!(flags & HL_RESET_HARD);
+ from_hard_reset_thread = !!(flags & HL_RESET_FROM_RESET_THREAD);
if (!hard_reset && !hdev->supports_soft_reset) {
hard_instead_soft = true;
diff --git a/drivers/misc/habanalabs/common/firmware_if.c b/drivers/misc/habanalabs/common/firmware_if.c
index 2e4d04ec6b53..653e8f5ef6ac 100644
--- a/drivers/misc/habanalabs/common/firmware_if.c
+++ b/drivers/misc/habanalabs/common/firmware_if.c
@@ -240,11 +240,15 @@ int hl_fw_send_cpu_message(struct hl_device *hdev, u32 hw_queue_id, u32 *msg,
/* set fence to a non valid value */
pkt->fence = cpu_to_le32(UINT_MAX);
- rc = hl_hw_queue_send_cb_no_cmpl(hdev, hw_queue_id, len, pkt_dma_addr);
- if (rc) {
- dev_err(hdev->dev, "Failed to send CB on CPU PQ (%d)\n", rc);
- goto out;
- }
+ /*
+ * The CPU queue is a synchronous queue with an effective depth of
+ * a single entry (although it is allocated with room for multiple
+ * entries). We lock on it using 'send_cpu_message_lock' which
+ * serializes accesses to the CPU queue.
+ * Which means that we don't need to lock the access to the entire H/W
+ * queues module when submitting a JOB to the CPU queue.
+ */
+ hl_hw_queue_submit_bd(hdev, queue, 0, len, pkt_dma_addr);
if (prop->fw_app_cpu_boot_dev_sts0 & CPU_BOOT_DEV_STS0_PKT_PI_ACK_EN)
expected_ack_val = queue->pi;
@@ -2235,6 +2239,10 @@ static int hl_fw_dynamic_init_cpu(struct hl_device *hdev,
dev_info(hdev->dev,
"Loading firmware to device, may take some time...\n");
+ /*
+ * In this stage, "cpu_dyn_regs" contains only LKD's hard coded values!
+ * It will be updated from FW after hl_fw_dynamic_request_descriptor().
+ */
dyn_regs = &fw_loader->dynamic_loader.comm_desc.cpu_dyn_regs;
rc = hl_fw_dynamic_send_protocol_cmd(hdev, fw_loader, COMMS_RST_STATE,
diff --git a/drivers/misc/habanalabs/common/habanalabs.h b/drivers/misc/habanalabs/common/habanalabs.h
index 6b3cdd7e068a..c63e26da5135 100644
--- a/drivers/misc/habanalabs/common/habanalabs.h
+++ b/drivers/misc/habanalabs/common/habanalabs.h
@@ -2436,7 +2436,9 @@ void destroy_hdev(struct hl_device *hdev);
int hl_hw_queues_create(struct hl_device *hdev);
void hl_hw_queues_destroy(struct hl_device *hdev);
int hl_hw_queue_send_cb_no_cmpl(struct hl_device *hdev, u32 hw_queue_id,
- u32 cb_size, u64 cb_ptr);
+ u32 cb_size, u64 cb_ptr);
+void hl_hw_queue_submit_bd(struct hl_device *hdev, struct hl_hw_queue *q,
+ u32 ctl, u32 len, u64 ptr);
int hl_hw_queue_schedule_cs(struct hl_cs *cs);
u32 hl_hw_queue_add_ptr(u32 ptr, u16 val);
void hl_hw_queue_inc_ci_kernel(struct hl_device *hdev, u32 hw_queue_id);
diff --git a/drivers/misc/habanalabs/common/hw_queue.c b/drivers/misc/habanalabs/common/hw_queue.c
index bcabfdbf1e01..0afead229e97 100644
--- a/drivers/misc/habanalabs/common/hw_queue.c
+++ b/drivers/misc/habanalabs/common/hw_queue.c
@@ -65,7 +65,7 @@ void hl_hw_queue_update_ci(struct hl_cs *cs)
}
/*
- * ext_and_hw_queue_submit_bd() - Submit a buffer descriptor to an external or a
+ * hl_hw_queue_submit_bd() - Submit a buffer descriptor to an external or a
* H/W queue.
* @hdev: pointer to habanalabs device structure
* @q: pointer to habanalabs queue structure
@@ -80,8 +80,8 @@ void hl_hw_queue_update_ci(struct hl_cs *cs)
* This function must be called when the scheduler mutex is taken
*
*/
-static void ext_and_hw_queue_submit_bd(struct hl_device *hdev,
- struct hl_hw_queue *q, u32 ctl, u32 len, u64 ptr)
+void hl_hw_queue_submit_bd(struct hl_device *hdev, struct hl_hw_queue *q,
+ u32 ctl, u32 len, u64 ptr)
{
struct hl_bd *bd;
@@ -222,8 +222,8 @@ static int hw_queue_sanity_checks(struct hl_device *hdev, struct hl_hw_queue *q,
* @cb_size: size of CB
* @cb_ptr: pointer to CB location
*
- * This function sends a single CB, that must NOT generate a completion entry
- *
+ * This function sends a single CB, that must NOT generate a completion entry.
+ * Sending CPU messages can be done instead via 'hl_hw_queue_submit_bd()'
*/
int hl_hw_queue_send_cb_no_cmpl(struct hl_device *hdev, u32 hw_queue_id,
u32 cb_size, u64 cb_ptr)
@@ -231,16 +231,7 @@ int hl_hw_queue_send_cb_no_cmpl(struct hl_device *hdev, u32 hw_queue_id,
struct hl_hw_queue *q = &hdev->kernel_queues[hw_queue_id];
int rc = 0;
- /*
- * The CPU queue is a synchronous queue with an effective depth of
- * a single entry (although it is allocated with room for multiple
- * entries). Therefore, there is a different lock, called
- * send_cpu_message_lock, that serializes accesses to the CPU queue.
- * As a result, we don't need to lock the access to the entire H/W
- * queues module when submitting a JOB to the CPU queue
- */
- if (q->queue_type != QUEUE_TYPE_CPU)
- hdev->asic_funcs->hw_queues_lock(hdev);
+ hdev->asic_funcs->hw_queues_lock(hdev);
if (hdev->disabled) {
rc = -EPERM;
@@ -258,11 +249,10 @@ int hl_hw_queue_send_cb_no_cmpl(struct hl_device *hdev, u32 hw_queue_id,
goto out;
}
- ext_and_hw_queue_submit_bd(hdev, q, 0, cb_size, cb_ptr);
+ hl_hw_queue_submit_bd(hdev, q, 0, cb_size, cb_ptr);
out:
- if (q->queue_type != QUEUE_TYPE_CPU)
- hdev->asic_funcs->hw_queues_unlock(hdev);
+ hdev->asic_funcs->hw_queues_unlock(hdev);
return rc;
}
@@ -328,7 +318,7 @@ static void ext_queue_schedule_job(struct hl_cs_job *job)
cq->pi = hl_cq_inc_ptr(cq->pi);
submit_bd:
- ext_and_hw_queue_submit_bd(hdev, q, ctl, len, ptr);
+ hl_hw_queue_submit_bd(hdev, q, ctl, len, ptr);
}
/*
@@ -407,7 +397,7 @@ static void hw_queue_schedule_job(struct hl_cs_job *job)
else
ptr = (u64) (uintptr_t) job->user_cb;
- ext_and_hw_queue_submit_bd(hdev, q, ctl, len, ptr);
+ hl_hw_queue_submit_bd(hdev, q, ctl, len, ptr);
}
static int init_signal_cs(struct hl_device *hdev,
--
2.30.2
next prev parent reply other threads:[~2021-09-17 2:33 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-17 2:32 [PATCH AUTOSEL 5.14 01/21] thermal/drivers/rcar_gen3_thermal: Store TSC id as unsigned int Sasha Levin
2021-09-17 2:32 ` [PATCH AUTOSEL 5.14 02/21] habanalabs: fix nullifying of destroyed mmu pgt pool Sasha Levin
2021-09-17 2:32 ` Sasha Levin [this message]
2021-09-17 2:32 ` [PATCH AUTOSEL 5.14 04/21] drm/amdgpu: Fixes to returning VBIOS RAS EEPROM address Sasha Levin
2021-09-17 2:32 ` [PATCH AUTOSEL 5.14 05/21] drm/amd/display: Fix memory leak reported by coverity Sasha Levin
2021-09-17 2:33 ` [PATCH AUTOSEL 5.14 06/21] drm/amdgpu: fix fdinfo race with process exit Sasha Levin
2021-09-17 2:33 ` [PATCH AUTOSEL 5.14 07/21] habanalabs: add validity check for event ID received from F/W Sasha Levin
2021-09-17 2:33 ` [PATCH AUTOSEL 5.14 08/21] habanalabs: fix mmu node address resolution in debugfs Sasha Levin
2021-09-17 2:33 ` [PATCH AUTOSEL 5.14 09/21] habanalabs: add "in device creation" status Sasha Levin
2021-09-17 2:33 ` [PATCH AUTOSEL 5.14 10/21] habanalabs: cannot sleep while holding spinlock Sasha Levin
2021-09-17 2:33 ` [PATCH AUTOSEL 5.14 11/21] pwm: img: Don't modify HW state in .remove() callback Sasha Levin
2021-09-17 2:33 ` [PATCH AUTOSEL 5.14 12/21] pwm: rockchip: " Sasha Levin
2021-09-17 2:33 ` [PATCH AUTOSEL 5.14 13/21] pwm: stm32-lp: " Sasha Levin
2021-09-17 2:33 ` [PATCH AUTOSEL 5.14 14/21] nvmet: fixup buffer overrun in nvmet_subsys_attr_serial() Sasha Levin
2021-09-17 2:33 ` [PATCH AUTOSEL 5.14 15/21] block: genhd: don't call blkdev_show() with major_names_lock held Sasha Levin
2021-09-17 2:33 ` [PATCH AUTOSEL 5.14 16/21] blk-throttle: fix UAF by deleteing timer in blk_throtl_exit() Sasha Levin
2021-09-17 2:33 ` [PATCH AUTOSEL 5.14 17/21] blk-mq: allow 4x BLK_MAX_REQUEST_COUNT at blk_plug for multiple_queues Sasha Levin
2021-09-17 2:33 ` [PATCH AUTOSEL 5.14 18/21] rtc: rx8010: select REGMAP_I2C Sasha Levin
2021-09-17 2:33 ` [PATCH AUTOSEL 5.14 19/21] sched/idle: Make the idle timer expire in hard interrupt context Sasha Levin
2021-09-17 2:33 ` [PATCH AUTOSEL 5.14 20/21] cifs: properly invalidate cached root handle when closing it Sasha Levin
2021-09-17 2:33 ` [PATCH AUTOSEL 5.14 21/21] io_uring: fix off-by-one in BUILD_BUG_ON check of __REQ_F_LAST_BIT Sasha Levin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210917023315.816225-3-sashal@kernel.org \
--to=sashal@kernel.org \
--cc=Arnd@vger.kernel.org \
--cc=amizrahi@habana.ai \
--cc=fkassabri@habana.ai \
--cc=gregkh@linuxfoundation.org \
--cc=kelbaz@habana.ai \
--cc=linux-kernel@vger.kernel.org \
--cc=mhaimovski@habana.ai \
--cc=obitton@habana.ai \
--cc=ogabbay@kernel.org \
--cc=osharabi@habana.ai \
--cc=oshpigelman@habana.ai \
--cc=stable@vger.kernel.org \
--cc=ttayar@habana.ai \
--cc=ynudelman@habana.ai \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®