From: Oded Gabbay <ogabbay@kernel.org>
To: linux-kernel@vger.kernel.org
Cc: Ohad Sharabi <osharabi@habana.ai>
Subject: [PATCH 25/26] habanalabs/gaudi2: wait for preboot ready if HW state is dirty
Date: Thu, 8 Dec 2022 17:13:49 +0200 [thread overview]
Message-ID: <20221208151350.1833823-25-ogabbay@kernel.org> (raw)
In-Reply-To: <20221208151350.1833823-1-ogabbay@kernel.org>
From: Ohad Sharabi <osharabi@habana.ai>
Instead of waiting for BTM indication we should wait for preboot ready.
Consider the below scenario:
1. FW update is being triggered
- setting the dirty bit
2. hard reset will be triggered due to the dirty bit
3. FW initiates the reset:
- dirty bit cleared
- BTM indication cleared
- preboot ready indication cleared
4. during hard reset:
- BTM indication will be set
- BIST test performed and another reset triggered
5. only after this reset the preboot will set the preboot ready
When polling on BTM indication alone we can lose sync with FW while
trying to communicate with FW that is during reset.
To overcome this we will always wait to preboot ready indication.
Signed-off-by: Ohad Sharabi <osharabi@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
---
drivers/misc/habanalabs/common/firmware_if.c | 2 +-
drivers/misc/habanalabs/common/habanalabs.h | 1 +
drivers/misc/habanalabs/gaudi2/gaudi2.c | 26 +++++++++++++++++++-
3 files changed, 27 insertions(+), 2 deletions(-)
diff --git a/drivers/misc/habanalabs/common/firmware_if.c b/drivers/misc/habanalabs/common/firmware_if.c
index 537b1ae3fcb7..cda0bf3dbf1b 100644
--- a/drivers/misc/habanalabs/common/firmware_if.c
+++ b/drivers/misc/habanalabs/common/firmware_if.c
@@ -1352,7 +1352,7 @@ static void detect_cpu_boot_status(struct hl_device *hdev, u32 status)
}
}
-static int hl_fw_wait_preboot_ready(struct hl_device *hdev)
+int hl_fw_wait_preboot_ready(struct hl_device *hdev)
{
struct pre_fw_load_props *pre_fw_load = &hdev->fw_loader.pre_fw_load;
u32 status;
diff --git a/drivers/misc/habanalabs/common/habanalabs.h b/drivers/misc/habanalabs/common/habanalabs.h
index de715c91a87e..e5443bf7fe12 100644
--- a/drivers/misc/habanalabs/common/habanalabs.h
+++ b/drivers/misc/habanalabs/common/habanalabs.h
@@ -3745,6 +3745,7 @@ int hl_fw_cpucp_power_get(struct hl_device *hdev, u64 *power);
void hl_fw_ask_hard_reset_without_linux(struct hl_device *hdev);
void hl_fw_ask_halt_machine_without_linux(struct hl_device *hdev);
int hl_fw_init_cpu(struct hl_device *hdev);
+int hl_fw_wait_preboot_ready(struct hl_device *hdev);
int hl_fw_read_preboot_status(struct hl_device *hdev);
int hl_fw_dynamic_send_protocol_cmd(struct hl_device *hdev,
struct fw_load_mgr *fw_loader,
diff --git a/drivers/misc/habanalabs/gaudi2/gaudi2.c b/drivers/misc/habanalabs/gaudi2/gaudi2.c
index ba3b0ae76ebf..5242b6f6bf95 100644
--- a/drivers/misc/habanalabs/gaudi2/gaudi2.c
+++ b/drivers/misc/habanalabs/gaudi2/gaudi2.c
@@ -5484,7 +5484,31 @@ static void gaudi2_hw_fini(struct hl_device *hdev, bool hard_reset, bool fw_rese
skip_reset:
if (driver_performs_reset || hard_reset)
- gaudi2_poll_btm_indication(hdev, reset_sleep_ms, poll_timeout_us);
+ /*
+ * Instead of waiting for BTM indication we should wait for preboot ready:
+ * Consider the below scenario:
+ * 1. FW update is being triggered
+ * - setting the dirty bit
+ * 2. hard reset will be triggered due to the dirty bit
+ * 3. FW initiates the reset:
+ * - dirty bit cleared
+ * - BTM indication cleared
+ * - preboot ready indication cleared
+ * 4. during hard reset:
+ * - BTM indication will be set
+ * - BIST test performed and another reset triggered
+ * 5. only after this reset the preboot will set the preboot ready
+ *
+ * when polling on BTM indication alone we can lose sync with FW while trying to
+ * communicate with FW that is during reset.
+ * to overcome this we will always wait to preboot ready indication
+ */
+ if ((hdev->fw_components & FW_TYPE_PREBOOT_CPU)) {
+ msleep(reset_sleep_ms);
+ hl_fw_wait_preboot_ready(hdev);
+ } else {
+ gaudi2_poll_btm_indication(hdev, reset_sleep_ms, poll_timeout_us);
+ }
else
gaudi2_get_soft_rst_done_indication(hdev, poll_timeout_us);
--
2.25.1
next prev parent reply other threads:[~2022-12-08 15:15 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-12-08 15:13 [PATCH 01/26] habanalabs/gaudi2: fix BMON 3rd address range Oded Gabbay
2022-12-08 15:13 ` [PATCH 02/26] habanalabs: read binning info from preboot Oded Gabbay
2022-12-08 15:13 ` [PATCH 03/26] habanalabs: remove releasing of user threads from device release Oded Gabbay
2022-12-08 15:13 ` [PATCH 04/26] habanalabs: abort waiting user threads upon error Oded Gabbay
2022-12-08 15:13 ` [PATCH 05/26] habanalabs: don't notify user about clk throttling due to power Oded Gabbay
2022-12-08 15:13 ` [PATCH 06/26] habanalabs: don't allow user to destroy CB handle more than once Oded Gabbay
2022-12-08 15:13 ` [PATCH 07/26] habanalabs: use dev_dbg() when hl_mmap_mem_buf_get() fails Oded Gabbay
2022-12-08 15:13 ` [PATCH 08/26] habanalabs: make set_dram_properties an ASIC function Oded Gabbay
2022-12-08 15:13 ` [PATCH 09/26] habanalabs: fix double assignment in MMU V1 Oded Gabbay
2022-12-08 15:13 ` [PATCH 10/26] habanalabs: update DRAM props according to preboot data Oded Gabbay
2022-12-08 15:13 ` [PATCH 11/26] habanalabs/gaudi2: count interrupt causes Oded Gabbay
2022-12-08 15:13 ` [PATCH 12/26] habanalabs/gaudi2: remove duplicated event prints Oded Gabbay
2022-12-08 15:13 ` [PATCH 13/26] habanalabs: adjacent timestamps should be more accurate Oded Gabbay
2022-12-08 15:13 ` [PATCH 14/26] habanalabs: skip device idle check in hpriv_release if in reset Oded Gabbay
2022-12-08 15:13 ` [PATCH 15/26] habanalabs/gaudi2: support abrupt device reset event Oded Gabbay
2022-12-08 15:13 ` [PATCH 16/26] habanalabs: define traces for COMMS protocol Oded Gabbay
2022-12-08 15:13 ` [PATCH 17/26] habanalabs: trace " Oded Gabbay
2022-12-08 15:13 ` [PATCH 18/26] habanalabs: set log level for descriptor validation to debug Oded Gabbay
2022-12-08 15:13 ` [PATCH 19/26] habanalabs: remove support to export dmabuf from handle Oded Gabbay
2022-12-08 15:13 ` [PATCH 20/26] habanalabs: helper function to validate export params Oded Gabbay
2022-12-08 15:13 ` [PATCH 21/26] habanalabs: modify export dmabuf API Oded Gabbay
2022-12-08 15:13 ` [PATCH 22/26] habanalabs: fix dmabuf to export only required size Oded Gabbay
2022-12-08 15:13 ` [PATCH 23/26] habanalabs: fix handling of wait CS for interrupting signals Oded Gabbay
2022-12-08 15:13 ` [PATCH 24/26] habanalabs: put fences in case of unexpected wait status Oded Gabbay
2022-12-08 15:13 ` Oded Gabbay [this message]
2022-12-08 15:13 ` [PATCH 26/26] habanalabs: fix wrong variable type used for vzalloc Oded Gabbay
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20221208151350.1833823-25-ogabbay@kernel.org \
--to=ogabbay@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=osharabi@habana.ai \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®