[PATCH 0/2] ipmi:watchdog: Fix panic, D-state hang, and lost protection on BMC reset

[PATCH 0/2] ipmi:watchdog: Fix panic, D-state hang, and lost protection on BMC reset

[Tony Camuso]

When the BMC resets while the IPMI watchdog is active, the driver has
three failure modes depending on timing:

 1. list_add double add panic -- the watchdog daemon retries while the
    static smi_msg/recv_msg structures are still queued in the IPMI
    layer from the previous (unanswered) request.

 2. D-state hang -- wait_for_completion() blocks indefinitely because
    the BMC never delivers a response.

 3. Silent loss of watchdog protection -- the BMC returns a non-zero
    completion code, the driver's internal state becomes inconsistent,
    writes to /dev/watchdog return -EINVAL, and the daemon gives up.
    The system continues running without hardware watchdog coverage.

All three stem from the same root cause: the static message structures
and unbounded completion waits were never designed for a BMC that
disappears mid-transaction.

This has been independently reported by Kenta Akagi on a Dell PowerEdge
R640 running 6.18.7, also triggered by a BMC reset with the watchdog
active:

  https://sourceforge.net/p/openipmi/mailman/message/59292850/

The fix takes a simple, deterministic approach: detect the failure via
BMC error response, guard against structure reuse (msg_in_flight) and
indefinite waits (completion timeout), then initiate orderly_reboot()
when the watchdog is active.  This produces the same outcome the
hardware watchdog would have -- a system reset -- but through a
controlled path with clear logging and no panics or hangs.

If the watchdog is stopped when the BMC resets, no reboot occurs and
the system continues normally.

Tested on Dell PowerEdge R640 with kernel 5.14 (RHEL 9) and verified
against mainline (both patches apply cleanly).

Corey Minyard's recent fix for list corruption in smi_work()
(ipmi_msghandler.c) addresses a related but separate code path.  The
watchdog driver's own static structure reuse requires this fix.

Tony Camuso (2):
  ipmi:watchdog: Reboot cleanly on BMC reset
  Documentation: ipmi: Update BMC reset behavior for watchdog

 Documentation/driver-api/ipmi.rst |  61 ++++++++++++++++++
 drivers/char/ipmi/ipmi_watchdog.c | 101 ++++++++++++++++++++++++------
 2 files changed, 144 insertions(+), 18 deletions(-)

-- 
2.53.0

[PATCH 1/2] ipmi:watchdog: Reboot cleanly on BMC reset

[Tony Camuso]

When the BMC resets while the IPMI watchdog is active, three problems
can occur:

1. The static smi_msg and recv_msg structures remain queued in the IPMI
   layer after a response timeout. If the watchdog daemon retries, the
   code reuses these structures while still on the IPMI layer's internal
   lists, causing:

     list_add double add: new=ffffffffc10063e0, prev=ffffffffc10063e0, ...
     kernel BUG at lib/list_debug.c:29!

2. Both __ipmi_heartbeat() and _ipmi_set_timeout() use
   wait_for_completion() with no timeout, blocking indefinitely if the
   BMC is unresponsive, leaving tasks stuck in D state.

3. When the BMC loses the watchdog timer state, the driver's internal
   state becomes inconsistent, causing subsequent writes to /dev/watchdog
   to return -EINVAL, leaving the system without watchdog protection.

Fix all three issues:

- Add msg_in_flight atomic flag to prevent re-entry into
  __ipmi_heartbeat() and _ipmi_set_timeout() while message structures
  are still queued in the IPMI layer.

- Convert wait_for_completion() to wait_for_completion_timeout() in
  both functions to prevent indefinite blocking.

- Add reinit_completion() before each use to prevent stale completion
  events from allowing premature wakeup.

- Detect BMC communication failure in ipmi_wdog_msg_handler() via
  non-zero completion codes and initiate orderly_reboot() when the
  watchdog is active.  This ensures the system reboots cleanly rather
  than being left without watchdog protection.  Error classification
  distinguishes TIMER_NOT_INIT (0x80), vendor-specific codes
  (0x81-0xBE), and standard IPMI completion codes.

- Guard all BMC communication paths (_ipmi_set_timeout,
  __ipmi_heartbeat, wdog_reboot_handler) with bmc_reset_shutdown flag
  to prevent further IPMI operations during shutdown.

Signed-off-by: Tony Camuso <tcamuso@redhat.com>
---
 drivers/char/ipmi/ipmi_watchdog.c | 101 ++++++++++++++++++++++++------
 1 file changed, 83 insertions(+), 18 deletions(-)

diff --git a/drivers/char/ipmi/ipmi_watchdog.c b/drivers/char/ipmi/ipmi_watchdog.c
index a013ddbf1466..1d8277cbe598 100644
--- a/drivers/char/ipmi/ipmi_watchdog.c
+++ b/drivers/char/ipmi/ipmi_watchdog.c
@@ -123,6 +123,16 @@
 
 #define IPMI_WDOG_TIMER_NOT_INIT_RESP	0x80
 
+/* Timeout for waiting for a heartbeat response (in jiffies). */
+#define IPMI_HEARTBEAT_WAIT_TIMEOUT	(HZ * 5)
+
+/*
+ * Set when the BMC becomes unreachable while the watchdog is active.
+ * Once set, all BMC communication is skipped and an orderly reboot
+ * is in progress.
+ */
+static bool bmc_reset_shutdown;
+
 static DEFINE_MUTEX(ipmi_watchdog_mutex);
 static bool nowayout = WATCHDOG_NOWAYOUT;
 
@@ -339,12 +349,14 @@ static int __ipmi_heartbeat(void);
  * and freed when both the send and receive messages are free.
  */
 static atomic_t msg_tofree = ATOMIC_INIT(0);
+static atomic_t msg_in_flight = ATOMIC_INIT(0);
 static DECLARE_COMPLETION(msg_wait);
 static void msg_free_smi(struct ipmi_smi_msg *msg)
 {
 	if (atomic_dec_and_test(&msg_tofree)) {
 		if (!oops_in_progress)
 			complete(&msg_wait);
+		atomic_set(&msg_in_flight, 0);
 	}
 }
 static void msg_free_recv(struct ipmi_recv_msg *msg)
@@ -352,6 +364,7 @@ static void msg_free_recv(struct ipmi_recv_msg *msg)
 	if (atomic_dec_and_test(&msg_tofree)) {
 		if (!oops_in_progress)
 			complete(&msg_wait);
+		atomic_set(&msg_in_flight, 0);
 	}
 }
 static struct ipmi_smi_msg smi_msg = INIT_IPMI_SMI_MSG(msg_free_smi);
@@ -429,19 +442,34 @@ static int _ipmi_set_timeout(int do_heartbeat)
 {
 	int send_heartbeat_now;
 	int rv;
+	unsigned long ret;
 
 	if (!watchdog_user)
 		return -ENODEV;
 
+	if (bmc_reset_shutdown)
+		return -ENODEV;
+
+	if (atomic_read(&msg_in_flight))
+		return -EBUSY;
+
+	reinit_completion(&msg_wait);
+	atomic_set(&msg_in_flight, 1);
 	atomic_set(&msg_tofree, 2);
 
 	rv = __ipmi_set_timeout(&smi_msg, &recv_msg, &send_heartbeat_now);
 	if (rv) {
 		atomic_set(&msg_tofree, 0);
+		atomic_set(&msg_in_flight, 0);
 		return rv;
 	}
 
-	wait_for_completion(&msg_wait);
+	ret = wait_for_completion_timeout(&msg_wait,
+					  IPMI_HEARTBEAT_WAIT_TIMEOUT);
+	if (ret == 0) {
+		atomic_set(&msg_tofree, 0);
+		return -ETIMEDOUT;
+	}
 
 	if ((do_heartbeat == IPMI_SET_TIMEOUT_FORCE_HB)
 		|| ((send_heartbeat_now)
@@ -510,10 +538,17 @@ static int __ipmi_heartbeat(void)
 {
 	struct kernel_ipmi_msg msg;
 	int rv;
+	unsigned long ret;
 	struct ipmi_system_interface_addr addr;
 	int timeout_retries = 0;
 
 restart:
+	if (bmc_reset_shutdown)
+		return -ENODEV;
+
+	if (atomic_read(&msg_in_flight))
+		return -EBUSY;
+
 	/*
 	 * Don't reset the timer if we have the timer turned off, that
 	 * re-enables the watchdog.
@@ -521,6 +556,8 @@ static int __ipmi_heartbeat(void)
 	if (ipmi_watchdog_state == WDOG_TIMEOUT_NONE)
 		return 0;
 
+	reinit_completion(&msg_wait);
+	atomic_set(&msg_in_flight, 1);
 	atomic_set(&msg_tofree, 2);
 
 	addr.addr_type = IPMI_SYSTEM_INTERFACE_ADDR_TYPE;
@@ -541,14 +578,17 @@ static int __ipmi_heartbeat(void)
 				      1);
 	if (rv) {
 		atomic_set(&msg_tofree, 0);
-		pr_warn("heartbeat send failure: %d\n", rv);
+		atomic_set(&msg_in_flight, 0);
 		return rv;
 	}
 
-	/* Wait for the heartbeat to be sent. */
-	wait_for_completion(&msg_wait);
+	ret = wait_for_completion_timeout(&msg_wait, IPMI_HEARTBEAT_WAIT_TIMEOUT);
+	if (ret == 0) {
+		atomic_set(&msg_tofree, 0);
+		return -ETIMEDOUT;
+	}
 
-	if (recv_msg.msg.data[0] == IPMI_WDOG_TIMER_NOT_INIT_RESP)  {
+	if (recv_msg.msg.data[0] >= 0x80) {
 		timeout_retries++;
 		if (timeout_retries > 3) {
 			pr_err("Unable to restore the IPMI watchdog's settings, giving up\n");
@@ -557,12 +597,11 @@ static int __ipmi_heartbeat(void)
 		}
 
 		/*
-		 * The timer was not initialized, that means the BMC was
-		 * probably reset and lost the watchdog information.  Attempt
-		 * to restore the timer's info.  Note that we still hold
-		 * the heartbeat lock, to keep a heartbeat from happening
-		 * in this process, so must say no heartbeat to avoid a
-		 * deadlock on this mutex
+		 * The BMC was probably reset and lost the watchdog
+		 * information.  Attempt to restore the timer's info.
+		 * Note that we still hold the heartbeat lock, to keep
+		 * a heartbeat from happening in this process, so must
+		 * say no heartbeat to avoid a deadlock on this mutex.
 		 */
 		rv = _ipmi_set_timeout(IPMI_SET_TIMEOUT_NO_HB);
 		if (rv) {
@@ -876,15 +915,38 @@ static struct miscdevice ipmi_wdog_miscdev = {
 static void ipmi_wdog_msg_handler(struct ipmi_recv_msg *msg,
 				  void                 *handler_data)
 {
-	if (msg->msg.cmd == IPMI_WDOG_RESET_TIMER &&
-			msg->msg.data[0] == IPMI_WDOG_TIMER_NOT_INIT_RESP)
-		pr_info("response: The IPMI controller appears to have been reset, will attempt to reinitialize the watchdog timer\n");
-	else if (msg->msg.data[0] != 0)
-		pr_err("response: Error %x on cmd %x\n",
-		       msg->msg.data[0],
-		       msg->msg.cmd);
+	if (msg->msg.data[0] != 0) {
+		if (msg->msg.data[0] == IPMI_WDOG_TIMER_NOT_INIT_RESP)
+			pr_crit("BMC error: watchdog timer not initialized "
+				"(0x%02x on cmd 0x%02x)\n",
+				msg->msg.data[0], msg->msg.cmd);
+		else if (msg->msg.data[0] > 0x80 &&
+			 msg->msg.data[0] <= 0xBE)
+			pr_crit("BMC error: vendor-specific completion code "
+				"0x%02x on cmd 0x%02x\n",
+				msg->msg.data[0], msg->msg.cmd);
+		else
+			pr_crit("BMC error: completion code 0x%02x "
+				"on cmd 0x%02x\n",
+				msg->msg.data[0], msg->msg.cmd);
+
+		if (ipmi_watchdog_state != WDOG_TIMEOUT_NONE &&
+		    !bmc_reset_shutdown) {
+			bmc_reset_shutdown = true;
+			pr_crit("BMC communication lost with watchdog active, "
+				"initiating system reboot\n");
+			orderly_reboot();
+		}
+	}
 
 	ipmi_free_recv_msg(msg);
+	/*
+	 * Ensure the in-flight flag is cleared after the message is freed.
+	 * In the normal path this is redundant (already cleared by the
+	 * recv_msg destructor). For late responses arriving after a
+	 * completion timeout, this is the only path that clears the flag.
+	 */
+	atomic_set(&msg_in_flight, 0);
 }
 
 static void ipmi_wdog_pretimeout_handler(void *handler_data)
@@ -1106,6 +1168,9 @@ static int wdog_reboot_handler(struct notifier_block *this,
 		/* Make sure we only do this once. */
 		reboot_event_handled = 1;
 
+		if (bmc_reset_shutdown)
+			return NOTIFY_OK;
+
 		if (code == SYS_POWER_OFF || code == SYS_HALT) {
 			/* Disable the WDT if we are shutting down. */
 			ipmi_watchdog_state = WDOG_TIMEOUT_NONE;
-- 
2.53.0

[PATCH 2/2] Documentation: ipmi: Update BMC reset behavior for watchdog

[Tony Camuso]

Update the IPMI watchdog BMC reset documentation to describe the
current behavior: when the BMC resets while the watchdog is active,
the driver detects the communication failure and initiates an orderly
system reboot rather than attempting to retry and recover.

Document the panic and hang prevention mechanisms, BMC failure
detection via completion code classification in the message handler,
the bmc_reset_shutdown guard that prevents further IPMI operations
during shutdown, and the late response handling for the msg_in_flight
flag.

Signed-off-by: Tony Camuso <tcamuso@redhat.com>
---
 Documentation/driver-api/ipmi.rst | 61 +++++++++++++++++++++++++++++++
 1 file changed, 61 insertions(+)

diff --git a/Documentation/driver-api/ipmi.rst b/Documentation/driver-api/ipmi.rst
index f52ab2df2569..dbdc1440d16e 100644
--- a/Documentation/driver-api/ipmi.rst
+++ b/Documentation/driver-api/ipmi.rst
@@ -734,6 +734,67 @@ device to close it, or the timer will not stop.  This is a new semantic
 for the driver, but makes it consistent with the rest of the watchdog
 drivers in Linux.
 
+BMC Reset Behavior
+------------------
+
+When the BMC (Baseboard Management Controller) resets while the IPMI
+watchdog is active, the hardware watchdog timer state on the BMC is
+lost.  The driver detects this condition and initiates a clean system
+reboot rather than leaving the system running without watchdog
+protection.
+
+The driver handles BMC resets as follows:
+
+1. **Panic prevention:** The static message structures (``smi_msg`` and
+   ``recv_msg``) are guarded by an ``msg_in_flight`` atomic flag.  If a
+   previous message is still queued in the IPMI layer, new operations
+   return ``-EBUSY`` instead of reusing the structures (which would cause
+   a ``list_add`` corruption BUG).
+
+2. **Hang prevention:** ``wait_for_completion_timeout()`` with a 5-second
+   timeout replaces the indefinite ``wait_for_completion()`` in both
+   ``__ipmi_heartbeat()`` and ``_ipmi_set_timeout()``.  This prevents
+   tasks from blocking in D state when the BMC is unresponsive.
+
+3. **BMC failure detection:** When ``ipmi_wdog_msg_handler()`` receives
+   a non-zero completion code while the watchdog is active, it sets the
+   ``bmc_reset_shutdown`` flag and calls ``orderly_reboot()``.  Error
+   classification distinguishes three categories:
+
+   - ``TIMER_NOT_INIT`` (0x80): the BMC lost the watchdog timer state.
+   - Vendor-specific codes (0x81-0xBE): BMC-specific error responses.
+   - Standard IPMI completion codes (0xC0+): general BMC errors.
+
+   All produce a critical-level log message::
+
+     IPMI Watchdog: BMC error: watchdog timer not initialized (0x80 on cmd 0x22)
+     IPMI Watchdog: BMC communication lost with watchdog active, initiating system reboot
+
+4. **Clean shutdown:** Once ``bmc_reset_shutdown`` is set, all BMC
+   communication paths (``_ipmi_set_timeout()``, ``__ipmi_heartbeat()``,
+   ``wdog_reboot_handler()``) return immediately without attempting
+   further IPMI operations.  This prevents panics, stack traces, and
+   hangs during the reboot sequence.
+
+5. **Late response handling:** The ``msg_in_flight`` flag is cleared in
+   ``ipmi_wdog_msg_handler()`` after the message is freed.  This handles
+   late responses arriving after a completion timeout, ensuring the flag
+   does not remain set permanently.
+
+The system reboot after a BMC reset is the expected and correct
+behavior.  The hardware watchdog timer lives on the BMC, and when
+that timer state is lost, the system must be restarted to restore
+watchdog protection.
+
+Administrators performing supervised BMC maintenance (firmware updates,
+manual resets) should disarm the watchdog before the operation::
+
+  systemctl stop watchdog
+
+And restart it after the BMC has fully recovered::
+
+  systemctl start watchdog
+
 
 Panic Timeouts
 --------------
-- 
2.53.0

Re: [PATCH 0/2] ipmi:watchdog: Fix panic, D-state hang, and lost protection on BMC reset

[Corey Minyard]

On Tue, Apr 07, 2026 at 01:51:32PM -0400, Tony Camuso wrote:
> When the BMC resets while the IPMI watchdog is active, the driver has
> three failure modes depending on timing:
> 
>  1. list_add double add panic -- the watchdog daemon retries while the
>     static smi_msg/recv_msg structures are still queued in the IPMI
>     layer from the previous (unanswered) request.

I'm trying to make sense of this.  Are you sure this didn't start
happening after you added a timeout on the wait_for_completion()?
Otherwise it would never return, the mutex would be held, and no new
message could be added.

Just timing out in wait_for_completion() there could cause all kinds of
bad things to happen.

> 
>  2. D-state hang -- wait_for_completion() blocks indefinitely because
>     the BMC never delivers a response.

This is an issue.  The lower level driver is *always* supposed to return
a failure.  Something else needs to be fixed.

I have seen several creative ways in which BMCs "fail to respond" that
have confused the lower level drivers.  If my guess is correct, there's
a bug in the low level driver that's causing it to not time out the
message.

If we don't fix this, it will cause other issues outside the watchdog.

> 
>  3. Silent loss of watchdog protection -- the BMC returns a non-zero
>     completion code, the driver's internal state becomes inconsistent,
>     writes to /dev/watchdog return -EINVAL, and the daemon gives up.
>     The system continues running without hardware watchdog coverage.

Again, are you sure this didn't start happening after you added the
timeout?

> 
> All three stem from the same root cause: the static message structures
> and unbounded completion waits were never designed for a BMC that
> disappears mid-transaction.

All that is supposed to be protected by a mutex.  That mutex is claimed
on all IPMI watchdog operations, and it shouldn't be released until all
resources have been freed.  Anything that violates that is asking for
trouble.

You don't mention the lower level interface (KCS, BT, SMIC, SSIF) but I
think we need to start looking there.

It may be that the timeouts on the watchdog messages need to be
adjusted.  The whole IPMI driver was designed on the presumption that
the BMC would go away for only a short period of time (5-10 seconds) and
not permanantly.  That has slowly been fixed over time, but things might
need to be adjusted in the watchdog.

-corey

> 
> This has been independently reported by Kenta Akagi on a Dell PowerEdge
> R640 running 6.18.7, also triggered by a BMC reset with the watchdog
> active:
> 
>   https://sourceforge.net/p/openipmi/mailman/message/59292850/
> 
> The fix takes a simple, deterministic approach: detect the failure via
> BMC error response, guard against structure reuse (msg_in_flight) and
> indefinite waits (completion timeout), then initiate orderly_reboot()
> when the watchdog is active.  This produces the same outcome the
> hardware watchdog would have -- a system reset -- but through a
> controlled path with clear logging and no panics or hangs.
> 
> If the watchdog is stopped when the BMC resets, no reboot occurs and
> the system continues normally.
> 
> Tested on Dell PowerEdge R640 with kernel 5.14 (RHEL 9) and verified
> against mainline (both patches apply cleanly).
> 
> Corey Minyard's recent fix for list corruption in smi_work()
> (ipmi_msghandler.c) addresses a related but separate code path.  The
> watchdog driver's own static structure reuse requires this fix.
> 
> Tony Camuso (2):
>   ipmi:watchdog: Reboot cleanly on BMC reset
>   Documentation: ipmi: Update BMC reset behavior for watchdog
> 
>  Documentation/driver-api/ipmi.rst |  61 ++++++++++++++++++
>  drivers/char/ipmi/ipmi_watchdog.c | 101 ++++++++++++++++++++++++------
>  2 files changed, 144 insertions(+), 18 deletions(-)
> 
> -- 
> 2.53.0
> 

Re: [PATCH 0/2] ipmi:watchdog: Fix panic, D-state hang, and lost protection on BMC reset

[Tony Camuso]

On 4/7/2026 5:54 PM, Corey Minyard wrote:
> On Tue, Apr 07, 2026 at 01:51:32PM -0400, Tony Camuso wrote:
>> When the BMC resets while the IPMI watchdog is active, the driver has
>> three failure modes depending on timing:
>>
>>   1. list_add double add panic -- the watchdog daemon retries while the
>>      static smi_msg/recv_msg structures are still queued in the IPMI
>>      layer from the previous (unanswered) request.
> 
> I'm trying to make sense of this.  Are you sure this didn't start
> happening after you added a timeout on the wait_for_completion()?
> Otherwise it would never return, the mutex would be held, and no new
> message could be added.
> 
> Just timing out in wait_for_completion() there could cause all kinds of
> bad things to happen.
> 

You're right. This work was done on a RHEL 9 kernel that did not yet have
your recent upstream KCS/SI fixes applied, so some of the behavior I
observed may have been caused/influenced by bugs you've more recently
addressed.

>>
>>   2. D-state hang -- wait_for_completion() blocks indefinitely because
>>      the BMC never delivers a response.
> 
> This is an issue.  The lower level driver is *always* supposed to return
> a failure.  Something else needs to be fixed.
> 
> I have seen several creative ways in which BMCs "fail to respond" that
> have confused the lower level drivers.  If my guess is correct, there's
> a bug in the low level driver that's causing it to not time out the
> message.
> 
> If we don't fix this, it will cause other issues outside the watchdog.
> 

Agreed -- the D-state hang is a symptom, not the root cause. If the
KCS driver correctly transitions through error recovery to
SI_SM_HOSED, and the SI layer returns an error completion to the
caller, then wait_for_completion() should never block indefinitely.

To get to the bottom of this, I've instrumented three layers:

   - ipmi_kcs_sm.c: trace entry into start_error_recovery() and the
     transition to KCS_HOSED after MAX_ERROR_RETRIES
   - ipmi_si_intf.c: trace return_hosed_msg(), the SI_SM_HOSED
     handler in smi_event_handler(), and HOSED recovery in
     smi_timeout()
   - ipmi_watchdog.c: trace message send/completion in
     _ipmi_set_timeout() and __ipmi_heartbeat(), and the completion
     code received in ipmi_wdog_msg_handler()

I've applied your recent upstream patches to my test kernel, so the
KCS/SI code is congruent with current mainline. The traces will show
whether the error recovery chain works correctly with your fixes in
place, or whether the BMC is doing something that still confuses the
low-level driver.

I'll collect the data and follow up.

>>
>>   3. Silent loss of watchdog protection -- the BMC returns a non-zero
>>      completion code, the driver's internal state becomes inconsistent,
>>      writes to /dev/watchdog return -EINVAL, and the daemon gives up.
>>      The system continues running without hardware watchdog coverage.
> 
> Again, are you sure this didn't start happening after you added the
> timeout?
> 

I think this one is pre-existing, independent of any timeout
changes. When the BMC comes back after a reset and returns a
non-zero completion code (e.g. 0xD5 or 0xFF), the watchdog handler
treats this as a permanent failure. The userspace daemon sees
-EINVAL on subsequent writes to /dev/watchdog and stops retrying.
The system continues running without hardware watchdog coverage,
with no indication to the administrator.

But I need to confirm this with the instrumented traces on the
the patched kernel.

I should have traces collected within the next week or so.

Tony

>>
>> All three stem from the same root cause: the static message structures
>> and unbounded completion waits were never designed for a BMC that
>> disappears mid-transaction.
> 
> All that is supposed to be protected by a mutex.  That mutex is claimed
> on all IPMI watchdog operations, and it shouldn't be released until all
> resources have been freed.  Anything that violates that is asking for
> trouble.
> 
> You don't mention the lower level interface (KCS, BT, SMIC, SSIF) but I
> think we need to start looking there.
> 
> It may be that the timeouts on the watchdog messages need to be
> adjusted.  The whole IPMI driver was designed on the presumption that
> the BMC would go away for only a short period of time (5-10 seconds) and
> not permanantly.  That has slowly been fixed over time, but things might
> need to be adjusted in the watchdog.
> 
> -corey
> 
>>>> This has been independently reported by Kenta Akagi on a Dell PowerEdge
>> R640 running 6.18.7, also triggered by a BMC reset with the watchdog
>> active:
>>
>>    https://sourceforge.net/p/openipmi/mailman/message/59292850/
>>
>> The fix takes a simple, deterministic approach: detect the failure via
>> BMC error response, guard against structure reuse (msg_in_flight) and
>> indefinite waits (completion timeout), then initiate orderly_reboot()
>> when the watchdog is active.  This produces the same outcome the
>> hardware watchdog would have -- a system reset -- but through a
>> controlled path with clear logging and no panics or hangs.
>>
>> If the watchdog is stopped when the BMC resets, no reboot occurs and
>> the system continues normally.
>>
>> Tested on Dell PowerEdge R640 with kernel 5.14 (RHEL 9) and verified
>> against mainline (both patches apply cleanly).
>>
>> Corey Minyard's recent fix for list corruption in smi_work()
>> (ipmi_msghandler.c) addresses a related but separate code path.  The
>> watchdog driver's own static structure reuse requires this fix.
>>
>> Tony Camuso (2):
>>    ipmi:watchdog: Reboot cleanly on BMC reset
>>    Documentation: ipmi: Update BMC reset behavior for watchdog
>>
>>   Documentation/driver-api/ipmi.rst |  61 ++++++++++++++++++
>>   drivers/char/ipmi/ipmi_watchdog.c | 101 ++++++++++++++++++++++++------
>>   2 files changed, 144 insertions(+), 18 deletions(-)
>>
>> -- 
>> 2.53.0
>>
>