From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8C93E3B6364 for ; Tue, 7 Apr 2026 17:51:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775584306; cv=none; b=X7+8fYk2e9wyApGTYgaIKvlSSCSNimWmBk5fRK7Je6p3wxoRXlpMLuevjEAtwwDqTZpxwkNsy4U5t1sSeD7vsneCrVAX1eO4Th037L/lE7tccbvjgNRsHf6DKMl/SiAO/ETPmIoafrPNCis7LhefHn5w9YoygoRSIvRabu7OXlA= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775584306; c=relaxed/simple; bh=zKuVClqgtppi4ywq1lk995n/crx1o613zPV8uapkU7c=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=Han/yHCKKIiC0YDsOg/rx9wZCDRvOidWR8zZc/XccdVmjGGRFYMuVGPTl5aRn3PiAy36lcfDhb7JBM5nmVJhoEz4E8RTLoFcTMO6kwFGNZackEJV7QjX6kQxZHoWBqwpi4zb+u5DkFQx2YVNebrKOPURDsEHIy2zgr3jaoirAnw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=VmNGHd9c; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="VmNGHd9c" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1775584300; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=CMYKKJ0/5vsZICXPLv3GetDWZ5/nNXf8fikSV9fvaKU=; b=VmNGHd9cC/Dckz/ItyOG7wJ7ugJWE7b+gxJ7F+VoRixB6hBwniFmb/4hlto5NQqfm2PTf0 IT4agL1uTTSkTKbIBxqsQD9Cj29fOJPhgn4sWsqw21bJtpH0JCK+kZUtUvszzKojFokIeV xxyvNDTPHw7mu1tuqvrU22ueI98CwPA= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-651-0zcLQDsiOCu33GuLoDabIw-1; Tue, 07 Apr 2026 13:51:37 -0400 X-MC-Unique: 0zcLQDsiOCu33GuLoDabIw-1 X-Mimecast-MFC-AGG-ID: 0zcLQDsiOCu33GuLoDabIw_1775584296 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 2869719560B0; Tue, 7 Apr 2026 17:51:36 +0000 (UTC) Received: from hp-dl380pgen9-07.khw.eng.rdu2.dc.redhat.com (hp-dl380pgen9-07.khw.eng.rdu2.dc.redhat.com [10.6.10.143]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 8B9EC300019F; Tue, 7 Apr 2026 17:51:35 +0000 (UTC) From: Tony Camuso To: openipmi-developer@lists.sourceforge.net, linux-kernel@vger.kernel.org Cc: minyard@acm.org, tcamuso@redhat.com Subject: [PATCH 0/2] ipmi:watchdog: Fix panic, D-state hang, and lost protection on BMC reset Date: Tue, 7 Apr 2026 13:51:32 -0400 Message-ID: <20260407175134.3367345-1-tcamuso@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 When the BMC resets while the IPMI watchdog is active, the driver has three failure modes depending on timing: 1. list_add double add panic -- the watchdog daemon retries while the static smi_msg/recv_msg structures are still queued in the IPMI layer from the previous (unanswered) request. 2. D-state hang -- wait_for_completion() blocks indefinitely because the BMC never delivers a response. 3. Silent loss of watchdog protection -- the BMC returns a non-zero completion code, the driver's internal state becomes inconsistent, writes to /dev/watchdog return -EINVAL, and the daemon gives up. The system continues running without hardware watchdog coverage. All three stem from the same root cause: the static message structures and unbounded completion waits were never designed for a BMC that disappears mid-transaction. This has been independently reported by Kenta Akagi on a Dell PowerEdge R640 running 6.18.7, also triggered by a BMC reset with the watchdog active: https://sourceforge.net/p/openipmi/mailman/message/59292850/ The fix takes a simple, deterministic approach: detect the failure via BMC error response, guard against structure reuse (msg_in_flight) and indefinite waits (completion timeout), then initiate orderly_reboot() when the watchdog is active. This produces the same outcome the hardware watchdog would have -- a system reset -- but through a controlled path with clear logging and no panics or hangs. If the watchdog is stopped when the BMC resets, no reboot occurs and the system continues normally. Tested on Dell PowerEdge R640 with kernel 5.14 (RHEL 9) and verified against mainline (both patches apply cleanly). Corey Minyard's recent fix for list corruption in smi_work() (ipmi_msghandler.c) addresses a related but separate code path. The watchdog driver's own static structure reuse requires this fix. Tony Camuso (2): ipmi:watchdog: Reboot cleanly on BMC reset Documentation: ipmi: Update BMC reset behavior for watchdog Documentation/driver-api/ipmi.rst | 61 ++++++++++++++++++ drivers/char/ipmi/ipmi_watchdog.c | 101 ++++++++++++++++++++++++------ 2 files changed, 144 insertions(+), 18 deletions(-) -- 2.53.0