mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Breno Leitao <leitao@debian.org>
To: Corey Minyard <corey@minyard.net>
Cc: openipmi-developer@lists.sourceforge.net,
	linux-kernel@vger.kernel.org,  kernel-team@meta.com,
	Dan Melnic <dmm@meta.com>,  Breno Leitao <leitao@debian.org>
Subject: [PATCH] ipmi:kcs: Rate limit the invalid state warning
Date: Fri, 30 Oct 2026 03:29:45 -0700	[thread overview]
Message-ID: <20261030-ipmi-v1-1-f84d5fa1fa6c@debian.org> (raw)

start_kcs_transaction() warns each time it is asked to start a
transaction while the KCS state machine is busy. The warning is not
rate limited, so if the SI layer believes the interface is idle while
the state machine is still running, every message sent to the BMC logs
another line.

Kernels carrying commit c608966f3f9c ("ipmi: fix msg stack when IPMI is
disconnected") hit this. When the interface is declared hosed, the
state machine is reset to KCS_ERROR0 rather than KCS_IDLE, and keeps
walking the error states while the SI layer, which treats it as idle,
tries to start new messages.

One host logged 29K of these lines in four days, several per second:

  [Thu Oct 29 14:55:54 2026] ipmi_si dmi-ipmi-si.0: KCS in invalid state 9
  [Thu Oct 29 14:55:55 2026] ipmi_si dmi-ipmi-si.0: KCS in invalid state 7
  [Thu Oct 29 14:55:55 2026] ipmi_si dmi-ipmi-si.0: KCS in invalid state 8
  [Thu Oct 29 14:55:55 2026] ipmi_si dmi-ipmi-si.0: KCS in invalid state 9
  [Thu Oct 29 14:55:55 2026] ipmi_si dmi-ipmi-si.0: KCS in invalid state 7

Across hosts running such kernels at Meta fleet, about 887 million of
these lines were logged in one week, all for states 6 to 9 (KCS_ERROR0
to KCS_ERROR3).

That commit was reverted by commit 5d09ee1bec87 ("Revert "ipmi: fix msg
stack when IPMI is disconnected""), but nothing bounds the warning if
the two layers get out of sync again.

Use dev_warn_ratelimited(), as the SI layer already does for its
transaction warnings, given it is cheap and will protect us from this
crazy flood.

Reported-by: Dan Melnic <dmm@meta.com>
Signed-off-by: Breno Leitao <leitao@debian.org>
---
 drivers/char/ipmi/ipmi_kcs_sm.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/char/ipmi/ipmi_kcs_sm.c b/drivers/char/ipmi/ipmi_kcs_sm.c
index c89055db39e52e..ae0399ccd82196 100644
--- a/drivers/char/ipmi/ipmi_kcs_sm.c
+++ b/drivers/char/ipmi/ipmi_kcs_sm.c
@@ -271,7 +271,8 @@ static int start_kcs_transaction(struct si_sm_data *kcs, unsigned char *data,
 		return IPMI_REQ_LEN_EXCEEDED_ERR;
 
 	if ((kcs->state != KCS_IDLE) && (kcs->state != KCS_HOSED)) {
-		dev_warn(kcs->io->dev, "KCS in invalid state %d\n", kcs->state);
+		dev_warn_ratelimited(kcs->io->dev, "KCS in invalid state %d\n",
+				     kcs->state);
 		return IPMI_NOT_IN_MY_STATE_ERR;
 	}
 

---
base-commit: a39c950806213f6861d00f90729acaaa21f2d95a
base-tree: git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git master
change-id: 20261030-ipmi-a2223415ce84

Best regards,
--  
Breno Leitao <leitao@debian.org>


             reply	other threads:[~2026-10-30 10:40 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-10-30 10:29 Breno Leitao [this message]
2026-10-30 13:00 ` Corey Minyard
2026-10-30 13:19   ` Breno Leitao

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261030-ipmi-v1-1-f84d5fa1fa6c@debian.org \
    --to=leitao@debian.org \
    --cc=corey@minyard.net \
    --cc=dmm@meta.com \
    --cc=kernel-team@meta.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=openipmi-developer@lists.sourceforge.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®