From: Corey Minyard <corey@minyard.net>
To: Breno Leitao <leitao@debian.org>
Cc: openipmi-developer@lists.sourceforge.net,
linux-kernel@vger.kernel.org, kernel-team@meta.com,
Dan Melnic <dmm@meta.com>
Subject: Re: [PATCH] ipmi:kcs: Rate limit the invalid state warning
Date: Fri, 30 Oct 2026 08:00:06 -0500 [thread overview]
Message-ID: <auSU1tHoDTrIkHoR@mail.minyard.net> (raw)
In-Reply-To: <20261030-ipmi-v1-1-f84d5fa1fa6c@debian.org>
On Fri, Oct 30, 2026 at 03:29:45AM -0700, Breno Leitao wrote:
> start_kcs_transaction() warns each time it is asked to start a
> transaction while the KCS state machine is busy. The warning is not
> rate limited, so if the SI layer believes the interface is idle while
> the state machine is still running, every message sent to the BMC logs
> another line.
>
> Kernels carrying commit c608966f3f9c ("ipmi: fix msg stack when IPMI is
> disconnected") hit this. When the interface is declared hosed, the
> state machine is reset to KCS_ERROR0 rather than KCS_IDLE, and keeps
> walking the error states while the SI layer, which treats it as idle,
> tries to start new messages.
>
> One host logged 29K of these lines in four days, several per second:
>
> [Thu Oct 29 14:55:54 2026] ipmi_si dmi-ipmi-si.0: KCS in invalid state 9
> [Thu Oct 29 14:55:55 2026] ipmi_si dmi-ipmi-si.0: KCS in invalid state 7
> [Thu Oct 29 14:55:55 2026] ipmi_si dmi-ipmi-si.0: KCS in invalid state 8
> [Thu Oct 29 14:55:55 2026] ipmi_si dmi-ipmi-si.0: KCS in invalid state 9
> [Thu Oct 29 14:55:55 2026] ipmi_si dmi-ipmi-si.0: KCS in invalid state 7
The rate limiting is a good idea. I've included the patch.
However, it also sounds like you don't have working IPMI hardware on
these systems. The change you mention just exacerbates the problem
here, but I think it would be a good idea to track down why this is
happening.
If you really have working IPMI hardware, then it would be a good
idea to trace down why this is happening.
If you don't have working IPMI hardware, then it sounds like your
system is advertising that it has good hardware through SMBIOS or
ACPI. It would be good to either fix your firmware or add a deny
for this particular system.
In either case, this points to a bug in the driver; it shouldn't try to
start a transaction while in error state. I can't see it, but I believe
I can reproduce in simulation.
What specific kernel version are you running?
BTW, I need a link to the discussion about this so I can add a Closes:
tag to the git change.
Thanks,
-corey
>
> Across hosts running such kernels at Meta fleet, about 887 million of
> these lines were logged in one week, all for states 6 to 9 (KCS_ERROR0
> to KCS_ERROR3).
>
> That commit was reverted by commit 5d09ee1bec87 ("Revert "ipmi: fix msg
> stack when IPMI is disconnected""), but nothing bounds the warning if
> the two layers get out of sync again.
>
> Use dev_warn_ratelimited(), as the SI layer already does for its
> transaction warnings, given it is cheap and will protect us from this
> crazy flood.
>
> Reported-by: Dan Melnic <dmm@meta.com>
> Signed-off-by: Breno Leitao <leitao@debian.org>
> ---
> drivers/char/ipmi/ipmi_kcs_sm.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/char/ipmi/ipmi_kcs_sm.c b/drivers/char/ipmi/ipmi_kcs_sm.c
> index c89055db39e52e..ae0399ccd82196 100644
> --- a/drivers/char/ipmi/ipmi_kcs_sm.c
> +++ b/drivers/char/ipmi/ipmi_kcs_sm.c
> @@ -271,7 +271,8 @@ static int start_kcs_transaction(struct si_sm_data *kcs, unsigned char *data,
> return IPMI_REQ_LEN_EXCEEDED_ERR;
>
> if ((kcs->state != KCS_IDLE) && (kcs->state != KCS_HOSED)) {
> - dev_warn(kcs->io->dev, "KCS in invalid state %d\n", kcs->state);
> + dev_warn_ratelimited(kcs->io->dev, "KCS in invalid state %d\n",
> + kcs->state);
> return IPMI_NOT_IN_MY_STATE_ERR;
> }
>
>
> ---
> base-commit: a39c950806213f6861d00f90729acaaa21f2d95a
> base-tree: git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git master
> change-id: 20261030-ipmi-a2223415ce84
>
> Best regards,
> --
> Breno Leitao <leitao@debian.org>
>
next prev parent reply other threads:[~2026-10-30 13:00 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-10-30 10:29 Breno Leitao
2026-10-30 13:00 ` Corey Minyard [this message]
2026-10-30 13:19 ` Breno Leitao
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=auSU1tHoDTrIkHoR@mail.minyard.net \
--to=corey@minyard.net \
--cc=dmm@meta.com \
--cc=kernel-team@meta.com \
--cc=leitao@debian.org \
--cc=linux-kernel@vger.kernel.org \
--cc=openipmi-developer@lists.sourceforge.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®