[PATCH] printk: fold consecutive duplicate messages

[PATCH] printk: fold consecutive duplicate messages

[林濬哲]

A kernel bug can flood the console with thousands of copies of the
same message, drowning out everything else. Fold consecutive
duplicates into a single "last message repeated N times" summary,
flushed when 10 repeats accumulate or a 1s window elapses.

The dedup key is built from facility, level and the format string
address; format parameters are not part of the key since va_list can
only be consumed once and hashing rendered text would put string
comparisons on the printk fast path.

Messages at LOGLEVEL_ERR and above are never folded, and dedup is
skipped after suppress_printk / panic take effect since the call site
sits behind those checks. The dedup state is protected by a raw
spinlock, and the summary is printed only after dropping the lock to
avoid recursive self-deadlock.

Disable with printk_dedup=0 on the kernel command line or via the
printk_dedup module parameter.

Known limitation: dedup_lock is not NMI safe; a message from NMI
context while another CPU holds the lock will spin.

Signed-off-by: 林濬哲 <m18667909625@163.com>
Assisted-by: AI coding assistant (disclosed per kernel AI guidelines)
---
 kernel/printk/printk.c | 89 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 89 insertions(+)

diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index 6d3d18a50da7..0c93767d041f 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -20,6 +20,7 @@
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
 
 #include <linux/kernel.h>
+#include <linux/hash.h>
 #include <linux/mm.h>
 #include <linux/tty.h>
 #include <linux/tty_driver.h>
@@ -104,6 +105,91 @@ DEFINE_STATIC_SRCU(console_srcu);
  */
 int __read_mostly suppress_printk;
 
+static u32 dedup_last_key;
+static u32 dedup_repeat;
+static bool dedup_active;
+static u64 dedup_window_start;
+#define PRINTK_DEDUP_WINDOW_NS	1000000000ULL	/* 1s */
+static bool printk_dedup = true;
+module_param(printk_dedup, bool, 0644);
+MODULE_PARM_DESC(printk_dedup, "fold consecutive duplicate printk messages");
+/* protects the dedup state above; never held while printing */
+static DEFINE_RAW_SPINLOCK(dedup_lock);
+
+static int __init printk_dedup_setup(char *str)
+{
+	return kstrtobool(str, &printk_dedup);
+}
+early_param("printk_dedup", printk_dedup_setup);
+
+/*
+ * Detect consecutive duplicate printk messages and fold them away.
+ * Returns true if this message should be dropped.
+ *
+ * The dedup key is built from facility, level and the format string.
+ * Format parameters are intentionally not part of the key: va_list
+ * can only be consumed once, and hashing rendered text would put
+ * string comparisons on the printk fast path.
+ */
+static bool outputs_dedupe(int facility, int level, const char *fmt)
+{
+	unsigned long flags;
+	bool drop;
+	u64 now;
+	u32 key;
+	u32 n;
+
+	/* error and above must never be folded */
+	if (level <= LOGLEVEL_ERR || !printk_dedup)
+		return false;
+
+	key = hash_64((unsigned long)(fmt ? : ""), 32) ^
+	      hash_32(facility ^ level, 32);
+	now = ktime_get_ns();
+
+	raw_spin_lock_irqsave(&dedup_lock, flags);
+
+	if (dedup_active && key == dedup_last_key) {
+		dedup_repeat++;
+		/*
+		 * Force a flush when 10 repeats have accumulated or
+		 * the 1s window has elapsed, so the summary shows up.
+		 */
+		if (dedup_repeat >= 10 ||
+		    now - dedup_window_start > PRINTK_DEDUP_WINDOW_NS) {
+			n = dedup_repeat;
+			dedup_repeat = 0;
+			dedup_window_start = now;
+			drop = false;
+		} else {
+			drop = true;
+		}
+	} else {
+		if (dedup_active && dedup_repeat) {
+			n = dedup_repeat;
+			drop = false;
+		} else {
+			n = 0;
+			drop = false;
+		}
+		dedup_last_key = key;
+		dedup_repeat = 0;
+		dedup_window_start = now;
+		dedup_active = true;
+	}
+
+	raw_spin_unlock_irqrestore(&dedup_lock, flags);
+
+	/*
+	 * Print only after dropping the lock: pr_info() re-enters
+	 * vprintk_emit() and would deadlock on dedup_lock otherwise.
+	 */
+	if (n)
+		pr_info("last message repeated %u times\n", n);
+
+	return drop;
+}
+
 #ifdef CONFIG_LOCKDEP
 static struct lockdep_map console_lock_dep_map = {
 	.name = "console_lock"
@@ -2441,6 +2527,9 @@ asmlinkage int vprintk_emit(int facility, int level,
 	    !panic_triggering_all_cpu_backtrace)
 		return 0;
 
+	if (outputs_dedupe(facility, level, fmt))
+		return 0;
+
 	printk_get_console_flush_type(&ft);
 
 	/* If called from the scheduler, we can not call up(). */
-- 
2.53.0

Re: [PATCH] printk: fold consecutive duplicate messages

[John Ogness]

Hi,

On 2026-09-21, 林濬哲 <m18667909625@163.com> wrote:
> A kernel bug can flood the console with thousands of copies of the
> same message, drowning out everything else. Fold consecutive
> duplicates into a single "last message repeated N times" summary,
> flushed when 10 repeats accumulate or a 1s window elapses.

Do you have an example of the _same_ message flooding the console? This
does not sound like a real-world case.

> Known limitation: dedup_lock is not NMI safe; a message from NMI
> context while another CPU holds the lock will spin.

printk _is_ NMI safe _and_ lockless. It needs to stay that way.

John Ogness

Re: [PATCH] printk: fold consecutive duplicate messages

[Lin Junzhe]

Hi John,

Thanks for the review.

> Could you provide some examples of consoles being flooded with
> the same message? Doesn't sound like a real case to me.

Real cases I have seen in the field and in bug reports:

1. GPU faults (nouveau): a misbehaving userspace program or a dying
   GPU can trigger a stream of identical fault reports from the
   in-tree nouveau driver, e.g. repeated "fifo: fault at ..." lines
   while the offending context keeps being rescheduled. Similar
   spam exists for other GPU drivers when a fence or scheduler
   loop misbehaves.

2. Failing storage: a dying SATA disk produces endless identical
   "ata1.00: failed command" / "ata1: SError" storms. This is a
   classic dmesg flood that buries everything else on machines
   with a serial console and no syslogd.

3. USB reset loops: a flaky cable or port makes the USB stack
   repeatedly print the identical "usb X-Y: reset <speed> USB
   device number N using <hcd>" line, sometimes for minutes.

4. IRQ storms: an unhandled level-triggered interrupt prints the
   identical "irq N: nobody cared" report for every retrigger
   until the IRQ is disabled.

In all four cases the messages come from kernel-side ratelimits or
error paths that repeat at line rate, and on systems where the only
observable output is the kernel console (serial console, early boot,
netconsole), no userspace dedup exists to save the log.

> printk is NMI safe and lockless. It needs to remain so.

Fully agreed, and thank you for the clear statement. My
implementation takes a raw spinlock in vprintk_emit(), which
violates exactly that invariant -- the in_nmi() guard only avoids
the deadlock by disabling the feature where it would be most
dangerous, which is not acceptable either.

Given this, I see two options:

a) I drop the patch entirely; or

b) I rework the idea as lockless per-CPU/per-console state at the
   console output layer (or on top of nbcon), so the printk
   fast path stays lock- and NMI-safe.

Please tell me whether (b) is worth exploring or whether the
consensus is that deduplication belongs in userspace and (a) is
the right outcome. Either way is fine with me.

Best regards,
林濬哲

--
Assisted-by: AI coding assistant (disclosed per kernel AI guidelines)

Re: [PATCH] printk: fold consecutive duplicate messages

[John Ogness]

Hi Lin,

On 2026-09-21, Lin Junzhe <m18667909625@163.com> wrote:
> Real cases I have seen in the field and in bug reports:
>
> 1. GPU faults (nouveau): a misbehaving userspace program or a dying
>    GPU can trigger a stream of identical fault reports from the
>    in-tree nouveau driver, e.g. repeated "fifo: fault at ..." lines
>    while the offending context keeps being rescheduled. Similar
>    spam exists for other GPU drivers when a fence or scheduler
>    loop misbehaves.

There are several such "fault at" messages. However, the ones I looked
at have other printk messages following in the same context, so your
patch would not even help in these cases.

Please specify the exact message (file + line number) you are talking
about. Perhaps it would be enough to change it to use a printk
ratelimited variant.

> 2. Failing storage: a dying SATA disk produces endless identical
>    "ata1.00: failed command" / "ata1: SError" storms. This is a
>    classic dmesg flood that buries everything else on machines
>    with a serial console and no syslogd.

The "failed command" ata printk is also followed by further printk's, so
your patch would not help.

Please specify the exact message you are concerned about.

> 3. USB reset loops: a flaky cable or port makes the USB stack
>    repeatedly print the identical "usb X-Y: reset <speed> USB
>    device number N using <hcd>" line, sometimes for minutes.

I could not find this pattern. Please specify the exact message.

> 4. IRQ storms: an unhandled level-triggered interrupt prints the
>    identical "irq N: nobody cared" report for every retrigger
>    until the IRQ is disabled.

This message also follows with more messages, so your patch would not
help.

>> printk is NMI safe and lockless. It needs to remain so.
>
> Fully agreed, and thank you for the clear statement. My
> implementation takes a raw spinlock in vprintk_emit(), which
> violates exactly that invariant -- the in_nmi() guard only avoids
> the deadlock by disabling the feature where it would be most
> dangerous, which is not acceptable either.
>
> Given this, I see two options:
>
> a) I drop the patch entirely; or
>
> b) I rework the idea as lockless per-CPU/per-console state at the
>    console output layer (or on top of nbcon), so the printk
>    fast path stays lock- and NMI-safe.
>
> Please tell me whether (b) is worth exploring or whether the
> consensus is that deduplication belongs in userspace and (a) is
> the right outcome. Either way is fine with me.

I am against a patch that drops messages just because a format string
repeats. The _data_ is not the same and that is important (particularly
with your GPU and SATA examples).

I am also skeptical that these are real-world issues as all of your
examples (that I could find) had different printk messages following,
which would lead to no drops.

There is also the ratelimited variant of printk. If there are indeed
messages that are not useful and can flood the kernel log, perhaps those
messages should be either removed or ratelimited.

If such a feature were to exist, I would prefer it is implemented such
that:

1. A duplicate message means contents are identical (except for the
timestamp of course).

2. It is implemented using flows similar to LOG_CONT to be certain that
the message being dropped is really the next message.

3. Records could be extended to include a counter for how often they
repeat (so that deferred consoles can print the "repeated" line).

Honestly, I do not see a real value for this feature. In my experience,
even if a console is being flooded with messages, I still want all those
messages. If a console is unable to keep up with the flood of incoming
records, I need to use a faster console and/or reduce my console
loglevel. And if there really are printk messages that can flood the
kernel log and are useless when repeated output, they should be make to
use the once or ratelimited variants.

John

Re: [PATCH] printk: fold consecutive duplicate messages

[Lin Junzhe]

Hi John,

Thanks for the detailed follow-up, and for taking the time to check
each of my examples against the actual source.

You are right on all counts. In every case I cited, the repeated
format string is interleaved with other messages, so a "consecutive
duplicate" check would not have helped. More fundamentally, your
point that "same format string does not mean same data" exposes a
real flaw in my implementation: the dedup key hashes only the fmt
pointer and facility/level, not the arguments (a va_list can only be
consumed once), so two messages with different argument values would
be wrongly folded. That alone makes the patch incorrect for exactly
the scenarios it was meant to address.

I also accept the broader point: a message that is spammy enough to
flood the console should be fixed at its source with a ratelimited
or once-per-event variant, rather than papered over in printk.

So I am withdrawing this patch:

  https://lore.kernel.org/lkml/20260921050304.73440-1-m18667909625@163.com/

Thank you and Petr for the reviews -- the "printk must remain lockless
and NMI-safe" invariant, and the guidance on where a feature like this
would have to live (store phase, exact-content match, LOG_CONT-style
ordering) were valuable lessons.

Best regards,
Lin Junzhe

--
Assisted-by: AI coding assistant (disclosed per kernel AI guidelines)