mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: John Ogness <john.ogness@linutronix.de>
To: Petr Mladek <pmladek@suse.com>
Cc: Sergey Senozhatsky <senozhatsky@chromium.org>,
	Steven Rostedt <rostedt@goodmis.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	linux-kernel@vger.kernel.org,
	Francesco Dolcini <francesco@dolcini.it>,
	kernel test robot <oliver.sang@intel.com>,
	Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Subject: Re: [PATCH printk v3 02/14] printk: Adjust mapping for 32bit seq macros
Date: Mon, 15 Jan 2024 11:58:39 +0106	[thread overview]
Message-ID: <87a5p6g96w.fsf@jogness.linutronix.de> (raw)
In-Reply-To: <ZaGBlDTuQ0hYBpeJ@alley>

On 2024-01-12, Petr Mladek <pmladek@suse.com> wrote:
>> > Reported-by: Francesco Dolcini <francesco@dolcini.it>
>> > Reported-by: kernel test robot <oliver.sang@intel.com>
>> > Closes: https://lore.kernel.org/oe-lkp/202311171611.78d41dbe-oliver.sang@intel.com
>> > Reported-by: kernel test robot <oliver.sang@intel.com>
>> > Closes: https://lore.kernel.org/oe-lkp/202311161555.3ee16fc9-oliver.sang@intel.com
>> > Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
>> > Signed-off-by: John Ogness <john.ogness@linutronix.de>
>> 
>> Great catch! It must have been complicated to debug this.
>> 
>> Reviewed-by: Petr Mladek <pmladek@suse.com>
>
> That said, I am a bit nervous that a bug like this might cause
> workqueue stall and panic() the kernel.
>
> At least, this is how I read
> https://lore.kernel.org/oe-lkp/202311171611.78d41dbe-oliver.sang@intel.com/

Note that the bug is reported for code that mainline will never
have. This patch "fixes" the problem before it is a problem. From the
perspective of mainline, the problem never existed. Perhaps it is
inappropriate to list these with the Closes tag.

> It looks like it caused some loop and refcout overlow or so.
> But I might be wrong.
>
> I would like to better understand this and check if we could prevent
> it somehow.

The code in question can be found here:

https://git.kernel.org/pub/scm/linux/kernel/git/rt/linux-rt-devel.git/tree/kernel/printk/printk_ringbuffer.c?id=768c33033e41ad195a9304ebb183afb730d5ae98#n2031

The URL above points to prb_next_reserve_seq(), which is in an infinite
loop.

desc_last_finalized_seq() is returning a huge garbage value due to the
32bit sequence value 1 being incorrectly mapped to 0xffffffff00000001
(which is 18446744069414584321). When desc_read_finalized_seq() is
called (line 2082), it obviously returns -EINVAL because there is no
record with this huge sequence number.

The code interrupts -EINVAL to mean that the record has been
overwritten, and so it tries again with an updated last finalized
sequence number => infinite loop.

With this patch applied, the 32bit value 1 is correctly mapped to
sequence number 1 and all is OK.

The problem was that desc_last_finalized_seq() returned a sequence
number that was higher than the highest record. That was the bug. As
long as desc_last_finalized_seq() always returns a sequence number that
exists or used to exist, the retry loop is fine. And since the tail
record is always in the finalized state, there will always be a
finalized record available.

John

  parent reply	other threads:[~2024-01-15 10:52 UTC|newest]

Thread overview: 43+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-12-14 21:41 [PATCH printk v3 00/14] fix console flushing John Ogness
2023-12-14 21:41 ` [PATCH printk v3 01/14] printk: nbcon: Relocate 32bit seq macros John Ogness
2024-01-12 10:14   ` Petr Mladek
2023-12-14 21:41 ` [PATCH printk v3 02/14] printk: Adjust mapping for " John Ogness
2023-12-15  9:55   ` Sebastian Andrzej Siewior
2023-12-15 10:10     ` John Ogness
2023-12-15 10:58       ` Sebastian Andrzej Siewior
2024-01-12 10:28   ` Petr Mladek
2024-01-12 18:14     ` Petr Mladek
2024-01-15  8:51       ` Sebastian Andrzej Siewior
2024-01-15 10:52       ` John Ogness [this message]
2024-01-15 16:17         ` Petr Mladek
2024-01-15 17:08           ` John Ogness
2023-12-14 21:41 ` [PATCH printk v3 03/14] printk: Use prb_first_seq() as base " John Ogness
2024-01-12 16:19   ` Petr Mladek
2023-12-14 21:41 ` [PATCH printk v3 04/14] printk: ringbuffer: Do not skip non-finalized records with prb_next_seq() John Ogness
2024-01-12 18:05   ` Petr Mladek
2024-01-15 11:55     ` John Ogness
2024-01-15 17:00       ` Petr Mladek
2024-02-05 11:33         ` John Ogness
2024-02-06 17:27           ` Petr Mladek
2023-12-14 21:41 ` [PATCH printk v3 05/14] printk: ringbuffer: Clarify special lpos values John Ogness
2024-01-30 13:12   ` Petr Mladek
2023-12-14 21:41 ` [PATCH printk v3 06/14] printk: For @suppress_panic_printk check for other CPU in panic John Ogness
2023-12-14 21:41 ` [PATCH printk v3 07/14] printk: Add this_cpu_in_panic() John Ogness
2023-12-14 21:41 ` [PATCH printk v3 08/14] printk: ringbuffer: Cleanup reader terminology John Ogness
2024-01-30 14:36   ` Petr Mladek
2023-12-14 21:41 ` [PATCH printk v3 09/14] printk: Wait for all reserved records with pr_flush() John Ogness
2024-01-31 11:36   ` Petr Mladek
2024-02-05 13:33     ` John Ogness
2024-02-07  9:20       ` Petr Mladek
2023-12-14 21:41 ` [PATCH printk v3 10/14] printk: ringbuffer: Skip non-finalized records in panic John Ogness
2024-02-01 16:56   ` Petr Mladek
2023-12-14 21:41 ` [PATCH printk v3 11/14] printk: ringbuffer: Consider committed as finalized " John Ogness
2024-02-01 18:00   ` Petr Mladek
2024-02-05 14:08     ` John Ogness
2024-02-07 10:11       ` Petr Mladek
2023-12-14 21:41 ` [PATCH printk v3 12/14] printk: Disable passing console lock owner completely during panic() John Ogness
2023-12-14 21:42 ` [PATCH printk v3 13/14] printk: Avoid non-panic CPUs writing to ringbuffer John Ogness
2024-02-02  9:26   ` Petr Mladek
2023-12-14 21:42 ` [PATCH printk v3 14/14] panic: Flush kernel log buffer at the end John Ogness
2024-02-02  9:30   ` Petr Mladek
2024-02-02  9:38 ` [PATCH printk v3 00/14] fix console flushing Petr Mladek

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87a5p6g96w.fsf@jogness.linutronix.de \
    --to=john.ogness@linutronix.de \
    --cc=bigeasy@linutronix.de \
    --cc=francesco@dolcini.it \
    --cc=linux-kernel@vger.kernel.org \
    --cc=oliver.sang@intel.com \
    --cc=pmladek@suse.com \
    --cc=rostedt@goodmis.org \
    --cc=senozhatsky@chromium.org \
    --cc=tglx@linutronix.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

Powered by JetHome