[PATCH RFC printk] printk: Postpone kthread startup

[PATCH RFC printk] printk: Postpone kthread startup

[John Ogness]

There have been various bug reports related to SMP systems hanging
on boot. The reports vary wildly but seem to have some overlap with
regard to missed IPI's during SMP setup. The reports also show that
reverting NBCON console usage allows the bugs to disappear.

The NBCON kthreads for console printing are started very early
during boot using early_initcall(). Since these kthreads are
actively enabling/disabling interrupts during early boot, they may
be interfering with SMP setup.

Postpone kthread starting until after all arch initcalls have
completed.

Signed-off-by: John Ogness <john.ogness@linutronix.de>
---
@pmladek: This patch came about from the RedHat reports we were talking
about at LPC. Derek is still doing final testing, but this patch most
likely fixes the issue. We probably will want this for the 7.3-hotfix in
order to preempt any other surprise bug reports.

It would be nice to start the kthreads sooner, but it is not worth
possibly hanging some sensitive (buggy?) hardware.

 kernel/printk/nbcon.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/printk/nbcon.c b/kernel/printk/nbcon.c
index f05cc90123bd4..04fdd71b37e55 100644
--- a/kernel/printk/nbcon.c
+++ b/kernel/printk/nbcon.c
@@ -1475,7 +1475,7 @@ static int __init printk_setup_threads(void)
 	console_list_unlock();
 	return 0;
 }
-early_initcall(printk_setup_threads);
+subsys_initcall(printk_setup_threads);
 
 /**
  * nbcon_alloc - Allocate buffers needed by the nbcon console

Re: [PATCH RFC printk] printk: Postpone kthread startup

[John Ogness]

On 2026-10-15, John Ogness <john.ogness@linutronix.de> wrote:
> There have been various bug reports related to SMP systems hanging
> on boot. The reports vary wildly but seem to have some overlap with
> regard to missed IPI's during SMP setup. The reports also show that
> reverting NBCON console usage allows the bugs to disappear.
>
> The NBCON kthreads for console printing are started very early
> during boot using early_initcall(). Since these kthreads are
> actively enabling/disabling interrupts during early boot, they may
> be interfering with SMP setup.
>
> Postpone kthread starting until after all arch initcalls have
> completed.

This should also contain:

Reported-by: Derek Barbosa <debarbos@redhat.com>
Closes: https://lore.kernel.org/linux-rt-users/arPmYPbFLf2EldIl@debarbos-thinkpadt14gen5.rmtusma.csb
Fixes: 5f1b414ca027 ("printk: nbcon: Start printing threads")

Re: [PATCH RFC printk] printk: Postpone kthread startup

[Derek Barbosa]

Hi all,

On Thu, Oct 15, 2026 at 03:13:23PM +0206, John Ogness wrote:
> There have been various bug reports related to SMP systems hanging
> on boot. The reports vary wildly but seem to have some overlap with
> regard to missed IPI's during SMP setup. The reports also show that
> reverting NBCON console usage allows the bugs to disappear.

FWIW, in some of those cases, the issue tends to be exacerbated by problematic
hardware. Some of the reports were coming from platforms running older microcode
that have had known issues on "complex workloads" with SMT enabled.

> 
> The NBCON kthreads for console printing are started very early
> during boot using early_initcall(). Since these kthreads are
> actively enabling/disabling interrupts during early boot, they may
> be interfering with SMP setup.
> 
> Postpone kthread starting until after all arch initcalls have
> completed.
> 
> Signed-off-by: John Ogness <john.ogness@linutronix.de>
> ---
> @pmladek: This patch came about from the RedHat reports we were talking
> about at LPC. Derek is still doing final testing, but this patch most
> likely fixes the issue. We probably will want this for the 7.3-hotfix in
> order to preempt any other surprise bug reports.

I am wrapping up my last set of trial runs here, but I haven't been able to
reproduce the issue using the "conventional" methods we've discussed. I would
just like to confirm "once and for all" some of the noise I am seeing is caused
by some network instability or whatever. I should be done with it soon.

> 
> It would be nice to start the kthreads sooner, but it is not worth
> possibly hanging some sensitive (buggy?) hardware.
> 

buggy is an appropriate word too :)

>  kernel/printk/nbcon.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/kernel/printk/nbcon.c b/kernel/printk/nbcon.c
> index f05cc90123bd4..04fdd71b37e55 100644
> --- a/kernel/printk/nbcon.c
> +++ b/kernel/printk/nbcon.c
> @@ -1475,7 +1475,7 @@ static int __init printk_setup_threads(void)
>  	console_list_unlock();
>  	return 0;
>  }
> -early_initcall(printk_setup_threads);
> +subsys_initcall(printk_setup_threads);
>  
>  /**
>   * nbcon_alloc - Allocate buffers needed by the nbcon console
> 

Cheers,

-- 
Derek <debarbos@redhat.com>