[PATCH] RISC-V: Enable the IPI before workqueue_online_cpu()

[PATCH] RISC-V: Enable the IPI before workqueue_online_cpu()

[Nick Hu]

Sometimes the hotplug cpu stalls at the arch_cpu_idle() for a while after
workqueue_online_cpu(). When cpu stalls at the idle loop, the reschedule
IPI is pending. However the enable bit is not enabled yet so the cpu stalls
at WFI until watchdog timeout. Therefore enable the IPI before the
workqueue_online_cpu() to fix the issue.

Fixes: 63c5484e7495 ("workqueue: Add multiple affinity scopes and interface to select them")
Signed-off-by: Nick Hu <nick.hu@sifive.com>
---
 arch/riscv/kernel/sbi-ipi.c | 2 +-
 include/linux/cpuhotplug.h  | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/riscv/kernel/sbi-ipi.c b/arch/riscv/kernel/sbi-ipi.c
index 1026e22955cc..e05e68de8871 100644
--- a/arch/riscv/kernel/sbi-ipi.c
+++ b/arch/riscv/kernel/sbi-ipi.c
@@ -71,7 +71,7 @@ void __init sbi_ipi_init(void)
 	 * the masking/unmasking of virtual IPIs is done
 	 * via generic IPI-Mux
 	 */
-	cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
+	cpuhp_setup_state(CPUHP_AP_IRQ_RISCV_SWINT_STARTING,
 			  "irqchip/sbi-ipi:starting",
 			  sbi_ipi_starting_cpu, NULL);
 
diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
index 7a5785f405b6..57dcf1229b27 100644
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -147,6 +147,7 @@ enum cpuhp_state {
 	CPUHP_AP_IRQ_LOONGARCH_STARTING,
 	CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
 	CPUHP_AP_IRQ_RISCV_IMSIC_STARTING,
+	CPUHP_AP_IRQ_RISCV_SWINT_STARTING,
 	CPUHP_AP_ARM_MVEBU_COHERENCY,
 	CPUHP_AP_PERF_X86_AMD_UNCORE_STARTING,
 	CPUHP_AP_PERF_X86_STARTING,
-- 
2.34.1

Re: [PATCH] RISC-V: Enable the IPI before workqueue_online_cpu()

[Anup Patel]

On Tue, Jul 16, 2024 at 2:35 PM Nick Hu <nick.hu@sifive.com> wrote:
>
> Sometimes the hotplug cpu stalls at the arch_cpu_idle() for a while after
> workqueue_online_cpu(). When cpu stalls at the idle loop, the reschedule
> IPI is pending. However the enable bit is not enabled yet so the cpu stalls
> at WFI until watchdog timeout. Therefore enable the IPI before the
> workqueue_online_cpu() to fix the issue.
>
> Fixes: 63c5484e7495 ("workqueue: Add multiple affinity scopes and interface to select them")
> Signed-off-by: Nick Hu <nick.hu@sifive.com>
> ---
>  arch/riscv/kernel/sbi-ipi.c | 2 +-
>  include/linux/cpuhotplug.h  | 1 +
>  2 files changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/arch/riscv/kernel/sbi-ipi.c b/arch/riscv/kernel/sbi-ipi.c
> index 1026e22955cc..e05e68de8871 100644
> --- a/arch/riscv/kernel/sbi-ipi.c
> +++ b/arch/riscv/kernel/sbi-ipi.c
> @@ -71,7 +71,7 @@ void __init sbi_ipi_init(void)
>          * the masking/unmasking of virtual IPIs is done
>          * via generic IPI-Mux
>          */
> -       cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
> +       cpuhp_setup_state(CPUHP_AP_IRQ_RISCV_SWINT_STARTING,
>                           "irqchip/sbi-ipi:starting",
>                           sbi_ipi_starting_cpu, NULL);
>
> diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
> index 7a5785f405b6..57dcf1229b27 100644
> --- a/include/linux/cpuhotplug.h
> +++ b/include/linux/cpuhotplug.h
> @@ -147,6 +147,7 @@ enum cpuhp_state {
>         CPUHP_AP_IRQ_LOONGARCH_STARTING,
>         CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
>         CPUHP_AP_IRQ_RISCV_IMSIC_STARTING,
> +       CPUHP_AP_IRQ_RISCV_SWINT_STARTING,

This is a misleading name.

I suggest s/_RISCV_SWINT_/_RISCV_SBI_IPI_/ .

>         CPUHP_AP_ARM_MVEBU_COHERENCY,
>         CPUHP_AP_PERF_X86_AMD_UNCORE_STARTING,
>         CPUHP_AP_PERF_X86_STARTING,
> --
> 2.34.1
>

Regards,
Anup

Re: [PATCH] RISC-V: Enable the IPI before workqueue_online_cpu()

[Nick Hu]

Hi Anup

On Tue, Jul 16, 2024 at 6:51 PM Anup Patel <anup@brainfault.org> wrote:
>
> On Tue, Jul 16, 2024 at 2:35 PM Nick Hu <nick.hu@sifive.com> wrote:
> >
> > Sometimes the hotplug cpu stalls at the arch_cpu_idle() for a while after
> > workqueue_online_cpu(). When cpu stalls at the idle loop, the reschedule
> > IPI is pending. However the enable bit is not enabled yet so the cpu stalls
> > at WFI until watchdog timeout. Therefore enable the IPI before the
> > workqueue_online_cpu() to fix the issue.
> >
> > Fixes: 63c5484e7495 ("workqueue: Add multiple affinity scopes and interface to select them")
> > Signed-off-by: Nick Hu <nick.hu@sifive.com>
> > ---
> >  arch/riscv/kernel/sbi-ipi.c | 2 +-
> >  include/linux/cpuhotplug.h  | 1 +
> >  2 files changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/riscv/kernel/sbi-ipi.c b/arch/riscv/kernel/sbi-ipi.c
> > index 1026e22955cc..e05e68de8871 100644
> > --- a/arch/riscv/kernel/sbi-ipi.c
> > +++ b/arch/riscv/kernel/sbi-ipi.c
> > @@ -71,7 +71,7 @@ void __init sbi_ipi_init(void)
> >          * the masking/unmasking of virtual IPIs is done
> >          * via generic IPI-Mux
> >          */
> > -       cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
> > +       cpuhp_setup_state(CPUHP_AP_IRQ_RISCV_SWINT_STARTING,
> >                           "irqchip/sbi-ipi:starting",
> >                           sbi_ipi_starting_cpu, NULL);
> >
> > diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
> > index 7a5785f405b6..57dcf1229b27 100644
> > --- a/include/linux/cpuhotplug.h
> > +++ b/include/linux/cpuhotplug.h
> > @@ -147,6 +147,7 @@ enum cpuhp_state {
> >         CPUHP_AP_IRQ_LOONGARCH_STARTING,
> >         CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
> >         CPUHP_AP_IRQ_RISCV_IMSIC_STARTING,
> > +       CPUHP_AP_IRQ_RISCV_SWINT_STARTING,
>
> This is a misleading name.
>
> I suggest s/_RISCV_SWINT_/_RISCV_SBI_IPI_/ .
>
Good point.
I'll update it in the next version.
Thank you for the feedback.

> >        CPUHP_AP_ARM_MVEBU_COHERENCY,
> >         CPUHP_AP_PERF_X86_AMD_UNCORE_STARTING,
> >         CPUHP_AP_PERF_X86_STARTING,
> > --
> > 2.34.1
> >
>
> Regards,
> Anup

Regards,
Nick