[PATCH v2] scsi: ufs: core: Ensure clk_gating.lock is used only after initialization

[PATCH v2] scsi: ufs: core: Ensure clk_gating.lock is used only after initialization

[Avri Altman]

This commit addresses a lockdep warning triggered by the use of the
clk_gating.lock before it is properly initialized. The warning is as
follows:

[    4.388838] INFO: trying to register non-static key.
[    4.395673] The code is fine but needs lockdep annotation, or maybe
[    4.402118] you didn't initialize this object before use?
[    4.407673] turning off the locking correctness validator.
[    4.413334] CPU: 5 UID: 0 PID: 58 Comm: kworker/u32:1 Not tainted 6.12-rc1 #185
[    4.413343] Hardware name: Qualcomm Technologies, Inc. Robotics RB5 (DT)
[    4.413362] Call trace:
[    4.413364]  show_stack+0x18/0x24 (C)
[    4.413374]  dump_stack_lvl+0x90/0xd0
[    4.413384]  dump_stack+0x18/0x24
[    4.413392]  register_lock_class+0x498/0x4a8
[    4.413400]  __lock_acquire+0xb4/0x1b90
[    4.413406]  lock_acquire+0x114/0x310
[    4.413413]  _raw_spin_lock_irqsave+0x60/0x88
[    4.413423]  ufshcd_setup_clocks+0x2c0/0x490
[    4.413433]  ufshcd_init+0x198/0x10ec
[    4.413437]  ufshcd_pltfrm_init+0x600/0x7c0
[    4.413444]  ufs_qcom_probe+0x20/0x58
[    4.413449]  platform_probe+0x68/0xd8
[    4.413459]  really_probe+0xbc/0x268
[    4.413466]  __driver_probe_device+0x78/0x12c
[    4.413473]  driver_probe_device+0x40/0x11c
[    4.413481]  __device_attach_driver+0xb8/0xf8
[    4.413489]  bus_for_each_drv+0x84/0xe4
[    4.413495]  __device_attach+0xfc/0x18c
[    4.413502]  device_initial_probe+0x14/0x20
[    4.413510]  bus_probe_device+0xb0/0xb4
[    4.413517]  deferred_probe_work_func+0x8c/0xc8
[    4.413524]  process_scheduled_works+0x250/0x658
[    4.413534]  worker_thread+0x15c/0x2c8
[    4.413542]  kthread+0x134/0x200
[    4.413550]  ret_from_fork+0x10/0x20

To fix this issue, ensure that the spinlock is only used after it
has been properly initialized before using it in `ufshcd_setup_clocks`.

Fixes: 209f4e43b806 ("scsi: ufs: core: Introduce a new clock_gating lock")
Reported-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Signed-off-by: Avri Altman <avri.altman@wdc.com>

---
Changes since v1:
 - move the spin_lock_init(&hba->clk_gating.lock) call from
   ufshcd_init_clk_gating() just before the ufshcd_hba_init() call in
   ufshcd_init() (Bart)
---
 drivers/ufs/core/ufshcd.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c
index f6c38cf10382..9b17f6e2f807 100644
--- a/drivers/ufs/core/ufshcd.c
+++ b/drivers/ufs/core/ufshcd.c
@@ -2120,8 +2120,6 @@ static void ufshcd_init_clk_gating(struct ufs_hba *hba)
 	INIT_DELAYED_WORK(&hba->clk_gating.gate_work, ufshcd_gate_work);
 	INIT_WORK(&hba->clk_gating.ungate_work, ufshcd_ungate_work);
 
-	spin_lock_init(&hba->clk_gating.lock);
-
 	hba->clk_gating.clk_gating_workq = alloc_ordered_workqueue(
 		"ufs_clk_gating_%d", WQ_MEM_RECLAIM | WQ_HIGHPRI,
 		hba->host->host_no);
@@ -10412,6 +10410,13 @@ int ufshcd_init(struct ufs_hba *hba, void __iomem *mmio_base, unsigned int irq)
 	hba->irq = irq;
 	hba->vps = &ufs_hba_vps;
 
+	/*
+	 * Initialize clk_gating.lock early since it is being used in
+	 * ufshcd_setup_clocks()
+	 */
+	if (ufshcd_is_clkgating_allowed(hba))
+		spin_lock_init(&hba->clk_gating.lock);
+
 	err = ufshcd_hba_init(hba);
 	if (err)
 		goto out_error;
-- 
2.25.1

Re: [PATCH v2] scsi: ufs: core: Ensure clk_gating.lock is used only after initialization

[Geert Uytterhoeven]

Hi Avri,

On Fri, Jan 24, 2025 at 8:36 AM Avri Altman <avri.altman@wdc.com> wrote:
> This commit addresses a lockdep warning triggered by the use of the
> clk_gating.lock before it is properly initialized. The warning is as
> follows:
>
> [    4.388838] INFO: trying to register non-static key.
> [    4.395673] The code is fine but needs lockdep annotation, or maybe
> [    4.402118] you didn't initialize this object before use?
> [    4.407673] turning off the locking correctness validator.
> [    4.413334] CPU: 5 UID: 0 PID: 58 Comm: kworker/u32:1 Not tainted 6.12-rc1 #185
> [    4.413343] Hardware name: Qualcomm Technologies, Inc. Robotics RB5 (DT)
> [    4.413362] Call trace:
> [    4.413364]  show_stack+0x18/0x24 (C)
> [    4.413374]  dump_stack_lvl+0x90/0xd0
> [    4.413384]  dump_stack+0x18/0x24
> [    4.413392]  register_lock_class+0x498/0x4a8
> [    4.413400]  __lock_acquire+0xb4/0x1b90
> [    4.413406]  lock_acquire+0x114/0x310
> [    4.413413]  _raw_spin_lock_irqsave+0x60/0x88
> [    4.413423]  ufshcd_setup_clocks+0x2c0/0x490
> [    4.413433]  ufshcd_init+0x198/0x10ec
> [    4.413437]  ufshcd_pltfrm_init+0x600/0x7c0
> [    4.413444]  ufs_qcom_probe+0x20/0x58
> [    4.413449]  platform_probe+0x68/0xd8
> [    4.413459]  really_probe+0xbc/0x268
> [    4.413466]  __driver_probe_device+0x78/0x12c
> [    4.413473]  driver_probe_device+0x40/0x11c
> [    4.413481]  __device_attach_driver+0xb8/0xf8
> [    4.413489]  bus_for_each_drv+0x84/0xe4
> [    4.413495]  __device_attach+0xfc/0x18c
> [    4.413502]  device_initial_probe+0x14/0x20
> [    4.413510]  bus_probe_device+0xb0/0xb4
> [    4.413517]  deferred_probe_work_func+0x8c/0xc8
> [    4.413524]  process_scheduled_works+0x250/0x658
> [    4.413534]  worker_thread+0x15c/0x2c8
> [    4.413542]  kthread+0x134/0x200
> [    4.413550]  ret_from_fork+0x10/0x20
>
> To fix this issue, ensure that the spinlock is only used after it
> has been properly initialized before using it in `ufshcd_setup_clocks`.
>
> Fixes: 209f4e43b806 ("scsi: ufs: core: Introduce a new clock_gating lock")
> Reported-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
> Signed-off-by: Avri Altman <avri.altman@wdc.com>
>
> ---
> Changes since v1:
>  - move the spin_lock_init(&hba->clk_gating.lock) call from
>    ufshcd_init_clk_gating() just before the ufshcd_hba_init() call in
>    ufshcd_init() (Bart)

Thanks for the update!

On Renesas R-Car S4 Starter Kit:

    BUG: spinlock bad magic on CPU#6, swapper/0/1
     lock: 0xffffff84443014e8, .magic: 00000000, .owner: <none>/-1,
.owner_cpu: 0
    CPU: 6 UID: 0 PID: 1 Comm: swapper/0 Not tainted
6.13.0-rcar3-initrd-08318-g75abbef32a94 #896
    Hardware name: R-Car S4 Starter Kit board (DT)
    Call trace:
     show_stack+0x18/0x24 (C)
     dump_stack_lvl+0x60/0x80
     dump_stack+0x18/0x24
     spin_bug+0x7c/0xa0
     do_raw_spin_lock+0x34/0xb4
     _raw_spin_lock_irqsave+0x1c/0x30
     class_spinlock_irqsave_constructor+0x18/0x30
     ufshcd_setup_clocks+0x98/0x23c
     ufshcd_init+0x288/0xd38
     ufshcd_pltfrm_init+0x618/0x738
     ufs_renesas_probe+0x18/0x24
     platform_probe+0x68/0xb8
     really_probe+0x138/0x268
     __driver_probe_device+0xf4/0x10c
     driver_probe_device+0x3c/0xf8
     __driver_attach+0xf0/0x100
     bus_for_each_dev+0x84/0xdc
     driver_attach+0x24/0x30
     bus_add_driver+0xe8/0x1dc
     driver_register+0xbc/0xf8
     __platform_driver_register+0x24/0x30
     ufs_renesas_platform_init+0x1c/0x28
     do_one_initcall+0x84/0x1f4
     kernel_init_freeable+0x238/0x23c
     kernel_init+0x20/0x120
     ret_from_fork+0x10/0x20

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

RE: [PATCH v2] scsi: ufs: core: Ensure clk_gating.lock is used only after initialization

[Avri Altman]

> >
> > To fix this issue, ensure that the spinlock is only used after it has
> > been properly initialized before using it in `ufshcd_setup_clocks`.
> >
> > Fixes: 209f4e43b806 ("scsi: ufs: core: Introduce a new clock_gating
> > lock")
> > Reported-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
> > Signed-off-by: Avri Altman <avri.altman@wdc.com>
> >
> > ---
> > Changes since v1:
> >  - move the spin_lock_init(&hba->clk_gating.lock) call from
> >    ufshcd_init_clk_gating() just before the ufshcd_hba_init() call in
> >    ufshcd_init() (Bart)
> 
> Thanks for the update!
> 
> On Renesas R-Car S4 Starter Kit:
Thanks.  This is not happening on my RB5.
I need some time to get a Renesas R-Car platform.
Meanwhile, let me revert the offending patch.
Will do that on Sunday after some more testing.

Thanks,
Avri

> 
>     BUG: spinlock bad magic on CPU#6, swapper/0/1
>      lock: 0xffffff84443014e8, .magic: 00000000, .owner: <none>/-1,
> .owner_cpu: 0
>     CPU: 6 UID: 0 PID: 1 Comm: swapper/0 Not tainted
> 6.13.0-rcar3-initrd-08318-g75abbef32a94 #896
>     Hardware name: R-Car S4 Starter Kit board (DT)
>     Call trace:
>      show_stack+0x18/0x24 (C)
>      dump_stack_lvl+0x60/0x80
>      dump_stack+0x18/0x24
>      spin_bug+0x7c/0xa0
>      do_raw_spin_lock+0x34/0xb4
>      _raw_spin_lock_irqsave+0x1c/0x30
>      class_spinlock_irqsave_constructor+0x18/0x30
>      ufshcd_setup_clocks+0x98/0x23c
>      ufshcd_init+0x288/0xd38
>      ufshcd_pltfrm_init+0x618/0x738
>      ufs_renesas_probe+0x18/0x24
>      platform_probe+0x68/0xb8
>      really_probe+0x138/0x268
>      __driver_probe_device+0xf4/0x10c
>      driver_probe_device+0x3c/0xf8
>      __driver_attach+0xf0/0x100
>      bus_for_each_dev+0x84/0xdc
>      driver_attach+0x24/0x30
>      bus_add_driver+0xe8/0x1dc
>      driver_register+0xbc/0xf8
>      __platform_driver_register+0x24/0x30
>      ufs_renesas_platform_init+0x1c/0x28
>      do_one_initcall+0x84/0x1f4
>      kernel_init_freeable+0x238/0x23c
>      kernel_init+0x20/0x120
>      ret_from_fork+0x10/0x20
> 
> Gr{oetje,eeting}s,
> 
>                         Geert
> 
> --
> Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-
> m68k.org
> 
> In personal conversations with technical people, I call myself a hacker. But when
> I'm talking to journalists I just say "programmer" or something like that.
>                                 -- Linus Torvalds

Re: [PATCH v2] scsi: ufs: core: Ensure clk_gating.lock is used only after initialization

[Geert Uytterhoeven]

Hi Avri,

On Fri, Jan 24, 2025 at 8:36 AM Avri Altman <avri.altman@wdc.com> wrote:
> This commit addresses a lockdep warning triggered by the use of the
> clk_gating.lock before it is properly initialized. The warning is as
> follows:
>
> [    4.388838] INFO: trying to register non-static key.
> [    4.395673] The code is fine but needs lockdep annotation, or maybe
> [    4.402118] you didn't initialize this object before use?
> [    4.407673] turning off the locking correctness validator.
> [    4.413334] CPU: 5 UID: 0 PID: 58 Comm: kworker/u32:1 Not tainted 6.12-rc1 #185
> [    4.413343] Hardware name: Qualcomm Technologies, Inc. Robotics RB5 (DT)
> [    4.413362] Call trace:
> [    4.413364]  show_stack+0x18/0x24 (C)
> [    4.413374]  dump_stack_lvl+0x90/0xd0
> [    4.413384]  dump_stack+0x18/0x24
> [    4.413392]  register_lock_class+0x498/0x4a8
> [    4.413400]  __lock_acquire+0xb4/0x1b90
> [    4.413406]  lock_acquire+0x114/0x310
> [    4.413413]  _raw_spin_lock_irqsave+0x60/0x88
> [    4.413423]  ufshcd_setup_clocks+0x2c0/0x490
> [    4.413433]  ufshcd_init+0x198/0x10ec
> [    4.413437]  ufshcd_pltfrm_init+0x600/0x7c0
> [    4.413444]  ufs_qcom_probe+0x20/0x58
> [    4.413449]  platform_probe+0x68/0xd8
> [    4.413459]  really_probe+0xbc/0x268
> [    4.413466]  __driver_probe_device+0x78/0x12c
> [    4.413473]  driver_probe_device+0x40/0x11c
> [    4.413481]  __device_attach_driver+0xb8/0xf8
> [    4.413489]  bus_for_each_drv+0x84/0xe4
> [    4.413495]  __device_attach+0xfc/0x18c
> [    4.413502]  device_initial_probe+0x14/0x20
> [    4.413510]  bus_probe_device+0xb0/0xb4
> [    4.413517]  deferred_probe_work_func+0x8c/0xc8
> [    4.413524]  process_scheduled_works+0x250/0x658
> [    4.413534]  worker_thread+0x15c/0x2c8
> [    4.413542]  kthread+0x134/0x200
> [    4.413550]  ret_from_fork+0x10/0x20
>
> To fix this issue, ensure that the spinlock is only used after it
> has been properly initialized before using it in `ufshcd_setup_clocks`.
>
> Fixes: 209f4e43b806 ("scsi: ufs: core: Introduce a new clock_gating lock")
> Reported-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
> Signed-off-by: Avri Altman <avri.altman@wdc.com>

> --- a/drivers/ufs/core/ufshcd.c
> +++ b/drivers/ufs/core/ufshcd.c

> @@ -10412,6 +10410,13 @@ int ufshcd_init(struct ufs_hba *hba, void __iomem *mmio_base, unsigned int irq)
>         hba->irq = irq;
>         hba->vps = &ufs_hba_vps;
>
> +       /*
> +        * Initialize clk_gating.lock early since it is being used in
> +        * ufshcd_setup_clocks()
> +        */
> +       if (ufshcd_is_clkgating_allowed(hba))

This checks if the UFSHCD_CAP_CLK_GATING flag is set, which is only
done in a subset of the host drivers:

drivers/ufs/host/ufs-exynos.c:  hba->caps |= UFSHCD_CAP_CLK_GATING |
UFSHCD_CAP_HIBERN8_WITH_CLK_GATING;
drivers/ufs/host/ufs-mediatek.c:        hba->caps |= UFSHCD_CAP_CLK_GATING;
drivers/ufs/host/ufs-qcom.c:    hba->caps |= UFSHCD_CAP_CLK_GATING |
UFSHCD_CAP_HIBERN8_WITH_CLK_GATING;
drivers/ufs/host/ufs-sprd.c:    hba->caps |= UFSHCD_CAP_CLK_GATING |

> +               spin_lock_init(&hba->clk_gating.lock);

Hence the spinlock is not uninitialized on all other host drivers.

> +
>         err = ufshcd_hba_init(hba);
>         if (err)
>                 goto out_error;

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

RE: [PATCH v2] scsi: ufs: core: Ensure clk_gating.lock is used only after initialization

[Avri Altman]

> > +       /*
> > +        * Initialize clk_gating.lock early since it is being used in
> > +        * ufshcd_setup_clocks()
> > +        */
> > +       if (ufshcd_is_clkgating_allowed(hba))
> 
> This checks if the UFSHCD_CAP_CLK_GATING flag is set, which is only done in a
> subset of the host drivers:
> 
> drivers/ufs/host/ufs-exynos.c:  hba->caps |= UFSHCD_CAP_CLK_GATING |
> UFSHCD_CAP_HIBERN8_WITH_CLK_GATING;
> drivers/ufs/host/ufs-mediatek.c:        hba->caps |= UFSHCD_CAP_CLK_GATING;
> drivers/ufs/host/ufs-qcom.c:    hba->caps |= UFSHCD_CAP_CLK_GATING |
> UFSHCD_CAP_HIBERN8_WITH_CLK_GATING;
> drivers/ufs/host/ufs-sprd.c:    hba->caps |= UFSHCD_CAP_CLK_GATING |
> 
> > +               spin_lock_init(&hba->clk_gating.lock);
> 
> Hence the spinlock is not uninitialized on all other host drivers.
Oh - Thanks for pointing this out.  I totally missed it.
At is appears when clock_gating was invented - 1ab27c9cf8b6 ("ufs: Add support for clock gating")
The below was added without checking if ufshcd_is_clkgating_allowed, or better yet - clk_gating.is_initialized:

@@ -4151,12 +4420,19 @@ static int __ufshcd_setup_clocks(struct ufs_hba *hba, bool on,
                                        clki->name, on ? "en" : "dis");
                }
        }
+       } else if (!ret && on) {
+               spin_lock_irqsave(hba->host->host_lock, flags);
+               hba->clk_gating.state = CLKS_ON;
+               spin_unlock_irqrestore(hba->host->host_lock, flags);
        }

Looks like this is the missing piece.
Will fix this in a 2nd patch.

Thanks,
Avri

> 
> > +
> >         err = ufshcd_hba_init(hba);
> >         if (err)
> >                 goto out_error;
> 
> Gr{oetje,eeting}s,
> 
>                         Geert
> 
> --
> Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-
> m68k.org
> 
> In personal conversations with technical people, I call myself a hacker. But when
> I'm talking to journalists I just say "programmer" or something like that.
>                                 -- Linus Torvalds

Re: [PATCH v2] scsi: ufs: core: Ensure clk_gating.lock is used only after initialization

[Bart Van Assche]

On 1/23/25 11:33 PM, Avri Altman wrote:
> +	/*
> +	 * Initialize clk_gating.lock early since it is being used in
> +	 * ufshcd_setup_clocks()
> +	 */
> +	if (ufshcd_is_clkgating_allowed(hba))
> +		spin_lock_init(&hba->clk_gating.lock);

Initializing a spinlock is a very fast operation so please do this
unconditionally.

Thanks,

Bart.

RE: [PATCH v2] scsi: ufs: core: Ensure clk_gating.lock is used only after initialization

[Avri Altman]

> On 1/23/25 11:33 PM, Avri Altman wrote:
> > +     /*
> > +      * Initialize clk_gating.lock early since it is being used in
> > +      * ufshcd_setup_clocks()
> > +      */
> > +     if (ufshcd_is_clkgating_allowed(hba))
> > +             spin_lock_init(&hba->clk_gating.lock);
> 
> Initializing a spinlock is a very fast operation so please do this unconditionally.
Done.

Thanks,
Avri

> 
> Thanks,
> 
> Bart.