* Re: [PATCH] drm/panthor: Do not change the GPU clock rate while suspended
2026-11-10 7:15 [PATCH] drm/panthor: Do not change the GPU clock rate while suspended Arun Iyengar via B4 Relay
@ 2026-11-10 10:36 ` Liviu Dudau
2026-11-10 13:02 ` Boris Brezillon
1 sibling, 0 replies; 3+ messages in thread
From: Liviu Dudau @ 2026-11-10 10:36 UTC (permalink / raw)
To: aiyengar1990
Cc: Boris Brezillon, Steven Price, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, David Airlie, Simona Vetter, Heiko Stuebner,
dri-devel, linux-kernel, linux-arm-kernel, linux-rockchip,
Jonas Karlman
On Tue, Nov 10, 2026 at 07:15:00AM +0000, Arun Iyengar via B4 Relay wrote:
> From: Arun Iyengar <aiyengar1990@gmail.com>
>
> From the start of panthor_device_suspend() until panthor_device_resume()
> has enabled the clocks again, the GPU clocks are off or about to be, and
> the GPU power domain may be down. devfreq can still call ->target() in
> that window: a write to min_freq or max_freq in sysfs, a thermal cooling
> state change or a PM QoS request all end up in update_devfreq() whatever
> the device state.
>
> With the RK3588 GPU core clock switched to the firmware-managed
> SCMI_CLK_GPU (needed to reach the PVTPLL rates of the upper OPPs), such
> a call is fatal. To set a PVTPLL rate TF-A programs registers in the GPU
> GRF, which sit in the GPU power domain, and with the domain off that
> access ends in an asynchronous SError and a kernel panic, for example
> after
>
> echo 1000000000 > /sys/class/devfreq/fb000000.gpu/min_freq
>
> while the GPU is runtime suspended.
Maybe what needs to be done is to disable the clock when the power domain
goes down.
>
> Only apply a rate to the hardware while the device is resuming or
> active, plus the suspend frequency that devfreq_suspend_device() applies
> from the suspend path while the GPU is still powered. Other requests are
> still accepted and recorded by devfreq, which applies them on resume or
> at the next governor update.
>
> The RFC this is based on (Jonas Karlman's panthor SCMI clock change,
> carried in Armbian's linux-rockchip pull request 380) checked the
> runtime PM status instead. That
> status is still RPM_SUSPENDING after panthor_devfreq_suspend() has run,
> which leaves a window where the rate can be raised again right before
> the power domain goes down. The devfreq suspend count cannot be used
> either: panthor's first runtime resume calls devfreq_resume_device()
> without a matching suspend, so the count is -1 while the GPU is active
> and 0 while it is suspended.
>
> Suggested-by: Jonas Karlman <jonas@kwiboo.se>
> Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
> ---
> Tested on an Orange Pi 5 (RK3588S) with the GPU core clock switched to
> SCMI_CLK_GPU and a 200 MHz opp-suspend (the configuration from the RFC,
> not yet in mainline): writing min_freq while the GPU was runtime
> suspended no longer ends in an SError, the request takes effect at the
> next job (1 GHz measured), max_freq likewise; 200 suspend/resume cycles
> with short GPU jobs and a 3 minute sustained load ran without faults.
> With the mainline device tree (CRU clock, no suspend OPP) the SError is
> not reachable and the guard only skips rate changes while the GPU is
> powered down.
>
> This patch was developed with Claude (Anthropic) as a coding assistant:
> it wrote the code and the commit message, and I reviewed and tested it
> as described above.
> ---
> drivers/gpu/drm/panthor/panthor_devfreq.c | 42 ++++++++++++++++++++++++++++---
> 1 file changed, 39 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/gpu/drm/panthor/panthor_devfreq.c b/drivers/gpu/drm/panthor/panthor_devfreq.c
> index 2249b41ca4af..cab86e9c9d46 100644
> --- a/drivers/gpu/drm/panthor/panthor_devfreq.c
> +++ b/drivers/gpu/drm/panthor/panthor_devfreq.c
> @@ -60,20 +60,56 @@ static void panthor_devfreq_update_utilization(struct panthor_devfreq *pdevfreq)
> pdevfreq->time_last_update = now;
> }
>
> +/*
> + * Whether a new rate can be applied to the hardware now.
> + *
> + * From the start of panthor_device_suspend() until panthor_device_resume()
> + * has enabled the clocks again, the GPU clocks are about to be disabled
> + * or are off, and the GPU power domain may be down. devfreq can still
> + * call ->target() in that window: sysfs
> + * min_freq/max_freq writes, thermal cooling and PM QoS requests all end
> + * up in update_devfreq() whatever the device state.
> + *
> + * Changing the rate then is pointless, and harmful where the core clock
> + * is managed by firmware: on RK3588, setting SCMI_CLK_GPU to a PVTPLL
> + * rate with the GPU power domain off makes TF-A access the powered-down
> + * GPU GRF, which ends in an asynchronous SError. So while the device is
> + * suspending only the suspend frequency, applied by
> + * devfreq_suspend_device(), reaches the hardware, and nothing does
> + * while it is suspended. devfreq records the other requests: they are
> + * applied on resume or at the next governor update.
Can this comment be rewritten to be less verbose?
> + */
> +static bool panthor_devfreq_can_set_rate(struct panthor_device *ptdev,
> + unsigned long freq)
> +{
> + struct devfreq *devfreq = ptdev->devfreq ? ptdev->devfreq->devfreq : NULL;
> +
> + switch (atomic_read(&ptdev->pm.state)) {
> + case PANTHOR_DEVICE_PM_STATE_SUSPENDED:
> + return false;
> + case PANTHOR_DEVICE_PM_STATE_SUSPENDING:
> + return devfreq && devfreq->suspend_freq &&
> + freq == devfreq->suspend_freq;
> + default:
> + return true;
> + }
> +}
> +
> static int panthor_devfreq_target(struct device *dev, unsigned long *freq,
> u32 flags)
> {
> + struct panthor_device *ptdev = dev_get_drvdata(dev);
> struct dev_pm_opp *opp;
> - int err;
>
> opp = devfreq_recommended_opp(dev, freq, flags);
> if (IS_ERR(opp))
> return PTR_ERR(opp);
> dev_pm_opp_put(opp);
>
> - err = dev_pm_opp_set_rate(dev, *freq);
> + if (!panthor_devfreq_can_set_rate(ptdev, *freq))
> + return 0;
Hmm, why not return an error here? -EINVAL?
>
> - return err;
> + return dev_pm_opp_set_rate(dev, *freq);
> }
>
> static void panthor_devfreq_reset(struct panthor_devfreq *pdevfreq)
>
> ---
> base-commit: 8deaade4a5a4ff635df91e246e42139ab2b33f6d
> change-id: 20261110-panthor-suspend-clk-11d5604fe5be
>
> Best regards,
> --
> Arun Iyengar <aiyengar1990@gmail.com>
>
>
Best regards,
Liviu
--
====================
| I would like to |
| fix the world, |
| but they're not |
| giving me the |
\ source code! /
---------------
¯\_(ツ)_/¯
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: [PATCH] drm/panthor: Do not change the GPU clock rate while suspended
2026-11-10 7:15 [PATCH] drm/panthor: Do not change the GPU clock rate while suspended Arun Iyengar via B4 Relay
2026-11-10 10:36 ` Liviu Dudau
@ 2026-11-10 13:02 ` Boris Brezillon
1 sibling, 0 replies; 3+ messages in thread
From: Boris Brezillon @ 2026-11-10 13:02 UTC (permalink / raw)
To: Arun Iyengar via B4 Relay
Cc: aiyengar1990, Steven Price, Liviu Dudau, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
Heiko Stuebner, dri-devel, linux-kernel, linux-arm-kernel,
linux-rockchip, Jonas Karlman
On Tue, 10 Nov 2026 07:15:00 +0000
Arun Iyengar via B4 Relay <devnull+aiyengar1990.gmail.com@kernel.org>
wrote:
> From: Arun Iyengar <aiyengar1990@gmail.com>
>
> From the start of panthor_device_suspend() until panthor_device_resume()
> has enabled the clocks again, the GPU clocks are off or about to be, and
> the GPU power domain may be down. devfreq can still call ->target() in
> that window: a write to min_freq or max_freq in sysfs, a thermal cooling
> state change or a PM QoS request all end up in update_devfreq() whatever
> the device state.
>
> With the RK3588 GPU core clock switched to the firmware-managed
> SCMI_CLK_GPU (needed to reach the PVTPLL rates of the upper OPPs), such
> a call is fatal. To set a PVTPLL rate TF-A programs registers in the GPU
> GRF, which sit in the GPU power domain, and with the domain off that
> access ends in an asynchronous SError and a kernel panic, for example
> after
>
> echo 1000000000 > /sys/class/devfreq/fb000000.gpu/min_freq
>
> while the GPU is runtime suspended.
>
> Only apply a rate to the hardware while the device is resuming or
> active, plus the suspend frequency that devfreq_suspend_device() applies
> from the suspend path while the GPU is still powered. Other requests are
> still accepted and recorded by devfreq, which applies them on resume or
> at the next governor update.
>
> The RFC this is based on (Jonas Karlman's panthor SCMI clock change,
> carried in Armbian's linux-rockchip pull request 380) checked the
> runtime PM status instead. That
> status is still RPM_SUSPENDING after panthor_devfreq_suspend() has run,
> which leaves a window where the rate can be raised again right before
> the power domain goes down. The devfreq suspend count cannot be used
> either: panthor's first runtime resume calls devfreq_resume_device()
> without a matching suspend, so the count is -1 while the GPU is active
> and 0 while it is suspended.
>
> Suggested-by: Jonas Karlman <jonas@kwiboo.se>
> Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
> ---
> Tested on an Orange Pi 5 (RK3588S) with the GPU core clock switched to
> SCMI_CLK_GPU and a 200 MHz opp-suspend (the configuration from the RFC,
> not yet in mainline): writing min_freq while the GPU was runtime
> suspended no longer ends in an SError, the request takes effect at the
> next job (1 GHz measured), max_freq likewise; 200 suspend/resume cycles
> with short GPU jobs and a 3 minute sustained load ran without faults.
> With the mainline device tree (CRU clock, no suspend OPP) the SError is
> not reachable and the guard only skips rate changes while the GPU is
> powered down.
>
> This patch was developed with Claude (Anthropic) as a coding assistant:
> it wrote the code and the commit message, and I reviewed and tested it
> as described above.
> ---
> drivers/gpu/drm/panthor/panthor_devfreq.c | 42 ++++++++++++++++++++++++++++---
> 1 file changed, 39 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/gpu/drm/panthor/panthor_devfreq.c b/drivers/gpu/drm/panthor/panthor_devfreq.c
> index 2249b41ca4af..cab86e9c9d46 100644
> --- a/drivers/gpu/drm/panthor/panthor_devfreq.c
> +++ b/drivers/gpu/drm/panthor/panthor_devfreq.c
> @@ -60,20 +60,56 @@ static void panthor_devfreq_update_utilization(struct panthor_devfreq *pdevfreq)
> pdevfreq->time_last_update = now;
> }
>
> +/*
> + * Whether a new rate can be applied to the hardware now.
> + *
> + * From the start of panthor_device_suspend() until panthor_device_resume()
> + * has enabled the clocks again, the GPU clocks are about to be disabled
> + * or are off, and the GPU power domain may be down. devfreq can still
> + * call ->target() in that window: sysfs
> + * min_freq/max_freq writes, thermal cooling and PM QoS requests all end
> + * up in update_devfreq() whatever the device state.
> + *
> + * Changing the rate then is pointless, and harmful where the core clock
> + * is managed by firmware: on RK3588, setting SCMI_CLK_GPU to a PVTPLL
> + * rate with the GPU power domain off makes TF-A access the powered-down
> + * GPU GRF, which ends in an asynchronous SError. So while the device is
> + * suspending only the suspend frequency, applied by
> + * devfreq_suspend_device(), reaches the hardware, and nothing does
> + * while it is suspended. devfreq records the other requests: they are
> + * applied on resume or at the next governor update.
> + */
> +static bool panthor_devfreq_can_set_rate(struct panthor_device *ptdev,
> + unsigned long freq)
> +{
> + struct devfreq *devfreq = ptdev->devfreq ? ptdev->devfreq->devfreq : NULL;
I'd rather use devfreq::suspend_count to detect if the devfreq
component is suspended, since it gives us a better granularity than
panthor_device::pm::state.
> +
> + switch (atomic_read(&ptdev->pm.state)) {
> + case PANTHOR_DEVICE_PM_STATE_SUSPENDED:
> + return false;
> + case PANTHOR_DEVICE_PM_STATE_SUSPENDING:
> + return devfreq && devfreq->suspend_freq &&
> + freq == devfreq->suspend_freq;
> + default:
When PANTHOR_DEVICE_PM_STATE_SUSPENDING is set, not all HW
components have been clocked/put out of reset. It's probably okay
for power domains, because those deps are described through device
links, and the dependencies are resumed by PM core before the
panthor PM hooks are called, but this is fragile, and that's another
reason I think we shouldn't rely on panthor_device::pm::state for that,
but on devfreq::suspend_count (or any other internal panthor_devfreq
state that tracks the panthor_devfreq component status).
> + return true;
> + }
> +}
> +
> static int panthor_devfreq_target(struct device *dev, unsigned long *freq,
> u32 flags)
> {
> + struct panthor_device *ptdev = dev_get_drvdata(dev);
> struct dev_pm_opp *opp;
> - int err;
>
> opp = devfreq_recommended_opp(dev, freq, flags);
> if (IS_ERR(opp))
> return PTR_ERR(opp);
> dev_pm_opp_put(opp);
>
> - err = dev_pm_opp_set_rate(dev, *freq);
> + if (!panthor_devfreq_can_set_rate(ptdev, *freq))
> + return 0;
If you return zero here, the new frequency is assumed to be applied,
and it messes up with the devfreq state (::previous_freq for instance)
that was set on devfreq_suspend_device(). If we assume that rate
changes should be rejected while we're suspended, this should probably
return an error.
>
> - return err;
> + return dev_pm_opp_set_rate(dev, *freq);
There's another race between devfreq_suspend_device(), which will
restore the ::resume_freq, and a concurrent devfreq_update_target() call
that's here to set a new rate. Using devfreq::suspend_count won't help,
because this atomic is decremented before the resume_freq is applied,
and the devfreq lock is only taken around the devfreq_set_target() call.
Honestly, it feels like we're trying to address the problem at the wrong
level. I would expect governors to stop calling devfreq_update_target()
between the DEVFREQ_GOV_SUSPEND and DEVFREQ_GOV_RESUME events (which
should probably be sent before suspend_count is incremented and the
suspend freq is applied). And I'd also expect some kind of
locking/synchronization to prevent races between
devfreq_update_target() and devfreq_{suspend,resume}_device().
> }
>
> static void panthor_devfreq_reset(struct panthor_devfreq *pdevfreq)
>
> ---
> base-commit: 8deaade4a5a4ff635df91e246e42139ab2b33f6d
> change-id: 20261110-panthor-suspend-clk-11d5604fe5be
>
> Best regards,
> --
> Arun Iyengar <aiyengar1990@gmail.com>
>
>
^ permalink raw reply [flat|nested] 3+ messages in thread