Re: [RFC] accel/rocket: DVFS on RK3588 - a hardware constraint, and some numbers

Re: [RFC] accel/rocket: DVFS on RK3588 - a hardware constraint, and some numbers

[Jiaxing Hu]

Hi Igor,

> all three NPU domains already list the NPU clock (rk3588-base.dtsi
> lines 864, 877 and 885)

Those lines list CLK_NPU_DSU0, but the clock the driver holds as "npu",
and the one devfreq scales, is <&scmi_clk SCMI_CLK_NPU>. The driver never
holds CLK_NPU_DSU0 at all. They may share a root, but if they do not then
the handshake is not breaking because you scaled the clock it needs, and
the notifier is treating a symptom. Worth a look at clk_summary first.

On RK3576 they really are the same clock ("npu" is CLK_RKNN_DSU0, which
PD_NPUTOP also lists), so that comparison at least is easy here.

Also relevant: the vendor does not scale the handshake side at all. Live
sample from a 6.1.115 BSP during a working inference, captured by
Olaf001au:

  clk_npu 950 MHz, clk_dsu 198 MHz, aclk_cbuf 198 MHz

Compute at 950, everything else left at boot rate. If something similar
exists on RK3588 you may be able to pick a clock that is not in any
domain's list and avoid the constraint rather than veto around it.

Two smaller data points:

We get an async SError from NPU domain power-on here too, which is why my
v3 has the settle delay and the reset cycling. But our DSU0 sits at 594 to
786 MHz permanently and only the cold power-on fails; later transitions at
the same rate are fine. So either your constraint is RK3588 specific or my
settle delay is masking it and my explanation is wrong. No idea which.

You said you had not tried PVTPLL because reading its registers is
supposed to hang. I tried it from the other end on RK3576, routing the NPU
clock through SCMI: zero jobs completed, 83 scheduler timeouts, everything
else unchanged. Not a drop-in for the CRU clock.

On the OPP table, your plateau looks memory bound, and the vendor pinning
aclk_cbuf at 198 while compute runs at 950 points the same way. If so, 600
is where MobileNetV1 stops scaling rather than where the hardware does.
Maybe worth one compute dense model before cutting the table there.

Cheers,
Jiaxing

Re: [RFC] accel/rocket: DVFS on RK3588 - a hardware constraint, and some numbers

[Igor Paunovic]

Hi Tomeu, hi Jiaxing,

I need to correct the measurements in my first mail. The conclusions I
drew from them do not survive better methodology, and since two of my
questions to Tomeu were based on those conclusions, I would rather fix
this now than have anyone reason from bad data.

Short version: the numbers were real, but they measured my interrupt
configuration, not the NPU.


1. What was wrong
=================

Each inference generates 41 interrupts on this driver (one per task,
more on that below).  All of them were landing on CPU0, a Cortex-A55
that was also entering its shallow idle state (exit latency 220 us)
between interrupts.  So every inference paid a large, frequency-
independent CPU-side cost, which compressed the differences between
NPU frequencies and produced the inversion I reported.

Three changes, none of them driver code:

  - NPU IRQ affinity moved to an idle A76
  - cpu-sleep (state1) disabled on that core
  - performance cpufreq governors during measurement

With that, the picture changes completely.  Even the fixed 200 MHz
baseline moves from 68.5 to ~91 inf/s - a quarter of the stock
performance was lost to configuration alone.  I should have caught
this; the per-inference interrupt count was in my own telemetry.


2. Corrected numbers
====================

Same rig as before otherwise: MobileNetV1 through Teflon, one inference
thread pinned to one A76, bit-exact sha256 oracle over intermediate
tensors on every deep-check, zero tolerance, oracle passed in every run
quoted here.  Kernel 7.0.0 + local devfreq for rocket.  Ambient
controlled, die temperature logged per row (all rows 42.5-45.3 degC).
Every row is an ~80 s run; key points were replicated 2-5x across the
day, spread within 1%.

  inf/s            800 mV    850 mV    950 mV
  200 MHz           91.1      90.7      90.8
  300 MHz          161.7     170.9     185.1
  400 MHz          187.3     196.2     211.4
  500 MHz          208.8     216.7     226.6
  600 MHz          221.1     227.9     236.5
  700 MHz          227.4     233.0     240.7
  800 MHz             -      232.2     241.9
  900 MHz             -      233.8     241.4
  1000 MHz            -      235.0     242.8

(The 800 mV column stops at 700 because my local safeguard refuses
higher OPPs on that rail.  The identical 200 MHz row across all three
rails is the negative control - at 200 MHz voltage headroom is
irrelevant, and the rig reproduces to 0.5%.)

Corrections to my three claims, in order:

a) "600 MHz is the optimum" - withdrawn.  The curve is monotonic to
   ~700 MHz and flat above it.  There is no inversion; the inversion
   was the A55 paying 41 wakeups per inference.

b) "Voltage does not help" - withdrawn.  At the same nominal clock,
   raising the rail 800 -> 950 mV buys +14% at 300 MHz and +7% at
   600 MHz.  Consistent with the clock being PVT-controlled: more
   voltage lets the PLL deliver a higher actual clock for the same
   nominal request.  (This also means the flat region above 700 is
   partly voltage-limited, not purely memory-limited: the per-chunk
   floor is ~72.5 us at 850 mV but ~69 us at 950 mV.)

c) "Should the OPP table stop at 600?" - wrong question, please ignore
   it.  The plateau is this board's memory (LPDDR4X-2112) plus the
   voltage effect above; boards with LPDDR5 should keep scaling
   further.  The table should carry the full vendor range and let each
   board cap via max_freq / thermal policy.

One methodological note that may be useful to anyone benchmarking this
hardware: the largest error sources I quantified today were, in order -
interrupt placement (+32% at stock clock), a busy desktop session
sharing the SoC (up to 18%), die temperature (~0.5%/degC via the PVT
loop), and an open browser (~1%).  The silicon was the most
reproducible part of the whole setup.


3. Jiaxing's points
===================

> Those lines list CLK_NPU_DSU0, but the clock the driver holds as
> "npu" [...] is <&scmi_clk SCMI_CLK_NPU>. [...] Worth a look at
> clk_summary first.

You are right about the identity, and thanks for pushing me to check
it properly.  Decoded from the live DT: "npu" is SCMI_CLK_NPU (SCMI
clock id 6); aclk/hclk/pclk come from the CRU.  The domains list
CLK_NPU_DSU0, which devfreq never touches.

I then took clk_summary snapshots idle, mid-benchmark at nominal 600,
and mid-benchmark at nominal 1000:

                    idle      during 600   during 1000
  clk_npu_dsu0      250 MHz   250 MHz      250 MHz
  aclk_npu0/1/2     250       250          250
  hclk_npu0/1/2     198       198          198
  pclk_npu_root     100       100          100

So the handshake-side clocks never move - this driver already does
exactly what your vendor sample shows (compute clock scaled, dsu/aclk/
hclk parked at boot rates).  And yet the domain transition still fails
whenever the SCMI compute clock is raised, reproducibly.  So the
constraint tracks the compute clock itself, not the DSU clock, and
"pick a clock outside the domain lists" unfortunately does not exist
here: the only rate anyone scales is already outside them.

Whether the mechanism is a PLL inside the domain remains my hypothesis
- the TRM's NPU chapter is silent on PVTPLL and I still have not risked
reading its registers.  The constraint itself is empirical.

A caveat worth passing on: clk_summary reports SCMI_CLK_NPU at
200 MHz even mid-benchmark at nominal 1000 - the CCF cache is stale for
firmware clocks, so clk_summary genuinely cannot answer this question.
I verified the actual rate two ways: clk_get_rate() (which round-trips
to firmware), and the per-chunk NPU time scaling 236 -> 69 us across
the sweep.

> your plateau looks memory bound [...] Maybe worth one compute dense
> model before cutting the table there.

Fair, and the answer is a few lines up: I am no longer proposing to cut
the table anywhere.  I did run the compute-dense model you asked for:
InceptionV1 (quant), same rig, 850 mV.  One caveat first: on this stack
Teflon leaves ~79% of an Inception inference on the CPU (many more ops
fall back than for MobileNet), so total throughput barely moves with
clock and the honest metric is the NPU-side time per inference, which
the harness separates out:

  nominal     NPU-side per inference    total
  300 MHz     7.2 ms                    43.7 inf/s
  700 MHz     4.16 ms                   50.5 inf/s
  1000 MHz    4.17 ms                   50.2 inf/s

It scales 300 -> 700 and then flattens at exactly the same knee as
MobileNetV1 (whose NPU-side time goes 4.54 -> 3.00 -> 2.97 ms over the
same points).  So on this board the plateau is the platform, not
MobileNetV1's memory profile - consistent with the LPDDR4X explanation,
and it does not change the conclusion: full table, per-board cap.
Boards with faster memory may well keep scaling; mine cannot test that.

(Two incidental data points from the same runs: Inception jobs get
scheduled across two NPU cores here - 33+27 interrupts per inference
against MobileNet's 41 on one core - and the bit-exact oracle passed at
every frequency for this model too.)

On your RK3576 SCMI experiment (zero jobs, 83 timeouts): one cross-data
point - on RK3588 the SCMI/PVTPLL path is what we use successfully for
the GPU (Mali-G610 at 1 GHz via SCMI, measured and stable), and the NPU
compute clock here is SCMI-routed and scales fine while the domain is
up.  So the SCMI plumbing itself can work on this family; whatever
failed on RK3576 may be specific to that port rather than the approach.


4. Thermal, update
==================

The missing NPU cooling chain from my first mail is now running here
end-to-end: #cooling-cells on the NPU node, a passive trip at 85 degC
with a cooling map, and devfreq_cooling registered by the driver - the
kernel bound them together without manual intervention.  So the two DT
patches are ready whenever the driver work lands; they need nothing
beyond what devfreq already provides.


5. Where this leaves the questions to Tomeu
===========================================

The two RFC questions that survive unchanged: whether you want DVFS
for rocket upstream at all, and whether the genpd-notifier shape is
acceptable.  The OPP question is answered above (full table, per-board
cap).  The code cleanup questions stand as written.

Sorry for the noise, and thanks to Jiaxing for the push to verify the
clock tree - the numbers are better for it.

Igor

Re: [RFC] accel/rocket: DVFS on RK3588 - a hardware constraint, and some numbers

[Tomeu Vizoso]

Hi Igor,

On Sun, Aug 2, 2026 at 2:04 PM Igor Paunovic <royalnet026@gmail.com> wrote:
>
> Hi Tomeu, hi Jiaxing,
>
> I need to correct the measurements in my first mail. The conclusions I
> drew from them do not survive better methodology, and since two of my
> questions to Tomeu were based on those conclusions, I would rather fix
> this now than have anyone reason from bad data.
>
> Short version: the numbers were real, but they measured my interrupt
> configuration, not the NPU.
>
>
> 1. What was wrong
> =================
>
> Each inference generates 41 interrupts on this driver (one per task,
> more on that below).  All of them were landing on CPU0, a Cortex-A55
> that was also entering its shallow idle state (exit latency 220 us)
> between interrupts.  So every inference paid a large, frequency-
> independent CPU-side cost, which compressed the differences between
> NPU frequencies and produced the inversion I reported.
>
> Three changes, none of them driver code:
>
>   - NPU IRQ affinity moved to an idle A76
>   - cpu-sleep (state1) disabled on that core
>   - performance cpufreq governors during measurement
>
> With that, the picture changes completely.  Even the fixed 200 MHz
> baseline moves from 68.5 to ~91 inf/s - a quarter of the stock
> performance was lost to configuration alone.  I should have caught
> this; the per-inference interrupt count was in my own telemetry.
>
>
> 2. Corrected numbers
> ====================
>
> Same rig as before otherwise: MobileNetV1 through Teflon, one inference
> thread pinned to one A76, bit-exact sha256 oracle over intermediate
> tensors on every deep-check, zero tolerance, oracle passed in every run
> quoted here.  Kernel 7.0.0 + local devfreq for rocket.  Ambient
> controlled, die temperature logged per row (all rows 42.5-45.3 degC).
> Every row is an ~80 s run; key points were replicated 2-5x across the
> day, spread within 1%.
>
>   inf/s            800 mV    850 mV    950 mV
>   200 MHz           91.1      90.7      90.8
>   300 MHz          161.7     170.9     185.1
>   400 MHz          187.3     196.2     211.4
>   500 MHz          208.8     216.7     226.6
>   600 MHz          221.1     227.9     236.5
>   700 MHz          227.4     233.0     240.7
>   800 MHz             -      232.2     241.9
>   900 MHz             -      233.8     241.4
>   1000 MHz            -      235.0     242.8
>
> (The 800 mV column stops at 700 because my local safeguard refuses
> higher OPPs on that rail.  The identical 200 MHz row across all three
> rails is the negative control - at 200 MHz voltage headroom is
> irrelevant, and the rig reproduces to 0.5%.)
>
> Corrections to my three claims, in order:
>
> a) "600 MHz is the optimum" - withdrawn.  The curve is monotonic to
>    ~700 MHz and flat above it.  There is no inversion; the inversion
>    was the A55 paying 41 wakeups per inference.
>
> b) "Voltage does not help" - withdrawn.  At the same nominal clock,
>    raising the rail 800 -> 950 mV buys +14% at 300 MHz and +7% at
>    600 MHz.  Consistent with the clock being PVT-controlled: more
>    voltage lets the PLL deliver a higher actual clock for the same
>    nominal request.  (This also means the flat region above 700 is
>    partly voltage-limited, not purely memory-limited: the per-chunk
>    floor is ~72.5 us at 850 mV but ~69 us at 950 mV.)
>
> c) "Should the OPP table stop at 600?" - wrong question, please ignore
>    it.  The plateau is this board's memory (LPDDR4X-2112) plus the
>    voltage effect above; boards with LPDDR5 should keep scaling
>    further.  The table should carry the full vendor range and let each
>    board cap via max_freq / thermal policy.
>
> One methodological note that may be useful to anyone benchmarking this
> hardware: the largest error sources I quantified today were, in order -
> interrupt placement (+32% at stock clock), a busy desktop session
> sharing the SoC (up to 18%), die temperature (~0.5%/degC via the PVT
> loop), and an open browser (~1%).  The silicon was the most
> reproducible part of the whole setup.
>
>
> 3. Jiaxing's points
> ===================
>
> > Those lines list CLK_NPU_DSU0, but the clock the driver holds as
> > "npu" [...] is <&scmi_clk SCMI_CLK_NPU>. [...] Worth a look at
> > clk_summary first.
>
> You are right about the identity, and thanks for pushing me to check
> it properly.  Decoded from the live DT: "npu" is SCMI_CLK_NPU (SCMI
> clock id 6); aclk/hclk/pclk come from the CRU.  The domains list
> CLK_NPU_DSU0, which devfreq never touches.
>
> I then took clk_summary snapshots idle, mid-benchmark at nominal 600,
> and mid-benchmark at nominal 1000:
>
>                     idle      during 600   during 1000
>   clk_npu_dsu0      250 MHz   250 MHz      250 MHz
>   aclk_npu0/1/2     250       250          250
>   hclk_npu0/1/2     198       198          198
>   pclk_npu_root     100       100          100
>
> So the handshake-side clocks never move - this driver already does
> exactly what your vendor sample shows (compute clock scaled, dsu/aclk/
> hclk parked at boot rates).  And yet the domain transition still fails
> whenever the SCMI compute clock is raised, reproducibly.  So the
> constraint tracks the compute clock itself, not the DSU clock, and
> "pick a clock outside the domain lists" unfortunately does not exist
> here: the only rate anyone scales is already outside them.
>
> Whether the mechanism is a PLL inside the domain remains my hypothesis
> - the TRM's NPU chapter is silent on PVTPLL and I still have not risked
> reading its registers.  The constraint itself is empirical.
>
> A caveat worth passing on: clk_summary reports SCMI_CLK_NPU at
> 200 MHz even mid-benchmark at nominal 1000 - the CCF cache is stale for
> firmware clocks, so clk_summary genuinely cannot answer this question.
> I verified the actual rate two ways: clk_get_rate() (which round-trips
> to firmware), and the per-chunk NPU time scaling 236 -> 69 us across
> the sweep.
>
> > your plateau looks memory bound [...] Maybe worth one compute dense
> > model before cutting the table there.
>
> Fair, and the answer is a few lines up: I am no longer proposing to cut
> the table anywhere.  I did run the compute-dense model you asked for:
> InceptionV1 (quant), same rig, 850 mV.  One caveat first: on this stack
> Teflon leaves ~79% of an Inception inference on the CPU (many more ops
> fall back than for MobileNet), so total throughput barely moves with
> clock and the honest metric is the NPU-side time per inference, which
> the harness separates out:
>
>   nominal     NPU-side per inference    total
>   300 MHz     7.2 ms                    43.7 inf/s
>   700 MHz     4.16 ms                   50.5 inf/s
>   1000 MHz    4.17 ms                   50.2 inf/s
>
> It scales 300 -> 700 and then flattens at exactly the same knee as
> MobileNetV1 (whose NPU-side time goes 4.54 -> 3.00 -> 2.97 ms over the
> same points).  So on this board the plateau is the platform, not
> MobileNetV1's memory profile - consistent with the LPDDR4X explanation,
> and it does not change the conclusion: full table, per-board cap.
> Boards with faster memory may well keep scaling; mine cannot test that.
>
> (Two incidental data points from the same runs: Inception jobs get
> scheduled across two NPU cores here - 33+27 interrupts per inference
> against MobileNet's 41 on one core - and the bit-exact oracle passed at
> every frequency for this model too.)
>
> On your RK3576 SCMI experiment (zero jobs, 83 timeouts): one cross-data
> point - on RK3588 the SCMI/PVTPLL path is what we use successfully for
> the GPU (Mali-G610 at 1 GHz via SCMI, measured and stable), and the NPU
> compute clock here is SCMI-routed and scales fine while the domain is
> up.  So the SCMI plumbing itself can work on this family; whatever
> failed on RK3576 may be specific to that port rather than the approach.
>
>
> 4. Thermal, update
> ==================
>
> The missing NPU cooling chain from my first mail is now running here
> end-to-end: #cooling-cells on the NPU node, a passive trip at 85 degC
> with a cooling map, and devfreq_cooling registered by the driver - the
> kernel bound them together without manual intervention.  So the two DT
> patches are ready whenever the driver work lands; they need nothing
> beyond what devfreq already provides.
>
>
> 5. Where this leaves the questions to Tomeu
> ===========================================
>
> The two RFC questions that survive unchanged: whether you want DVFS
> for rocket upstream at all,

Sure!

> and whether the genpd-notifier shape is
> acceptable.

To be honest, I don't feel qualified to answer that. If you don't get
an answer here or know whom to ask (you could check git blame), you
can just send your best try to elicit a discussion.

> The OPP question is answered above (full table, per-board
> cap).

Right.

>  The code cleanup questions stand as written.

Regarding code organization, please check how devfreq is used in other
DRM drivers for ideas.

Thanks,

Tomeu

> Sorry for the noise, and thanks to Jiaxing for the push to verify the
> clock tree - the numbers are better for it.
>
> Igor