[PATCH 0/3] cpufreq: Avoid fallback values in cpuinfo_avg_freq

[PATCH 0/3] cpufreq: Avoid fallback values in cpuinfo_avg_freq

[Chuyi Zhou]

Introduction
============

cpuinfo_avg_freq is intended to expose an average operating frequency
derived from hardware feedback over a short time interval. On x86, it
can instead return a policy value or cpu_khz when no usable sample is
available. Userspace cannot distinguish that fallback from a measurement.

Provide a separate hardware feedback getter for cpuinfo_avg_freq so that
missing measurements are reported as errors. Keep the existing frequency
reporting behavior of scaling_cur_freq and /proc/cpuinfo.

Background
==========

The x86 frequency reporting path uses APERF/MPERF samples collected by
the scheduler tick. Reading the cached samples avoids sending IPIs to
wake idle CPUs or interrupt isolated CPUs. If a sample is older than
approximately 20 ms or has a zero MPERF delta, arch_freq_get_on_cpu()
falls back to cpufreq_quick_get(), and then to cpu_khz if that returns
zero.

Commit fbb4a4759b54 ("cpufreq: Introduce an optional cpuinfo_avg_freq sysfs
entry") reused arch_freq_get_on_cpu() for cpuinfo_avg_freq. This exposed
the x86 fallback through an interface intended to report the frequency
at which the hardware actually ran.

For example, the active amd-pstate-epp driver sets policy->cur to
policy->min, including under the performance policy. When a sample
expires, cpuinfo_avg_freq can therefore switch from a measured frequency
to the policy minimum. Monitoring software can interpret this as a
frequency drop even though no measurement supports that conclusion.

Commit 9e4cb21f2940 ("cpufreq: intel_pstate: Adjust policy->cur in active
mode to policy") changed the Intel performance-policy fallback to the
requested frequency. That value cannot establish that the hardware
reached the requested frequency, so the same ambiguity remains.

Missing samples are not limited to idle CPUs. The scheduler skips
frequency sampling on CPUs excluded from HK_TYPE_KERNEL_NOISE
housekeeping. A nohz_full CPU can lack a usable sample while running a
busy workload. Conversely, a CPU that has just entered idle can still
have a valid cached sample.

Changes
=======

Introduce arch_freq_get_avg() to separate hardware feedback from the
compatibility fallback. Use it for both cpuinfo_avg_freq reads and the
attribute's support probe. On x86, the behavior changes as follows:

  Condition               Before                 After
  ---------               ------                 -----
  Usable sample           Measured frequency     Measured frequency
  Expired sample          Fallback frequency     EAGAIN
  Zero MPERF delta        Fallback frequency     EAGAIN
  No APERF/MPERF support   Fallback frequency     Attribute absent

ARM64 reuses its existing AMU implementation without changing its behavior.

Chuyi Zhou (3):
  x86/aperfmperf: Separate hardware feedback from fallback reporting
  cpufreq: Use hardware feedback for cpuinfo_avg_freq
  x86/aperfmperf: Clean up includes after removing IPI sampling

 Documentation/admin-guide/pm/cpufreq.rst | 11 ++++++--
 arch/arm64/kernel/topology.c             |  7 ++++-
 arch/x86/kernel/cpu/aperfmperf.c         | 33 +++++++++++++++++-------
 drivers/cpufreq/cpufreq.c                | 19 ++++++++++++--
 include/linux/cpufreq.h                  |  1 +
 5 files changed, 56 insertions(+), 15 deletions(-)


base-commit: e81ee06308379a5f2ededf997bcf17551bce5db7
-- 
2.20.1

[PATCH 1/3] x86/aperfmperf: Separate hardware feedback from fallback reporting

[Chuyi Zhou]

arch_freq_get_on_cpu() reads cached APERF/MPERF samples and falls back to
cpufreq_quick_get() or cpu_khz when no usable sample is available. Its
callers cannot distinguish that fallback from a measured frequency.
This prevents a hardware feedback consumer from reporting missing
samples while preserving the fallback used by existing interfaces.

Extract the sample lookup and calculation into arch_freq_get_avg().
Return -EOPNOTSUPP when APERF/MPERF is unavailable and -EAGAIN when the
sample is too old or has a zero MPERF delta. Keep the existing seqcount
read, age limit and frequency calculation.

No functional change intended. This prepares cpuinfo_avg_freq to use
hardware feedback without fallback values.

Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
---
 arch/x86/kernel/cpu/aperfmperf.c | 25 ++++++++++++++++++++-----
 include/linux/cpufreq.h          |  1 +
 2 files changed, 21 insertions(+), 5 deletions(-)

diff --git a/arch/x86/kernel/cpu/aperfmperf.c b/arch/x86/kernel/cpu/aperfmperf.c
index 7ffc78d5ebf2..de6297b1284b 100644
--- a/arch/x86/kernel/cpu/aperfmperf.c
+++ b/arch/x86/kernel/cpu/aperfmperf.c
@@ -503,15 +503,22 @@ void arch_scale_freq_tick(void)
  */
 #define MAX_SAMPLE_AGE	((unsigned long)HZ / 50)
 
-int arch_freq_get_on_cpu(int cpu)
+/**
+ * arch_freq_get_avg() - Read cached APERF/MPERF frequency feedback
+ * @cpu: CPU to read.
+ *
+ * Return: Frequency in kHz, -EAGAIN if no usable sample is available, or
+ * -EOPNOTSUPP if APERF/MPERF is unsupported.
+ */
+int arch_freq_get_avg(int cpu)
 {
 	struct aperfmperf *s = per_cpu_ptr(&cpu_samples, cpu);
-	unsigned int seq, freq;
 	unsigned long last;
+	unsigned int seq;
 	u64 acnt, mcnt;
 
 	if (!cpu_feature_enabled(X86_FEATURE_APERFMPERF))
-		goto fallback;
+		return -EOPNOTSUPP;
 
 	do {
 		seq = raw_read_seqcount_begin(&s->seq);
@@ -525,11 +532,19 @@ int arch_freq_get_on_cpu(int cpu)
 	 * which covers idle and NOHZ full CPUs.
 	 */
 	if (!mcnt || (jiffies - last) > MAX_SAMPLE_AGE)
-		goto fallback;
+		return -EAGAIN;
 
 	return div64_u64((cpu_khz * acnt), mcnt);
+}
+
+int arch_freq_get_on_cpu(int cpu)
+{
+	int freq = arch_freq_get_avg(cpu);
+
+	if (freq >= 0)
+		return freq;
 
-fallback:
+	/* Preserve the fallback used by scaling_cur_freq and /proc/cpuinfo. */
 	freq = cpufreq_quick_get(cpu);
 	return freq ? freq : cpu_khz;
 }
diff --git a/include/linux/cpufreq.h b/include/linux/cpufreq.h
index 35ce665edfd8..dab1d921183d 100644
--- a/include/linux/cpufreq.h
+++ b/include/linux/cpufreq.h
@@ -1224,6 +1224,7 @@ static inline int of_perf_domain_get_sharing_cpumask(int pcpu, const char *list_
 }
 #endif
 
+extern int arch_freq_get_avg(int cpu);
 extern int arch_freq_get_on_cpu(int cpu);
 
 #ifndef arch_set_freq_scale
-- 
2.20.1

[PATCH 2/3] cpufreq: Use hardware feedback for cpuinfo_avg_freq

[Chuyi Zhou]

cpuinfo_avg_freq is documented as a frequency derived from hardware
feedback, but it uses arch_freq_get_on_cpu(), which supplies a fallback
on x86 when the cached sample is unavailable. Userspace cannot distinguish
that value from a measurement. With amd-pstate-epp, for example, the
fallback is policy->min even under the performance policy. A fallback
based on the requested frequency also does not establish the frequency
at which the hardware actually ran.

Use arch_freq_get_avg() for cpuinfo_avg_freq reads and support detection.
On x86, unavailable samples then return -EAGAIN instead of a fallback
frequency, and the attribute is omitted when APERF/MPERF is unsupported.

Provide a weak default returning -EOPNOTSUPP and reuse the existing ARM64
AMU implementation. Preserve the behavior of ARM64, scaling_cur_freq
and /proc/cpuinfo.

Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
---
 Documentation/admin-guide/pm/cpufreq.rst | 11 +++++++++--
 arch/arm64/kernel/topology.c             |  7 ++++++-
 drivers/cpufreq/cpufreq.c                | 19 +++++++++++++++++--
 3 files changed, 32 insertions(+), 5 deletions(-)

diff --git a/Documentation/admin-guide/pm/cpufreq.rst b/Documentation/admin-guide/pm/cpufreq.rst
index 34baf20cc202..3057829b01bd 100644
--- a/Documentation/admin-guide/pm/cpufreq.rst
+++ b/Documentation/admin-guide/pm/cpufreq.rst
@@ -255,12 +255,19 @@ are the following:
 
         This is expected to be based on the frequency the hardware actually runs
         at and, as such, might require specialised hardware support (such as AMU
-        extension on ARM). If one cannot be determined, this attribute should
-        not be present.
+        extension on ARM or APERF/MPERF on x86). This attribute is not present
+        when hardware feedback is unsupported.
 
         Note that failed attempt to retrieve current frequency for a given
         CPU(s) will result in an appropriate error, i.e.: EAGAIN for CPU that
         remains idle (raised on ARM).
+        The attribute remains present during temporary sampling gaps.
+
+        On x86, reads use cached APERF/MPERF samples without waking the target
+        CPU to collect new samples. An expired sample or a zero MPERF delta
+        results in ``EAGAIN`` instead of a fallback to a policy or reference
+        frequency. A CPU that has just entered idle can still have a usable
+        sample, while a busy CPU excluded from periodic sampling can lack one.
 
 ``cpuinfo_max_freq``
 	Maximum possible operating frequency the CPUs belonging to this policy
diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
index d28438f8b83f..939d3e4ce5d2 100644
--- a/arch/arm64/kernel/topology.c
+++ b/arch/arm64/kernel/topology.c
@@ -181,7 +181,7 @@ void arch_cpu_idle_enter(void)
 
 #define AMU_SAMPLE_EXP_MS	20
 
-int arch_freq_get_on_cpu(int cpu)
+int arch_freq_get_avg(int cpu)
 {
 	struct amu_cntr_sample *amu_sample;
 	unsigned int start_cpu = cpu;
@@ -250,6 +250,11 @@ int arch_freq_get_on_cpu(int cpu)
 	return freq;
 }
 
+int arch_freq_get_on_cpu(int cpu)
+{
+	return arch_freq_get_avg(cpu);
+}
+
 static void amu_fie_setup(const struct cpumask *cpus)
 {
 	int cpu;
diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c
index 0d0df986fa3d..21bec6b7f538 100644
--- a/drivers/cpufreq/cpufreq.c
+++ b/drivers/cpufreq/cpufreq.c
@@ -705,9 +705,24 @@ __weak int arch_freq_get_on_cpu(int cpu)
 	return -EOPNOTSUPP;
 }
 
+/**
+ * arch_freq_get_avg() - Get an average frequency from hardware feedback
+ * @cpu: CPU to read.
+ *
+ * Provide cpuinfo_avg_freq with an average operating frequency derived
+ * from recent hardware feedback for @cpu or its frequency domain.
+ *
+ * Return: Frequency in kHz, -EOPNOTSUPP if feedback is unsupported for the
+ * CPU or policy under the current configuration.
+ */
+__weak int arch_freq_get_avg(int cpu)
+{
+	return -EOPNOTSUPP;
+}
+
 static inline bool cpufreq_avg_freq_supported(struct cpufreq_policy *policy)
 {
-	return arch_freq_get_on_cpu(policy->cpu) != -EOPNOTSUPP;
+	return arch_freq_get_avg(policy->cpu) != -EOPNOTSUPP;
 }
 
 static ssize_t show_scaling_cur_freq(struct cpufreq_policy *policy, char *buf)
@@ -769,7 +784,7 @@ static ssize_t show_cpuinfo_cur_freq(struct cpufreq_policy *policy,
 static ssize_t show_cpuinfo_avg_freq(struct cpufreq_policy *policy,
 				     char *buf)
 {
-	int avg_freq = arch_freq_get_on_cpu(policy->cpu);
+	int avg_freq = arch_freq_get_avg(policy->cpu);
 
 	if (avg_freq > 0)
 		return sysfs_emit(buf, "%u\n", avg_freq);
-- 
2.20.1

[PATCH 3/3] x86/aperfmperf: Clean up includes after removing IPI sampling

[Chuyi Zhou]

Commit f3eca381bd49 ("x86/aperfmperf: Replace arch_freq_get_on_cpu()")
removed the ad hoc sampling code, including its delays, ktime timestamps,
RCU idle checks, housekeeping checks and remote function calls. The
corresponding includes remain even though the file no longer uses those
interfaces directly.

Remove the obsolete includes and explicitly include the headers for
jiffies, sequence counters and workqueues used by the current code.

Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
---
 arch/x86/kernel/cpu/aperfmperf.c | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/arch/x86/kernel/cpu/aperfmperf.c b/arch/x86/kernel/cpu/aperfmperf.c
index de6297b1284b..8cf9decea07c 100644
--- a/arch/x86/kernel/cpu/aperfmperf.c
+++ b/arch/x86/kernel/cpu/aperfmperf.c
@@ -7,15 +7,13 @@
  * Author: Len Brown <len.brown@intel.com>
  */
 #include <linux/cpufreq.h>
-#include <linux/delay.h>
-#include <linux/ktime.h>
+#include <linux/jiffies.h>
 #include <linux/math64.h>
 #include <linux/percpu.h>
-#include <linux/rcupdate.h>
-#include <linux/sched/isolation.h>
 #include <linux/sched/topology.h>
-#include <linux/smp.h>
+#include <linux/seqlock.h>
 #include <linux/syscore_ops.h>
+#include <linux/workqueue.h>
 
 #include <asm/cpu.h>
 #include <asm/cpu_device_id.h>
-- 
2.20.1

Re: [PATCH 2/3] cpufreq: Use hardware feedback for cpuinfo_avg_freq

[Beata Michalska]

Hello Chuyi,

On Wed, Sep 16, 2026 at 10:26:22PM +0800, Chuyi Zhou wrote:
> cpuinfo_avg_freq is documented as a frequency derived from hardware
> feedback, but it uses arch_freq_get_on_cpu(), which supplies a fallback
> on x86 when the cached sample is unavailable. Userspace cannot distinguish
> that value from a measurement. With amd-pstate-epp, for example, the
> fallback is policy->min even under the performance policy. A fallback
> based on the requested frequency also does not establish the frequency
> at which the hardware actually ran.
> 
> Use arch_freq_get_avg() for cpuinfo_avg_freq reads and support detection.
> On x86, unavailable samples then return -EAGAIN instead of a fallback
> frequency, and the attribute is omitted when APERF/MPERF is unsupported.
> 
> Provide a weak default returning -EOPNOTSUPP and reuse the existing ARM64
> AMU implementation. Preserve the behavior of ARM64, scaling_cur_freq
> and /proc/cpuinfo.
> 
> Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
> ---
>  Documentation/admin-guide/pm/cpufreq.rst | 11 +++++++++--
>  arch/arm64/kernel/topology.c             |  7 ++++++-
>  drivers/cpufreq/cpufreq.c                | 19 +++++++++++++++++--
>  3 files changed, 32 insertions(+), 5 deletions(-)
> 
> diff --git a/Documentation/admin-guide/pm/cpufreq.rst b/Documentation/admin-guide/pm/cpufreq.rst
> index 34baf20cc202..3057829b01bd 100644
> --- a/Documentation/admin-guide/pm/cpufreq.rst
> +++ b/Documentation/admin-guide/pm/cpufreq.rst
> @@ -255,12 +255,19 @@ are the following:
>  
>          This is expected to be based on the frequency the hardware actually runs
>          at and, as such, might require specialised hardware support (such as AMU
> -        extension on ARM). If one cannot be determined, this attribute should
> -        not be present.
> +        extension on ARM or APERF/MPERF on x86). This attribute is not present
> +        when hardware feedback is unsupported.
>  
>          Note that failed attempt to retrieve current frequency for a given
>          CPU(s) will result in an appropriate error, i.e.: EAGAIN for CPU that
>          remains idle (raised on ARM).
> +        The attribute remains present during temporary sampling gaps.
That's bit ambigous - what are 'temporary sampling gaps' ?
Sampling did not take place, samples were outdated or somewhat invalid ?
Also, isn't that implied by the sentence above.
> +
> +        On x86, reads use cached APERF/MPERF samples without waking the target
Is there a case when the target is woken up for this particular attribute
readings ?
> +        CPU to collect new samples. An expired sample or a zero MPERF delta
> +        results in ``EAGAIN`` instead of a fallback to a policy or reference
> +        frequency. A CPU that has just entered idle can still have a usable
> +        sample, while a busy CPU excluded from periodic sampling can lack one.
>  
>  ``cpuinfo_max_freq``
>  	Maximum possible operating frequency the CPUs belonging to this policy
> diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
> index d28438f8b83f..939d3e4ce5d2 100644
> --- a/arch/arm64/kernel/topology.c
> +++ b/arch/arm64/kernel/topology.c
> @@ -181,7 +181,7 @@ void arch_cpu_idle_enter(void)
>  
>  #define AMU_SAMPLE_EXP_MS	20
>  
> -int arch_freq_get_on_cpu(int cpu)
> +int arch_freq_get_avg(int cpu)
>  {
>  	struct amu_cntr_sample *amu_sample;
>  	unsigned int start_cpu = cpu;
> @@ -250,6 +250,11 @@ int arch_freq_get_on_cpu(int cpu)
>  	return freq;
>  }
>  
> +int arch_freq_get_on_cpu(int cpu)
> +{
> +	return arch_freq_get_avg(cpu);
> +}
> +
I do not think this is needed.

---
BR
Beata
>  static void amu_fie_setup(const struct cpumask *cpus)
>  {
>  	int cpu;
> diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c
> index 0d0df986fa3d..21bec6b7f538 100644
> --- a/drivers/cpufreq/cpufreq.c
> +++ b/drivers/cpufreq/cpufreq.c
> @@ -705,9 +705,24 @@ __weak int arch_freq_get_on_cpu(int cpu)
>  	return -EOPNOTSUPP;
>  }
>  
> +/**
> + * arch_freq_get_avg() - Get an average frequency from hardware feedback
> + * @cpu: CPU to read.
> + *
> + * Provide cpuinfo_avg_freq with an average operating frequency derived
> + * from recent hardware feedback for @cpu or its frequency domain.
> + *
> + * Return: Frequency in kHz, -EOPNOTSUPP if feedback is unsupported for the
> + * CPU or policy under the current configuration.
> + */
> +__weak int arch_freq_get_avg(int cpu)
> +{
> +	return -EOPNOTSUPP;
> +}
> +
>  static inline bool cpufreq_avg_freq_supported(struct cpufreq_policy *policy)
>  {
> -	return arch_freq_get_on_cpu(policy->cpu) != -EOPNOTSUPP;
> +	return arch_freq_get_avg(policy->cpu) != -EOPNOTSUPP;
>  }
>  
>  static ssize_t show_scaling_cur_freq(struct cpufreq_policy *policy, char *buf)
> @@ -769,7 +784,7 @@ static ssize_t show_cpuinfo_cur_freq(struct cpufreq_policy *policy,
>  static ssize_t show_cpuinfo_avg_freq(struct cpufreq_policy *policy,
>  				     char *buf)
>  {
> -	int avg_freq = arch_freq_get_on_cpu(policy->cpu);
> +	int avg_freq = arch_freq_get_avg(policy->cpu);
>  
>  	if (avg_freq > 0)
>  		return sysfs_emit(buf, "%u\n", avg_freq);
> -- 
> 2.20.1