Re: [RFT][PATCH v1 0/5] cpuidle: menu: Avoid discarding useful information when processing recent idle intervals

[Aboorva Devarajan <aboorvad@linux.ibm.com>]

On Mon, 2025-02-24 at 11:57 +0530, Aboorva Devarajan wrote:

> On Thu, 2025-02-06 at 15:21 +0100, Rafael J. Wysocki wrote:
> 
> > Hi Everyone,
> > 
> > ...
> > 
> > 
> > 
> 
> Hi Rafael,
> 
> I ran some tests using a cpuidle microbenchmark that I have [1]:

I missed including the system configuration in my previous email, so here it is,

I carried out these experiments on a Power10 - pseries logical partition with
the following configuration:


$ lscpu
Architecture:             ppc64le
  Byte Order:             Little Endian
CPU(s):                   80
  On-line CPU(s) list:    0-79
Model name:               POWER10 (architected), altivec supported
  Model:                  2.0 (pvr 0080 0200)
  Thread(s) per core:     8
  Core(s) per socket:     10
  Socket(s):              1
  Physical sockets:       8
  Physical chips:         1
  Physical cores/chip:    10
Virtualization features:  
  Hypervisor vendor:      pHyp
  Virtualization type:    para
Caches (sum of all):      
  L1d:                    640 KiB (20 instances)
  L1i:                    960 KiB (20 instances)
  L2:                     20 MiB (20 instances)
  L3:                     80 MiB (20 instances)
NUMA:                     
  NUMA node(s):           1
  NUMA node4 CPU(s):      0-79

$ cpupower idle-info
CPUidle driver: pseries_idle
CPUidle governor: menu
analyzing CPU 47:

Number of idle states: 2
Available idle states: snooze CEDE
snooze:
Flags/Description: snooze
Latency: 0
Residency: 0
Usage: 5899
Duration: 360940
CEDE:
Flags/Description: CEDE
Latency: 12
Residency: 120
Usage: 31037
Duration: 16664409173


> 
> The test does a uniform distribution of idle durations, which means the past eight idle intervals approximately match 
> the sleep duration. So as anticipated, this change-set does not impact this case, and the behavior remains mostly
> consistent even after applying the patch.
> 
>  Pipe based wakeup - Above %
>  ----------------------------------------------------------------------
> > Sleep time (s)  | Menu No Patch(%) | Menu Patch(%)| Teo No Patch (%) |
> > ---------------- |------------------|--------------|------------------|
> > 5               | 0.00%            | 0.00%        | 0.00%            |
> > 10              | 0.00%            | 0.00%        | 0.00%            |
> ------------------------------------------------------------------------
> > 30              | 99.97%           | 99.96%       | 0.00%            |
> > 100             | 98.61%           | 98.67%       | 0.00%            |
> > 120             | 97.88%           | 98.42%       | 1.03%            | -> (*)
> ------------------------------------------------------------------------
> > 130             | 2.82%            | 1.26%        | 0.22%            |
> > 150             | 1.68%            | 1.43%        | 0.32%            |
> > 200             | 2.09%            | 1.91%        | 0.15%            |
> > 300             | 1.22%            | 1.22%        | 0.19%            |
> > 400             | 1.20%            | 1.21%        | 0.19%            |
> > 500             | 1.16%            | 1.12%        | 0.12%            |
> ------------------------------------------------------------------------
> 
>  Pipe based wakeup - Below %
>  -----------------------------------------------------------------------
> > Sleep time (s)  | Menu No Patch(%) | Menu Patch(%)| Teo No Patch (%) |
> > ---------------- |------------------|--------------|------------------|
> > 5               | 0.00%            | 0.00%        | 0.00%            |
> > 10              | 0.00%            | 0.00%        | 0.00%            |
> > 30              | 0.00%            | 0.00%        | 0.00%            |
> > 100             | 0.00%            | 0.00%        | 0.00%            |
> > 120             | 2.76%            | 1.14%        | 0.93%            |
> > 130             | 3.11%            | 1.13%        | 0.12%            |
> > 150             | 1.34%            | 1.16%        | 0.18%            |
> > 200             | 1.38%            | 1.15%        | 0.09%            |
> > 300             | 1.33%            | 1.21%        | 0.11%            |
> > 400             | 1.36%            | 1.25%        | 0.10%            |
> > 500             | 1.25%            | 1.22%        | 0.10%            |
> > ---------------- |------------------|--------------|------------------|
> 
>  Time based wakeup - Above %
>  -----------------------------------------------------------------------
> > Sleep time (s)  | Menu No Patch(%) | Menu Patch(%)| Teo No Patch (%) |
> > ---------------- |------------------|--------------|------------------|
> > 5               | 0.00%            | 0.00%        | 0.00%            |
> > 10              | 0.00%            | 0.00%        | 0.00%            |
> > 30              | 0.00%            | 0.00%        | 0.00%            |
> > 100             | 0.01%            | 0.00%        | 0.00%            |
> > 120             | 0.00%            | 0.00%        | 0.15%            |
> > 130             | 15.84%           | 0.14%        | 0.23%            |
> > 150             | 0.39%            | 0.23%        | 0.48%            |
> > 200             | 0.95%            | 0.87%        | 0.10%            |
> > 300             | 0.20%            | 0.17%        | 0.15%            |
> > 400             | 0.14%            | 0.12%        | 0.17%            |
> > 500             | 0.10%            | 0.20%        | 0.11%            |
> > ---------------- |------------------|--------------|------------------|
> 
>  Time based wakeup - Below %
>  -----------------------------------------------------------------------
> > Sleep time (s)  | Menu No Patch(%) | Menu Patch(%)| Teo No Patch (%) |
> > ---------------- |------------------|--------------|------------------|
> > 5               | 0.00%            | 0.00%        | 0.00%            |
> > 10              | 0.00%            | 0.00%        | 0.00%            |
> > 30              | 0.00%            | 0.00%        | 0.00%            |
> > 100             | 0.00%            | 0.00%        | 0.00%            |
> > 120             | 1.85%            | 1.66%        | 2.67%            |
> > 130             | 16.71%           | 1.13%        | 1.11%            |
> > 150             | 1.36%            | 1.16%        | 1.13%            |
> > 200             | 1.33%            | 1.14%        | 1.19%            |
> > 300             | 1.44%            | 1.20%        | 1.17%            |
> > 400             | 1.51%            | 1.21%        | 1.21%            |
> > 500             | 1.42%            | 1.24%        | 1.25%            |
> > ---------------- |------------------|--------------|------------------|
> 
> 
> (*) - Above and below values are higher even without the patch with menu governor,
>       this issue still persists, as previously reported in [2]. I will investigate
>       further and submit a revision to get additional feedback.
> 
> I also carried out some benchmarks using pgbench:
> 
> pgbench Results
> 
> Without Patch:
> -----------------------------------------------------------------------------
> > Run  | Transactions Processed | Latency Avg (ms) | TPS (without init time) |
> -----------------------------------------------------------------------------
> >  1   |  11,936,327            |  0.050           |  198,946.141025         |
> >  2   |  11,899,540            |  0.050           |  198,333.097547         |
> >  3   |  11,870,792            |  0.051           |  197,853.728614         |
> >  4   |  11,901,670            |  0.050           |  198,368.526139         |
> >  5   |  11,922,046            |  0.050           |  198,708.112243         |
> -----------------------------------------------------------------------------
> > Avg  |  11,906,075            |  0.0502          |  198,441.921114         |
> -----------------------------------------------------------------------------
> 
> With Patch:
> -----------------------------------------------------------------------------
> > Run  | Transactions Processed | Latency Avg (ms) | TPS (without init time) |
> -----------------------------------------------------------------------------
> >  1   |  12,052,865            |  0.050           |  200,888.492771         |
> >  2   |  12,058,359            |  0.050           |  200,979.895325         |
> >  3   |  12,071,012            |  0.050           |  201,190.809734         |
> >  4   |  12,054,646            |  0.050           |  200,918.076736         |
> >  5   |  12,053,087            |  0.050           |  200,892.045581         |
> -----------------------------------------------------------------------------
> > Avg  |  12,058,394            |  0.0500          |  200,973.464029         |
> ------------------------------------------------------------------------------
> 
> Performance Improvement After Patch:
> --------------------------------------------------------------------------------------
> > Metric                  | Without Patch (Avg.)| With Patch (Avg.)  | % Improvement |
> --------------------------------------------------------------------------------------
> > Transactions Processed  |  11,906,075         |  12,058,394        |  +1.28%       | 
> > Latency Avg (ms)        |  0.0502             |  0.0500            |  -0.40%       |
> > TPS (without init time) |  198,441.921114     |  200,973.464029    |  +1.28%       |
> --------------------------------------------------------------------------------------
> 
> So with the patch pgbench's "Transactions Processed" improves by ~1.28%, and I did not observe
> any major variations on other benchmarks with the patch.
> 
> So for the entire series:
> 
> Tested-by: Aboorva Devarajan <aboorvad@linux.ibm.com>
> 
> I'm also trying a minimal unit fuzz-test with the pre- and post- patched version of the get_typical_interval 
> function to understand this better, will post the results soon.
> 
> [1] - https://github.com/AboorvaDevarajan/linux-utils/tree/main/cpuidle/cpuidle_wakeup
> [2] - https://lore.kernel.org/all/20240809073120.250974-1-aboorvad@linux.ibm.com/
> 
> 
> Thanks,
> Aboorva