From: Dhananjay Ugwekar <Dhananjay.Ugwekar@amd.com>
To: K Prateek Nayak <kprateek.nayak@amd.com>
Cc: linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-hardening@vger.kernel.org, ananth.narayan@amd.com,
gautham.shenoy@amd.com, ravi.bangoria@amd.com,
sandipan.das@amd.com, linux-pm@vger.kernel.org,
rui.zhang@intel.com, oleksandr@natalenko.name,
peterz@infradead.org, mingo@redhat.com, acme@kernel.org,
namhyung@kernel.org, mark.rutland@arm.com,
alexander.shishkin@linux.intel.com, jolsa@kernel.org,
irogers@google.com, adrian.hunter@intel.com,
kan.liang@linux.intel.com, tglx@linutronix.de, bp@alien8.de,
dave.hansen@linux.intel.com, x86@kernel.org, kees@kernel.org,
gustavoars@kernel.org
Subject: Re: [PATCH v2 0/9] Add per-core RAPL energy counter support for AMD CPUs
Date: Fri, 21 Jun 2024 14:19:13 +0530 [thread overview]
Message-ID: <79fd7c1d-2d3a-46b4-8a0c-56bfda036a94@amd.com> (raw)
In-Reply-To: <a26b9774-f9da-763e-aebf-5d66a6d44377@amd.com>
Hello Prateek,
On 6/21/2024 1:54 PM, K Prateek Nayak wrote:
> Hello Dhananjay,
>
> On 6/20/2024 6:26 PM, Dhananjay Ugwekar wrote:
>> Currently the energy-cores event in the power PMU aggregates energy
>> consumption data at a package level. On the other hand the core energy
>> RAPL counter in AMD CPUs has a core scope (which means the energy
>> consumption is recorded separately for each core). Earlier efforts to add
>> the core event in the power PMU had failed [1], due to the difference in
>> the scope of these two events. Hence, there is a need for a new core scope
>> PMU.
>>
>> This patchset adds a new "power_per_core" PMU alongside the existing
>> "power" PMU, which will be responsible for collecting the new
>> "energy-per-core" event.
>>
>> Tested the package level and core level PMU counters with workloads
>> pinned to different CPUs.
>>
>> Results with workload pinned to CPU 1 in Core 1 on an AMD Zen4 Genoa
>> machine:
>>
>> $ perf stat -a --per-core -e power_per_core/energy-per-core/ sleep 1
>
> When testing this on a 2P 3rd Generation EPYC System (2 x 64/128T), I
> ran into an issue where it seems like the energy reporting for the
> system is coming from the second socket. Following are the CPUs on each
> socket of the system:
>
> Node 0: 0-63, 128-191
> Node 1: 64-127, 192-255
>
> Following are the experiments I ran:
>
> $ # Run a busy loop on each thread of the first socket
> $ for i in `seq 0 63` `seq 128 191`; do taskset -c $i ~/scripts/loop & done
> $ sudo perf stat -a --per-core -e power_per_core/energy-per-core/ -- sleep 5
>
> S0-D0-C0 1 0.00 Joules power_per_core/energy-per-core/
> S0-D0-C1 1 0.00 Joules power_per_core/energy-per-core/
> S0-D0-C2 1 0.00 Joules power_per_core/energy-per-core/
> S0-D0-C3 1 0.00 Joules power_per_core/energy-per-core/
> ...
> S0-D0-C63 1 0.00 Joules power_per_core/energy-per-core/
> S1-D1-C0 1 0.00 Joules power_per_core/energy-per-core/
> S1-D1-C1 1 0.00 Joules power_per_core/energy-per-core/
> S1-D1-C2 1 0.00 Joules power_per_core/energy-per-core/
> S1-D1-C3 1 0.00 Joules power_per_core/energy-per-core/
> ...
> S1-D1-C63 1 0.00 Joules power_per_core/energy-per-core/
>
> From the energy data, it looks as if the system is entirely idle.
>
> If I repeat the same, pinning the running busy loop on the threads of
> second socket, I see the following:
>
> $ # Run a busy loop on each thread of the second socket
> $ for i in `seq 64 127` `seq 192 255`; do taskset -c $i ~/scripts/loop & done
> $ sudo perf stat -a --per-core -e power_per_core/energy-per-core/ -- sleep 5
>
> S0-D0-C0 1 11.79 Joules power_per_core/energy-per-core/
> S0-D0-C1 1 11.80 Joules power_per_core/energy-per-core/
> S0-D0-C2 1 11.90 Joules power_per_core/energy-per-core/
> S0-D0-C3 1 11.88 Joules power_per_core/energy-per-core/
> ...
> S0-D0-C63 1 11.76 Joules power_per_core/energy-per-core/
> S1-D1-C0 1 11.81 Joules power_per_core/energy-per-core/
> S1-D1-C1 1 11.80 Joules power_per_core/energy-per-core/
> S1-D1-C2 1 11.90 Joules power_per_core/energy-per-core/
> S1-D1-C3 1 11.88 Joules power_per_core/energy-per-core/
> ...
> S1-D1-C63 1 11.76 Joules power_per_core/energy-per-core/
>
> The whole system seems to be busy this time around. I've verified that
> only half the system is busy using htop in either case.
>
> Running some more experiments, I see the following:
>
> $ taskset -c 1 ~/scripts/loop& # First thread from Core 1, Socket
> $ sudo perf stat -a --per-core -e power_per_core/energy-per-core/ -- sleep 5
>
> S0-D0-C0 1 0.02 Joules power_per_core/energy-per-core/
> S0-D0-C1 1 0.21 Joules power_per_core/energy-per-core/
> S0-D0-C2 1 0.20 Joules power_per_core/energy-per-core/
> S0-D0-C3 1 0.00 Joules power_per_core/energy-per-core/
> ...
> (Seemingly idle system)
>
>
> $ taskset -c 65 ~/scripts/loop&
> $ sudo perf stat -a --per-core -e power_per_core/energy-per-core/ -- sleep 5
>
> S0-D0-C0 1 0.01 Joules power_per_core/energy-per-core/
> S0-D0-C1 1 16.73 Joules power_per_core/energy-per-core/
> S0-D0-C2 1 0.00 Joules power_per_core/energy-per-core/
> S0-D0-C3 1 0.00 Joules power_per_core/energy-per-core/
> ...
> S0-D0-C63 1 0.00 Joules power_per_core/energy-per-core/
> S1-D1-C0 1 0.01 Joules power_per_core/energy-per-core/
> S1-D1-C1 1 16.73 Joules power_per_core/energy-per-core/
> S1-D1-C2 1 0.00 Joules power_per_core/energy-per-core/
> S1-D1-C3 1 0.00 Joules power_per_core/energy-per-core/
> ...
> S1-D1-C63 1 0.00 Joules power_per_core/energy-per-core/
>
> (Core 1 from both sockets look busy reporting identical energy
> values)
>
> Hope it helps narrow down the issue.
I think my assumption that topology_core_id() will return a unique core ID
across the system might not be correct. It seems the core ID is unique only
within a package, will fix this in the next version.
Thanks a lot for testing and helping narrow down the issue!
Regards,
Dhananjay
> >>
>> Performance counter stats for 'system wide':
>>
>> S0-D0-C0 1 0.02 Joules power_per_core/energy-per-core/
>> S0-D0-C1 1 5.72 Joules power_per_core/energy-per-core/
>> S0-D0-C2 1 0.02 Joules power_per_core/energy-per-core/
>> S0-D0-C3 1 0.02 Joules power_per_core/energy-per-core/
>> S0-D0-C4 1 0.02 Joules power_per_core/energy-per-core/
>> S0-D0-C5 1 0.02 Joules power_per_core/energy-per-core/
>> S0-D0-C6 1 0.02 Joules power_per_core/energy-per-core/
>> S0-D0-C7 1 0.02 Joules power_per_core/energy-per-core/
>> S0-D0-C8 1 0.02 Joules power_per_core/energy-per-core/
>> S0-D0-C9 1 0.02 Joules power_per_core/energy-per-core/
>> S0-D0-C10 1 0.02 Joules power_per_core/energy-per-core/
>>
>> [1]: https://lore.kernel.org/lkml/3e766f0e-37d4-0f82-3868-31b14228868d@linux.intel.com/
>>
>> This patchset applies cleanly on top of v6.10-rc4 as well as latest
>> tip/master.
>
> P.S. I tested these changes on top of tip:perf/core
>
>>
>> [..snip..]
>>
>
prev parent reply other threads:[~2024-06-21 8:49 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-06-20 12:56 Dhananjay Ugwekar
2024-06-20 12:56 ` [PATCH v2 1/9] perf/x86/rapl: Fix the energy-pkg event " Dhananjay Ugwekar
2024-06-20 12:56 ` [PATCH v2 2/9] perf/x86/rapl: Rename rapl_pmu variables Dhananjay Ugwekar
2024-06-20 12:56 ` [PATCH v2 3/9] perf/x86/rapl: Make rapl_model struct global Dhananjay Ugwekar
2024-06-20 12:56 ` [PATCH v2 4/9] perf/x86/rapl: Move cpumask variable to rapl_pmus struct Dhananjay Ugwekar
2024-06-20 12:56 ` [PATCH v2 5/9] perf/x86/rapl: Add wrapper for online/offline functions Dhananjay Ugwekar
2024-06-20 12:57 ` [PATCH v2 6/9] perf/x86/rapl: Add an argument to the cleanup and init functions Dhananjay Ugwekar
2024-06-20 12:57 ` [PATCH v2 7/9] perf/x86/rapl: Modify the generic variable names to *_pkg* Dhananjay Ugwekar
2024-06-20 12:57 ` [PATCH v2 8/9] perf/x86/rapl: Remove the global variable rapl_msrs Dhananjay Ugwekar
2024-06-20 12:57 ` [PATCH v2 9/9] perf/x86/rapl: Add per-core energy counter support for AMD CPUs Dhananjay Ugwekar
2024-06-21 8:24 ` [PATCH v2 0/9] Add per-core RAPL " K Prateek Nayak
2024-06-21 8:49 ` Dhananjay Ugwekar [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=79fd7c1d-2d3a-46b4-8a0c-56bfda036a94@amd.com \
--to=dhananjay.ugwekar@amd.com \
--cc=acme@kernel.org \
--cc=adrian.hunter@intel.com \
--cc=alexander.shishkin@linux.intel.com \
--cc=ananth.narayan@amd.com \
--cc=bp@alien8.de \
--cc=dave.hansen@linux.intel.com \
--cc=gautham.shenoy@amd.com \
--cc=gustavoars@kernel.org \
--cc=irogers@google.com \
--cc=jolsa@kernel.org \
--cc=kan.liang@linux.intel.com \
--cc=kees@kernel.org \
--cc=kprateek.nayak@amd.com \
--cc=linux-hardening@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=linux-pm@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=mingo@redhat.com \
--cc=namhyung@kernel.org \
--cc=oleksandr@natalenko.name \
--cc=peterz@infradead.org \
--cc=ravi.bangoria@amd.com \
--cc=rui.zhang@intel.com \
--cc=sandipan.das@amd.com \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®