* [PATCH v2 0/6] perf: add PMU driver for the MIPS Coherence Manager 3 performance counters
@ 2026-11-12 12:28 Vladimir Kondratiev
2026-11-12 12:28 ` [PATCH v2 1/6] MIPS: CM: Make cm_core_lock raw spinlocks Vladimir Kondratiev
` (5 more replies)
0 siblings, 6 replies; 8+ messages in thread
From: Vladimir Kondratiev @ 2026-11-12 12:28 UTC (permalink / raw)
To: Thomas Bogendoerfer, Sebastian Andrzej Siewior, Clark Williams,
Steven Rostedt, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Jiaxun Yang, Jonathan Corbet, Shuah Khan, Randy Dunlap,
Will Deacon, Mark Rutland, Kees Cook, Gustavo A. R. Silva,
Benoît Monin, Gregory CLEMENT, Théo Lebrun,
Fu Chao-Ying
Cc: Vladimir Kondratiev, linux-mips, linux-kernel, linux-rt-devel,
devicetree, linux-doc, linux-arm-kernel, linux-perf-users,
linux-hardening, Paul Burton
The Coherence Manager 3 (CM3) of the MIPS I6500 multiprocessing system
is the point where the requests of all cores, I/O coherence units and
the hardware prefetcher are serialized before reaching the shared L2
cache and the memory interface. It carries a small set of performance
counters which observe this traffic: two programmable 32-bit event
counters and a 32-bit cycle counter, programmed through the GCR global
debug block (GCR_DB_PC_*).
The counters can count requests serialized by the CM, requests received
by an IOCU, requests issued on the memory interface, AXI handshake
cycles on the memory and IOCU buses, and PDTrace statistics. Each event
is refined by a 64-bit qualifier made of groups of bits (originating
port, thread, command, cacheability, L1/L2 line state, eviction, bank,
scheduler, ...): a request is counted when it matches a set bit in
every group, so the same event can be narrowed down to, for example,
L2 misses caused by coherent reads from core 1.
This is the only visibility into the L2 and the memory interface that
the I6500 offers; the per-core counters stop at the L1. It is the
natural place to attribute memory bandwidth to cores and IOCUs, or to
measure the L2 hit rate of a workload, on SoCs such as the Mobileye
EyeQ6H.
This series adds an uncore perf driver for these counters.
Design notes
------------
The driver binds to the existing coherency-manager device tree node
("mti,mips-cm" / "mobileye,eyeq6-cm") and reuses the GCR mapping
already set up by mips_cm_probe(), so no new reg property is needed:
the GCR base is discovered from CP0 CMGCRBase as for everything else in
the CM. The register definitions go into asm/mips-cm.h next to the
other GCRs, following the existing accessor scheme.
Events are described by two fields: 'event' (config:0-8) is the number
written to GCR_DB_PC_EVENT, with 0x100 as a pseudo event selecting the
cycle counter, and 'qual' (config1:0-63) is the raw qualifier. A
qualifier of 0 picks a per-event default counting every occurrence;
this matters for the I/O and AXI events, whose qualifiers contain
encoded values (IOCU number, AXI channel) rather than masks, so "all
ones" is not a sensible default there. Named events cover the common
combinations and, since they are plain event=/qual= strings, serve as
templates for custom qualifiers.
A multi-cluster system has one CM3, and thus one set of counters, per
cluster. The number of clusters comes from GCR_CONFIG and one PMU is
registered per cluster, named mips_cm3_pmu_<cluster>. A cluster need
not have CPUs, online or at all, for its CM to be busy: its L2 and
memory interface may serve the I/O devices behind its IOCUs while its
cores are absent or powered down. The counters of a cluster are thus
accessed through the GCR redirect block (GCR_Cx_OTHER with BLOCK set
to the debug block) from whichever CPU the PMU runs on, which may be in
any cluster. The one thing required is that the CM of the counted
cluster is powered, which the driver checks in the CPC (the CM "core"
STAT_CONF sequencer state) when an event is opened; the counters of a
cluster that is powered down at probe time are initialized on first
use, and events on it fail with -ENODEV until then. The counters are
shared by the whole cluster: events are system-wide only, pinned to one
CPU which follows CPU hotplug, and sampling, task events and the
exclude_* attributes are rejected.
A cluster holding CPUs is a special case of the above: Linux leaves
CPC_PWRUP_CTL.CM_PWRUP clear once it has booted a cluster, so the CM
powers down together with the last of its cores and the counters go
with it. The hotplug callbacks track this: when the last online CPU
of a cluster goes down and CM_PWRUP is clear, the PMU of that cluster
is paused (counts harvested, running events kept but not counting,
new events refused with -ENODEV) and it is resumed, the events
restarted, when a CPU of the cluster comes back online and the CM
with it. Pause and resume run on the PMU's CPU with interrupts
disabled, like the perf callbacks, so they do not race with them;
the pause precedes the death of the last CPU, hence the power down,
so everything that touches the counters either runs on that CPU
with interrupts disabled or is waited for by the pause (the poll
timer). The remaining exposure is a cluster without CPUs whose power
is not managed by this kernel: only the CPC check at event open
covers it.
Overflow is where the SoC integration leaks in. The CM3 has a
so_cm_perf_cnt_int output, but whether it is wired to a GIC input is
the integrator's choice and is not discoverable. The binding therefore
gains an optional 'interrupts' property: when present the driver uses
it to extend the 32-bit counters, otherwise it polls them every 250 ms,
which is comfortably below the time the fastest event needs to wrap
at the CM3 clock rates. Both paths yield 64-bit counts. In interrupt
mode the counter is only read on demand and may make a full turn
between two reads, so the overflow status is folded into the delta.
No DTS change is included because the EyeQ6H routing of that signal
is not documented on my side; the driver works in polling mode
without it.
The interrupt can only serve the cluster the CPUs live in, and only
when they all live in that one cluster. Each cluster's CM drives its
own interrupt output into its own GIC, and the GIC driver routes a
shared interrupt to the cluster of its affinity CPU, so a single
'interrupts' entry cannot serve several clusters at once. The PMUs of
the other clusters, and all PMUs when CPUs are spread over clusters,
are polled.
The driver can be built as a module. That requires exporting
mips_gcr_base, mips_cpc_base, mips_cm_{lock,unlock}_other() and, less
obviously, mips_cm_is64, since the CPS accessor macros consult it on
every 64-bit GCR access. All are exported _GPL.
The redirect block is reached from the perf callbacks, which run under
the raw perf context lock, and from the polling timer in hard
interrupt context. mips_cm_lock_other() takes cm_core_lock, a plain
spinlock, which is an invalid wait context there and on PREEMPT_RT.
Paul Burton's patch turning it into a raw spinlock, posted in 2019 but
never merged, is picked up as the first patch of the series.
Patches
-------
Patch 1 is Paul Burton's conversion of cm_core_lock to a raw spinlock.
Patch 2 adds the GCR_DB_PC_* register and field definitions, with
local and redirect accessors.
Patch 3 exports the CM and CPC symbols needed by a modular user of the
GCR accessors.
Patch 4 extends the mti,mips-cm binding with the optional performance
counter interrupt.
Patch 5 is the driver itself together with its admin-guide document,
which describes the perf interface and transcribes the event numbers
and qualifier layouts so that users do not need the TRM to build a
qualifier.
Patch 6 enables the driver (as a module) in eyeq6_defconfig.
Testing
-------
Built with eyeq6_defconfig with the driver both built-in and as a
module; every patch of the series builds on its own, and the binding
and the document pass their respective checkers.
Run on the EyeQ6H board with 2 clusters, loaded as module in polling
mode. The overflow interrupt path could not be validated on real
hardware: I have no platform where so_cm_perf_cnt_int is wired to the
GIC, so that mode is build-tested and reviewed only.
[ 66.128375] mips_cm3_pmu coherency-manager: mips_cm3_pmu_0: 2 counters + cycles, polled
[ 66.129617] mips_cm3_pmu coherency-manager: mips_cm3_pmu_1: 2 counters + cycles, polled
dev / # perf stat -a -e mips_cm3_pmu_0/cm_requests_l2_hit/,mips_cm3_pmu_0/cm_requests_l2_miss/,mips_cm3_p
mu_1/cm_requests_l2_hit/,mips_cm3_pmu_1/cm_requests_l2_miss/,mips_cm3_pmu_0/cycles/,mips_cm3_pmu_1/cycles
/ sleep 1
Performance counter stats for 'system wide':
505821 mips_cm3_pmu_0/cm_requests_l2_hit/
49083 mips_cm3_pmu_0/cm_requests_l2_miss/
184893 mips_cm3_pmu_1/cm_requests_l2_hit/
37760 mips_cm3_pmu_1/cm_requests_l2_miss/
1807514589 mips_cm3_pmu_0/cycles/
1807491360 mips_cm3_pmu_1/cycles/
1.005857149 seconds time elapsed
Signed-off-by: Vladimir Kondratiev <vladimir.kondratiev@mobileye.com>
---
Changes in v2:
- Pick up Paul Burton's "MIPS: CM: Make cm_core_lock raw spinlocks" as
patch 1: the redirect block is accessed under the raw perf context
lock and from hard interrupt context (Sashiko).
- Account a full counter turn in interrupt mode: fold the overflow
status into the delta instead of relying on the 32-bit difference
alone, which lost 2^32 on every wrap (Sashiko).
- Keep the 64-bit qualifier in hwc->config1 rather than the
unsigned long hwc->config_base, truncated on 32-bit (Sashiko).
- Set hwc->config before validating the group, and do not walk the
siblings of an event that is its own leader: it has none yet and
its context is not set (Sashiko).
- Use cpuhp_state_remove_instance_nocalls() so that the unregistered
PMU is not migrated (Sashiko).
- Check that the cluster is powered on every event open, not only
the first time (Sashiko).
- Pause the PMU of a cluster when its last online CPU goes down and
the CM is set to power down with its cores, resume it when a CPU
of the cluster comes back online; closes the window where the
counters of a cluster could vanish under running events. The
poll timer stops with the pause and is re-armed by the resume.
- Note in the cover letter that the interrupt mode was not validated
on hardware.
- Link to v1: https://patch.msgid.link/20261108-i6500-cm3-pmu-v1-0-bd119584891a@mobileye.com
To: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
To: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
To: Clark Williams <clrkwllms@kernel.org>
To: Steven Rostedt <rostedt@goodmis.org>
To: Rob Herring <robh@kernel.org>
To: Krzysztof Kozlowski <krzk+dt@kernel.org>
To: Conor Dooley <conor+dt@kernel.org>
To: Jiaxun Yang <jiaxun.yang@flygoat.com>
To: Jonathan Corbet <corbet@lwn.net>
To: Shuah Khan <skhan@linuxfoundation.org>
To: Randy Dunlap <rdunlap@infradead.org>
To: Will Deacon <will@kernel.org>
To: Mark Rutland <mark.rutland@arm.com>
To: Kees Cook <kees@kernel.org>
To: "Gustavo A. R. Silva" <gustavoars@kernel.org>
To: Vladimir Kondratiev <vladimir.kondratiev@mobileye.com>
To: Benoît Monin <benoit.monin@bootlin.com>
To: Gregory CLEMENT <gregory.clement@bootlin.com>
To: Théo Lebrun <theo.lebrun@bootlin.com>
To: Fu Chao-Ying <chao-ying.fu@globalfoundries.com>
Cc: linux-mips@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: linux-rt-devel@lists.linux.dev
Cc: devicetree@vger.kernel.org
Cc: linux-doc@vger.kernel.org
Cc: linux-arm-kernel@lists.infradead.org
Cc: linux-perf-users@vger.kernel.org
Cc: linux-hardening@vger.kernel.org
---
Paul Burton (1):
MIPS: CM: Make cm_core_lock raw spinlocks
Vladimir Kondratiev (5):
MIPS: CM: add CM3 performance counter GCR definitions
MIPS: CM: export the symbols needed to access the GCRs from a module
dt-bindings: mips: mti,mips-cm: add performance counter interrupt
perf: add MIPS Coherence Manager 3 (CM3) PMU driver
MIPS: eyeq6_defconfig: enable the CM3 PMU
Documentation/admin-guide/perf/index.rst | 1 +
Documentation/admin-guide/perf/mips_cm3_pmu.rst | 346 ++++++
.../devicetree/bindings/mips/mti,mips-cm.yaml | 6 +
arch/mips/configs/eyeq6_defconfig | 2 +
arch/mips/include/asm/mips-cm.h | 51 +-
arch/mips/kernel/mips-cm.c | 25 +-
arch/mips/kernel/mips-cpc.c | 2 +
drivers/perf/Kconfig | 12 +
drivers/perf/Makefile | 1 +
drivers/perf/mips_cm3_pmu.c | 1237 ++++++++++++++++++++
10 files changed, 1672 insertions(+), 11 deletions(-)
---
base-commit: c03c1a14507dfaf8b856c2608f9ac216021fc222
change-id: 20261101-i6500-cm3-pmu-4a11f399b430
Best regards,
--
Vladimir Kondratiev <vladimir.kondratiev@mobileye.com>
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v2 1/6] MIPS: CM: Make cm_core_lock raw spinlocks
2026-11-12 12:28 [PATCH v2 0/6] perf: add PMU driver for the MIPS Coherence Manager 3 performance counters Vladimir Kondratiev
@ 2026-11-12 12:28 ` Vladimir Kondratiev
2026-11-12 12:39 ` sashiko-bot
2026-11-12 12:28 ` [PATCH v2 2/6] MIPS: CM: add CM3 performance counter GCR definitions Vladimir Kondratiev
` (4 subsequent siblings)
5 siblings, 1 reply; 8+ messages in thread
From: Vladimir Kondratiev @ 2026-11-12 12:28 UTC (permalink / raw)
To: Thomas Bogendoerfer, Sebastian Andrzej Siewior, Clark Williams,
Steven Rostedt, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Jiaxun Yang, Jonathan Corbet, Shuah Khan, Randy Dunlap,
Will Deacon, Mark Rutland, Kees Cook, Gustavo A. R. Silva,
Benoît Monin, Gregory CLEMENT, Théo Lebrun,
Fu Chao-Ying
Cc: Vladimir Kondratiev, linux-mips, linux-kernel, linux-rt-devel,
devicetree, linux-doc, linux-arm-kernel, linux-perf-users,
linux-hardening, Paul Burton
From: Paul Burton <paul.burton@mips.com>
cm_core_lock is used to protect access to a CPU's core-other/redirect CM
register block. Since each CPU sees its own copy of this block, we need
to ensure that critical sections holding a cm_core_lock run on one CPU -
ie. we need to disable preemption.
Use raw_spinlock_t to implement cm_core_lock in order to provide the
guarantees we need on PREEMPT_RT systems.
Signed-off-by: Paul Burton <paul.burton@mips.com>
Signed-off-by: Vladimir Kondratiev <vladimir.kondratiev@mobileye.com>
---
arch/mips/kernel/mips-cm.c | 20 ++++++++++----------
1 file changed, 10 insertions(+), 10 deletions(-)
diff --git a/arch/mips/kernel/mips-cm.c b/arch/mips/kernel/mips-cm.c
index 7c9c5dc38823..3bfe85e86a9a 100644
--- a/arch/mips/kernel/mips-cm.c
+++ b/arch/mips/kernel/mips-cm.c
@@ -179,7 +179,7 @@ static char *cm3_causes[32] = {
"0x19", "0x1a", "0x1b", "0x1c", "0x1d", "0x1e", "0x1f"
};
-static DEFINE_PER_CPU_ALIGNED(spinlock_t, cm_core_lock);
+static DEFINE_PER_CPU_ALIGNED(raw_spinlock_t, cm_core_lock);
static DEFINE_PER_CPU_ALIGNED(unsigned long, cm_core_lock_flags);
phys_addr_t __weak mips_cm_phys_base(void)
@@ -309,7 +309,7 @@ int mips_cm_probe(void)
mips_cm_is64 = IS_ENABLED(CONFIG_64BIT) && (mips_cm_revision() >= CM_REV_CM3);
for_each_possible_cpu(cpu)
- spin_lock_init(&per_cpu(cm_core_lock, cpu));
+ raw_spin_lock_init(&per_cpu(cm_core_lock, cpu));
return 0;
}
@@ -347,8 +347,8 @@ void mips_cm_lock_other(unsigned int cluster, unsigned int core,
* mips_cm_lock_other() leading to a deadlock or a nice warning
* with lockdep enabled.
*/
- spin_lock_irqsave(this_cpu_ptr(&cm_core_lock),
- *this_cpu_ptr(&cm_core_lock_flags));
+ raw_spin_lock_irqsave(this_cpu_ptr(&cm_core_lock),
+ *this_cpu_ptr(&cm_core_lock_flags));
} else {
WARN_ON(cluster != 0);
WARN_ON(block != CM_GCR_Cx_OTHER_BLOCK_LOCAL);
@@ -359,8 +359,8 @@ void mips_cm_lock_other(unsigned int cluster, unsigned int core,
* race with us.
*/
curr_core = cpu_core(¤t_cpu_data);
- spin_lock_irqsave(&per_cpu(cm_core_lock, curr_core),
- per_cpu(cm_core_lock_flags, curr_core));
+ raw_spin_lock_irqsave(&per_cpu(cm_core_lock, curr_core),
+ per_cpu(cm_core_lock_flags, curr_core));
val = FIELD_PREP(CM_GCR_Cx_OTHER_CORENUM, core);
}
@@ -380,11 +380,11 @@ void mips_cm_unlock_other(void)
if (mips_cm_revision() < CM_REV_CM3) {
curr_core = cpu_core(¤t_cpu_data);
- spin_unlock_irqrestore(&per_cpu(cm_core_lock, curr_core),
- per_cpu(cm_core_lock_flags, curr_core));
+ raw_spin_unlock_irqrestore(&per_cpu(cm_core_lock, curr_core),
+ per_cpu(cm_core_lock_flags, curr_core));
} else {
- spin_unlock_irqrestore(this_cpu_ptr(&cm_core_lock),
- *this_cpu_ptr(&cm_core_lock_flags));
+ raw_spin_unlock_irqrestore(this_cpu_ptr(&cm_core_lock),
+ *this_cpu_ptr(&cm_core_lock_flags));
}
preempt_enable();
--
2.43.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v2 2/6] MIPS: CM: add CM3 performance counter GCR definitions
2026-11-12 12:28 [PATCH v2 0/6] perf: add PMU driver for the MIPS Coherence Manager 3 performance counters Vladimir Kondratiev
2026-11-12 12:28 ` [PATCH v2 1/6] MIPS: CM: Make cm_core_lock raw spinlocks Vladimir Kondratiev
@ 2026-11-12 12:28 ` Vladimir Kondratiev
2026-11-12 12:29 ` [PATCH v2 3/6] MIPS: CM: export the symbols needed to access the GCRs from a module Vladimir Kondratiev
` (3 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Vladimir Kondratiev @ 2026-11-12 12:28 UTC (permalink / raw)
To: Thomas Bogendoerfer, Sebastian Andrzej Siewior, Clark Williams,
Steven Rostedt, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Jiaxun Yang, Jonathan Corbet, Shuah Khan, Randy Dunlap,
Will Deacon, Mark Rutland, Kees Cook, Gustavo A. R. Silva,
Benoît Monin, Gregory CLEMENT, Théo Lebrun,
Fu Chao-Ying
Cc: Vladimir Kondratiev, linux-mips, linux-kernel, linux-rt-devel,
devicetree, linux-doc, linux-arm-kernel, linux-perf-users,
linux-hardening
The CM3 implements two programmable 32-bit performance counters and a
32-bit cycle counter, controlled through registers in the GCR global
debug block (GCR_DB_PC_*, offset 0x6100..0x61a8 from GCR_BASE).
Add GCR_DB_ACCESSOR_{RO,RW} helpers for the debug block and define the
control, overflow status, event select, cycle counter, qualifier and
counter registers along with their fields, as described in the I6500
register reference (MD01189).
Each accessor comes in two flavours, as the CPC ones do: db_<reg> for
the local cluster and redir_db_<reg> for the debug block of another
cluster through the redirect block. GCR_Cx_OTHER.BLOCK value 2 selects
the debug block in the GCR, where the existing _BLOCK_USER name only
describes its GIC meaning; add CM_GCR_Cx_OTHER_BLOCK_DEBUG for it.
Signed-off-by: Vladimir Kondratiev <vladimir.kondratiev@mobileye.com>
---
arch/mips/include/asm/mips-cm.h | 51 ++++++++++++++++++++++++++++++++++++++++-
1 file changed, 50 insertions(+), 1 deletion(-)
diff --git a/arch/mips/include/asm/mips-cm.h b/arch/mips/include/asm/mips-cm.h
index 407f253bb4a1..390d06a8f1d1 100644
--- a/arch/mips/include/asm/mips-cm.h
+++ b/arch/mips/include/asm/mips-cm.h
@@ -161,6 +161,20 @@ static inline bool mips_cm_has_l2sync(void)
CPS_ACCESSOR_RW(gcr, sz, MIPS_CM_CLCB_OFS + off, cl_##name) \
CPS_ACCESSOR_RW(gcr, sz, MIPS_CM_COCB_OFS + off, co_##name)
+/*
+ * Global debug block registers. The redir_db_* accessors reach the debug
+ * block of another cluster through the redirect block, which must have been
+ * pointed at it with mips_cm_lock_other(cluster, 0, 0,
+ * CM_GCR_Cx_OTHER_BLOCK_DEBUG).
+ */
+#define GCR_DB_ACCESSOR_RO(sz, off, name) \
+ CPS_ACCESSOR_RO(gcr, sz, MIPS_CM_GDB_OFS + off, db_##name) \
+ CPS_ACCESSOR_RO(gcr, sz, MIPS_CM_COCB_OFS + off, redir_db_##name)
+
+#define GCR_DB_ACCESSOR_RW(sz, off, name) \
+ CPS_ACCESSOR_RW(gcr, sz, MIPS_CM_GDB_OFS + off, db_##name) \
+ CPS_ACCESSOR_RW(gcr, sz, MIPS_CM_COCB_OFS + off, redir_db_##name)
+
/* GCR_CONFIG - Information about the system */
GCR_ACCESSOR_RO(64, 0x000, config)
#define CM_GCR_CONFIG_CLUSTER_COH_CAPABLE BIT_ULL(43)
@@ -357,7 +371,8 @@ GCR_CX_ACCESSOR_RW(32, 0x018, other)
#define CM_GCR_Cx_OTHER_BLOCK GENMASK(25, 24) /* CM >= 3.5 */
#define CM_GCR_Cx_OTHER_BLOCK_LOCAL 0
#define CM_GCR_Cx_OTHER_BLOCK_GLOBAL 1
-#define CM_GCR_Cx_OTHER_BLOCK_USER 2
+#define CM_GCR_Cx_OTHER_BLOCK_USER 2 /* GIC */
+#define CM_GCR_Cx_OTHER_BLOCK_DEBUG 2 /* GCR */
#define CM_GCR_Cx_OTHER_BLOCK_GLOBAL_HIGH 3
#define CM_GCR_Cx_OTHER_CLUSTER GENMASK(21, 16) /* CM >= 3.5 */
#define CM3_GCR_Cx_OTHER_CORE GENMASK(13, 8) /* CM >= 3 */
@@ -384,6 +399,40 @@ GCR_CX_ACCESSOR_RW(32, 0x030, reset_ext_base)
#define CM_GCR_Cx_RESET_EXT_BASE_BEVEXCPA GENMASK(7, 1)
#define CM_GCR_Cx_RESET_EXT_BASE_PRESENT BIT(0)
+/* GCR_DB_PC_CTL - Control of the CM3 performance counters */
+GCR_DB_ACCESSOR_RW(32, 0x100, pc_ctl)
+#define CM_GCR_DB_PC_CTL_PERF_INT_EN BIT(30)
+#define CM_GCR_DB_PC_CTL_PERF_OVF_STOP BIT(29)
+#define CM_GCR_DB_PC_CTL_P1_RESET BIT(9)
+#define CM_GCR_DB_PC_CTL_P1_COUNTON BIT(8)
+#define CM_GCR_DB_PC_CTL_P0_RESET BIT(7)
+#define CM_GCR_DB_PC_CTL_P0_COUNTON BIT(6)
+#define CM_GCR_DB_PC_CTL_CYCL_CNT_RESET BIT(5)
+#define CM_GCR_DB_PC_CTL_CYCL_CNT_COUNTON BIT(4)
+#define CM_GCR_DB_PC_CTL_PERF_NUM_CNT GENMASK(3, 0)
+
+/* GCR_DB_PC_OV - CM3 performance counter overflow status (write 1 to clear) */
+GCR_DB_ACCESSOR_RW(32, 0x120, pc_ov)
+#define CM_GCR_DB_PC_OV_P1_OF BIT(2)
+#define CM_GCR_DB_PC_OV_P0_OF BIT(1)
+#define CM_GCR_DB_PC_OV_CYCL_CNT_OF BIT(0)
+
+/* GCR_DB_PC_EVENT - CM3 performance counter event selection */
+GCR_DB_ACCESSOR_RW(32, 0x130, pc_event)
+#define CM_GCR_DB_PC_EVENT_P1_EVENT GENMASK(15, 8)
+#define CM_GCR_DB_PC_EVENT_P0_EVENT GENMASK(7, 0)
+
+/* GCR_DB_PC_CYCL - CM3 performance cycle counter */
+GCR_DB_ACCESSOR_RW(32, 0x180, pc_cycl)
+
+/* GCR_DB_PC_QUALn - CM3 performance counter n event qualifier */
+GCR_DB_ACCESSOR_RW(64, 0x190, pc_qual0)
+GCR_DB_ACCESSOR_RW(64, 0x1a0, pc_qual1)
+
+/* GCR_DB_PC_CNTn - CM3 performance counter n value */
+GCR_DB_ACCESSOR_RW(32, 0x198, pc_cnt0)
+GCR_DB_ACCESSOR_RW(32, 0x1a8, pc_cnt1)
+
/**
* mips_cm_l2sync - perform an L2-only sync operation
*
--
2.43.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v2 3/6] MIPS: CM: export the symbols needed to access the GCRs from a module
2026-11-12 12:28 [PATCH v2 0/6] perf: add PMU driver for the MIPS Coherence Manager 3 performance counters Vladimir Kondratiev
2026-11-12 12:28 ` [PATCH v2 1/6] MIPS: CM: Make cm_core_lock raw spinlocks Vladimir Kondratiev
2026-11-12 12:28 ` [PATCH v2 2/6] MIPS: CM: add CM3 performance counter GCR definitions Vladimir Kondratiev
@ 2026-11-12 12:29 ` Vladimir Kondratiev
2026-11-12 12:29 ` [PATCH v2 4/6] dt-bindings: mips: mti,mips-cm: add performance counter interrupt Vladimir Kondratiev
` (2 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Vladimir Kondratiev @ 2026-11-12 12:29 UTC (permalink / raw)
To: Thomas Bogendoerfer, Sebastian Andrzej Siewior, Clark Williams,
Steven Rostedt, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Jiaxun Yang, Jonathan Corbet, Shuah Khan, Randy Dunlap,
Will Deacon, Mark Rutland, Kees Cook, Gustavo A. R. Silva,
Benoît Monin, Gregory CLEMENT, Théo Lebrun,
Fu Chao-Ying
Cc: Vladimir Kondratiev, linux-mips, linux-kernel, linux-rt-devel,
devicetree, linux-doc, linux-arm-kernel, linux-perf-users,
linux-hardening
Allow modules to use the GCR and CPC accessors from asm/mips-cm.h and
asm/mips-cpc.h:
- mips_gcr_base and mips_cpc_base are the mapped register blocks;
- the CPS_ACCESSOR macros consult mips_cm_is64 on every 64-bit register
access;
- mips_cm_lock_other() / mips_cm_unlock_other() are the only way to
reach the registers of another cluster through the redirect block.
This is a prerequisite for building the CM3 PMU driver as a module.
Signed-off-by: Vladimir Kondratiev <vladimir.kondratiev@mobileye.com>
---
arch/mips/kernel/mips-cm.c | 5 +++++
arch/mips/kernel/mips-cpc.c | 2 ++
2 files changed, 7 insertions(+)
diff --git a/arch/mips/kernel/mips-cm.c b/arch/mips/kernel/mips-cm.c
index 3bfe85e86a9a..ed98021418b5 100644
--- a/arch/mips/kernel/mips-cm.c
+++ b/arch/mips/kernel/mips-cm.c
@@ -5,6 +5,7 @@
*/
#include <linux/errno.h>
+#include <linux/export.h>
#include <linux/of.h>
#include <linux/percpu.h>
#include <linux/spinlock.h>
@@ -14,8 +15,10 @@
#include <asm/mipsregs.h>
void __iomem *mips_gcr_base;
+EXPORT_SYMBOL_GPL(mips_gcr_base);
void __iomem *mips_cm_l2sync_base;
int mips_cm_is64;
+EXPORT_SYMBOL_GPL(mips_cm_is64);
bool mips_cm_is_l2_hci_broken;
static char *cm2_tr[8] = {
@@ -373,6 +376,7 @@ void mips_cm_lock_other(unsigned int cluster, unsigned int core,
*/
mb();
}
+EXPORT_SYMBOL_GPL(mips_cm_lock_other);
void mips_cm_unlock_other(void)
{
@@ -389,6 +393,7 @@ void mips_cm_unlock_other(void)
preempt_enable();
}
+EXPORT_SYMBOL_GPL(mips_cm_unlock_other);
void mips_cm_error_report(void)
{
diff --git a/arch/mips/kernel/mips-cpc.c b/arch/mips/kernel/mips-cpc.c
index 3e386f7e1545..4a8bd56bc2e8 100644
--- a/arch/mips/kernel/mips-cpc.c
+++ b/arch/mips/kernel/mips-cpc.c
@@ -6,6 +6,7 @@
#include <linux/bitfield.h>
#include <linux/errno.h>
+#include <linux/export.h>
#include <linux/percpu.h>
#include <linux/of.h>
#include <linux/of_address.h>
@@ -14,6 +15,7 @@
#include <asm/mips-cps.h>
void __iomem *mips_cpc_base;
+EXPORT_SYMBOL_GPL(mips_cpc_base);
static DEFINE_PER_CPU_ALIGNED(spinlock_t, cpc_core_lock);
--
2.43.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v2 4/6] dt-bindings: mips: mti,mips-cm: add performance counter interrupt
2026-11-12 12:28 [PATCH v2 0/6] perf: add PMU driver for the MIPS Coherence Manager 3 performance counters Vladimir Kondratiev
` (2 preceding siblings ...)
2026-11-12 12:29 ` [PATCH v2 3/6] MIPS: CM: export the symbols needed to access the GCRs from a module Vladimir Kondratiev
@ 2026-11-12 12:29 ` Vladimir Kondratiev
2026-11-12 12:29 ` [PATCH v2 5/6] perf: add MIPS Coherence Manager 3 (CM3) PMU driver Vladimir Kondratiev
2026-11-12 12:29 ` [PATCH v2 6/6] MIPS: eyeq6_defconfig: enable the CM3 PMU Vladimir Kondratiev
5 siblings, 0 replies; 8+ messages in thread
From: Vladimir Kondratiev @ 2026-11-12 12:29 UTC (permalink / raw)
To: Thomas Bogendoerfer, Sebastian Andrzej Siewior, Clark Williams,
Steven Rostedt, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Jiaxun Yang, Jonathan Corbet, Shuah Khan, Randy Dunlap,
Will Deacon, Mark Rutland, Kees Cook, Gustavo A. R. Silva,
Benoît Monin, Gregory CLEMENT, Théo Lebrun,
Fu Chao-Ying
Cc: Vladimir Kondratiev, linux-mips, linux-kernel, linux-rt-devel,
devicetree, linux-doc, linux-arm-kernel, linux-perf-users,
linux-hardening
The CM3 asserts so_cm_perf_cnt_int when one of its performance counters
overflows. Whether that output is connected to an interrupt controller
input is up to the SoC integrator, so describe it as an optional
'interrupts' property of the coherency-manager node. When absent, the
counters have to be polled by software.
Signed-off-by: Vladimir Kondratiev <vladimir.kondratiev@mobileye.com>
---
Documentation/devicetree/bindings/mips/mti,mips-cm.yaml | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml b/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml
index d129d6382847..ca7a6717ecdf 100644
--- a/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml
+++ b/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml
@@ -38,6 +38,12 @@ properties:
maxItems: 1
+ interrupts:
+ description:
+ Performance counter overflow interrupt of the CM (so_cm_perf_cnt_int).
+ When absent, the performance counters are polled by software.
+ maxItems: 1
+
required:
- compatible
--
2.43.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v2 5/6] perf: add MIPS Coherence Manager 3 (CM3) PMU driver
2026-11-12 12:28 [PATCH v2 0/6] perf: add PMU driver for the MIPS Coherence Manager 3 performance counters Vladimir Kondratiev
` (3 preceding siblings ...)
2026-11-12 12:29 ` [PATCH v2 4/6] dt-bindings: mips: mti,mips-cm: add performance counter interrupt Vladimir Kondratiev
@ 2026-11-12 12:29 ` Vladimir Kondratiev
2026-11-12 12:29 ` [PATCH v2 6/6] MIPS: eyeq6_defconfig: enable the CM3 PMU Vladimir Kondratiev
5 siblings, 0 replies; 8+ messages in thread
From: Vladimir Kondratiev @ 2026-11-12 12:29 UTC (permalink / raw)
To: Thomas Bogendoerfer, Sebastian Andrzej Siewior, Clark Williams,
Steven Rostedt, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Jiaxun Yang, Jonathan Corbet, Shuah Khan, Randy Dunlap,
Will Deacon, Mark Rutland, Kees Cook, Gustavo A. R. Silva,
Benoît Monin, Gregory CLEMENT, Théo Lebrun,
Fu Chao-Ying
Cc: Vladimir Kondratiev, linux-mips, linux-kernel, linux-rt-devel,
devicetree, linux-doc, linux-arm-kernel, linux-perf-users,
linux-hardening
The CM3 of the MIPS I6500 multiprocessing system serializes the requests
of all cores, IOCUs and the hardware prefetcher before they reach the
shared L2 and the memory interface. It provides two programmable 32-bit
event counters and a 32-bit cycle counter to observe this traffic:
coherence manager requests, I/O and memory interface requests, AXI bus
utilization and PDTrace statistics. Each event is refined by a 64-bit
qualifier split into groups of bits (port, thread, command, CCA, L1/L2
state, ...), where a request is counted when it matches a set bit in
every group.
Expose the counters as uncore PMUs bound to the coherency-manager DT
node, using the GCR block already mapped by mips_cm_probe(). Events are
described by 'event' (config:0-8, 0x100 selecting the cycle counter)
and 'qual' (config1:0-63); a qualifier of 0 picks a per-event default
that counts every occurrence. Named events are provided for the common
combinations and double as templates for custom qualifiers.
Every cluster has its own CM3 and its own counters, so one PMU named
mips_cm3_pmu_<cluster> is registered per cluster found in GCR_CONFIG.
A cluster does not need CPUs, online or at all, for its L2 and memory
interface to be busy, so the counters of other clusters are accessed
through the GCR redirect block from whichever CPU runs the PMU. This
only requires the CM of that cluster to be powered, which is checked
in the CPC (CM core STAT_CONF) when an event is opened: the counters
of a powered down cluster are initialized lazily on first use and
events on it fail with -ENODEV meanwhile. A cluster that Linux booted
powers down with the last of its cores (CPC_PWRUP_CTL.CM_PWRUP is
left clear), so when the last online CPU of a cluster goes offline
its PMU is paused: counts are harvested, running events keep their
state but stop counting, new events are refused; it resumes when a
CPU of the cluster comes back. The counters are shared by the whole
cluster: events are system-wide only, pinned to a single CPU which
follows CPU hotplug; sampling, task events and the exclude_*
attributes are rejected.
Counter overflow is handled through the CM interrupt when the DT node
provides one, and by periodic polling otherwise, giving 64-bit counts
in both cases. Each cluster's CM drives its own interrupt output into
its own GIC, and a GIC shared interrupt is only routed to the cluster
of its affinity CPU, so the interrupt serves the cluster the CPUs live
in, and only when they all live in that one cluster; everything else
is polled.
Document the perf interface, the event numbers and the qualifier layouts
in Documentation/admin-guide/perf/mips_cm3_pmu.rst.
Signed-off-by: Vladimir Kondratiev <vladimir.kondratiev@mobileye.com>
---
Documentation/admin-guide/perf/index.rst | 1 +
Documentation/admin-guide/perf/mips_cm3_pmu.rst | 346 +++++++
drivers/perf/Kconfig | 12 +
drivers/perf/Makefile | 1 +
drivers/perf/mips_cm3_pmu.c | 1237 +++++++++++++++++++++++
5 files changed, 1597 insertions(+)
diff --git a/Documentation/admin-guide/perf/index.rst b/Documentation/admin-guide/perf/index.rst
index aa12708ddb96..97620aa20465 100644
--- a/Documentation/admin-guide/perf/index.rst
+++ b/Documentation/admin-guide/perf/index.rst
@@ -14,6 +14,7 @@ Performance monitor support
qcom_l2_pmu
qcom_l3_pmu
starfive_starlink_pmu
+ mips_cm3_pmu
mrvl-odyssey-ddr-pmu
mrvl-odyssey-tad-pmu
arm-ccn
diff --git a/Documentation/admin-guide/perf/mips_cm3_pmu.rst b/Documentation/admin-guide/perf/mips_cm3_pmu.rst
new file mode 100644
index 000000000000..f952b43f668e
--- /dev/null
+++ b/Documentation/admin-guide/perf/mips_cm3_pmu.rst
@@ -0,0 +1,346 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=================================================================
+MIPS Coherence Manager 3 (CM3) Performance Monitoring Unit (PMU)
+=================================================================
+
+The Coherence Manager 3 (CM3) of the MIPS I6500 multiprocessing system is
+the point where the requests of all cores, I/O coherence units (IOCU) and
+the hardware prefetcher are serialized before reaching the shared L2 cache
+and the memory interface. The CM3 implements a small set of performance
+counters which observe this traffic:
+
+- two programmable 32-bit event counters, and
+- one 32-bit cycle counter clocked at the CM3 clock.
+
+The counters live in the global debug block of the CM Global Configuration
+Registers (``GCR_DB_PC_*``), are shared by all CPUs of the cluster, and count
+regardless of which CPU, privilege level or task generated the traffic. They
+are exposed by the ``mips_cm3_pmu`` driver (``CONFIG_MIPS_CM3_PMU``, built-in
+or as the ``mips_cm3_pmu`` module) as uncore PMUs.
+
+Clusters
+========
+
+A multi-cluster system has one CM3, and so one set of counters, per
+cluster. The number of clusters is read from ``GCR_CONFIG`` and the driver
+registers one PMU per cluster, named after the cluster number::
+
+ /sys/bus/event_source/devices/mips_cm3_pmu_0/
+ /sys/bus/event_source/devices/mips_cm3_pmu_1/
+
+A single-cluster system only has ``mips_cm3_pmu_0``. Each PMU counts the
+traffic of its own cluster: a request from a core in cluster 1 to the memory
+interface of cluster 1 is seen by ``mips_cm3_pmu_1`` only.
+
+A cluster does not need CPUs for its CM to be busy: its L2 cache and memory
+interface may serve I/O devices behind its IOCUs while its cores are absent,
+offline or powered down. The counters of any cluster are therefore reached
+through the GCR redirect block from whichever CPU runs the PMU, which can be
+in any cluster. What is needed is that the CM of the counted cluster is
+powered: the driver checks the CM power sequencer state in the Cluster Power
+Controller (``CPC_STAT_CONF`` of the CM "core") when an event is opened and
+fails with ``-ENODEV`` if the cluster is powered down. The PMU of the
+cluster the CPUs run in is always usable.
+
+A cluster that Linux booted powers down with the last of its cores, as the
+kernel leaves ``CPC_PWRUP_CTL.CM_PWRUP`` clear once the cluster is up. When
+the last online CPU of such a cluster is taken down, its PMU is paused: the
+counts of its events are harvested, the events keep their state but stop
+counting, and opening new events on the PMU fails with ``-ENODEV``. The PMU
+resumes, its running events counting again from where they stood, when a
+CPU of the cluster comes back online. A cluster whose CM is kept powered
+(``CM_PWRUP`` set) is not paused and keeps counting without CPUs.
+
+Because the counters are shared, events are system wide only. Sampling
+(``perf record``), attaching to a task, and the ``exclude_*`` attributes are
+not supported; ``perf stat -a`` is the intended interface. The ``cpumask``
+attribute names the single CPU on which the counters are programmed and
+read; it migrates to another online CPU when that one goes offline::
+
+ $ cat /sys/bus/event_source/devices/mips_cm3_pmu_1/cpumask
+ 0
+
+Counter overflow
+================
+
+The CM3 raises its ``so_cm_perf_cnt_int`` output when a counter reaches
+0xFFFFFFFF. Whether that output reaches an interrupt controller input is a
+decision of the SoC integrator. If the ``coherency-manager`` device tree node
+carries an ``interrupts`` property, the driver uses it to extend the counters
+to 64 bits. Otherwise the driver polls the counters every 250 ms, which is
+well below the time the fastest event needs to wrap a 32-bit counter. Either
+way, the values reported by perf are 64-bit.
+
+The interrupt can only serve the cluster the CPUs live in, and only when
+they all live in that one cluster. Every cluster's CM drives its own
+interrupt output into its own GIC, and a GIC shared interrupt is routed to
+the cluster of its affinity CPU, so a single ``interrupts`` entry cannot
+serve several clusters at once. The PMUs of all other clusters, and all PMUs
+when CPUs are spread over several clusters, are polled.
+
+The mode chosen is printed at probe time, or the fact that the cluster was
+powered down when the driver probed; the counters are then initialized when
+the first event is opened::
+
+ mips_cm3_pmu coherency-manager: mips_cm3_pmu_0: 2 counters + cycles, polled
+ mips_cm3_pmu coherency-manager: mips_cm3_pmu_1: cluster powered down
+
+Event encoding
+==============
+
+An event is described by two fields, documented in the ``format`` directory::
+
+ $ cat /sys/bus/event_source/devices/mips_cm3_pmu_0/format/event
+ config:0-8
+ $ cat /sys/bus/event_source/devices/mips_cm3_pmu_0/format/qual
+ config1:0-63
+
+``event`` selects the event number written to ``GCR_DB_PC_EVENT``. The
+value ``0x100`` is a pseudo event selecting the cycle counter.
+
+``qual`` is the 64-bit qualifier written to ``GCR_DB_PC_QUALn``. It refines
+the event: the qualifier is split into groups of bits, each group covering
+one attribute of a request (originating port, command, cacheability, L2
+state, ...). A request is counted when, in **every** group, the bit
+matching its attribute is set. Setting several bits of a group ORs the
+alternatives; setting all bits of a group makes it a "don't care". A ``qual``
+of 0 (or omitting it) selects a per-event default that counts every
+occurrence of the event.
+
+Event numbers
+-------------
+
+=========== ========================== ======================================
+Event Name Description
+=========== ========================== ======================================
+0x01 CM requests Requests serialized by the coherence
+ manager (qualifier: Table 1)
+0x02 I/O traffic requests Requests received by an IOCU
+ (qualifier: Table 2)
+0x03 Memory interface requests Requests issued on the memory
+ interface (qualifier: Table 3)
+0x07 MEM AXI utilization Cycles on the memory AXI/ACE bus
+ (qualifier: Table 4)
+0x08 - 0x0f IOCU0-7 AXI utilization Cycles on the AXI bus of IOCU
+ (event - 0x08) (qualifier: Table 4)
+0x11 PDTrace dropped messages Messages dropped by CM PDTrace
+ (qualifier: Table 5)
+0x12 PDTrace overflow cycles Cycles spent in CM PDTrace overflow
+0x100 cycles CM3 cycle counter (pseudo event)
+=========== ========================== ======================================
+
+Table 1: Coherence Manager Requests qualifier (event 0x01)
+----------------------------------------------------------
+
+====== =========== ===================================================
+Bits Group Values (one bit each, LSB first)
+====== =========== ===================================================
+9:0 Port Input port 0..7, prefetcher (8), intervention (9).
+ Ports are numbered cores first, then IOCUs.
+13:10 Thread Thread 0..3
+15:14 Target Memory, register bus (GCR, GIC, CPC, ...)
+39:16 Command Legacy read, legacy write, CohReadOwn,
+ CohReadShare, CohReadDiscard, CohEvict,
+ CohUpgrade, CohUpgradeSC, CohWriteBack,
+ CohWriteInvalidate, CohReadDiscardAllocate,
+ prefetch own, prefetch share, prefetch write
+ invalidate, GetToOwn, Tag_Err, RegRead,
+ RegWrite, Sync, L1I cacheop, L1D cacheop,
+ L2 cacheop, L3 cacheop, other
+42:40 CCA Uncached, uncached accelerated, other
+44:43 Size Less than a line, one line
+47:45 L1 state Invalid, shared, exclusive/modified
+51:48 L2 state Invalid, shared, exclusive, modified
+53:52 L2 locked Not locked, locked
+56:54 Eviction None, L2 and L1 eviction, L2 eviction only
+58:57 Bank L2 bank 0, bank 1
+60:59 Scheduler Scheduler 0, scheduler 1
+63:61 Reserved Must be set
+====== =========== ===================================================
+
+Default: all bits set.
+
+Table 2: I/O Traffic Requests qualifier (event 0x02)
+----------------------------------------------------
+
+====== =============== ===============================================
+Bits Group Values
+====== =============== ===============================================
+2:0 IOCU number Encoded 0..7 (a value, not a mask)
+7:3 Direction/CCA Uncached read, coherent read with allocate,
+ coherent read without allocate, uncached
+ write, coherent write
+15:8 Size 1, 2, 4, 8, 16, 32, 64, 128 bytes per beat
+24:16 Length 1, 2, 3-4, 5-8, 9-16, 17-32, 33-64, 65-128,
+ 129-256 beats
+26:25 Alignment Aligned, misaligned
+28:27 Prefetch Not prefetch, prefetch
+29 All device IDs Set to ignore bits 35:30
+35:30 Device ID Encoded device ID
+36 All txn IDs Set to ignore bits 41:37
+41:37 Transaction ID Encoded transaction ID
+====== =============== ===============================================
+
+Default: IOCU 0, every other group "don't care" (``0x3fffffffff8``).
+
+Table 3: Memory Interface Requests qualifier (event 0x03)
+---------------------------------------------------------
+
+====== =============== ===============================================
+Bits Group Values
+====== =============== ===============================================
+1:0 Direction Read, write
+9:2 Size 1, 2, 4, 8, 16, 32, 64, 128 bytes per beat
+17:10 Length 0..7 beats
+21:18 Cacheability Uncached, uncached accelerated, cacheable read
+ discard, cacheable not read discard
+29:22 Source Input port 0..7
+35:32 Thread Thread 0..3
+37:36 Code/data Data, code
+39:38 Prefetch Not prefetch, prefetch
+40 All guest IDs Set to ignore bits 45:41
+45:41 Guest ID Encoded guest ID
+====== =============== ===============================================
+
+Default: all groups "don't care" (``0x3fffffffffff``).
+
+Table 4: AXI Bus Utilization qualifier (events 0x07 - 0x0f)
+-----------------------------------------------------------
+
+These events count cycles, qualified by the state of the handshake signals
+of one AXI channel.
+
+====== =============== ===============================================
+Bits Group Values
+====== =============== ===============================================
+0 Not valid Count cycles where xVALID is deasserted
+1 Valid Count cycles where xVALID is asserted
+2 Not ready Count cycles where xREADY is deasserted
+3 Ready Count cycles where xREADY is asserted
+6:4 Channel Encoded: 0 AR, 1 AW, 2 W, 3 R, 4 B
+====== =============== ===============================================
+
+Setting both ``valid`` and ``ready`` counts completed transfers. Default:
+transfers on the AR channel (``0xa``).
+
+Table 5: CM PDTrace Dropped Messages qualifier (event 0x11)
+-----------------------------------------------------------
+
+====== =============== ===============================================
+Bits Group Values
+====== =============== ===============================================
+5:0 Trace port ubrh, prsh, tmh0, tmh0_mulp, tmh1, tmh1_mulp
+7:6 Trace type Request, response
+====== =============== ===============================================
+
+Default: all bits set.
+
+Named events
+============
+
+The ``events`` directory provides ready-made event/qualifier pairs for the
+most useful combinations::
+
+ $ perf list mips_cm3_pmu_0
+
+ mips_cm3_pmu_0/cm_requests/ [Kernel PMU event]
+ mips_cm3_pmu_0/cm_requests_coh_writeback/ [Kernel PMU event]
+ mips_cm3_pmu_0/cm_requests_l2_hit/ [Kernel PMU event]
+ mips_cm3_pmu_0/cm_requests_l2_miss/ [Kernel PMU event]
+ mips_cm3_pmu_0/cm_requests_legacy_read/ [Kernel PMU event]
+ mips_cm3_pmu_0/cm_requests_legacy_write/ [Kernel PMU event]
+ mips_cm3_pmu_0/cm_requests_mem/ [Kernel PMU event]
+ mips_cm3_pmu_0/cm_requests_regbus/ [Kernel PMU event]
+ mips_cm3_pmu_0/cycles/ [Kernel PMU event]
+ mips_cm3_pmu_0/iocu0_requests/ [Kernel PMU event]
+ mips_cm3_pmu_0/mem_axi_ar/ [Kernel PMU event]
+ mips_cm3_pmu_0/mem_axi_aw/ [Kernel PMU event]
+ mips_cm3_pmu_0/mem_axi_b/ [Kernel PMU event]
+ mips_cm3_pmu_0/mem_axi_r/ [Kernel PMU event]
+ mips_cm3_pmu_0/mem_axi_w/ [Kernel PMU event]
+ mips_cm3_pmu_0/mem_reads/ [Kernel PMU event]
+ mips_cm3_pmu_0/mem_requests/ [Kernel PMU event]
+ mips_cm3_pmu_0/mem_writes/ [Kernel PMU event]
+ mips_cm3_pmu_0/pdtrace_dropped/ [Kernel PMU event]
+ mips_cm3_pmu_0/pdtrace_overflow_cycles/ [Kernel PMU event]
+
+============================== =============================================
+Name Counts
+============================== =============================================
+cycles CM3 clock cycles
+cm_requests Every request serialized by the CM
+cm_requests_mem Requests targeting memory
+cm_requests_regbus Requests targeting the register bus
+cm_requests_legacy_read Non-coherent reads, including code fetches
+cm_requests_legacy_write Non-coherent writes
+cm_requests_coh_writeback Coherent write backs
+cm_requests_l2_hit Memory requests whose line is present in L2
+cm_requests_l2_miss Memory requests whose line is not in L2
+iocu0_requests Requests received by IOCU 0
+mem_requests Requests issued on the memory interface
+mem_reads Memory interface reads
+mem_writes Memory interface writes
+mem_axi_ar/aw/w/r/b Completed transfers on the given channel
+ of the memory AXI bus
+pdtrace_dropped Messages dropped by CM PDTrace
+pdtrace_overflow_cycles Cycles spent in CM PDTrace overflow
+============================== =============================================
+
+The named events are plain ``event=...,qual=...`` strings, so they serve as
+templates: read one from sysfs and clear the bits of the group you want to
+narrow down.
+
+Examples
+========
+
+Count all CM requests and the share coming from port 1 (core 1), together
+with cycles, for one second::
+
+ $ perf stat -a -e mips_cm3_pmu_0/cycles/ \
+ -e mips_cm3_pmu_0/cm_requests/ \
+ -e mips_cm3_pmu_0/event=0x1,qual=0xfffffffffffffc02/ \
+ sleep 1
+
+The second qualifier is the ``cm_requests`` default with the Port group
+(bits 9:0) reduced to bit 1.
+
+L2 hit rate of memory requests::
+
+ $ perf stat -a -e mips_cm3_pmu_0/cm_requests_l2_hit/ \
+ -e mips_cm3_pmu_0/cm_requests_l2_miss/ sleep 1
+
+Utilization of the memory read data channel: cycles with a transfer versus
+cycles where the CM waits for data (valid deasserted)::
+
+ $ perf stat -a -e mips_cm3_pmu_0/cycles/ \
+ -e mips_cm3_pmu_0/mem_axi_r/ \
+ -e mips_cm3_pmu_0/event=0x7,qual=0x3d/ sleep 1
+
+Requests received by IOCU 1 (IOCU number is an encoded value in bits 2:0)::
+
+ $ perf stat -a -e mips_cm3_pmu_0/event=0x2,qual=0x3fffffffff9/ sleep 1
+
+Memory interface traffic of every cluster of a two-cluster system::
+
+ $ perf stat -a -e mips_cm3_pmu_0/mem_requests/ \
+ -e mips_cm3_pmu_1/mem_requests/ sleep 1
+
+Only two programmable counters exist; a group requesting more than two
+non-cycle events, or more than one ``cycles`` event, is rejected with
+``-EINVAL``.
+
+Device tree
+===========
+
+The driver binds to the ``coherency-manager`` node (``mti,mips-cm`` or
+``mobileye,eyeq6-cm``) and uses the GCR base address already discovered by
+the kernel from the ``CMGCRBase`` register. An ``interrupts`` property on
+that node, when the SoC routes ``so_cm_perf_cnt_int`` to an interrupt
+controller, enables interrupt-driven overflow handling::
+
+ coherency-manager {
+ compatible = "mobileye,eyeq6-cm";
+ interrupt-parent = <&gic>;
+ interrupts = <GIC_SHARED 42 IRQ_TYPE_LEVEL_HIGH>;
+ };
diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig
index 245e7bb763b9..2dbed2fc7b03 100644
--- a/drivers/perf/Kconfig
+++ b/drivers/perf/Kconfig
@@ -75,6 +75,18 @@ config ARM_XSCALE_PMU
depends on ARM_PMU && CPU_XSCALE
def_bool y
+config MIPS_CM3_PMU
+ tristate "MIPS Coherence Manager 3 (CM3) PMU"
+ depends on MIPS_CM
+ help
+ Support for the performance counters of the MIPS Coherence Manager 3
+ (CM3) found in the I6500 multiprocessing system. The CM3 provides two
+ programmable event counters and a cycle counter which can count
+ requests serialized by the coherence manager, I/O and memory interface
+ traffic, and AXI bus utilization, each refined by a qualifier.
+
+ If compiled as a module, it will be called mips_cm3_pmu.
+
config RISCV_PMU
depends on RISCV
bool "RISC-V PMU framework"
diff --git a/drivers/perf/Makefile b/drivers/perf/Makefile
index eb8a022dad9a..5f0920ba7058 100644
--- a/drivers/perf/Makefile
+++ b/drivers/perf/Makefile
@@ -17,6 +17,7 @@ obj-$(CONFIG_FUJITSU_UNCORE_PMU) += fujitsu_uncore_pmu.o
obj-$(CONFIG_HISI_PMU) += hisilicon/
obj-$(CONFIG_QCOM_L2_PMU) += qcom_l2_pmu.o
obj-$(CONFIG_QCOM_L3_PMU) += qcom_l3_pmu.o
+obj-$(CONFIG_MIPS_CM3_PMU) += mips_cm3_pmu.o
obj-$(CONFIG_RISCV_PMU) += riscv_pmu.o
obj-$(CONFIG_RISCV_PMU_LEGACY) += riscv_pmu_legacy.o
obj-$(CONFIG_RISCV_PMU_SBI) += riscv_pmu_sbi.o
diff --git a/drivers/perf/mips_cm3_pmu.c b/drivers/perf/mips_cm3_pmu.c
new file mode 100644
index 000000000000..6d53535473b9
--- /dev/null
+++ b/drivers/perf/mips_cm3_pmu.c
@@ -0,0 +1,1237 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * MIPS Coherence Manager 3 (CM3) performance counter PMU driver
+ *
+ * The CM3 found in the MIPS I6500 multiprocessing system implements two
+ * programmable 32-bit event counters and a 32-bit cycle counter. They are
+ * controlled through the GCR global debug block (GCR_DB_PC_*) and count
+ * events happening at the coherence manager: serialized requests, I/O and
+ * memory traffic, AXI bus utilization, ...
+ *
+ * perf interface:
+ *
+ * config [8:0] event number (GCR_DB_PC_EVENT.Px_Event), 0x100 selects
+ * the cycle counter.
+ * config1 [63:0] event qualifier (GCR_DB_PC_QUALn). A qualifier is split
+ * into groups of bits, each selecting one attribute of the
+ * request; an event is counted when it matches at least one
+ * set bit of every group. Setting all bits of a group makes
+ * it a "don't care". A qualifier of 0 selects a sensible
+ * default which counts every occurrence of the event.
+ *
+ * Every cluster has its own CM3 and therefore its own set of counters, and
+ * one PMU instance (mips_cm3_pmu_<cluster>) is registered per cluster. A
+ * cluster does not need CPUs, online or at all, for its L2 and memory
+ * interface to be busy, so the counters of other clusters are reached
+ * through the GCR redirect block from whichever CPU runs the PMU. This
+ * requires the CM of that cluster to be powered, which is checked in the
+ * CPC before an event is opened. A cluster holding CPUs powers down with
+ * the last of them; its PMU is then paused until one comes back.
+ *
+ * The counters are shared by the whole cluster, so events are system wide,
+ * pinned to a single CPU, and cannot be sampled.
+ */
+
+#define CM3_PMU_NAME "mips_cm3_pmu"
+
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/perf_event.h>
+#include <linux/platform_device.h>
+
+#include <asm/cpu-info.h>
+#include <asm/mips-cps.h>
+
+/* Two programmable counters followed by the cycle counter */
+#define CM3_PMU_MAX_PCS 2
+#define CM3_PMU_IDX_CYCLES CM3_PMU_MAX_PCS
+#define CM3_PMU_NUM_COUNTERS (CM3_PMU_MAX_PCS + 1)
+#define CM3_PMU_COUNTER_MASK GENMASK_ULL(31, 0)
+
+/*
+ * Without an overflow interrupt the 32-bit counters are read periodically.
+ * The fastest events count a couple of occurrences per CM clock cycle, so
+ * even at 2GHz a counter takes more than a second to wrap.
+ */
+#define CM3_PMU_POLL_PERIOD_NS (250 * NSEC_PER_MSEC)
+
+/* Event numbers (GCR_DB_PC_EVENT.Px_Event) */
+#define CM3_PMU_EV_NONE 0x00
+#define CM3_PMU_EV_CM_REQ 0x01 /* Coherence Manager requests */
+#define CM3_PMU_EV_IO_REQ 0x02 /* I/O traffic requests (IOCU) */
+#define CM3_PMU_EV_MEM_REQ 0x03 /* Memory interface requests */
+#define CM3_PMU_EV_MEM_AXI 0x07 /* MEM AXI bus utilization */
+#define CM3_PMU_EV_IOCU_AXI(n) (0x08 + (n)) /* IOCUn AXI bus utilization */
+#define CM3_PMU_EV_IOCU_AXI_LAST CM3_PMU_EV_IOCU_AXI(7)
+#define CM3_PMU_EV_PDTRACE_DROPPED 0x11 /* CM PDTrace dropped messages */
+#define CM3_PMU_EV_PDTRACE_OVF_CYCLES 0x12 /* CM PDTrace overflow cycles */
+#define CM3_PMU_EV_HW_MASK GENMASK(7, 0)
+/* Pseudo event selecting the cycle counter */
+#define CM3_PMU_EV_CYCLES 0x100
+#define CM3_PMU_EV_MASK GENMASK(8, 0)
+
+/*
+ * Qualifier of the Coherence Manager Requests event. Ports are numbered
+ * with the cores first, followed by the IOCUs; port bit 8 is the prefetcher
+ * and bit 9 interventions.
+ */
+#define CM3_PMU_Q_CMREQ_PORT GENMASK_ULL(9, 0)
+#define CM3_PMU_Q_CMREQ_THREAD GENMASK_ULL(13, 10)
+#define CM3_PMU_Q_CMREQ_TARGET GENMASK_ULL(15, 14)
+#define CM3_PMU_Q_CMREQ_TARGET_MEM BIT_ULL(14)
+#define CM3_PMU_Q_CMREQ_TARGET_REGBUS BIT_ULL(15)
+#define CM3_PMU_Q_CMREQ_CMD GENMASK_ULL(39, 16)
+#define CM3_PMU_Q_CMREQ_CMD_LEGACY_RD BIT_ULL(16)
+#define CM3_PMU_Q_CMREQ_CMD_LEGACY_WR BIT_ULL(17)
+#define CM3_PMU_Q_CMREQ_CMD_COH_WB BIT_ULL(24)
+#define CM3_PMU_Q_CMREQ_CCA GENMASK_ULL(42, 40) /* UC, UCA, other */
+#define CM3_PMU_Q_CMREQ_SIZE GENMASK_ULL(44, 43) /* sub-line, line */
+#define CM3_PMU_Q_CMREQ_L1_STATE GENMASK_ULL(47, 45) /* I, S, E/M */
+#define CM3_PMU_Q_CMREQ_L2_STATE GENMASK_ULL(51, 48) /* I, S, E, M */
+#define CM3_PMU_Q_CMREQ_L2_INVALID BIT_ULL(48)
+#define CM3_PMU_Q_CMREQ_L2_LOCKED GENMASK_ULL(53, 52)
+#define CM3_PMU_Q_CMREQ_EVICTION GENMASK_ULL(56, 54)
+#define CM3_PMU_Q_CMREQ_BANK GENMASK_ULL(58, 57)
+#define CM3_PMU_Q_CMREQ_SCHEDULER GENMASK_ULL(60, 59)
+#define CM3_PMU_Q_CMREQ_RESERVED GENMASK_ULL(63, 61) /* must be set */
+#define CM3_PMU_Q_CMREQ_DEFAULT GENMASK_ULL(63, 0)
+
+/*
+ * Qualifier of the I/O Traffic Requests event. The IOCU number is an
+ * encoded value, not a mask. Direction bits: UC read, coherent read with
+ * allocate, coherent read without allocate, UC write, coherent write.
+ */
+#define CM3_PMU_Q_IO_IOCU GENMASK_ULL(2, 0)
+#define CM3_PMU_Q_IO_DIR GENMASK_ULL(7, 3)
+#define CM3_PMU_Q_IO_SIZE GENMASK_ULL(15, 8) /* 1 .. 128 bytes */
+#define CM3_PMU_Q_IO_LENGTH GENMASK_ULL(24, 16) /* 1 .. 129-256 beats */
+#define CM3_PMU_Q_IO_ALIGNED GENMASK_ULL(26, 25)
+#define CM3_PMU_Q_IO_PREFETCH GENMASK_ULL(28, 27)
+#define CM3_PMU_Q_IO_ALL_DEVID BIT_ULL(29)
+#define CM3_PMU_Q_IO_DEVID GENMASK_ULL(35, 30)
+#define CM3_PMU_Q_IO_ALL_TXNID BIT_ULL(36)
+#define CM3_PMU_Q_IO_TXNID GENMASK_ULL(41, 37)
+#define CM3_PMU_Q_IO_DEFAULT GENMASK_ULL(41, 3) /* IOCU 0 */
+
+/* Qualifier of the Memory Interface Requests event */
+#define CM3_PMU_Q_MEM_DIR GENMASK_ULL(1, 0)
+#define CM3_PMU_Q_MEM_DIR_READ BIT_ULL(0)
+#define CM3_PMU_Q_MEM_DIR_WRITE BIT_ULL(1)
+#define CM3_PMU_Q_MEM_SIZE GENMASK_ULL(9, 2) /* 1 .. 128 bytes */
+#define CM3_PMU_Q_MEM_LENGTH GENMASK_ULL(17, 10) /* 0 .. 7 */
+#define CM3_PMU_Q_MEM_CCA GENMASK_ULL(21, 18) /* UC, UCA, read discard, cacheable */
+#define CM3_PMU_Q_MEM_SOURCE GENMASK_ULL(29, 22) /* input port */
+#define CM3_PMU_Q_MEM_THREAD GENMASK_ULL(35, 32)
+#define CM3_PMU_Q_MEM_CODE_DATA GENMASK_ULL(37, 36)
+#define CM3_PMU_Q_MEM_PREFETCH GENMASK_ULL(39, 38)
+#define CM3_PMU_Q_MEM_ALL_GUESTID BIT_ULL(40)
+#define CM3_PMU_Q_MEM_GUESTID GENMASK_ULL(45, 41)
+#define CM3_PMU_Q_MEM_DEFAULT GENMASK_ULL(45, 0)
+
+/* Qualifier of the AXI Bus Utilization events, counts cycles */
+#define CM3_PMU_Q_AXI_NOT_VALID BIT_ULL(0)
+#define CM3_PMU_Q_AXI_VALID BIT_ULL(1)
+#define CM3_PMU_Q_AXI_NOT_READY BIT_ULL(2)
+#define CM3_PMU_Q_AXI_READY BIT_ULL(3)
+#define CM3_PMU_Q_AXI_CHANNEL GENMASK_ULL(6, 4)
+#define CM3_PMU_Q_AXI_CHANNEL_AR 0
+#define CM3_PMU_Q_AXI_CHANNEL_AW 1
+#define CM3_PMU_Q_AXI_CHANNEL_W 2
+#define CM3_PMU_Q_AXI_CHANNEL_R 3
+#define CM3_PMU_Q_AXI_CHANNEL_B 4
+/* Completed transfers (VALID && READY) on a channel */
+#define CM3_PMU_Q_AXI_XFER(ch) (FIELD_PREP_CONST(CM3_PMU_Q_AXI_CHANNEL, ch) | \
+ CM3_PMU_Q_AXI_VALID | CM3_PMU_Q_AXI_READY)
+#define CM3_PMU_Q_AXI_DEFAULT CM3_PMU_Q_AXI_XFER(CM3_PMU_Q_AXI_CHANNEL_AR)
+
+/* Qualifier of the CM PDTrace Dropped Messages event */
+#define CM3_PMU_Q_PDTRACE_PORT GENMASK_ULL(5, 0)
+#define CM3_PMU_Q_PDTRACE_TYPE GENMASK_ULL(7, 6) /* request, response */
+#define CM3_PMU_Q_PDTRACE_DEFAULT GENMASK_ULL(7, 0)
+
+/* Restrict a single qualifier group of a default qualifier to @bits */
+#define CM3_PMU_QUAL(def, group, bits) (((def) & ~(group)) | (bits))
+
+struct cm3_pmu {
+ struct pmu pmu;
+ struct hlist_node node;
+ struct hrtimer hrtimer;
+ struct perf_event *events[CM3_PMU_NUM_COUNTERS];
+ struct mutex init_lock; /* serializes cm3_pmu_hw_init() */
+ bool initialized;
+ bool paused; /* cluster CM powered down */
+ unsigned int cluster;
+ unsigned int num_pcs;
+ unsigned int num_active;
+ int on_cpu;
+ int irq;
+};
+
+struct cm3_pmu_dev {
+ unsigned int num_pmus;
+ struct cm3_pmu pmus[] __counted_by(num_pmus);
+};
+
+#define to_cm3_pmu(p) container_of(p, struct cm3_pmu, pmu)
+
+static int cm3_pmu_cpuhp_state;
+
+/* sysfs attributes */
+
+static ssize_t cm3_pmu_format_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct dev_ext_attribute *eattr;
+
+ eattr = container_of(attr, struct dev_ext_attribute, attr);
+ return sysfs_emit(buf, "%s\n", (char *)eattr->var);
+}
+
+#define CM3_PMU_FORMAT_ATTR(_name, _config) \
+ (&((struct dev_ext_attribute[]) { \
+ { .attr = __ATTR(_name, 0444, cm3_pmu_format_show, NULL), \
+ .var = (void *)_config, } \
+ })[0].attr.attr)
+
+static struct attribute *cm3_pmu_format_attrs[] = {
+ CM3_PMU_FORMAT_ATTR(event, "config:0-8"),
+ CM3_PMU_FORMAT_ATTR(qual, "config1:0-63"),
+ NULL
+};
+
+static const struct attribute_group cm3_pmu_format_attr_group = {
+ .name = "format",
+ .attrs = cm3_pmu_format_attrs,
+};
+
+struct cm3_pmu_event_attr {
+ struct device_attribute attr;
+ u16 event;
+ u64 qual;
+};
+
+static ssize_t cm3_pmu_event_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct cm3_pmu_event_attr *eattr;
+
+ eattr = container_of(attr, struct cm3_pmu_event_attr, attr);
+ if (!eattr->qual)
+ return sysfs_emit(buf, "event=0x%x\n", eattr->event);
+
+ return sysfs_emit(buf, "event=0x%x,qual=0x%llx\n",
+ eattr->event, eattr->qual);
+}
+
+#define CM3_PMU_EVENT_ATTR(_name, _event, _qual) \
+ (&((struct cm3_pmu_event_attr[]) { \
+ { .attr = __ATTR(_name, 0444, cm3_pmu_event_show, NULL), \
+ .event = _event, .qual = _qual, } \
+ })[0].attr.attr)
+
+static struct attribute *cm3_pmu_event_attrs[] = {
+ CM3_PMU_EVENT_ATTR(cycles, CM3_PMU_EV_CYCLES, 0),
+
+ /* Requests serialized by the coherence manager */
+ CM3_PMU_EVENT_ATTR(cm_requests, CM3_PMU_EV_CM_REQ,
+ CM3_PMU_Q_CMREQ_DEFAULT),
+ CM3_PMU_EVENT_ATTR(cm_requests_mem, CM3_PMU_EV_CM_REQ,
+ CM3_PMU_QUAL(CM3_PMU_Q_CMREQ_DEFAULT,
+ CM3_PMU_Q_CMREQ_TARGET,
+ CM3_PMU_Q_CMREQ_TARGET_MEM)),
+ CM3_PMU_EVENT_ATTR(cm_requests_regbus, CM3_PMU_EV_CM_REQ,
+ CM3_PMU_QUAL(CM3_PMU_Q_CMREQ_DEFAULT,
+ CM3_PMU_Q_CMREQ_TARGET,
+ CM3_PMU_Q_CMREQ_TARGET_REGBUS)),
+ CM3_PMU_EVENT_ATTR(cm_requests_legacy_read, CM3_PMU_EV_CM_REQ,
+ CM3_PMU_QUAL(CM3_PMU_Q_CMREQ_DEFAULT,
+ CM3_PMU_Q_CMREQ_CMD,
+ CM3_PMU_Q_CMREQ_CMD_LEGACY_RD)),
+ CM3_PMU_EVENT_ATTR(cm_requests_legacy_write, CM3_PMU_EV_CM_REQ,
+ CM3_PMU_QUAL(CM3_PMU_Q_CMREQ_DEFAULT,
+ CM3_PMU_Q_CMREQ_CMD,
+ CM3_PMU_Q_CMREQ_CMD_LEGACY_WR)),
+ CM3_PMU_EVENT_ATTR(cm_requests_coh_writeback, CM3_PMU_EV_CM_REQ,
+ CM3_PMU_QUAL(CM3_PMU_Q_CMREQ_DEFAULT,
+ CM3_PMU_Q_CMREQ_CMD,
+ CM3_PMU_Q_CMREQ_CMD_COH_WB)),
+ /* Memory requests whose line is (not) present in the L2 */
+ CM3_PMU_EVENT_ATTR(cm_requests_l2_hit, CM3_PMU_EV_CM_REQ,
+ CM3_PMU_QUAL(CM3_PMU_QUAL(CM3_PMU_Q_CMREQ_DEFAULT,
+ CM3_PMU_Q_CMREQ_TARGET,
+ CM3_PMU_Q_CMREQ_TARGET_MEM),
+ CM3_PMU_Q_CMREQ_L2_STATE,
+ CM3_PMU_Q_CMREQ_L2_STATE &
+ ~CM3_PMU_Q_CMREQ_L2_INVALID)),
+ CM3_PMU_EVENT_ATTR(cm_requests_l2_miss, CM3_PMU_EV_CM_REQ,
+ CM3_PMU_QUAL(CM3_PMU_QUAL(CM3_PMU_Q_CMREQ_DEFAULT,
+ CM3_PMU_Q_CMREQ_TARGET,
+ CM3_PMU_Q_CMREQ_TARGET_MEM),
+ CM3_PMU_Q_CMREQ_L2_STATE,
+ CM3_PMU_Q_CMREQ_L2_INVALID)),
+
+ /* Requests received by IOCU 0 */
+ CM3_PMU_EVENT_ATTR(iocu0_requests, CM3_PMU_EV_IO_REQ,
+ CM3_PMU_Q_IO_DEFAULT),
+
+ /* Requests issued on the memory interface */
+ CM3_PMU_EVENT_ATTR(mem_requests, CM3_PMU_EV_MEM_REQ,
+ CM3_PMU_Q_MEM_DEFAULT),
+ CM3_PMU_EVENT_ATTR(mem_reads, CM3_PMU_EV_MEM_REQ,
+ CM3_PMU_QUAL(CM3_PMU_Q_MEM_DEFAULT,
+ CM3_PMU_Q_MEM_DIR,
+ CM3_PMU_Q_MEM_DIR_READ)),
+ CM3_PMU_EVENT_ATTR(mem_writes, CM3_PMU_EV_MEM_REQ,
+ CM3_PMU_QUAL(CM3_PMU_Q_MEM_DEFAULT,
+ CM3_PMU_Q_MEM_DIR,
+ CM3_PMU_Q_MEM_DIR_WRITE)),
+
+ /* Transfers on each channel of the memory AXI bus */
+ CM3_PMU_EVENT_ATTR(mem_axi_ar, CM3_PMU_EV_MEM_AXI,
+ CM3_PMU_Q_AXI_XFER(CM3_PMU_Q_AXI_CHANNEL_AR)),
+ CM3_PMU_EVENT_ATTR(mem_axi_aw, CM3_PMU_EV_MEM_AXI,
+ CM3_PMU_Q_AXI_XFER(CM3_PMU_Q_AXI_CHANNEL_AW)),
+ CM3_PMU_EVENT_ATTR(mem_axi_w, CM3_PMU_EV_MEM_AXI,
+ CM3_PMU_Q_AXI_XFER(CM3_PMU_Q_AXI_CHANNEL_W)),
+ CM3_PMU_EVENT_ATTR(mem_axi_r, CM3_PMU_EV_MEM_AXI,
+ CM3_PMU_Q_AXI_XFER(CM3_PMU_Q_AXI_CHANNEL_R)),
+ CM3_PMU_EVENT_ATTR(mem_axi_b, CM3_PMU_EV_MEM_AXI,
+ CM3_PMU_Q_AXI_XFER(CM3_PMU_Q_AXI_CHANNEL_B)),
+
+ CM3_PMU_EVENT_ATTR(pdtrace_dropped, CM3_PMU_EV_PDTRACE_DROPPED,
+ CM3_PMU_Q_PDTRACE_DEFAULT),
+ CM3_PMU_EVENT_ATTR(pdtrace_overflow_cycles,
+ CM3_PMU_EV_PDTRACE_OVF_CYCLES, 0),
+ NULL
+};
+
+static const struct attribute_group cm3_pmu_events_attr_group = {
+ .name = "events",
+ .attrs = cm3_pmu_event_attrs,
+};
+
+static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct cm3_pmu *cm3_pmu = to_cm3_pmu(dev_get_drvdata(dev));
+
+ return sysfs_emit(buf, "%*pbl\n",
+ cpumask_pr_args(cpumask_of(cm3_pmu->on_cpu)));
+}
+static DEVICE_ATTR_RO(cpumask);
+
+static struct attribute *cm3_pmu_cpumask_attrs[] = {
+ &dev_attr_cpumask.attr,
+ NULL
+};
+
+static const struct attribute_group cm3_pmu_cpumask_attr_group = {
+ .attrs = cm3_pmu_cpumask_attrs,
+};
+
+static const struct attribute_group *cm3_pmu_attr_groups[] = {
+ &cm3_pmu_format_attr_group,
+ &cm3_pmu_events_attr_group,
+ &cm3_pmu_cpumask_attr_group,
+ NULL
+};
+
+/*
+ * Hardware access
+ *
+ * The debug block of the local cluster is accessed directly. That of another
+ * cluster is accessed through the redirect block, pointed at the debug block
+ * of the target cluster, from whichever CPU this runs on. Both paths are
+ * valid in any context; the redirect path takes the per-CPU redirect lock
+ * with interrupts disabled.
+ */
+
+static bool cm3_pmu_lock(struct cm3_pmu *cm3_pmu)
+{
+ preempt_disable();
+ if (cm3_pmu->cluster == cpu_cluster(¤t_cpu_data))
+ return false;
+
+ mips_cm_lock_other(cm3_pmu->cluster, 0, 0, CM_GCR_Cx_OTHER_BLOCK_DEBUG);
+ return true;
+}
+
+static void cm3_pmu_unlock(bool other)
+{
+ if (other)
+ mips_cm_unlock_other();
+ preempt_enable();
+}
+
+#define CM3_PMU_READ(cm3_pmu, reg) \
+({ \
+ bool __other = cm3_pmu_lock(cm3_pmu); \
+ typeof(read_gcr_db_##reg()) __val; \
+ \
+ __val = __other ? read_gcr_redir_db_##reg() : read_gcr_db_##reg(); \
+ cm3_pmu_unlock(__other); \
+ __val; \
+})
+
+#define CM3_PMU_WRITE(cm3_pmu, reg, val) \
+do { \
+ typeof(read_gcr_db_##reg()) __val = (val); \
+ bool __other = cm3_pmu_lock(cm3_pmu); \
+ \
+ if (__other) \
+ write_gcr_redir_db_##reg(__val); \
+ else \
+ write_gcr_db_##reg(__val); \
+ cm3_pmu_unlock(__other); \
+} while (0)
+
+/*
+ * Whether the CM of the cluster is powered. The CPC is always powered and
+ * reports the CM power sequencer state in the STAT_CONF of its "CM core".
+ * Accessing the GCRs of a powered down cluster would fault.
+ */
+static bool cm3_pmu_cluster_powered(struct cm3_pmu *cm3_pmu)
+{
+ u32 stat;
+
+ /* No other cluster before CM 3.5, and ours is obviously powered */
+ if (cm3_pmu->cluster == cpu_cluster(&raw_current_cpu_data))
+ return true;
+
+ if (!mips_cpc_present())
+ return false;
+
+ mips_cm_lock_other(cm3_pmu->cluster, CM_GCR_Cx_OTHER_CORE_CM, 0,
+ CM_GCR_Cx_OTHER_BLOCK_LOCAL);
+ stat = read_cpc_co_stat_conf();
+ mips_cm_unlock_other();
+
+ return FIELD_GET(CPC_Cx_STAT_CONF_SEQSTATE, stat) ==
+ CPC_Cx_STAT_CONF_SEQSTATE_U5;
+}
+
+static u32 cm3_pmu_counton_bit(int idx)
+{
+ switch (idx) {
+ case 0:
+ return CM_GCR_DB_PC_CTL_P0_COUNTON;
+ case 1:
+ return CM_GCR_DB_PC_CTL_P1_COUNTON;
+ default:
+ return CM_GCR_DB_PC_CTL_CYCL_CNT_COUNTON;
+ }
+}
+
+static u32 cm3_pmu_reset_bit(int idx)
+{
+ switch (idx) {
+ case 0:
+ return CM_GCR_DB_PC_CTL_P0_RESET;
+ case 1:
+ return CM_GCR_DB_PC_CTL_P1_RESET;
+ default:
+ return CM_GCR_DB_PC_CTL_CYCL_CNT_RESET;
+ }
+}
+
+static u32 cm3_pmu_overflow_bit(int idx)
+{
+ switch (idx) {
+ case 0:
+ return CM_GCR_DB_PC_OV_P0_OF;
+ case 1:
+ return CM_GCR_DB_PC_OV_P1_OF;
+ default:
+ return CM_GCR_DB_PC_OV_CYCL_CNT_OF;
+ }
+}
+
+static u32 cm3_pmu_read_counter(struct cm3_pmu *cm3_pmu, int idx)
+{
+ switch (idx) {
+ case 0:
+ return CM3_PMU_READ(cm3_pmu, pc_cnt0);
+ case 1:
+ return CM3_PMU_READ(cm3_pmu, pc_cnt1);
+ default:
+ return CM3_PMU_READ(cm3_pmu, pc_cycl);
+ }
+}
+
+static void cm3_pmu_program_counter(struct cm3_pmu *cm3_pmu, int idx,
+ u32 event, u64 qual)
+{
+ u32 val = CM3_PMU_READ(cm3_pmu, pc_event);
+
+ if (idx == 0) {
+ val &= ~CM_GCR_DB_PC_EVENT_P0_EVENT;
+ val |= FIELD_PREP(CM_GCR_DB_PC_EVENT_P0_EVENT, event);
+ CM3_PMU_WRITE(cm3_pmu, pc_event, val);
+ CM3_PMU_WRITE(cm3_pmu, pc_qual0, qual);
+ } else {
+ val &= ~CM_GCR_DB_PC_EVENT_P1_EVENT;
+ val |= FIELD_PREP(CM_GCR_DB_PC_EVENT_P1_EVENT, event);
+ CM3_PMU_WRITE(cm3_pmu, pc_event, val);
+ CM3_PMU_WRITE(cm3_pmu, pc_qual1, qual);
+ }
+}
+
+static void cm3_pmu_set_ctl(struct cm3_pmu *cm3_pmu, u32 clear, u32 set)
+{
+ u32 val = CM3_PMU_READ(cm3_pmu, pc_ctl);
+
+ CM3_PMU_WRITE(cm3_pmu, pc_ctl, (val & ~clear) | set);
+}
+
+/* Stop everything, clear pending overflows, enable the interrupt if used */
+static void cm3_pmu_hw_reset(struct cm3_pmu *cm3_pmu)
+{
+ CM3_PMU_WRITE(cm3_pmu, pc_ctl,
+ cm3_pmu->irq > 0 ? CM_GCR_DB_PC_CTL_PERF_INT_EN : 0);
+ CM3_PMU_WRITE(cm3_pmu, pc_event, 0);
+ CM3_PMU_WRITE(cm3_pmu, pc_ov, CM_GCR_DB_PC_OV_P1_OF |
+ CM_GCR_DB_PC_OV_P0_OF |
+ CM_GCR_DB_PC_OV_CYCL_CNT_OF);
+}
+
+/*
+ * Run @fn on the CPU the PMU is on, with interrupts disabled. This orders
+ * it with the perf callbacks and with the pause of the PMU, which precedes
+ * the power down of a cluster holding CPUs: a cluster seen unpaused there
+ * stays powered until @fn returns. That CPU may be on its way offline, in
+ * which case either @fn still runs on it, before the hotplug sequence can
+ * go on, or the call fails and is retried on the PMU's new CPU.
+ */
+static int cm3_pmu_call_on_cpu(struct cm3_pmu *cm3_pmu, smp_call_func_t fn,
+ void *info)
+{
+ int cpu, ret;
+
+ do {
+ cpu = READ_ONCE(cm3_pmu->on_cpu);
+ if (cpu < 0)
+ return -ENODEV;
+
+ ret = smp_call_function_single(cpu, fn, info, 1);
+ } while (ret == -ENXIO);
+
+ return ret;
+}
+
+struct cm3_pmu_hw_init_args {
+ struct cm3_pmu *cm3_pmu;
+ int ret;
+};
+
+static void cm3_pmu_hw_init_on_cpu(void *data)
+{
+ struct cm3_pmu_hw_init_args *args = data;
+ struct cm3_pmu *cm3_pmu = args->cm3_pmu;
+
+ if (cm3_pmu->paused) {
+ args->ret = -ENODEV;
+ return;
+ }
+
+ cm3_pmu->num_pcs = FIELD_GET(CM_GCR_DB_PC_CTL_PERF_NUM_CNT,
+ CM3_PMU_READ(cm3_pmu, pc_ctl));
+ cm3_pmu->num_pcs = min_t(unsigned int, cm3_pmu->num_pcs,
+ CM3_PMU_MAX_PCS);
+ cm3_pmu_hw_reset(cm3_pmu);
+ cm3_pmu->initialized = true;
+ args->ret = 0;
+}
+
+/*
+ * Check that the cluster is powered, and bring its counters to a known
+ * state the first time they are needed: the CM may not have been powered
+ * at probe time. A later power cycle puts the registers back to their
+ * reset values, with the counters stopped, so there is nothing to redo.
+ *
+ * The CPC check is all there is for a cluster without CPUs, whose power is
+ * not in the hands of this kernel. For a cluster with CPUs the pause flag
+ * is authoritative, and is consulted where it cannot change under us.
+ */
+static int cm3_pmu_hw_init(struct cm3_pmu *cm3_pmu)
+{
+ struct cm3_pmu_hw_init_args args = { .cm3_pmu = cm3_pmu };
+ int ret;
+
+ guard(mutex)(&cm3_pmu->init_lock);
+
+ if (!cm3_pmu_cluster_powered(cm3_pmu))
+ return -ENODEV;
+
+ if (cm3_pmu->initialized)
+ return 0;
+
+ ret = cm3_pmu_call_on_cpu(cm3_pmu, cm3_pmu_hw_init_on_cpu, &args);
+
+ return ret ?: args.ret;
+}
+
+static u64 cm3_pmu_default_qual(u32 event)
+{
+ switch (event) {
+ case CM3_PMU_EV_CM_REQ:
+ return CM3_PMU_Q_CMREQ_DEFAULT;
+ case CM3_PMU_EV_IO_REQ:
+ return CM3_PMU_Q_IO_DEFAULT;
+ case CM3_PMU_EV_MEM_REQ:
+ return CM3_PMU_Q_MEM_DEFAULT;
+ case CM3_PMU_EV_MEM_AXI:
+ case CM3_PMU_EV_IOCU_AXI(0) ... CM3_PMU_EV_IOCU_AXI_LAST:
+ return CM3_PMU_Q_AXI_DEFAULT;
+ case CM3_PMU_EV_PDTRACE_DROPPED:
+ return CM3_PMU_Q_PDTRACE_DEFAULT;
+ default:
+ /* All qualifier groups set to "don't care" */
+ return GENMASK_ULL(63, 0);
+ }
+}
+
+/* perf callbacks */
+
+static void cm3_pmu_update(struct perf_event *event)
+{
+ struct cm3_pmu *cm3_pmu = to_cm3_pmu(event->pmu);
+ struct hw_perf_event *hwc = &event->hw;
+ u32 ov_bit = cm3_pmu_overflow_bit(hwc->idx);
+ u64 delta, prev, new;
+ bool overflowed = false;
+
+ /* Counts were harvested when the cluster went down, nothing since */
+ if (READ_ONCE(cm3_pmu->paused))
+ return;
+
+ /*
+ * When polling, the counter cannot wrap twice between two reads and
+ * the 32-bit difference tells the whole story. With the overflow
+ * interrupt the counter is only read on demand and may have made a
+ * full turn since the last read, which the overflow status tells.
+ * It is read and cleared before the counter so that a wrap seen
+ * here is also reflected in the counter value; a wrap happening in
+ * between is accounted by the next update, which the interrupt it
+ * raises makes sure happens. In interrupt mode every access to the
+ * event runs on cm3_pmu->on_cpu with interrupts disabled, so the
+ * status and the counter are read consistently.
+ */
+ if (cm3_pmu->irq > 0) {
+ overflowed = CM3_PMU_READ(cm3_pmu, pc_ov) & ov_bit;
+ if (overflowed)
+ CM3_PMU_WRITE(cm3_pmu, pc_ov, ov_bit);
+ }
+
+ do {
+ prev = local64_read(&hwc->prev_count);
+ new = cm3_pmu_read_counter(cm3_pmu, hwc->idx);
+ } while (local64_cmpxchg(&hwc->prev_count, prev, new) != prev);
+
+ /*
+ * If the counter wrapped and now shows a value not below the
+ * previous one, it made a full turn on top of the 32-bit difference.
+ */
+ delta = (new - prev) & CM3_PMU_COUNTER_MASK;
+ if (overflowed && new >= prev)
+ delta += CM3_PMU_COUNTER_MASK + 1;
+
+ local64_add(delta, &event->count);
+}
+
+/* Program the counter of the event and start it from zero */
+static void cm3_pmu_start_counter(struct cm3_pmu *cm3_pmu,
+ struct perf_event *event)
+{
+ struct hw_perf_event *hwc = &event->hw;
+ int idx = hwc->idx;
+
+ if (idx != CM3_PMU_IDX_CYCLES)
+ cm3_pmu_program_counter(cm3_pmu, idx, hwc->config,
+ hwc->config1);
+
+ /*
+ * Setting the reset bit together with the count-on bit resets the
+ * counter and its overflow status before counting is started.
+ */
+ local64_set(&hwc->prev_count, 0);
+ cm3_pmu_set_ctl(cm3_pmu, 0,
+ cm3_pmu_reset_bit(idx) | cm3_pmu_counton_bit(idx));
+}
+
+static void cm3_pmu_start(struct perf_event *event, int flags)
+{
+ struct cm3_pmu *cm3_pmu = to_cm3_pmu(event->pmu);
+ struct hw_perf_event *hwc = &event->hw;
+
+ if (WARN_ON_ONCE(!(hwc->state & PERF_HES_STOPPED)))
+ return;
+
+ hwc->state = 0;
+
+ /* A paused PMU starts its running events when the cluster is back */
+ if (!cm3_pmu->paused)
+ cm3_pmu_start_counter(cm3_pmu, event);
+
+ perf_event_update_userpage(event);
+}
+
+static void cm3_pmu_stop(struct perf_event *event, int flags)
+{
+ struct cm3_pmu *cm3_pmu = to_cm3_pmu(event->pmu);
+ struct hw_perf_event *hwc = &event->hw;
+
+ if (hwc->state & PERF_HES_STOPPED)
+ return;
+
+ if (!cm3_pmu->paused)
+ cm3_pmu_set_ctl(cm3_pmu, cm3_pmu_counton_bit(hwc->idx), 0);
+
+ cm3_pmu_update(event);
+ hwc->state |= PERF_HES_STOPPED | PERF_HES_UPTODATE;
+}
+
+static int cm3_pmu_add(struct perf_event *event, int flags)
+{
+ struct cm3_pmu *cm3_pmu = to_cm3_pmu(event->pmu);
+ struct hw_perf_event *hwc = &event->hw;
+ int idx;
+
+ if (hwc->config == CM3_PMU_EV_CYCLES) {
+ idx = CM3_PMU_IDX_CYCLES;
+ if (cm3_pmu->events[idx])
+ return -EAGAIN;
+ } else {
+ for (idx = 0; idx < cm3_pmu->num_pcs; idx++)
+ if (!cm3_pmu->events[idx])
+ break;
+ if (idx == cm3_pmu->num_pcs)
+ return -EAGAIN;
+ }
+
+ hwc->idx = idx;
+ hwc->state = PERF_HES_STOPPED | PERF_HES_UPTODATE;
+ cm3_pmu->events[idx] = event;
+
+ /* No polling while the cluster is down, resume takes care of it */
+ if (!cm3_pmu->num_active++ && cm3_pmu->irq <= 0 && !cm3_pmu->paused)
+ hrtimer_start(&cm3_pmu->hrtimer,
+ ns_to_ktime(CM3_PMU_POLL_PERIOD_NS),
+ HRTIMER_MODE_REL_PINNED);
+
+ if (flags & PERF_EF_START)
+ cm3_pmu_start(event, PERF_EF_RELOAD);
+
+ perf_event_update_userpage(event);
+
+ return 0;
+}
+
+static void cm3_pmu_del(struct perf_event *event, int flags)
+{
+ struct cm3_pmu *cm3_pmu = to_cm3_pmu(event->pmu);
+ struct hw_perf_event *hwc = &event->hw;
+
+ cm3_pmu_stop(event, PERF_EF_UPDATE);
+
+ cm3_pmu->events[hwc->idx] = NULL;
+ if (!--cm3_pmu->num_active && cm3_pmu->irq <= 0)
+ hrtimer_cancel(&cm3_pmu->hrtimer);
+
+ perf_event_update_userpage(event);
+}
+
+static void cm3_pmu_count_group_event(struct perf_event *event,
+ struct perf_event *member,
+ unsigned int *cycles, unsigned int *pcs)
+{
+ if (member->pmu != event->pmu)
+ return;
+
+ if (member->hw.config == CM3_PMU_EV_CYCLES)
+ (*cycles)++;
+ else
+ (*pcs)++;
+}
+
+static bool cm3_pmu_validate_group(struct perf_event *event)
+{
+ struct cm3_pmu *cm3_pmu = to_cm3_pmu(event->pmu);
+ struct perf_event *leader = event->group_leader;
+ struct perf_event *sibling;
+ unsigned int cycles = 0, pcs = 0;
+
+ cm3_pmu_count_group_event(event, event, &cycles, &pcs);
+
+ /* A new leader has no siblings yet, and no context to assert on */
+ if (leader == event)
+ return pcs <= cm3_pmu->num_pcs;
+
+ /* Hardware events of the group must all be ours */
+ if (leader->pmu != event->pmu && !is_software_event(leader))
+ return false;
+
+ cm3_pmu_count_group_event(event, leader, &cycles, &pcs);
+
+ for_each_sibling_event(sibling, leader) {
+ if (sibling->pmu != event->pmu && !is_software_event(sibling))
+ return false;
+
+ cm3_pmu_count_group_event(event, sibling, &cycles, &pcs);
+ }
+
+ return cycles <= 1 && pcs <= cm3_pmu->num_pcs;
+}
+
+static int cm3_pmu_event_init(struct perf_event *event)
+{
+ struct cm3_pmu *cm3_pmu = to_cm3_pmu(event->pmu);
+ struct hw_perf_event *hwc = &event->hw;
+ u64 config = event->attr.config;
+ int ret;
+
+ if (event->attr.type != event->pmu->type)
+ return -ENOENT;
+
+ /* Counters are shared by all CPUs: no sampling, no per-task events */
+ if (is_sampling_event(event) ||
+ event->attach_state & PERF_ATTACH_TASK)
+ return -EOPNOTSUPP;
+
+ if (event->cpu < 0)
+ return -EINVAL;
+
+ if (config & ~CM3_PMU_EV_MASK)
+ return -EINVAL;
+
+ if (config != CM3_PMU_EV_CYCLES &&
+ (config & ~CM3_PMU_EV_HW_MASK || config == CM3_PMU_EV_NONE))
+ return -EINVAL;
+
+ /* The cluster's CM must be powered for its counters to be usable */
+ if (READ_ONCE(cm3_pmu->paused))
+ return -ENODEV;
+
+ ret = cm3_pmu_hw_init(cm3_pmu);
+ if (ret)
+ return ret;
+
+ hwc->idx = -1;
+ hwc->config = config;
+ hwc->config1 = event->attr.config1 ?: cm3_pmu_default_qual(config);
+
+ /* Group validation looks at hwc->config of every member, ours too */
+ if (!cm3_pmu_validate_group(event))
+ return -EINVAL;
+
+ event->cpu = cm3_pmu->on_cpu;
+
+ return 0;
+}
+
+static enum hrtimer_restart cm3_pmu_poll(struct hrtimer *hrtimer)
+{
+ struct cm3_pmu *cm3_pmu = container_of(hrtimer, struct cm3_pmu,
+ hrtimer);
+ int idx;
+
+ /*
+ * Nothing to poll while the cluster is down, resume re-arms the
+ * timer. The timer may run on another CPU than the pause did.
+ */
+ if (READ_ONCE(cm3_pmu->paused))
+ return HRTIMER_NORESTART;
+
+ for (idx = 0; idx < CM3_PMU_NUM_COUNTERS; idx++) {
+ struct perf_event *event = cm3_pmu->events[idx];
+
+ if (event && !(event->hw.state & PERF_HES_STOPPED))
+ cm3_pmu_update(event);
+ }
+
+ hrtimer_forward_now(hrtimer, ns_to_ktime(CM3_PMU_POLL_PERIOD_NS));
+
+ return HRTIMER_RESTART;
+}
+
+static irqreturn_t cm3_pmu_handle_irq(int irq, void *data)
+{
+ struct cm3_pmu *cm3_pmu = data;
+ u32 ov = CM3_PMU_READ(cm3_pmu, pc_ov);
+ int idx;
+
+ if (!ov)
+ return IRQ_NONE;
+
+ /*
+ * Status bits are write-1-to-clear and clearing them deasserts the
+ * interrupt. cm3_pmu_update() clears the bit of the counter it
+ * accounts; only bits of counters without an event are left over.
+ */
+ for (idx = 0; idx < CM3_PMU_NUM_COUNTERS; idx++) {
+ struct perf_event *event = cm3_pmu->events[idx];
+
+ if (!(ov & cm3_pmu_overflow_bit(idx)))
+ continue;
+
+ if (event)
+ cm3_pmu_update(event);
+ else
+ CM3_PMU_WRITE(cm3_pmu, pc_ov, cm3_pmu_overflow_bit(idx));
+ }
+
+ return IRQ_HANDLED;
+}
+
+/*
+ * CPU hotplug
+ *
+ * Besides following the CPU the PMUs run on, this tracks the power state of
+ * the clusters holding CPUs. Once Linux has booted a cluster it leaves
+ * CPC_PWRUP_CTL.CM_PWRUP clear, so that the CM powers down together with
+ * the last of its cores, taking the counters with it. The PMU of such a
+ * cluster is paused when its last CPU goes offline: the counts are
+ * harvested, the running events stop counting but keep their state, and no
+ * new event can be opened on it. It resumes when a CPU of the cluster comes
+ * back online, the CM having been powered up for it.
+ *
+ * The pause and resume run on cm3_pmu->on_cpu with interrupts disabled,
+ * like the perf callbacks, so they cannot race with them. As the pause
+ * happens before the last CPU of the cluster dies, and thus before the CM
+ * can power down, code running on cm3_pmu->on_cpu with interrupts disabled
+ * that finds the PMU unpaused can rely on the cluster staying powered.
+ */
+
+static void cm3_pmu_pause(void *data)
+{
+ struct cm3_pmu *cm3_pmu = data;
+ int idx;
+
+ for (idx = 0; idx < CM3_PMU_NUM_COUNTERS; idx++) {
+ struct perf_event *event = cm3_pmu->events[idx];
+
+ if (event && !(event->hw.state & PERF_HES_STOPPED))
+ cm3_pmu_update(event);
+ }
+
+ WRITE_ONCE(cm3_pmu->paused, true);
+}
+
+static void cm3_pmu_resume(void *data)
+{
+ struct cm3_pmu *cm3_pmu = data;
+ int idx;
+
+ WRITE_ONCE(cm3_pmu->paused, false);
+
+ /* The registers are back to their reset values; make sure of it */
+ cm3_pmu_hw_reset(cm3_pmu);
+
+ for (idx = 0; idx < CM3_PMU_NUM_COUNTERS; idx++) {
+ struct perf_event *event = cm3_pmu->events[idx];
+
+ if (event && !(event->hw.state & PERF_HES_STOPPED))
+ cm3_pmu_start_counter(cm3_pmu, event);
+ }
+
+ /*
+ * Restart polling if the pause stopped it. Running here, on
+ * cm3_pmu->on_cpu, also pins the timer to the CPU the PMU is on.
+ */
+ if (cm3_pmu->num_active && cm3_pmu->irq <= 0)
+ hrtimer_start(&cm3_pmu->hrtimer,
+ ns_to_ktime(CM3_PMU_POLL_PERIOD_NS),
+ HRTIMER_MODE_REL_PINNED);
+}
+
+/* Whether another online CPU than @cpu lives in @cluster */
+static bool cm3_pmu_cluster_has_other_cpus(unsigned int cluster,
+ unsigned int cpu)
+{
+ unsigned int c;
+
+ for_each_online_cpu(c)
+ if (c != cpu && cpu_cluster(&cpu_data[c]) == cluster)
+ return true;
+
+ return false;
+}
+
+/*
+ * Whether the CM of the local cluster is kept powered when all its cores
+ * power down. Must run on a CPU of that cluster, as the hotplug callbacks
+ * do. Without a CPC, or before clusters, nothing powers the CM down.
+ */
+static bool cm3_pmu_cluster_stays_powered(void)
+{
+ if (mips_cm_revision() < CM_REV_CM3_5 || !mips_cpc_present())
+ return true;
+
+ return read_cpc_pwrup_ctl() & CPC_PWRUP_CTL_CM_PWRUP;
+}
+
+static int cm3_pmu_online_cpu(unsigned int cpu, struct hlist_node *node)
+{
+ struct cm3_pmu *cm3_pmu = hlist_entry_safe(node, struct cm3_pmu, node);
+ unsigned int cluster = cpu_cluster(&cpu_data[cpu]);
+
+ if (cm3_pmu->on_cpu < 0) {
+ cm3_pmu->on_cpu = cpu;
+ if (cm3_pmu->irq > 0)
+ WARN_ON(irq_set_affinity(cm3_pmu->irq, cpumask_of(cpu)));
+ }
+
+ /* First CPU of the cluster back: its CM is powered again */
+ if (cm3_pmu->cluster == cluster && cm3_pmu->paused &&
+ !cm3_pmu_cluster_has_other_cpus(cluster, cpu))
+ smp_call_function_single(cm3_pmu->on_cpu, cm3_pmu_resume,
+ cm3_pmu, 1);
+
+ return 0;
+}
+
+static int cm3_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
+{
+ struct cm3_pmu *cm3_pmu = hlist_entry_safe(node, struct cm3_pmu, node);
+ unsigned int cluster = cpu_cluster(&cpu_data[cpu]);
+ unsigned int target;
+
+ if (cm3_pmu->on_cpu == cpu) {
+ target = cpumask_any_but(cpu_online_mask, cpu);
+ if (target >= nr_cpu_ids) {
+ cm3_pmu->on_cpu = -1;
+ return 0;
+ }
+
+ perf_pmu_migrate_context(&cm3_pmu->pmu, cpu, target);
+ cm3_pmu->on_cpu = target;
+ if (cm3_pmu->irq > 0)
+ WARN_ON(irq_set_affinity(cm3_pmu->irq,
+ cpumask_of(target)));
+ }
+
+ /* Last CPU of the cluster going: its CM may follow */
+ if (cm3_pmu->cluster == cluster && !cm3_pmu->paused &&
+ !cm3_pmu_cluster_has_other_cpus(cluster, cpu) &&
+ !cm3_pmu_cluster_stays_powered()) {
+ smp_call_function_single(cm3_pmu->on_cpu, cm3_pmu_pause,
+ cm3_pmu, 1);
+ /*
+ * The poll timer may have been migrated away from the PMU's
+ * CPU by an earlier hotplug and be running right now; wait
+ * for it. Resume starts it again.
+ */
+ hrtimer_cancel(&cm3_pmu->hrtimer);
+ }
+
+ return 0;
+}
+
+/* Platform driver */
+
+/* Leave the counters stopped, overflow interrupt disabled */
+static void cm3_pmu_hw_exit_on_cpu(void *data)
+{
+ struct cm3_pmu *cm3_pmu = data;
+
+ if (cm3_pmu->paused)
+ return;
+
+ cm3_pmu->irq = 0;
+ cm3_pmu_hw_reset(cm3_pmu);
+}
+
+static void cm3_pmu_unregister(struct cm3_pmu *cm3_pmu)
+{
+ perf_pmu_unregister(&cm3_pmu->pmu);
+
+ /* While the hotplug callbacks still track the cluster's power */
+ if (cm3_pmu->initialized && cm3_pmu_cluster_powered(cm3_pmu))
+ cm3_pmu_call_on_cpu(cm3_pmu, cm3_pmu_hw_exit_on_cpu, cm3_pmu);
+
+ cpuhp_state_remove_instance_nocalls(cm3_pmu_cpuhp_state,
+ &cm3_pmu->node);
+ hrtimer_cancel(&cm3_pmu->hrtimer);
+}
+
+static int cm3_pmu_register(struct device *dev, struct cm3_pmu *cm3_pmu)
+{
+ const char *name;
+ int ret;
+
+ cm3_pmu->on_cpu = -1;
+ cm3_pmu->num_pcs = CM3_PMU_MAX_PCS;
+ mutex_init(&cm3_pmu->init_lock);
+ hrtimer_setup(&cm3_pmu->hrtimer, cm3_pmu_poll, CLOCK_MONOTONIC,
+ HRTIMER_MODE_REL);
+
+ cm3_pmu->pmu = (struct pmu) {
+ .parent = dev,
+ .module = THIS_MODULE,
+ .task_ctx_nr = perf_invalid_context,
+ .capabilities = PERF_PMU_CAP_NO_EXCLUDE |
+ PERF_PMU_CAP_NO_INTERRUPT,
+ .event_init = cm3_pmu_event_init,
+ .add = cm3_pmu_add,
+ .del = cm3_pmu_del,
+ .start = cm3_pmu_start,
+ .stop = cm3_pmu_stop,
+ .read = cm3_pmu_update,
+ .attr_groups = cm3_pmu_attr_groups,
+ };
+
+ name = devm_kasprintf(dev, GFP_KERNEL, CM3_PMU_NAME "_%u",
+ cm3_pmu->cluster);
+ if (!name)
+ return -ENOMEM;
+
+ /* Picks the CPU the PMU runs on, needed by cm3_pmu_hw_init() */
+ ret = cpuhp_state_add_instance(cm3_pmu_cpuhp_state, &cm3_pmu->node);
+ if (ret)
+ return dev_err_probe(dev, ret, "failed to register hotplug\n");
+
+ /* Initialize now if the cluster is powered, else on first use */
+ ret = cm3_pmu_hw_init(cm3_pmu);
+ if (ret && ret != -ENODEV)
+ goto err_remove_instance;
+
+ ret = perf_pmu_register(&cm3_pmu->pmu, name, -1);
+ if (ret) {
+ dev_err_probe(dev, ret, "failed to register %s\n", name);
+ goto err_remove_instance;
+ }
+
+ if (cm3_pmu->initialized)
+ dev_info(dev, "%s: %u counters + cycles, %s\n", name,
+ cm3_pmu->num_pcs,
+ cm3_pmu->irq > 0 ? "overflow interrupt" : "polled");
+ else
+ dev_info(dev, "%s: cluster powered down\n", name);
+
+ return 0;
+
+err_remove_instance:
+ cpuhp_state_remove_instance_nocalls(cm3_pmu_cpuhp_state,
+ &cm3_pmu->node);
+
+ return ret;
+}
+
+static int cm3_pmu_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct cm3_pmu_dev *cm3_dev;
+ unsigned int nclusters, i;
+ int irq, ret;
+
+ if (!mips_cm_present() || mips_cm_revision() < CM_REV_CM3)
+ return -ENODEV;
+
+ /* One PMU per cluster, whether or not it holds CPUs */
+ nclusters = mips_cps_numclusters();
+ cm3_dev = devm_kzalloc(dev, struct_size(cm3_dev, pmus, nclusters),
+ GFP_KERNEL);
+ if (!cm3_dev)
+ return -ENOMEM;
+
+ cm3_dev->num_pmus = nclusters;
+ for (i = 0; i < nclusters; i++)
+ cm3_dev->pmus[i].cluster = i;
+
+ /*
+ * Each cluster's CM raises its own overflow interrupt into its own
+ * GIC, and a GIC shared interrupt is routed to a single cluster at a
+ * time (the one of its affinity CPU). The interrupt can therefore
+ * only serve the cluster the CPUs live in, and only if they all live
+ * in that one cluster so that any CPU may handle it. Other clusters
+ * are polled.
+ */
+ irq = platform_get_irq_optional(pdev, 0);
+ if (irq > 0 && !mips_cps_multicluster_cpus()) {
+ struct cm3_pmu *cm3_pmu;
+
+ cm3_pmu = &cm3_dev->pmus[cpu_cluster(&boot_cpu_data)];
+ ret = devm_request_irq(dev, irq, cm3_pmu_handle_irq,
+ IRQF_NOBALANCING | IRQF_NO_THREAD,
+ CM3_PMU_NAME, cm3_pmu);
+ if (ret)
+ return dev_err_probe(dev, ret, "failed to request IRQ\n");
+
+ cm3_pmu->irq = irq;
+ } else if (irq > 0) {
+ dev_info(dev, "CPUs in several clusters: overflow interrupt not usable, polling\n");
+ } else if (irq != -ENXIO) {
+ return dev_err_probe(dev, irq, "failed to get IRQ\n");
+ }
+
+ for (i = 0; i < cm3_dev->num_pmus; i++) {
+ ret = cm3_pmu_register(dev, &cm3_dev->pmus[i]);
+ if (ret)
+ goto err_unregister;
+ }
+
+ platform_set_drvdata(pdev, cm3_dev);
+
+ return 0;
+
+err_unregister:
+ while (i--)
+ cm3_pmu_unregister(&cm3_dev->pmus[i]);
+
+ return ret;
+}
+
+static void cm3_pmu_remove(struct platform_device *pdev)
+{
+ struct cm3_pmu_dev *cm3_dev = platform_get_drvdata(pdev);
+ unsigned int i;
+
+ for (i = 0; i < cm3_dev->num_pmus; i++)
+ cm3_pmu_unregister(&cm3_dev->pmus[i]);
+}
+
+static const struct of_device_id cm3_pmu_of_match[] = {
+ { .compatible = "mti,mips-cm" },
+ { .compatible = "mobileye,eyeq6-cm" },
+ {}
+};
+MODULE_DEVICE_TABLE(of, cm3_pmu_of_match);
+
+static struct platform_driver cm3_pmu_driver = {
+ .driver = {
+ .name = CM3_PMU_NAME,
+ .of_match_table = cm3_pmu_of_match,
+ .suppress_bind_attrs = true,
+ },
+ .probe = cm3_pmu_probe,
+ .remove = cm3_pmu_remove,
+};
+
+static int __init cm3_pmu_init(void)
+{
+ int ret;
+
+ ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
+ "perf/mips/cm3_pmu:online",
+ cm3_pmu_online_cpu, cm3_pmu_offline_cpu);
+ if (ret < 0)
+ return ret;
+
+ cm3_pmu_cpuhp_state = ret;
+
+ ret = platform_driver_register(&cm3_pmu_driver);
+ if (ret)
+ cpuhp_remove_multi_state(cm3_pmu_cpuhp_state);
+
+ return ret;
+}
+module_init(cm3_pmu_init);
+
+static void __exit cm3_pmu_exit(void)
+{
+ platform_driver_unregister(&cm3_pmu_driver);
+ cpuhp_remove_multi_state(cm3_pmu_cpuhp_state);
+}
+module_exit(cm3_pmu_exit);
+
+MODULE_DESCRIPTION("MIPS Coherence Manager 3 (CM3) PMU driver");
+MODULE_LICENSE("GPL");
--
2.43.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v2 6/6] MIPS: eyeq6_defconfig: enable the CM3 PMU
2026-11-12 12:28 [PATCH v2 0/6] perf: add PMU driver for the MIPS Coherence Manager 3 performance counters Vladimir Kondratiev
` (4 preceding siblings ...)
2026-11-12 12:29 ` [PATCH v2 5/6] perf: add MIPS Coherence Manager 3 (CM3) PMU driver Vladimir Kondratiev
@ 2026-11-12 12:29 ` Vladimir Kondratiev
5 siblings, 0 replies; 8+ messages in thread
From: Vladimir Kondratiev @ 2026-11-12 12:29 UTC (permalink / raw)
To: Thomas Bogendoerfer, Sebastian Andrzej Siewior, Clark Williams,
Steven Rostedt, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Jiaxun Yang, Jonathan Corbet, Shuah Khan, Randy Dunlap,
Will Deacon, Mark Rutland, Kees Cook, Gustavo A. R. Silva,
Benoît Monin, Gregory CLEMENT, Théo Lebrun,
Fu Chao-Ying
Cc: Vladimir Kondratiev, linux-mips, linux-kernel, linux-rt-devel,
devicetree, linux-doc, linux-arm-kernel, linux-perf-users,
linux-hardening
Enable perf events and build the Coherence Manager 3 PMU driver as a
module.
Signed-off-by: Vladimir Kondratiev <vladimir.kondratiev@mobileye.com>
---
arch/mips/configs/eyeq6_defconfig | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/mips/configs/eyeq6_defconfig b/arch/mips/configs/eyeq6_defconfig
index 0a00a201937b..2d06d9ed6526 100644
--- a/arch/mips/configs/eyeq6_defconfig
+++ b/arch/mips/configs/eyeq6_defconfig
@@ -19,6 +19,7 @@ CONFIG_USER_NS=y
CONFIG_SCHED_AUTOGROUP=y
CONFIG_BLK_DEV_INITRD=y
CONFIG_EXPERT=y
+CONFIG_PERF_EVENTS=y
CONFIG_EYEQ=y
CONFIG_MACH_EYEQ6H=y
CONFIG_MIPS_CPS=y
@@ -86,6 +87,7 @@ CONFIG_MMC_SDHCI_PLTFM=y
CONFIG_MMC_SDHCI_CADENCE=y
# CONFIG_IOMMU_SUPPORT is not set
CONFIG_RESET_CONTROLLER=y
+CONFIG_MIPS_CM3_PMU=m
# CONFIG_NVMEM is not set
CONFIG_EXT4_FS=y
CONFIG_EXT4_FS_POSIX_ACL=y
--
2.43.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v2 1/6] MIPS: CM: Make cm_core_lock raw spinlocks
2026-11-12 12:28 ` [PATCH v2 1/6] MIPS: CM: Make cm_core_lock raw spinlocks Vladimir Kondratiev
@ 2026-11-12 12:39 ` sashiko-bot
0 siblings, 0 replies; 8+ messages in thread
From: sashiko-bot @ 2026-11-12 12:39 UTC (permalink / raw)
To: Vladimir Kondratiev
Cc: linux-rt-devel, devicetree, linux-perf-users, robh, conor+dt,
linux-kernel
Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [High] mips: cm: invalid wait context in mips_cpc_lock_other() under
cm_core_lock
--
commit 4e39a6ca6a7f0be5ae6999f6aa8d3e3e05b7551d
Author: Paul Burton <paulburton@kernel.org>
MIPS: CM: Make cm_core_lock raw spinlocks
This commit converts the per-CPU cm_core_lock from spinlock_t to
raw_spinlock_t so that preemption remains disabled while accessing the
core-other CM register block on PREEMPT_RT kernels.
> diff --git a/arch/mips/kernel/mips-cm.c b/arch/mips/kernel/mips-cm.c
> index 7c9c5dc38823f..3bfe85e86a9a5 100644
> --- a/arch/mips/kernel/mips-cm.c
> +++ b/arch/mips/kernel/mips-cm.c
[ ... ]
> @@ -359,8 +359,8 @@ void mips_cm_lock_other(unsigned int cluster, unsigned int core,
> * race with us.
> */
> curr_core = cpu_core(¤t_cpu_data);
> - spin_lock_irqsave(&per_cpu(cm_core_lock, curr_core),
> - per_cpu(cm_core_lock_flags, curr_core));
> + raw_spin_lock_irqsave(&per_cpu(cm_core_lock, curr_core),
> + per_cpu(cm_core_lock_flags, curr_core));
>
> val = FIELD_PREP(CM_GCR_Cx_OTHER_CORENUM, core);
> }
[Severity: High]
Does converting cm_core_lock to raw_spinlock_t without also converting
cpc_core_lock cause an invalid wait context in mips_cpc_lock_other() on
systems with CM < 3?
mips_cpc_lock_other() is always called inside a mips_cm_lock_other() /
mips_cm_unlock_other() section (for example in boot_core() and
mips_smp_send_ipi_mask()). On CM < 3 systems, mips_cpc_lock_other()
acquires cpc_core_lock, which is still a spinlock_t:
arch/mips/kernel/mips-cpc.c:mips_cpc_lock_other() {
if (mips_cm_revision() >= CM_REV_CM3)
/* Systems with CM >= 3 lock the CPC via mips_cm_lock_other */
return;
preempt_disable();
curr_core = cpu_core(¤t_cpu_data);
spin_lock_irqsave(&per_cpu(cpc_core_lock, curr_core),
per_cpu(cpc_core_lock_flags, curr_core));
...
}
Taking spinlock_t cpc_core_lock while holding raw_spinlock_t cm_core_lock
triggers a lockdep invalid wait context warning and sleeps in atomic
context on PREEMPT_RT kernels.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261112-i6500-cm3-pmu-v2-0-cd2569571819@mobileye.com?part=1
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2026-11-12 12:39 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-12 12:28 [PATCH v2 0/6] perf: add PMU driver for the MIPS Coherence Manager 3 performance counters Vladimir Kondratiev
2026-11-12 12:28 ` [PATCH v2 1/6] MIPS: CM: Make cm_core_lock raw spinlocks Vladimir Kondratiev
2026-11-12 12:39 ` sashiko-bot
2026-11-12 12:28 ` [PATCH v2 2/6] MIPS: CM: add CM3 performance counter GCR definitions Vladimir Kondratiev
2026-11-12 12:29 ` [PATCH v2 3/6] MIPS: CM: export the symbols needed to access the GCRs from a module Vladimir Kondratiev
2026-11-12 12:29 ` [PATCH v2 4/6] dt-bindings: mips: mti,mips-cm: add performance counter interrupt Vladimir Kondratiev
2026-11-12 12:29 ` [PATCH v2 5/6] perf: add MIPS Coherence Manager 3 (CM3) PMU driver Vladimir Kondratiev
2026-11-12 12:29 ` [PATCH v2 6/6] MIPS: eyeq6_defconfig: enable the CM3 PMU Vladimir Kondratiev
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®