[PATCH v9 00/12] Introduce MMIO-based CMT access for Enhanced RDT

[PATCH v9 00/12] Introduce MMIO-based CMT access for Enhanced RDT

[Chen Yu]

v8: https://lore.kernel.org/lkml/cover.1789705667.git.yu.c.chen@intel.com/
v7: https://lore.kernel.org/lkml/cover.1787976868.git.yu.c.chen@intel.com/
v6: https://lore.kernel.org/lkml/cover.1784968626.git.yu.c.chen@intel.com/
v5: https://lore.kernel.org/lkml/cover.1782866200.git.yu.c.chen@intel.com/
v4: https://lore.kernel.org/lkml/cover.1781332698.git.yu.c.chen@intel.com/
v3: https://lore.kernel.org/lkml/cover.1780710620.git.yu.c.chen@intel.com/
v2: https://lore.kernel.org/lkml/cover.1780587063.git.yu.c.chen@intel.com/
v1: https://lore.kernel.org/lkml/cover.1779872016.git.yu.c.chen@intel.com/

Intel Enhanced Resource Director Technology (ERDT) extends the existing
RDT framework with two major capabilities:

  1. MMIO-based access to monitoring and allocation registers, replacing
     the legacy MSR-based interface.
  2. Region-aware RDT for fine-grained control over different tiers of
     memory (e.g., CXL.mem, DDR).

This is described in the Intel RDT Architecture Specification:
https://cdrdv2-public.intel.com/789566/356688-intel-rdt-arch-spec.pdf

This patch set focuses on the first part: enabling MMIO-based access for
Cache Monitoring Technology (CMT), while CAT/MBM/MBA are still using MSR.
The platform advertises the MMIO register layout through the ACPI ERDT
(Enhanced Resource Director Technology) table, which contains sub-tables
describing per-domain register regions for monitoring and allocation.

With ERDT, L3 cache occupancy counters are read via MMIO rather than
MSR, allowing the reads to be performed from any CPU without requiring
cross-CPU IPIs. This series parses the relevant ACPI sub-tables (RMDD,
CMRC), prepares the resctrl monitor infrastructure for MMIO-based reads,
and adds initial support for reading L3 occupancy via the CMRC interface.

kselftest of CMT and L3_CAT has passed with minor adjustment at
https://lore.kernel.org/lkml/20260523101715.3964456-1-yu.c.chen@intel.com/.

Thanks Tony, Reinette, Thomas, Dave, Boris, Peter, Hongyu for your time to
look at this patch set.

V9 has gone through local sashiko review, with false positives left
unaddressed. Also used AI to confirm each review feedback has been
addressed by v9. The change history in the section "Changes since v8"
in each patch is generated by AI.

Major functional changes from V8 to V9:
- Do not enable ERDT when Sub-NUMA Cluster (SNC) is enabled.
  New patch [PATCH 04/12]. (Reinette Chatre, Tony Luck, sashiko)
- CPUID remains the architectural enumeration mechanism: erdt_cpu_has()
  now requires boot_cpu_has(flag) in addition to ERDT firmware support.
  (Reinette Chatre, Tony Luck)
- [bug fix] Attach the ERDT information to the L3 monitoring domain before the
  domain is published on the RCU protected domain list.
  (Reinette Chatre, sashiko)
- Do not discard the whole ERDT table when a CACD x2APIC ID has no
  logical CPU, e.g. (Reinette Chatre, sashiko)

Changes from V7 to V8:
- Rebased on latest x86/cache branch
  git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git
- git commit log revisement
- pick Tony's x86_64 dependency patch from AET patch set and replaced
  mine.
- No other code logic changes

Changes from V6 to V7:
- Let get_rdt_resources() clean up its own state on failure and drop the
  asymmetric __resctrl_arch_late_init() wrapper. (Reinette Chatre)
- Make erdt_max_rmid and num_ids unsigned and use a plain min(), match
  erdt_ioremap() argument types to ioremap(), and reset erdt_max_rmid in
  erdt_exit(). (Reinette Chatre)
- Run erdt_cpu_valid() once in resctrl_arch_online_cpu(), and match an
  ERDT domain to a resctrl monitoring domain by domain id rather than by
  cpumask. Warn on an invalid dom_id and on a duplicate ERDT-to-resctrl
  domain mapping. (Reinette Chatre)
- Fix the silent truncation of the 64-bit CMRC up_scale: reject an
  out-of-range value with a FW_BUG warning and fail closed instead of
  using max_t(int, ...). (Reinette Chatre, Tony Luck)
- Keep the original resctrl_arch_rmid_read() parameter order (resource
  first) and drop the unused closid/arch_priv arguments. (Reinette Chatre)
- Keep a single erdt_cpu_has() declaration in asm/resctrl.h, required by
  the inline resctrl_arch_round_mon_val(). (Reinette Chatre)
- Use switch() in erdt_support(), clamp max_rmid with min_t() so MSR-based
  MBM events are not handed the ERDT RMID, and divide the ERDT scale by
  snc_nodes_per_l3_cache under SNC. (Reinette Chatre)

Changes from V5 to V6:
- Reorder the series so that the x86/topology change, which touches a
  different subsystem, comes first. (Reinette Chatre)
- Drop the v5 "x86/resctrl: Replace 'msr' in monitoring data identifiers"
  patch. None of the renamed identifiers are used by the MMIO code.
  (Reinette Chatre)
- Split the domain setup into erdt_cpu_valid() and
  erdt_l3_mon_domain_setup(). Validation now happens in
  domain_add_cpu_mon() before the CPU is added to the domain cpumask,
  and attaching ERDT data to a freshly created domain (Reinette Chatre)
- Drop the SNC special case and the new resctrl_disable_mon_event().
  (Reinette Chatre)
- resctrl_arch_round_mon_val() now rounds to the ERDT scale instead of
  returning the value unchanged (Reinette Chatre)
- Make erdt_get_max_rmid() a global value instead of a per-CPU lookup.
  (Reinette Chatre)
- Use struct cpumask instead of cpumask_var_t, list_for_each_entry()
  instead of open coded container_of(), and document every member of
  struct erdt_domain_info. (Reinette Chatre)

Changes from V4 to V5:
There are some major changes since v4:
- (biggest change) Eliminate the xarray for runtime lookups; embed
  struct erdt_domain_info directly in rdt_hw_l3_mon_domain and assign
  during l3_mon_domain_setup().
- Separate CPUID and ACPI enumeration cleanly. Do not use CPUID feature
  flags to gate MMIO-based monitoring. Use ACPI table presence (e.g., CMRC table)
  to determine event enablement.
- Use ACPI RMDD's own "Max RMID" field for MMIO access instead of relying
  on CPUID's max RMID (which applies to MSR).
- Enforce the SNC constraint in code rather than burying it behind a comment
  WARN. Disable mon_capable in rdt_get_l3_mon_config() when ERDT is
  enabled and snc_nodes_per_l3_cache > 1.
- Split non-resctrl changes (topology.c, apic.h) into a separate preparatory
  patch prefixed with x86/topology.
- Move the "depends on X86" to "X86_64" adjustment to a separate patch with explicit
  justification in its changelog.

Changes from V3 to V4:
- Remove the redundant table length check in subtbl_valid() (Thomas Gleixner)
- Reuse subtbl_valid() for all the table iteration (Thomas Gleixner)
- Refine the commit log of [PATCH 5/6] to state that this change is a
  preparation for [PATCH 6/6] rather than fixing an existing issue
  (Thomas Gleixner, Reinette Chatre, Tony Luck)
- Fix if CACD lists all CPUs in the LLC domain (sashiko)
- Deal with a corner case that if there is no valid RMDD tables,
  the erdt_enabled_flag should remain false.(sashiko)
- Add Thomas's Reviewed-by and Hongyu's Tested-by.

Changes from V2 to V3:
- Wrap __resctrl_arch_late_init() to avoid the goto logic. (Thomas Gleixner)
- Make the variables in struct erdt_domain_info tabular format (Thomas Gleixner)
- Remove tail comments (Thomas Gleixner)
- Make the name of erdt_enabled() and variable in it consistent and
  comprehensible. (Thomas Gleixner)
- Use topo_lookup_cpuid() to search the CPU id according to the x2apic id
  (Thomas Gleixner)
- Fix kernel doc comment format (Thomas Gleixner)
- Use brackets for multiple lines "if" case. (Thomas Gleixner)
- Let the parameter for cacd_init() to fully utilize 100 characters.
  (Thomas Gleixner)
- Variables are reordered in reverse fir-tree.(Thomas Gleixner)
- Added a named constant and use it in the rmdd->flags check.
  (Thomas Gleixner)
- Introduce helper functions to make the code readable when iterating
  the RMDD tables. (Thomas Gleixner)
- Make the macros tabular format. (Thomas Gleixner)

Changes from V1 to V2:
- Add #include <linux/cleanup.h> to follow the "include-what-you-use" best
  practice (Tony Luck)
- Fix 3 issues reported by:
  https://sashiko.dev/#/patchset/cover.1779872016.git.yu.c.chen%40intel.com
  Remove the variable of cacd in struct erdt_domain_info as it will
  never be used after initialization.
  Invoke erdt_exit() to avoid resource leak if rdt_alloc_capable and
  rdt_mon_capable are both false.
  Adjust the comments suggested by sashiko.

Chen Yu (10):
  x86/topology: Export topo_lookup_cpuid() for resctrl use
  x86/resctrl: Parse ACPI ERDT table and save CACD cpumask per RMD
  x86/resctrl: Do not enable ERDT when Sub-NUMA Cluster is enabled
  x86/resctrl: Attach ACPI ERDT information to L3 mon domain on CPU
    online
  x86,fs/resctrl: Move resctrl_arch_round_mon_val() out of line
  x86/resctrl: Add a global ERDT monitor scale
  x86/resctrl: Parse ACPI CMRC table
  x86/resctrl: Refactor the monitor read function
  x86/resctrl: Introduce erdt_cpu_has() and erdt_support()
  x86/resctrl: Add MMIO-based LLC occupancy monitoring support

Tony Luck (2):
  x86/resctrl: Require 64-bit x86 for resctrl support
  fs/resctrl: Do not invoke smp_processor_id() in preemptible context

 arch/x86/Kconfig                       |   4 +-
 arch/x86/include/asm/apic.h            |   1 +
 arch/x86/include/asm/resctrl.h         |   9 -
 arch/x86/kernel/cpu/resctrl/Makefile   |   1 +
 arch/x86/kernel/cpu/resctrl/core.c     |  45 ++-
 arch/x86/kernel/cpu/resctrl/erdt.c     | 525 +++++++++++++++++++++++++
 arch/x86/kernel/cpu/resctrl/internal.h |  45 +++
 arch/x86/kernel/cpu/resctrl/monitor.c  |  60 ++-
 arch/x86/kernel/cpu/topology.c         |   2 +-
 drivers/resctrl/mpam_resctrl.c         |   5 +
 fs/resctrl/monitor.c                   |  47 ++-
 include/linux/arm_mpam.h               |   5 -
 include/linux/resctrl.h                |   1 +
 13 files changed, 703 insertions(+), 47 deletions(-)
 create mode 100644 arch/x86/kernel/cpu/resctrl/erdt.c

-- 
2.43.0

[PATCH v9 01/12] x86/topology: Export topo_lookup_cpuid() for resctrl use

[Chen Yu]

Enhanced RDT (ERDT) enumerates its monitoring and control resources through
ACPI tables that identify each CPU by its x2APIC ID. To associate this
firmware-provided data with the kernel's per-CPU structures, ERDT must
translate those x2APIC IDs into logical CPU numbers.

topo_lookup_cpuid() already performs this translation, but it is static to
topology.c and therefore not reachable from resctrl.

Export topo_lookup_cpuid() and add its declaration to asm/apic.h that already
exposes the inverse mapping via cpuid_to_apicid[], so that ERDT can resolve
x2APIC IDs from ACPI tables to logical CPU numbers.

No functional change intended.

Signed-off-by: Chen Yu <yu.c.chen@intel.com>
Tested-by: Hongyu Ning <hongyu.ning@linux.intel.com>
Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
---
Changes since v8:
- No change.

 arch/x86/include/asm/apic.h    | 1 +
 arch/x86/kernel/cpu/topology.c | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index 9cd493d467d4..bb84651b14bd 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -54,6 +54,7 @@ static inline void x86_32_probe_apic(void) { }
 #endif
 
 extern u32 cpuid_to_apicid[];
+int topo_lookup_cpuid(u32 apic_id);
 
 #define CPU_ACPIID_INVALID	U32_MAX
 
diff --git a/arch/x86/kernel/cpu/topology.c b/arch/x86/kernel/cpu/topology.c
index 4913b64ec592..bcee70fb9277 100644
--- a/arch/x86/kernel/cpu/topology.c
+++ b/arch/x86/kernel/cpu/topology.c
@@ -92,7 +92,7 @@ static inline u32 topo_apicid(u32 apicid, enum x86_topology_domains dom)
 	return apicid & (UINT_MAX << x86_topo_system.dom_shifts[dom - 1]);
 }
 
-static int topo_lookup_cpuid(u32 apic_id)
+int topo_lookup_cpuid(u32 apic_id)
 {
 	int i;
 
-- 
2.43.0

[PATCH v9 02/12] x86/resctrl: Require 64-bit x86 for resctrl support

[Chen Yu]

From: Tony Luck <tony.luck@intel.com>

Intel Application Energy Telemetry (AET) provides event counts in 64-bit
registers in MMIO space. For accurate results these should be read with a
single 64-bit load operation from the CPU.

In preparation for the removal of the separate CONFIG_X86_CPU_RESCTRL_INTEL_AET
configuration option, move the X86_64 dependency to X86_CPU_RESCTRL.

This is in line with Dave Hansen's proposal[1] for an addition to Linux
documentation that details reduced 32-bit x86 support.

Signed-off-by: Tony Luck <tony.luck@intel.com>
Signed-off-by: Chen Yu <yu.c.chen@intel.com>
Link: https://lore.kernel.org/all/c6ab0c24-8931-4f0d-9be6-23498ab4c38b@intel.com/ # [1]
---
Changes since v8:
- Pick up the latest version of this patch from the AET series so that
  both series carry identical versions. Changelog wording only, no code
  change. (Reinette Chatre)

 arch/x86/Kconfig | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 15fd9ec5ecac..06b5d0c4633e 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -518,7 +518,7 @@ config X86_MPPARSE
 
 config X86_CPU_RESCTRL
 	bool "x86 CPU resource control support"
-	depends on X86 && (CPU_SUP_INTEL || CPU_SUP_AMD)
+	depends on X86_64 && (CPU_SUP_INTEL || CPU_SUP_AMD)
 	depends on MISC_FILESYSTEMS
 	select ARCH_HAS_CPU_RESCTRL
 	select RESCTRL_FS
@@ -541,7 +541,7 @@ config X86_CPU_RESCTRL
 
 config X86_CPU_RESCTRL_INTEL_AET
 	bool "Intel Application Energy Telemetry"
-	depends on X86_64 && X86_CPU_RESCTRL && CPU_SUP_INTEL && INTEL_PMT_TELEMETRY=y && INTEL_TPMI=y
+	depends on X86_CPU_RESCTRL && CPU_SUP_INTEL && INTEL_PMT_TELEMETRY=y && INTEL_TPMI=y
 	help
 	  Enable per-RMID telemetry events in resctrl.
 
-- 
2.43.0

[PATCH v9 03/12] x86/resctrl: Parse ACPI ERDT table and save CACD cpumask per RMD

[Chen Yu]

Enhanced RDT (ERDT) exposes per-domain MMIO registers that resctrl needs
in order to read hardware monitoring counters on the upcoming MMIO-based
path. The kernel discovers this hardware through a single per-platform
ERDT ACPI table.

Each Resource Management Domain Description (RMDD) sub-table within the
ERDT table represents one resource management domain (RMD). When
RMDD_FLAG_CPU_L3_DOMAIN is set, it identifies the domain as a CPU-scoped
L3 domain. The MMIO base of the domain's CMT counters is carried by the
RMDD's Cache Monitoring Registers for CPU Agents Description (CMRC)
sub-table. The set of CPUs in the domain is listed by x2APIC ID in the
CPU Agent Collection Description (CACD) sub-table.

The hierarchy of these ACPI tables is illustrated below:

  +---------------------------------------------------------------+
  | ERDT  (one instance per platform)                             |
  |                                                               |
  |   +-------------------------------------------------------+   |
  |   | RMDD #0                                               |   |
  |   |   domain_id, max_rmid                                 |   |
  |   |                                                       |   |
  |   |   +-----------------------------------------------+   |   |
  |   |   | CACD                                          |   |   |
  |   |   |   domain_id, x2APIC ID[0], x2APIC ID[1], ...  |   |   |
  |   |   +-----------------------------------------------+   |   |
  |   +-------------------------------------------------------+   |
  |   +-------------------------------------------------------+   |
  |   | RMDD #1                                               |   |
  |   |   ...                                                 |   |
  |   +-------------------------------------------------------+   |
  |   ...                                                         |
  |   +-------------------------------------------------------+   |
  |   | RMDD #N-1                                             |   |
  |   +-------------------------------------------------------+   |
  +---------------------------------------------------------------+

Walk the ERDT table's RMDD sub-tables in preparation for attaching each
ERDT domain to a resctrl L3 domain. For each CPU-based L3 RMDD,
ioremap its control-register region, walk its CACD entries,
translate each x2APIC ID to a logical CPU, and record the result on the
ERDT domain's erdt_domain_info. Record the largest RMID that is valid
on every RMD so a later reader cannot access an RMID that is
unsupported on some domain, and require every RMDD to advertise the
same set of sub-table types so downstream code can rely on a uniform
shape.

Originally-by: Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com>
Suggested-by: Tony Luck <tony.luck@intel.com>
Suggested-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Chen Yu <yu.c.chen@intel.com>
Tested-by: Hongyu Ning <hongyu.ning@linux.intel.com>
---
Changes since v8:
- Rename the subject from "... save CACD cpumask for RMDD domains" to
  "... save CACD cpumask per RMD". (Reinette Chatre)
- Rewrite the changelog to lead with the motivation, drop the duplicated
  first paragraph, and add a diagram of the ERDT/RMDD/CACD hierarchy.
  (Reinette Chatre)
- Replace "Based on original work from Anil S Keshavamurthy" with an
  Originally-by: tag. (Reinette Chatre)
- Keep the CONFIG_X86_CPU_RESCTRL entries grouped together in the
  Makefile. (Reinette Chatre)
- Rename the local "succeed" to "ret" in get_rdt_resources() to match
  the surrounding style. (Reinette Chatre)
- Make erdt_init() return void because its return value was ignored, and
  place __init before the return type. (Reinette Chatre)
- Move the "ERDT: " prefix into pr_fmt() and drop it from the individual
  messages. (Reinette Chatre)
- Rename ERDT_VALID_VERSION to ERDT_VALID_REVISION to match the ACPI
  header field it is compared against. (Reinette Chatre)
- Add ERDT_DOMAIN_ID_UNSET instead of the open coded -1. (Reinette Chatre)
- Make cacd_init() return bool instead of errno values that no caller
  consumed. (Reinette Chatre)
- Do not discard the whole ERDT table when a CACD x2APIC ID has no
  logical CPU, for example when the possible CPU mask is limited by
  nr_cpus=. Skip such a CPU instead. (sashiko, Reinette Chatre)
- Drop rmdd_subtbl() and open code the sub-table walk to match the style
  of enumerate_erdt_table(). (Reinette Chatre)
- Drop the always-zero argument from the RMDD max_rmid FW_BUG message.
  (Reinette Chatre)
- Comments: use the grep friendly "max_rmid", upper case the RMID
  acronym, and describe struct erdt_domain_info as "L3 domain ERDT
  information". (Reinette Chatre)

 arch/x86/kernel/cpu/resctrl/Makefile   |   1 +
 arch/x86/kernel/cpu/resctrl/core.c     |  16 +-
 arch/x86/kernel/cpu/resctrl/erdt.c     | 255 +++++++++++++++++++++++++
 arch/x86/kernel/cpu/resctrl/internal.h |  29 +++
 4 files changed, 299 insertions(+), 2 deletions(-)
 create mode 100644 arch/x86/kernel/cpu/resctrl/erdt.c

diff --git a/arch/x86/kernel/cpu/resctrl/Makefile b/arch/x86/kernel/cpu/resctrl/Makefile
index 273ddfa30836..a975c73e273e 100644
--- a/arch/x86/kernel/cpu/resctrl/Makefile
+++ b/arch/x86/kernel/cpu/resctrl/Makefile
@@ -1,6 +1,7 @@
 # SPDX-License-Identifier: GPL-2.0
 obj-$(CONFIG_X86_CPU_RESCTRL)		+= core.o rdtgroup.o monitor.o
 obj-$(CONFIG_X86_CPU_RESCTRL)		+= ctrlmondata.o
+obj-$(CONFIG_X86_CPU_RESCTRL)		+= erdt.o
 obj-$(CONFIG_X86_CPU_RESCTRL_INTEL_AET)	+= intel_aet.o
 obj-$(CONFIG_RESCTRL_FS_PSEUDO_LOCK)	+= pseudo_lock.o
 
diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index 55214d6fdc49..7fadfebb5312 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -1016,10 +1016,17 @@ static __init void check_quirks(void)
 
 static __init bool get_rdt_resources(void)
 {
+	bool ret;
+
+	erdt_init();
 	rdt_alloc_capable = get_rdt_alloc_resources();
 	rdt_mon_capable = get_rdt_mon_resources();
 
-	return (rdt_mon_capable || rdt_alloc_capable);
+	ret = (rdt_mon_capable || rdt_alloc_capable);
+	if (!ret)
+		erdt_exit();
+
+	return ret;
 }
 
 static __init void rdt_init_res_defs_intel(void)
@@ -1141,12 +1148,15 @@ static int __init resctrl_arch_late_init(void)
 				  "x86/resctrl/cat:online:",
 				  resctrl_arch_online_cpu,
 				  resctrl_arch_offline_cpu);
-	if (state < 0)
+	if (state < 0) {
+		erdt_exit();
 		return state;
+	}
 
 	ret = resctrl_init();
 	if (ret) {
 		cpuhp_remove_state(state);
+		erdt_exit();
 		return ret;
 	}
 	rdt_online = state;
@@ -1169,6 +1179,8 @@ static void __exit resctrl_arch_exit(void)
 	cpuhp_remove_state(rdt_online);
 
 	resctrl_exit();
+
+	erdt_exit();
 }
 
 __exitcall(resctrl_arch_exit);
diff --git a/arch/x86/kernel/cpu/resctrl/erdt.c b/arch/x86/kernel/cpu/resctrl/erdt.c
new file mode 100644
index 000000000000..eb5e262e243a
--- /dev/null
+++ b/arch/x86/kernel/cpu/resctrl/erdt.c
@@ -0,0 +1,255 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Enhanced Resource Director Technology (ERDT)
+ *
+ * Copyright (C) 2026 Intel Corporation
+ *
+ */
+
+#define pr_fmt(fmt)     "resctrl: ERDT: " fmt
+
+#include <linux/acpi.h>
+#include <linux/overflow.h>
+#include <linux/resctrl.h>
+#include <linux/sizes.h>
+
+#include <asm/apic.h>
+
+#include "internal.h"
+
+static LIST_HEAD(domain_info_list);
+
+/* True when the ERDT ACPI table describes at least one domain with at least one CPU. */
+static bool erdt_enabled;
+
+#define ERDT_VALID_REVISION		1
+#define RMDD_FLAG_CPU_L3_DOMAIN		BIT(0)
+#define ERDT_DOMAIN_ID_UNSET		-1
+
+/* Bitmask of valid sub-tables found in the first RMDD, used to ensure all RMDDs match. */
+static u32 valid_subtbl_mask;
+
+/* Domain ID of the first RMDD that established @valid_subtbl_mask, for diagnostics. */
+static u16 first_rmdd_domain_id;
+
+/*
+ * The minimal max_rmid of different domains. Using minimal is to avoid the domain with
+ * small RMID accessing an invalid RMID.
+ */
+static unsigned int erdt_max_rmid;
+
+static void __iomem *erdt_ioremap(resource_size_t base, u32 num_pages, const char *desc)
+{
+	void __iomem *addr;
+	unsigned long size;
+
+	if (check_mul_overflow(num_pages, SZ_4K, &size))
+		return NULL;
+
+	addr = ioremap(base, size);
+	if (!addr)
+		pr_warn(FW_BUG "Failed to map %s at phys addr %pa (size: %u pages)\n",
+			desc, &base, num_pages);
+
+	return addr;
+}
+
+static void erdt_iounmap_domain(struct erdt_domain_info *domain)
+{
+	for (int i = 0; i < ERDT_MMIO_NUM_TYPES; i++) {
+		if (domain->base[i]) {
+			iounmap(domain->base[i]);
+			domain->base[i] = NULL;
+		}
+	}
+}
+
+static void cleanup_one_domain(struct erdt_domain_info *d)
+{
+	erdt_iounmap_domain(d);
+	kfree(d);
+}
+
+/*
+ * Save CACD information for this RMDD:
+ * convert the x2APIC IDs to CPUs and save them in a mask.
+ */
+static __init bool cacd_init(struct acpi_subtbl_hdr_16 *subtbl,
+			     struct erdt_domain_info *domain_info)
+{
+	struct acpi_erdt_cacd *cacd = (struct acpi_erdt_cacd *)subtbl;
+	unsigned int num_ids;
+	int cpu;
+
+	if (cacd->header.length < struct_size(cacd, X2APICIDS, 1)) {
+		pr_warn(FW_BUG "Invalid x2APIC ID CACD table\n");
+		return false;
+	}
+
+	num_ids = (cacd->header.length - sizeof(*cacd)) / sizeof(cacd->X2APICIDS[0]);
+
+	for (unsigned int i = 0; i < num_ids; i++) {
+		cpu = topo_lookup_cpuid(cacd->X2APICIDS[i]);
+		/* Skip CPUs not in the possible mask, e.g. excluded by nr_cpus= */
+		if (cpu < 0)
+			continue;
+
+		cpumask_set_cpu(cpu, &domain_info->cpu_mask);
+	}
+
+	return true;
+}
+
+static inline struct acpi_subtbl_hdr_16 *next_subtbl(struct acpi_subtbl_hdr_16 *subtbl)
+{
+	return (void *)subtbl + subtbl->length;
+}
+
+static inline bool subtbl_valid(void *end, struct acpi_subtbl_hdr_16 *subtbl)
+{
+	/* Ensure the header is within bounds before dereferencing it. */
+	if ((void *)subtbl + sizeof(*subtbl) > end)
+		return false;
+
+	/* A sub-table must be at least as large as its header. */
+	if (subtbl->length < sizeof(*subtbl))
+		return false;
+
+	/* The entire sub-table (including body) must fit within the parent. */
+	if ((void *)subtbl + subtbl->length > end)
+		return false;
+
+	return true;
+}
+
+static __init bool parse_rmdd_table(struct acpi_subtbl_hdr_16 *rmdd_hdr)
+{
+	struct acpi_erdt_rmdd *rmdd = (struct acpi_erdt_rmdd *)rmdd_hdr;
+	struct erdt_domain_info *domain_info;
+	struct acpi_subtbl_hdr_16 *subtbl;
+	u32 subtbl_mask = 0;
+
+	if (rmdd->header.length < sizeof(*rmdd)) {
+		pr_warn(FW_BUG "Invalid RMDD length %u bytes\n", rmdd->header.length);
+		return false;
+	}
+
+	/* Quietly ignore non-CPU-based L3 domains */
+	if (!(rmdd->flags & RMDD_FLAG_CPU_L3_DOMAIN))
+		return true;
+
+	domain_info = kzalloc_obj(*domain_info, GFP_KERNEL);
+	if (!domain_info)
+		return false;
+
+	domain_info->dom_id = ERDT_DOMAIN_ID_UNSET;
+
+	domain_info->base[ERDT_MMIO_RMDD_CREG] =
+		erdt_ioremap(rmdd->creg_base, rmdd->creg_size, "RMDD ctrl base");
+	if (!domain_info->base[ERDT_MMIO_RMDD_CREG])
+		goto cleanup;
+
+	for (subtbl = (void *)rmdd + sizeof(*rmdd);
+	     subtbl_valid((void *)rmdd + rmdd->header.length, subtbl);
+	     subtbl = next_subtbl(subtbl)) {
+		switch (subtbl->type) {
+		/* An RMDD table has one or more CACD sub-table(s) */
+		case ACPI_ERDT_TYPE_CACD:
+			if (!cacd_init(subtbl, domain_info))
+				goto cleanup;
+
+			subtbl_mask |= BIT(ACPI_ERDT_TYPE_CACD);
+			break;
+		default:
+			break;
+		}
+	}
+
+	if (!subtbl_mask)
+		goto cleanup;
+
+	/*
+	 * Require all RMDDs to support same set of sub-tables
+	 */
+	if (!valid_subtbl_mask) {
+		valid_subtbl_mask = subtbl_mask;
+		first_rmdd_domain_id = rmdd->domain_id;
+	} else if (subtbl_mask != valid_subtbl_mask) {
+		pr_warn(FW_BUG "RMDD %u sub-table set does not match the first RMDD %u\n",
+			rmdd->domain_id, first_rmdd_domain_id);
+		goto cleanup;
+	}
+
+	if (!rmdd->max_rmid) {
+		pr_warn(FW_BUG "Unreasonable RMDD max_rmid 0\n");
+		goto cleanup;
+	}
+
+	if (!erdt_max_rmid)
+		erdt_max_rmid = rmdd->max_rmid;
+	else
+		erdt_max_rmid = min(erdt_max_rmid, rmdd->max_rmid);
+
+	list_add(&domain_info->entry, &domain_info_list);
+
+	return true;
+
+cleanup:
+	cleanup_one_domain(domain_info);
+	return false;
+}
+
+void erdt_exit(void)
+{
+	struct erdt_domain_info *d, *tmp;
+
+	list_for_each_entry_safe(d, tmp, &domain_info_list, entry) {
+		list_del(&d->entry);
+		cleanup_one_domain(d);
+	}
+	erdt_enabled = false;
+	valid_subtbl_mask = 0;
+	first_rmdd_domain_id = 0;
+	erdt_max_rmid = 0;
+}
+
+static __init int enumerate_erdt_table(struct acpi_table_header *table_hdr)
+{
+	struct acpi_table_erdt *erdt = (struct acpi_table_erdt *)table_hdr;
+	struct acpi_subtbl_hdr_16 *subtbl;
+
+	if (erdt->header.revision != ERDT_VALID_REVISION) {
+		pr_info("Unsupported table revision %u (expected %u)\n",
+			erdt->header.revision, ERDT_VALID_REVISION);
+		return -EINVAL;
+	}
+
+	if (erdt->header.length < sizeof(*erdt)) {
+		pr_warn(FW_BUG "Invalid table length %u bytes\n", erdt->header.length);
+		return -EINVAL;
+	}
+
+	for (subtbl = (void *)erdt + sizeof(*erdt);
+	     subtbl_valid((void *)erdt + erdt->header.length, subtbl);
+	     subtbl = next_subtbl(subtbl)) {
+		if (subtbl->type == ACPI_ERDT_TYPE_RMDD &&
+		    !parse_rmdd_table(subtbl))
+			goto cleanup;
+	}
+
+	if (list_empty(&domain_info_list))
+		goto cleanup;
+
+	erdt_enabled = true;
+
+	return 0;
+
+cleanup:
+	erdt_exit();
+	return -EINVAL;
+}
+
+__init void erdt_init(void)
+{
+	acpi_table_parse(ACPI_SIG_ERDT, enumerate_erdt_table);
+}
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index e3cfa0c10e92..e2f3b884847c 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -21,6 +21,32 @@
 
 #define RMID_VAL_UNAVAIL		BIT_ULL(62)
 
+/*
+ * Index into erdt_domain_info::base[] for each MMIO region.
+ * @ERDT_MMIO_RMDD_CREG: RMDD control register base address
+ */
+enum erdt_mmio_type {
+	ERDT_MMIO_RMDD_CREG,
+	ERDT_MMIO_LAST = ERDT_MMIO_RMDD_CREG
+};
+
+#define ERDT_MMIO_NUM_TYPES	(ERDT_MMIO_LAST + 1)
+
+/**
+ * struct erdt_domain_info - L3 domain ERDT information
+ * @base:	Array of ioremapped MMIO region base addresses, indexed by ERDT_MMIO_*
+ * @cpu_mask:	CPUs belonging to this resource management domain
+ * @dom_id:	L3 cache ID shared by all CPUs in this domain
+ *		(ERDT_DOMAIN_ID_UNSET if unset)
+ * @entry:	Links into the global domain_info_list
+ */
+struct erdt_domain_info {
+	void __iomem		*base[ERDT_MMIO_NUM_TYPES];
+	struct cpumask		cpu_mask;
+	int			dom_id;
+	struct list_head	entry;
+};
+
 /*
  * With the above fields in use 62 bits remain in MSR_IA32_QM_CTR for
  * data to be returned. The counter width is discovered from the hardware
@@ -253,4 +279,7 @@ static inline void intel_aet_mon_domain_setup(int cpu, int id, struct rdt_resour
 static inline bool intel_handle_aet_option(bool force_off, char *tok) { return false; }
 #endif
 
+void erdt_init(void);
+void erdt_exit(void);
+
 #endif /* _ASM_X86_RESCTRL_INTERNAL_H */
-- 
2.43.0

[PATCH v9 04/12] x86/resctrl: Do not enable ERDT when Sub-NUMA Cluster is enabled

[Chen Yu]

ERDT cannot coexist with SNC(Sub-NUMA-Clustering). The definition of SNC
is to 'divide' an L3 into smaller L3 slices by mapping addresses to
different L3 slices. The current platform with ERDT enabled has several
L3 per socket, and it will not be divided into smaller slices.
Furthermore, ERDT has only L3 scope for the CPU agent, and there are no
node-scope domains.

snc_nodes_per_l3_cache is calculated in rdt_get_l3_mon_config(), which
runs after erdt_init() has parsed the ACPI ERDT table. Introduce
resctrl_arch_update_snc() to do that calculation and call it from
get_rdt_resources() before erdt_init(). Leave ERDT disabled if SNC is
detected and keep using the legacy MSR based CMT interface. Do this check
while parsing the ERDT table so that platforms that do not provide an
ERDT table at all stay silent.

Suggested-by: Tony Luck <tony.luck@intel.com>
Suggested-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Chen Yu <yu.c.chen@intel.com>
Tested-by: Hongyu Ning <hongyu.ning@linux.intel.com>
---
Changes since v8:
- New patch. v8 made it look as though SNC was supported while silently
  depending on it being disabled. Make the restriction explicit by
  leaving ERDT disabled when SNC is enabled, and introduce
  resctrl_arch_update_snc() so that snc_nodes_per_l3_cache is computed
  before erdt_init() parses the ACPI table.
  (Reinette Chatre, sashiko)

 arch/x86/kernel/cpu/resctrl/core.c     |  1 +
 arch/x86/kernel/cpu/resctrl/erdt.c     | 12 ++++++++++++
 arch/x86/kernel/cpu/resctrl/internal.h |  3 +++
 arch/x86/kernel/cpu/resctrl/monitor.c  |  9 ++++++---
 4 files changed, 22 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index 7fadfebb5312..0ea888807bd0 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -1018,6 +1018,7 @@ static __init bool get_rdt_resources(void)
 {
 	bool ret;
 
+	resctrl_arch_update_snc();
 	erdt_init();
 	rdt_alloc_capable = get_rdt_alloc_resources();
 	rdt_mon_capable = get_rdt_mon_resources();
diff --git a/arch/x86/kernel/cpu/resctrl/erdt.c b/arch/x86/kernel/cpu/resctrl/erdt.c
index eb5e262e243a..d94c54617fa1 100644
--- a/arch/x86/kernel/cpu/resctrl/erdt.c
+++ b/arch/x86/kernel/cpu/resctrl/erdt.c
@@ -229,6 +229,18 @@ static __init int enumerate_erdt_table(struct acpi_table_header *table_hdr)
 		return -EINVAL;
 	}
 
+	/*
+	 * ERDT describes monitoring resources per L3 cache domain, while SNC
+	 * splits an L3 cache into NUMA node scoped monitoring domains. The two
+	 * cannot be matched up, so leave ERDT disabled and fall back to the
+	 * MSR based CMT interface. snc_nodes_per_l3_cache has been computed by
+	 * resctrl_arch_update_snc() before erdt_init() is called.
+	 */
+	if (snc_nodes_per_l3_cache > 1) {
+		pr_info("Not supported with Sub-NUMA Cluster enabled, using legacy CMT\n");
+		return -EINVAL;
+	}
+
 	for (subtbl = (void *)erdt + sizeof(*erdt);
 	     subtbl_valid((void *)erdt + erdt->header.length, subtbl);
 	     subtbl = next_subtbl(subtbl)) {
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index e2f3b884847c..34d29f719076 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -177,6 +177,8 @@ static inline struct rdt_hw_resource *resctrl_to_arch_res(struct rdt_resource *r
 
 extern struct rdt_hw_resource rdt_resources_all[];
 
+extern int snc_nodes_per_l3_cache;
+
 void arch_mon_domain_online(struct rdt_resource *r, struct rdt_l3_mon_domain *d);
 
 /* CPUID.(EAX=10H, ECX=ResID=1).EAX */
@@ -251,6 +253,7 @@ union l3_qos_abmc_cfg {
 void rdt_ctrl_update(void *arg);
 
 int rdt_get_l3_mon_config(struct rdt_resource *r);
+__init void resctrl_arch_update_snc(void);
 
 bool rdt_cpu_has(int flag);
 
diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
index 3838e0a13d36..6e46f10d8640 100644
--- a/arch/x86/kernel/cpu/resctrl/monitor.c
+++ b/arch/x86/kernel/cpu/resctrl/monitor.c
@@ -34,7 +34,7 @@ bool rdt_mon_capable;
 
 #define CF(cf)	((unsigned long)(1048576 * (cf) + 0.5))
 
-static int snc_nodes_per_l3_cache = 1;
+int snc_nodes_per_l3_cache = 1;
 
 /*
  * The correction factor table is documented in Documentation/filesystems/resctrl.rst.
@@ -416,6 +416,11 @@ static __init int snc_get_config(void)
 	return ret;
 }
 
+__init void resctrl_arch_update_snc(void)
+{
+	snc_nodes_per_l3_cache = snc_get_config();
+}
+
 int __init rdt_get_l3_mon_config(struct rdt_resource *r)
 {
 	unsigned int mbm_offset = boot_cpu_data.x86_cache_mbm_width_offset;
@@ -423,8 +428,6 @@ int __init rdt_get_l3_mon_config(struct rdt_resource *r)
 	unsigned int threshold;
 	u32 eax, ebx, ecx, edx;
 
-	snc_nodes_per_l3_cache = snc_get_config();
-
 	resctrl_rmid_realloc_limit = boot_cpu_data.x86_cache_size * 1024;
 	hw_res->mon_scale = boot_cpu_data.x86_cache_occ_scale / snc_nodes_per_l3_cache;
 	r->mon.num_rmid = (boot_cpu_data.x86_cache_max_rmid + 1) / snc_nodes_per_l3_cache;
-- 
2.43.0

[PATCH v9 05/12] x86/resctrl: Attach ACPI ERDT information to L3 mon domain on CPU online

[Chen Yu]

In preparation for MMIO-based LLC occupancy monitoring, link the per-domain
ERDT information to the resctrl L3 monitoring domain, so the MMIO read path
can locate a domain's MMIO registers.

ERDT and CPUID enumerate CPU-to-L3-domain membership independently: CPUID leaf 4
describes the L3 cache topology, while the firmware CACD sub-table lists the
CPUs of each ERDT domain. Both views must agree on a CPU's L3 domain for that
CPU to be monitored safely.

When a CPU comes online, validate that firmware and CPUID agree on its L3 domain
and bind the CPU to its ERDT domain before adding it to any resctrl domain.
Fail the CPU online operation on a mismatch for now. Leaving the CPU
online but outside all resctrl domains is doable, but it would require a
change to cpus_ctrl_write() to reject CPUs that are online but not tracked by
any resctrl group. The current approach is stricter, but simpler.

The ID used to match a resctrl monitoring domain against an ERDT domain is
always the L3 cache ID. ERDT is only enabled when Sub-NUMA Clustering is
disabled, so resctrl keeps the L3 monitoring domains at L3 cache scope
instead of node scope.

Suggested-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Chen Yu <yu.c.chen@intel.com>
Tested-by: Hongyu Ning <hongyu.ning@linux.intel.com>
---
Changes since v8:
- Reword the changelog that read like a bug fix, and state that the ID
  used for the match is always the L3 cache ID because ERDT is not
  enabled when SNC is enabled. (Reinette Chatre)
- Rename erdt_cpu_valid() to erdt_try_bind_cpu() because the function
  also binds the CPU's ERDT domain to its L3 domain. (Reinette Chatre)
- Attach the ERDT information to the monitoring domain before the domain
  is published on the RCU protected domain list, so that the domain is
  never visible to the resctrl filesystem in a half initialized state.
  (Reinette Chatre, sashiko)
- Move the added comment above the "All writers take this mutex:"
  sentence that introduces domain_list_lock. (Reinette Chatre)
- Move the binding example into the function comment of
  erdt_try_bind_cpu(), refer to @cpu, and fix the mixed tense.
  (Reinette Chatre)
- Drop the @id parameter of erdt_l3_mon_domain_setup() and use hdr->id.
  (Reinette Chatre)
- Drop the comment that duplicated the function comment and the
  hw_dom->d_info check that cannot trigger on a freshly kzalloc()ed
  domain. (Reinette Chatre)
- Use pr_warn_once() for the messages that can be triggered repeatedly
  through CPU hotplug. (Reinette Chatre)
- Move the new prototypes next to the other erdt_*() prototypes in
  internal.h. (Reinette Chatre)

 arch/x86/kernel/cpu/resctrl/core.c     |  11 +++
 arch/x86/kernel/cpu/resctrl/erdt.c     | 109 +++++++++++++++++++++++++
 arch/x86/kernel/cpu/resctrl/internal.h |   4 +
 3 files changed, 124 insertions(+)

diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index 0ea888807bd0..9a59643f366c 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -33,6 +33,8 @@
  * The rdt_resource's domain list is updated when this happens. Readers of
  * the domain list must either take cpus_read_lock(), or rely on an RCU
  * read-side critical section, to avoid observing concurrent modification.
+ * This mutex also protects the ERDT domain_info_list, which is modified when a
+ * CPU comes online.
  * All writers take this mutex:
  */
 static DEFINE_MUTEX(domain_list_lock);
@@ -558,6 +560,8 @@ static void l3_mon_domain_setup(int cpu, int id, struct rdt_resource *r, struct
 		return;
 	}
 
+	erdt_l3_mon_domain_setup(&d->hdr);
+
 	err = resctrl_online_mon_domain(r, &d->hdr);
 	if (err) {
 		l3_mon_domain_free(hw_dom);
@@ -742,6 +746,13 @@ static int resctrl_arch_online_cpu(unsigned int cpu)
 	struct rdt_resource *r;
 
 	mutex_lock(&domain_list_lock);
+	/* Refuse to online a CPU whose ERDT and CPUID L3 domain views disagree. */
+	if (!erdt_try_bind_cpu(cpu)) {
+		mutex_unlock(&domain_list_lock);
+		clear_closid_rmid(cpu);
+		return -EINVAL;
+	}
+
 	for_each_capable_rdt_resource(r)
 		domain_add_cpu(cpu, r);
 	mutex_unlock(&domain_list_lock);
diff --git a/arch/x86/kernel/cpu/resctrl/erdt.c b/arch/x86/kernel/cpu/resctrl/erdt.c
index d94c54617fa1..d4b630358d6d 100644
--- a/arch/x86/kernel/cpu/resctrl/erdt.c
+++ b/arch/x86/kernel/cpu/resctrl/erdt.c
@@ -199,6 +199,115 @@ static __init bool parse_rmdd_table(struct acpi_subtbl_hdr_16 *rmdd_hdr)
 	return false;
 }
 
+/*
+ * erdt_try_bind_cpu() - Validate @cpu's L3 domain against ERDT and bind the
+ * ERDT domain containing @cpu to that L3 domain.
+ *
+ * Find the erdt_domain_info whose CACD lists @cpu, then bind that ERDT domain
+ * to @cpu's L3 id. A CPU whose L3 id does not match the binding of its ERDT
+ * domain cannot be covered by resctrl.
+ *
+ * For example, the CACD sub-tables report:
+ * domain0: CPU0, CPU2, domain1: CPU1, CPU3
+ * while CPUID/cacheinfo reports the L3 cache is shared by:
+ * id0: CPU0, CPU1, id1: CPU2, CPU3
+ * With the CPUs coming online in order:
+ * - CPU0 binds domain0 to L3 id0.
+ * - CPU1 is in the unbound domain1, but L3 id0 is already bound to domain0,
+ *   so CPU1 is not covered by resctrl.
+ * - CPU2 is in domain0, which is bound to L3 id0 rather than CPU2's L3 id1,
+ *   so CPU2 is not covered by resctrl.
+ * - CPU3 binds domain1 to L3 id1.
+ *
+ * Return: true if @cpu can be covered by resctrl, false otherwise.
+ */
+bool erdt_try_bind_cpu(int cpu)
+{
+	struct erdt_domain_info *d, *cpu_dom = NULL;
+	int dom_id;
+
+	/* Without ERDT there is no firmware topology to disagree with. */
+	if (!erdt_enabled)
+		return true;
+
+	dom_id = get_cpu_cacheinfo_id(cpu, RESCTRL_L3_CACHE);
+	if (dom_id < 0) {
+		pr_warn_once(FW_BUG "Can't find L3 id for CPU:%d\n", cpu);
+		return false;
+	}
+
+	list_for_each_entry(d, &domain_info_list, entry) {
+		if (cpumask_test_cpu(cpu, &d->cpu_mask)) {
+			cpu_dom = d;
+			break;
+		}
+	}
+
+	if (!cpu_dom) {
+		pr_warn_once(FW_BUG "Cannot find the ERDT domain which has CPU%d\n", cpu);
+		return false;
+	}
+
+	/* This ERDT domain is already bound to this CPU's L3 domain. */
+	if (cpu_dom->dom_id == dom_id)
+		return true;
+
+	/*
+	 * This ERDT domain is already bound to a different L3 domain. Rebinding it
+	 * would leave two L3 domains reading the counters of one ERDT domain, so
+	 * skip @cpu instead (CPU2 in the example above).
+	 */
+	if (cpu_dom->dom_id != ERDT_DOMAIN_ID_UNSET) {
+		pr_warn_once(FW_BUG "CPU%d's id=%d not equal to CACD domain(%*pbl) id=%d, skip this CPU\n",
+			     cpu, dom_id, cpumask_pr_args(&cpu_dom->cpu_mask), cpu_dom->dom_id);
+
+		return false;
+	}
+
+	/*
+	 * A possible new binding. Check if another ERDT domain shares the same
+	 * L3 id. If yes, this is a conflict and @cpu should not be considered
+	 * by resctrl (CPU1 in the example above).
+	 */
+	list_for_each_entry(d, &domain_info_list, entry) {
+		if (d == cpu_dom)
+			continue;
+
+		if (d->dom_id == dom_id) {
+			pr_warn_once(FW_BUG "CPU%d's id=%d is already used by CACD domain(%*pbl), skip this CPU\n",
+				     cpu, dom_id, cpumask_pr_args(&d->cpu_mask));
+
+			return false;
+		}
+	}
+
+	/* Eligible new binding, assign the L3 id. */
+	cpu_dom->dom_id = dom_id;
+
+	return true;
+}
+
+/*
+ * Associate ERDT table information with this domain.
+ */
+void erdt_l3_mon_domain_setup(struct rdt_domain_hdr *hdr)
+{
+	struct rdt_hw_l3_mon_domain *hw_dom;
+	struct erdt_domain_info *d;
+
+	if (!erdt_enabled)
+		return;
+
+	hw_dom = resctrl_to_arch_mon_dom(container_of(hdr, struct rdt_l3_mon_domain, hdr));
+
+	list_for_each_entry(d, &domain_info_list, entry) {
+		if (d->dom_id == hdr->id) {
+			hw_dom->d_info = d;
+			return;
+		}
+	}
+}
+
 void erdt_exit(void)
 {
 	struct erdt_domain_info *d, *tmp;
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index 34d29f719076..48aa64c1ff96 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -98,12 +98,14 @@ struct rdt_hw_ctrl_domain {
  * @arch_mbm_states:	Per-event pointer to the MBM event's saved state.
  *			An MBM event's state is an array of struct arch_mbm_state
  *			indexed by RMID on x86.
+ * @d_info:		ERDT table information of this domain
  *
  * Members of this structure are accessed via helpers that provide abstraction.
  */
 struct rdt_hw_l3_mon_domain {
 	struct rdt_l3_mon_domain	d_resctrl;
 	struct arch_mbm_state		*arch_mbm_states[QOS_NUM_L3_MBM_EVENTS];
+	const struct erdt_domain_info	*d_info;
 };
 
 static inline struct rdt_hw_ctrl_domain *resctrl_to_arch_ctrl_dom(struct rdt_ctrl_domain *r)
@@ -284,5 +286,7 @@ static inline bool intel_handle_aet_option(bool force_off, char *tok) { return f
 
 void erdt_init(void);
 void erdt_exit(void);
+bool erdt_try_bind_cpu(int cpu);
+void erdt_l3_mon_domain_setup(struct rdt_domain_hdr *hdr);
 
 #endif /* _ASM_X86_RESCTRL_INTERNAL_H */
-- 
2.43.0

[PATCH v9 06/12] x86,fs/resctrl: Move resctrl_arch_round_mon_val() out of line

[Chen Yu]

resctrl_arch_round_mon_val() is an inline function in the architecture
header. It is only called during resctrl initialization and when user space
writes "max_threshold_occupancy", neither of which is a hot path that
benefits from inlining.

Upcoming ERDT support needs resctrl_arch_round_mon_val() to consult
ERDT-specific state. Keeping it inline would force ERDT prototypes into the
architecture header, fragmenting the ERDT code across files.

Move resctrl_arch_round_mon_val() into monitor.c and declare it in
include/linux/resctrl.h alongside the other resctrl_arch_*() calls. Move the
MPAM stub from arm_mpam.h into mpam_resctrl.c accordingly.

No functional change.

Suggested-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Chen Yu <yu.c.chen@intel.com>
---
Changes since v8:
- New patch. v8 added erdt_get_scale() and erdt_cpu_has() prototypes to
  asm/resctrl.h only to keep resctrl_arch_round_mon_val() inline, which
  fragmented the ERDT code. Move the function out of line instead so the
  ERDT prototypes can stay in the resctrl internal header.
  (Reinette Chatre)

 arch/x86/include/asm/resctrl.h        | 9 ---------
 arch/x86/kernel/cpu/resctrl/monitor.c | 9 +++++++++
 drivers/resctrl/mpam_resctrl.c        | 5 +++++
 include/linux/arm_mpam.h              | 5 -----
 include/linux/resctrl.h               | 1 +
 5 files changed, 15 insertions(+), 14 deletions(-)

diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h
index 8f6edcdcfd87..738de74235bd 100644
--- a/arch/x86/include/asm/resctrl.h
+++ b/arch/x86/include/asm/resctrl.h
@@ -130,15 +130,6 @@ static inline void __resctrl_sched_in(struct task_struct *tsk)
 	}
 }
 
-static inline unsigned int resctrl_arch_round_mon_val(unsigned int val)
-{
-	unsigned int scale = boot_cpu_data.x86_cache_occ_scale;
-
-	/* h/w works in units of "boot_cpu_data.x86_cache_occ_scale" */
-	val /= scale;
-	return val * scale;
-}
-
 static inline void resctrl_arch_set_cpu_default_closid_rmid(int cpu, u32 closid,
 							    u32 rmid)
 {
diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
index 6e46f10d8640..e42ee2769a39 100644
--- a/arch/x86/kernel/cpu/resctrl/monitor.c
+++ b/arch/x86/kernel/cpu/resctrl/monitor.c
@@ -421,6 +421,15 @@ __init void resctrl_arch_update_snc(void)
 	snc_nodes_per_l3_cache = snc_get_config();
 }
 
+unsigned int resctrl_arch_round_mon_val(unsigned int val)
+{
+	unsigned int scale = boot_cpu_data.x86_cache_occ_scale;
+
+	/* h/w works in units of "boot_cpu_data.x86_cache_occ_scale" */
+	val /= scale;
+	return val * scale;
+}
+
 int __init rdt_get_l3_mon_config(struct rdt_resource *r)
 {
 	unsigned int mbm_offset = boot_cpu_data.x86_cache_mbm_width_offset;
diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c
index 9d223057953a..5799c6bd4f98 100644
--- a/drivers/resctrl/mpam_resctrl.c
+++ b/drivers/resctrl/mpam_resctrl.c
@@ -143,6 +143,11 @@ void resctrl_arch_pre_mount(void)
 {
 }
 
+unsigned int resctrl_arch_round_mon_val(unsigned int val)
+{
+	return val;
+}
+
 bool resctrl_arch_get_cdp_enabled(enum resctrl_res_level rid)
 {
 	return mpam_resctrl_controls[rid].cdp_enabled;
diff --git a/include/linux/arm_mpam.h b/include/linux/arm_mpam.h
index f92a36187a52..4c639e9c8e5b 100644
--- a/include/linux/arm_mpam.h
+++ b/include/linux/arm_mpam.h
@@ -76,11 +76,6 @@ static inline void resctrl_arch_disable_mon(void) { }
 static inline void resctrl_arch_enable_alloc(void) { }
 static inline void resctrl_arch_disable_alloc(void) { }
 
-static inline unsigned int resctrl_arch_round_mon_val(unsigned int val)
-{
-	return val;
-}
-
 /**
  * mpam_register_requestor() - Register a requestor with the MPAM driver
  * @partid_max:		The maximum PARTID value the requestor can generate.
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index dd09c2ce9a0f..c99561b19e7a 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -418,6 +418,7 @@ static inline u32 resctrl_get_default_ctrl(struct rdt_resource *r)
 u32 resctrl_arch_get_num_closid(struct rdt_resource *r);
 u32 resctrl_arch_system_num_rmid_idx(void);
 int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid);
+unsigned int resctrl_arch_round_mon_val(unsigned int val);
 
 bool resctrl_enable_mon_event(enum resctrl_event_id eventid, bool any_cpu,
 			      unsigned int binary_bits, void *arch_priv);
-- 
2.43.0

[PATCH v9 07/12] x86/resctrl: Add a global ERDT monitor scale

[Chen Yu]

The limbo handler polls the LLC occupancy of stale RMIDs and frees an RMID
once its occupancy drops below max_threshold_occupancy. User space writes
that threshold in bytes, but the hardware counts occupancy in units of the
occupancy scale, so the threshold is rounded down to a multiple of that
scale. The scale is the number of bytes one counter unit represents, i.e.
the granularity of the occupancy counter.

ERDT may provide per-domain monitoring scale factors, but resctrl rounds
max_threshold_occupancy through a single global
resctrl_rmid_realloc_threshold value that is visible to user space.

Prepare for that by adding erdt_scale/erdt_get_scale() and teaching
resctrl_arch_round_mon_val() to consult the ERDT scale when firmware has
populated one.

When CMRC parsing provides per-domain up_scale values, erdt_scale will be
updated with max(). The reason to use max is because rounding is a floor:
a larger scale yields a slightly lower threshold, i.e. an RMID has to
drop to a slightly lower occupancy before it is reused.

For now erdt_scale is intentionally left at its default value of 0.

Suggested-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Chen Yu <yu.c.chen@intel.com>
---
Changes since v8:
- New patch. erdt_scale and erdt_get_scale() were introduced inside
  "x86/resctrl: Parse ACPI CMRC table" without any mention in its
  changelog. Split them out and describe why a single global scale is
  used. (Reinette Chatre)
- Reword the erdt_scale comment: it is not used by the limbo handler.
  (Reinette Chatre)

 arch/x86/kernel/cpu/resctrl/erdt.c     | 16 ++++++++++++++++
 arch/x86/kernel/cpu/resctrl/internal.h |  1 +
 arch/x86/kernel/cpu/resctrl/monitor.c  |  5 ++++-
 3 files changed, 21 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/cpu/resctrl/erdt.c b/arch/x86/kernel/cpu/resctrl/erdt.c
index d4b630358d6d..09072e478bde 100644
--- a/arch/x86/kernel/cpu/resctrl/erdt.c
+++ b/arch/x86/kernel/cpu/resctrl/erdt.c
@@ -38,6 +38,22 @@ static u16 first_rmdd_domain_id;
  */
 static unsigned int erdt_max_rmid;
 
+/*
+ * Used only to round the global resctrl_rmid_realloc_threshold for the limbo
+ * handler. resctrl_rmid_realloc_threshold is exposed to user space as a single
+ * global value, so ERDT uses one scale derived from the per-domain
+ * cmrc->up_scale values. max() keeps this single threshold conservative across
+ * domains because the rounding is a floor: a larger scale yields a slightly
+ * lower threshold, i.e. an RMID has to drop to a slightly lower occupancy
+ * before it is reused.
+ */
+static unsigned int erdt_scale;
+
+unsigned int erdt_get_scale(void)
+{
+	return erdt_scale;
+}
+
 static void __iomem *erdt_ioremap(resource_size_t base, u32 num_pages, const char *desc)
 {
 	void __iomem *addr;
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index 48aa64c1ff96..75dd4736937d 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -284,6 +284,7 @@ static inline void intel_aet_mon_domain_setup(int cpu, int id, struct rdt_resour
 static inline bool intel_handle_aet_option(bool force_off, char *tok) { return false; }
 #endif
 
+unsigned int erdt_get_scale(void);
 void erdt_init(void);
 void erdt_exit(void);
 bool erdt_try_bind_cpu(int cpu);
diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
index e42ee2769a39..f1a28a3c81bc 100644
--- a/arch/x86/kernel/cpu/resctrl/monitor.c
+++ b/arch/x86/kernel/cpu/resctrl/monitor.c
@@ -423,7 +423,10 @@ __init void resctrl_arch_update_snc(void)
 
 unsigned int resctrl_arch_round_mon_val(unsigned int val)
 {
-	unsigned int scale = boot_cpu_data.x86_cache_occ_scale;
+	unsigned int scale = erdt_get_scale();
+
+	if (!scale)
+		scale = boot_cpu_data.x86_cache_occ_scale;
 
 	/* h/w works in units of "boot_cpu_data.x86_cache_occ_scale" */
 	val /= scale;
-- 
2.43.0

[PATCH v9 08/12] x86/resctrl: Parse ACPI CMRC table

[Chen Yu]

The CMRC (Cache Monitoring Registers for CPU Agents Description) sub-table of
ERDT describes the MMIO registers used to read cache monitoring counters (e.g.
LLC occupancy) for a monitoring domain.

Parse each CMRC sub-table, ioremap its register window, and save a copy of the
CMRC table in the corresponding ERDT domain entry so that monitoring code can
read the counters via MMIO.

Suggested-by: Tony Luck <tony.luck@intel.com>
Signed-off-by: Chen Yu <yu.c.chen@intel.com>
Tested-by: Hongyu Ning <hongyu.ning@linux.intel.com>
---
Changes since v8:
- Split the introduction of erdt_scale/erdt_get_scale() out into
  "x86/resctrl: Add a global ERDT monitor scale". (Reinette Chatre)
- Make cmrc_init() return bool instead of errno values that no caller
  consumed. (Reinette Chatre)
- Use one consistent comment style for the RMDD sub-table cases: a
  single line comment above each "case" label. (Reinette Chatre)

 arch/x86/kernel/cpu/resctrl/erdt.c     | 53 ++++++++++++++++++++++++++
 arch/x86/kernel/cpu/resctrl/internal.h |  6 ++-
 2 files changed, 58 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/cpu/resctrl/erdt.c b/arch/x86/kernel/cpu/resctrl/erdt.c
index 09072e478bde..1e6cfc6dbc38 100644
--- a/arch/x86/kernel/cpu/resctrl/erdt.c
+++ b/arch/x86/kernel/cpu/resctrl/erdt.c
@@ -23,6 +23,7 @@ static LIST_HEAD(domain_info_list);
 static bool erdt_enabled;
 
 #define ERDT_VALID_REVISION		1
+#define CMRC_SUPPORTED_INDEX_FN		1
 #define RMDD_FLAG_CPU_L3_DOMAIN		BIT(0)
 #define ERDT_DOMAIN_ID_UNSET		-1
 
@@ -83,6 +84,7 @@ static void erdt_iounmap_domain(struct erdt_domain_info *domain)
 static void cleanup_one_domain(struct erdt_domain_info *d)
 {
 	erdt_iounmap_domain(d);
+	kfree(d->cmrc);
 	kfree(d);
 }
 
@@ -116,6 +118,47 @@ static __init bool cacd_init(struct acpi_subtbl_hdr_16 *subtbl,
 	return true;
 }
 
+static __init bool cmrc_init(struct acpi_subtbl_hdr_16 *subtbl,
+			     struct erdt_domain_info *domain_info)
+{
+	struct acpi_erdt_cmrc *cmrc = (struct acpi_erdt_cmrc *)subtbl;
+
+	if (cmrc->header.length < sizeof(*cmrc)) {
+		pr_warn(FW_BUG "Truncated CMRC sub-table\n");
+		return false;
+	}
+
+	if (cmrc->index_fn != CMRC_SUPPORTED_INDEX_FN) {
+		pr_info("Unsupported CMRC index function %u\n", cmrc->index_fn);
+		return false;
+	}
+
+	if (!cmrc->clump_size) {
+		pr_warn(FW_BUG "CMRC clump_size is zero\n");
+		return false;
+	}
+
+	/* resctrl scales monitoring values with an unsigned int. */
+	if (cmrc->up_scale > UINT_MAX) {
+		pr_warn(FW_BUG "Insane CMRC up_scale value 0x%llx\n", cmrc->up_scale);
+		return false;
+	}
+
+	domain_info->base[ERDT_MMIO_CMRC_BASE] =
+		erdt_ioremap(cmrc->cmt_reg_base, cmrc->cmt_reg_size, "CMRC base");
+	if (!domain_info->base[ERDT_MMIO_CMRC_BASE])
+		return false;
+
+	domain_info->cmrc = kmemdup(cmrc, cmrc->header.length, GFP_KERNEL);
+	if (!domain_info->cmrc) {
+		iounmap(domain_info->base[ERDT_MMIO_CMRC_BASE]);
+		domain_info->base[ERDT_MMIO_CMRC_BASE] = NULL;
+		return false;
+	}
+
+	return true;
+}
+
 static inline struct acpi_subtbl_hdr_16 *next_subtbl(struct acpi_subtbl_hdr_16 *subtbl)
 {
 	return (void *)subtbl + subtbl->length;
@@ -176,6 +219,16 @@ static __init bool parse_rmdd_table(struct acpi_subtbl_hdr_16 *rmdd_hdr)
 
 			subtbl_mask |= BIT(ACPI_ERDT_TYPE_CACD);
 			break;
+		/* An RMDD table supports at most one CMRC sub-table */
+		case ACPI_ERDT_TYPE_CMRC:
+			if (subtbl_mask & BIT(ACPI_ERDT_TYPE_CMRC))
+				break;
+
+			if (!cmrc_init(subtbl, domain_info))
+				goto cleanup;
+
+			subtbl_mask |= BIT(ACPI_ERDT_TYPE_CMRC);
+			break;
 		default:
 			break;
 		}
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index 75dd4736937d..0de174df26e9 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -24,10 +24,12 @@
 /*
  * Index into erdt_domain_info::base[] for each MMIO region.
  * @ERDT_MMIO_RMDD_CREG: RMDD control register base address
+ * @ERDT_MMIO_CMRC_BASE: CMRC monitoring register base address
  */
 enum erdt_mmio_type {
 	ERDT_MMIO_RMDD_CREG,
-	ERDT_MMIO_LAST = ERDT_MMIO_RMDD_CREG
+	ERDT_MMIO_CMRC_BASE,
+	ERDT_MMIO_LAST = ERDT_MMIO_CMRC_BASE
 };
 
 #define ERDT_MMIO_NUM_TYPES	(ERDT_MMIO_LAST + 1)
@@ -35,6 +37,7 @@ enum erdt_mmio_type {
 /**
  * struct erdt_domain_info - L3 domain ERDT information
  * @base:	Array of ioremapped MMIO region base addresses, indexed by ERDT_MMIO_*
+ * @cmrc:	Copy of the ACPI CMRC sub-table for this domain
  * @cpu_mask:	CPUs belonging to this resource management domain
  * @dom_id:	L3 cache ID shared by all CPUs in this domain
  *		(ERDT_DOMAIN_ID_UNSET if unset)
@@ -42,6 +45,7 @@ enum erdt_mmio_type {
  */
 struct erdt_domain_info {
 	void __iomem		*base[ERDT_MMIO_NUM_TYPES];
+	struct acpi_erdt_cmrc	*cmrc;
 	struct cpumask		cpu_mask;
 	int			dom_id;
 	struct list_head	entry;
-- 
2.43.0

[PATCH v9 09/12] x86/resctrl: Refactor the monitor read function

[Chen Yu]

Split the monitor read helper into an L3 read path and an AET (Intel Application
Energy Telemetry) read path. This makes the two distinct monitoring sources
easier to extend independently and prepares the L3 path for ERDT-based MMIO
reads.

No functional change.

Signed-off-by: Chen Yu <yu.c.chen@intel.com>
Tested-by: Hongyu Ning <hongyu.ning@linux.intel.com>
---
Changes since v8:
- No change.

 arch/x86/kernel/cpu/resctrl/monitor.c | 26 ++++++++++++++++++--------
 1 file changed, 18 insertions(+), 8 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
index f1a28a3c81bc..e66e4c845739 100644
--- a/arch/x86/kernel/cpu/resctrl/monitor.c
+++ b/arch/x86/kernel/cpu/resctrl/monitor.c
@@ -239,9 +239,8 @@ static u64 get_corrected_val(struct rdt_resource *r, struct rdt_l3_mon_domain *d
 	return chunks * hw_res->mon_scale;
 }
 
-int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain_hdr *hdr,
-			   u32 unused, u32 rmid, enum resctrl_event_id eventid,
-			   void *arch_priv, u64 *val, void *ignored)
+static int arch_l3_read_event(struct rdt_resource *r, struct rdt_domain_hdr *hdr,
+			      u32 rmid, enum resctrl_event_id eventid, u64 *val)
 {
 	struct rdt_hw_l3_mon_domain *hw_dom;
 	struct rdt_l3_mon_domain *d;
@@ -251,11 +250,6 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain_hdr *hdr,
 	int cpu;
 	int ret;
 
-	resctrl_arch_rmid_read_context_check();
-
-	if (r->rid == RDT_RESOURCE_PERF_PKG)
-		return intel_aet_read_event(hdr->id, rmid, arch_priv, val);
-
 	if (!domain_header_is_valid(hdr, RESCTRL_MON_DOMAIN, RDT_RESOURCE_L3))
 		return -EINVAL;
 
@@ -281,6 +275,22 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain_hdr *hdr,
 	return ret;
 }
 
+int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain_hdr *hdr,
+			   u32 unused, u32 rmid, enum resctrl_event_id eventid,
+			   void *arch_priv, u64 *val, void *ignored)
+{
+	resctrl_arch_rmid_read_context_check();
+
+	switch (r->rid) {
+	case RDT_RESOURCE_L3:
+		return arch_l3_read_event(r, hdr, rmid, eventid, val);
+	case RDT_RESOURCE_PERF_PKG:
+		return intel_aet_read_event(hdr->id, rmid, arch_priv, val);
+	default:
+		return -EINVAL;
+	}
+}
+
 static int __cntr_id_read(u32 cntr_id, u64 *val)
 {
 	struct msr msr_val = {
-- 
2.43.0

[PATCH v9 10/12] fs/resctrl: Do not invoke smp_processor_id() in preemptible context

[Chen Yu]

From: Tony Luck <tony.luck@intel.com>

An event flagged with mon_evt::any_cpu is read directly in preemptible task
context instead of being dispatched to a CPU of the monitoring domain. Only
Intel AET events set this flag, so __mon_event_count() rejects an any_cpu L3
event with WARN_ON_ONCE(). The L3 read paths can therefore call
smp_processor_id() unconditionally to check that the current CPU belongs to the
domain being read.

MMIO-based LLC occupancy counters can be read from any CPU, so the L3 read
paths have to tolerate an any_cpu event. Move that check into a helper which
returns early for any_cpu events, avoiding smp_processor_id() in preemptible
context.

Signed-off-by: Tony Luck <tony.luck@intel.com>
Signed-off-by: Chen Yu <yu.c.chen@intel.com>
Tested-by: Hongyu Ning <hongyu.ning@linux.intel.com>
---
Changes since v8:
- Rename cpu_on_correct_domain() to cpu_can_read_event() because the
  domain is not always relevant to the answer. (Reinette Chatre)
- Drop "report a wrong CPU via rmid_read::err" from the changelog, it is
  obvious from the diff. (Reinette Chatre)

 fs/resctrl/monitor.c | 47 +++++++++++++++++++++++++++++++-------------
 1 file changed, 33 insertions(+), 14 deletions(-)

diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
index 73413cb128ea..5060640b4f50 100644
--- a/fs/resctrl/monitor.c
+++ b/fs/resctrl/monitor.c
@@ -422,9 +422,36 @@ static void mbm_cntr_free(struct rdt_l3_mon_domain *d, int cntr_id)
 	memset(&d->cntr_cfg[cntr_id], 0, sizeof(*d->cntr_cfg));
 }
 
+/**
+ * cpu_can_read_event() - Check if the current CPU can read the event.
+ * @rr: The rmid_read structure containing event and domain information.
+ *
+ * Context: Preemptible process context when @rr->evt->any_cpu is set.
+ *          Non-migratable process context (via smp_call_on_cpu()) or
+ *          non-preemptible context (via smp_call_function_any()) when
+ *          the event must be read on a specific CPU.
+ * Return: true if the current CPU can read this event, false otherwise.
+ */
+static bool cpu_can_read_event(struct rmid_read *rr)
+{
+	int cpu;
+
+	/* Any CPU is OK for this event */
+	if (rr->evt->any_cpu)
+		return true;
+
+	cpu = smp_processor_id();
+
+	/* Single domain. Must be on a CPU in that domain. */
+	if (rr->hdr)
+		return cpumask_test_cpu(cpu, &rr->hdr->cpu_mask);
+
+	/* Summing domains that share a cache, must be on a CPU for that cache. */
+	return cpumask_test_cpu(cpu, &rr->ci->shared_cpu_map);
+}
+
 static int __l3_mon_event_count(struct rdtgroup *rdtgrp, struct rmid_read *rr)
 {
-	int cpu = smp_processor_id();
 	u32 closid = rdtgrp->closid;
 	u32 rmid = rdtgrp->mon.rmid;
 	struct rdt_l3_mon_domain *d;
@@ -457,11 +484,6 @@ static int __l3_mon_event_count(struct rdtgroup *rdtgrp, struct rmid_read *rr)
 		return 0;
 	}
 
-	/* Reading a single domain, must be on a CPU in that domain. */
-	if (!cpumask_test_cpu(cpu, &d->hdr.cpu_mask)) {
-		rr->err = -EIO;
-		return -EINVAL;
-	}
 	if (rr->is_mbm_cntr)
 		rr->err = resctrl_arch_cntr_read(rr->r, d, closid, rmid, cntr_id,
 						 rr->evt->evtid, &tval);
@@ -479,7 +501,6 @@ static int __l3_mon_event_count(struct rdtgroup *rdtgrp, struct rmid_read *rr)
 
 static int __l3_mon_event_count_sum(struct rdtgroup *rdtgrp, struct rmid_read *rr)
 {
-	int cpu = smp_processor_id();
 	u32 closid = rdtgrp->closid;
 	u32 rmid = rdtgrp->mon.rmid;
 	struct rdt_l3_mon_domain *d;
@@ -497,12 +518,6 @@ static int __l3_mon_event_count_sum(struct rdtgroup *rdtgrp, struct rmid_read *r
 		return -EINVAL;
 	}
 
-	/* Summing domains that share a cache, must be on a CPU for that cache. */
-	if (!cpumask_test_cpu(cpu, &rr->ci->shared_cpu_map)) {
-		rr->err = -EIO;
-		return -EINVAL;
-	}
-
 	/*
 	 * Legacy files must report the sum of an event across all
 	 * domains that share the same L3 cache instance.
@@ -538,7 +553,11 @@ static int __mon_event_count(struct rdtgroup *rdtgrp, struct rmid_read *rr)
 {
 	switch (rr->r->rid) {
 	case RDT_RESOURCE_L3:
-		WARN_ON_ONCE(rr->evt->any_cpu);
+		if (!cpu_can_read_event(rr)) {
+			rr->err = -EIO;
+			return -EINVAL;
+		}
+
 		if (rr->hdr)
 			return __l3_mon_event_count(rdtgrp, rr);
 		else
-- 
2.43.0

[PATCH v9 11/12] x86/resctrl: Introduce erdt_cpu_has() and erdt_support()

[Chen Yu]

Introduce erdt_cpu_has() to verify if a specific RDT feature is backed by an
ERDT table. A feature is only reported as ERDT backed when the CPU enumerates
it via CPUID and firmware describes it in the ERDT table (and its sub-tables).
erdt_cpu_has() also considers userspace input like "rdt=!cmt". Similar to
rdt_cpu_has(), erdt_cpu_has() expects an x86 feature flag as parameter.

Introduce erdt_support() as a helper called by erdt_cpu_has() to check whether
the ERDT firmware supports a given feature.

erdt_support() deliberately claims no feature yet and returns false. Features
should be added to erdt_support() as their MMIO implementations are introduced.

Suggested-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Chen Yu <yu.c.chen@intel.com>
Tested-by: Hongyu Ning <hongyu.ning@linux.intel.com>
---
Changes since v8:
- Factor the "rdt=" option handling out of rdt_cpu_has() into
  apply_rdt_options() and build both rdt_cpu_has() and erdt_cpu_has() on
  top of it instead of duplicating the loop. (Reinette Chatre)
- CPUID stays the architectural enumeration mechanism, so erdt_cpu_has()
  now requires boot_cpu_has(flag) in addition to ERDT firmware support.
  (Reinette Chatre)
- Do not add the erdt_cpu_has()/erdt_get_scale() prototypes to
  asm/resctrl.h, they live in the resctrl internal header now that
  resctrl_arch_round_mon_val() is out of line. (Reinette Chatre)
- Reword the changelog: use imperative mood instead of "is derived
  from", and do not imply that rdt_cpu_has() considers ERDT.
  (Reinette Chatre)

 arch/x86/kernel/cpu/resctrl/core.c     | 13 +++++++++++--
 arch/x86/kernel/cpu/resctrl/erdt.c     |  5 +++++
 arch/x86/kernel/cpu/resctrl/internal.h |  2 ++
 3 files changed, 18 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index 9a59643f366c..f8737d9b2e20 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -872,9 +872,8 @@ static int __init set_rdt_options(char *str)
 }
 __setup("rdt", set_rdt_options);
 
-bool rdt_cpu_has(int flag)
+static bool apply_rdt_options(int flag, bool ret)
 {
-	bool ret = boot_cpu_has(flag);
 	struct rdt_options *o;
 
 	if (!ret)
@@ -892,6 +891,16 @@ bool rdt_cpu_has(int flag)
 	return ret;
 }
 
+bool rdt_cpu_has(int flag)
+{
+	return apply_rdt_options(flag, boot_cpu_has(flag));
+}
+
+bool erdt_cpu_has(int flag)
+{
+	return apply_rdt_options(flag, boot_cpu_has(flag) && erdt_support(flag));
+}
+
 bool resctrl_arch_is_evt_configurable(enum resctrl_event_id evt)
 {
 	if (!rdt_cpu_has(X86_FEATURE_BMEC))
diff --git a/arch/x86/kernel/cpu/resctrl/erdt.c b/arch/x86/kernel/cpu/resctrl/erdt.c
index 1e6cfc6dbc38..110e7b9806b4 100644
--- a/arch/x86/kernel/cpu/resctrl/erdt.c
+++ b/arch/x86/kernel/cpu/resctrl/erdt.c
@@ -50,6 +50,11 @@ static unsigned int erdt_max_rmid;
  */
 static unsigned int erdt_scale;
 
+bool erdt_support(int flag)
+{
+	return false;
+}
+
 unsigned int erdt_get_scale(void)
 {
 	return erdt_scale;
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index 0de174df26e9..88fae7a84fc7 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -288,6 +288,8 @@ static inline void intel_aet_mon_domain_setup(int cpu, int id, struct rdt_resour
 static inline bool intel_handle_aet_option(bool force_off, char *tok) { return false; }
 #endif
 
+bool erdt_cpu_has(int flag);
+bool erdt_support(int flag);
 unsigned int erdt_get_scale(void);
 void erdt_init(void);
 void erdt_exit(void);
-- 
2.43.0

[PATCH v9 12/12] x86/resctrl: Add MMIO-based LLC occupancy monitoring support

[Chen Yu]

Add erdt_mon_read() to read LLC occupancy via MMIO and use it
when the platform supports ERDT. Register the L3 occupancy event
with ERDT enabled when available, falling back to the MSR-based
path otherwise.

Use the CMRC (Cache Monitoring Registers for CPU Agents Description)
ACPI sub-table to read LLC occupancy counters for each RMID via MMIO
when ERDT is enabled.

Although the occupancy counters can now be read from any CPU via
MMIO, keep the per-domain limbo handler. A global handler would
still have to iterate over every domain and would save nr_domains - 1
worker threads, which does not justify maintaining two limbo handler
models.

Suggested-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Chen Yu <yu.c.chen@intel.com>
Tested-by: Hongyu Ning <hongyu.ning@linux.intel.com>
---
Changes since v8:
- Keep rdt_cpu_has(X86_FEATURE_CQM_OCCUP_LLC) as the outer check in
  get_rdt_mon_resources() and only use erdt_cpu_has() to select the MMIO
  read path, so that CPUID remains the enumeration mechanism and this
  patch does not conflict with Tony's CMT enumeration cleanup.
  (Reinette Chatre)
- Drop the snc_nodes_per_l3_cache division from erdt_read_l3_occupancy()
  and erdt_get_scale(). ERDT is not enabled when SNC is enabled, see
  "x86/resctrl: Do not enable ERDT when Sub-NUMA Cluster is enabled".
  (Reinette Chatre, sashiko)
- Rename UNAVAILABLE_COUNTER to CMRC_UNAVAILABLE_COUNTER. (Reinette Chatre)
- Add a function comment for cmrc_index_function_1() that connects it to
  CMRC_SUPPORTED_INDEX_FN, documents the Intel RDT Architecture
  Specification as the origin of the formula, and keeps the formula on a
  single line. (Reinette Chatre)
- Make erdt_get_scale() and erdt_get_max_rmid() return 0 unless
  erdt_cpu_has(X86_FEATURE_CQM_OCCUP_LLC), so the MSR path is used
  whenever the MMIO path is not.
- Fix the stale "h/w works in units of boot_cpu_data.x86_cache_occ_scale"
  comment. (Reinette Chatre)
- Changelog: drop the description of storing the CMRC data, which is
  done by "x86/resctrl: Parse ACPI CMRC table", use imperative mood for
  keeping the per-domain limbo handler, and say that a global handler
  would save nr_domains - 1 worker threads. (Reinette Chatre)

 arch/x86/kernel/cpu/resctrl/core.c     |  4 +-
 arch/x86/kernel/cpu/resctrl/erdt.c     | 77 +++++++++++++++++++++++++-
 arch/x86/kernel/cpu/resctrl/internal.h |  2 +
 arch/x86/kernel/cpu/resctrl/monitor.c  | 17 +++++-
 4 files changed, 96 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index f8737d9b2e20..f71584b10b19 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -989,7 +989,9 @@ static __init bool get_rdt_mon_resources(void)
 	bool ret = false;
 
 	if (rdt_cpu_has(X86_FEATURE_CQM_OCCUP_LLC)) {
-		resctrl_enable_mon_event(QOS_L3_OCCUP_EVENT_ID, false, 0, NULL);
+		resctrl_enable_mon_event(QOS_L3_OCCUP_EVENT_ID,
+					 erdt_cpu_has(X86_FEATURE_CQM_OCCUP_LLC),
+					 0, NULL);
 		ret = true;
 	}
 	if (rdt_cpu_has(X86_FEATURE_CQM_MBM_TOTAL)) {
diff --git a/arch/x86/kernel/cpu/resctrl/erdt.c b/arch/x86/kernel/cpu/resctrl/erdt.c
index 110e7b9806b4..1a9847a11de8 100644
--- a/arch/x86/kernel/cpu/resctrl/erdt.c
+++ b/arch/x86/kernel/cpu/resctrl/erdt.c
@@ -27,6 +27,9 @@ static bool erdt_enabled;
 #define RMDD_FLAG_CPU_L3_DOMAIN		BIT(0)
 #define ERDT_DOMAIN_ID_UNSET		-1
 
+#define CMRC_UNAVAILABLE_COUNTER	BIT_ULL(63)
+#define CMRC_FLAG_UNAVAILABLE_BIT	BIT(0)
+
 /* Bitmask of valid sub-tables found in the first RMDD, used to ensure all RMDDs match. */
 static u32 valid_subtbl_mask;
 
@@ -52,14 +55,84 @@ static unsigned int erdt_scale;
 
 bool erdt_support(int flag)
 {
-	return false;
+	if (!erdt_enabled)
+		return false;
+
+	switch (flag) {
+	case X86_FEATURE_CQM_OCCUP_LLC:
+		return valid_subtbl_mask & BIT(ACPI_ERDT_TYPE_CMRC);
+	default:
+		return false;
+	}
+}
+
+unsigned int erdt_get_max_rmid(void)
+{
+	if (!erdt_cpu_has(X86_FEATURE_CQM_OCCUP_LLC))
+		return 0;
+
+	return erdt_max_rmid;
 }
 
 unsigned int erdt_get_scale(void)
 {
+	if (!erdt_cpu_has(X86_FEATURE_CQM_OCCUP_LLC))
+		return 0;
+
 	return erdt_scale;
 }
 
+/*
+ * Compute the CMRC MMIO offset for CMRC_SUPPORTED_INDEX_FN using the
+ * MMIO_offset_for_RMID# = (RMID / ClumpSize) * Stride + (RMID % ClumpSize) * 8
+ * formula from the Intel RDT Architecture Specification.
+ */
+static u32 cmrc_index_function_1(struct acpi_erdt_cmrc *cmrc, u32 rmid)
+{
+	return (rmid / cmrc->clump_size) * cmrc->clump_stride +
+	       (rmid % cmrc->clump_size) * sizeof(u64);
+}
+
+static int erdt_read_l3_occupancy(const struct erdt_domain_info *d, u32 rmid, u64 *val)
+{
+	struct acpi_erdt_cmrc *cmrc;
+	u64 l3_cmt_count;
+	u32 offset;
+
+	cmrc = d->cmrc;
+	if (!cmrc)
+		return -EIO;
+
+	offset = cmrc_index_function_1(cmrc, rmid);
+	if (offset + sizeof(u64) > (u32)cmrc->cmt_reg_size * SZ_4K)
+		return -EINVAL;
+
+	l3_cmt_count = readq(d->base[ERDT_MMIO_CMRC_BASE] + offset);
+	if ((cmrc->flags & CMRC_FLAG_UNAVAILABLE_BIT) &&
+	    (l3_cmt_count & CMRC_UNAVAILABLE_COUNTER))
+		return -EINVAL;
+
+	*val = l3_cmt_count * cmrc->up_scale;
+
+	return 0;
+}
+
+int erdt_mon_read(struct rdt_domain_hdr *hdr, enum resctrl_event_id evtid, u32 rmid, u64 *val)
+{
+	struct rdt_hw_l3_mon_domain *hw_dom;
+	const struct erdt_domain_info *d;
+
+	hw_dom = resctrl_to_arch_mon_dom(container_of(hdr, struct rdt_l3_mon_domain, hdr));
+	d = hw_dom->d_info;
+	if (!d)
+		return -EIO;
+
+	if (evtid == QOS_L3_OCCUP_EVENT_ID)
+		return erdt_read_l3_occupancy(d, rmid, val);
+
+	return -EIO;
+}
+
 static void __iomem *erdt_ioremap(resource_size_t base, u32 num_pages, const char *desc)
 {
 	void __iomem *addr;
@@ -161,6 +234,8 @@ static __init bool cmrc_init(struct acpi_subtbl_hdr_16 *subtbl,
 		return false;
 	}
 
+	erdt_scale = max(erdt_scale, cmrc->up_scale);
+
 	return true;
 }
 
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index 88fae7a84fc7..3561a7ae8c7e 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -291,6 +291,8 @@ static inline bool intel_handle_aet_option(bool force_off, char *tok) { return f
 bool erdt_cpu_has(int flag);
 bool erdt_support(int flag);
 unsigned int erdt_get_scale(void);
+unsigned int erdt_get_max_rmid(void);
+int erdt_mon_read(struct rdt_domain_hdr *hdr, enum resctrl_event_id evtid, u32 rmid, u64 *val);
 void erdt_init(void);
 void erdt_exit(void);
 bool erdt_try_bind_cpu(int cpu);
diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
index e66e4c845739..93ee638119d3 100644
--- a/arch/x86/kernel/cpu/resctrl/monitor.c
+++ b/arch/x86/kernel/cpu/resctrl/monitor.c
@@ -283,6 +283,10 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain_hdr *hdr,
 
 	switch (r->rid) {
 	case RDT_RESOURCE_L3:
+		if (eventid == QOS_L3_OCCUP_EVENT_ID &&
+		    erdt_cpu_has(X86_FEATURE_CQM_OCCUP_LLC))
+			return erdt_mon_read(hdr, eventid, rmid, val);
+
 		return arch_l3_read_event(r, hdr, rmid, eventid, val);
 	case RDT_RESOURCE_PERF_PKG:
 		return intel_aet_read_event(hdr->id, rmid, arch_priv, val);
@@ -438,7 +442,7 @@ unsigned int resctrl_arch_round_mon_val(unsigned int val)
 	if (!scale)
 		scale = boot_cpu_data.x86_cache_occ_scale;
 
-	/* h/w works in units of "boot_cpu_data.x86_cache_occ_scale" */
+	/* h/w works in units of the occupancy scale  */
 	val /= scale;
 	return val * scale;
 }
@@ -447,12 +451,21 @@ int __init rdt_get_l3_mon_config(struct rdt_resource *r)
 {
 	unsigned int mbm_offset = boot_cpu_data.x86_cache_mbm_width_offset;
 	struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r);
+	unsigned int erdt_max_rmid;
 	unsigned int threshold;
+	int max_rmid;
 	u32 eax, ebx, ecx, edx;
 
+	erdt_max_rmid = erdt_get_max_rmid();
+	if (erdt_max_rmid)
+		max_rmid = min_t(unsigned int, erdt_max_rmid,
+				 boot_cpu_data.x86_cache_max_rmid);
+	else
+		max_rmid = boot_cpu_data.x86_cache_max_rmid;
+
 	resctrl_rmid_realloc_limit = boot_cpu_data.x86_cache_size * 1024;
 	hw_res->mon_scale = boot_cpu_data.x86_cache_occ_scale / snc_nodes_per_l3_cache;
-	r->mon.num_rmid = (boot_cpu_data.x86_cache_max_rmid + 1) / snc_nodes_per_l3_cache;
+	r->mon.num_rmid = (max_rmid + 1) / snc_nodes_per_l3_cache;
 	hw_res->mbm_width = MBM_CNTR_WIDTH_BASE;
 
 	if (mbm_offset > 0 && mbm_offset <= MBM_CNTR_WIDTH_OFFSET_MAX)
-- 
2.43.0