[PATCH v2 0/2] fs/resctrl: Fix two counter ID bugs.

[PATCH v2 0/2] fs/resctrl: Fix two counter ID bugs.

[tizi4590]

Use the domain's counter ID for arm mpam driver, and give a monitor
domain the counters the groups already use

Changes in v2:
- Fix the mpam bug in mpam_resctrl.c.
- Link to v1: https://lore.kernel.org/all/20261106094537.2337188-1-ruanjinjie@huawei.com/

Jinjie Ruan (2):
  arm_mpam: resctrl: Use the domain's counter ID
  fs/resctrl: Give a monitor domain the counters the groups already use

 drivers/resctrl/mpam_internal.h |  15 ---
 drivers/resctrl/mpam_resctrl.c  | 195 ++++++++++++--------------------
 fs/resctrl/internal.h           |   2 +
 fs/resctrl/monitor.c            | 121 +++++++++++++++++++-
 fs/resctrl/rdtgroup.c           |   4 +
 5 files changed, 196 insertions(+), 141 deletions(-)

-- 
2.34.1

[PATCH v2 1/2] arm_mpam: resctrl: Use the domain's counter ID

[tizi4590]

From: Jinjie Ruan <ruanjinjie@huawei.com>

resctrl assigns MBM counters per domain: mbm_L3_assignments takes a
domain ID, a group can have a counter in one domain and none in
another, and resctrl_arch_cntr_read() and resctrl_arch_reset_cntr()
are given the counter to use.

The driver ignores that counter and looks the monitor up in
mbwu_idx_to_mon[], indexed by CLOSID and RMID.
resctrl_arch_config_cntr() fills that array without looking at the
domain, so each group has one monitor for the whole resource.
Whichever assignment is written last selects the monitor the group
reads in every domain.

For example, with the counter ID each group uses for mbm_total_bytes
on each domain and "_" for none:

                                domain 0         domain 1
  mount, mkdir g1               default=0 g1=1   default=0 g1=1
  unassign both on domain 1     default=0 g1=1   default=_ g1=_
  assign g1 on domain 1         default=0 g1=1   default=_ g1=0

g1 is assigned the lowest free counter on domain 1, counter 0. That
assignment is the last one written for g1, so it selects the monitor
g1 reads in every domain. On domain 0, g1 uses counter 1, but reads
the monitor of counter 0:

                      counter whose monitor is read on domain 0
                      default      g1
  expected            0            1
  actual              0            0

That is the default group's monitor.

Translate the counter ID with assigned_counters[], which maps a counter
ID to a monitor of the class and is valid in every domain, and drop
mbwu_idx_to_mon[] with the lookups in __read_mon() and __reset_mon().
With nothing left to record, resctrl_arch_config_cntr() only resets the
monitors of a counter when it is assigned.

Nothing consumes USE_PRE_ALLOCATED, the marker that triggered those
lookups, once they are gone. resctrl only allocates a monitor context
for an MBM event when resctrl_arch_mbm_cntr_assign_enabled() reports
false, and this driver always reports true for the L3, so the MBM
events are always read as assigned counters. Remove the marker, and
the MBM cases of resctrl_arch_mon_ctx_alloc_no_wait() that returned it.

Fixes: a5ff6ef2d73e ("arm_mpam: resctrl: Add resctrl_arch_cntr_read() & resctrl_arch_reset_cntr()")
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
Cc: <stable@vger.kernel.org>
---
 drivers/resctrl/mpam_internal.h |  15 ---
 drivers/resctrl/mpam_resctrl.c  | 195 ++++++++++++--------------------
 2 files changed, 71 insertions(+), 139 deletions(-)

diff --git a/drivers/resctrl/mpam_internal.h b/drivers/resctrl/mpam_internal.h
index 1f86cc158100..e1348d958d05 100644
--- a/drivers/resctrl/mpam_internal.h
+++ b/drivers/resctrl/mpam_internal.h
@@ -30,14 +30,6 @@ struct platform_device;
 #define PACKED_FOR_KUNIT
 #endif
 
-/*
- * This 'mon' values must not alias an actual monitor, so must be larger than
- * U16_MAX, but not be confused with an errno value, so smaller than
- * (u32)-SZ_4K.
- * USE_PRE_ALLOCATED is used to avoid confusion with an actual monitor.
- */
-#define USE_PRE_ALLOCATED	(U16_MAX + 1)
-
 static inline bool mpam_is_enabled(void)
 {
 	return static_branch_likely(&mpam_enabled);
@@ -309,10 +301,6 @@ enum mon_filter_options {
 };
 
 struct mon_cfg {
-	/*
-	 * mon must be large enough to hold out of range values like
-	 * USE_PRE_ALLOCATED
-	 */
 	u32			mon;
 	u8			pmg;
 	bool			match_pmg;
@@ -455,9 +443,6 @@ struct mpam_resctrl_res {
 struct mpam_resctrl_mon {
 	struct mpam_class	*class;
 
-	/* Array of allocated MBWU monitors, indexed by (closid, rmid). */
-	int			*mbwu_idx_to_mon;
-
 	/* Array of assigned MBWU monitors, indexed by resctrl's cntr_id. */
 	int			*assigned_counters;
 };
diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c
index b28c5c04a87c..05c1bf97c43d 100644
--- a/drivers/resctrl/mpam_resctrl.c
+++ b/drivers/resctrl/mpam_resctrl.c
@@ -397,20 +397,13 @@ static int resctrl_arch_mon_ctx_alloc_no_wait(enum resctrl_event_id evtid)
 {
 	struct mpam_resctrl_mon *mon = &mpam_resctrl_counters[evtid];
 
-	if (!mpam_is_enabled())
-		return -EINVAL;
-
-	if (!mon->class)
+	if (!mpam_is_enabled() || !mon->class)
 		return -EINVAL;
 
-	switch (evtid) {
-	case QOS_L3_OCCUP_EVENT_ID:
+	if (evtid == QOS_L3_OCCUP_EVENT_ID) {
 		/* With CDP, one monitor gets used for both code/data reads */
 		return mpam_alloc_csu_mon(mon->class);
-	case QOS_L3_MBM_LOCAL_EVENT_ID:
-	case QOS_L3_MBM_TOTAL_EVENT_ID:
-		return USE_PRE_ALLOCATED;
-	default:
+	} else {
 		return -EOPNOTSUPP;
 	}
 }
@@ -464,9 +457,15 @@ void resctrl_arch_mon_ctx_free(struct rdt_resource *r,
 	resctrl_arch_mon_ctx_free_no_wait(evtid, mon_idx);
 }
 
-static int __read_mon(struct mpam_resctrl_mon *mon, struct mpam_component *mon_comp,
-		      enum mpam_device_features mon_type,
-		      int mon_idx,
+/* The monitor resctrl's counter @cntr_id uses for @cdp_type. */
+static int mon_for_cntr(struct mpam_resctrl_mon *mon, int cntr_id,
+			enum resctrl_conf_type cdp_type)
+{
+	return mon->assigned_counters[resctrl_get_config_index(cntr_id, cdp_type)];
+}
+
+static int __read_mon(struct mpam_component *mon_comp,
+		      enum mpam_device_features mon_type, int mon_idx,
 		      enum resctrl_conf_type cdp_type, u32 closid, u32 rmid, u64 *val)
 {
 	struct mon_cfg cfg;
@@ -477,14 +476,6 @@ static int __read_mon(struct mpam_resctrl_mon *mon, struct mpam_component *mon_c
 	/* Shift closid to account for CDP */
 	closid = resctrl_get_config_index(closid, cdp_type);
 
-	if (mon_idx == USE_PRE_ALLOCATED) {
-		int mbwu_idx = resctrl_arch_rmid_idx_encode(closid, rmid);
-
-		mon_idx = mon->mbwu_idx_to_mon[mbwu_idx];
-		if (mon_idx == -1)
-			return -ENOENT;
-	}
-
 	if (irqs_disabled()) {
 		/* Check if we can access this domain without an IPI */
 		return -EIO;
@@ -500,32 +491,6 @@ static int __read_mon(struct mpam_resctrl_mon *mon, struct mpam_component *mon_c
 	return mpam_msmon_read(mon_comp, &cfg, mon_type, val);
 }
 
-static int read_mon_cdp_safe(struct mpam_resctrl_mon *mon, struct mpam_component *mon_comp,
-			     enum mpam_device_features mon_type,
-			     int mon_idx, u32 closid, u32 rmid, u64 *val)
-{
-	if (cdp_enabled) {
-		u64 code_val = 0, data_val = 0;
-		int err;
-
-		err = __read_mon(mon, mon_comp, mon_type, mon_idx,
-				 CDP_CODE, closid, rmid, &code_val);
-		if (err)
-			return err;
-
-		err = __read_mon(mon, mon_comp, mon_type, mon_idx,
-				 CDP_DATA, closid, rmid, &data_val);
-		if (err)
-			return err;
-
-		*val = code_val + data_val;
-		return 0;
-	}
-
-	return __read_mon(mon, mon_comp, mon_type, mon_idx,
-			  CDP_NONE, closid, rmid, val);
-}
-
 /* MBWU when not in ABMC mode (not supported), and CSU counters. */
 int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain_hdr *hdr,
 			   u32 closid, u32 rmid, enum resctrl_event_id eventid,
@@ -553,13 +518,31 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain_hdr *hdr,
 
 	mon_type = mpam_feat_msmon_csu;
 
-	return read_mon_cdp_safe(mon, mon_comp, mon_type, mon_idx,
-				 closid, rmid, val);
+	if (cdp_enabled) {
+		u64 code_val = 0, data_val = 0;
+		int err;
+
+		err = __read_mon(mon_comp, mon_type, mon_idx,
+				 CDP_CODE, closid, rmid, &code_val);
+		if (err)
+			return err;
+
+		err = __read_mon(mon_comp, mon_type, mon_idx,
+				 CDP_DATA, closid, rmid, &data_val);
+		if (err)
+			return err;
+
+		*val = code_val + data_val;
+		return 0;
+	}
+
+	return __read_mon(mon_comp, mon_type, mon_idx,
+			  CDP_NONE, closid, rmid, val);
 }
 
 /* MBWU counters when in ABMC mode */
 int resctrl_arch_cntr_read(struct rdt_resource *r, struct rdt_l3_mon_domain *d,
-			   u32 closid, u32 rmid, int mon_idx,
+			   u32 closid, u32 rmid, int cntr_id,
 			   enum resctrl_event_id eventid, u64 *val)
 {
 	struct mpam_resctrl_mon *mon = &mpam_resctrl_counters[eventid];
@@ -575,43 +558,38 @@ int resctrl_arch_cntr_read(struct rdt_resource *r, struct rdt_l3_mon_domain *d,
 	l3_dom = container_of(d, struct mpam_resctrl_dom, resctrl_mon_dom);
 	mon_comp = l3_dom->mon_comp[eventid];
 
-	return read_mon_cdp_safe(mon, mon_comp, mpam_feat_msmon_mbwu,
-				 USE_PRE_ALLOCATED, closid, rmid, val);
-}
-
-static void __reset_mon(struct mpam_resctrl_mon *mon, struct mpam_component *mon_comp,
-			int mon_idx,
-			enum resctrl_conf_type cdp_type, u32 closid, u32 rmid)
-{
-	struct mon_cfg cfg = { };
-
-	if (!mpam_is_enabled())
-		return;
+	if (cdp_enabled) {
+		u64 code_val = 0, data_val = 0;
+		int err;
 
-	/* Shift closid to account for CDP */
-	closid = resctrl_get_config_index(closid, cdp_type);
+		err = __read_mon(mon_comp, mpam_feat_msmon_mbwu,
+				 mon_for_cntr(mon, cntr_id, CDP_CODE),
+				 CDP_CODE, closid, rmid, &code_val);
+		if (err)
+			return err;
 
-	if (mon_idx == USE_PRE_ALLOCATED) {
-		int mbwu_idx = resctrl_arch_rmid_idx_encode(closid, rmid);
+		err = __read_mon(mon_comp, mpam_feat_msmon_mbwu,
+				 mon_for_cntr(mon, cntr_id, CDP_DATA),
+				 CDP_DATA, closid, rmid, &data_val);
+		if (err)
+			return err;
 
-		mon_idx = mon->mbwu_idx_to_mon[mbwu_idx];
+		*val = code_val + data_val;
+		return 0;
 	}
 
-	if (mon_idx == -1)
-		return;
-	cfg.mon = mon_idx;
-	mpam_msmon_reset_mbwu(mon_comp, &cfg);
+	return __read_mon(mon_comp, mpam_feat_msmon_mbwu,
+			  mon_for_cntr(mon, cntr_id, CDP_NONE),
+			  CDP_NONE, closid, rmid, val);
 }
 
-static void reset_mon_cdp_safe(struct mpam_resctrl_mon *mon, struct mpam_component *mon_comp,
-			       int mon_idx, u32 closid, u32 rmid)
+/* Reset the monitor resctrl's counter @cntr_id uses for @cdp_type. */
+static void __reset_mon(struct mpam_resctrl_mon *mon, struct mpam_component *mon_comp,
+			int cntr_id, enum resctrl_conf_type cdp_type)
 {
-	if (cdp_enabled) {
-		__reset_mon(mon, mon_comp, mon_idx, CDP_CODE, closid, rmid);
-		__reset_mon(mon, mon_comp, mon_idx, CDP_DATA, closid, rmid);
-	} else {
-		__reset_mon(mon, mon_comp, mon_idx, CDP_NONE, closid, rmid);
-	}
+	struct mon_cfg cfg = { .mon = mon_for_cntr(mon, cntr_id, cdp_type) };
+
+	mpam_msmon_reset_mbwu(mon_comp, &cfg);
 }
 
 /* Reset an assigned counter */
@@ -632,7 +610,12 @@ void resctrl_arch_reset_cntr(struct rdt_resource *r, struct rdt_l3_mon_domain *d
 	l3_dom = container_of(d, struct mpam_resctrl_dom, resctrl_mon_dom);
 	mon_comp = l3_dom->mon_comp[eventid];
 
-	reset_mon_cdp_safe(mon, mon_comp, USE_PRE_ALLOCATED, closid, rmid);
+	if (cdp_enabled) {
+		__reset_mon(mon, mon_comp, cntr_id, CDP_CODE);
+		__reset_mon(mon, mon_comp, cntr_id, CDP_DATA);
+	} else {
+		__reset_mon(mon, mon_comp, cntr_id, CDP_NONE);
+	}
 }
 
 /*
@@ -1170,22 +1153,6 @@ static void mpam_resctrl_pick_counters(void)
 	}
 }
 
-static void __config_cntr(struct mpam_resctrl_mon *mon, u32 cntr_id,
-			  enum resctrl_conf_type cdp_type, u32 closid, u32 rmid,
-			  bool assign)
-{
-	/* Same CDP index remap as closid; maps cntr_id to assigned_counters[]. */
-	u32 mbwu_idx, mon_idx = resctrl_get_config_index(cntr_id, cdp_type);
-
-	closid = resctrl_get_config_index(closid, cdp_type);
-	mbwu_idx = resctrl_arch_rmid_idx_encode(closid, rmid);
-
-	if (assign)
-		mon->mbwu_idx_to_mon[mbwu_idx] = mon->assigned_counters[mon_idx];
-	else
-		mon->mbwu_idx_to_mon[mbwu_idx] = -1;
-}
-
 void resctrl_arch_config_cntr(struct rdt_resource *r, struct rdt_l3_mon_domain *d,
 			      enum resctrl_event_id evtid, u32 rmid, u32 closid,
 			      u32 cntr_id, bool assign)
@@ -1197,19 +1164,13 @@ void resctrl_arch_config_cntr(struct rdt_resource *r, struct rdt_l3_mon_domain *
 		return;
 	}
 
-	if (!mon->mbwu_idx_to_mon || !mon->assigned_counters) {
-		pr_debug("monitor arrays not allocated\n");
+	if (!mon->assigned_counters) {
+		pr_debug("monitor array not allocated\n");
 		return;
 	}
 
-	if (cdp_enabled) {
-		__config_cntr(mon, cntr_id, CDP_CODE, closid, rmid, assign);
-		__config_cntr(mon, cntr_id, CDP_DATA, closid, rmid, assign);
-	} else {
-		__config_cntr(mon, cntr_id, CDP_NONE, closid, rmid, assign);
-	}
-
-	resctrl_arch_reset_cntr(r, d, closid, rmid, cntr_id, QOS_L3_MBM_TOTAL_EVENT_ID);
+	if (assign)
+		resctrl_arch_reset_cntr(r, d, closid, rmid, cntr_id, evtid);
 }
 
 static int mpam_resctrl_control_init(struct mpam_resctrl_res *res)
@@ -1296,25 +1257,16 @@ static int mpam_resctrl_pick_domain_id(int cpu, struct mpam_component *comp)
 static int mpam_resctrl_monitor_init_abmc(struct mpam_resctrl_mon *mon)
 {
 	struct mpam_resctrl_res *res = &mpam_resctrl_controls[RDT_RESOURCE_L3];
-	size_t num_rmid = resctrl_arch_system_num_rmid_idx();
 	struct rdt_resource *l3 = &res->resctrl_res;
 	struct mpam_class *class = mon->class;
 	u16 num_mbwu_mon;
 	int *cntrs;
 
-	int *rmid_array __free(kvfree) = kvmalloc_objs(*rmid_array, num_rmid);
-	if (!rmid_array) {
-		pr_debug("Failed to allocate RMID array\n");
-		return -ENOMEM;
-	}
-	memset(rmid_array, -1, num_rmid * sizeof(*rmid_array));
-
 	num_mbwu_mon = class->props.num_mbwu_mon;
 	cntrs = __alloc_mbwu_array(mon->class, num_mbwu_mon);
 	if (IS_ERR(cntrs))
 		return PTR_ERR(cntrs);
 	mon->assigned_counters = cntrs;
-	mon->mbwu_idx_to_mon = no_free_ptr(rmid_array);
 
 	l3->mon.mbm_cntr_assignable = true;
 	l3->mon.mbm_assign_on_mkdir = true;
@@ -1943,17 +1895,12 @@ static void mpam_resctrl_teardown_mon(struct mpam_resctrl_mon *mon, struct mpam_
 {
 	u32 num_mbwu_mon = class->props.num_mbwu_mon;
 
-	if (!mon->mbwu_idx_to_mon)
+	if (!mon->assigned_counters)
 		return;
 
-	if (mon->assigned_counters) {
-		__free_mbwu_mon(class, mon->assigned_counters, num_mbwu_mon);
-		kvfree(mon->assigned_counters);
-		mon->assigned_counters = NULL;
-	}
-
-	kvfree(mon->mbwu_idx_to_mon);
-	mon->mbwu_idx_to_mon = NULL;
+	__free_mbwu_mon(class, mon->assigned_counters, num_mbwu_mon);
+	kvfree(mon->assigned_counters);
+	mon->assigned_counters = NULL;
 }
 
 void mpam_resctrl_teardown_class(struct mpam_class *class)
-- 
2.34.1

[PATCH v2 2/2] fs/resctrl: Give a monitor domain the counters the groups already use

[tizi4590]

From: Jinjie Ruan <ruanjinjie@huawei.com>

With assignable MBM counters, resctrl can be mounted while all CPUs of
a monitor domain go offline and one of them comes back, either through
CPU hotplug or because hibernation takes the non-boot CPUs offline. The
domain is destroyed and re-created, and the new domain starts with no
counters assigned. Counters are otherwise only assigned when a group is
created, when rdtgroup_assign_cntrs() takes a free counter in each
monitor domain for each MBM event, so no code revisits the groups that
already exist.

Those groups read "Unassigned" on the new domain, on any system with
assignable MBM counters. A counter assigned to them later through
mbm_L3_assignments is the lowest free one in that domain, so the same
group can end up with a different counter ID on different domains. For
example, with the counter ID each group uses for mbm_total_bytes and
"_" for none:

                               domain 0         domain 1
  mount, mkdir g1              default=0 g1=1   default=0 g1=1
  all CPUs of domain 1 offline default=0 g1=1   (destroyed)
  one CPU of domain 1 online   default=0 g1=1   default=_ g1=_
  assign g1 on domain 1        default=0 g1=1   default=_ g1=0

Each domain has its own counters, so g1 can use counter 0 on domain 1
and counter 1 on domain 0 without affecting the other domain, but until
it is assigned on domain 1 it reads Unassigned there.

Assign the counters when the domain comes online, after its state is
set up. Give each enabled MBM event of every group, including the
monitor groups under mon_groups/, the counter ID the group uses in
another domain, and leave the event unassigned when that ID is already
taken on the new domain. Skip groups in pseudo-lock mode, as their RMID
has been freed and they are not monitored.

A group that has no counter for an event in any domain stays unassigned
on the new domain, as it was before. mbm_assign_on_mkdir only applies
when a group is created, and counters that the user removed through
mbm_L3_assignments stay removed.

Fixes: ef712fe97ec5 ("fs/resctrl: Auto assign counters on mkdir and clean up on group removal")
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
Cc: <stable@vger.kernel.org>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Babu Moger <babu.moger@amd.com>
Cc: James Morse <james.morse@arm.com>
Cc: Ben Horgan <ben.horgan@arm.com>
Cc: Dave Martin <Dave.Martin@arm.com>
---
Changes in v2:
- Drop the MPAM part of the changelog, the MPAM driver is fixed by
  "arm_mpam: resctrl: Use the domain's counter ID".
- Link to v1: https://lore.kernel.org/all/20261106094537.2337188-1-ruanjinjie@huawei.com/
---
 fs/resctrl/internal.h |   2 +
 fs/resctrl/monitor.c  | 121 +++++++++++++++++++++++++++++++++++++++++-
 fs/resctrl/rdtgroup.c |   4 ++
 3 files changed, 125 insertions(+), 2 deletions(-)

diff --git a/fs/resctrl/internal.h b/fs/resctrl/internal.h
index 5a3a11a5bd5c..2e092fbd3051 100644
--- a/fs/resctrl/internal.h
+++ b/fs/resctrl/internal.h
@@ -438,6 +438,8 @@ int resctrl_available_mbm_cntrs_show(struct kernfs_open_file *of, struct seq_fil
 
 void rdtgroup_assign_cntrs(struct rdtgroup *rdtgrp);
 
+void resctrl_mbm_cntr_assign_domain(struct rdt_resource *r, struct rdt_l3_mon_domain *d);
+
 void rdtgroup_unassign_cntrs(struct rdtgroup *rdtgrp);
 
 int event_filter_show(struct kernfs_open_file *of, struct seq_file *seq, void *v);
diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
index bb6e6e509f4d..604b42d1133f 100644
--- a/fs/resctrl/monitor.c
+++ b/fs/resctrl/monitor.c
@@ -427,6 +427,52 @@ static int mbm_cntr_get(struct rdt_resource *r, struct rdt_l3_mon_domain *d,
 	return -ENOENT;
 }
 
+/*
+ * mbm_cntr_get_other_domain() - Return the counter ID @rdtgrp uses for @evtid
+ * on a domain other than @d.
+ *
+ * Counters are allocated on every domain together, so a group usually has the
+ * same ID everywhere. The first domain that has a counter ID for the event is
+ * used. @d is skipped in case it is already on the list.
+ *
+ * Context: Called with cpus_read_lock() held.
+ *
+ * Return:
+ * Valid counter ID on success, or -ENOENT on failure.
+ */
+static int mbm_cntr_get_other_domain(struct rdt_resource *r,
+				     struct rdt_l3_mon_domain *d,
+				     struct rdtgroup *rdtgrp,
+				     enum resctrl_event_id evtid)
+{
+	struct rdt_l3_mon_domain *dom;
+
+	lockdep_assert_cpus_held();
+
+	list_for_each_entry_rcu(dom, &r->mon_domains, hdr.list, lockdep_is_cpus_held()) {
+		int cntr_id;
+
+		if (dom == d)
+			continue;
+
+		cntr_id = mbm_cntr_get(r, dom, rdtgrp, evtid);
+		if (cntr_id >= 0)
+			return cntr_id;
+	}
+
+	return -ENOENT;
+}
+
+/*
+ * mbm_cntr_claim() - Mark the counter @cntr_id in domain @d as used by @rdtgrp.
+ */
+static void mbm_cntr_claim(struct rdt_l3_mon_domain *d, int cntr_id,
+			   struct rdtgroup *rdtgrp, enum resctrl_event_id evtid)
+{
+	d->cntr_cfg[cntr_id].rdtgrp = rdtgrp;
+	d->cntr_cfg[cntr_id].evtid = evtid;
+}
+
 /*
  * mbm_cntr_alloc() - Initialize and return a new counter ID in the domain @d.
  * Caller must ensure that the specified event is not assigned already.
@@ -441,8 +487,7 @@ static int mbm_cntr_alloc(struct rdt_resource *r, struct rdt_l3_mon_domain *d,
 
 	for (cntr_id = 0; cntr_id < r->mon.num_mbm_cntrs; cntr_id++) {
 		if (!d->cntr_cfg[cntr_id].rdtgrp) {
-			d->cntr_cfg[cntr_id].rdtgrp = rdtgrp;
-			d->cntr_cfg[cntr_id].evtid = evtid;
+			mbm_cntr_claim(d, cntr_id, rdtgrp, evtid);
 			return cntr_id;
 		}
 	}
@@ -1423,6 +1468,78 @@ void rdtgroup_assign_cntrs(struct rdtgroup *rdtgrp)
 					   &mon_event_all[QOS_L3_MBM_LOCAL_EVENT_ID]);
 }
 
+/*
+ * resctrl_mbm_cntr_assign_group() - Give @rdtgrp, on @d, the counter IDs it
+ * uses on the other domains.
+ */
+static void resctrl_mbm_cntr_assign_group(struct rdt_resource *r,
+					  struct rdt_l3_mon_domain *d,
+					  struct rdtgroup *rdtgrp)
+{
+	struct mon_evt *mevt;
+
+	/* Pseudo-locked groups have no RMID and are not monitored. */
+	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP ||
+	    rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED)
+		return;
+
+	for_each_mon_event(mevt) {
+		int cntr_id;
+
+		if (mevt->rid != r->rid || !mevt->enabled ||
+		    !resctrl_is_mbm_event(mevt->evtid))
+			continue;
+
+		/* A group without a counter on any domain stays unassigned. */
+		cntr_id = mbm_cntr_get_other_domain(r, d, rdtgrp, mevt->evtid);
+		if (cntr_id < 0)
+			continue;
+
+		/* Keep the group's ID; do not allocate a different one. */
+		if (d->cntr_cfg[cntr_id].rdtgrp) {
+			pr_warn_ratelimited("Counter %d for %s is in use in domain %d\n",
+					    cntr_id, mevt->name, d->hdr.id);
+			continue;
+		}
+
+		mbm_cntr_claim(d, cntr_id, rdtgrp, mevt->evtid);
+		rdtgroup_assign_cntr(r, d, mevt->evtid, rdtgrp->mon.rmid,
+				     rdtgrp->closid, cntr_id, true);
+	}
+}
+
+/**
+ * resctrl_mbm_cntr_assign_domain() - Assign counters to the existing groups on
+ *				      a monitor domain that has just come online.
+ * @r:	The resource the domain belongs to.
+ * @d:	The domain that has just come online.
+ *
+ * A new domain has no counters assigned. Give each group, including the
+ * monitor groups below a control group, the counter IDs it uses in the other
+ * domains. A group that has no counter for an event in any domain stays
+ * unassigned: mbm_assign_on_mkdir only applies when a group is created, and
+ * counters that the user removed through mbm_L3_assignments stay removed.
+ *
+ * Context: Called with rdtgroup_mutex and cpus_read_lock() held while resctrl
+ * is mounted.
+ */
+void resctrl_mbm_cntr_assign_domain(struct rdt_resource *r, struct rdt_l3_mon_domain *d)
+{
+	struct rdtgroup *prgrp, *crgrp;
+
+	lockdep_assert_held(&rdtgroup_mutex);
+
+	if (!resctrl_arch_mbm_cntr_assign_enabled(r))
+		return;
+
+	list_for_each_entry(prgrp, &rdt_all_groups, rdtgroup_list) {
+		resctrl_mbm_cntr_assign_group(r, d, prgrp);
+
+		list_for_each_entry(crgrp, &prgrp->mon.crdtgrp_list, mon.crdtgrp_list)
+			resctrl_mbm_cntr_assign_group(r, d, crgrp);
+	}
+}
+
 /*
  * rdtgroup_free_unassign_cntr() - Unassign and reset the counter ID configuration
  * for the event pointed to by @mevt within the domain @d and resctrl group @rdtgrp.
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 40b8f34c3d56..abe5244da379 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -4744,6 +4744,10 @@ int resctrl_online_mon_domain(struct rdt_resource *r, struct rdt_domain_hdr *hdr
 	if (err)
 		goto out_unlock;
 
+	/* The new domain has no counters for the existing groups yet. */
+	if (resctrl_mounted)
+		resctrl_mbm_cntr_assign_domain(r, d);
+
 	if (resctrl_is_mbm_enabled()) {
 		INIT_DELAYED_WORK(&d->mbm_over, mbm_handle_overflow);
 		mbm_setup_overflow_handler(d, MBM_OVERFLOW_INTERVAL,
-- 
2.34.1

Re: [PATCH v2 2/2] fs/resctrl: Give a monitor domain the counters the groups already use

[Ben Horgan]

Hi Jinjie,

On 13/11/2026 08:35, tizi4590 wrote:
> From: Jinjie Ruan <ruanjinjie@huawei.com>
> 
> With assignable MBM counters, resctrl can be mounted while all CPUs of
> a monitor domain go offline and one of them comes back, either through
> CPU hotplug or because hibernation takes the non-boot CPUs offline. The
> domain is destroyed and re-created, and the new domain starts with no
> counters assigned. Counters are otherwise only assigned when a group is
> created, when rdtgroup_assign_cntrs() takes a free counter in each
> monitor domain for each MBM event, so no code revisits the groups that
> already exist.
> 
> Those groups read "Unassigned" on the new domain, on any system with
> assignable MBM counters. A counter assigned to them later through
> mbm_L3_assignments is the lowest free one in that domain, so the same
> group can end up with a different counter ID on different domains. For
> example, with the counter ID each group uses for mbm_total_bytes and
> "_" for none:
> 
>                                domain 0         domain 1
>   mount, mkdir g1              default=0 g1=1   default=0 g1=1
>   all CPUs of domain 1 offline default=0 g1=1   (destroyed)
>   one CPU of domain 1 online   default=0 g1=1   default=_ g1=_
>   assign g1 on domain 1        default=0 g1=1   default=_ g1=0
> 
> Each domain has its own counters, so g1 can use counter 0 on domain 1
> and counter 1 on domain 0 without affecting the other domain, but until
> it is assigned on domain 1 it reads Unassigned there.

I brought this up before and it was assessed that no change was required:
https://lore.kernel.org/lkml/171f92a3-fa2e-44fb-b0d1-1250c72e31d1@amd.com/

Do you have any particular reason for wanting this behaviour?

Thanks,

Ben

> 
> Assign the counters when the domain comes online, after its state is
> set up. Give each enabled MBM event of every group, including the
> monitor groups under mon_groups/, the counter ID the group uses in
> another domain, and leave the event unassigned when that ID is already
> taken on the new domain. Skip groups in pseudo-lock mode, as their RMID
> has been freed and they are not monitored.
> 
> A group that has no counter for an event in any domain stays unassigned
> on the new domain, as it was before. mbm_assign_on_mkdir only applies
> when a group is created, and counters that the user removed through
> mbm_L3_assignments stay removed.
> 
> Fixes: ef712fe97ec5 ("fs/resctrl: Auto assign counters on mkdir and clean up on group removal")
> Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
> Cc: <stable@vger.kernel.org>
> Cc: Tony Luck <tony.luck@intel.com>
> Cc: Reinette Chatre <reinette.chatre@intel.com>
> Cc: Babu Moger <babu.moger@amd.com>
> Cc: James Morse <james.morse@arm.com>
> Cc: Ben Horgan <ben.horgan@arm.com>
> Cc: Dave Martin <Dave.Martin@arm.com>
> ---
> Changes in v2:
> - Drop the MPAM part of the changelog, the MPAM driver is fixed by
>   "arm_mpam: resctrl: Use the domain's counter ID".
> - Link to v1: https://lore.kernel.org/all/20261106094537.2337188-1-ruanjinjie@huawei.com/