* [PATCH v4 1/3] x86/resctrl: Fix ABMC counter programming
2026-10-09 23:44 [PATCH v4 0/3] x86,fs/resctrl: Keep default MBM mode at boot and fix ABMC programming Babu Moger
@ 2026-10-09 23:44 ` Babu Moger
2026-10-09 23:44 ` [PATCH v4 2/3] fs/resctrl: Assign counters to existing groups when enabling mbm_event Babu Moger
2026-10-09 23:44 ` [PATCH v4 3/3] x86,fs/resctrl: Keep mbm_assign_mode in default mode at boot Babu Moger
2 siblings, 0 replies; 4+ messages in thread
From: Babu Moger @ 2026-10-09 23:44 UTC (permalink / raw)
To: babu.moger, tony.luck, reinette.chatre, bp
Cc: x86, Dave.Martin, james.morse, corbet, skhan, rdunlap, tglx,
mingo, dave.hansen, hpa, kas, rick.p.edgecombe, linux-kernel,
linux-doc, linux-coco, kvm
AMD's Assignable Bandwidth Monitoring Counters (ABMC) are configured via
MSR_IA32_L3_QOS_ABMC_CFG; MSR_IA32_L3_QOS_ABMC_CFG.cntr_id selects the
counter the configuration applies to. The number of counters a platform
supports (the number of possible values written to
MSR_IA32_L3_QOS_ABMC_CFG.cntr_id) is enumerated separately via CPUID.
On platforms that enumerate more than 32 counters, the current 5-bit
encoding truncates the counter ID and misprograms ABMC.
The AMD64 Architecture Programmer's Manual [1], available from [2], has
been updated to widen MSR_IA32_L3_QOS_ABMC_CFG.cntr_id from 5 bits to 12
bits (the published revision 3.41 does not yet reflect this; a future
revision will). The CPUID enumeration reports the maximum counter ID in a
16-bit field and may therefore report more counters than a 12-bit
MSR_IA32_L3_QOS_ABMC_CFG.cntr_id can address.
Widen MSR_IA32_L3_QOS_ABMC_CFG.cntr_id to 12 bits to match the
architecture. Cap the enumerated counter count at BIT(12) so every counter
ID used by resctrl can be written to MSR_IA32_L3_QOS_ABMC_CFG.cntr_id
without truncation.
[1] AMD64 Architecture Programmer's Manual Volume 2: System Programming,
Publication #24593, Revision 3.41, Section 19.3.3.3 "Assignable
Bandwidth Monitoring (ABMC)"
Fixes: 84ecefb76674 ("x86/resctrl: Add data structures and definitions for ABMC assignment")
Signed-off-by: Babu Moger <babu.moger@amd.com>
Cc: stable@vger.kernel.org
Link: https://bugzilla.kernel.org/show_bug.cgi?id=206537 # [2]
---
v4: Changelog update(from Reinette's input).
v3: Dropped the fix for truncation on 32-bit x86.
Removed the change bw_src field(RMID) width to 15 bits.
Added new check to limit the number of counters to 12 bits.
v2: Moved the link tag to the last.
v1: https://lore.kernel.org/lkml/980f39d3a0e0d9f73925e362f835aeef070a1bc5.1784322818.git.babu.moger@amd.com/
---
arch/x86/kernel/cpu/resctrl/internal.h | 4 ++--
arch/x86/kernel/cpu/resctrl/monitor.c | 3 ++-
2 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index e3cfa0c10e92..ffd74a68671b 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -214,8 +214,8 @@ union l3_qos_abmc_cfg {
bw_src :12,
reserved1: 3,
is_clos : 1,
- cntr_id : 5,
- reserved : 9,
+ cntr_id :12,
+ reserved : 2,
cntr_en : 1,
cfg_en : 1;
} split;
diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
index 3838e0a13d36..5d0d3b18f9b8 100644
--- a/arch/x86/kernel/cpu/resctrl/monitor.c
+++ b/arch/x86/kernel/cpu/resctrl/monitor.c
@@ -470,7 +470,8 @@ int __init rdt_get_l3_mon_config(struct rdt_resource *r)
r->mon.mbm_cntr_assignable = true;
r->mon.mbm_cntr_configurable = true;
cpuid_count(0x80000020, 5, &eax, &ebx, &ecx, &edx);
- r->mon.num_mbm_cntrs = (ebx & GENMASK(15, 0)) + 1;
+ /* cntr_id is 12 bits and can only encode 4096 counters. */
+ r->mon.num_mbm_cntrs = min((ebx & GENMASK(15, 0)) + 1, BIT(12));
hw_res->mbm_cntr_assign_enabled = true;
}
--
2.43.0
^ permalink raw reply [flat|nested] 4+ messages in thread* [PATCH v4 2/3] fs/resctrl: Assign counters to existing groups when enabling mbm_event
2026-10-09 23:44 [PATCH v4 0/3] x86,fs/resctrl: Keep default MBM mode at boot and fix ABMC programming Babu Moger
2026-10-09 23:44 ` [PATCH v4 1/3] x86/resctrl: Fix ABMC counter programming Babu Moger
@ 2026-10-09 23:44 ` Babu Moger
2026-10-09 23:44 ` [PATCH v4 3/3] x86,fs/resctrl: Keep mbm_assign_mode in default mode at boot Babu Moger
2 siblings, 0 replies; 4+ messages in thread
From: Babu Moger @ 2026-10-09 23:44 UTC (permalink / raw)
To: babu.moger, tony.luck, reinette.chatre, bp
Cc: x86, Dave.Martin, james.morse, corbet, skhan, rdunlap, tglx,
mingo, dave.hansen, hpa, kas, rick.p.edgecombe, linux-kernel,
linux-doc, linux-coco, kvm
When the user switches counter assignment mode by writing "mbm_event" to
/sys/fs/resctrl/info/L3_MON/mbm_assign_mode, resctrl frees all assignable
counters, resets per-domain RMID state, and enables counter auto-assignment
exposed to user space as mbm_assign_on_mkdir.
Even though counter auto-assignment is enabled, groups that already exist
at the time of the switch, including the default group created at mount,
are not assigned a counter. All MBM events read "Unassigned" until the user
assigns a counter by hand.
Walk every existing CTRL_MON and its MON children after the reset and
assign counters to their MBM events. Enabling "mbm_event" now leaves the
same per-group state that auto-assignment would have produced. There may be
fewer available counters than MBM events across the existing groups; in
that case stop assignment when no counters remain. Events in the remaining
groups read "Unassigned", matching the behavior of creating a group when no
counters are available. Also revise the rdtgroup_assign_cntrs() function
comments to account for the new callers.
Fixes: 8004ea01cf63 ("fs/resctrl: Introduce the interface to switch between monitor modes")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/8cb66e18e32e4087a9712c1e68ee6da614efe244.1784322818.git.babu.moger%40amd.com
Signed-off-by: Babu Moger <babu.moger@amd.com>
Cc: stable@vger.kernel.org
---
v4: Changelog update (Reinette's input).
Function comment update for rdtgroup_assign_cntrs().
v3: Changelog update.
Code comment cleanup.
Added Cc to stable.
Added Reported-by and Closes tags.
v2: New patch.
This patch addresses the Sashiko comment about documentation issue where
counters are not assigned automatically when mode is switched to mbm_event.
https://sashiko.dev/#/patchset/8cb66e18e32e4087a9712c1e68ee6da614efe244.1784322818.git.babu.moger%40amd.com
In fact, it exposed a real issue. When switching to mbm_event mode, existing
monitoring groups should be assigned counters whenever counters are available.
This provides a smooth transition between modes and aligns the behavior with
the existing auto-assignment mechanism.
---
Documentation/filesystems/resctrl.rst | 7 ++--
fs/resctrl/monitor.c | 47 ++++++++++++++++++++++-----
2 files changed, 43 insertions(+), 11 deletions(-)
diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
index b52795e03303..c8507580474a 100644
--- a/Documentation/filesystems/resctrl.rst
+++ b/Documentation/filesystems/resctrl.rst
@@ -370,8 +370,11 @@ with the following files:
of counters available is described in the "num_mbm_cntrs" file. Changing the
mode may cause all counters on the resource to reset.
- Moving to mbm_event counter assignment mode requires users to assign the counters
- to the events. Otherwise, the MBM event counters will return 'Unassigned' when read.
+ Moving to mbm_event counter assignment mode enables "mbm_assign_on_mkdir" and
+ assigns counters to the events of all existing monitoring groups, including
+ the default group, while counters remain available. Consult
+ "mbm_L3_assignments" after switching to "mbm_event" mode for counter
+ assignment states of all monitoring groups.
The mode is beneficial for AMD platforms that support more CTRL_MON
and MON groups than available hardware counters. By default, this
diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
index 04e4ea612b7c..9e705eafe873 100644
--- a/fs/resctrl/monitor.c
+++ b/fs/resctrl/monitor.c
@@ -1302,14 +1302,17 @@ static int rdtgroup_assign_cntr_event(struct rdt_l3_mon_domain *d, struct rdtgro
}
/*
- * rdtgroup_assign_cntrs() - Assign counters to MBM events. Called when
- * a new group is created.
+ * rdtgroup_assign_cntrs() - Assign counters to enabled MBM events of a
+ * resource group
*
* Each group can accommodate two counters per domain: one for the total
- * event and one for the local event. Assignments may fail due to the limited
- * number of counters. However, it is not necessary to fail the group creation
- * and thus no failure is returned. Users have the option to modify the
- * counter assignments after the group has been created.
+ * event and one for the local event. Assignments may fail due to the
+ * limited number of counters.
+ *
+ * Intended to be called in auto assignment scenarios ("mbm_assign_on_mkdir"
+ * enabled in "mbm_event" counter assignment mode) where counter assignment
+ * is not required to succeed. Thus no failure is returned. Users can modify
+ * the counter assignments afterwards if needed.
*/
void rdtgroup_assign_cntrs(struct rdtgroup *rdtgrp)
{
@@ -1328,6 +1331,22 @@ void rdtgroup_assign_cntrs(struct rdtgroup *rdtgrp)
&mon_event_all[QOS_L3_MBM_LOCAL_EVENT_ID]);
}
+/*
+ * resctrl_assign_cntrs_allrdtgrp() - Assign counters to the MBM events of
+ * every existing group.
+ */
+static void resctrl_assign_cntrs_allrdtgrp(void)
+{
+ struct rdtgroup *prgrp, *crgrp;
+
+ list_for_each_entry(prgrp, &rdt_all_groups, rdtgroup_list) {
+ rdtgroup_assign_cntrs(prgrp);
+
+ list_for_each_entry(crgrp, &prgrp->mon.crdtgrp_list, mon.crdtgrp_list)
+ rdtgroup_assign_cntrs(crgrp);
+ }
+}
+
/*
* rdtgroup_free_unassign_cntr() - Unassign and reset the counter ID configuration
* for the event pointed to by @mevt within the domain @d and resctrl group @rdtgrp.
@@ -1601,9 +1620,6 @@ ssize_t resctrl_mbm_assign_mode_write(struct kernfs_open_file *of, char *buf,
(READS_TO_LOCAL_MEM |
READS_TO_LOCAL_S_MEM |
NON_TEMP_WRITE_TO_LOCAL_MEM);
- /* Enable auto assignment when switching to "mbm_event" mode */
- if (enable)
- r->mon.mbm_assign_on_mkdir = true;
/*
* Reset all the non-achitectural RMID state and assignable counters.
*/
@@ -1611,6 +1627,19 @@ ssize_t resctrl_mbm_assign_mode_write(struct kernfs_open_file *of, char *buf,
mbm_cntr_free_all(r, d);
resctrl_reset_rmid_all(r, d);
}
+
+ /*
+ * Counters were freed above, so both new groups (via mkdir) and
+ * the groups that already exist need assignments. Groups created
+ * while in "default" mode have no counter assigned, including the
+ * default group created when resctrl is mounted. Assign counters
+ * to them so that enabling the mode leaves the same assignments
+ * that mkdir would have made.
+ */
+ if (enable) {
+ r->mon.mbm_assign_on_mkdir = true;
+ resctrl_assign_cntrs_allrdtgrp();
+ }
}
out_unlock:
--
2.43.0
^ permalink raw reply [flat|nested] 4+ messages in thread* [PATCH v4 3/3] x86,fs/resctrl: Keep mbm_assign_mode in default mode at boot
2026-10-09 23:44 [PATCH v4 0/3] x86,fs/resctrl: Keep default MBM mode at boot and fix ABMC programming Babu Moger
2026-10-09 23:44 ` [PATCH v4 1/3] x86/resctrl: Fix ABMC counter programming Babu Moger
2026-10-09 23:44 ` [PATCH v4 2/3] fs/resctrl: Assign counters to existing groups when enabling mbm_event Babu Moger
@ 2026-10-09 23:44 ` Babu Moger
2 siblings, 0 replies; 4+ messages in thread
From: Babu Moger @ 2026-10-09 23:44 UTC (permalink / raw)
To: babu.moger, tony.luck, reinette.chatre, bp
Cc: x86, Dave.Martin, james.morse, corbet, skhan, rdunlap, tglx,
mingo, dave.hansen, hpa, kas, rick.p.edgecombe, linux-kernel,
linux-doc, linux-coco, kvm
The mbm_event mode assigns hardware MBM counters to RMID/event pairs.
It is intended for deployments that need to manage counter assignment
on platforms where the number of monitoring groups exceeds the available
hardware counters. Hardware counters are scarce on these platforms, so
mbm_event mode suits workflows that monitor a subset of groups at a time
and rotate assignments as needed.
resctrl enables the mbm_event mode by default on hardware that supports
it. This breaks the pqos tool [1], which assumes the historical default
mode. It creates 16 or more monitoring groups and uses two counters per
group. On a platform with 32 counters per domain, that exhausts the
mbm_event counters pool, causing additional groups to read "Unassigned".
pqos interprets a non-numeric event read as zero bandwidth and reports 0
MB/s for those groups:
$sudo pqos -m all:0-4
CORE IPC MISSES LLC[KB] MBL[MB/s] MBR[MB/s]
0 0.69 16k 192.0 0.0 0.0
1 0.40 3k 0.0 0.0 0.0
2 0.44 1k 64.0 0.0 0.0
3 0.39 1k 64.0 0.0 0.0
4 0.43 1k 0.0 0.0 0.0
Leave mbm_assign_mode in default mode during initialization. Default mode
uses one counter per monitoring group. On existing AMD platforms, the
default active counters pool is 64 and may be larger on newer hardware.
The active counters pool is the number of counters that the hardware can
actively track in default mode.
Deployments within the active counter pool receive accurate bandwidth
measurements in the default mode. Users can create more monitoring groups
(4096) than there are active counters in the pool. Beyond that pool,
readings may be misleading or report as "Unavailable", with no user-visible
indication.
Default mode is still limited once the number of monitoring groups exceeds
that active counters pool. After hardware re-allocates a counter, a read
may return "Unavailable". pqos still treats "Unavailable" as zero.
Successive reads can be a count, "Unavailable", then another count. pqos
treats those jumps as wraparound and can report an inconsistent rate:
$sudo pqos -m all:0-4
CORE IPC MISSES LLC[KB] MBL[MB/s] MBR[MB/s]
0 0.76 60k 32.0 0.0 0.0
1 0.45 1k 32.0 0.0 0.0
2 1.57 107k 64.0 0.0 17592186044184.9
3 1.62 276k 4928.0 3.2 0.5
Users that need stable measurements beyond the active countes pool should
use mbm_event mode and rotate assignments as needed. Enable it with:
$echo mbm_event > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
Fixes: 0f1576e43adc ("x86/resctrl: Configure mbm_event mode if supported")
Closes: https://github.com/intel/intel-cmt-cat/issues/311
Signed-off-by: Babu Moger <babu.moger@amd.com>
Cc: stable@vger.kernel.org
Link: https://github.com/intel/intel-cmt-cat # [1]
---
v4: Introduced mbm_event counter pool and default active counter pool.
Removed duplicate texts in resctrl.rst.
Re-arranged the tag order.
v3: Added pqos output to show the problem.
More changelog to detail the issue.
Updated the resctrl.rst to mention changes.
v2:
Added documentation describing the known issue with the default mode.
Will add cc to stable once we have all the things in order.
Let me know if I missed anything.
v1:
https://lore.kernel.org/lkml/8cb66e18e32e4087a9712c1e68ee6da614efe244.1784322818.git.babu.moger@amd.com/
---
Documentation/filesystems/resctrl.rst | 43 +++++++++++++++++----------
arch/x86/kernel/cpu/resctrl/monitor.c | 1 -
2 files changed, 28 insertions(+), 16 deletions(-)
diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
index c8507580474a..a896bcae3319 100644
--- a/Documentation/filesystems/resctrl.rst
+++ b/Documentation/filesystems/resctrl.rst
@@ -354,8 +354,8 @@ with the following files:
::
# cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
- [mbm_event]
- default
+ [default]
+ mbm_event
"mbm_event":
@@ -376,20 +376,35 @@ with the following files:
"mbm_L3_assignments" after switching to "mbm_event" mode for counter
assignment states of all monitoring groups.
- The mode is beneficial for AMD platforms that support more CTRL_MON
- and MON groups than available hardware counters. By default, this
- feature is enabled on AMD platforms with the ABMC (Assignable Bandwidth
- Monitoring Counters) capability, ensuring counters remain assigned even
- when the corresponding RMID is not actively used by any processor.
+ It is intended for deployments that need to manage counter assignment on
+ platforms where the number of monitoring groups exceeds the available
+ hardware counters. Hardware counters are scarce on these platforms, so
+ mbm_event mode suits workflows that monitor a subset of groups at a time and
+ rotate assignments as needed. The mbm_event mode ensures counters remain
+ assigned even when the corresponding RMID is not actively monitored.
"default":
In default mode, resctrl assumes there is a hardware counter for each
- event within every CTRL_MON and MON group. On AMD platforms, it is
- recommended to use the mbm_event mode, if supported, to prevent reset of MBM
- events between reads resulting from hardware re-allocating counters. This can
- result in misleading values or display "Unavailable" if no counter is assigned
- to the event.
+ event within every CTRL_MON and MON group. This mode is enabled by default.
+
+ On AMD platforms that support more CTRL_MON and MON groups than hardware
+ counters, hardware dynamically shares a smaller pool of counters among
+ RMIDs. One counter from this pool is used per monitoring group and counts
+ every MBM event of that group's RMID. The size of the pool is not
+ enumerated to software (unlike "num_mbm_cntrs"), and "num_rmids" may be
+ much larger. For example, 64 counters in this pool monitor 64 groups.
+ Newer hardware may provide a larger pool.
+
+ While the number of monitoring groups does not exceed that pool, a counter
+ stays attached to each RMID and readings remain accurate. Creating more
+ groups than the pool can cause hardware to re-allocate those counters
+ between successive reads of an event. Bandwidth values may then be
+ misleading, or a read may return "Unavailable" if no counter is allocated
+ to the RMID. There is no user-visible indication when this begins.
+ Users who need stable readings beyond that pool should switch to mbm_event
+ mode, if supported, and assign counters to the groups of interest
+ (rotating assignments as needed).
* To enable "mbm_event" counter assignment mode:
::
@@ -1787,11 +1802,9 @@ View the llc occupancy snapshot::
Examples on working with mbm_assign_mode
========================================
-a. Check if MBM counter assignment mode is supported.
+a. Check if MBM counter assignment mode is supported and enabled.
::
- # mount -t resctrl resctrl /sys/fs/resctrl/
-
# cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
[mbm_event]
default
diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
index 5d0d3b18f9b8..a6a9090c4808 100644
--- a/arch/x86/kernel/cpu/resctrl/monitor.c
+++ b/arch/x86/kernel/cpu/resctrl/monitor.c
@@ -472,7 +472,6 @@ int __init rdt_get_l3_mon_config(struct rdt_resource *r)
cpuid_count(0x80000020, 5, &eax, &ebx, &ecx, &edx);
/* cntr_id is 12 bits and can only encode 4096 counters. */
r->mon.num_mbm_cntrs = min((ebx & GENMASK(15, 0)) + 1, BIT(12));
- hw_res->mbm_cntr_assign_enabled = true;
}
r->mon_capable = true;
--
2.43.0
^ permalink raw reply [flat|nested] 4+ messages in thread