* [PATCH v2 1/3] x86/resctrl: Fix ABMC counter programming for extended counter ranges
2026-09-04 18:06 [PATCH v2 0/3] x86/resctrl: Keep default MBM mode at boot and fix ABMC Babu Moger
@ 2026-09-04 18:06 ` Babu Moger
2026-09-11 22:03 ` Reinette Chatre
2026-09-04 18:06 ` [PATCH v2 2/3] fs/resctrl: Assign counters to existing groups when enabling mbm_event Babu Moger
2026-09-04 18:06 ` [PATCH v2 3/3] x86/resctrl: Keep mbm_assign_mode in default mode at boot Babu Moger
2 siblings, 1 reply; 7+ messages in thread
From: Babu Moger @ 2026-09-04 18:06 UTC (permalink / raw)
To: babu.moger, tony.luck, reinette.chatre, bp
Cc: x86, Dave.Martin, james.morse, corbet, skhan, tglx, mingo,
dave.hansen, hpa, linux-kernel, linux-doc, eranian, peternewman
Memory Bandwidth Monitoring (MBM) can report incorrect values when ABMC is
enabled on systems supporting more than 32 ABMC counters. As the number of
active monitoring groups increases beyond the range supported by the
existing counter ID encoding, programming an ABMC counter may inadvertently
affect a different counter, resulting in unexpected counter resets and
abnormally large MBM readings.
The issue originates from the ABMC counter programming interface in the
L3_QOS_ABMC_CFG MSR. The counter ID field is currently defined as 5 bits,
which limits the addressable counter range to 32 counters. On systems
implementing more than 32 ABMC counters, counter IDs above 31 cannot be
encoded correctly. Consequently, programming a counter ID beyond the
supported range may target an unintended counter and reset bandwidth
statistics associated with another monitoring group.
While updating this logic, it was also observed that the bw_src field,
which encodes the RMID, is currently at its 12-bit limit with support for
4096 RMIDs. This field also needs to be updated for future expansion.
Also found one more pre-existing issue. This union structure can truncate
data on 32-bit x86 systems when unsigned long is used.
Fix the issues with the following changes:
1. Update the cntr_id field handling to support the full hardware ABMC
counter range and ensure that counter programming does not interfere with
unrelated counters.
2. Expand the bw_src field to 15 bits.
3. Change "unsigned long" to u64 to fix truncation on 32-bit x86.
The AMD64 Architecture Programmer's Manual [1] available at [2] will be
updated accordingly in a future revision to document the expanded cntr_id
and bw_src field definitions.
[1] AMD64 Architecture Programmer's Manual Volume 2: System Programming,
Publication #24593, Revision 3.41, Section 19.3.3.3 "Assignable
Bandwidth Monitoring (ABMC)"
Fixes: 84ecefb76674 ("x86/resctrl: Add data structures and definitions for ABMC assignment")
Signed-off-by: Babu Moger <babu.moger@amd.com>
Link: https://bugzilla.kernel.org/show_bug.cgi?id=206537 # [2]
---
v2: Moved the link tag to the last.
v1: https://lore.kernel.org/lkml/980f39d3a0e0d9f73925e362f835aeef070a1bc5.1784322818.git.babu.moger@amd.com/
---
arch/x86/kernel/cpu/resctrl/internal.h | 18 ++++++++----------
1 file changed, 8 insertions(+), 10 deletions(-)
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index e3cfa0c10e92..b3d780eda8a7 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -192,7 +192,6 @@ union cpuid_0x10_x_edx {
* @bw_type : Event configuration that represents the memory
* transactions being tracked by the @cntr_id.
* @bw_src : Bandwidth source (RMID or CLOSID).
- * @reserved1 : Reserved.
* @is_clos : @bw_src field is a CLOSID (not an RMID).
* @cntr_id : Counter identifier.
* @reserved : Reserved.
@@ -210,16 +209,15 @@ union cpuid_0x10_x_edx {
*/
union l3_qos_abmc_cfg {
struct {
- unsigned long bw_type :32,
- bw_src :12,
- reserved1: 3,
- is_clos : 1,
- cntr_id : 5,
- reserved : 9,
- cntr_en : 1,
- cfg_en : 1;
+ u64 bw_type :32,
+ bw_src :15,
+ is_clos : 1,
+ cntr_id :12,
+ reserved : 2,
+ cntr_en : 1,
+ cfg_en : 1;
} split;
- unsigned long full;
+ u64 full;
};
void rdt_ctrl_update(void *arg);
--
2.43.0
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH v2 1/3] x86/resctrl: Fix ABMC counter programming for extended counter ranges
2026-09-04 18:06 ` [PATCH v2 1/3] x86/resctrl: Fix ABMC counter programming for extended counter ranges Babu Moger
@ 2026-09-11 22:03 ` Reinette Chatre
0 siblings, 0 replies; 7+ messages in thread
From: Reinette Chatre @ 2026-09-11 22:03 UTC (permalink / raw)
To: Babu Moger, tony.luck, bp
Cc: x86, Dave.Martin, james.morse, corbet, skhan, tglx, mingo,
dave.hansen, hpa, linux-kernel, linux-doc, eranian, peternewman
Hi Babu,
What does the "for extended counter ranges" in subject refer to? As I understand
"extended events" is another term for ABMC so this seems redundant?
"counter ranges" also just seems to refer to one "part" of this patch so perhaps
it could just be:
x86/resctrl: Fix ABMC counter programming
On 9/4/26 11:06 AM, Babu Moger wrote:
> Memory Bandwidth Monitoring (MBM) can report incorrect values when ABMC is
> enabled on systems supporting more than 32 ABMC counters. As the number of
> active monitoring groups increases beyond the range supported by the
> existing counter ID encoding, programming an ABMC counter may inadvertently
> affect a different counter, resulting in unexpected counter resets and
> abnormally large MBM readings.
>
> The issue originates from the ABMC counter programming interface in the
> L3_QOS_ABMC_CFG MSR. The counter ID field is currently defined as 5 bits,
> which limits the addressable counter range to 32 counters. On systems
> implementing more than 32 ABMC counters, counter IDs above 31 cannot be
> encoded correctly. Consequently, programming a counter ID beyond the
> supported range may target an unintended counter and reset bandwidth
> statistics associated with another monitoring group.
>
> While updating this logic, it was also observed that the bw_src field,
> which encodes the RMID, is currently at its 12-bit limit with support for
> 4096 RMIDs. This field also needs to be updated for future expansion.
>
> Also found one more pre-existing issue. This union structure can truncate
> data on 32-bit x86 systems when unsigned long is used.
>
> Fix the issues with the following changes:
>
> 1. Update the cntr_id field handling to support the full hardware ABMC
> counter range and ensure that counter programming does not interfere with
> unrelated counters.
Sashiko's assessment that this statement is not accurate looks correct to me.
Looks like the enumeration needs a check to limit the number of supported
counters if the hardware supports more than what can be configured? It seems
awkward that such hardware could exist and unclear why the spec has a mismatch in
the number of bits here. Even if cntr_id expands further to use the remaining
reserved bits it would still not be sufficient to configure all the numbers
that hardware may claim to support. Are there perhaps more field width changes
in this upcoming spec update?
>
> 2. Expand the bw_src field to 15 bits.
Do the comments describing the RMID field width when reading the monitoring
data (__cntr_id_read() and __rmid_read_phys()) need an update also?
> 3. Change "unsigned long" to u64 to fix truncation on 32-bit x86.
Sashiko found that this change by itself is not sufficient to address issues
with 32-bit. What do you think of dropping this part of the patch and instead
adding your support to:
https://lore.kernel.org/lkml/20260831174421.13921-22-tony.luck@intel.com/ ?
>
> The AMD64 Architecture Programmer's Manual [1] available at [2] will be
> updated accordingly in a future revision to document the expanded cntr_id
> and bw_src field definitions.
This changelog uses a lot of text to describe one of these spec updates and then,
seemingly as an afterthought, describe two more changes in a way that hints that
these should be separate patches.
Could the changelog be simplified (after dropping the 64-bit change) to something
like:
AMD's Assignable Bandwidth Monitoring Counters (ABMC) are configured via
MSR_IA32_L3_QOS_ABMC_CFG. The architecture [1] received an update that
increases the width of two of the MSR's fields:
1. The counter ID (represented by l3_qos_abmc_cfg.split.cntr_id) increases
from 5 to 12 bits.
2. The bandwidth source (represented by l3_qos_abmc_cfg.split.bw_src), used
for the RMID, increases from 12 to 15 bits.
Use the new field widths. The number of supported counters and RMID are
enumerated separately. Designate this update as a fix to original enabling
to avoid misconfigurations resulting from truncating the counter ID and RMID
on hardware that support a large number of these IDs.
The changelog is just a suggestion based on its current form - other potential changes
like the comments and enumeration checks are not captured by it, but should be if/when
they are added.
>
> [1] AMD64 Architecture Programmer's Manual Volume 2: System Programming,
> Publication #24593, Revision 3.41, Section 19.3.3.3 "Assignable
> Bandwidth Monitoring (ABMC)"
>
> Fixes: 84ecefb76674 ("x86/resctrl: Add data structures and definitions for ABMC assignment")
Is this a stable candidate?
> Signed-off-by: Babu Moger <babu.moger@amd.com>
> Link: https://bugzilla.kernel.org/show_bug.cgi?id=206537 # [2]
> ---
Reinette
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH v2 2/3] fs/resctrl: Assign counters to existing groups when enabling mbm_event
2026-09-04 18:06 [PATCH v2 0/3] x86/resctrl: Keep default MBM mode at boot and fix ABMC Babu Moger
2026-09-04 18:06 ` [PATCH v2 1/3] x86/resctrl: Fix ABMC counter programming for extended counter ranges Babu Moger
@ 2026-09-04 18:06 ` Babu Moger
2026-09-11 22:12 ` Reinette Chatre
2026-09-04 18:06 ` [PATCH v2 3/3] x86/resctrl: Keep mbm_assign_mode in default mode at boot Babu Moger
2 siblings, 1 reply; 7+ messages in thread
From: Babu Moger @ 2026-09-04 18:06 UTC (permalink / raw)
To: babu.moger, tony.luck, reinette.chatre, bp
Cc: x86, Dave.Martin, james.morse, corbet, skhan, tglx, mingo,
dave.hansen, hpa, linux-kernel, linux-doc, eranian, peternewman
resctrl_mbm_assign_mode_write() frees all counters and sets
mbm_assign_on_mkdir for subsequent mkdir, but does not assign counters to
groups that already exist, including the default group created at mount.
Those events then read "Unassigned" until the user assigns counters by
hand.
Enable mbm_assign_on_mkdir and assign counters to existing CTRL_MON and MON
groups with resctrl_assign_cntrs_allrdtgrp() so the switch matches mkdir
auto-assignment. Groups left without a counter still read "Unassigned".
Update Documentation/filesystems/resctrl.rst to describe this.
Fixes: 8004ea01cf63 ("fs/resctrl: Introduce the interface to switch between monitor modes")
Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v2: New patch.
This patch addresses the Sashiko comment about documentation issue where
counters are not assigned automatically when mode is switched to mbm_event.
https://sashiko.dev/#/patchset/8cb66e18e32e4087a9712c1e68ee6da614efe244.1784322818.git.babu.moger%40amd.com
In fact, it exposed a real issue. When switching to mbm_event mode, existing
monitoring groups should be assigned counters whenever counters are available.
This provides a smooth transition between modes and aligns the behavior with
the existing auto-assignment mechanism.
---
Documentation/filesystems/resctrl.rst | 7 +++++--
fs/resctrl/monitor.c | 30 ++++++++++++++++++++++++---
2 files changed, 32 insertions(+), 5 deletions(-)
diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
index e4b66af55ffb..79feeb1dc296 100644
--- a/Documentation/filesystems/resctrl.rst
+++ b/Documentation/filesystems/resctrl.rst
@@ -371,8 +371,11 @@ with the following files:
of counters available is described in the "num_mbm_cntrs" file. Changing the
mode may cause all counters on the resource to reset.
- Moving to mbm_event counter assignment mode requires users to assign the counters
- to the events. Otherwise, the MBM event counters will return 'Unassigned' when read.
+ Moving to mbm_event counter assignment mode enables "mbm_assign_on_mkdir" and
+ assigns counters to the events of all existing groups, including the default
+ group, for as long as counters remain available. Events left without a counter
+ will return 'Unassigned' when read until the user assigns one using
+ "mbm_L3_assignments".
The mode is beneficial for AMD platforms that support more CTRL_MON
and MON groups than available hardware counters. By default, this
diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
index 73413cb128ea..61463741b91b 100644
--- a/fs/resctrl/monitor.c
+++ b/fs/resctrl/monitor.c
@@ -1326,6 +1326,27 @@ void rdtgroup_assign_cntrs(struct rdtgroup *rdtgrp)
&mon_event_all[QOS_L3_MBM_LOCAL_EVENT_ID]);
}
+/*
+ * resctrl_assign_cntrs_allrdtgrp() - Assign counters to the MBM events of every
+ * existing group. Called when "mbm_event" mode
+ * is enabled.
+ *
+ * Groups created while in "default" mode have no counter assigned, including the
+ * default group created when resctrl is mounted. Assign counters to them so that
+ * enabling the mode leaves the same assignments that mkdir would have made.
+ */
+static void resctrl_assign_cntrs_allrdtgrp(void)
+{
+ struct rdtgroup *prgrp, *crgrp;
+
+ list_for_each_entry(prgrp, &rdt_all_groups, rdtgroup_list) {
+ rdtgroup_assign_cntrs(prgrp);
+
+ list_for_each_entry(crgrp, &prgrp->mon.crdtgrp_list, mon.crdtgrp_list)
+ rdtgroup_assign_cntrs(crgrp);
+ }
+}
+
/*
* rdtgroup_free_unassign_cntr() - Unassign and reset the counter ID configuration
* for the event pointed to by @mevt within the domain @d and resctrl group @rdtgrp.
@@ -1599,9 +1620,6 @@ ssize_t resctrl_mbm_assign_mode_write(struct kernfs_open_file *of, char *buf,
(READS_TO_LOCAL_MEM |
READS_TO_LOCAL_S_MEM |
NON_TEMP_WRITE_TO_LOCAL_MEM);
- /* Enable auto assignment when switching to "mbm_event" mode */
- if (enable)
- r->mon.mbm_assign_on_mkdir = true;
/*
* Reset all the non-achitectural RMID state and assignable counters.
*/
@@ -1609,6 +1627,12 @@ ssize_t resctrl_mbm_assign_mode_write(struct kernfs_open_file *of, char *buf,
mbm_cntr_free_all(r, d);
resctrl_reset_rmid_all(r, d);
}
+
+ if (enable) {
+ /* Enable auto assignment when switching to "mbm_event" mode */
+ r->mon.mbm_assign_on_mkdir = true;
+ resctrl_assign_cntrs_allrdtgrp();
+ }
}
out_unlock:
--
2.43.0
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH v2 2/3] fs/resctrl: Assign counters to existing groups when enabling mbm_event
2026-09-04 18:06 ` [PATCH v2 2/3] fs/resctrl: Assign counters to existing groups when enabling mbm_event Babu Moger
@ 2026-09-11 22:12 ` Reinette Chatre
0 siblings, 0 replies; 7+ messages in thread
From: Reinette Chatre @ 2026-09-11 22:12 UTC (permalink / raw)
To: Babu Moger, tony.luck, bp
Cc: x86, Dave.Martin, james.morse, corbet, skhan, tglx, mingo,
dave.hansen, hpa, linux-kernel, linux-doc, eranian, peternewman
Hi Babu,
On 9/4/26 11:06 AM, Babu Moger wrote:
> resctrl_mbm_assign_mode_write() frees all counters and sets
The changelog is easier to read if it documents what the code does
instead of documenting the function names. In its current form the reader needs
to stop at the first word of this changelog, go to the source code,
figure out when resctrl_mbm_assign_mode_write() is called, and then be able
to return to changelog to further try and understand the change.
Consider an alternative like:
When the user enables counter assignment mode by writing "mbm_event"
to /sys/fs/resctrl/info/L3_MON/mbm_assign_mode resctrl resets all
monitoring state and sets ...
> mbm_assign_on_mkdir for subsequent mkdir, but does not assign counters to
> groups that already exist, including the default group created at mount.
> Those events then read "Unassigned" until the user assigns counters by
"Those events"? No mention of "events" before this.
> hand.
>
> Enable mbm_assign_on_mkdir and assign counters to existing CTRL_MON and MON
> groups with resctrl_assign_cntrs_allrdtgrp() so the switch matches mkdir
no need to mention the function name, this can be seen from the patch. Just mention
what the change achieves. Looks like "with resctrl_assign_cntrs_allrdtgrp()" can
just be dropped.
> auto-assignment. Groups left without a counter still read "Unassigned".
"Groups" -> "An event ..."?
"still read" -> "reads"?
This implies that counters are assigned to all groups but that may not be possible.
I am not sure what would be best text here. How about something like:
Enable mbm_assign_on_mkdir and assign counters, while there are some
available, to existing ...
>
> Update Documentation/filesystems/resctrl.rst to describe this.
>
> Fixes: 8004ea01cf63 ("fs/resctrl: Introduce the interface to switch between monitor modes")
> Signed-off-by: Babu Moger <babu.moger@amd.com>
> ---
> v2: New patch.
> This patch addresses the Sashiko comment about documentation issue where
> counters are not assigned automatically when mode is switched to mbm_event.
> https://sashiko.dev/#/patchset/8cb66e18e32e4087a9712c1e68ee6da614efe244.1784322818.git.babu.moger%40amd.com
Sounds like this is needed:
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/8cb66e18e32e4087a9712c1e68ee6da614efe244.1784322818.git.babu.moger%40amd.com
Is this a stable candidate?
> In fact, it exposed a real issue. When switching to mbm_event mode, existing
> monitoring groups should be assigned counters whenever counters are available.
> This provides a smooth transition between modes and aligns the behavior with
> the existing auto-assignment mechanism.
> ---
> Documentation/filesystems/resctrl.rst | 7 +++++--
> fs/resctrl/monitor.c | 30 ++++++++++++++++++++++++---
> 2 files changed, 32 insertions(+), 5 deletions(-)
>
> diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
> index e4b66af55ffb..79feeb1dc296 100644
> --- a/Documentation/filesystems/resctrl.rst
> +++ b/Documentation/filesystems/resctrl.rst
> @@ -371,8 +371,11 @@ with the following files:
> of counters available is described in the "num_mbm_cntrs" file. Changing the
> mode may cause all counters on the resource to reset.
>
> - Moving to mbm_event counter assignment mode requires users to assign the counters
> - to the events. Otherwise, the MBM event counters will return 'Unassigned' when read.
> + Moving to mbm_event counter assignment mode enables "mbm_assign_on_mkdir" and
> + assigns counters to the events of all existing groups, including the default
"all existing groups" -> "all existing monitoring groups"
> + group, for as long as counters remain available. Events left without a counter
"for as long as" implies duration. Perhaps "while counters remain available"?
> + will return 'Unassigned' when read until the user assigns one using
> + "mbm_L3_assignments".
hmmm ... guiding users to read each event and use the return value to learn whether
a counter is assigned or not seems inefficient. How about replacing last sentence with
something similar to the "mbm_assign_on_mkdir" doc:
Consult "mbm_L3_assignments" after switching to "mbm_event" mode for
counter assignment states of all monitoring groups.
>
> The mode is beneficial for AMD platforms that support more CTRL_MON
> and MON groups than available hardware counters. By default, this
> diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
> index 73413cb128ea..61463741b91b 100644
> --- a/fs/resctrl/monitor.c
> +++ b/fs/resctrl/monitor.c
> @@ -1326,6 +1326,27 @@ void rdtgroup_assign_cntrs(struct rdtgroup *rdtgrp)
> &mon_event_all[QOS_L3_MBM_LOCAL_EVENT_ID]);
> }
>
> +/*
> + * resctrl_assign_cntrs_allrdtgrp() - Assign counters to the MBM events of every
> + * existing group. Called when "mbm_event" mode
> + * is enabled.
Please do not include caller information in function comments. This does not age well
and this patch clearly demonstrates this: rdtgroup_assign_cntrs()'s comments
read "Called when a new group is created.", after this patch those comments are no
longer accurate and thus also needs to change as part of this patch.
> + *
> + * Groups created while in "default" mode have no counter assigned, including the
> + * default group created when resctrl is mounted. Assign counters to them so that
> + * enabling the mode leaves the same assignments that mkdir would have made.
Above comment belongs in caller.
> + */
> +static void resctrl_assign_cntrs_allrdtgrp(void)
> +{
> + struct rdtgroup *prgrp, *crgrp;
> +
> + list_for_each_entry(prgrp, &rdt_all_groups, rdtgroup_list) {
> + rdtgroup_assign_cntrs(prgrp);
> +
> + list_for_each_entry(crgrp, &prgrp->mon.crdtgrp_list, mon.crdtgrp_list)
> + rdtgroup_assign_cntrs(crgrp);
> + }
> +}
I think sashiko's feedback about needing to test for pseudo-locked groups need not
be followed since there is no overlap between systems supporting assigned counters
and those that support pseudo-locking.
> +
> /*
> * rdtgroup_free_unassign_cntr() - Unassign and reset the counter ID configuration
> * for the event pointed to by @mevt within the domain @d and resctrl group @rdtgrp.
> @@ -1599,9 +1620,6 @@ ssize_t resctrl_mbm_assign_mode_write(struct kernfs_open_file *of, char *buf,
> (READS_TO_LOCAL_MEM |
> READS_TO_LOCAL_S_MEM |
> NON_TEMP_WRITE_TO_LOCAL_MEM);
> - /* Enable auto assignment when switching to "mbm_event" mode */
> - if (enable)
> - r->mon.mbm_assign_on_mkdir = true;
> /*
> * Reset all the non-achitectural RMID state and assignable counters.
> */
> @@ -1609,6 +1627,12 @@ ssize_t resctrl_mbm_assign_mode_write(struct kernfs_open_file *of, char *buf,
> mbm_cntr_free_all(r, d);
> resctrl_reset_rmid_all(r, d);
> }
> +
Comment within the block can be dropped and instead a new comment can be placed here.
Something like:
/*
* Counters were freed above, so both new groups (via mkdir) and the
* groups that already exist need assignments.
*/
> + if (enable) {
> + /* Enable auto assignment when switching to "mbm_event" mode */
> + r->mon.mbm_assign_on_mkdir = true;
> + resctrl_assign_cntrs_allrdtgrp();
> + }
> }
>
> out_unlock:
Reinette
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH v2 3/3] x86/resctrl: Keep mbm_assign_mode in default mode at boot
2026-09-04 18:06 [PATCH v2 0/3] x86/resctrl: Keep default MBM mode at boot and fix ABMC Babu Moger
2026-09-04 18:06 ` [PATCH v2 1/3] x86/resctrl: Fix ABMC counter programming for extended counter ranges Babu Moger
2026-09-04 18:06 ` [PATCH v2 2/3] fs/resctrl: Assign counters to existing groups when enabling mbm_event Babu Moger
@ 2026-09-04 18:06 ` Babu Moger
2026-09-11 22:25 ` Reinette Chatre
2 siblings, 1 reply; 7+ messages in thread
From: Babu Moger @ 2026-09-04 18:06 UTC (permalink / raw)
To: babu.moger, tony.luck, reinette.chatre, bp
Cc: x86, Dave.Martin, james.morse, corbet, skhan, tglx, mingo,
dave.hansen, hpa, linux-kernel, linux-doc, eranian, peternewman
ABMC ("mbm_event" mode) allows explicit assignment of hardware MBM counters
to RMID/event pairs. It is intended for deployments that need to manage
counter assignment on platforms where the number of monitoring groups
exceeds the available hardware counters. Because hardware MBM counters are
scarce, mbm_event mode is best suited for snapshot-and-rotation workflows
that monitor a subset of groups at a time.
Commit 0f1576e43adc ("x86/resctrl: Configure mbm_event mode if supported")
enabled ABMC automatically during initialization. This causes problems on
systems with limited MBM counters and breaks existing userspace that
assumes the historical default mode, including the pqos tool from
intel-cmt-cat [1].
For example, pqos mounts resctrl and creates 16 or more monitoring groups,
using two counters per group (mbm_local_bytes and mbm_total_bytes). On
platforms that provide 32 MBM counters per domain, this consumes the entire
counter pool. Additional groups cannot be assigned counters and pqos
reports zero bandwidth for them.
Leave mbm_assign_mode in "default" mode during initialization. Default mode
can support more monitoring groups (up to 64) than mbm_event mode, which is
typically limited to 16 groups because of hardware counter availability.
Common deployments with a modest number of monitoring groups continue to
receive accurate bandwidth measurements.
Users that require ABMC functionality can enable it explicitly:
echo mbm_event > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
Note that default mode has a long-standing limitation when the number of
monitoring groups exceeds the available counter pool. After hardware
counter reallocation, reads may return "Unavailable" or misleading values.
Users that need stable measurements across a large number of monitoring
groups should use mbm_event mode and rotate assignments as needed.
Update Documentation/filesystems/resctrl.rst to reflect the default boot
behavior, document the limitations of default mode, and adjust
mbm_assign_mode examples accordingly.
Signed-off-by: Babu Moger <babu.moger@amd.com>
Link: https://github.com/intel/intel-cmt-cat/issues/311 # [1]
---
v2:
Added documentation describing the known issue with the default mode.
Will add cc to stable once we have all the things in order.
Let me know if I missed anything.
v1:
https://lore.kernel.org/lkml/8cb66e18e32e4087a9712c1e68ee6da614efe244.1784322818.git.babu.moger@amd.com/
---
Documentation/filesystems/resctrl.rst | 79 +++++++++++++++++----------
arch/x86/kernel/cpu/resctrl/monitor.c | 1 -
2 files changed, 49 insertions(+), 31 deletions(-)
diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
index 79feeb1dc296..a43ed3c89a93 100644
--- a/Documentation/filesystems/resctrl.rst
+++ b/Documentation/filesystems/resctrl.rst
@@ -355,8 +355,8 @@ with the following files:
::
# cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
- [mbm_event]
- default
+ [default]
+ mbm_event
"mbm_event":
@@ -377,20 +377,30 @@ with the following files:
will return 'Unassigned' when read until the user assigns one using
"mbm_L3_assignments".
- The mode is beneficial for AMD platforms that support more CTRL_MON
- and MON groups than available hardware counters. By default, this
- feature is enabled on AMD platforms with the ABMC (Assignable Bandwidth
- Monitoring Counters) capability, ensuring counters remain assigned even
- when the corresponding RMID is not actively used by any processor.
+ The mode is beneficial for AMD platforms that support more CTRL_MON and MON
+ groups than available hardware counters. The mbm_event mode ensures counters
+ remain assigned even when the corresponding RMID is not actively monitored.
"default":
In default mode, resctrl assumes there is a hardware counter for each
- event within every CTRL_MON and MON group. On AMD platforms, it is
- recommended to use the mbm_event mode, if supported, to prevent reset of MBM
- events between reads resulting from hardware re-allocating counters. This can
- result in misleading values or display "Unavailable" if no counter is assigned
- to the event.
+ event within every CTRL_MON and MON group. This mode is enabled by default.
+
+ Default mode has a long-standing limitation on AMD platforms that support
+ more CTRL_MON and MON groups than hardware counters. Hardware dynamically
+ shares a smaller pool of counters among RMIDs. The size of that pool is
+ not enumerated to software (unlike "num_mbm_cntrs" in mbm_event mode), and
+ "num_rmids" may be much larger. On current AMD platforms this pool can
+ provide more counters than mbm_event mode (for example 64, versus 32 ABMC
+ counters), so more groups can be monitored accurately than with mbm_event.
+ Typical usage with fewer groups keeps a counter attached and readings
+ remain accurate. Creating more groups than that pool (for example 64 or
+ more) can cause hardware to re-allocate counters
+ between reads. Bandwidth values may then be misleading, or reads may return
+ "Unavailable" if no counter is allocated to the event. There is no
+ user-visible indication when this begins. Users who need stable readings
+ for many groups should switch to mbm_event mode, if supported, and assign
+ counters to the groups of interest (rotating assignments as needed).
* To enable "mbm_event" counter assignment mode:
::
@@ -474,8 +484,8 @@ with the following files:
Determines if a counter will automatically be assigned to an RMID, MBM event
pair when its associated monitor group is created via mkdir. Enabled by default
- on boot, also when switched from "default" mode to "mbm_event" counter assignment
- mode. Users can disable this capability by writing to the interface.
+ when switched to "mbm_event" counter assignment mode. Users can disable this
+ capability by writing to the interface.
"0":
Auto assignment is disabled.
@@ -1791,32 +1801,41 @@ a. Check if MBM counter assignment mode is supported.
# mount -t resctrl resctrl /sys/fs/resctrl/
+ # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
+ [default]
+ mbm_event
+
+The "mbm_event" and "default" modes are supported. The "default" mode
+is enabled by default.
+
+b. Enable "mbm_event" counter assignment mode.
+::
+
+ # echo "mbm_event" > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
# cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
[mbm_event]
default
-The "mbm_event" mode is detected and enabled.
-
-b. Check how many assignable counters are supported.
+c. Check how many assignable counters are supported.
::
# cat /sys/fs/resctrl/info/L3_MON/num_mbm_cntrs
0=32;1=32
-c. Check how many assignable counters are available for assignment in each domain.
+d. Check how many assignable counters are available for assignment in each domain.
::
# cat /sys/fs/resctrl/info/L3_MON/available_mbm_cntrs
0=30;1=30
-d. To list the default group's assign states.
+e. To list the default group's assign states.
::
# cat /sys/fs/resctrl/mbm_L3_assignments
mbm_total_bytes:0=e;1=e
mbm_local_bytes:0=e;1=e
-e. To unassign the counter associated with the mbm_total_bytes event on domain 0.
+f. To unassign the counter associated with the mbm_total_bytes event on domain 0.
::
# echo "mbm_total_bytes:0=_" > /sys/fs/resctrl/mbm_L3_assignments
@@ -1824,7 +1843,7 @@ e. To unassign the counter associated with the mbm_total_bytes event on domain
mbm_total_bytes:0=_;1=e
mbm_local_bytes:0=e;1=e
-f. To unassign the counter associated with the mbm_total_bytes event on all domains.
+g. To unassign the counter associated with the mbm_total_bytes event on all domains.
::
# echo "mbm_total_bytes:*=_" > /sys/fs/resctrl/mbm_L3_assignments
@@ -1832,7 +1851,7 @@ f. To unassign the counter associated with the mbm_total_bytes event on all doma
mbm_total_bytes:0=_;1=_
mbm_local_bytes:0=e;1=e
-g. To assign a counter associated with the mbm_total_bytes event on all domains in
+h. To assign a counter associated with the mbm_total_bytes event on all domains in
exclusive mode.
::
@@ -1841,7 +1860,7 @@ exclusive mode.
mbm_total_bytes:0=e;1=e
mbm_local_bytes:0=e;1=e
-h. Read the events mbm_total_bytes and mbm_local_bytes of the default group. There is
+i. Read the events mbm_total_bytes and mbm_local_bytes of the default group. There is
no change in reading the events with the assignment.
::
@@ -1854,7 +1873,7 @@ no change in reading the events with the assignment.
# cat /sys/fs/resctrl/mon_data/mon_L3_01/mbm_local_bytes
121212144
-i. Check the event configurations.
+j. Check the event configurations.
::
# cat /sys/fs/resctrl/info/L3_MON/event_configs/mbm_total_bytes/event_filter
@@ -1864,7 +1883,7 @@ i. Check the event configurations.
# cat /sys/fs/resctrl/info/L3_MON/event_configs/mbm_local_bytes/event_filter
local_reads,local_non_temporal_writes,local_reads_slow_memory
-j. Change the event configuration for mbm_local_bytes.
+k. Change the event configuration for mbm_local_bytes.
::
# echo "local_reads, local_non_temporal_writes, local_reads_slow_memory, remote_reads" >
@@ -1873,7 +1892,7 @@ j. Change the event configuration for mbm_local_bytes.
# cat /sys/fs/resctrl/info/L3_MON/event_configs/mbm_local_bytes/event_filter
local_reads,local_non_temporal_writes,local_reads_slow_memory,remote_reads
-k. Now read the local events again. The first read may come back with "Unavailable"
+l. Now read the local events again. The first read may come back with "Unavailable"
status. The subsequent read of mbm_local_bytes will display the current value.
::
@@ -1886,9 +1905,9 @@ status. The subsequent read of mbm_local_bytes will display the current value.
# cat /sys/fs/resctrl/mon_data/mon_L3_01/mbm_local_bytes
1566565
-l. Users have the option to go back to 'default' mbm_assign_mode if required. This can be
-done using the following command. Note that switching the mbm_assign_mode may reset all
-the MBM counters (and thus all MBM events) of all the resctrl groups.
+m. Users have the option to switch back to 'default' mbm_assign_mode if required. This
+can be done using the following command. Note that switching the mbm_assign_mode may
+reset all the MBM counters (and thus all MBM events) of all the resctrl groups.
::
# echo "default" > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
@@ -1896,7 +1915,7 @@ the MBM counters (and thus all MBM events) of all the resctrl groups.
mbm_event
[default]
-m. Unmount the resctrl filesystem.
+n. Unmount the resctrl filesystem.
::
# umount /sys/fs/resctrl/
diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
index 3838e0a13d36..8a0d6086518b 100644
--- a/arch/x86/kernel/cpu/resctrl/monitor.c
+++ b/arch/x86/kernel/cpu/resctrl/monitor.c
@@ -471,7 +471,6 @@ int __init rdt_get_l3_mon_config(struct rdt_resource *r)
r->mon.mbm_cntr_configurable = true;
cpuid_count(0x80000020, 5, &eax, &ebx, &ecx, &edx);
r->mon.num_mbm_cntrs = (ebx & GENMASK(15, 0)) + 1;
- hw_res->mbm_cntr_assign_enabled = true;
}
r->mon_capable = true;
--
2.43.0
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH v2 3/3] x86/resctrl: Keep mbm_assign_mode in default mode at boot
2026-09-04 18:06 ` [PATCH v2 3/3] x86/resctrl: Keep mbm_assign_mode in default mode at boot Babu Moger
@ 2026-09-11 22:25 ` Reinette Chatre
0 siblings, 0 replies; 7+ messages in thread
From: Reinette Chatre @ 2026-09-11 22:25 UTC (permalink / raw)
To: Babu Moger, tony.luck, bp
Cc: x86, Dave.Martin, james.morse, corbet, skhan, tglx, mingo,
dave.hansen, hpa, linux-kernel, linux-doc, eranian, peternewman
Hi Babu,
On 9/4/26 11:06 AM, Babu Moger wrote:
> ABMC ("mbm_event" mode) allows explicit assignment of hardware MBM counters
> to RMID/event pairs. It is intended for deployments that need to manage
> counter assignment on platforms where the number of monitoring groups
> exceeds the available hardware counters. Because hardware MBM counters are
> scarce, mbm_event mode is best suited for snapshot-and-rotation workflows
> that monitor a subset of groups at a time.
>
> Commit 0f1576e43adc ("x86/resctrl: Configure mbm_event mode if supported")
> enabled ABMC automatically during initialization. This causes problems on
Se "Fixes:" tag notes in Documentation/process/maintainer-tip.rst
> systems with limited MBM counters and breaks existing userspace that
> assumes the historical default mode, including the pqos tool from
> intel-cmt-cat [1].
There is no record of this breaking pqos. *you* created [1] *after* you
submitted v1. What a strategy! I mentioned a couple of times that this is
misleading. Since you insist on proclaiming "we cannot break pqos!" as
motivation for this change you have to also disclose the consequence of
this change on pqos followed by motivation why that is acceptable. Specifically:
https://lore.kernel.org/lkml/77f77d02-fae7-401d-9bb5-c62b244d23cd@intel.com/
>
> For example, pqos mounts resctrl and creates 16 or more monitoring groups,
> using two counters per group (mbm_local_bytes and mbm_total_bytes). On
> platforms that provide 32 MBM counters per domain, this consumes the entire
> counter pool. Additional groups cannot be assigned counters and pqos
> reports zero bandwidth for them.
>
> Leave mbm_assign_mode in "default" mode during initialization. Default mode
> can support more monitoring groups (up to 64) than mbm_event mode, which is
"up to 64" - so it may be fewer than 64? What is guidance to users about
how many monitoring groups in "default" mode are "safe"?
> typically limited to 16 groups because of hardware counter availability.
> Common deployments with a modest number of monitoring groups continue to
> receive accurate bandwidth measurements.
Please be specific and do not hide the consequences in a note at the end of changelog.
For example,
Deployments with 64 or fewer monitoring groups will receive accurate
bandwidth measurements. The hardware supports 4096 monitoring groups.
Deployments with 65 to 4096 monitoring groups may (without user-visible
indication) receive misleading values or "Unavailable".
Users that need stable measurements across 65 or more monitoring
groups should use mbm_event mode and rotate assignments as needed.
Although, the earlier text is "up to 64" so above attempt at guidance may not
be correct and there is no knowing how many monitoring groups are guaranteed
to receive accurate counts?
>
> Users that require ABMC functionality can enable it explicitly:
>
> echo mbm_event > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
>
> Note that default mode has a long-standing limitation when the number of
> monitoring groups exceeds the available counter pool. After hardware
> counter reallocation, reads may return "Unavailable" or misleading values.
> Users that need stable measurements across a large number of monitoring
> groups should use mbm_event mode and rotate assignments as needed.
>
> Update Documentation/filesystems/resctrl.rst to reflect the default boot
> behavior, document the limitations of default mode, and adjust
> mbm_assign_mode examples accordingly.
>
If the plan is to send this to stable then it needs a "Fixes:" tag.
> Signed-off-by: Babu Moger <babu.moger@amd.com>
> Link: https://github.com/intel/intel-cmt-cat/issues/311 # [1]
"Link:" -> "Closes:" (and then move it above SoB)?
> ---
> v2:
> Added documentation describing the known issue with the default mode.
> Will add cc to stable once we have all the things in order.
> Let me know if I missed anything.
>
> v1:
> https://lore.kernel.org/lkml/8cb66e18e32e4087a9712c1e68ee6da614efe244.1784322818.git.babu.moger@amd.com/
> ---
> Documentation/filesystems/resctrl.rst | 79 +++++++++++++++++----------
> arch/x86/kernel/cpu/resctrl/monitor.c | 1 -
> 2 files changed, 49 insertions(+), 31 deletions(-)
>
> diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
> index 79feeb1dc296..a43ed3c89a93 100644
> --- a/Documentation/filesystems/resctrl.rst
> +++ b/Documentation/filesystems/resctrl.rst
> @@ -355,8 +355,8 @@ with the following files:
> ::
>
> # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
> - [mbm_event]
> - default
> + [default]
> + mbm_event
>
> "mbm_event":
>
> @@ -377,20 +377,30 @@ with the following files:
> will return 'Unassigned' when read until the user assigns one using
> "mbm_L3_assignments".
>
> - The mode is beneficial for AMD platforms that support more CTRL_MON
> - and MON groups than available hardware counters. By default, this
> - feature is enabled on AMD platforms with the ABMC (Assignable Bandwidth
> - Monitoring Counters) capability, ensuring counters remain assigned even
> - when the corresponding RMID is not actively used by any processor.
> + The mode is beneficial for AMD platforms that support more CTRL_MON and MON
> + groups than available hardware counters. The mbm_event mode ensures counters
> + remain assigned even when the corresponding RMID is not actively monitored.
>
> "default":
>
> In default mode, resctrl assumes there is a hardware counter for each
> - event within every CTRL_MON and MON group. On AMD platforms, it is
> - recommended to use the mbm_event mode, if supported, to prevent reset of MBM
> - events between reads resulting from hardware re-allocating counters. This can
> - result in misleading values or display "Unavailable" if no counter is assigned
> - to the event.
> + event within every CTRL_MON and MON group. This mode is enabled by default.
> +
> + Default mode has a long-standing limitation on AMD platforms that support
Please be specific and drop unnecessary words. For example, "long-standing" can be dropped.
> + more CTRL_MON and MON groups than hardware counters. Hardware dynamically
> + shares a smaller pool of counters among RMIDs. The size of that pool is
> + not enumerated to software (unlike "num_mbm_cntrs" in mbm_event mode), and
I do not think access to "num_mbm_cntrs" depends on mbm_event mode being enabled
so "in mbm_event mode" can just be dropped?
> + "num_rmids" may be much larger. On current AMD platforms this pool can
"current AMD platforms" does not age well in documentation. Can "current" just be dropped?
> + provide more counters than mbm_event mode (for example 64, versus 32 ABMC
"ABMC" -> "mbm_event mode"?
There seems to be another distinction that just "more counters": what can be counted
by such counter. Specifically, a single counter from from the "pool of 64" seems to count
*all* events associated with an RMID, while a single counter from the "pool of 32 mbm_event
mode counters" can only count a single event associated with an RMID? This documentation
uses the term "counter" interchangeably and is difficult to follow.
> + counters), so more groups can be monitored accurately than with mbm_event.
"mbm_event" -> "mbm_event mode"?
> + Typical usage with fewer groups keeps a counter attached and readings
Drop "Typical usage" and just be specific about the different scenarios. It will be easier
for user to determine how their usage matches to specific scenarios than to try and
determine if their usage is "typical".
Also related to above text, since resctrl documentation usually refers to counters being
assigned to event/group pairs this is not clear about what the counter is attached to.
> + remain accurate. Creating more groups than that pool (for example 64 or
"64" -> "65"?
> + more) can cause hardware to re-allocate counters
> + between reads. Bandwidth values may then be misleading, or reads may return
"between reads" - what reads are referred to here?
> + "Unavailable" if no counter is allocated to the event. There is no
hmmm ... now it is refering to counter being allocated to event, but this seems to
be referring to the counters assigned to RMID which would mean it counts all the events?
> + user-visible indication when this begins. Users who need stable readings
> + for many groups should switch to mbm_event mode, if supported, and assign
"many groups" -> "65 or more"?
Since this is not enumerated this may be the best guidance that can be provided ... although
since the changelog mentions "up to 64" there really seems no way for users to know how
many monitor groups are "safe"?
> + counters to the groups of interest (rotating assignments as needed).
>
> * To enable "mbm_event" counter assignment mode:
> ::
> @@ -474,8 +484,8 @@ with the following files:
>
> Determines if a counter will automatically be assigned to an RMID, MBM event
> pair when its associated monitor group is created via mkdir. Enabled by default
> - on boot, also when switched from "default" mode to "mbm_event" counter assignment
> - mode. Users can disable this capability by writing to the interface.
> + when switched to "mbm_event" counter assignment mode. Users can disable this
Why is this change necessary? It seems to drop the text that mbm_assign_on_mkdir is
enabled on boot ... but it is still enabled on boot, no?
> + capability by writing to the interface.
>
> "0":
> Auto assignment is disabled.
> @@ -1791,32 +1801,41 @@ a. Check if MBM counter assignment mode is supported.
>
> # mount -t resctrl resctrl /sys/fs/resctrl/
>
> + # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
> + [default]
> + mbm_event
> +
> +The "mbm_event" and "default" modes are supported. The "default" mode
> +is enabled by default.
> +
> +b. Enable "mbm_event" counter assignment mode.
> +::
> +
> + # echo "mbm_event" > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
> # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
> [mbm_event]
> default
>
> -The "mbm_event" mode is detected and enabled.
> -
> -b. Check how many assignable counters are supported.
> +c. Check how many assignable counters are supported.
<snip>
So many hunks follow and all they do is relabel the steps. This is a lot of churn for
a fix.
What if "step a" instead just drops the
# mount -t resctrl resctrl /sys/fs/resctrl/
step that implies "mbm_event" is the default?
If so, all these hunks could just be replaced with, for example:
diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
index e4b66af55ffb..1c3c27444dd3 100644
--- a/Documentation/filesystems/resctrl.rst
+++ b/Documentation/filesystems/resctrl.rst
@@ -1783,11 +1783,9 @@ View the llc occupancy snapshot::
Examples on working with mbm_assign_mode
========================================
-a. Check if MBM counter assignment mode is supported.
+a. Check if MBM counter assignment mode is supported and enabled.
::
- # mount -t resctrl resctrl /sys/fs/resctrl/
-
# cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
[mbm_event]
default
> diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
> index 3838e0a13d36..8a0d6086518b 100644
> --- a/arch/x86/kernel/cpu/resctrl/monitor.c
> +++ b/arch/x86/kernel/cpu/resctrl/monitor.c
> @@ -471,7 +471,6 @@ int __init rdt_get_l3_mon_config(struct rdt_resource *r)
> r->mon.mbm_cntr_configurable = true;
> cpuid_count(0x80000020, 5, &eax, &ebx, &ecx, &edx);
> r->mon.num_mbm_cntrs = (ebx & GENMASK(15, 0)) + 1;
> - hw_res->mbm_cntr_assign_enabled = true;
> }
>
> r->mon_capable = true;
Reinette
^ permalink raw reply [flat|nested] 7+ messages in thread