[PATCH v5 0/2] x86,fs/resctrl: Keep default MBM mode at boot and fix ABMC programming

[Babu Moger <babu.moger@amd.com>]

Hi All,

This series restores default MBM assignment mode at boot, and fixes ABMC
programming for more than 32 counters.

Commit 0f1576e43adc ("x86/resctrl: Configure mbm_event mode if supported")
enabled ABMC at boot. That breaks the pqos tool from intel-cmt-cat [1],
which treats a non-numeric event read as zero bandwidth. pqos creates 16
or more groups and uses two counters per group. On a platform with 32
counters per domain, that consumes the pool, so further groups read
"Unassigned" and pqos reports 0 MB/s.

Leaving the mode at default does not make pqos correct. pqos still
treats "Unavailable" as zero. Once the default-mode counter pool is
exceeded, hardware can re-allocate a counter between reads and pqos can
report that as a wraparound. That shows up only after the pool is
exceeded. Default mode uses one counter per monitoring group. On existing
AMD platforms that pool is 64 counters and may be larger on newer
hardware, so common deployments stay within it. mbm_event mode uses two
counters per group, so the same 32 counters cover only 16 groups.

mbm_event remains available as an opt-in. Users that need stable
measurements beyond the default-mode pool should enable it and rotate
assignments. The limitation of default mode is documented.

Patch 1: widen cntr_id in L3_QOS_ABMC_CFG from 5 bits to 12 bits, and
limit the enumerated counter count to 4096, which is what that field can
encode. The APM [2] at [3] will be updated for the expanded cntr_id field.

Patch 2: leave mbm_assign_mode as "default" at boot. Counters are not
assigned automatically. So, update resctrl documentation to replect that.

Based on tip/master:
  6b27d3127c05 Merge branch into tip/master: 'x86/tdx'

[1] https://github.com/intel/intel-cmt-cat/issues/311
[2] AMD64 Architecture Programmer's Manual Volume 2: System Programming,
    Publication #24593, Revision 3.41, Section 19.3.3.3 "Assignable
    Bandwidth Monitoring (ABMC)"
[3] https://bugzilla.kernel.org/show_bug.cgi?id=206537

v5:
 - Dropped patch 2 from v4, as auto-assignment is not a requirement
   and should be left to the user.
 - Patch 1: Added Reviewed-by tag from Reinette.
 - Patch 2: Fixed a few typos.
            Auto-assignment is not a requirement based on our discussion.
            Updated the resctrl documentation to reflect this.

v4:
 - Patch 1: Updated changelog(from Reinette's input).
 - Patch 2: Changelog update (Reinette's input).
            Function comment update for rdtgroup_assign_cntrs().
            Tag order update.
 - Patch 3: Introduced mbm_event counter pool and default active counter pool.
            Removed duplicate texts in resctrl.rst.
            Re-arranged the tag order.
v3:
- Patch 1: drop the 32-bit u64 change and the bw_src widening. Cap the
  enumerated counter count at 4096 instead of truncating the CPUID field.
- Patch 2: clean up comments. Add Reported-by, Closes, and Cc: stable.
- Patch 3: document one counter per group in default mode and two counters
  per group in mbm_event mode. Describe the default-mode pool and the pqos
  wraparound once that pool is exceeded.

v2:
- Added patch 2 to address Sashiko's comment regarding the documentation
  issue. In fact, it exposed a real issue. When switching to mbm_event mode,
  existing monitoring groups should be assigned counters whenever counters
  are available. This provides a smooth transition between modes and aligns
  the behavior with the existing auto-assignment mechanism.
  https://sashiko.dev/#/patchset/8cb66e18e32e4087a9712c1e68ee6da614efe244.1784322818.git.babu.moger%40amd.com

- Combine the two v1 patches and add patch 2 (assign existing groups
  on mbm_event switch; Sashiko review).
- Keep boot-default separate from the encoding/truncation fixes.
- Document the default-mode counter-pool limitation.

Previous versions:

v1: https://lore.kernel.org/lkml/980f39d3a0e0d9f73925e362f835aeef070a1bc5.1784322818.git.babu.moger@amd.com/
v2: https://lore.kernel.org/lkml/cover.1788545152.git.babu.moger@amd.com/
v3: https://lore.kernel.org/lkml/cover.1790976400.git.babu.moger@amd.com/
v4: https://lore.kernel.org/lkml/cover.1791589412.git.babu.moger@amd.com/

Thanks
Babu Moger


Babu Moger (2):
  x86/resctrl: Fix ABMC counter programming
  x86,fs/resctrl: Keep mbm_assign_mode in default mode at boot

 Documentation/filesystems/resctrl.rst  | 101 +++++++++++++++----------
 arch/x86/kernel/cpu/resctrl/internal.h |   4 +-
 arch/x86/kernel/cpu/resctrl/monitor.c  |   4 +-
 3 files changed, 67 insertions(+), 42 deletions(-)

-- 
2.43.0