Re: [RFC PATCH 00/6] resctrl: Support mba_MBps on MPAM

Re: [RFC PATCH 00/6] resctrl: Support mba_MBps on MPAM

[Ben Horgan]

Hi Jinjie,

On 09/11/2026 13:26, Jinjie Ruan wrote:
> From: Jinjie Ruan <ruanjinjie@huawei.com>
> 
> Resctrl offers two ways to describe an MBA policy:
> - The percentage interface writes a fraction of the maximum bandwidth
>   into the hardware control register
> - The MBA software controller (mounted with -o mba_MBps) takes
>   an absolute target in MiB/s and adjusts that percentage based on
>   the MBM counters sampled once per second.
> 
> The problem is that a percentage alone isn't enough. How much bandwidth
> a percentage actually corresponds to depends on the memory frequency,
> the channel count, how many threads are generating traffic, the access
> pattern, and the traffic of other groups. Neither MPAM nor x86 resctrl
> reports the maximum bandwidth in MiB/s. So an administrator who wants
> to enforce an absolute budget (a tenant SLA, a noisy-neighbour limit)
> has to measure and re-tune the percentage on every machine. x86 resctrl
> has had this software controller since 2018; MPAM has never been able
> to use it. This series makes it work on MPAM.

I see that you are enabling the software controller at the same time as
assignable mbm counters. This means that if you enable the software
controller and create too many monitor groups then the software
controller will stop working. Any user applications already using the
software controller would be broken.

Would instead teaching MPAM about the mbm_default fixed allocation of
counters be better? If a platform has sufficient bandwidth counters then
we can ensure there is a counter for every monitor group. If necessary,
we can also reduce the number of CLOSID/RMID the driver reports so that
there is one per monitor group.

Thanks,

Ben

> 
> What this series does
> =====================
> 
> MPAM refuses mba_MBps in two places:
> - Its MBM counters are assignable, and supports_mba_mbps() refuses
>   the option when they are.
> - Its maximum bandwidth control is a soft limit by default, which
>   can't hold a single requester down to the target.
>   MPAMF_MBW_IDR.MAX_LIM reports whether the hard limit exists.
> 
> MPAM also makes two generic defects reachable:
> - Writing the control register sleeps (smp_call_on_cpu()), while
>   the worker runs under the RCU read lock
> - A 16-bit fraction lets the minimum control value reach zero, which
>   divides by zero in the estimator
> 
> The patches:
> 
> - Patches 1 and 2 fix the two generic defects.
> - Patches 3 and 4 make the loop runnable on MPAM: the worker takes the
>   CPU hotplug lock around the sleeping control update, and assignable
>   counters are accepted with a fixed assignment mode.
> - Patches 5 and 6 handle the maximum: the driver sets HARDLIM and
>   records the MAX_LIM behaviours, and mba_MBps is refused where only
>   the soft limit exists.
> 
> x86 has no assignable counters in the mode where the controller is
> allowed, and does not set the soft-limit flag, so its behaviour is
> unchanged.
> 
> Testing
> =======
> 
> With the MPAM MSC model in QEMU:
> 
> - From a 9606 MiB/s baseline, a 7000 MiB/s target converges to
>   6888 MiB/s and a 9000 MiB/s target to 8766 MiB/s.
> - The memory-side MSC register shows MBW_MAX.HARDLIM set.
> - A modelled MSC that implements only the soft limit refuses the
>   mba_MBps mount, while a plain mount succeeds.
> - With mbm_assign_on_mkdir forced on while the controller is in use,
>   a write of 0 is refused, and the user's value is restored after
>   unmount.
> - The KUnit tests for the MPAM driver pass.
> 
> Jinjie Ruan (6):
>   fs/resctrl: Avoid division by zero in the MBA software controller
>   fs/resctrl: Do not update bandwidth from a failed counter read
>   fs/resctrl: Take the CPU hotplug lock for software controller updates
>   fs/resctrl: Allow mba_MBps with assignable MBM counters
>   fs/resctrl: Reject mba_MBps when the maximum is only a soft limit
>   arm_mpam: Program the hard limit for the software controller
> 
>  Documentation/filesystems/resctrl.rst | 13 +++-
>  drivers/resctrl/mpam_devices.c        | 55 +++++++++++++----
>  drivers/resctrl/mpam_internal.h       |  4 ++
>  drivers/resctrl/mpam_resctrl.c        |  9 +++
>  fs/resctrl/monitor.c                  | 86 +++++++++++++++++++++------
>  fs/resctrl/rdtgroup.c                 | 48 ++++++++++++---
>  include/linux/resctrl.h               |  5 ++
>  7 files changed, 179 insertions(+), 41 deletions(-)
> 

[RFC PATCH 00/6] resctrl: Support mba_MBps on MPAM

[Jinjie Ruan]

From: Jinjie Ruan <ruanjinjie@huawei.com>

Resctrl offers two ways to describe an MBA policy:
- The percentage interface writes a fraction of the maximum bandwidth
  into the hardware control register
- The MBA software controller (mounted with -o mba_MBps) takes
  an absolute target in MiB/s and adjusts that percentage based on
  the MBM counters sampled once per second.

The problem is that a percentage alone isn't enough. How much bandwidth
a percentage actually corresponds to depends on the memory frequency,
the channel count, how many threads are generating traffic, the access
pattern, and the traffic of other groups. Neither MPAM nor x86 resctrl
reports the maximum bandwidth in MiB/s. So an administrator who wants
to enforce an absolute budget (a tenant SLA, a noisy-neighbour limit)
has to measure and re-tune the percentage on every machine. x86 resctrl
has had this software controller since 2018; MPAM has never been able
to use it. This series makes it work on MPAM.

What this series does
=====================

MPAM refuses mba_MBps in two places:
- Its MBM counters are assignable, and supports_mba_mbps() refuses
  the option when they are.
- Its maximum bandwidth control is a soft limit by default, which
  can't hold a single requester down to the target.
  MPAMF_MBW_IDR.MAX_LIM reports whether the hard limit exists.

MPAM also makes two generic defects reachable:
- Writing the control register sleeps (smp_call_on_cpu()), while
  the worker runs under the RCU read lock
- A 16-bit fraction lets the minimum control value reach zero, which
  divides by zero in the estimator

The patches:

- Patches 1 and 2 fix the two generic defects.
- Patches 3 and 4 make the loop runnable on MPAM: the worker takes the
  CPU hotplug lock around the sleeping control update, and assignable
  counters are accepted with a fixed assignment mode.
- Patches 5 and 6 handle the maximum: the driver sets HARDLIM and
  records the MAX_LIM behaviours, and mba_MBps is refused where only
  the soft limit exists.

x86 has no assignable counters in the mode where the controller is
allowed, and does not set the soft-limit flag, so its behaviour is
unchanged.

Testing
=======

With the MPAM MSC model in QEMU:

- From a 9606 MiB/s baseline, a 7000 MiB/s target converges to
  6888 MiB/s and a 9000 MiB/s target to 8766 MiB/s.
- The memory-side MSC register shows MBW_MAX.HARDLIM set.
- A modelled MSC that implements only the soft limit refuses the
  mba_MBps mount, while a plain mount succeeds.
- With mbm_assign_on_mkdir forced on while the controller is in use,
  a write of 0 is refused, and the user's value is restored after
  unmount.
- The KUnit tests for the MPAM driver pass.

Jinjie Ruan (6):
  fs/resctrl: Avoid division by zero in the MBA software controller
  fs/resctrl: Do not update bandwidth from a failed counter read
  fs/resctrl: Take the CPU hotplug lock for software controller updates
  fs/resctrl: Allow mba_MBps with assignable MBM counters
  fs/resctrl: Reject mba_MBps when the maximum is only a soft limit
  arm_mpam: Program the hard limit for the software controller

 Documentation/filesystems/resctrl.rst | 13 +++-
 drivers/resctrl/mpam_devices.c        | 55 +++++++++++++----
 drivers/resctrl/mpam_internal.h       |  4 ++
 drivers/resctrl/mpam_resctrl.c        |  9 +++
 fs/resctrl/monitor.c                  | 86 +++++++++++++++++++++------
 fs/resctrl/rdtgroup.c                 | 48 ++++++++++++---
 include/linux/resctrl.h               |  5 ++
 7 files changed, 179 insertions(+), 41 deletions(-)

-- 
2.43.0