* [RFC PATCH v2 0/9] fs/resctrl: Support mba_MBps on MPAM
@ 2026-11-12 13:12 Jinjie Ruan
2026-11-12 13:12 ` [RFC PATCH v2 1/9] fs/resctrl: Extract the software controller's step decision Jinjie Ruan
` (8 more replies)
0 siblings, 9 replies; 11+ messages in thread
From: Jinjie Ruan @ 2026-11-12 13:12 UTC (permalink / raw)
To: tony.luck, reinette.chatre, Dave.Martin, james.morse, babu.moger,
corbet, skhan, rdunlap, tglx, mingo, bp, dave.hansen, hpa,
ben.horgan, fenghuay, tan.shaopeng, quic_jiles, x86,
linux-kernel, linux-doc, linux-arm-kernel
Cc: ruanjinjie
Resctrl offers two ways to describe an MBA policy:
- The percentage interface writes a fraction of the maximum bandwidth
into the hardware control register
- The MBA software controller (mounted with -o mba_MBps) takes
an absolute target in MiB/s and adjusts that percentage based on
the MBM counters sampled once per second.
The problem is that a percentage alone isn't enough. How much bandwidth
a percentage actually corresponds to depends on the memory frequency,
the channel count, how many threads are generating traffic, the access
pattern, and the traffic of other groups. And neither MPAM nor x86
resctrl reports the maximum bandwidth in MiB/s. So an administrator who
wants to enforce an absolute budget (a tenant SLA, a noisy-neighbour
limit) has to measure and re-tune the percentage on every machine. x86
resctrl has had this software controller since 2018; MPAM has never been
able to use it.
This series makes mba_MBps work on MPAM.
What this series does
=====================
MPAM refuses mba_MBps in two places:
- Its MBM counters are assignable, and supports_mba_mbps() refuses
the option when they are.
- Its maximum bandwidth control is a soft limit by default, which
can't hold a single requester down to the target.
MPAMF_MBW_IDR.MAX_LIM reports whether the hard limit exists.
MPAM's monitor groups also each take a bandwidth counter from a shared
pool, while nothing relates the number of groups to the number of
counters: a group created once the pool is empty is invisible to the
controller.
MPAM also makes two generic defects reachable:
- Writing the control register sleeps (smp_call_on_cpu()), while
the worker runs under the RCU read lock
- A 16-bit fraction lets the minimum control value reach zero, which
divides by zero in the estimator
The patches:
- Patches 1 to 3 fix the two generic defects
- Patches 4 and 5 make the loop runnable on MPAM: the worker takes the
CPU hotplug lock around the sleeping control update, and assignable
counters are accepted with a fixed assignment mode
- Patches 6 and 7 handle the maximum: mba_MBps is refused where only
the soft limit exists, and the driver sets HARDLIM.
- Patches 8 and 9 cap the CLOSID/RMID space resctrl may use to the
number of counters while the controller is enabled, so that every
group it measures has a counter.
x86 has no assignable counters in the mode where the controller is
allowed, and does not set the soft-limit flag, so its behaviour is
unchanged.
Changes in RFC v2
=============
- Extract the step decision into a helper, with no behaviour change
- Use bw_gran instead of min_bw in the estimate
- Drop the warning when a monitor group gets no counter, as
the new cap makes it unreachable (Ben)
- While the controller is enabled, cap the CLOSID/RMID space to the
number of bandwidth counters (Ben)
- Link to RFC v1:
- https://lore.kernel.org/all/20261109132621.3862497-1-ruanjinjie01@gmail.com/
- https://lore.kernel.org/all/dc1398f4-b81c-4ad8-b706-df89a1622e6c@arm.com/
Testing
=======
With the MPAM MSC model in QEMU:
- From a 9595 MiB/s baseline, a 7000 MiB/s target converges to
6978 MiB/s and a 9000 MiB/s target to 8732 MiB/s.
- The memory-side MSC register shows MBW_MAX.HARDLIM set.
- A modelled MSC that implements only the soft limit refuses the
mba_MBps mount, while a plain mount succeeds.
- With 8 counters and 32 PARTIDs, an mba_MBps mount reports 8 CLOSIDs
and 8 RMIDs, and the 8th mkdir fails with ENOSPC instead of
creating a group the controller cannot see
- With 4 PMGs and 64 counters, a plain mount reports 32 CLOSIDs and
128 RMIDs while an mba_MBps mount reports 32 and 64: only the RMIDs
per CLOSID shrink
- With mbm_assign_on_mkdir forced on while the controller is in use,
a write of 0 is refused, and the user's value is restored after
unmount
- The KUnit tests for the MPAM driver pass.
Jinjie Ruan (9):
fs/resctrl: Extract the software controller's step decision
fs/resctrl: Avoid division by zero in the MBA software controller
fs/resctrl: Do not update bandwidth from a failed counter read
fs/resctrl: Take the CPU hotplug lock for software controller updates
fs/resctrl: Allow mba_MBps with assignable MBM counters
fs/resctrl: Reject mba_MBps when the maximum is only a soft limit
arm_mpam: Program the hard limit for the software controller
x86,fs/resctrl: Let the architecture cap the RMIDs for the software
controller
arm_mpam: Cap the CLOSID/RMID space by the bandwidth counters for
mba_MBps
Documentation/filesystems/resctrl.rst | 7 +-
arch/x86/include/asm/resctrl.h | 6 ++
drivers/resctrl/mpam_devices.c | 55 +++++++++---
drivers/resctrl/mpam_internal.h | 4 +
drivers/resctrl/mpam_resctrl.c | 56 +++++++++++-
fs/resctrl/monitor.c | 123 +++++++++++++++++++-------
fs/resctrl/rdtgroup.c | 66 +++++++++++---
include/linux/arm_mpam.h | 2 +
include/linux/resctrl.h | 2 +
9 files changed, 259 insertions(+), 62 deletions(-)
--
2.34.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [RFC PATCH v2 1/9] fs/resctrl: Extract the software controller's step decision
2026-11-12 13:12 [RFC PATCH v2 0/9] fs/resctrl: Support mba_MBps on MPAM Jinjie Ruan
@ 2026-11-12 13:12 ` Jinjie Ruan
2026-11-12 13:12 ` [RFC PATCH v2 2/9] fs/resctrl: Avoid division by zero in the MBA software controller Jinjie Ruan
` (7 subsequent siblings)
8 siblings, 0 replies; 11+ messages in thread
From: Jinjie Ruan @ 2026-11-12 13:12 UTC (permalink / raw)
To: tony.luck, reinette.chatre, Dave.Martin, james.morse, babu.moger,
corbet, skhan, rdunlap, tglx, mingo, bp, dave.hansen, hpa,
ben.horgan, fenghuay, tan.shaopeng, quic_jiles, x86,
linux-kernel, linux-doc, linux-arm-kernel
Cc: ruanjinjie
Move the condition that picks the next control value out of
update_mba_bw() and into a helper of its own.
No functional change intended.
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
---
fs/resctrl/monitor.c | 54 ++++++++++++++++++++++++++++----------------
1 file changed, 34 insertions(+), 20 deletions(-)
diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
index 04e4ea612b7c..75abbf90d090 100644
--- a/fs/resctrl/monitor.c
+++ b/fs/resctrl/monitor.c
@@ -661,6 +661,38 @@ static struct rdt_ctrl_domain *get_sc_ctrl_domain_from_cpu(int cpu,
return NULL;
}
+/*
+ * Pick the control value for the next interval: step down while the
+ * measured bandwidth is above the target, and step up when the target is
+ * above the bandwidth the next step up is expected to yield.
+ *
+ * Always increase throttling if current bandwidth is above the target set
+ * by user. But avoid thrashing up and down on every poll by checking
+ * whether a decrease in throttling is likely to push the group back over
+ * target. E.g. if currently throttling to 30% of bandwidth on a system
+ * with 10% granularity steps, check whether moving to 40% would go past
+ * the limit by multiplying current bandwidth by "(30 + 10) / 30".
+ *
+ * Return: true when the control value changes, false otherwise.
+ */
+static bool mba_sc_next_ctrl_val(struct rdt_resource *r_mba, u32 cur_msr_val,
+ u32 cur_bw, u32 user_bw, u32 *new_msr_val)
+{
+ if (cur_msr_val > r_mba->membw.min_bw && user_bw < cur_bw) {
+ *new_msr_val = cur_msr_val - r_mba->membw.bw_gran;
+ return true;
+ }
+
+ if (cur_msr_val < MAX_MBA_BW &&
+ user_bw > (cur_bw * (cur_msr_val + r_mba->membw.min_bw) /
+ cur_msr_val)) {
+ *new_msr_val = cur_msr_val + r_mba->membw.bw_gran;
+ return true;
+ }
+
+ return false;
+}
+
/*
* Feedback loop for MBA software controller (mba_sc)
*
@@ -737,27 +769,9 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_l3_mon_domain *dom_m
cur_bw += cmbm_data->prev_bw;
}
- /*
- * Scale up/down the bandwidth linearly for the ctrl group. The
- * bandwidth step is the bandwidth granularity specified by the
- * hardware.
- * Always increase throttling if current bandwidth is above the
- * target set by user.
- * But avoid thrashing up and down on every poll by checking
- * whether a decrease in throttling is likely to push the group
- * back over target. E.g. if currently throttling to 30% of bandwidth
- * on a system with 10% granularity steps, check whether moving to
- * 40% would go past the limit by multiplying current bandwidth by
- * "(30 + 10) / 30".
- */
- if (cur_msr_val > r_mba->membw.min_bw && user_bw < cur_bw) {
- new_msr_val = cur_msr_val - r_mba->membw.bw_gran;
- } else if (cur_msr_val < MAX_MBA_BW &&
- (user_bw > (cur_bw * (cur_msr_val + r_mba->membw.min_bw) / cur_msr_val))) {
- new_msr_val = cur_msr_val + r_mba->membw.bw_gran;
- } else {
+ if (!mba_sc_next_ctrl_val(r_mba, cur_msr_val, cur_bw, user_bw,
+ &new_msr_val))
return;
- }
resctrl_arch_update_one(r_mba, dom_mba, closid, CDP_NONE, new_msr_val);
}
--
2.34.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [RFC PATCH v2 2/9] fs/resctrl: Avoid division by zero in the MBA software controller
2026-11-12 13:12 [RFC PATCH v2 0/9] fs/resctrl: Support mba_MBps on MPAM Jinjie Ruan
2026-11-12 13:12 ` [RFC PATCH v2 1/9] fs/resctrl: Extract the software controller's step decision Jinjie Ruan
@ 2026-11-12 13:12 ` Jinjie Ruan
2026-11-12 13:12 ` [RFC PATCH v2 3/9] fs/resctrl: Do not update bandwidth from a failed counter read Jinjie Ruan
` (6 subsequent siblings)
8 siblings, 0 replies; 11+ messages in thread
From: Jinjie Ruan @ 2026-11-12 13:12 UTC (permalink / raw)
To: tony.luck, reinette.chatre, Dave.Martin, james.morse, babu.moger,
corbet, skhan, rdunlap, tglx, mingo, bp, dave.hansen, hpa,
ben.horgan, fenghuay, tan.shaopeng, quic_jiles, x86,
linux-kernel, linux-doc, linux-arm-kernel
Cc: ruanjinjie
The software controller raises the control while the target is above
the bandwidth expected after the next step, and it uses min_bw instead
of bw_gran. The two are equal on x86, but when MPAM's maximum control
has 8 or more bits, min_bw rounds to 0% while bw_gran is 1%. The estimate
then falls back to the current bandwidth, so the control steps up
whenever the target is above it and oscillates across the target.
With min_bw at 0% the control can also reach zero, and the estimate
then divides by zero in the MBM overflow worker.
Use bw_gran in the estimate, and step up without evaluating it when the
control is zero. Also require the target to be above the measured
bandwidth, so a zero target does not raise the control.
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
---
fs/resctrl/monitor.c | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
index 75abbf90d090..f067329e92c6 100644
--- a/fs/resctrl/monitor.c
+++ b/fs/resctrl/monitor.c
@@ -673,6 +673,9 @@ static struct rdt_ctrl_domain *get_sc_ctrl_domain_from_cpu(int cpu,
* with 10% granularity steps, check whether moving to 40% would go past
* the limit by multiplying current bandwidth by "(30 + 10) / 30".
*
+ * A zero min_bw lets the control reach zero, where the ratio is
+ * undefined. Take the step up without evaluating it.
+ *
* Return: true when the control value changes, false otherwise.
*/
static bool mba_sc_next_ctrl_val(struct rdt_resource *r_mba, u32 cur_msr_val,
@@ -683,9 +686,9 @@ static bool mba_sc_next_ctrl_val(struct rdt_resource *r_mba, u32 cur_msr_val,
return true;
}
- if (cur_msr_val < MAX_MBA_BW &&
- user_bw > (cur_bw * (cur_msr_val + r_mba->membw.min_bw) /
- cur_msr_val)) {
+ if (cur_msr_val < MAX_MBA_BW && user_bw > cur_bw &&
+ (cur_msr_val == 0 ||
+ user_bw > (cur_bw * (cur_msr_val + r_mba->membw.bw_gran) / cur_msr_val))) {
*new_msr_val = cur_msr_val + r_mba->membw.bw_gran;
return true;
}
--
2.34.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [RFC PATCH v2 3/9] fs/resctrl: Do not update bandwidth from a failed counter read
2026-11-12 13:12 [RFC PATCH v2 0/9] fs/resctrl: Support mba_MBps on MPAM Jinjie Ruan
2026-11-12 13:12 ` [RFC PATCH v2 1/9] fs/resctrl: Extract the software controller's step decision Jinjie Ruan
2026-11-12 13:12 ` [RFC PATCH v2 2/9] fs/resctrl: Avoid division by zero in the MBA software controller Jinjie Ruan
@ 2026-11-12 13:12 ` Jinjie Ruan
2026-11-12 13:12 ` [RFC PATCH v2 4/9] fs/resctrl: Take the CPU hotplug lock for software controller updates Jinjie Ruan
` (5 subsequent siblings)
8 siblings, 0 replies; 11+ messages in thread
From: Jinjie Ruan @ 2026-11-12 13:12 UTC (permalink / raw)
To: tony.luck, reinette.chatre, Dave.Martin, james.morse, babu.moger,
corbet, skhan, rdunlap, tglx, mingo, bp, dave.hansen, hpa,
ben.horgan, fenghuay, tan.shaopeng, quic_jiles, x86,
linux-kernel, linux-doc, linux-arm-kernel
Cc: ruanjinjie
Every second, mbm_update_one_event() reads a monitoring group's
counter and passes the result to mbm_bw_count(). When the read fails,
rmid_read::err is set but the value stays zero, and mbm_bw_count()
runs anyway. Subtracting the previous sample from zero wraps to a
huge bandwidth, and the controller moves the control the wrong way.
The read fails when the architecture reports the counter as
unavailable, and, with assignable counters, when the group has no
counter because the shared pool is exhausted.
Update the bandwidth only after a successful read. A failed read keeps
the previous bandwidth; the counter keeps accumulating, so the next
sample spans the failed interval and is larger. The controller steps
down once and recovers in the next round.
Fixes: 30442571ec81 ("x86/resctrl: Calculate bandwidth from the previous __mon_event_count() chunks")
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
Cc: <stable@vger.kernel.org> # v6.1+
---
fs/resctrl/monitor.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
index f067329e92c6..49a4b4ce95d4 100644
--- a/fs/resctrl/monitor.c
+++ b/fs/resctrl/monitor.c
@@ -801,10 +801,11 @@ static void mbm_update_one_event(struct rdt_resource *r, struct rdt_l3_mon_domai
__mon_event_count(rdtgrp, &rr);
/*
- * If the software controller is enabled, compute the
- * bandwidth for this event id.
+ * If the software controller is enabled, compute the bandwidth for
+ * this event id. Do not update it when the counter could not be read:
+ * that would store a bogus zero sample as the previous bandwidth.
*/
- if (is_mba_sc(NULL))
+ if (is_mba_sc(NULL) && !rr.err)
mbm_bw_count(rdtgrp, &rr);
if (rr.arch_mon_ctx)
--
2.34.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [RFC PATCH v2 4/9] fs/resctrl: Take the CPU hotplug lock for software controller updates
2026-11-12 13:12 [RFC PATCH v2 0/9] fs/resctrl: Support mba_MBps on MPAM Jinjie Ruan
` (2 preceding siblings ...)
2026-11-12 13:12 ` [RFC PATCH v2 3/9] fs/resctrl: Do not update bandwidth from a failed counter read Jinjie Ruan
@ 2026-11-12 13:12 ` Jinjie Ruan
2026-11-12 13:12 ` [RFC PATCH v2 5/9] fs/resctrl: Allow mba_MBps with assignable MBM counters Jinjie Ruan
` (4 subsequent siblings)
8 siblings, 0 replies; 11+ messages in thread
From: Jinjie Ruan @ 2026-11-12 13:12 UTC (permalink / raw)
To: tony.luck, reinette.chatre, Dave.Martin, james.morse, babu.moger,
corbet, skhan, rdunlap, tglx, mingo, bp, dave.hansen, hpa,
ben.horgan, fenghuay, tan.shaopeng, quic_jiles, x86,
linux-kernel, linux-doc, linux-arm-kernel
Cc: ruanjinjie
update_mba_bw() runs from the MBM overflow worker and calls the
architecture to read and write the control of its CPU's domain, under
the RCU read lock.
Writing the control can sleep: MPAM uses smp_call_on_cpu() and waits
for a CPU that can reach the MSC, so that CPU must stay online. The
hotplug lock cannot be taken after rdtgroup_mutex, because the offline
path takes the hotplug lock first, and sleeping in an RCU read-side
section is invalid.
Take the hotplug lock with cpus_read_trylock() before rdtgroup_mutex.
Skip a round when it is unavailable; a blocking lock would deadlock
against the offline path, which holds the write lock while it drains
this worker. The lock keeps the control domain in place, so drop the
RCU read lock.
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
---
fs/resctrl/monitor.c | 28 +++++++++++++++++++---------
fs/resctrl/rdtgroup.c | 8 +++++---
2 files changed, 24 insertions(+), 12 deletions(-)
diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
index 49a4b4ce95d4..8f3b8c3e3010 100644
--- a/fs/resctrl/monitor.c
+++ b/fs/resctrl/monitor.c
@@ -640,19 +640,18 @@ void mon_event_count(void *info)
/*
* Find the software controller's ctrl domain that contains @cpu on resource @r.
*
- * Only called from the mbm_over worker via update_mba_bw() where the returned
- * domain is kept alive by cancel_delayed_work_sync() in
- * resctrl_offline_ctrl_domain(). This drains this worker and then waits on
- * rdtgroup_mutex held here before the architecture can free the ctrl domain.
+ * Only called from the mbm_over worker via update_mba_bw(), which holds the
+ * CPU hotplug lock so the domain cannot be removed while it is being used.
*
- * Context: Call from RCU read-side critical section.
+ * Context: Call with the CPU hotplug lock held.
*/
static struct rdt_ctrl_domain *get_sc_ctrl_domain_from_cpu(int cpu,
struct rdt_resource *r)
{
struct rdt_ctrl_domain *d;
- list_for_each_entry_rcu(d, &r->ctrl_domains, hdr.list) {
+ list_for_each_entry_rcu(d, &r->ctrl_domains, hdr.list,
+ lockdep_is_cpus_held()) {
/* Find the domain that contains this CPU */
if (cpumask_test_cpu(cpu, &d->hdr.cpu_mask))
return d;
@@ -748,7 +747,8 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_l3_mon_domain *dom_m
if (WARN_ON_ONCE(!pmbm_data))
return;
- guard(rcu)();
+ lockdep_assert_cpus_held();
+
dom_mba = get_sc_ctrl_domain_from_cpu(smp_processor_id(), r_mba);
if (!dom_mba) {
pr_warn_once("Failure to get domain for MBA update\n");
@@ -895,11 +895,19 @@ void mbm_handle_overflow(struct work_struct *work)
struct rdt_l3_mon_domain *d;
struct list_head *head;
struct rdt_resource *r;
+ bool cpus_read_locked = false;
/*
- * Safe to run without CPU hotplug lock. Work is guaranteed to be
+ * Reading counters is safe without the CPU hotplug lock: the work is
* canceled before the domain structure is removed.
+ *
+ * Programming the control can sleep and may need the hotplug lock,
+ * which cannot be taken after rdtgroup_mutex. Try it first; if
+ * hotplug is in progress skip the update and retry next round.
*/
+ if (is_mba_sc(NULL))
+ cpus_read_locked = cpus_read_trylock();
+
mutex_lock(&rdtgroup_mutex);
/*
@@ -934,7 +942,7 @@ void mbm_handle_overflow(struct work_struct *work)
list_for_each_entry(crgrp, head, mon.crdtgrp_list)
mbm_update(r, d, crgrp);
- if (is_mba_sc(NULL))
+ if (is_mba_sc(NULL) && cpus_read_locked)
update_mba_bw(prgrp, d);
}
@@ -948,6 +956,8 @@ void mbm_handle_overflow(struct work_struct *work)
out_unlock:
mutex_unlock(&rdtgroup_mutex);
+ if (cpus_read_locked)
+ cpus_read_unlock();
}
/**
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 57ab090072c2..b32831871f70 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -4519,9 +4519,11 @@ void resctrl_offline_ctrl_domain(struct rdt_resource *r, struct rdt_ctrl_domain
* no new worker iteration can find it; drain any worker that already
* holds a pointer to it before the architecture frees the domain.
*
- * Software controller is enabled/disabled on mount/unmount with
- * cpus_read_lock() held. Running here with cpus_write_lock() so
- * there are no concurrent changes to software controller status.
+ * The overflow worker holds the CPU hotplug read lock around its
+ * software controller updates, and mount/unmount enable/disable the
+ * software controller with the read lock held. Running here with
+ * cpus_write_lock() means there are no in-flight software controller
+ * updates and no concurrent change of the software controller status.
*/
if (r->rid == RDT_RESOURCE_MBA && is_mba_sc(r)) {
struct rdt_resource *l3 = resctrl_arch_get_resource(RDT_RESOURCE_L3);
--
2.34.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [RFC PATCH v2 5/9] fs/resctrl: Allow mba_MBps with assignable MBM counters
2026-11-12 13:12 [RFC PATCH v2 0/9] fs/resctrl: Support mba_MBps on MPAM Jinjie Ruan
` (3 preceding siblings ...)
2026-11-12 13:12 ` [RFC PATCH v2 4/9] fs/resctrl: Take the CPU hotplug lock for software controller updates Jinjie Ruan
@ 2026-11-12 13:12 ` Jinjie Ruan
2026-11-17 11:37 ` Ben Horgan
2026-11-12 13:12 ` [RFC PATCH v2 6/9] fs/resctrl: Reject mba_MBps when the maximum is only a soft limit Jinjie Ruan
` (3 subsequent siblings)
8 siblings, 1 reply; 11+ messages in thread
From: Jinjie Ruan @ 2026-11-12 13:12 UTC (permalink / raw)
To: tony.luck, reinette.chatre, Dave.Martin, james.morse, babu.moger,
corbet, skhan, rdunlap, tglx, mingo, bp, dave.hansen, hpa,
ben.horgan, fenghuay, tan.shaopeng, quic_jiles, x86,
linux-kernel, linux-doc, linux-arm-kernel
Cc: ruanjinjie
The controller needs a counter for the event selected by mba_MBps_event
in every monitoring group, so mba_MBps is refused when MBM counters are
assignable. On MPAM a group's monitor is assigned from a fixed pool
when the group is created, and users can otherwise only pick a
percentage of a maximum that firmware does not report.
Allow the controller when the architecture has a fixed assignment mode
and assigns counters when a group is created. Keep mbm_assign_on_mkdir
enabled while the controller is in use, restore its previous value
when the controller is disabled, and refuse to unassign the
controller's counter meanwhile.
Only MPAM has this mode; other systems keep the previous behavior.
Suggested-by: Ben Horgan <ben.horgan@arm.com>
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
---
Documentation/filesystems/resctrl.rst | 7 +++++--
fs/resctrl/monitor.c | 25 ++++++++++++++++++++++
fs/resctrl/rdtgroup.c | 30 +++++++++++++++++++++------
3 files changed, 54 insertions(+), 8 deletions(-)
diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
index b52795e03303..d70aec6f7096 100644
--- a/Documentation/filesystems/resctrl.rst
+++ b/Documentation/filesystems/resctrl.rst
@@ -471,7 +471,8 @@ with the following files:
Determines if a counter will automatically be assigned to an RMID, MBM event
pair when its associated monitor group is created via mkdir. Enabled by default
on boot, also when switched from "default" mode to "mbm_event" counter assignment
- mode. Users can disable this capability by writing to the interface.
+ mode. Users can disable this capability by writing to the interface. While the
+ MBA software controller is enabled, auto assignment cannot be disabled.
"0":
Auto assignment is disabled.
@@ -704,7 +705,9 @@ When monitoring is enabled all MON groups may also contain:
mbm_total_bytes:0=e;1=e
mbm_local_bytes:0=e;1=e
- Assignments can be modified by writing to the interface.
+ Assignments can be modified by writing to the interface. While the
+ MBA software controller is enabled, the counter of the event the
+ controller uses cannot be unassigned.
Examples:
diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
index 8f3b8c3e3010..5a67c8089615 100644
--- a/fs/resctrl/monitor.c
+++ b/fs/resctrl/monitor.c
@@ -1210,6 +1210,12 @@ ssize_t resctrl_mbm_assign_on_mkdir_write(struct kernfs_open_file *of, char *buf
goto out_unlock;
}
+ if (!value && is_mba_sc(NULL)) {
+ rdt_last_cmd_puts("mbm_assign_on_mkdir is required by the software controller\n");
+ ret = -EBUSY;
+ goto out_unlock;
+ }
+
r->mon.mbm_assign_on_mkdir = value;
out_unlock:
@@ -1776,6 +1782,19 @@ static struct mon_evt *mbm_get_mon_event_by_name(struct rdt_resource *r, char *n
return NULL;
}
+/*
+ * Return the event the MBA software controller uses as its input for @rdtgrp.
+ * A monitor group has no event of its own: its bandwidth is summed into the
+ * parent control group, which is the group with the mba_MBps_event setting.
+ */
+static enum resctrl_event_id rdtgrp_mba_mbps_event(struct rdtgroup *rdtgrp)
+{
+ if (rdtgrp->type == RDTMON_GROUP)
+ return rdtgrp->mon.parent->mba_mbps_event;
+
+ return rdtgrp->mba_mbps_event;
+}
+
static int rdtgroup_modify_assign_state(char *assign, struct rdt_l3_mon_domain *d,
struct rdtgroup *rdtgrp, struct mon_evt *mevt)
{
@@ -1789,6 +1808,12 @@ static int rdtgroup_modify_assign_state(char *assign, struct rdt_l3_mon_domain *
ret = rdtgroup_assign_cntr_event(d, rdtgrp, mevt);
break;
case '_':
+ if (is_mba_sc(NULL) &&
+ mevt->evtid == rdtgrp_mba_mbps_event(rdtgrp)) {
+ rdt_last_cmd_puts("Counter is in use by the software controller\n");
+ ret = -EBUSY;
+ break;
+ }
rdtgroup_unassign_cntr_event(d, rdtgrp, mevt);
break;
default:
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index b32831871f70..518d89b0c67d 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -2610,10 +2610,9 @@ static void mba_sc_domain_destroy(struct rdt_resource *r,
* The MBA software controller is supported only if MBM is supported and MBA is
* in linear scale, and the MBM monitor scope is the same as MBA control scope.
*
- * The software controller cannot be supported when the MBM counters are
- * assignable. There is no guarantee that MBM counters are assigned to the
- * event backing the software controller in all monitoring domains of all
- * monitoring groups.
+ * The controller needs a counter for its event in every monitoring group.
+ * With assignable counters that requires a fixed assignment mode, with
+ * counters assigned automatically while the controller is in use.
*/
static bool supports_mba_mbps(void)
{
@@ -2623,9 +2622,13 @@ static bool supports_mba_mbps(void)
return (resctrl_is_mbm_enabled() &&
r->alloc_capable && is_mba_linear() &&
r->ctrl_scope == rmbm->mon_scope &&
- !rmbm->mon.mbm_cntr_assignable);
+ (!rmbm->mon.mbm_cntr_assignable ||
+ rmbm->mon.mbm_cntr_assign_fixed));
}
+/* mbm_assign_on_mkdir before the software controller forced it on. */
+static bool mba_sc_saved_assign_on_mkdir;
+
/*
* Enable or disable the MBA software controller
* which helps user specify bandwidth in MBps.
@@ -2633,6 +2636,7 @@ static bool supports_mba_mbps(void)
static int set_mba_sc(bool mba_sc)
{
struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
+ struct rdt_resource *rmbm = resctrl_arch_get_resource(RDT_RESOURCE_L3);
u32 num_closid = resctrl_arch_get_num_closid(r);
struct rdt_ctrl_domain *d;
unsigned long fflags;
@@ -2643,6 +2647,20 @@ static int set_mba_sc(bool mba_sc)
r->membw.mba_sc = mba_sc;
+ /*
+ * The controller needs its event counted in every monitoring group:
+ * force automatic assignment on, and restore the user's setting when
+ * the controller is disabled.
+ */
+ if (rmbm->mon.mbm_cntr_assignable) {
+ if (mba_sc) {
+ mba_sc_saved_assign_on_mkdir = rmbm->mon.mbm_assign_on_mkdir;
+ rmbm->mon.mbm_assign_on_mkdir = true;
+ } else {
+ rmbm->mon.mbm_assign_on_mkdir = mba_sc_saved_assign_on_mkdir;
+ }
+ }
+
rdtgroup_default.mba_mbps_event = mba_mbps_default_event;
list_for_each_entry_rcu(d, &r->ctrl_domains, hdr.list, lockdep_is_cpus_held()) {
@@ -3352,7 +3370,7 @@ static int rdt_parse_param(struct fs_context *fc, struct fs_parameter *param)
ctx->enable_cdpl2 = true;
return 0;
case Opt_mba_mbps:
- msg = "mba_MBps requires MBM (mbm_event mode not supported) and linear scale MBA at L3 scope";
+ msg = "mba_MBps requires MBM and linear scale MBA at L3 scope";
if (!supports_mba_mbps())
return invalfc(fc, msg);
ctx->enable_mba_mbps = true;
--
2.34.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [RFC PATCH v2 6/9] fs/resctrl: Reject mba_MBps when the maximum is only a soft limit
2026-11-12 13:12 [RFC PATCH v2 0/9] fs/resctrl: Support mba_MBps on MPAM Jinjie Ruan
` (4 preceding siblings ...)
2026-11-12 13:12 ` [RFC PATCH v2 5/9] fs/resctrl: Allow mba_MBps with assignable MBM counters Jinjie Ruan
@ 2026-11-12 13:12 ` Jinjie Ruan
2026-11-12 13:12 ` [RFC PATCH v2 7/9] arm_mpam: Program the hard limit for the software controller Jinjie Ruan
` (2 subsequent siblings)
8 siblings, 0 replies; 11+ messages in thread
From: Jinjie Ruan @ 2026-11-12 13:12 UTC (permalink / raw)
To: tony.luck, reinette.chatre, Dave.Martin, james.morse, babu.moger,
corbet, skhan, rdunlap, tglx, mingo, bp, dave.hansen, hpa,
ben.horgan, fenghuay, tan.shaopeng, quic_jiles, x86,
linux-kernel, linux-doc, linux-arm-kernel
Cc: ruanjinjie
The controller lowers the maximum bandwidth control when the measured
bandwidth exceeds the target, and relies on the hardware enforcing
that maximum. A hard limit does not service requests above the
maximum; a soft limit services them when no other request is waiting,
so a group without competing traffic keeps exceeding the target and
the controller never converges.
Add resctrl_membw::max_is_soft_limit for architectures whose maximum
control implements only the soft limit, and refuse to mount with
mba_MBps when it is set. Architectures that leave the flag clear are
unchanged.
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
---
fs/resctrl/rdtgroup.c | 5 ++++-
include/linux/resctrl.h | 2 ++
2 files changed, 6 insertions(+), 1 deletion(-)
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 518d89b0c67d..9f10b6ceff8a 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -2613,6 +2613,8 @@ static void mba_sc_domain_destroy(struct rdt_resource *r,
* The controller needs a counter for its event in every monitoring group.
* With assignable counters that requires a fixed assignment mode, with
* counters assigned automatically while the controller is in use.
+ *
+ * The maximum control must also be a hard limit, see max_is_soft_limit.
*/
static bool supports_mba_mbps(void)
{
@@ -2622,6 +2624,7 @@ static bool supports_mba_mbps(void)
return (resctrl_is_mbm_enabled() &&
r->alloc_capable && is_mba_linear() &&
r->ctrl_scope == rmbm->mon_scope &&
+ !r->membw.max_is_soft_limit &&
(!rmbm->mon.mbm_cntr_assignable ||
rmbm->mon.mbm_cntr_assign_fixed));
}
@@ -3370,7 +3373,7 @@ static int rdt_parse_param(struct fs_context *fc, struct fs_parameter *param)
ctx->enable_cdpl2 = true;
return 0;
case Opt_mba_mbps:
- msg = "mba_MBps requires MBM and linear scale MBA at L3 scope";
+ msg = "mba_MBps requires MBM, linear scale MBA at L3 scope and an enforced maximum";
if (!supports_mba_mbps())
return invalfc(fc, msg);
ctx->enable_mba_mbps = true;
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index 10dfdca7f4bf..933bab2b8289 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -250,6 +250,7 @@ enum membw_throttle_mode {
* @bw_gran: Granularity at which the memory bandwidth is allocated
* @delay_linear: True if memory B/W delay is in linear scale
* @arch_needs_linear: True if we can't configure non-linear resources
+ * @max_is_soft_limit: True if maximum control only marks traffic as low priority
* @throttle_mode: Bandwidth throttling mode when threads request
* different memory bandwidths
* @mba_sc: True if MBA software controller(mba_sc) is enabled
@@ -261,6 +262,7 @@ struct resctrl_membw {
u32 bw_gran;
u32 delay_linear;
bool arch_needs_linear;
+ bool max_is_soft_limit;
enum membw_throttle_mode throttle_mode;
bool mba_sc;
u32 *mb_map;
--
2.34.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [RFC PATCH v2 7/9] arm_mpam: Program the hard limit for the software controller
2026-11-12 13:12 [RFC PATCH v2 0/9] fs/resctrl: Support mba_MBps on MPAM Jinjie Ruan
` (5 preceding siblings ...)
2026-11-12 13:12 ` [RFC PATCH v2 6/9] fs/resctrl: Reject mba_MBps when the maximum is only a soft limit Jinjie Ruan
@ 2026-11-12 13:12 ` Jinjie Ruan
2026-11-12 13:12 ` [RFC PATCH v2 8/9] x86,fs/resctrl: Let the architecture cap the RMIDs " Jinjie Ruan
2026-11-12 13:12 ` [RFC PATCH v2 9/9] arm_mpam: Cap the CLOSID/RMID space by the bandwidth counters for mba_MBps Jinjie Ruan
8 siblings, 0 replies; 11+ messages in thread
From: Jinjie Ruan @ 2026-11-12 13:12 UTC (permalink / raw)
To: tony.luck, reinette.chatre, Dave.Martin, james.morse, babu.moger,
corbet, skhan, rdunlap, tglx, mingo, bp, dave.hansen, hpa,
ben.horgan, fenghuay, tan.shaopeng, quic_jiles, x86,
linux-kernel, linux-doc, linux-arm-kernel
Cc: ruanjinjie
MPAM's maximum bandwidth control has a HARDLIM bit:
- When set, a PARTID that uses more than its maximum gets no further
bandwidth
- When clear, requests above the maximum are serviced when no other
request is waiting
The driver writes only the 16-bit maximum, so the control is a soft
limit and the controller cannot keep a single requester below its target.
Carry the hard limit in mpam_config and set it when the controller is
enabled. MPAMF_MBW_IDR.MAX_LIM reports the implemented behaviors: 00
both, 10 hard only, 01 soft only. Record the soft-only case and the
reserved 0b11 in resctrl_membw::max_is_soft_limit so mba_MBps is
refused there. Merging resources into a class propagates the soft-only
case, because the hard limit is usable only when every resource
implements it.
The percentage interface never sets the hard limit, and resctrl clears
a limit set by the controller when it resets the controls on unmount.
Write the bit from the controller state on every write, so that a hard
limit left by an interrupted reset is cleared.
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
---
drivers/resctrl/mpam_devices.c | 55 ++++++++++++++++++++++++++-------
drivers/resctrl/mpam_internal.h | 4 +++
drivers/resctrl/mpam_resctrl.c | 8 +++++
3 files changed, 56 insertions(+), 11 deletions(-)
diff --git a/drivers/resctrl/mpam_devices.c b/drivers/resctrl/mpam_devices.c
index c067b6d57d71..2c8f60c96d3f 100644
--- a/drivers/resctrl/mpam_devices.c
+++ b/drivers/resctrl/mpam_devices.c
@@ -1025,8 +1025,22 @@ static int mpam_ris_hw_probe(struct mpam_msc_ris *ris)
*/
props->bwa_wd = min(props->bwa_wd, 16);
- if (props->bwa_wd && FIELD_GET(MPAMF_MBW_IDR_HAS_MAX, mbw_features))
+ if (props->bwa_wd && FIELD_GET(MPAMF_MBW_IDR_HAS_MAX, mbw_features)) {
+ /*
+ * MAX_LIM: 0b00 both limits, 0b01 the soft limit
+ * only, 0b10 the hard limit only. 0b11 is reserved;
+ * treat it as the soft limit only so the software
+ * controller is refused rather than rely on an
+ * unknown limit behavior.
+ */
+ props->max_lim = FIELD_GET(MPAMF_MBW_IDR_MAX_LIM, mbw_features);
+ if (props->max_lim == 3)
+ props->max_lim = 1;
+
mpam_set_feature(mpam_feat_mbw_max, props);
+ /* The hard-limit bit is a configuration bit. */
+ mpam_set_feature(mpam_feat_max_limit, props);
+ }
if (props->bwa_wd && FIELD_GET(MPAMF_MBW_IDR_HAS_MIN, mbw_features))
mpam_set_feature(mpam_feat_mbw_min, props);
@@ -1911,16 +1925,21 @@ static int mpam_reprogram_ris_partid(struct mpam_msc_ris *ris, u16 partid,
}
if (mpam_has_feature(mpam_feat_mbw_max, rprops)) {
- if (mpam_has_feature(mpam_feat_mbw_max, cfg)) {
- ret = mpam_write_partsel_reg(msc, MBW_MAX, cfg->mbw_max);
- if (ret)
- return ret;
- } else {
- ret = mpam_write_partsel_reg(msc, MBW_MAX,
- MPAMCFG_MBW_MAX_MAX);
- if (ret)
- return ret;
- }
+ u32 val = MPAMCFG_MBW_MAX_MAX;
+
+ if (mpam_has_feature(mpam_feat_mbw_max, cfg))
+ val = cfg->mbw_max;
+
+ /*
+ * The hard-limit bit follows its own feature bit so that it
+ * can be set without changing the maximum.
+ */
+ if (mpam_has_feature(mpam_feat_max_limit, cfg) && cfg->max_limit)
+ val |= MPAMCFG_MBW_MAX_HARDLIM;
+
+ ret = mpam_write_partsel_reg(msc, MBW_MAX, val);
+ if (ret)
+ return ret;
}
if (mpam_has_feature(mpam_feat_mbw_prop, rprops)) {
@@ -2661,6 +2680,19 @@ static void __props_mismatch(struct mpam_props *parent,
parent->bwa_wd = min(parent->bwa_wd, child->bwa_wd);
}
+ /*
+ * The hard limit is only usable if every resource implements it,
+ * so the soft limit only wins the merge. The encoding follows
+ * MPAMF_MBW_IDR.MAX_LIM.
+ */
+ if (alias && !mpam_has_feature(mpam_feat_mbw_max, parent) &&
+ mpam_has_feature(mpam_feat_mbw_max, child)) {
+ parent->max_lim = child->max_lim;
+ } else if (mpam_has_feature(mpam_feat_mbw_max, parent) &&
+ child->max_lim == 1) {
+ parent->max_lim = 1;
+ }
+
if (alias && !mpam_has_cmax_wd_feature(parent) && mpam_has_cmax_wd_feature(child)) {
parent->cmax_wd = child->cmax_wd;
} else if (MISMATCHED_HELPER(parent, child, mpam_has_cmax_wd_feature,
@@ -3405,6 +3437,7 @@ static bool mpam_update_config(struct mpam_config *cfg,
maybe_update_config(cfg, mpam_feat_cpor_part, newcfg, cpbm, has_changes);
maybe_update_config(cfg, mpam_feat_mbw_part, newcfg, mbw_pbm, has_changes);
maybe_update_config(cfg, mpam_feat_mbw_max, newcfg, mbw_max, has_changes);
+ maybe_update_config(cfg, mpam_feat_max_limit, newcfg, max_limit, has_changes);
return has_changes;
}
diff --git a/drivers/resctrl/mpam_internal.h b/drivers/resctrl/mpam_internal.h
index 1f86cc158100..184fbc01fa91 100644
--- a/drivers/resctrl/mpam_internal.h
+++ b/drivers/resctrl/mpam_internal.h
@@ -234,6 +234,7 @@ enum mpam_device_features {
mpam_feat_msmon_mbwu_capture,
mpam_feat_msmon_mbwu_rwbw,
mpam_feat_partid_nrw,
+ mpam_feat_max_limit,
MPAM_FEATURE_LAST
};
@@ -243,6 +244,7 @@ struct mpam_props {
u16 cpbm_wd;
u16 mbw_pbm_bits;
u16 bwa_wd;
+ u8 max_lim;
u16 cmax_wd;
u16 cassoc_wd;
u16 intpri_wd;
@@ -364,6 +366,7 @@ struct mpam_config {
u32 cpbm;
u32 mbw_pbm;
u16 mbw_max;
+ bool max_limit;
struct mpam_garbage garbage;
};
@@ -628,6 +631,7 @@ int mpam_fb_check_shared_buffer_size(struct mpam_msc *msc);
/* MPAMF_MBW_IDR - MPAM features memory bandwidth partitioning ID register */
#define MPAMF_MBW_IDR_BWA_WD GENMASK(5, 0)
+#define MPAMF_MBW_IDR_MAX_LIM GENMASK(9, 8)
#define MPAMF_MBW_IDR_HAS_MIN BIT(10)
#define MPAMF_MBW_IDR_HAS_MAX BIT(11)
#define MPAMF_MBW_IDR_HAS_PBM BIT(12)
diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c
index 5468367776e2..53fcffb23984 100644
--- a/drivers/resctrl/mpam_resctrl.c
+++ b/drivers/resctrl/mpam_resctrl.c
@@ -1236,6 +1236,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res)
r->ctrl_scope = RESCTRL_L3_CACHE;
r->membw.delay_linear = true;
+ r->membw.max_is_soft_limit = (cprops->max_lim == 1);
r->membw.throttle_mode = THREAD_THROTTLE_UNDEFINED;
r->membw.min_bw = get_mba_min(cprops);
r->membw.max_bw = MAX_MBA_BW;
@@ -1481,6 +1482,13 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_ctrl_domain *d,
if (mpam_has_feature(mpam_feat_mbw_max, cprops)) {
cfg.mbw_max = percent_to_mbw_max(cfg_val, cprops);
mpam_set_feature(mpam_feat_mbw_max, &cfg);
+ /*
+ * Always carry the hard-limit bit: the control follows
+ * the software controller, so a hard limit left by an
+ * earlier mount is cleared by the next write.
+ */
+ mpam_set_feature(mpam_feat_max_limit, &cfg);
+ cfg.max_limit = r->membw.mba_sc;
break;
}
fallthrough;
--
2.34.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [RFC PATCH v2 8/9] x86,fs/resctrl: Let the architecture cap the RMIDs for the software controller
2026-11-12 13:12 [RFC PATCH v2 0/9] fs/resctrl: Support mba_MBps on MPAM Jinjie Ruan
` (6 preceding siblings ...)
2026-11-12 13:12 ` [RFC PATCH v2 7/9] arm_mpam: Program the hard limit for the software controller Jinjie Ruan
@ 2026-11-12 13:12 ` Jinjie Ruan
2026-11-12 13:12 ` [RFC PATCH v2 9/9] arm_mpam: Cap the CLOSID/RMID space by the bandwidth counters for mba_MBps Jinjie Ruan
8 siblings, 0 replies; 11+ messages in thread
From: Jinjie Ruan @ 2026-11-12 13:12 UTC (permalink / raw)
To: tony.luck, reinette.chatre, Dave.Martin, james.morse, babu.moger,
corbet, skhan, rdunlap, tglx, mingo, bp, dave.hansen, hpa,
ben.horgan, fenghuay, tan.shaopeng, quic_jiles, x86,
linux-kernel, linux-doc, linux-arm-kernel
Cc: ruanjinjie
The MBA software controller measures every monitor group. When the
architecture assigns each group a counter from a shared pool as the
group is created, the number of groups must not exceed the number of
counters while the controller is in use. resctrl has no way to learn
this from the architecture.
The rmid_idx space cannot change for this: rmid_ptrs[] is set up on
the first mount, and RMIDs in limbo outlive an unmount.
Add two hooks instead.
- resctrl_arch_set_mba_sc(). Called when the controller is enabled
or disabled, so the architecture can update what it reports
- resctrl_arch_rmid_valid(). Skips the RMIDs that arch refuses.
The hook runs before the number of CLOSIDs is read again, so the reset
of the bandwidth targets covers the CLOSIDs the controller may use.
The reset is skipped when the controller is disabled: the targets are
not read while it is off, and a domain that came online while it was
enabled may have allocated mbps_val for the capped, smaller number of
CLOSIDs.
Both hooks are no-ops on x86 and, for now, on MPAM.
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
---
arch/x86/include/asm/resctrl.h | 6 ++++++
fs/resctrl/monitor.c | 6 ++++--
fs/resctrl/rdtgroup.c | 25 ++++++++++++++++++++-----
include/linux/arm_mpam.h | 8 ++++++++
4 files changed, 38 insertions(+), 7 deletions(-)
diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h
index 8f6edcdcfd87..ac51b63f6dde 100644
--- a/arch/x86/include/asm/resctrl.h
+++ b/arch/x86/include/asm/resctrl.h
@@ -181,6 +181,12 @@ static inline u32 resctrl_arch_rmid_idx_encode(u32 ignored, u32 rmid)
return rmid;
}
+static inline void resctrl_arch_set_mba_sc(bool enable) { }
+static inline bool resctrl_arch_rmid_valid(u32 closid, u32 rmid)
+{
+ return true;
+}
+
/* x86 can always read an rmid, nothing needs allocating */
struct rdt_resource;
static inline void *resctrl_arch_mon_ctx_alloc(struct rdt_resource *r,
diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
index 5a67c8089615..bcd809083f18 100644
--- a/fs/resctrl/monitor.c
+++ b/fs/resctrl/monitor.c
@@ -216,12 +216,14 @@ static struct rmid_entry *resctrl_find_free_rmid(u32 closid)
* to be if it were used with this CLOSID.
* If the CLOSID is irrelevant on this architecture, the two
* index values are always the same on every entry and thus the
- * very first entry will be returned.
+ * very first entry will be returned. Skip the RMIDs the
+ * architecture does not allow.
*/
itr_idx = resctrl_arch_rmid_idx_encode(itr->closid, itr->rmid);
cmp_idx = resctrl_arch_rmid_idx_encode(closid, itr->rmid);
- if (itr_idx == cmp_idx)
+ if (itr_idx == cmp_idx &&
+ resctrl_arch_rmid_valid(closid, itr->rmid))
return itr;
}
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 9f10b6ceff8a..e186ae322f08 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -2640,16 +2640,20 @@ static int set_mba_sc(bool mba_sc)
{
struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
struct rdt_resource *rmbm = resctrl_arch_get_resource(RDT_RESOURCE_L3);
- u32 num_closid = resctrl_arch_get_num_closid(r);
struct rdt_ctrl_domain *d;
unsigned long fflags;
- int i;
if (!supports_mba_mbps() || mba_sc == is_mba_sc(r))
return -EINVAL;
r->membw.mba_sc = mba_sc;
+ /*
+ * The architecture may cap the groups to what the controller can
+ * measure. Do it before the number of CLOSIDs is used below.
+ */
+ resctrl_arch_set_mba_sc(mba_sc);
+
/*
* The controller needs its event counted in every monitoring group:
* force automatic assignment on, and restore the user's setting when
@@ -2666,9 +2670,20 @@ static int set_mba_sc(bool mba_sc)
rdtgroup_default.mba_mbps_event = mba_mbps_default_event;
- list_for_each_entry_rcu(d, &r->ctrl_domains, hdr.list, lockdep_is_cpus_held()) {
- for (i = 0; i < num_closid; i++)
- d->mbps_val[i] = MBA_MAX_MBPS;
+ /*
+ * Reset the targets of the CLOSIDs the controller uses. Skip this
+ * when disabling: nothing reads the targets while the controller is
+ * off, and a domain that came online while it was enabled may have
+ * allocated mbps_val for the capped, smaller number of CLOSIDs.
+ */
+ if (mba_sc) {
+ u32 num_closid = resctrl_arch_get_num_closid(r);
+ int i;
+
+ list_for_each_entry_rcu(d, &r->ctrl_domains, hdr.list, lockdep_is_cpus_held()) {
+ for (i = 0; i < num_closid; i++)
+ d->mbps_val[i] = MBA_MAX_MBPS;
+ }
}
fflags = mba_sc ? RFTYPE_CTRL_BASE | RFTYPE_MON_BASE : 0;
diff --git a/include/linux/arm_mpam.h b/include/linux/arm_mpam.h
index 002f56e15362..73a6277f688c 100644
--- a/include/linux/arm_mpam.h
+++ b/include/linux/arm_mpam.h
@@ -63,6 +63,14 @@ u32 resctrl_arch_rmid_idx_encode(u32 closid, u32 rmid);
void resctrl_arch_rmid_idx_decode(u32 idx, u32 *closid, u32 *rmid);
u32 resctrl_arch_system_num_rmid_idx(void);
+/* MPAM does not limit the RMIDs for the MBA software controller yet. */
+static inline void resctrl_arch_set_mba_sc(bool enable) { }
+
+static inline bool resctrl_arch_rmid_valid(u32 closid, u32 rmid)
+{
+ return true;
+}
+
struct rdt_resource;
void *resctrl_arch_mon_ctx_alloc(struct rdt_resource *r, enum resctrl_event_id evtid);
void resctrl_arch_mon_ctx_free(struct rdt_resource *r, enum resctrl_event_id evtid, void *ctx);
--
2.34.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [RFC PATCH v2 9/9] arm_mpam: Cap the CLOSID/RMID space by the bandwidth counters for mba_MBps
2026-11-12 13:12 [RFC PATCH v2 0/9] fs/resctrl: Support mba_MBps on MPAM Jinjie Ruan
` (7 preceding siblings ...)
2026-11-12 13:12 ` [RFC PATCH v2 8/9] x86,fs/resctrl: Let the architecture cap the RMIDs " Jinjie Ruan
@ 2026-11-12 13:12 ` Jinjie Ruan
8 siblings, 0 replies; 11+ messages in thread
From: Jinjie Ruan @ 2026-11-12 13:12 UTC (permalink / raw)
To: tony.luck, reinette.chatre, Dave.Martin, james.morse, babu.moger,
corbet, skhan, rdunlap, tglx, mingo, bp, dave.hansen, hpa,
ben.horgan, fenghuay, tan.shaopeng, quic_jiles, x86,
linux-kernel, linux-doc, linux-arm-kernel
Cc: ruanjinjie
MPAM assigns each monitor group a bandwidth counter from a shared pool
when the group is created, but the number of groups comes from the
PARTID and PMG spaces. Once the pool is empty, new groups get no
counter, their reads fail with -ENOENT, and the MBA software controller
cannot keep them below their target.
While the controller is enabled, cap the CLOSIDs and the RMIDs per
CLOSID to the number of counters, shrinking the RMIDs first and keeping
one per CLOSID for the default monitor group. CDP hides the MBA
resource, as MBW_MAX is configured per PARTID and the two PARTIDs of a
CLOSID would alias, so the controller cannot be enabled together with
CDP and the cap needs no CDP adjustment.
resctrl_arch_get_num_closid() reports the capped CLOSIDs,
resctrl_arch_rmid_valid() refuses the RMIDs above the cap, and
resctrl_arch_set_mba_sc() updates num_rmid and the limbo threshold.
Without mba_MBps nothing changes.
Suggested-by: Ben Horgan <ben.horgan@arm.com>
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
---
drivers/resctrl/mpam_resctrl.c | 48 +++++++++++++++++++++++++++++++++-
include/linux/arm_mpam.h | 10 ++-----
2 files changed, 49 insertions(+), 9 deletions(-)
diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c
index 53fcffb23984..1e23a9a41ea9 100644
--- a/drivers/resctrl/mpam_resctrl.c
+++ b/drivers/resctrl/mpam_resctrl.c
@@ -62,6 +62,14 @@ static DEFINE_MUTEX(domain_list_lock);
*/
static bool cdp_enabled;
+/*
+ * The CLOSIDs and the RMIDs per CLOSID resctrl may use while the MBA software
+ * controller is enabled. Set by resctrl_arch_set_mba_sc().
+ */
+static u32 mpam_capped_num_closid;
+static u32 mpam_capped_num_rmid;
+static bool mba_sc_enabled;
+
/*
* We use cacheinfo to discover the size of the caches and their id. cacheinfo
* populates this from a device_initcall(). mpam_resctrl_setup() must wait.
@@ -271,7 +279,7 @@ static bool mpam_resctrl_hide_cdp(enum resctrl_res_level rid)
*/
u32 resctrl_arch_get_num_closid(struct rdt_resource *ignored)
{
- return mpam_partid_max + 1;
+ return mba_sc_enabled ? mpam_capped_num_closid : mpam_partid_max + 1;
}
u32 resctrl_arch_system_num_rmid_idx(void)
@@ -290,6 +298,12 @@ void resctrl_arch_rmid_idx_decode(u32 idx, u32 *closid, u32 *rmid)
*rmid = idx % (mpam_pmg_max + 1);
}
+/* The CLOSID is already capped by resctrl_arch_get_num_closid(). */
+bool resctrl_arch_rmid_valid(u32 closid, u32 rmid)
+{
+ return !mba_sc_enabled || rmid < mpam_capped_num_rmid;
+}
+
void resctrl_arch_sched_in(struct task_struct *tsk)
{
lockdep_assert_preemption_disabled();
@@ -1154,6 +1168,38 @@ static void mpam_resctrl_pick_counters(void)
}
}
+/*
+ * The controller needs a bandwidth counter for every monitor group, so while
+ * it is enabled cap the CLOSIDs and the RMIDs per CLOSID to the number of
+ * counters, shrinking the RMIDs first. The number of monitor groups changes,
+ * and so does the limbo threshold. CDP hides the MBA resource, so it cannot
+ * be enabled together with the controller and needs no adjustment here.
+ */
+void resctrl_arch_set_mba_sc(bool enable)
+{
+ struct mpam_resctrl_mon *mon = &mpam_resctrl_counters[QOS_L3_MBM_TOTAL_EVENT_ID];
+ struct rdt_resource *l3 = &mpam_resctrl_controls[RDT_RESOURCE_L3].resctrl_res;
+ u32 num_closid = mpam_partid_max + 1;
+ u32 num_rmid = mpam_pmg_max + 1;
+
+ if (enable && mon->class) {
+ u32 num_mon = mon->class->props.num_mbwu_mon;
+
+ num_closid = min(num_closid, num_mon);
+ num_rmid = clamp(num_mon / num_closid, 1U, num_rmid);
+ }
+
+ mpam_capped_num_closid = num_closid;
+ mpam_capped_num_rmid = num_rmid;
+ mba_sc_enabled = enable;
+
+ l3->mon.num_rmid = num_closid * num_rmid;
+
+ if (resctrl_rmid_realloc_limit)
+ resctrl_rmid_realloc_threshold =
+ resctrl_rmid_realloc_limit / (num_closid * num_rmid);
+}
+
static void __config_cntr(struct mpam_resctrl_mon *mon, u32 cntr_id,
enum resctrl_conf_type cdp_type, u32 closid, u32 rmid,
bool assign)
diff --git a/include/linux/arm_mpam.h b/include/linux/arm_mpam.h
index 73a6277f688c..7c939a6f670b 100644
--- a/include/linux/arm_mpam.h
+++ b/include/linux/arm_mpam.h
@@ -62,14 +62,8 @@ bool resctrl_arch_match_rmid(struct task_struct *tsk, u32 closid, u32 rmid);
u32 resctrl_arch_rmid_idx_encode(u32 closid, u32 rmid);
void resctrl_arch_rmid_idx_decode(u32 idx, u32 *closid, u32 *rmid);
u32 resctrl_arch_system_num_rmid_idx(void);
-
-/* MPAM does not limit the RMIDs for the MBA software controller yet. */
-static inline void resctrl_arch_set_mba_sc(bool enable) { }
-
-static inline bool resctrl_arch_rmid_valid(u32 closid, u32 rmid)
-{
- return true;
-}
+void resctrl_arch_set_mba_sc(bool enable);
+bool resctrl_arch_rmid_valid(u32 closid, u32 rmid);
struct rdt_resource;
void *resctrl_arch_mon_ctx_alloc(struct rdt_resource *r, enum resctrl_event_id evtid);
--
2.34.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [RFC PATCH v2 5/9] fs/resctrl: Allow mba_MBps with assignable MBM counters
2026-11-12 13:12 ` [RFC PATCH v2 5/9] fs/resctrl: Allow mba_MBps with assignable MBM counters Jinjie Ruan
@ 2026-11-17 11:37 ` Ben Horgan
0 siblings, 0 replies; 11+ messages in thread
From: Ben Horgan @ 2026-11-17 11:37 UTC (permalink / raw)
To: Jinjie Ruan, tony.luck, reinette.chatre, Dave.Martin,
james.morse, babu.moger, corbet, skhan, rdunlap, tglx, mingo, bp,
dave.hansen, hpa, fenghuay, tan.shaopeng, quic_jiles, x86,
linux-kernel, linux-doc, linux-arm-kernel
Hi Jinjie,
On 12/11/2026 13:12, Jinjie Ruan wrote:
> The controller needs a counter for the event selected by mba_MBps_event
> in every monitoring group, so mba_MBps is refused when MBM counters are
> assignable. On MPAM a group's monitor is assigned from a fixed pool
> when the group is created, and users can otherwise only pick a
> percentage of a maximum that firmware does not report.
>
> Allow the controller when the architecture has a fixed assignment mode
> and assigns counters when a group is created. Keep mbm_assign_on_mkdir
> enabled while the controller is in use, restore its previous value
> when the controller is disabled, and refuse to unassign the
> controller's counter meanwhile.
I'm afraid there has been some miscommunication here. What I was trying
to suggest is that MPAM could when using the software controller use the
default mbm_assign_mode as is used in intel systems. All the counters
are allocated up front and there is one of each supported type per
monitor group. Using mbm_assign_on_mkdir doesn't always give you
guaranteed assigned counters as you point out in your
patch "arm_mpam: resctrl: Use the domain's counter ID". I haven't
thought through all the abi consideration but it seems plausible, what
do you think?
Thanks,
Ben
>
> Only MPAM has this mode; other systems keep the previous behavior.
>
> Suggested-by: Ben Horgan <ben.horgan@arm.com>
> Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
> ---
> Documentation/filesystems/resctrl.rst | 7 +++++--
> fs/resctrl/monitor.c | 25 ++++++++++++++++++++++
> fs/resctrl/rdtgroup.c | 30 +++++++++++++++++++++------
> 3 files changed, 54 insertions(+), 8 deletions(-)
>
> diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
> index b52795e03303..d70aec6f7096 100644
> --- a/Documentation/filesystems/resctrl.rst
> +++ b/Documentation/filesystems/resctrl.rst
> @@ -471,7 +471,8 @@ with the following files:
> Determines if a counter will automatically be assigned to an RMID, MBM event
> pair when its associated monitor group is created via mkdir. Enabled by default
> on boot, also when switched from "default" mode to "mbm_event" counter assignment
> - mode. Users can disable this capability by writing to the interface.
> + mode. Users can disable this capability by writing to the interface. While the
> + MBA software controller is enabled, auto assignment cannot be disabled.
>
> "0":
> Auto assignment is disabled.
> @@ -704,7 +705,9 @@ When monitoring is enabled all MON groups may also contain:
> mbm_total_bytes:0=e;1=e
> mbm_local_bytes:0=e;1=e
>
> - Assignments can be modified by writing to the interface.
> + Assignments can be modified by writing to the interface. While the
> + MBA software controller is enabled, the counter of the event the
> + controller uses cannot be unassigned.
>
> Examples:
>
> diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
> index 8f3b8c3e3010..5a67c8089615 100644
> --- a/fs/resctrl/monitor.c
> +++ b/fs/resctrl/monitor.c
> @@ -1210,6 +1210,12 @@ ssize_t resctrl_mbm_assign_on_mkdir_write(struct kernfs_open_file *of, char *buf
> goto out_unlock;
> }
>
> + if (!value && is_mba_sc(NULL)) {
> + rdt_last_cmd_puts("mbm_assign_on_mkdir is required by the software controller\n");
> + ret = -EBUSY;
> + goto out_unlock;
> + }
> +
> r->mon.mbm_assign_on_mkdir = value;
>
> out_unlock:
> @@ -1776,6 +1782,19 @@ static struct mon_evt *mbm_get_mon_event_by_name(struct rdt_resource *r, char *n
> return NULL;
> }
>
> +/*
> + * Return the event the MBA software controller uses as its input for @rdtgrp.
> + * A monitor group has no event of its own: its bandwidth is summed into the
> + * parent control group, which is the group with the mba_MBps_event setting.
> + */
> +static enum resctrl_event_id rdtgrp_mba_mbps_event(struct rdtgroup *rdtgrp)
> +{
> + if (rdtgrp->type == RDTMON_GROUP)
> + return rdtgrp->mon.parent->mba_mbps_event;
> +
> + return rdtgrp->mba_mbps_event;
> +}
> +
> static int rdtgroup_modify_assign_state(char *assign, struct rdt_l3_mon_domain *d,
> struct rdtgroup *rdtgrp, struct mon_evt *mevt)
> {
> @@ -1789,6 +1808,12 @@ static int rdtgroup_modify_assign_state(char *assign, struct rdt_l3_mon_domain *
> ret = rdtgroup_assign_cntr_event(d, rdtgrp, mevt);
> break;
> case '_':
> + if (is_mba_sc(NULL) &&
> + mevt->evtid == rdtgrp_mba_mbps_event(rdtgrp)) {
> + rdt_last_cmd_puts("Counter is in use by the software controller\n");
> + ret = -EBUSY;
> + break;
> + }
> rdtgroup_unassign_cntr_event(d, rdtgrp, mevt);
> break;
> default:
> diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
> index b32831871f70..518d89b0c67d 100644
> --- a/fs/resctrl/rdtgroup.c
> +++ b/fs/resctrl/rdtgroup.c
> @@ -2610,10 +2610,9 @@ static void mba_sc_domain_destroy(struct rdt_resource *r,
> * The MBA software controller is supported only if MBM is supported and MBA is
> * in linear scale, and the MBM monitor scope is the same as MBA control scope.
> *
> - * The software controller cannot be supported when the MBM counters are
> - * assignable. There is no guarantee that MBM counters are assigned to the
> - * event backing the software controller in all monitoring domains of all
> - * monitoring groups.
> + * The controller needs a counter for its event in every monitoring group.
> + * With assignable counters that requires a fixed assignment mode, with
> + * counters assigned automatically while the controller is in use.
> */
> static bool supports_mba_mbps(void)
> {
> @@ -2623,9 +2622,13 @@ static bool supports_mba_mbps(void)
> return (resctrl_is_mbm_enabled() &&
> r->alloc_capable && is_mba_linear() &&
> r->ctrl_scope == rmbm->mon_scope &&
> - !rmbm->mon.mbm_cntr_assignable);
> + (!rmbm->mon.mbm_cntr_assignable ||
> + rmbm->mon.mbm_cntr_assign_fixed));
> }
>
> +/* mbm_assign_on_mkdir before the software controller forced it on. */
> +static bool mba_sc_saved_assign_on_mkdir;
> +
> /*
> * Enable or disable the MBA software controller
> * which helps user specify bandwidth in MBps.
> @@ -2633,6 +2636,7 @@ static bool supports_mba_mbps(void)
> static int set_mba_sc(bool mba_sc)
> {
> struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_MBA);
> + struct rdt_resource *rmbm = resctrl_arch_get_resource(RDT_RESOURCE_L3);
> u32 num_closid = resctrl_arch_get_num_closid(r);
> struct rdt_ctrl_domain *d;
> unsigned long fflags;
> @@ -2643,6 +2647,20 @@ static int set_mba_sc(bool mba_sc)
>
> r->membw.mba_sc = mba_sc;
>
> + /*
> + * The controller needs its event counted in every monitoring group:
> + * force automatic assignment on, and restore the user's setting when
> + * the controller is disabled.
> + */
> + if (rmbm->mon.mbm_cntr_assignable) {
> + if (mba_sc) {
> + mba_sc_saved_assign_on_mkdir = rmbm->mon.mbm_assign_on_mkdir;
> + rmbm->mon.mbm_assign_on_mkdir = true;
> + } else {
> + rmbm->mon.mbm_assign_on_mkdir = mba_sc_saved_assign_on_mkdir;
> + }
> + }
> +
> rdtgroup_default.mba_mbps_event = mba_mbps_default_event;
>
> list_for_each_entry_rcu(d, &r->ctrl_domains, hdr.list, lockdep_is_cpus_held()) {
> @@ -3352,7 +3370,7 @@ static int rdt_parse_param(struct fs_context *fc, struct fs_parameter *param)
> ctx->enable_cdpl2 = true;
> return 0;
> case Opt_mba_mbps:
> - msg = "mba_MBps requires MBM (mbm_event mode not supported) and linear scale MBA at L3 scope";
> + msg = "mba_MBps requires MBM and linear scale MBA at L3 scope";
> if (!supports_mba_mbps())
> return invalfc(fc, msg);
> ctx->enable_mba_mbps = true;
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2026-11-17 11:38 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-12 13:12 [RFC PATCH v2 0/9] fs/resctrl: Support mba_MBps on MPAM Jinjie Ruan
2026-11-12 13:12 ` [RFC PATCH v2 1/9] fs/resctrl: Extract the software controller's step decision Jinjie Ruan
2026-11-12 13:12 ` [RFC PATCH v2 2/9] fs/resctrl: Avoid division by zero in the MBA software controller Jinjie Ruan
2026-11-12 13:12 ` [RFC PATCH v2 3/9] fs/resctrl: Do not update bandwidth from a failed counter read Jinjie Ruan
2026-11-12 13:12 ` [RFC PATCH v2 4/9] fs/resctrl: Take the CPU hotplug lock for software controller updates Jinjie Ruan
2026-11-12 13:12 ` [RFC PATCH v2 5/9] fs/resctrl: Allow mba_MBps with assignable MBM counters Jinjie Ruan
2026-11-17 11:37 ` Ben Horgan
2026-11-12 13:12 ` [RFC PATCH v2 6/9] fs/resctrl: Reject mba_MBps when the maximum is only a soft limit Jinjie Ruan
2026-11-12 13:12 ` [RFC PATCH v2 7/9] arm_mpam: Program the hard limit for the software controller Jinjie Ruan
2026-11-12 13:12 ` [RFC PATCH v2 8/9] x86,fs/resctrl: Let the architecture cap the RMIDs " Jinjie Ruan
2026-11-12 13:12 ` [RFC PATCH v2 9/9] arm_mpam: Cap the CLOSID/RMID space by the bandwidth counters for mba_MBps Jinjie Ruan
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®