* [PATCH v9 0/9] arm_mpam: Introduce Narrow-PARTID feature
@ 2026-10-12 8:23 Jinjie Ruan
2026-10-12 8:23 ` [PATCH v9 1/9] fs/resctrl: Unmount resctrl with the CDP state it was mounted with Jinjie Ruan
` (9 more replies)
0 siblings, 10 replies; 12+ messages in thread
From: Jinjie Ruan @ 2026-10-12 8:23 UTC (permalink / raw)
To: tony.luck, reinette.chatre, Dave.Martin, james.morse, babu.moger,
tglx, mingo, bp, dave.hansen, hpa, ben.horgan, fenghuay, x86,
linux-kernel
Cc: ruanjinjie
Background
==========
In ARM MPAM, a monitoring group is defined by a (PARTID, PMG) pair.
However, many production platforms implement only a single PMG bit,
limiting the total number of monitoring groups long before the request
PARTID space is exhausted.
Hardware Mechanism
===================
Platforms with "PARTID narrowing" mitigate this by mapping multiple
request PARTIDs onto a single internal PARTID. The internal PARTID
dictates the control group configuration, while each individual request
PARTID functions as an independent monitoring group. This decouples
control and monitoring allocation, allowing more monitoring groups
than the PMG constraint normally permits.
Software Strategy
==================
Using request PARTIDs beyond the internal range is only safe when no MPAM
class loses a non-aliasing control. While aliasing controls (e.g., cache
portion bitmaps) can be safely replicated across request PARTIDs,
non-aliasing controls (e.g., cache capacity) are additive and cannot.
To ensure safety, this series implements a completely static grouping
policy evaluated once before resctrl_init(). If any class contains
non-aliasing controls without narrowing capabilities, the expansion
is bypassed. This infrastructure remains completely transparent
to user-space.
Patch Architecture
====================
- Patches 1-4: Bugfix and preparatory rework.
- Patch 5: Track both internal and request PARTID limits per class.
- Patches 6-7: Implement driver-side modulo interleaving for request
PARTIDs and flat global RMID indexing.
- Patches 8-9: Identify non-aliasing controls, and enable the narrowing
expansion by lifting driver-side partid_max clamping and freeing
the extra request PARTIDs for resctrl.
Changes since v8
- Replaced dynamic allocation with pure static grouping as requested by James.
- Removed the 'mpam.intpartid_max' boot parameter; expansion is now automatic.
- Extended safety verification to all classes, preserving
forward-compatibility for unknown classes that might gain resctrl
schemas later.
- Decoupled the driver infrastructure patch from the resctrl glue patch.
- Introduced resctrl_arch_rmid_idx_match() to clean up the RMID allocator.
- Link to v8: https://lore.kernel.org/all/20260413085405.1166412-1-zengheng4@huawei.com/
Previous Versions
=================
v7: https://lore.kernel.org/all/20260317132141.1272506-1-zengheng4@huawei.com/
v6: https://lore.kernel.org/all/20250222112448.2438586-1-zengheng4@huawei.com/
v5: https://lore.kernel.org/all/20250217031852.2014939-1-zengheng4@huawei.com/
RFC-v4: https://lore.kernel.org/all/20250104101224.873926-1-zengheng4@huawei.com/
RFC-v3: https://lore.kernel.org/all/20241207092136.2488426-1-zengheng4@huawei.com/
RFC-v2: https://lore.kernel.org/all/20241119135104.595630-1-zengheng4@huawei.com/
RFC-v1: https://lore.kernel.org/all/20241114135037.918470-1-zengheng4@huawei.com/
Jinjie Ruan (6):
fs/resctrl: Add resctrl_arch_rmid_idx_match() helper
arm_mpam: resctrl: Add resctrl_closid_rmid_to_partid_pmg() helper
arm_mpam: Add a helper to check a PARTID is in a CLOSID
arm_mpam: Track the PARTID and internal PARTID limits per class
arm_mpam: Identify the non-aliasing controls
arm_mpam: Use the request PARTIDs beyond the internal range
Zeng Heng (3):
fs/resctrl: Unmount resctrl with the CDP state it was mounted with
arm_mpam: Divide the PARTIDs between the control groups
arm_mpam: Map resctrl RMIDs onto the request PARTIDs
arch/x86/include/asm/resctrl.h | 6 +
drivers/resctrl/mpam_devices.c | 105 +++++++++++++---
drivers/resctrl/mpam_internal.h | 24 ++++
drivers/resctrl/mpam_resctrl.c | 213 +++++++++++++++++++++++++++-----
fs/resctrl/monitor.c | 13 +-
fs/resctrl/rdtgroup.c | 38 +++++-
include/linux/arm_mpam.h | 1 +
7 files changed, 341 insertions(+), 59 deletions(-)
--
2.34.1
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v9 1/9] fs/resctrl: Unmount resctrl with the CDP state it was mounted with
2026-10-12 8:23 [PATCH v9 0/9] arm_mpam: Introduce Narrow-PARTID feature Jinjie Ruan
@ 2026-10-12 8:23 ` Jinjie Ruan
2026-10-12 8:23 ` [PATCH v9 2/9] fs/resctrl: Add resctrl_arch_rmid_idx_match() helper Jinjie Ruan
` (8 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Jinjie Ruan @ 2026-10-12 8:23 UTC (permalink / raw)
To: tony.luck, reinette.chatre, Dave.Martin, james.morse, babu.moger,
tglx, mingo, bp, dave.hansen, hpa, ben.horgan, fenghuay, x86,
linux-kernel
Cc: ruanjinjie
From: Zeng Heng <zengheng4@huawei.com>
resctrl disables CDP before the control and monitor groups are torn down
on unmount. The architecture translates between CLOSID and PARTID using
the CDP state, so the RMIDs freed by rmdir_all_sub() and
resctrl_fs_teardown() are parsed with the wrong mapping. On arm64 the
RMID limbo handler can still be running, and a CLOSID left in the limbo
list is re-interpreted with the CDP state of the next mount. Enabling
CDP doubles the CLOSID on arm64, which can push the stale value out of
range and take an MPAM error interrupt.
Disable CDP after the groups have been torn down, so every CLOSID freed
on unmount is parsed with the CDP state it was allocated under. Return
the RMIDs still in limbo to the free list and stop the worker, so
nothing carries over to the next mount.
Cc: Tony Luck <tony.luck@intel.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Dave Martin <Dave.Martin@arm.com>
Cc: James Morse <james.morse@arm.com>
Cc: Babu Moger <babu.moger@amd.com>
Signed-off-by: Zeng Heng <zengheng4@huawei.com>
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
---
fs/resctrl/rdtgroup.c | 38 ++++++++++++++++++++++++++++++++++++--
1 file changed, 36 insertions(+), 2 deletions(-)
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 5dcbb0a964e8..6778a7ef2e95 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -3145,6 +3145,33 @@ static void resctrl_fs_teardown(void)
rdtgroup_destroy_root();
}
+/*
+ * The RMID limbo handler keeps running after unmount. Any CLOSID still in
+ * limbo would be re-interpreted with the CDP state of the next mount. MPAM
+ * doubles CLOSID when CDP is enabled, which can push the stale value out of
+ * range. Force the limbo list back to the free list before the CLOSID state
+ * is torn down.
+ */
+static void rdt_flush_limbo(void)
+{
+ struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
+ struct rdt_l3_mon_domain *d;
+
+ if (!IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID))
+ return;
+
+ if (!resctrl_is_mon_event_enabled(QOS_L3_OCCUP_EVENT_ID))
+ return;
+
+ list_for_each_entry(d, &r->mon_domains, hdr.list) {
+ if (!has_busy_rmid(d))
+ continue;
+
+ cancel_delayed_work(&d->cqm_limbo);
+ __check_limbo(d, true);
+ }
+}
+
static void resctrl_unmount(void)
{
struct rdt_resource *r;
@@ -3152,13 +3179,20 @@ static void resctrl_unmount(void)
cpus_read_lock();
mutex_lock(&rdtgroup_mutex);
- rdt_disable_ctx();
-
/* Put everything back to default values. */
for_each_alloc_capable_rdt_resource(r)
resctrl_arch_reset_all_ctrls(r);
resctrl_fs_teardown();
+ rdt_flush_limbo();
+
+ /*
+ * Clear the CDP state last. free_rmid() translates between CLOSID and
+ * PARTID using the CDP state; clearing it first would make the
+ * translation inconsistent for the RMIDs being torn down.
+ */
+ rdt_disable_ctx();
+
if (resctrl_arch_alloc_capable())
resctrl_arch_disable_alloc();
if (resctrl_arch_mon_capable())
--
2.34.1
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v9 2/9] fs/resctrl: Add resctrl_arch_rmid_idx_match() helper
2026-10-12 8:23 [PATCH v9 0/9] arm_mpam: Introduce Narrow-PARTID feature Jinjie Ruan
2026-10-12 8:23 ` [PATCH v9 1/9] fs/resctrl: Unmount resctrl with the CDP state it was mounted with Jinjie Ruan
@ 2026-10-12 8:23 ` Jinjie Ruan
2026-10-12 8:23 ` [PATCH v9 3/9] arm_mpam: resctrl: Add resctrl_closid_rmid_to_partid_pmg() helper Jinjie Ruan
` (7 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Jinjie Ruan @ 2026-10-12 8:23 UTC (permalink / raw)
To: tony.luck, reinette.chatre, Dave.Martin, james.morse, babu.moger,
tglx, mingo, bp, dave.hansen, hpa, ben.horgan, fenghuay, x86,
linux-kernel
Cc: ruanjinjie
The current RMID allocator assumes that RMID and CLOSID are entirely
independent or follow a rigid 2D encoding. This prevents architectures
where an RMID is bound to a specific CLOSID from properly filtering
free RMIDs.
Introduce resctrl_arch_rmid_idx_match() to allow architectures to define
their own compatibility rules between a target CLOSID and an allocated
RMID's CLOSID.
No functional change.
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Dave Martin <Dave.Martin@arm.com>
Cc: James Morse <james.morse@arm.com>
Cc: Babu Moger <babu.moger@amd.com>
Cc: Thomas Gleixner <tglx@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Ben Horgan <ben.horgan@arm.com>
Cc: Fenghua Yu <fenghuay@nvidia.com>
Suggested-by: James Morse <james.morse@arm.com>
Signed-off-by: Zeng Heng <zengheng4@huawei.com>
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
---
arch/x86/include/asm/resctrl.h | 6 ++++++
drivers/resctrl/mpam_resctrl.c | 6 ++++++
fs/resctrl/monitor.c | 13 +++----------
include/linux/arm_mpam.h | 1 +
4 files changed, 16 insertions(+), 10 deletions(-)
diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h
index 8f6edcdcfd87..7bb297a0ab6e 100644
--- a/arch/x86/include/asm/resctrl.h
+++ b/arch/x86/include/asm/resctrl.h
@@ -181,6 +181,12 @@ static inline u32 resctrl_arch_rmid_idx_encode(u32 ignored, u32 rmid)
return rmid;
}
+/* x86 RMIDs are independent; any RMID can be used with any CLOSID. */
+static inline bool resctrl_arch_rmid_idx_match(u32 closid, u32 rmid_closid)
+{
+ return true;
+}
+
/* x86 can always read an rmid, nothing needs allocating */
struct rdt_resource;
static inline void *resctrl_arch_mon_ctx_alloc(struct rdt_resource *r,
diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c
index 9d223057953a..385c745787f2 100644
--- a/drivers/resctrl/mpam_resctrl.c
+++ b/drivers/resctrl/mpam_resctrl.c
@@ -279,6 +279,12 @@ u32 resctrl_arch_rmid_idx_encode(u32 closid, u32 rmid)
return closid * (mpam_pmg_max + 1) + rmid;
}
+/* Check if the RMID's associated closid matches the target CLOSID. */
+bool resctrl_arch_rmid_idx_match(u32 closid, u32 rmid_closid)
+{
+ return closid == rmid_closid;
+}
+
void resctrl_arch_rmid_idx_decode(u32 idx, u32 *closid, u32 *rmid)
{
*closid = idx / (mpam_pmg_max + 1);
diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
index 73413cb128ea..39908f67dea4 100644
--- a/fs/resctrl/monitor.c
+++ b/fs/resctrl/monitor.c
@@ -205,23 +205,16 @@ bool has_busy_rmid(struct rdt_l3_mon_domain *d)
static struct rmid_entry *resctrl_find_free_rmid(u32 closid)
{
struct rmid_entry *itr;
- u32 itr_idx, cmp_idx;
if (list_empty(&rmid_free_lru))
return rmid_limbo_count ? ERR_PTR(-EBUSY) : ERR_PTR(-ENOSPC);
list_for_each_entry(itr, &rmid_free_lru, list) {
/*
- * Get the index of this free RMID, and the index it would need
- * to be if it were used with this CLOSID.
- * If the CLOSID is irrelevant on this architecture, the two
- * index values are always the same on every entry and thus the
- * very first entry will be returned.
+ * Check if the free RMID matches the target CLOSID.
+ * Architecture-specific rules apply.
*/
- itr_idx = resctrl_arch_rmid_idx_encode(itr->closid, itr->rmid);
- cmp_idx = resctrl_arch_rmid_idx_encode(closid, itr->rmid);
-
- if (itr_idx == cmp_idx)
+ if (resctrl_arch_rmid_idx_match(closid, itr->closid))
return itr;
}
diff --git a/include/linux/arm_mpam.h b/include/linux/arm_mpam.h
index f92a36187a52..0bc8049a3733 100644
--- a/include/linux/arm_mpam.h
+++ b/include/linux/arm_mpam.h
@@ -60,6 +60,7 @@ void resctrl_arch_sched_in(struct task_struct *tsk);
bool resctrl_arch_match_closid(struct task_struct *tsk, u32 closid);
bool resctrl_arch_match_rmid(struct task_struct *tsk, u32 closid, u32 rmid);
u32 resctrl_arch_rmid_idx_encode(u32 closid, u32 rmid);
+bool resctrl_arch_rmid_idx_match(u32 closid, u32 rmid_closid);
void resctrl_arch_rmid_idx_decode(u32 idx, u32 *closid, u32 *rmid);
u32 resctrl_arch_system_num_rmid_idx(void);
--
2.34.1
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v9 3/9] arm_mpam: resctrl: Add resctrl_closid_rmid_to_partid_pmg() helper
2026-10-12 8:23 [PATCH v9 0/9] arm_mpam: Introduce Narrow-PARTID feature Jinjie Ruan
2026-10-12 8:23 ` [PATCH v9 1/9] fs/resctrl: Unmount resctrl with the CDP state it was mounted with Jinjie Ruan
2026-10-12 8:23 ` [PATCH v9 2/9] fs/resctrl: Add resctrl_arch_rmid_idx_match() helper Jinjie Ruan
@ 2026-10-12 8:23 ` Jinjie Ruan
2026-10-12 8:23 ` [PATCH v9 4/9] arm_mpam: Add a helper to check a PARTID is in a CLOSID Jinjie Ruan
` (6 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Jinjie Ruan @ 2026-10-12 8:23 UTC (permalink / raw)
To: tony.luck, reinette.chatre, Dave.Martin, james.morse, babu.moger,
tglx, mingo, bp, dave.hansen, hpa, ben.horgan, fenghuay, x86,
linux-kernel
Cc: ruanjinjie
The translation from resctrl (CLOSID, RMID) to MPAM (PARTID, PMG)
is currently open-coded across multiple paths (CPU default settings,
task setters, matching logic, and monitor readers). This duplicates
the special-case logic for CDP handling in multiple places.
Consolidate this translation into a single helper. This not only
eliminates duplication but also provides a unified checkpoint to support
non-linear partitioning mappings (such as PARTID narrowing) in subsequent
patches.
No functional change.
Cc: James Morse <james.morse@arm.com>
Cc: Ben Horgan <ben.horgan@arm.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Fenghua Yu <fenghuay@nvidia.com>
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
---
drivers/resctrl/mpam_resctrl.c | 64 ++++++++++++++++++++--------------
1 file changed, 38 insertions(+), 26 deletions(-)
diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c
index 385c745787f2..dbafa2a7586d 100644
--- a/drivers/resctrl/mpam_resctrl.c
+++ b/drivers/resctrl/mpam_resctrl.c
@@ -298,25 +298,37 @@ void resctrl_arch_sched_in(struct task_struct *tsk)
mpam_thread_switch(tsk);
}
-void resctrl_arch_set_cpu_default_closid_rmid(int cpu, u32 closid, u32 rmid)
+static void resctrl_closid_rmid_to_partid_pmg(u32 closid, u32 rmid,
+ u32 *partid_d, u32 *partid_i,
+ u8 *pmg_d, u8 *pmg_i)
{
WARN_ON_ONCE(closid > U16_MAX);
WARN_ON_ONCE(rmid > U8_MAX);
if (!cdp_enabled) {
- mpam_set_cpu_defaults(cpu, closid, closid, rmid, rmid);
+ *partid_d = *partid_i = closid;
+ *pmg_d = *pmg_i = rmid;
} else {
/*
* When CDP is enabled, resctrl halves the closid range and we
* use odd/even partid for one closid.
*/
- u32 partid_d = resctrl_get_config_index(closid, CDP_DATA);
- u32 partid_i = resctrl_get_config_index(closid, CDP_CODE);
-
- mpam_set_cpu_defaults(cpu, partid_d, partid_i, rmid, rmid);
+ *partid_d = resctrl_get_config_index(closid, CDP_DATA);
+ *partid_i = resctrl_get_config_index(closid, CDP_CODE);
+ *pmg_d = *pmg_i = rmid;
}
}
+void resctrl_arch_set_cpu_default_closid_rmid(int cpu, u32 closid, u32 rmid)
+{
+ u32 partid_d, partid_i;
+ u8 pmg_d, pmg_i;
+
+ resctrl_closid_rmid_to_partid_pmg(closid, rmid, &partid_d, &partid_i,
+ &pmg_d, &pmg_i);
+ mpam_set_cpu_defaults(cpu, partid_d, partid_i, pmg_d, pmg_i);
+}
+
void resctrl_arch_sync_cpu_closid_rmid(void *info)
{
struct resctrl_cpu_defaults *r = info;
@@ -333,17 +345,12 @@ void resctrl_arch_sync_cpu_closid_rmid(void *info)
void resctrl_arch_set_closid_rmid(struct task_struct *tsk, u32 closid, u32 rmid)
{
- WARN_ON_ONCE(closid > U16_MAX);
- WARN_ON_ONCE(rmid > U8_MAX);
+ u32 partid_d, partid_i;
+ u8 pmg_d, pmg_i;
- if (!cdp_enabled) {
- mpam_set_task_partid_pmg(tsk, closid, closid, rmid, rmid);
- } else {
- u32 partid_d = resctrl_get_config_index(closid, CDP_DATA);
- u32 partid_i = resctrl_get_config_index(closid, CDP_CODE);
-
- mpam_set_task_partid_pmg(tsk, partid_d, partid_i, rmid, rmid);
- }
+ resctrl_closid_rmid_to_partid_pmg(closid, rmid, &partid_d, &partid_i,
+ &pmg_d, &pmg_i);
+ mpam_set_task_partid_pmg(tsk, partid_d, partid_i, pmg_d, pmg_i);
}
bool resctrl_arch_match_closid(struct task_struct *tsk, u32 closid)
@@ -361,13 +368,15 @@ bool resctrl_arch_match_closid(struct task_struct *tsk, u32 closid)
bool resctrl_arch_match_rmid(struct task_struct *tsk, u32 closid, u32 rmid)
{
u64 regval = mpam_get_regval(tsk);
- u32 tsk_closid = FIELD_GET(MPAM0_EL1_PARTID_D, regval);
- u32 tsk_rmid = FIELD_GET(MPAM0_EL1_PMG_D, regval);
+ u32 tsk_partid = FIELD_GET(MPAM0_EL1_PARTID_D, regval);
+ u32 tsk_pmg = FIELD_GET(MPAM0_EL1_PMG_D, regval);
+ u32 partid_d, partid_i;
+ u8 pmg_d, pmg_i;
- if (cdp_enabled)
- tsk_closid >>= 1;
+ resctrl_closid_rmid_to_partid_pmg(closid, rmid, &partid_d, &partid_i,
+ &pmg_d, &pmg_i);
- return (tsk_closid == closid) && (tsk_rmid == rmid);
+ return (tsk_partid == partid_d) && (tsk_pmg == pmg_d);
}
struct rdt_resource *resctrl_arch_get_resource(enum resctrl_res_level l)
@@ -454,16 +463,19 @@ static int __read_mon(struct mpam_resctrl_mon *mon, struct mpam_component *mon_c
int mon_idx,
enum resctrl_conf_type cdp_type, u32 closid, u32 rmid, u64 *val)
{
+ u32 partid, partid_d, partid_i;
struct mon_cfg cfg;
+ u8 pmg_d, pmg_i;
if (!mpam_is_enabled())
return -EINVAL;
- /* Shift closid to account for CDP */
- closid = resctrl_get_config_index(closid, cdp_type);
+ resctrl_closid_rmid_to_partid_pmg(closid, rmid, &partid_d, &partid_i,
+ &pmg_d, &pmg_i);
+ partid = (cdp_type == CDP_CODE) ? partid_i : partid_d;
if (mon_idx == USE_PRE_ALLOCATED) {
- int mbwu_idx = resctrl_arch_rmid_idx_encode(closid, rmid);
+ int mbwu_idx = resctrl_arch_rmid_idx_encode(partid, rmid);
mon_idx = mon->mbwu_idx_to_mon[mbwu_idx];
if (mon_idx == -1)
@@ -478,8 +490,8 @@ static int __read_mon(struct mpam_resctrl_mon *mon, struct mpam_component *mon_c
cfg = (struct mon_cfg) {
.mon = mon_idx,
.match_pmg = true,
- .partid = closid,
- .pmg = rmid,
+ .partid = partid,
+ .pmg = pmg_d,
};
return mpam_msmon_read(mon_comp, &cfg, mon_type, val);
--
2.34.1
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v9 4/9] arm_mpam: Add a helper to check a PARTID is in a CLOSID
2026-10-12 8:23 [PATCH v9 0/9] arm_mpam: Introduce Narrow-PARTID feature Jinjie Ruan
` (2 preceding siblings ...)
2026-10-12 8:23 ` [PATCH v9 3/9] arm_mpam: resctrl: Add resctrl_closid_rmid_to_partid_pmg() helper Jinjie Ruan
@ 2026-10-12 8:23 ` Jinjie Ruan
2026-10-12 8:23 ` [PATCH v9 5/9] arm_mpam: Track the PARTID and internal PARTID limits per class Jinjie Ruan
` (5 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Jinjie Ruan @ 2026-10-12 8:23 UTC (permalink / raw)
To: tony.luck, reinette.chatre, Dave.Martin, james.morse, babu.moger,
tglx, mingo, bp, dave.hansen, hpa, ben.horgan, fenghuay, x86,
linux-kernel
Cc: ruanjinjie
match_closid() open codes the CDP special case that turns the task
PARTID into the CLOSID it belongs to. Move it into
resctrl_partid_in_closid(), so the question "does this PARTID belong to
this control group" is answered in one place.
No functional change.
Cc: James Morse <james.morse@arm.com>
Cc: Ben Horgan <ben.horgan@arm.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Fenghua Yu <fenghuay@nvidia.com>
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
---
drivers/resctrl/mpam_resctrl.c | 16 +++++++++++-----
1 file changed, 11 insertions(+), 5 deletions(-)
diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c
index dbafa2a7586d..0edededdf7a5 100644
--- a/drivers/resctrl/mpam_resctrl.c
+++ b/drivers/resctrl/mpam_resctrl.c
@@ -353,15 +353,21 @@ void resctrl_arch_set_closid_rmid(struct task_struct *tsk, u32 closid, u32 rmid)
mpam_set_task_partid_pmg(tsk, partid_d, partid_i, pmg_d, pmg_i);
}
+/* Does the request PARTID belong to this control group? */
+static bool resctrl_partid_in_closid(u16 partid, u32 closid)
+{
+ if (cdp_enabled)
+ partid >>= 1;
+
+ return partid == closid;
+}
+
bool resctrl_arch_match_closid(struct task_struct *tsk, u32 closid)
{
u64 regval = mpam_get_regval(tsk);
- u32 tsk_closid = FIELD_GET(MPAM0_EL1_PARTID_D, regval);
-
- if (cdp_enabled)
- tsk_closid >>= 1;
+ u32 tsk_partid = FIELD_GET(MPAM0_EL1_PARTID_D, regval);
- return tsk_closid == closid;
+ return resctrl_partid_in_closid(tsk_partid, closid);
}
/* The task's pmg is not unique, the partid must be considered too */
--
2.34.1
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v9 5/9] arm_mpam: Track the PARTID and internal PARTID limits per class
2026-10-12 8:23 [PATCH v9 0/9] arm_mpam: Introduce Narrow-PARTID feature Jinjie Ruan
` (3 preceding siblings ...)
2026-10-12 8:23 ` [PATCH v9 4/9] arm_mpam: Add a helper to check a PARTID is in a CLOSID Jinjie Ruan
@ 2026-10-12 8:23 ` Jinjie Ruan
2026-10-12 8:23 ` [PATCH v9 6/9] arm_mpam: Divide the PARTIDs between the control groups Jinjie Ruan
` (4 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Jinjie Ruan @ 2026-10-12 8:23 UTC (permalink / raw)
To: tony.luck, reinette.chatre, Dave.Martin, james.morse, babu.moger,
tglx, mingo, bp, dave.hansen, hpa, ben.horgan, fenghuay, x86,
linux-kernel
Cc: ruanjinjie
The number of PARTIDs and PMGs a resource supports is global, and the
number of internal PARTIDs is not tracked at all. Tracking this per
class lets the resctrl glue code find the limits of the classes it will
use, and pick the number of control groups before it is initialised.
Resources that implement PARTID narrowing reduce msc->partid_max,
because they don't have one configuration per PARTID. Record the
internal PARTID limit alongside partid_max in struct mpam_props, and
take the smaller of the values a class's RIS advertise.
Cc: James Morse <james.morse@arm.com>
Cc: Ben Horgan <ben.horgan@arm.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Fenghua Yu <fenghuay@nvidia.com>
Suggested-by: James Morse <james.morse@arm.com>
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
---
drivers/resctrl/mpam_devices.c | 23 ++++++++++++++---------
drivers/resctrl/mpam_internal.h | 2 ++
2 files changed, 16 insertions(+), 9 deletions(-)
diff --git a/drivers/resctrl/mpam_devices.c b/drivers/resctrl/mpam_devices.c
index dd422c56fbb1..029f105d5915 100644
--- a/drivers/resctrl/mpam_devices.c
+++ b/drivers/resctrl/mpam_devices.c
@@ -793,7 +793,6 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)
struct mpam_msc *msc = ris->vmsc->msc;
struct device *dev = &msc->pdev->dev;
struct mpam_props *props = &ris->props;
- struct mpam_class *class = ris->vmsc->comp->class;
lockdep_assert_held(&msc->probe_lock);
lockdep_assert_held(&msc->part_sel_lock);
@@ -938,18 +937,20 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)
}
/*
- * RIS with PARTID narrowing don't have enough storage for one
- * configuration per PARTID. If these are in a class we could use,
- * reduce the supported partid_max to match the number of intpartid.
- * If the class is unknown, just ignore it.
+ * A RIS that implements PARTID narrowing has one configuration per
+ * internal PARTID, and several request PARTIDs share it. Record both
+ * limits in the resource properties.
*/
- if (FIELD_GET(MPAMF_IDR_HAS_PARTID_NRW, ris->idr) &&
- class->type != MPAM_CLASS_UNKNOWN) {
+ props->partid_max = msc->partid_max;
+ props->intpartid_max = msc->partid_max;
+
+ if (FIELD_GET(MPAMF_IDR_HAS_PARTID_NRW, ris->idr)) {
u32 nrwidr = mpam_read_partsel_reg(msc, PARTID_NRW_IDR);
- u16 partid_max = FIELD_GET(MPAMF_PARTID_NRW_IDR_INTPARTID_MAX, nrwidr);
+ u16 intpartid_max = FIELD_GET(MPAMF_PARTID_NRW_IDR_INTPARTID_MAX, nrwidr);
mpam_set_feature(mpam_feat_partid_nrw, props);
- msc->partid_max = min(msc->partid_max, partid_max);
+ props->intpartid_max = min(props->intpartid_max, intpartid_max);
+ msc->partid_max = min(msc->partid_max, intpartid_max);
}
}
@@ -2259,6 +2260,10 @@ static void __props_mismatch(struct mpam_props *parent,
child->num_mbwu_mon);
}
+ /* Take the smallest, so the result is safe for every RIS in the class. */
+ parent->partid_max = min(parent->partid_max, child->partid_max);
+ parent->intpartid_max = min(parent->intpartid_max, child->intpartid_max);
+
if (CAN_MERGE_FEAT(parent, child, mpam_feat_intpri_part, alias)) {
parent->intpri_wd = child->intpri_wd;
} else if (MISMATCHED_FEAT(parent, child, mpam_feat_intpri_part,
diff --git a/drivers/resctrl/mpam_internal.h b/drivers/resctrl/mpam_internal.h
index def0e3a65c23..0a2177241615 100644
--- a/drivers/resctrl/mpam_internal.h
+++ b/drivers/resctrl/mpam_internal.h
@@ -203,6 +203,8 @@ struct mpam_props {
u16 dspri_wd;
u16 num_csu_mon;
u16 num_mbwu_mon;
+ u16 partid_max;
+ u16 intpartid_max;
/*
* Kunit tests use memset() to set up feature combinations that should be
--
2.34.1
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v9 6/9] arm_mpam: Divide the PARTIDs between the control groups
2026-10-12 8:23 [PATCH v9 0/9] arm_mpam: Introduce Narrow-PARTID feature Jinjie Ruan
` (4 preceding siblings ...)
2026-10-12 8:23 ` [PATCH v9 5/9] arm_mpam: Track the PARTID and internal PARTID limits per class Jinjie Ruan
@ 2026-10-12 8:23 ` Jinjie Ruan
2026-10-12 8:23 ` [PATCH v9 7/9] arm_mpam: Map resctrl RMIDs onto the request PARTIDs Jinjie Ruan
` (3 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Jinjie Ruan @ 2026-10-12 8:23 UTC (permalink / raw)
To: tony.luck, reinette.chatre, Dave.Martin, james.morse, babu.moger,
tglx, mingo, bp, dave.hansen, hpa, ben.horgan, fenghuay, x86,
linux-kernel
Cc: ruanjinjie
From: Zeng Heng <zengheng4@huawei.com>
To program a class that implements PARTID narrowing once per control
group, the driver needs to know which request PARTIDs belong to which
control group. Add a map from request PARTID to control group, and
program a group's configuration into every request PARTID that maps onto
it.
The map is empty until the resctrl glue code sets one up, so every
PARTID is its own control group and the behaviour is unchanged. A class
that implements narrowing writes the request-PARTID to internal-PARTID
mapping, so its configuration is only programmed for the request PARTID
that maps onto itself.
Cc: James Morse <james.morse@arm.com>
Cc: Ben Horgan <ben.horgan@arm.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Fenghua Yu <fenghuay@nvidia.com>
Signed-off-by: Zeng Heng <zengheng4@huawei.com>
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
---
drivers/resctrl/mpam_devices.c | 68 ++++++++++++++++++++++++++++++---
drivers/resctrl/mpam_internal.h | 2 +
2 files changed, 65 insertions(+), 5 deletions(-)
diff --git a/drivers/resctrl/mpam_devices.c b/drivers/resctrl/mpam_devices.c
index 029f105d5915..17c29022d75d 100644
--- a/drivers/resctrl/mpam_devices.c
+++ b/drivers/resctrl/mpam_devices.c
@@ -67,6 +67,25 @@ u8 mpam_pmg_max;
static bool partid_max_init, partid_max_published;
static DEFINE_SPINLOCK(partid_max_lock);
+/*
+ * Configuration constraints for resctrl-to-MPAM partitioning.
+ * mpam_num_closid == 0 indicates resctrl is unmounted or inactive.
+ */
+static u16 mpam_num_closid;
+static u16 mpam_num_partid;
+
+/*
+ * Map a request PARTID onto its owning control group using modulo
+ * interleaving. Without resctrl active, every PARTID maps 1:1 onto itself.
+ */
+u16 mpam_partid_to_intpartid(u16 partid)
+{
+ if (!mpam_num_closid || partid >= mpam_num_partid)
+ return partid;
+
+ return partid % mpam_num_closid;
+}
+
/*
* mpam is enabled once all devices have been probed from CPU online callbacks,
* scheduled via this work_struct. If access to an MSC depends on a CPU that
@@ -1560,14 +1579,24 @@ static void mpam_reprogram_ris_partid(struct mpam_msc_ris *ris, u16 partid,
if (mpam_has_feature(mpam_feat_partid_nrw, rprops)) {
/* Update the intpartid mapping */
+ u16 intpartid = mpam_partid_to_intpartid(partid);
+
mpam_write_partsel_reg(msc, INTPARTID,
- MPAMCFG_INTPARTID_INTERNAL | partid);
+ MPAMCFG_INTPARTID_INTERNAL | intpartid);
+
+ /*
+ * The internal PARTID holds the configuration. Only the
+ * request PARTID that maps onto itself writes the controls,
+ * the others share the same internal PARTID.
+ */
+ if (partid != intpartid)
+ goto out;
/*
* Then switch to the 'internal' partid to update the
* configuration.
*/
- __mpam_intpart_sel(ris->ris_idx, partid, msc);
+ __mpam_intpart_sel(ris->ris_idx, intpartid, msc);
}
if (mpam_has_feature(mpam_feat_cpor_part, rprops)) {
@@ -1643,6 +1672,7 @@ static void mpam_reprogram_ris_partid(struct mpam_msc_ris *ris, u16 partid,
mpam_quirk_post_config_change(ris, partid, cfg);
+out:
mutex_unlock(&msc->part_sel_lock);
}
@@ -1770,15 +1800,37 @@ struct mpam_write_config_arg {
static int __write_config(void *arg)
{
struct mpam_write_config_arg *c = arg;
+ u16 partid;
+
+ if (!mpam_num_closid) {
+ mpam_reprogram_ris_partid(c->ris, c->partid,
+ &c->comp->cfg[c->partid]);
+ return 0;
+ }
- mpam_reprogram_ris_partid(c->ris, c->partid, &c->comp->cfg[c->partid]);
+ /* c->partid is the control group, and must be a valid one. */
+ WARN_ON_ONCE(c->partid >= mpam_num_closid);
+
+ /*
+ * A control group owns every request PARTID that maps onto it. Write
+ * the group's configuration into each of them. Resources that
+ * implement narrowing collapse them back onto a single internal
+ * PARTID, so only the first write changes the hardware.
+ */
+ for (partid = 0; partid < mpam_num_partid; partid++) {
+ if (mpam_partid_to_intpartid(partid) != c->partid)
+ continue;
+
+ mpam_reprogram_ris_partid(c->ris, partid,
+ &c->comp->cfg[c->partid]);
+ }
return 0;
}
static void mpam_reprogram_msc(struct mpam_msc *msc)
{
- u16 partid;
+ u16 partid, loop_max;
bool reset;
struct mpam_config *cfg;
struct mpam_msc_ris *ris;
@@ -1802,7 +1854,13 @@ static void mpam_reprogram_msc(struct mpam_msc *msc)
arg.comp = ris->vmsc->comp;
arg.ris = ris;
reset = true;
- for (partid = 0; partid <= mpam_partid_max; partid++) {
+ /*
+ * Configurations are stored per control group. When resctrl
+ * has grouped several PARTIDs together, __write_config()
+ * expands one group into all of its PARTIDs.
+ */
+ loop_max = mpam_num_closid ? mpam_num_closid : mpam_partid_max + 1;
+ for (partid = 0; partid < loop_max; partid++) {
cfg = &ris->vmsc->comp->cfg[partid];
if (!bitmap_empty(cfg->features, MPAM_FEATURE_LAST))
reset = false;
diff --git a/drivers/resctrl/mpam_internal.h b/drivers/resctrl/mpam_internal.h
index 0a2177241615..5a7590568a05 100644
--- a/drivers/resctrl/mpam_internal.h
+++ b/drivers/resctrl/mpam_internal.h
@@ -458,6 +458,8 @@ extern struct list_head mpam_classes;
extern u16 mpam_partid_max;
extern u8 mpam_pmg_max;
+u16 mpam_partid_to_intpartid(u16 partid);
+
/* Scheduled work callback to enable mpam once all MSC have been probed */
void mpam_enable(struct work_struct *work);
void mpam_disable(struct work_struct *work);
--
2.34.1
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v9 7/9] arm_mpam: Map resctrl RMIDs onto the request PARTIDs
2026-10-12 8:23 [PATCH v9 0/9] arm_mpam: Introduce Narrow-PARTID feature Jinjie Ruan
` (5 preceding siblings ...)
2026-10-12 8:23 ` [PATCH v9 6/9] arm_mpam: Divide the PARTIDs between the control groups Jinjie Ruan
@ 2026-10-12 8:23 ` Jinjie Ruan
2026-10-12 8:23 ` [PATCH v9 8/9] arm_mpam: Identify the non-aliasing controls Jinjie Ruan
` (2 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Jinjie Ruan @ 2026-10-12 8:23 UTC (permalink / raw)
To: tony.luck, reinette.chatre, Dave.Martin, james.morse, babu.moger,
tglx, mingo, bp, dave.hansen, hpa, ben.horgan, fenghuay, x86,
linux-kernel
Cc: ruanjinjie
From: Zeng Heng <zengheng4@huawei.com>
The current resctrl RMID allocation assumes a rigid 2D encoding, which
limits the total number of monitoring groups to the hardware CLOSID count.
To decouple control and monitoring, redefine resctrl_arch_rmid_idx_encode()
as a flat, global monitoring index. When extra request PARTIDs are active,
the task's hardware PARTID and PMG are derived directly from this global
RMID via modulo division. Otherwise, legacy 1:1 behavior is preserved.
CDP halves the CLOSID range and uses two PARTIDs per control group,
which conflicts with the grouping, so it is refused while the extra
PARTIDs are in use.
No functional change for now as resctrl_using_extra_partids
remains false.
Cc: James Morse <james.morse@arm.com>
Cc: Ben Horgan <ben.horgan@arm.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Fenghua Yu <fenghuay@nvidia.com>
Suggested-by: James Morse <james.morse@arm.com>
Signed-off-by: Zeng Heng <zengheng4@huawei.com>
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
---
drivers/resctrl/mpam_resctrl.c | 63 +++++++++++++++++++++++++++++++---
1 file changed, 59 insertions(+), 4 deletions(-)
diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c
index 0edededdf7a5..8479fdd7f713 100644
--- a/drivers/resctrl/mpam_resctrl.c
+++ b/drivers/resctrl/mpam_resctrl.c
@@ -62,6 +62,9 @@ static DEFINE_MUTEX(domain_list_lock);
*/
static bool cdp_enabled;
+/* If resctrl is leveraging extra request PARTIDs for monitoring. */
+static bool resctrl_using_extra_partids;
+
/*
* We use cacheinfo to discover the size of the caches and their id. cacheinfo
* populates this from a device_initcall(). mpam_resctrl_setup() must wait.
@@ -205,6 +208,16 @@ int resctrl_arch_set_cdp_enabled(enum resctrl_res_level rid, bool enable)
if (enable)
pr_warn("CDP is an expert feature and may cause MPAM to malfunction.\n");
+ /*
+ * CDP halves the CLOSID range and uses two PARTIDs per control group.
+ * That conflicts with the PARTID grouping used to provide the extra
+ * monitoring groups.
+ */
+ if (enable && resctrl_using_extra_partids) {
+ pr_warn("CDP is not supported with PARTID narrowing\n");
+ return -EOPNOTSUPP;
+ }
+
/*
* resctrl_arch_set_cdp_enabled() is only called with enable set to
* false on error and unmount.
@@ -274,8 +287,28 @@ u32 resctrl_arch_system_num_rmid_idx(void)
return (mpam_pmg_max + 1) * (mpam_partid_max + 1);
}
+/* The request PARTID and PMG a resctrl RMID maps onto. */
+static u16 resctrl_rmid_to_partid(u32 rmid)
+{
+ return rmid / (mpam_pmg_max + 1);
+}
+
+static u8 resctrl_rmid_to_pmg(u32 rmid)
+{
+ return rmid % (mpam_pmg_max + 1);
+}
+
+/* The control group that owns a request PARTID. */
+static u16 resctrl_partid_to_closid(u16 partid)
+{
+ return mpam_partid_to_intpartid(partid);
+}
+
u32 resctrl_arch_rmid_idx_encode(u32 closid, u32 rmid)
{
+ if (resctrl_using_extra_partids)
+ return rmid;
+
return closid * (mpam_pmg_max + 1) + rmid;
}
@@ -287,6 +320,14 @@ bool resctrl_arch_rmid_idx_match(u32 closid, u32 rmid_closid)
void resctrl_arch_rmid_idx_decode(u32 idx, u32 *closid, u32 *rmid)
{
+ if (resctrl_using_extra_partids) {
+ if (rmid)
+ *rmid = idx;
+ if (closid)
+ *closid = resctrl_partid_to_closid(resctrl_rmid_to_partid(idx));
+ return;
+ }
+
*closid = idx / (mpam_pmg_max + 1);
*rmid = idx % (mpam_pmg_max + 1);
}
@@ -303,11 +344,23 @@ static void resctrl_closid_rmid_to_partid_pmg(u32 closid, u32 rmid,
u8 *pmg_d, u8 *pmg_i)
{
WARN_ON_ONCE(closid > U16_MAX);
- WARN_ON_ONCE(rmid > U8_MAX);
+ WARN_ON_ONCE(!resctrl_using_extra_partids && rmid > U8_MAX);
if (!cdp_enabled) {
- *partid_d = *partid_i = closid;
- *pmg_d = *pmg_i = rmid;
+ u16 partid;
+ u8 pmg;
+
+ if (resctrl_using_extra_partids) {
+ partid = resctrl_rmid_to_partid(rmid);
+ WARN_ON_ONCE(partid > U16_MAX);
+ pmg = resctrl_rmid_to_pmg(rmid);
+ } else {
+ partid = closid;
+ pmg = rmid;
+ }
+
+ *partid_d = *partid_i = partid;
+ *pmg_d = *pmg_i = pmg;
} else {
/*
* When CDP is enabled, resctrl halves the closid range and we
@@ -356,7 +409,9 @@ void resctrl_arch_set_closid_rmid(struct task_struct *tsk, u32 closid, u32 rmid)
/* Does the request PARTID belong to this control group? */
static bool resctrl_partid_in_closid(u16 partid, u32 closid)
{
- if (cdp_enabled)
+ if (resctrl_using_extra_partids)
+ partid = resctrl_partid_to_closid(partid);
+ else if (cdp_enabled)
partid >>= 1;
return partid == closid;
--
2.34.1
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v9 8/9] arm_mpam: Identify the non-aliasing controls
2026-10-12 8:23 [PATCH v9 0/9] arm_mpam: Introduce Narrow-PARTID feature Jinjie Ruan
` (6 preceding siblings ...)
2026-10-12 8:23 ` [PATCH v9 7/9] arm_mpam: Map resctrl RMIDs onto the request PARTIDs Jinjie Ruan
@ 2026-10-12 8:23 ` Jinjie Ruan
2026-10-12 8:23 ` [PATCH v9 9/9] arm_mpam: Use the request PARTIDs beyond the internal range Jinjie Ruan
2026-10-22 0:02 ` [PATCH v9 0/9] arm_mpam: Introduce Narrow-PARTID feature Shaopeng Tan (Fujitsu)
9 siblings, 0 replies; 12+ messages in thread
From: Jinjie Ruan @ 2026-10-12 8:23 UTC (permalink / raw)
To: tony.luck, reinette.chatre, Dave.Martin, james.morse, babu.moger,
tglx, mingo, bp, dave.hansen, hpa, ben.horgan, fenghuay, x86,
linux-kernel
Cc: ruanjinjie
A non-aliasing control can't be presented by writing the same value to
several PARTIDs, because its value is additive. This is used later to
decide whether a class's non-aliasing controls must be dropped before
its PARTIDs can be used as separate monitoring groups.
A control that aliases shares the resource, so writing the same value to
each PARTID keeps its meaning. Several PARTIDs that map onto one
internal PARTID are configured as one, so a class that implements
narrowing does not have this problem.
Cc: James Morse <james.morse@arm.com>
Cc: Ben Horgan <ben.horgan@arm.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Fenghua Yu <fenghuay@nvidia.com>
Suggested-by: James Morse <james.morse@arm.com>
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
---
drivers/resctrl/mpam_internal.h | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/drivers/resctrl/mpam_internal.h b/drivers/resctrl/mpam_internal.h
index 5a7590568a05..0433ca6faaba 100644
--- a/drivers/resctrl/mpam_internal.h
+++ b/drivers/resctrl/mpam_internal.h
@@ -221,6 +221,24 @@ struct mpam_props {
#define mpam_set_feature(_feat, x) __set_bit(_feat, (x)->features)
#define mpam_clear_feature(_feat, x) __clear_bit(_feat, (x)->features)
+/*
+ * A non-aliasing control (e.g. a cache capacity) can't be presented by
+ * writing the same value to several PARTIDs: its value is additive, two 50%
+ * values add up to 100%. An aliasing control (e.g. a cache portion bitmap)
+ * shares the resource, so the same value written to each PARTID keeps its
+ * meaning.
+ */
+static inline bool
+mpam_props_has_non_aliasing_controls(const struct mpam_props *props)
+{
+ return mpam_has_feature(mpam_feat_mbw_max, props) ||
+ mpam_has_feature(mpam_feat_mbw_min, props) ||
+ mpam_has_feature(mpam_feat_mbw_prop, props) ||
+ mpam_has_feature(mpam_feat_cmax_cmax, props) ||
+ mpam_has_feature(mpam_feat_cmax_cmin, props) ||
+ mpam_has_feature(mpam_feat_cmax_softlim, props);
+}
+
/* Workaround bits for msc->quirks */
enum mpam_device_quirks {
T241_SCRUB_SHADOW_REGS,
--
2.34.1
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v9 9/9] arm_mpam: Use the request PARTIDs beyond the internal range
2026-10-12 8:23 [PATCH v9 0/9] arm_mpam: Introduce Narrow-PARTID feature Jinjie Ruan
` (7 preceding siblings ...)
2026-10-12 8:23 ` [PATCH v9 8/9] arm_mpam: Identify the non-aliasing controls Jinjie Ruan
@ 2026-10-12 8:23 ` Jinjie Ruan
2026-10-22 0:02 ` [PATCH v9 0/9] arm_mpam: Introduce Narrow-PARTID feature Shaopeng Tan (Fujitsu)
9 siblings, 0 replies; 12+ messages in thread
From: Jinjie Ruan @ 2026-10-12 8:23 UTC (permalink / raw)
To: tony.luck, reinette.chatre, Dave.Martin, james.morse, babu.moger,
tglx, mingo, bp, dave.hansen, hpa, ben.horgan, fenghuay, x86,
linux-kernel
Cc: ruanjinjie
A class that implements PARTID narrowing maps several request PARTIDs
onto one internal PARTID. The internal PARTIDs hold its controls, and
the request PARTIDs can each be a monitoring group. This gives one
control group more monitoring groups than the PMG count alone allows.
The extra request PARTIDs can only be used when no class would lose a
non-aliasing control: its value can not be written to every PARTID of a
control group, whereas an aliasing control's value can. Every class is
considered, including the ones resctrl does not use, so that an unknown
class can still gain a resctrl schema later. This is decided once,
before resctrl_init(), and there is no user interface for it.
The driver stops reducing msc->partid_max, so the full request PARTID
range is available, and the resctrl glue code starts handing out the
extra ones. A control group's configuration is written to every request
PARTID that maps onto it.
Cc: James Morse <james.morse@arm.com>
Cc: Ben Horgan <ben.horgan@arm.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Fenghua Yu <fenghuay@nvidia.com>
Suggested-by: James Morse <james.morse@arm.com>
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
---
drivers/resctrl/mpam_devices.c | 16 +++++++-
drivers/resctrl/mpam_internal.h | 2 +
drivers/resctrl/mpam_resctrl.c | 72 ++++++++++++++++++++++++++++++++-
3 files changed, 87 insertions(+), 3 deletions(-)
diff --git a/drivers/resctrl/mpam_devices.c b/drivers/resctrl/mpam_devices.c
index 17c29022d75d..207456bcab08 100644
--- a/drivers/resctrl/mpam_devices.c
+++ b/drivers/resctrl/mpam_devices.c
@@ -74,6 +74,18 @@ static DEFINE_SPINLOCK(partid_max_lock);
static u16 mpam_num_closid;
static u16 mpam_num_partid;
+void mpam_partid_map_init(u16 num_partid, u16 num_closid)
+{
+ mpam_num_partid = num_partid;
+ mpam_num_closid = num_closid;
+}
+
+void mpam_partid_map_exit(void)
+{
+ mpam_num_closid = 0;
+ mpam_num_partid = 0;
+}
+
/*
* Map a request PARTID onto its owning control group using modulo
* interleaving. Without resctrl active, every PARTID maps 1:1 onto itself.
@@ -959,6 +971,9 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)
* A RIS that implements PARTID narrowing has one configuration per
* internal PARTID, and several request PARTIDs share it. Record both
* limits in the resource properties.
+ * Keep msc->partid_max at the full request PARTID range. The resctrl
+ * glue code will broadcast a control group's configuration to every
+ * request PARTID that maps onto that group's internal PARTID.
*/
props->partid_max = msc->partid_max;
props->intpartid_max = msc->partid_max;
@@ -969,7 +984,6 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)
mpam_set_feature(mpam_feat_partid_nrw, props);
props->intpartid_max = min(props->intpartid_max, intpartid_max);
- msc->partid_max = min(msc->partid_max, intpartid_max);
}
}
diff --git a/drivers/resctrl/mpam_internal.h b/drivers/resctrl/mpam_internal.h
index 0433ca6faaba..2ee8f8069685 100644
--- a/drivers/resctrl/mpam_internal.h
+++ b/drivers/resctrl/mpam_internal.h
@@ -477,6 +477,8 @@ extern u16 mpam_partid_max;
extern u8 mpam_pmg_max;
u16 mpam_partid_to_intpartid(u16 partid);
+void mpam_partid_map_init(u16 num_partid, u16 num_closid);
+void mpam_partid_map_exit(void);
/* Scheduled work callback to enable mpam once all MSC have been probed */
void mpam_enable(struct work_struct *work);
diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c
index 8479fdd7f713..56611ef13bca 100644
--- a/drivers/resctrl/mpam_resctrl.c
+++ b/drivers/resctrl/mpam_resctrl.c
@@ -62,6 +62,9 @@ static DEFINE_MUTEX(domain_list_lock);
*/
static bool cdp_enabled;
+static u16 resctrl_num_closid = 1;
+static u16 resctrl_num_mon_partid = 1;
+
/* If resctrl is leveraging extra request PARTIDs for monitoring. */
static bool resctrl_using_extra_partids;
@@ -279,12 +282,12 @@ static bool mpam_resctrl_hide_cdp(enum resctrl_res_level rid)
*/
u32 resctrl_arch_get_num_closid(struct rdt_resource *ignored)
{
- return mpam_partid_max + 1;
+ return resctrl_num_closid;
}
u32 resctrl_arch_system_num_rmid_idx(void)
{
- return (mpam_pmg_max + 1) * (mpam_partid_max + 1);
+ return (mpam_pmg_max + 1) * resctrl_num_mon_partid;
}
/* The request PARTID and PMG a resctrl RMID maps onto. */
@@ -1927,6 +1930,65 @@ void mpam_resctrl_offline_cpu(unsigned int cpu)
}
}
+/*
+ * Decide how the PARTID space is presented to resctrl. This runs once the
+ * classes are known, and before resctrl_init() advertises any limits to
+ * user-space.
+ *
+ * The extra request PARTIDs on a class that implements narrowing can be used
+ * as monitoring groups. That is only safe when no class would lose a
+ * non-aliasing control: its value can not be written to each PARTID of a
+ * control group, whereas an aliasing control's value can. A class that
+ * implements narrowing doesn't have the problem, as its PARTIDs are already
+ * configured as one.
+ *
+ * Every class is considered, including the ones resctrl doesn't use. An
+ * unknown class may gain a resctrl schema later, and using the extra PARTIDs
+ * would prevent that.
+ */
+static void mpam_resctrl_configure_partids(void)
+{
+ struct mpam_class *class;
+ u16 num_closid = mpam_partid_max + 1;
+ bool can_expand = false, blocked = false;
+
+ guard(srcu)(&mpam_srcu);
+ list_for_each_entry_srcu(class, &mpam_classes, classes_list,
+ srcu_read_lock_held(&mpam_srcu)) {
+ struct mpam_props *cprops = &class->props;
+
+ num_closid = min(num_closid, (u16)(cprops->intpartid_max + 1));
+
+ if (mpam_has_feature(mpam_feat_partid_nrw, cprops)) {
+ if (cprops->intpartid_max < cprops->partid_max)
+ can_expand = true;
+ } else if (mpam_props_has_non_aliasing_controls(cprops)) {
+ pr_debug("class %u has non-aliasing controls without narrowing\n",
+ class->level);
+ blocked = true;
+ }
+ }
+
+ if (!num_closid)
+ num_closid = 1;
+
+ resctrl_num_closid = num_closid;
+ if (can_expand && !blocked) {
+ resctrl_using_extra_partids = true;
+ resctrl_num_mon_partid = mpam_partid_max + 1;
+ } else {
+ resctrl_using_extra_partids = false;
+ resctrl_num_mon_partid = num_closid;
+ }
+
+ /* Tell the driver how the request PARTIDs map onto control groups. */
+ mpam_partid_map_init(resctrl_num_mon_partid, resctrl_num_closid);
+
+ pr_debug("PARTIDs: monitor=%u closid=%u extra=%u\n",
+ resctrl_num_mon_partid, resctrl_num_closid,
+ resctrl_using_extra_partids);
+}
+
int mpam_resctrl_setup(void)
{
int err = 0;
@@ -1948,6 +2010,9 @@ int mpam_resctrl_setup(void)
mpam_resctrl_pick_caches();
mpam_resctrl_pick_mba();
+ /* Decide how the request PARTIDs map onto the control groups */
+ mpam_resctrl_configure_partids();
+
/* Initialise the resctrl structures from the classes */
for_each_mpam_resctrl_control(res, rid) {
if (!res->class)
@@ -2003,6 +2068,9 @@ void mpam_resctrl_exit(void)
WRITE_ONCE(resctrl_enabled, false);
resctrl_exit();
+
+ /* resctrl is gone: forget how the PARTIDs were grouped. */
+ mpam_partid_map_exit();
}
static void mpam_resctrl_teardown_mon(struct mpam_resctrl_mon *mon, struct mpam_class *class)
--
2.34.1
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v9 0/9] arm_mpam: Introduce Narrow-PARTID feature
2026-10-12 8:23 [PATCH v9 0/9] arm_mpam: Introduce Narrow-PARTID feature Jinjie Ruan
` (8 preceding siblings ...)
2026-10-12 8:23 ` [PATCH v9 9/9] arm_mpam: Use the request PARTIDs beyond the internal range Jinjie Ruan
@ 2026-10-22 0:02 ` Shaopeng Tan (Fujitsu)
2026-10-22 1:51 ` Jinjie Ruan
9 siblings, 1 reply; 12+ messages in thread
From: Shaopeng Tan (Fujitsu) @ 2026-10-22 0:02 UTC (permalink / raw)
To: Jinjie Ruan <ruanjinjie@huawei.com>,
tony.luck, reinette.chatre, Dave.Martin, james.morse, babu.moger,
tglx, mingo, bp, dave.hansen, hpa, ben.horgan, fenghuay, x86,
linux-kernel
Hello Jinjie,
I would like to confirm the following points.
- Is the number of reqPARTIDs per intPARTID dependent on the total number of intPARTIDs and reqPARTIDs?
- Given that requirements for the number of control and monitoring groups vary by user,
would dynamic configuration of the mapping ratio between intPARTID and reqPARTID be a more suitable approach?
- Regarding the mapping logic, some reqPARTIDs may remain unused. Is this the expected behavior?
- Considering that Arm has previously implemented this,
are there any gaps between their implementation and Huawei’s requirements?
https://lore.kernel.org/linux-arm-kernel/20241212154000.330467-1-Dave.Martin@arm.com/
Best regards,
Shaopeng TAN
> Background
> ==========
>
> In ARM MPAM, a monitoring group is defined by a (PARTID, PMG) pair.
> However, many production platforms implement only a single PMG bit,
> limiting the total number of monitoring groups long before the request
> PARTID space is exhausted.
>
> Hardware Mechanism
> ===================
>
> Platforms with "PARTID narrowing" mitigate this by mapping multiple
> request PARTIDs onto a single internal PARTID. The internal PARTID
> dictates the control group configuration, while each individual request
> PARTID functions as an independent monitoring group. This decouples
> control and monitoring allocation, allowing more monitoring groups
> than the PMG constraint normally permits.
>
> Software Strategy
> ==================
>
> Using request PARTIDs beyond the internal range is only safe when no MPAM
> class loses a non-aliasing control. While aliasing controls (e.g., cache
> portion bitmaps) can be safely replicated across request PARTIDs,
> non-aliasing controls (e.g., cache capacity) are additive and cannot.
>
> To ensure safety, this series implements a completely static grouping
> policy evaluated once before resctrl_init(). If any class contains
> non-aliasing controls without narrowing capabilities, the expansion
> is bypassed. This infrastructure remains completely transparent
> to user-space.
>
> Patch Architecture
> ====================
>
> - Patches 1-4: Bugfix and preparatory rework.
>
> - Patch 5: Track both internal and request PARTID limits per class.
>
> - Patches 6-7: Implement driver-side modulo interleaving for request
> PARTIDs and flat global RMID indexing.
>
> - Patches 8-9: Identify non-aliasing controls, and enable the narrowing
> expansion by lifting driver-side partid_max clamping and freeing
> the extra request PARTIDs for resctrl.
>
> Changes since v8
> - Replaced dynamic allocation with pure static grouping as requested by James.
> - Removed the 'mpam.intpartid_max' boot parameter; expansion is now automatic.
> - Extended safety verification to all classes, preserving
> forward-compatibility for unknown classes that might gain resctrl
> schemas later.
> - Decoupled the driver infrastructure patch from the resctrl glue patch.
> - Introduced resctrl_arch_rmid_idx_match() to clean up the RMID allocator.
> - Link to v8: https://lore.kernel.org/all/20260413085405.1166412-1-zengheng4@huawei.com/
>
> Previous Versions
> =================
>
> v7: https://lore.kernel.org/all/20260317132141.1272506-1-zengheng4@huawei.com/
> v6: https://lore.kernel.org/all/20250222112448.2438586-1-zengheng4@huawei.com/
> v5: https://lore.kernel.org/all/20250217031852.2014939-1-zengheng4@huawei.com/
> RFC-v4: https://lore.kernel.org/all/20250104101224.873926-1-zengheng4@huawei.com/
> RFC-v3: https://lore.kernel.org/all/20241207092136.2488426-1-zengheng4@huawei.com/
> RFC-v2: https://lore.kernel.org/all/20241119135104.595630-1-zengheng4@huawei.com/
> RFC-v1: https://lore.kernel.org/all/20241114135037.918470-1-zengheng4@huawei.com/
>
> Jinjie Ruan (6):
> fs/resctrl: Add resctrl_arch_rmid_idx_match() helper
> arm_mpam: resctrl: Add resctrl_closid_rmid_to_partid_pmg() helper
> arm_mpam: Add a helper to check a PARTID is in a CLOSID
> arm_mpam: Track the PARTID and internal PARTID limits per class
> arm_mpam: Identify the non-aliasing controls
> arm_mpam: Use the request PARTIDs beyond the internal range
>
> Zeng Heng (3):
> fs/resctrl: Unmount resctrl with the CDP state it was mounted with
> arm_mpam: Divide the PARTIDs between the control groups
> arm_mpam: Map resctrl RMIDs onto the request PARTIDs
>
> arch/x86/include/asm/resctrl.h | 6 +
> drivers/resctrl/mpam_devices.c | 105 +++++++++++++---
> drivers/resctrl/mpam_internal.h | 24 ++++
> drivers/resctrl/mpam_resctrl.c | 213 +++++++++++++++++++++++++++-----
> fs/resctrl/monitor.c | 13 +-
> fs/resctrl/rdtgroup.c | 38 +++++-
> include/linux/arm_mpam.h | 1 +
> 7 files changed, 341 insertions(+), 59 deletions(-)
>
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v9 0/9] arm_mpam: Introduce Narrow-PARTID feature
2026-10-22 0:02 ` [PATCH v9 0/9] arm_mpam: Introduce Narrow-PARTID feature Shaopeng Tan (Fujitsu)
@ 2026-10-22 1:51 ` Jinjie Ruan
0 siblings, 0 replies; 12+ messages in thread
From: Jinjie Ruan @ 2026-10-22 1:51 UTC (permalink / raw)
To: Shaopeng Tan (Fujitsu),
tony.luck, reinette.chatre, Dave.Martin, james.morse, babu.moger,
tglx, mingo, bp, dave.hansen, hpa, ben.horgan, fenghuay, x86,
linux-kernel
在 2026/10/22 8:02, Shaopeng Tan (Fujitsu) 写道:
> Hello Jinjie,
>
> I would like to confirm the following points.
>
> - Is the number of reqPARTIDs per intPARTID dependent on the total number of intPARTIDs and reqPARTIDs?
Hi Shaopeng,
You are right,
reqPARTID % mpam_num_closid(the total number of intPARTIDs) -> intPARTID
so reqPARTIDs per intPARTID = Total number of reqPARTIDs / Total number
of intPARTIDs
> - Given that requirements for the number of control and monitoring groups vary by user,
> would dynamic configuration of the mapping ratio between intPARTID and reqPARTID be a more suitable approach?
No, in v8 James has suggested "static first" and avoid exposing user ABI
in multiple places below:
"I think we should leave anything dynamic here for the further future,
lets get static partitioning working first"
"I'd prefer we get the static grouping working first, before we work out
what needs changing in resctrl to do it dynamically"
"I'd like to avoid exposing user ABI to control this until we get it
working, then we can talk about whether to try making the grouping
dynamically managed by resctrl."
Link:
https://lore.kernel.org/all/2944b506-a462-42f8-95cf-404241fb27f0@arm.com/
Link:
https://lore.kernel.org/all/ec8bc617-9e74-4749-ab33-39d1079415cc@arm.com/
> - Regarding the mapping logic, some reqPARTIDs may remain unused. Is this the expected behavior?
As mentioned above, we are implementing the static solution first, so it
is expected behavior that some of the reqpartids are not used.
> - Considering that Arm has previously implemented this,
> are there any gaps between their implementation and Huawei’s requirements?
> https://lore.kernel.org/linux-arm-kernel/20241212154000.330467-1-Dave.Martin@arm.com/
I haven't looked at the ARM implementation yet (I'll refer to it later),
but currently the code is being gradually aligned with the maintainer's
review comments.
>
> Best regards,
> Shaopeng TAN
>
>
>> Background
>> ==========
>>
>> In ARM MPAM, a monitoring group is defined by a (PARTID, PMG) pair.
>> However, many production platforms implement only a single PMG bit,
>> limiting the total number of monitoring groups long before the request
>> PARTID space is exhausted.
>>
>> Hardware Mechanism
>> ===================
>>
>> Platforms with "PARTID narrowing" mitigate this by mapping multiple
>> request PARTIDs onto a single internal PARTID. The internal PARTID
>> dictates the control group configuration, while each individual request
>> PARTID functions as an independent monitoring group. This decouples
>> control and monitoring allocation, allowing more monitoring groups
>> than the PMG constraint normally permits.
>>
>> Software Strategy
>> ==================
>>
>> Using request PARTIDs beyond the internal range is only safe when no MPAM
>> class loses a non-aliasing control. While aliasing controls (e.g., cache
>> portion bitmaps) can be safely replicated across request PARTIDs,
>> non-aliasing controls (e.g., cache capacity) are additive and cannot.
>>
>> To ensure safety, this series implements a completely static grouping
>> policy evaluated once before resctrl_init(). If any class contains
>> non-aliasing controls without narrowing capabilities, the expansion
>> is bypassed. This infrastructure remains completely transparent
>> to user-space.
>>
>> Patch Architecture
>> ====================
>>
>> - Patches 1-4: Bugfix and preparatory rework.
>>
>> - Patch 5: Track both internal and request PARTID limits per class.
>>
>> - Patches 6-7: Implement driver-side modulo interleaving for request
>> PARTIDs and flat global RMID indexing.
>>
>> - Patches 8-9: Identify non-aliasing controls, and enable the narrowing
>> expansion by lifting driver-side partid_max clamping and freeing
>> the extra request PARTIDs for resctrl.
>>
>> Changes since v8
>> - Replaced dynamic allocation with pure static grouping as requested by James.
>> - Removed the 'mpam.intpartid_max' boot parameter; expansion is now automatic.
>> - Extended safety verification to all classes, preserving
>> forward-compatibility for unknown classes that might gain resctrl
>> schemas later.
>> - Decoupled the driver infrastructure patch from the resctrl glue patch.
>> - Introduced resctrl_arch_rmid_idx_match() to clean up the RMID allocator.
>> - Link to v8: https://lore.kernel.org/all/20260413085405.1166412-1-zengheng4@huawei.com/
>>
>> Previous Versions
>> =================
>>
>> v7: https://lore.kernel.org/all/20260317132141.1272506-1-zengheng4@huawei.com/
>> v6: https://lore.kernel.org/all/20250222112448.2438586-1-zengheng4@huawei.com/
>> v5: https://lore.kernel.org/all/20250217031852.2014939-1-zengheng4@huawei.com/
>> RFC-v4: https://lore.kernel.org/all/20250104101224.873926-1-zengheng4@huawei.com/
>> RFC-v3: https://lore.kernel.org/all/20241207092136.2488426-1-zengheng4@huawei.com/
>> RFC-v2: https://lore.kernel.org/all/20241119135104.595630-1-zengheng4@huawei.com/
>> RFC-v1: https://lore.kernel.org/all/20241114135037.918470-1-zengheng4@huawei.com/
>>
>> Jinjie Ruan (6):
>> fs/resctrl: Add resctrl_arch_rmid_idx_match() helper
>> arm_mpam: resctrl: Add resctrl_closid_rmid_to_partid_pmg() helper
>> arm_mpam: Add a helper to check a PARTID is in a CLOSID
>> arm_mpam: Track the PARTID and internal PARTID limits per class
>> arm_mpam: Identify the non-aliasing controls
>> arm_mpam: Use the request PARTIDs beyond the internal range
>>
>> Zeng Heng (3):
>> fs/resctrl: Unmount resctrl with the CDP state it was mounted with
>> arm_mpam: Divide the PARTIDs between the control groups
>> arm_mpam: Map resctrl RMIDs onto the request PARTIDs
>>
>> arch/x86/include/asm/resctrl.h | 6 +
>> drivers/resctrl/mpam_devices.c | 105 +++++++++++++---
>> drivers/resctrl/mpam_internal.h | 24 ++++
>> drivers/resctrl/mpam_resctrl.c | 213 +++++++++++++++++++++++++++-----
>> fs/resctrl/monitor.c | 13 +-
>> fs/resctrl/rdtgroup.c | 38 +++++-
>> include/linux/arm_mpam.h | 1 +
>> 7 files changed, 341 insertions(+), 59 deletions(-)
>>
>> --
>> 2.34.1
>>
--
Best regards,
Jinjie
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2026-10-22 1:51 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-10-12 8:23 [PATCH v9 0/9] arm_mpam: Introduce Narrow-PARTID feature Jinjie Ruan
2026-10-12 8:23 ` [PATCH v9 1/9] fs/resctrl: Unmount resctrl with the CDP state it was mounted with Jinjie Ruan
2026-10-12 8:23 ` [PATCH v9 2/9] fs/resctrl: Add resctrl_arch_rmid_idx_match() helper Jinjie Ruan
2026-10-12 8:23 ` [PATCH v9 3/9] arm_mpam: resctrl: Add resctrl_closid_rmid_to_partid_pmg() helper Jinjie Ruan
2026-10-12 8:23 ` [PATCH v9 4/9] arm_mpam: Add a helper to check a PARTID is in a CLOSID Jinjie Ruan
2026-10-12 8:23 ` [PATCH v9 5/9] arm_mpam: Track the PARTID and internal PARTID limits per class Jinjie Ruan
2026-10-12 8:23 ` [PATCH v9 6/9] arm_mpam: Divide the PARTIDs between the control groups Jinjie Ruan
2026-10-12 8:23 ` [PATCH v9 7/9] arm_mpam: Map resctrl RMIDs onto the request PARTIDs Jinjie Ruan
2026-10-12 8:23 ` [PATCH v9 8/9] arm_mpam: Identify the non-aliasing controls Jinjie Ruan
2026-10-12 8:23 ` [PATCH v9 9/9] arm_mpam: Use the request PARTIDs beyond the internal range Jinjie Ruan
2026-10-22 0:02 ` [PATCH v9 0/9] arm_mpam: Introduce Narrow-PARTID feature Shaopeng Tan (Fujitsu)
2026-10-22 1:51 ` Jinjie Ruan
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®