* [RFC PATCH v2 00/11] Introduce PARTID grouping support
@ 2025-01-17 15:10 Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 01/11] arm_mpam: Fix read-back of cloned resource controls under CDP emulation Dave Martin
` (22 more replies)
0 siblings, 23 replies; 26+ messages in thread
From: Dave Martin @ 2025-01-17 15:10 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-arm-kernel, Zeng Heng, Shaopeng Tan, James Morse
(Was: Introduce flexible CLOSID/RMID translation) [5]
This series introduces an ID remapping scheme for mapping multiple MPAM
PARTIDs to each resctrl control group, and programs the intPARTID
appropriately on MSCs that require and support PARTID Narrowing.
Rudimentary support is included for detecting whether this
configuration will work correctly on the detected hardware, but this
detection is currently inflexible and may disable this multiple PARTID
mapping behaviour unnecessarily on some platforms.
(Feedback would be appreciated from anyone who knows of hardware where
this approach is insufficient.)
On supported platforms, this allows the number of monitoring groups per
resctrl control group to be scaled up by the value of the (provisional)
mpam.partid_per_closid kernel command-line parameter, at the expense of
reducing the number of control groups that can be created.
See "Background" below, and the subsequent sections, for more details.
This series is not a complete implementation, but should be sufficient
to provide additional resctrl monitoring groups beyond the number of
PMGs supported by the hardware (with some limitations; see
"Limitations", below).
Based on:
git://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/snapshot/v6.12-rc1
4d17b37b670795954fd7a70c8ec83fb705a5e2ad ("arm64: mpam: Restore the expected MPAM sysregs on cpuhp")
Changes Since RFCv1: [5]
---
DROPPED:
* arm_mpam: Clean up config update checks in mpam_apply_config()
(present in the base branch)
CHANGED:
* arm_mpam: Introduce flexible CLOSID/RMID translation
* [bug fix] Add CDP index properly in mpam_resctrl_partid_range().
(Also reported independently by Shaopeng Tan.)
* [relaxation] Remove requirement that mpam_pmg_max + 1 is a power
of two.
(This constraint is redundant, due to the use of multiply/add
transformations of MPAM IDs in place of shift/OR.)
NEW:
* arm_mpam: Set INTERNAL as needed when setting MSC controls
(resurrected from my local MPAM fixes branch; previously posted,
but was not included in RFC v1 of this series)
* fs/resctrl: Add arch hook to set up the default CLOSID
* arm_mpam: resctrl: Implement arch hook to set up the default CLOSID
Add and wire up a hook to update reprogram intPARTID everywhere
when the resctrl<->MPAM ID mappings change. This is needed for
clean initialisation of the hardware, since the MPAM architecture
doesn't specify the default mappings -- purging everything to a
consistent state avoids the possibility of spurious aliasing.
* arm_mpam: Add control partition identifier to mpam_config
Add a mpam_config field to tell mpam_devices.c what to program
for intPARTID. (The idea is to keep the mapping logic in
mpam_resctrl.c so that mpam_devices.c does not need to understand
how this value is determined.)
* arm_mpam: Skip inapplicable cases when reprogramming a resctrl domain
To simplify programming of intPARTIDs, never try to program a
PARTID with controls for the wrong resctrl_conf_type. (If
resctrl passed any control changes for the wrong
resctrl_conf_type, that's a bug anyway...)
* arm_mpam: Detect whether the system can alias PARTIDs
Initial attempt to detect whether it is safe a map multiple
PARTIDs to the same resctrl control group: if not, the
user-specified mpam.partid_per_closid parameter is ignored, and
the driver reverts back to the default behaviour of mapping a
single PARTID to each CLOSID.
This is currently simplistic, and only considers all MPAM MSCs
and controls in the system (irrespective if they are going to be
exposed through resctrl or not).
Background
==========
An MPAM Memory System Component (MSC) that does not implement the
PARTID Narrowing feature (Narrowing) provides independently
programmable resource controls for each PARTID up to the maximum PARTID
that the MSC supports.
An MSC that _does_ implement Narrowing performs a remapping step to
convert the PARTID from the incoming memory transaction ("request
PARTID", or "reqPARTID") to an internal ID ("internal PARTID", or
"intPARTID") before looking up and applying resource controls. Such an
MSC does not necessarily need to provide the same number of
independently programmable resource control partitions (indexed by the
intPARTID) as that which the size of the supported PARTID ("reqPARTID")
space would otherwise require. Instead, independently programmable
resource controls are provided for each intPARTID, accompanied by a
software programmable mapping that allows each reqPARTID to be mapped
independently to a selected intPARTID.
Typically, the number of intPARTIDs supported by a Narrowing MSC is
smaller than the number of reqPARTIDs. Narrowing permits multiple
reqPARTIDs to be mapped to each intPARTID, so allowing the whole MPAM
PARTID space to be served.
A non-Narrowing MSC can be understood as implementing special case of
Narrowing, where the reqPARTID-to-intPARTID mapping is the identity
function and is not programmable.
The number of independently configurable control partitions at a given
MSC cannot exceed the number of intPARTIDs available, so the number of
resctrl CLOSIDs (which need to be independently controllable) must
typically be limited not to exceed the maximum number of intPARTIDs
available on any Narrowing MSC.
However, the presence of a programmable reqPARTID-to-intPARTID mapping
in each MSC creates an opportunity: because a group of multiple MPAM
PARTIDs can be mapped onto the same controls, they behave as a single
resource control partition. Because the PARTID is used as a match
criterion for monitoring purposes along with the MPAM Performance
Monitoring Group (PMG) identifier assigned each memory system
transaction, the specific PARTID in the group can be used to
distinguish transactions for monitoring purposes, even within the same
resctrl control group and when the PMGs of the transactions are the
same.
This means that it becomes possible to provide a larger number of
resctrl monitoring groups than the number of PMGs supported by the MPAM
hardware implementation.
In order to take advantage of this, a more flexible mapping scheme is
required for mapping resctrl CLOSIDs and RMIDs to MPAM PARTIDs and PMGs
than the current MPAM driver provides.
Similar approaches have been suggested by other people
(see Acknowledgements).
However, since I was already working on this, I'm posting my approach
for comparison.
This Series
===========
This series introduces a remapping scheme as follows:
1) The resctrl partition identifiers are combined into a single
identifier (formally the Cartesian product of the CLOSID, Code/Data
partition where applicable, and RMID ID spaces).
2) The resulting identifier is then broken down into the MPAM PARTID
and PMG, in such a way that each possible <CLOSID, CDP index, RMID>
maps to a unique <PARTID, PMG> pair and vice versa, and so that any two
<PARTID, PMG> that differ only in the PMG value, map onto the same
<CLOSID, CDP index> pair.
This transformation could be done in intuitive way using bitwise
concatenation, shift and masking operations, but since the size of the
MPAM hardware ID spaces may limited, I have tried to generalise this
approach using multiplication in place of bit-shifts, to permit ID
spaces whose size is not a power of two.
Limitations
===========
This series aims to contain all translations within the mpam_resctrl.c
code. This seems the most sensible place for it, since this is a
conversion scheme that will be used to interface the MPAM hardware with
the resctrl core code (but has nothing to do with the "pure" MPAM
architecture or the resctrl core code itself).
To simplify the interface between mpam_resctrl.c and mpam_devices.c,
the ID conversion scheme is arranged so that each resctrl CLOSID maps
to a contiguous sequence of MPAM PARTIDs. This means that programming
controls for a CLOSID can be achieved by iterating over the appropriate
range of PARTIDs.
This series provides a basic configuration interface that may be too
inflexible for real hardware.
Acknowledgements
================
Thanks to Shaopeng Tan for pointing out this opportunity and suggesting
a related idea, and to Zeng Heng who has also implemented something
similar, albeit with a slightly different mapping scheme [2], [3], [4];
this has continued to develop. [1]
References
==========
Zeng Heng <zengheng4@huawei.com>:
[1] (v4) arm_mpam: Introduce the Narrow-PARTID feature for MPAM driver
https://lore.kernel.org/linux-arm-kernel/20250104101224.873926-1-zengheng4@huawei.com/
[2] (v3) arm_mpam: Introduce the Narrow-PARTID feature for MPAM driver
https://lore.kernel.org/linux-arm-kernel/20241207092136.2488426-1-zengheng4@huawei.com/
[3] (v2) arm_mpam: Introduce the Narrow-PARTID feature for MPAM driver
https://lore.kernel.org/linux-arm-kernel/20241119135104.595630-1-zengheng4@huawei.com/
[4] (v1) arm_mpam: Introduce the definitions of intPARTID and reqPARTID
https://lore.kernel.org/linux-arm-kernel/20241114135037.918470-1-zengheng4@huawei.com/
Dave Martin <Dave.Martin@arm.com>:
[5] (previous post) [RFC PATCH 0/6] Introduce flexible CLOSID/RMID translation
https://lore.kernel.org/lkml/20241212154000.330467-1-Dave.Martin@arm.com/
Dave Martin (11):
arm_mpam: Fix read-back of cloned resource controls under CDP
emulation
arm_mpam: Delete unused function resctrl_arch_set_rmid()
arm_mpam: Set INTERNAL as needed when setting MSC controls
fs/resctrl: Add arch hook to set up the default CLOSID
arm_mpam: resctrl: Implement arch hook to set up the default CLOSID
arm_mpam: Introduce flexible CLOSID/RMID translation
arm_mpam: [NFU] Rework ID remapping to use a kernel command-line
argument
arm_mpam: [NFU] Development diagnostics for MPAM ID assignments
arm_mpam: Add control partition identifier to mpam_config
arm_mpam: Skip inapplicable cases when reprogramming a resctrl domain
arm_mpam: Detect whether the system can alias PARTIDs
arch/arm64/include/asm/mpam.h | 21 +-
arch/x86/include/asm/resctrl.h | 12 +
drivers/platform/arm64/mpam/mpam_devices.c | 88 +++++--
drivers/platform/arm64/mpam/mpam_internal.h | 9 +
drivers/platform/arm64/mpam/mpam_resctrl.c | 260 +++++++++++++-------
fs/resctrl/rdtgroup.c | 26 +-
include/linux/arm_mpam.h | 5 +
include/linux/resctrl.h | 2 +
8 files changed, 299 insertions(+), 124 deletions(-)
base-commit: 4d17b37b670795954fd7a70c8ec83fb705a5e2ad
--
2.34.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* [RFC PATCH v2 01/11] arm_mpam: Fix read-back of cloned resource controls under CDP emulation
2025-01-17 15:10 [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
@ 2025-01-17 15:10 ` Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 02/11] arm_mpam: Delete unused function resctrl_arch_set_rmid() Dave Martin
` (21 subsequent siblings)
22 siblings, 0 replies; 26+ messages in thread
From: Dave Martin @ 2025-01-17 15:10 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-arm-kernel, Zeng Heng, Shaopeng Tan, James Morse
When reading resource control values through a resctrl schemata
file, the index into the configuration array in the MPAM driver
is currently translated incorrectly.
This means that when resctrl is mounted with the "cdp" option, the
wrong array entry may be returned or an out-of-bounds array access
may occur when reading back of control values for the MB resource.
Fix it by translating the index for affected resources as if they
were "DATA" resources. ("CODE" would also work, but because
resctrl_arch_update_one() clones schemata writes across "CODE" and
"DATA" for affected resources, it does not matter which one is
read.)
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
This issue was introduced by the non-upstream commit
"arm_mpam: resctrl: Improve CDP emulation"
(commit b097a6193546dbbd27cd19d7e16e1e750ebb030f in
git://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/snapshot/v6.12-rc1 )
---
drivers/platform/arm64/mpam/mpam_resctrl.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/drivers/platform/arm64/mpam/mpam_resctrl.c b/drivers/platform/arm64/mpam/mpam_resctrl.c
index 76ddbce0ea9c..30f2caec11d7 100644
--- a/drivers/platform/arm64/mpam/mpam_resctrl.c
+++ b/drivers/platform/arm64/mpam/mpam_resctrl.c
@@ -1097,6 +1097,14 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_ctrl_domain *d,
dom = container_of(d, struct mpam_resctrl_dom, resctrl_ctrl_dom);
cprops = &res->class->props;
+ /*
+ * When CDP is enabled, but the resource doesn't support it,
+ * the control is cloned across both partids.
+ * Pick one at random to read:
+ */
+ if (mpam_resctrl_hide_cdp(r->rid))
+ type = CDP_DATA;
+
partid = resctrl_get_config_index(closid, type);
cfg = &dom->comp->cfg[partid];
--
2.34.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* [RFC PATCH v2 02/11] arm_mpam: Delete unused function resctrl_arch_set_rmid()
2025-01-17 15:10 [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 01/11] arm_mpam: Fix read-back of cloned resource controls under CDP emulation Dave Martin
@ 2025-01-17 15:10 ` Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 03/11] arm_mpam: Set INTERNAL as needed when setting MSC controls Dave Martin
` (20 subsequent siblings)
22 siblings, 0 replies; 26+ messages in thread
From: Dave Martin @ 2025-01-17 15:10 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-arm-kernel, Zeng Heng, Shaopeng Tan, James Morse
The function resctrl_arch_set_rmid() appears to have been left
behind after earlier refactoring and is now unused.
To avoid needing to answer awkward questions about how this
function should apply partition ID namespace remappings, just get
rid of it.
No functional change.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
I'm assuming here that there are no pending branches that implement /
use this function.
When moving tasks between monitoring groups, resctrlfs currently
extracts the CLOSID from struct rdtgroup and passes it explicitly
to resctrl_arch_set_closid_rmid(), so there seems to be no need for
a separate helper that doesn't overwrite the arch's control
partition identifier.
---
arch/arm64/include/asm/mpam.h | 14 --------------
1 file changed, 14 deletions(-)
diff --git a/arch/arm64/include/asm/mpam.h b/arch/arm64/include/asm/mpam.h
index e5f385767174..df725d2d9d05 100644
--- a/arch/arm64/include/asm/mpam.h
+++ b/arch/arm64/include/asm/mpam.h
@@ -93,20 +93,6 @@ static inline u64 mpam_get_regval(struct task_struct *tsk)
#endif
}
-static inline void resctrl_arch_set_rmid(struct task_struct *tsk, u32 rmid)
-{
-#ifdef CONFIG_ARM64_MPAM
- u64 regval = mpam_get_regval(tsk);
-
- regval &= ~MPAM1_EL1_PMG_D;
- regval &= ~MPAM1_EL1_PMG_I;
- regval |= FIELD_PREP(MPAM1_EL1_PMG_D, rmid);
- regval |= FIELD_PREP(MPAM1_EL1_PMG_I, rmid);
-
- WRITE_ONCE(task_thread_info(tsk)->mpam_partid_pmg, regval);
-#endif
-}
-
static inline void mpam_thread_switch(struct task_struct *tsk)
{
u64 oldregval;
--
2.34.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* [RFC PATCH v2 03/11] arm_mpam: Set INTERNAL as needed when setting MSC controls
2025-01-17 15:10 [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 01/11] arm_mpam: Fix read-back of cloned resource controls under CDP emulation Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 02/11] arm_mpam: Delete unused function resctrl_arch_set_rmid() Dave Martin
@ 2025-01-17 15:10 ` Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 04/11] fs/resctrl: Add arch hook to set up the default CLOSID Dave Martin
` (19 subsequent siblings)
22 siblings, 0 replies; 26+ messages in thread
From: Dave Martin @ 2025-01-17 15:10 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-arm-kernel, Zeng Heng, Shaopeng Tan, James Morse
Currently, when an MSC implements PARTID narrowing, MPAMCFG_PART_SEL is
left set to the reqPARTID while programming resource controls in
an MSC.
The MPAM architecture does not guarantee that any particular resource
controls will be updated correctly in this scenario. Instead,
MPAMCFG_PART_SEL must be written with the corresponding intPARTID and
with the INTERNAL bit set before attempting to program resource controls.
Only the PARTID->intPARTID mappings can be written without the INTERNAL
bit set in MPAMCFG_PART_SEL.
Fix it, by rewriting MPAMCFG_PART_SEL appropriately after setting the
intPARTID mapping.
The MPAMCFG_INTPARTID_INTERNAL flag is not currently accepted as
input to the __mpam_part_sel() helper. In the interest of keeping
the code clean, break this helper up.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Signed-off-by: James Morse <james.morse@arm.com>
---
drivers/platform/arm64/mpam/mpam_devices.c | 26 +++++++++++++++++-----
1 file changed, 20 insertions(+), 6 deletions(-)
diff --git a/drivers/platform/arm64/mpam/mpam_devices.c b/drivers/platform/arm64/mpam/mpam_devices.c
index 9463045c0011..f965e6639e0d 100644
--- a/drivers/platform/arm64/mpam/mpam_devices.c
+++ b/drivers/platform/arm64/mpam/mpam_devices.c
@@ -204,15 +204,27 @@ static u64 mpam_msc_read_esr(struct mpam_msc *msc)
return (esr_high << 32) | esr_low;
}
+static void __mpam_part_sel_raw(u32 partsel, struct mpam_msc *msc)
+{
+ lockdep_assert_held(&msc->part_sel_lock);
+ mpam_write_partsel_reg(msc, PART_SEL, partsel);
+}
+
static void __mpam_part_sel(u8 ris_idx, u16 partid, struct mpam_msc *msc)
{
- u32 partsel;
+ u32 partsel = FIELD_PREP(MPAMCFG_PART_SEL_RIS, ris_idx) |
+ FIELD_PREP(MPAMCFG_PART_SEL_PARTID_SEL, partid);
- lockdep_assert_held(&msc->part_sel_lock);
+ __mpam_part_sel_raw(partsel, msc);
+}
- partsel = FIELD_PREP(MPAMCFG_PART_SEL_RIS, ris_idx) |
- FIELD_PREP(MPAMCFG_PART_SEL_PARTID_SEL, partid);
- mpam_write_partsel_reg(msc, PART_SEL, partsel);
+static void __mpam_intpart_sel(u8 ris_idx, u16 intpartid, struct mpam_msc *msc)
+{
+ u32 partsel = FIELD_PREP(MPAMCFG_PART_SEL_RIS, ris_idx) |
+ FIELD_PREP(MPAMCFG_PART_SEL_PARTID_SEL, intpartid) |
+ MPAMCFG_PART_SEL_INTERNAL;
+
+ __mpam_part_sel_raw(partsel, msc);
}
int mpam_register_requestor(u16 partid_max, u8 pmg_max)
@@ -1530,9 +1542,11 @@ static void mpam_reprogram_ris_partid(struct mpam_msc_ris *ris, u16 partid,
mutex_lock(&msc->part_sel_lock);
__mpam_part_sel(ris->ris_idx, partid, msc);
- if (mpam_has_feature(mpam_feat_partid_nrw, rprops))
+ if (mpam_has_feature(mpam_feat_partid_nrw, rprops)) {
mpam_write_partsel_reg(msc, INTPARTID,
MPAMCFG_INTPARTID_INTERNAL | partid);
+ __mpam_intpart_sel(ris->ris_idx, partid, msc);
+ }
if (mpam_has_feature(mpam_feat_cpor_part, rprops)) {
if (mpam_has_feature(mpam_feat_cpor_part, cfg))
--
2.34.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* [RFC PATCH v2 04/11] fs/resctrl: Add arch hook to set up the default CLOSID
2025-01-17 15:10 [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
` (2 preceding siblings ...)
2025-01-17 15:10 ` [RFC PATCH v2 03/11] arm_mpam: Set INTERNAL as needed when setting MSC controls Dave Martin
@ 2025-01-17 15:10 ` Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 05/11] arm_mpam: resctrl: Implement " Dave Martin
` (18 subsequent siblings)
22 siblings, 0 replies; 26+ messages in thread
From: Dave Martin @ 2025-01-17 15:10 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-arm-kernel, Zeng Heng, Shaopeng Tan, James Morse
Currently, the resctrl core code assumes that CLOSID 0 requires no
special initialisation before use.
This is true on x86, but may not be true on other architectures
(including Arm MPAM, in certain configurations).
Since the allocation of CLOSIDs can depend on parameters such as
resctrl filesystem mount options that are not available at arch driver
init time, an arch hook is needed to set up the default CLOSID when
the resctrl filesystem is mounted.
Add a suitable arch hook resctrl_arch_init_domains().
Wire it up to arch stubs that do nothing, for now.
(This is also the right thing for x86.)
No functional change.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
arch/x86/include/asm/resctrl.h | 12 ++++++++++++
fs/resctrl/rdtgroup.c | 26 ++++++++++++++++++++++++--
include/linux/arm_mpam.h | 10 ++++++++++
include/linux/resctrl.h | 2 ++
4 files changed, 48 insertions(+), 2 deletions(-)
diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h
index 332b0617d728..b5fbb94e756a 100644
--- a/arch/x86/include/asm/resctrl.h
+++ b/arch/x86/include/asm/resctrl.h
@@ -50,6 +50,18 @@ DECLARE_STATIC_KEY_FALSE(rdt_enable_key);
DECLARE_STATIC_KEY_FALSE(rdt_alloc_enable_key);
DECLARE_STATIC_KEY_FALSE(rdt_mon_enable_key);
+/* The default CLOSID needs no special setup on x86: */
+static inline bool resctrl_arch_default_closid_needs_init(void)
+{
+ return false;
+}
+
+/* ... so this won't be needed either: */
+static inline int resctrl_arch_init_domains(struct rdt_resource *r, u32 closid)
+{
+ return 0;
+}
+
static inline bool resctrl_arch_alloc_capable(void)
{
return rdt_alloc_capable;
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 30f1dd05e10b..200f92e8f619 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -65,6 +65,8 @@ static struct seq_buf last_cmd_status;
static char last_cmd_status_buf[512];
+static int rdtgroup_init_first_alloc(struct rdtgroup *rdtgrp);
+
static int rdtgroup_setup_root(struct rdt_fs_context *ctx);
static void rdtgroup_destroy_root(void);
@@ -2542,6 +2544,9 @@ static int rdt_get_tree(struct fs_context *fc)
}
closid_init();
+ ret = rdtgroup_init_first_alloc(&rdtgroup_default);
+ if (ret)
+ goto out_schemata_free;
if (resctrl_arch_mon_capable())
flags |= RFTYPE_MON;
@@ -3230,7 +3235,7 @@ static void rdtgroup_init_mba(struct rdt_resource *r, u32 closid)
}
/* Initialize the RDT group's allocations. */
-static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp)
+static int __rdtgroup_init_alloc(struct rdtgroup *rdtgrp, bool first_time_init)
{
struct resctrl_schema *s;
struct rdt_resource *r;
@@ -3251,7 +3256,11 @@ static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp)
goto out;
}
- ret = resctrl_arch_update_domains(r, rdtgrp->closid);
+ if (first_time_init)
+ ret = resctrl_arch_init_domains(r, rdtgrp->closid);
+ else
+ ret = resctrl_arch_update_domains(r, rdtgrp->closid);
+
if (ret < 0) {
rdt_last_cmd_puts("Failed to initialize allocations\n");
goto out;
@@ -3265,6 +3274,19 @@ static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp)
return ret;
}
+static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp)
+{
+ return __rdtgroup_init_alloc(rdtgrp, false);
+}
+
+static int rdtgroup_init_first_alloc(struct rdtgroup *rdtgrp)
+{
+ if (!resctrl_arch_default_closid_needs_init())
+ return 0;
+
+ return __rdtgroup_init_alloc(rdtgrp, true);
+}
+
static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp)
{
int ret;
diff --git a/include/linux/arm_mpam.h b/include/linux/arm_mpam.h
index c00b5fc1d235..dd5f18474846 100644
--- a/include/linux/arm_mpam.h
+++ b/include/linux/arm_mpam.h
@@ -45,6 +45,16 @@ static inline unsigned int resctrl_arch_round_mon_val(unsigned int val)
return val;
}
+static inline bool resctrl_arch_default_closid_needs_init(void)
+{
+ return false;
+}
+
+static inline int resctrl_arch_init_domains(struct rdt_resource *r, u32 closid)
+{
+ return 0;
+}
+
bool resctrl_arch_alloc_capable(void);
bool resctrl_arch_mon_capable(void);
bool resctrl_arch_is_llc_occupancy_enabled(void);
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index be9d9f4c7b81..fc00b080f1a1 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -371,6 +371,8 @@ u32 resctrl_arch_system_num_rmid_idx(void);
struct rdt_domain_hdr *resctrl_arch_find_domain(struct list_head *domain_list,
int id);
+bool resctrl_arch_default_closid_needs_init(void);
+int resctrl_arch_init_domains(struct rdt_resource *r, u32 closid);
int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid);
bool resctrl_arch_is_evt_configurable(enum resctrl_event_id evt);
--
2.34.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* [RFC PATCH v2 05/11] arm_mpam: resctrl: Implement arch hook to set up the default CLOSID
2025-01-17 15:10 [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
` (3 preceding siblings ...)
2025-01-17 15:10 ` [RFC PATCH v2 04/11] fs/resctrl: Add arch hook to set up the default CLOSID Dave Martin
@ 2025-01-17 15:10 ` Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 06/11] arm_mpam: Introduce flexible CLOSID/RMID translation Dave Martin
` (17 subsequent siblings)
22 siblings, 0 replies; 26+ messages in thread
From: Dave Martin @ 2025-01-17 15:10 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-arm-kernel, Zeng Heng, Shaopeng Tan, James Morse
Since the allocation of CLOSIDs can now depend on parameters such
as resctrl filesystem mount options that are not available at arch
driver init time, all MPAM resource allocations need to be
reprogrammed when resctrl is mounted.
Implement resctrl_arch_init_domains() so that resctrl will now
reprogram the default CLOSID's resource allocations at filesystem
mount time, in the same way as when creating a new resctrl control
group.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
drivers/platform/arm64/mpam/mpam_resctrl.c | 15 +++++++++++++--
include/linux/arm_mpam.h | 7 +------
2 files changed, 14 insertions(+), 8 deletions(-)
diff --git a/drivers/platform/arm64/mpam/mpam_resctrl.c b/drivers/platform/arm64/mpam/mpam_resctrl.c
index 30f2caec11d7..aa8a6c077918 100644
--- a/drivers/platform/arm64/mpam/mpam_resctrl.c
+++ b/drivers/platform/arm64/mpam/mpam_resctrl.c
@@ -1209,7 +1209,8 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_ctrl_domain *d,
}
/* TODO: this is IPI heavy */
-int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)
+static int __resctrl_arch_update_domains(struct rdt_resource *r, u32 closid,
+ bool force)
{
int err = 0;
enum resctrl_conf_type t;
@@ -1222,7 +1223,7 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)
list_for_each_entry(d, &r->ctrl_domains, hdr.list) {
for (t = 0; t < CDP_NUM_TYPES; t++) {
cfg = &d->staged_config[t];
- if (!cfg->have_new_ctrl)
+ if (!force && !cfg->have_new_ctrl)
continue;
err = resctrl_arch_update_one(r, d, closid, t,
@@ -1235,6 +1236,16 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)
return err;
}
+int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)
+{
+ return __resctrl_arch_update_domains(r, closid, false);
+}
+
+int resctrl_arch_init_domains(struct rdt_resource *r, u32 closid)
+{
+ return __resctrl_arch_update_domains(r, closid, true);
+}
+
void resctrl_arch_reset_resources(void)
{
int i, idx;
diff --git a/include/linux/arm_mpam.h b/include/linux/arm_mpam.h
index dd5f18474846..e6b188ab735b 100644
--- a/include/linux/arm_mpam.h
+++ b/include/linux/arm_mpam.h
@@ -47,12 +47,7 @@ static inline unsigned int resctrl_arch_round_mon_val(unsigned int val)
static inline bool resctrl_arch_default_closid_needs_init(void)
{
- return false;
-}
-
-static inline int resctrl_arch_init_domains(struct rdt_resource *r, u32 closid)
-{
- return 0;
+ return true;
}
bool resctrl_arch_alloc_capable(void);
--
2.34.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* [RFC PATCH v2 06/11] arm_mpam: Introduce flexible CLOSID/RMID translation
2025-01-17 15:10 [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
` (4 preceding siblings ...)
2025-01-17 15:10 ` [RFC PATCH v2 05/11] arm_mpam: resctrl: Implement " Dave Martin
@ 2025-01-17 15:10 ` Dave Martin
2025-01-29 10:40 ` Shaopeng Tan (Fujitsu)
2025-01-17 15:10 ` [RFC PATCH v2 07/11] arm_mpam: [NFU] Rework ID remapping to use a kernel command-line argument Dave Martin
` (16 subsequent siblings)
22 siblings, 1 reply; 26+ messages in thread
From: Dave Martin @ 2025-01-17 15:10 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-arm-kernel, Zeng Heng, Shaopeng Tan, James Morse
Currently, the MPAM driver uses the resctrl CLOSID directly as the
MPAM Partition Identifier (PARTID; possibly modified to give
different PARTIDs to code and data when Code/Data Partitioning /
CDP is enabled), and uses the resctrl RMID directly as the MPAM
Performance Monitoring Group identifier (PMG).
In preparation for using the MPAM PARTID Narrowing feature to allow
more resctrl monitoring groups to be provided than the number of
PMG values supported by the hardware, a more flexible remapping
scheme is needed.
Factor out ID translation operations in the MPAM resctrl glue code
into a couple of self-contained helpers, and call them as
appropriate.
The translation scheme has a single parameter, partid_per_closid,
which is currently hard-wired to 1.
As a result, this patch should have no effect on functionality.
Logic to determine / control the mapping parameters can be
introduced later.
The ID transformation may be visualised as follows:
without CDP:
+---------------------------------------------------+-----------+
| CLOSID | RMID |
+---------------------------------------------------+-------+---+
| PARTID : |PMG|
+-----------------------------------------------------------+---+
and with CDP, where the "CDP" field is 0 for the data sub-
partition and 1 for the code sub-partition:
+-----------------------------------------------+---+-----------+
| CLOSID |CDP| RMID |
+-----------------------------------------------+---+-------+---+
| PARTID : |PMG|
+-----------------------------------------------------------+---+
where each box represents a non-negative integer spanning some
fixed range starting from zero, and horizontal concatenation
denotes multiplying the value denoted by the left-hand box by the
span of the right-hand box, and adding the result to the value
denoted by the right-hand box; thus, most-significant fields are on
the left.
(Except for the span of each field not necessarily being a power of
two, this is conceptually identical to concatenation and cutting of
bitfields.)
The dotted segment indicates the least significant part of PARTID,
which spans partid_per_closid PARTIDs and discriminates monitoring
groups that use the same MPAM PMG value.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
Changes since RFCv1:
* [bugfix] Add CDP index properly in mpam_resctrl_partid_range().
(Also reported by Shaopeng Tan.)
* [relaxation] Remove requirement that mpam_pmg_max + 1 is a power
of two.
(This constraint is redundant, due to the use of multiply/add
transformations of MPAM IDs in place of shift/OR.)
---
drivers/platform/arm64/mpam/mpam_resctrl.c | 211 +++++++++++----------
1 file changed, 112 insertions(+), 99 deletions(-)
diff --git a/drivers/platform/arm64/mpam/mpam_resctrl.c b/drivers/platform/arm64/mpam/mpam_resctrl.c
index aa8a6c077918..f834753a3274 100644
--- a/drivers/platform/arm64/mpam/mpam_resctrl.c
+++ b/drivers/platform/arm64/mpam/mpam_resctrl.c
@@ -156,6 +156,18 @@ static bool mpam_resctrl_hide_cdp(enum resctrl_res_level rid)
return cdp_enabled && !resctrl_arch_get_cdp_enabled(rid);
}
+static unsigned int partid_per_closid = 1;
+
+static unsigned int mpam_num_pmg(void)
+{
+ return mpam_pmg_max + 1;
+}
+
+static unsigned int mpam_num_rmid(void)
+{
+ return mpam_num_pmg() * partid_per_closid;
+}
+
/*
* MSC may raise an error interrupt if it sees an out or range partid/pmg,
* and go on to truncate the value. Regardless of what the hardware supports,
@@ -163,35 +175,73 @@ static bool mpam_resctrl_hide_cdp(enum resctrl_res_level rid)
*/
u32 resctrl_arch_get_num_closid(struct rdt_resource *ignored)
{
- return mpam_partid_max + 1;
+ u32 res = (mpam_partid_max + 1) / partid_per_closid;
+
+ WARN_ON(res < 1);
+ return res;
}
-u32 resctrl_arch_system_num_rmid_idx(void)
+static void mpam_resctrl_partid_range(u32 closid, enum resctrl_conf_type type,
+ const struct rdt_resource *r,
+ u16 *min_partid, u16 *max_partid)
{
- u8 closid_shift = fls(mpam_pmg_max);
- u32 num_partid = resctrl_arch_get_num_closid(NULL);
+ u16 base_partid = closid;
+ u16 span = 1;
- return num_partid << closid_shift;
+ if (cdp_enabled) {
+ base_partid *= 2;
+ if (mpam_resctrl_hide_cdp(r->rid) ||
+ type == CDP_NONE)
+ span *= 2;
+ else if (type == CDP_CODE)
+ base_partid++;
+ }
+
+ *min_partid = base_partid * partid_per_closid;
+ if (max_partid)
+ *max_partid = *min_partid + (span * partid_per_closid - 1);
}
-u32 resctrl_arch_rmid_idx_encode(u32 closid, u32 rmid)
+static void mpam_resctrl_hwid(u32 closid, u32 rmid,
+ u16 *partid_d, u16 *partid_i, u8 *pmg)
{
- u8 closid_shift = fls(mpam_pmg_max);
+ const u16 pmg_hi = rmid / mpam_num_pmg();
+ const u16 pmg_lo = rmid % mpam_num_pmg();
+ u16 base_partid = closid;
+ u16 partid_stride = 0;
+
+ if (cdp_enabled) {
+ base_partid *= 2;
+ partid_stride = 1;
+ }
+
+ partid_stride *= partid_per_closid;
+ base_partid *= partid_per_closid;
+ base_partid += pmg_hi;
- BUG_ON(closid_shift > 8);
+ *partid_d = base_partid;
+ if (partid_i)
+ *partid_i = base_partid + partid_stride;
- return (closid << closid_shift) | rmid;
+ *pmg = pmg_lo;
}
-void resctrl_arch_rmid_idx_decode(u32 idx, u32 *closid, u32 *rmid)
+u32 resctrl_arch_rmid_idx_encode(u32 closid, u32 rmid)
+{
+ return closid * mpam_num_rmid() + rmid;
+}
+
+u32 resctrl_arch_system_num_rmid_idx(void)
{
- u8 closid_shift = fls(mpam_pmg_max);
- u32 pmg_mask = ~(~0 << closid_shift);
+ u32 num_closid = resctrl_arch_get_num_closid(NULL);
- BUG_ON(closid_shift > 8);
+ return resctrl_arch_rmid_idx_encode(num_closid, 0);
+}
- *closid = idx >> closid_shift;
- *rmid = idx & pmg_mask;
+void resctrl_arch_rmid_idx_decode(u32 idx, u32 *closid, u32 *rmid)
+{
+ *closid = idx / mpam_num_rmid();
+ *rmid = idx % mpam_num_rmid();
}
void resctrl_arch_sched_in(struct task_struct *tsk)
@@ -203,21 +253,14 @@ void resctrl_arch_sched_in(struct task_struct *tsk)
void resctrl_arch_set_cpu_default_closid_rmid(int cpu, u32 closid, u32 rmid)
{
+ u16 partid_d, partid_i;
+ u8 pmg;
+
BUG_ON(closid > U16_MAX);
BUG_ON(rmid > U8_MAX);
- if (!cdp_enabled) {
- mpam_set_cpu_defaults(cpu, closid, closid, rmid, rmid);
- } else {
- /*
- * When CDP is enabled, resctrl halves the closid range and we
- * use odd/even partid for one closid.
- */
- u32 partid_d = resctrl_get_config_index(closid, CDP_DATA);
- u32 partid_i = resctrl_get_config_index(closid, CDP_CODE);
-
- mpam_set_cpu_defaults(cpu, partid_d, partid_i, rmid, rmid);
- }
+ mpam_resctrl_hwid(closid, rmid, &partid_d, &partid_i, &pmg);
+ mpam_set_cpu_defaults(cpu, partid_d, partid_i, pmg, pmg);
}
void resctrl_arch_sync_cpu_closid_rmid(void *info)
@@ -236,41 +279,38 @@ void resctrl_arch_sync_cpu_closid_rmid(void *info)
void resctrl_arch_set_closid_rmid(struct task_struct *tsk, u32 closid, u32 rmid)
{
+ u16 partid_d, partid_i;
+ u8 pmg;
+
BUG_ON(closid > U16_MAX);
BUG_ON(rmid > U8_MAX);
- if (!cdp_enabled) {
- mpam_set_task_partid_pmg(tsk, closid, closid, rmid, rmid);
- } else {
- u32 partid_d = resctrl_get_config_index(closid, CDP_DATA);
- u32 partid_i = resctrl_get_config_index(closid, CDP_CODE);
-
- mpam_set_task_partid_pmg(tsk, partid_d, partid_i, rmid, rmid);
- }
+ mpam_resctrl_hwid(closid, rmid, &partid_d, &partid_i, &pmg);
+ mpam_set_task_partid_pmg(tsk, partid_d, partid_i, pmg, pmg);
}
bool resctrl_arch_match_closid(struct task_struct *tsk, u32 closid)
{
+ u16 min_partid, max_partid;
u64 regval = mpam_get_regval(tsk);
u32 tsk_closid = FIELD_GET(MPAM1_EL1_PARTID_D, regval);
- if (cdp_enabled)
- tsk_closid >>= 1;
-
- return tsk_closid == closid;
+ mpam_resctrl_partid_range(closid, CDP_NONE, NULL,
+ &min_partid, &max_partid);
+ return tsk_closid >= min_partid && tsk_closid <= max_partid;
}
/* The task's pmg is not unique, the partid must be considered too */
bool resctrl_arch_match_rmid(struct task_struct *tsk, u32 closid, u32 rmid)
{
+ u16 partid_d;
+ u8 pmg;
u64 regval = mpam_get_regval(tsk);
- u32 tsk_closid = FIELD_GET(MPAM1_EL1_PARTID_D, regval);
- u32 tsk_rmid = FIELD_GET(MPAM1_EL1_PMG_D, regval);
-
- if (cdp_enabled)
- tsk_closid >>= 1;
+ u16 tsk_partid_d = FIELD_GET(MPAM1_EL1_PARTID_D, regval);
+ u8 tsk_pmg = FIELD_GET(MPAM1_EL1_PMG_D, regval);
- return (tsk_closid == closid) && (tsk_rmid == rmid);
+ mpam_resctrl_hwid(closid, rmid, &partid_d, NULL, &pmg);
+ return (tsk_partid_d == partid_d) && (tsk_pmg == pmg);
}
struct rdt_resource *resctrl_arch_get_resource(enum resctrl_res_level l)
@@ -370,6 +410,7 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_mon_domain *d,
struct mpam_resctrl_dom *dom;
u32 mon = *(u32 *)arch_mon_ctx;
enum mpam_device_features type;
+ u16 partid_d, partid_i, pmg;
resctrl_arch_rmid_read_context_check();
@@ -391,27 +432,23 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_mon_domain *d,
if (cfg.mon == USE_RMID_IDX)
cfg.mon = resctrl_arch_rmid_idx_encode(closid, rmid);
+ mpam_resctrl_hwid(closid, rmid, &partid_d, &partid_i, &cfg.pmg);
cfg.match_pmg = true;
- cfg.pmg = rmid;
+ cfg.pmg = pmg;
cfg.opts = resctrl_evt_config_to_mpam(dom->mbm_local_evt_cfg);
if (irqs_disabled()) {
/* Check if we can access this domain without an IPI */
err = -EIO;
} else {
- if (cdp_enabled) {
- cfg.partid = closid << 1;
- err = mpam_msmon_read(dom->comp, &cfg, type, val);
- if (err)
- return err;
+ cfg.partid = partid_d;
+ err = mpam_msmon_read(dom->comp, &cfg, type, val);
- cfg.partid += 1;
+ if (partid_i != partid_d) {
+ cfg.partid = partid_i;
err = mpam_msmon_read(dom->comp, &cfg, type, &cdp_val);
if (!err)
*val += cdp_val;
- } else {
- cfg.partid = closid;
- err = mpam_msmon_read(dom->comp, &cfg, type, val);
}
}
@@ -423,6 +460,7 @@ void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_mon_domain *d,
{
struct mon_cfg cfg;
struct mpam_resctrl_dom *dom;
+ u16 partid_d, partid_i;
if (eventid != QOS_L3_MBM_LOCAL_EVENT_ID)
return;
@@ -433,14 +471,13 @@ void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_mon_domain *d,
dom = container_of(d, struct mpam_resctrl_dom, resctrl_mon_dom);
- if (cdp_enabled) {
- cfg.partid = closid << 1;
- mpam_msmon_reset_mbwu(dom->comp, &cfg);
+ mpam_resctrl_hwid(closid, rmid, &partid_d, &partid_i, &cfg.pmg);
- cfg.partid += 1;
- mpam_msmon_reset_mbwu(dom->comp, &cfg);
- } else {
- cfg.partid = closid;
+ cfg.partid = partid_d;
+ mpam_msmon_reset_mbwu(dom->comp, &cfg);
+
+ if (partid_i != partid_d) {
+ cfg.partid = partid_i;
mpam_msmon_reset_mbwu(dom->comp, &cfg);
}
}
@@ -1051,15 +1088,6 @@ int mpam_resctrl_setup(void)
err = -EOPNOTSUPP;
if (!err) {
- if (!is_power_of_2(mpam_pmg_max + 1)) {
- /*
- * If not all the partid*pmg values are valid indexes,
- * resctrl may allocate pmg that don't exist. This
- * should cause an error interrupt.
- */
- pr_warn("Number of PMG is not a power of 2! resctrl may misbehave");
- }
-
err = resctrl_init();
if (!err)
WRITE_ONCE(resctrl_enabled, true);
@@ -1081,7 +1109,7 @@ static void mpam_resctrl_exit(void)
u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_ctrl_domain *d,
u32 closid, enum resctrl_conf_type type)
{
- u32 partid;
+ u16 partid;
struct mpam_config *cfg;
struct mpam_props *cprops;
struct mpam_resctrl_res *res;
@@ -1097,15 +1125,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_ctrl_domain *d,
dom = container_of(d, struct mpam_resctrl_dom, resctrl_ctrl_dom);
cprops = &res->class->props;
- /*
- * When CDP is enabled, but the resource doesn't support it,
- * the control is cloned across both partids.
- * Pick one at random to read:
- */
- if (mpam_resctrl_hide_cdp(r->rid))
- type = CDP_DATA;
-
- partid = resctrl_get_config_index(closid, type);
+ mpam_resctrl_partid_range(closid, type, r, &partid, NULL);
cfg = &dom->comp->cfg[partid];
switch (r->rid) {
@@ -1126,7 +1146,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_ctrl_domain *d,
return -EINVAL;
}
- if (!r->alloc_capable || partid >= resctrl_arch_get_num_closid(r) ||
+ if (!r->alloc_capable || partid > mpam_partid_max ||
!mpam_has_feature(configured_by, cfg))
return resctrl_get_default_ctrl(r);
@@ -1147,8 +1167,8 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_ctrl_domain *d,
int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_ctrl_domain *d,
u32 closid, enum resctrl_conf_type t, u32 cfg_val)
{
- int err;
- u32 partid;
+ int err = 0;
+ u16 partid, min_partid, max_partid;
struct mpam_config cfg;
struct mpam_props *cprops;
struct mpam_resctrl_res *res;
@@ -1163,8 +1183,10 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_ctrl_domain *d,
dom = container_of(d, struct mpam_resctrl_dom, resctrl_ctrl_dom);
cprops = &res->class->props;
- partid = resctrl_get_config_index(closid, t);
- if (!r->alloc_capable || partid >= resctrl_arch_get_num_closid(r))
+ mpam_resctrl_partid_range(closid, t, r, &min_partid, &max_partid);
+ if (!r->alloc_capable ||
+ min_partid > mpam_partid_max ||
+ max_partid > mpam_partid_max)
return -EINVAL;
cfg.features = 0;
@@ -1190,22 +1212,13 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_ctrl_domain *d,
return -EINVAL;
}
- /*
- * When CDP is enabled, but the resource doesn't support it, we need to
- * apply the same configuration to the other partid.
- */
- if (mpam_resctrl_hide_cdp(r->rid)) {
- partid = resctrl_get_config_index(closid, CDP_CODE);
+ for (partid = min_partid; partid <= max_partid; partid++) {
err = mpam_apply_config(dom->comp, partid, &cfg);
if (err)
- return err;
-
- partid = resctrl_get_config_index(closid, CDP_DATA);
- return mpam_apply_config(dom->comp, partid, &cfg);
-
- } else {
- return mpam_apply_config(dom->comp, partid, &cfg);
+ break;
}
+
+ return err;
}
/* TODO: this is IPI heavy */
--
2.34.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* [RFC PATCH v2 07/11] arm_mpam: [NFU] Rework ID remapping to use a kernel command-line argument
2025-01-17 15:10 [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
` (5 preceding siblings ...)
2025-01-17 15:10 ` [RFC PATCH v2 06/11] arm_mpam: Introduce flexible CLOSID/RMID translation Dave Martin
@ 2025-01-17 15:10 ` Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 08/11] arm_mpam: [NFU] Development diagnostics for MPAM ID assignments Dave Martin
` (15 subsequent siblings)
22 siblings, 0 replies; 26+ messages in thread
From: Dave Martin @ 2025-01-17 15:10 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-arm-kernel, Zeng Heng, Shaopeng Tan, James Morse
Hacking the source is an acceptable interface for experimentation,
but not very convenient.
Instead, add a kernel command-line parameter
mpam_partid_per_closid=<n> to specify how many PARTIDs are used to
provide monitoring groups for each resctrl CLOSID.
This change is not intended for upstream.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
drivers/platform/arm64/mpam/mpam_resctrl.c | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)
diff --git a/drivers/platform/arm64/mpam/mpam_resctrl.c b/drivers/platform/arm64/mpam/mpam_resctrl.c
index f834753a3274..eb1ef5d2bc57 100644
--- a/drivers/platform/arm64/mpam/mpam_resctrl.c
+++ b/drivers/platform/arm64/mpam/mpam_resctrl.c
@@ -3,6 +3,9 @@
#define pr_fmt(fmt) "mpam: resctrl: " fmt
+#undef KBUILD_MODNAME
+#define KBUILD_MODNAME "mpam"
+
#include <linux/arm_mpam.h>
#include <linux/cacheinfo.h>
#include <linux/cpu.h>
@@ -10,6 +13,7 @@
#include <linux/errno.h>
#include <linux/limits.h>
#include <linux/list.h>
+#include <linux/moduleparam.h>
#include <linux/printk.h>
#include <linux/rculist.h>
#include <linux/resctrl.h>
@@ -158,6 +162,24 @@ static bool mpam_resctrl_hide_cdp(enum resctrl_res_level rid)
static unsigned int partid_per_closid = 1;
+static int mpam_resctrl_partid_per_closid_set(const char *val,
+ const struct kernel_param *kp)
+{
+ /*
+ * 16 in an arbitrary maximum, sufficient for experimentation
+ * but not ridiculously large:
+ */
+ return param_set_uint_minmax(val, kp, 1, 16);
+}
+
+static const struct kernel_param_ops mpam_resctrl_partid_per_closid_ops = {
+ .set = mpam_resctrl_partid_per_closid_set,
+ .get = param_get_uint,
+};
+
+device_param_cb(partid_per_closid, &mpam_resctrl_partid_per_closid_ops,
+ &partid_per_closid, 0444);
+
static unsigned int mpam_num_pmg(void)
{
return mpam_pmg_max + 1;
--
2.34.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* [RFC PATCH v2 08/11] arm_mpam: [NFU] Development diagnostics for MPAM ID assignments
2025-01-17 15:10 [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
` (6 preceding siblings ...)
2025-01-17 15:10 ` [RFC PATCH v2 07/11] arm_mpam: [NFU] Rework ID remapping to use a kernel command-line argument Dave Martin
@ 2025-01-17 15:10 ` Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 09/11] arm_mpam: Add control partition identifier to mpam_config Dave Martin
` (14 subsequent siblings)
22 siblings, 0 replies; 26+ messages in thread
From: Dave Martin @ 2025-01-17 15:10 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-arm-kernel, Zeng Heng, Shaopeng Tan, James Morse
Purely for development purposes, add some printks to show what MPAM
IDs actually get assigned and configured.
This would otherwise be observable only though statistical
measurements of performance (or not observable at all).
This change is not intended for upstream.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
arch/arm64/include/asm/mpam.h | 7 +++++++
drivers/platform/arm64/mpam/mpam_devices.c | 6 ++++++
2 files changed, 13 insertions(+)
diff --git a/arch/arm64/include/asm/mpam.h b/arch/arm64/include/asm/mpam.h
index df725d2d9d05..4ae2e635e261 100644
--- a/arch/arm64/include/asm/mpam.h
+++ b/arch/arm64/include/asm/mpam.h
@@ -66,6 +66,9 @@ static inline void mpam_set_cpu_defaults(int cpu, u16 partid_d, u16 partid_i,
default_val |= FIELD_PREP(MPAM1_EL1_PMG_I, pmg_i);
WRITE_ONCE(per_cpu(arm64_mpam_default, cpu), default_val);
+
+ pr_info("MPAM: CPU%d -> (D=%d:%d, I=%d:%d)\n",
+ cpu, partid_d, pmg_d, partid_i, pmg_i);
}
static inline void mpam_set_task_partid_pmg(struct task_struct *tsk,
@@ -81,6 +84,10 @@ static inline void mpam_set_task_partid_pmg(struct task_struct *tsk,
regval |= FIELD_PREP(MPAM1_EL1_PMG_I, pmg_i);
WRITE_ONCE(task_thread_info(tsk)->mpam_partid_pmg, regval);
+
+ pr_info("MPAM: task %s[%d/%d] -> (D=%d:%d, I=%d:%d)\n",
+ current->comm, current->pid, current->tgid,
+ partid_d, pmg_d, partid_i, pmg_i);
#endif
}
diff --git a/drivers/platform/arm64/mpam/mpam_devices.c b/drivers/platform/arm64/mpam/mpam_devices.c
index f965e6639e0d..6b326506c359 100644
--- a/drivers/platform/arm64/mpam/mpam_devices.c
+++ b/drivers/platform/arm64/mpam/mpam_devices.c
@@ -3149,6 +3149,12 @@ int mpam_apply_config(struct mpam_component *comp, u16 partid,
mpam_extend_config(comp->class, cfg);
+ pr_info("mpam_apply_config(): comp %d partid %d cfg={0x%x: 0x%lx, 0x%lx, %u, %u}\n",
+ comp->comp_id, partid, cfg->features,
+ (unsigned long)cfg->cpbm,
+ (unsigned long)cfg->mbw_pbm,
+ cfg->mbw_min, cfg->mbw_max);
+
if (!mpam_update_config(&comp->cfg[partid], cfg))
return 0;
--
2.34.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* [RFC PATCH v2 09/11] arm_mpam: Add control partition identifier to mpam_config
2025-01-17 15:10 [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
` (7 preceding siblings ...)
2025-01-17 15:10 ` [RFC PATCH v2 08/11] arm_mpam: [NFU] Development diagnostics for MPAM ID assignments Dave Martin
@ 2025-01-17 15:10 ` Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 10/11] arm_mpam: Skip inapplicable cases when reprogramming a resctrl domain Dave Martin
` (13 subsequent siblings)
22 siblings, 0 replies; 26+ messages in thread
From: Dave Martin @ 2025-01-17 15:10 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-arm-kernel, Zeng Heng, Shaopeng Tan, James Morse
Currently, MPAM PARTIDs are assigned 1:1 to resctrl control groups.
This means that an MSC that implements PARTID Narrowing can simply
program the PARTID value for intPARTID where needed.
In order to enable multiple PARTIDs to be assigned to a single
control group in a flexible manner, the PARTID value cannot be used
directly for the intPARTID, so in the mpam_devices backend,
Narrowing MSCs need an explicit identifier for the control /
resource regulation group.
Add a field cpartid to struct mpam_config to hold this identifier,
and wire up the resctrl glue code to pass a suitable value.
In common with the handling of other parameters, make programming
of MPAMCFG_INTPARTID unconditional when applying control updates.
When no cpartid is supplied, e.g., during driver startup, intPARTID
is programmed with 0. (There is no architectural reset value for
MPAMCFG_INTPARTID even for PARTID 0, but the architecture can be
interpreted as suggesting intPARTID 0 as the default, so this is as
good a choice as any.) Control updates through resctrl will always
supply a cpartid, identifying the affected resctrl control
partition.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
drivers/platform/arm64/mpam/mpam_devices.c | 10 ++++++++--
drivers/platform/arm64/mpam/mpam_internal.h | 7 +++++++
drivers/platform/arm64/mpam/mpam_resctrl.c | 16 +++++++++++-----
3 files changed, 26 insertions(+), 7 deletions(-)
diff --git a/drivers/platform/arm64/mpam/mpam_devices.c b/drivers/platform/arm64/mpam/mpam_devices.c
index 6b326506c359..4b43dfef1cdc 100644
--- a/drivers/platform/arm64/mpam/mpam_devices.c
+++ b/drivers/platform/arm64/mpam/mpam_devices.c
@@ -1531,6 +1531,7 @@ static void mpam_quirk_post_config_change(struct mpam_msc_ris *ris, u16 partid,
static void mpam_reprogram_ris_partid(struct mpam_msc_ris *ris, u16 partid,
struct mpam_config *cfg)
{
+ u16 intpartid = 0;
u32 pri_val = 0;
u16 cmax = MPAMCFG_CMAX_CMAX;
u16 bwa_fract = MPAMCFG_MBW_MAX_MAX;
@@ -1543,9 +1544,13 @@ static void mpam_reprogram_ris_partid(struct mpam_msc_ris *ris, u16 partid,
__mpam_part_sel(ris->ris_idx, partid, msc);
if (mpam_has_feature(mpam_feat_partid_nrw, rprops)) {
+ if (mpam_has_feature(mpam_feat_partid_nrw, cfg))
+ intpartid = cfg->cpartid;
+
mpam_write_partsel_reg(msc, INTPARTID,
- MPAMCFG_INTPARTID_INTERNAL | partid);
- __mpam_intpart_sel(ris->ris_idx, partid, msc);
+ MPAMCFG_INTPARTID_INTERNAL | intpartid);
+
+ __mpam_intpart_sel(ris->ris_idx, intpartid, msc);
}
if (mpam_has_feature(mpam_feat_cpor_part, rprops)) {
@@ -3084,6 +3089,7 @@ static mpam_features_t mpam_update_config(struct mpam_config *cfg,
{
mpam_features_t changes = 0;
+ maybe_update_config(cfg, mpam_feat_partid_nrw, newcfg, cpartid, changes);
maybe_update_config(cfg, mpam_feat_cpor_part, newcfg, cpbm, changes);
maybe_update_config(cfg, mpam_feat_mbw_part, newcfg, mbw_pbm, changes);
maybe_update_config(cfg, mpam_feat_mbw_max, newcfg, mbw_max, changes);
diff --git a/drivers/platform/arm64/mpam/mpam_internal.h b/drivers/platform/arm64/mpam/mpam_internal.h
index 5af6ed60272e..965f4fea3012 100644
--- a/drivers/platform/arm64/mpam/mpam_internal.h
+++ b/drivers/platform/arm64/mpam/mpam_internal.h
@@ -300,6 +300,13 @@ struct mpam_config {
u16 mbw_max;
u16 mbw_min;
+ /*
+ * Control partition; resource regulation context for stateful controls
+ * Valid if features has mpam_feat_partid_nrw.
+ * For MSCs that implement PARTID Narrowing, this is intPARTID.
+ */
+ u16 cpartid;
+
struct mpam_garbage garbage;
};
diff --git a/drivers/platform/arm64/mpam/mpam_resctrl.c b/drivers/platform/arm64/mpam/mpam_resctrl.c
index eb1ef5d2bc57..f1d0d9f59771 100644
--- a/drivers/platform/arm64/mpam/mpam_resctrl.c
+++ b/drivers/platform/arm64/mpam/mpam_resctrl.c
@@ -205,7 +205,8 @@ u32 resctrl_arch_get_num_closid(struct rdt_resource *ignored)
static void mpam_resctrl_partid_range(u32 closid, enum resctrl_conf_type type,
const struct rdt_resource *r,
- u16 *min_partid, u16 *max_partid)
+ u16 *min_partid, u16 *max_partid,
+ u16 *cpartid)
{
u16 base_partid = closid;
u16 span = 1;
@@ -222,6 +223,9 @@ static void mpam_resctrl_partid_range(u32 closid, enum resctrl_conf_type type,
*min_partid = base_partid * partid_per_closid;
if (max_partid)
*max_partid = *min_partid + (span * partid_per_closid - 1);
+
+ if (cpartid)
+ *cpartid = base_partid;
}
static void mpam_resctrl_hwid(u32 closid, u32 rmid,
@@ -318,7 +322,7 @@ bool resctrl_arch_match_closid(struct task_struct *tsk, u32 closid)
u32 tsk_closid = FIELD_GET(MPAM1_EL1_PARTID_D, regval);
mpam_resctrl_partid_range(closid, CDP_NONE, NULL,
- &min_partid, &max_partid);
+ &min_partid, &max_partid, NULL);
return tsk_closid >= min_partid && tsk_closid <= max_partid;
}
@@ -1147,7 +1151,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_ctrl_domain *d,
dom = container_of(d, struct mpam_resctrl_dom, resctrl_ctrl_dom);
cprops = &res->class->props;
- mpam_resctrl_partid_range(closid, type, r, &partid, NULL);
+ mpam_resctrl_partid_range(closid, type, r, &partid, NULL, NULL);
cfg = &dom->comp->cfg[partid];
switch (r->rid) {
@@ -1205,13 +1209,15 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_ctrl_domain *d,
dom = container_of(d, struct mpam_resctrl_dom, resctrl_ctrl_dom);
cprops = &res->class->props;
- mpam_resctrl_partid_range(closid, t, r, &min_partid, &max_partid);
+ cfg.features = 0;
+ mpam_set_feature(mpam_feat_partid_nrw, &cfg);
+ mpam_resctrl_partid_range(closid, t, r, &min_partid, &max_partid,
+ &cfg.cpartid);
if (!r->alloc_capable ||
min_partid > mpam_partid_max ||
max_partid > mpam_partid_max)
return -EINVAL;
- cfg.features = 0;
switch (r->rid) {
case RDT_RESOURCE_L2:
case RDT_RESOURCE_L3:
--
2.34.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* [RFC PATCH v2 10/11] arm_mpam: Skip inapplicable cases when reprogramming a resctrl domain
2025-01-17 15:10 [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
` (8 preceding siblings ...)
2025-01-17 15:10 ` [RFC PATCH v2 09/11] arm_mpam: Add control partition identifier to mpam_config Dave Martin
@ 2025-01-17 15:10 ` Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 11/11] arm_mpam: Detect whether the system can alias PARTIDs Dave Martin
` (12 subsequent siblings)
22 siblings, 0 replies; 26+ messages in thread
From: Dave Martin @ 2025-01-17 15:10 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-arm-kernel, Zeng Heng, Shaopeng Tan, James Morse
The implementation of resctrl_arch_update_domains() currently
iterates over all types of staged configuration changes passed from
resctrl, irrespective of whether a given type (CDP_CODE, CDP_DATA
or CDP_NONE) makes sense for the affected MPAM resource. Since
resctrl will not stage any updates of inappropriate type for the
resource, this is mostly harmless.
When doing a forced update as a result of a call via
resctrl_arch_init_domains() however, there is a need to update the
cpartid (intPARTID) even if no staged configuration changes are
supplied. This can result in intPARTID being repeatedly
reprogrammed with different values. As well as being inefficient,
the final value may not be the correct one.
To program just once with the correct intPARTID, only apply the
update types that actually make sense for the resource.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
drivers/platform/arm64/mpam/mpam_resctrl.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/drivers/platform/arm64/mpam/mpam_resctrl.c b/drivers/platform/arm64/mpam/mpam_resctrl.c
index f1d0d9f59771..1f31b1455f66 100644
--- a/drivers/platform/arm64/mpam/mpam_resctrl.c
+++ b/drivers/platform/arm64/mpam/mpam_resctrl.c
@@ -1263,6 +1263,10 @@ static int __resctrl_arch_update_domains(struct rdt_resource *r, u32 closid,
list_for_each_entry(d, &r->ctrl_domains, hdr.list) {
for (t = 0; t < CDP_NUM_TYPES; t++) {
+ if ((t == CDP_NONE) !=
+ (!cdp_enabled || mpam_resctrl_hide_cdp(r->rid)))
+ continue;
+
cfg = &d->staged_config[t];
if (!force && !cfg->have_new_ctrl)
continue;
--
2.34.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* [RFC PATCH v2 11/11] arm_mpam: Detect whether the system can alias PARTIDs
2025-01-17 15:10 [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
` (9 preceding siblings ...)
2025-01-17 15:10 ` [RFC PATCH v2 10/11] arm_mpam: Skip inapplicable cases when reprogramming a resctrl domain Dave Martin
@ 2025-01-17 15:10 ` Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
` (11 subsequent siblings)
22 siblings, 0 replies; 26+ messages in thread
From: Dave Martin @ 2025-01-17 15:10 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-arm-kernel, Zeng Heng, Shaopeng Tan, James Morse
In order to be able to assign multiple PARTIDs freely to resource
control groups, it is necessary that all MSCs that are used by resctrl
either implement PARTID Narrowing (so that PARTIDs can be remapped
explicitly) or otherwise only have controls whose resource regulation
is stateless and so whose behaviour would not be affected by splitting
a control group across multiple PARTIDs.
Detect this property at the RIS level and merge it up the MPAM devices
tree.
Export the common global result as mpam_partid_aliasing, and apply
it to the PARTIDs-per-CLOSID configuration.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
Currently, this the logic is simple and may forbid PARTID aliasing
on some systems that can support it as a result of the precise set
of MSCs and controls that are exposed through resctrl.
The aim is to make this logic more flexible later.
---
drivers/platform/arm64/mpam/mpam_devices.c | 48 ++++++++++++++++++---
drivers/platform/arm64/mpam/mpam_internal.h | 2 +
drivers/platform/arm64/mpam/mpam_resctrl.c | 12 +++++-
3 files changed, 53 insertions(+), 9 deletions(-)
diff --git a/drivers/platform/arm64/mpam/mpam_devices.c b/drivers/platform/arm64/mpam/mpam_devices.c
index 4b43dfef1cdc..0182c922638f 100644
--- a/drivers/platform/arm64/mpam/mpam_devices.c
+++ b/drivers/platform/arm64/mpam/mpam_devices.c
@@ -68,6 +68,7 @@ static DEFINE_MUTEX(mpam_cpuhp_state_lock);
*/
u16 mpam_partid_max;
u8 mpam_pmg_max;
+bool mpam_partid_aliasing; /* PARTID aliasing supported */
static bool partid_max_init, partid_max_published;
static DEFINE_SPINLOCK(partid_max_lock);
@@ -828,6 +829,7 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)
struct mpam_msc *msc = ris->vmsc->msc;
struct mpam_props *props = &ris->props;
struct mpam_class *class = ris->vmsc->comp->class;
+ bool partid_can_alias = true;
lockdep_assert_held(&msc->probe_lock);
lockdep_assert_held(&msc->part_sel_lock);
@@ -836,6 +838,8 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)
/* Cache Capacity Partitioning */
if (FIELD_GET(MPAMF_IDR_HAS_CCAP_PART, ris->idr)) {
+ partid_can_alias = false;
+
ris->ccap_idr = mpam_read_partsel_reg(msc, CCAP_IDR);
props->cmax_wd = FIELD_GET(MPAMF_CCAP_IDR_CMAX_WD, ris->ccap_idr);
@@ -884,14 +888,20 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)
*/
props->bwa_wd = min(props->bwa_wd, 16);
- if (props->bwa_wd && FIELD_GET(MPAMF_MBW_IDR_HAS_MAX, mbw_features))
+ if (props->bwa_wd && FIELD_GET(MPAMF_MBW_IDR_HAS_MAX, mbw_features)) {
mpam_set_feature(mpam_feat_mbw_max, props);
+ partid_can_alias = false;
+ }
- if (props->bwa_wd && FIELD_GET(MPAMF_MBW_IDR_HAS_MIN, mbw_features))
+ if (props->bwa_wd && FIELD_GET(MPAMF_MBW_IDR_HAS_MIN, mbw_features)) {
mpam_set_feature(mpam_feat_mbw_min, props);
+ partid_can_alias = false;
+ }
- if (props->bwa_wd && FIELD_GET(MPAMF_MBW_IDR_HAS_PROP, mbw_features))
+ if (props->bwa_wd && FIELD_GET(MPAMF_MBW_IDR_HAS_PROP, mbw_features)) {
mpam_set_feature(mpam_feat_mbw_prop, props);
+ partid_can_alias = false;
+ }
}
/* Priority partitioning */
@@ -988,17 +998,26 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)
* RIS with PARTID narrowing don't have enough storage for one
* configuration per PARTID. If these are in a class we could use,
* reduce the supported partid_max to match the numer of intpartid.
- * If the class is unknown, just ignore it.
+ * If the MSC won't be used for anything or implements PARTID
+ * Narrowing then PARTIDs can be assigned freely, so PARTID
+ * aliasing can be supported regardless of the resource
+ * controls present.
*/
- if (FIELD_GET(MPAMF_IDR_HAS_PARTID_NRW, ris->idr) &&
- class->type != MPAM_CLASS_UNKNOWN) {
+ if (class->type == MPAM_CLASS_UNKNOWN)
+ partid_can_alias = true;
+ else if (FIELD_GET(MPAMF_IDR_HAS_PARTID_NRW, ris->idr)) {
u32 nrwidr = mpam_read_partsel_reg(msc, PARTID_NRW_IDR);
u16 partid_max = FIELD_GET(MPAMF_PARTID_NRW_IDR_INTPARTID_MAX, nrwidr);
mpam_set_feature(mpam_feat_partid_nrw, props);
msc->partid_max = min(msc->partid_max, partid_max);
+
+ partid_can_alias = true;
}
+ if (partid_can_alias)
+ mpam_set_feature(mpam_feat_partid_aliasing, props);
+
mpam_mon_sel_outer_unlock(msc);
}
@@ -2368,6 +2387,10 @@ static void __props_mismatch(struct mpam_props *parent,
mpam_clear_feature(mpam_feat_dspri_part_0_low, &parent->features);
}
+ /* PARTID aliasing must be supported everywhere, or not at all: */
+ if (!mpam_has_feature(mpam_feat_partid_aliasing, parent))
+ mpam_clear_feature(mpam_feat_partid_aliasing, &parent->features);
+
if (alias) {
/* Merge features for aliased resources */
parent->features |= child->features;
@@ -2486,6 +2509,7 @@ static void mpam_enable_merge_features(struct list_head *all_classes_list)
{
struct mpam_class *class;
struct mpam_component *comp;
+ bool partid_can_alias = true, comp_partid_aliasing;
lockdep_assert_held(&mpam_list_lock);
@@ -2495,9 +2519,19 @@ static void mpam_enable_merge_features(struct list_head *all_classes_list)
mpam_enable_init_class_features(class);
- list_for_each_entry(comp, &class->components, class_list)
+ list_for_each_entry(comp, &class->components, class_list) {
mpam_enable_merge_class_features(comp);
+
+ comp_partid_aliasing = mpam_has_feature(
+ mpam_feat_partid_aliasing, &class->props);
+ partid_can_alias = partid_can_alias &&
+ comp_partid_aliasing;
+ }
}
+
+ spin_lock(&partid_max_lock);
+ mpam_partid_aliasing = partid_can_alias;
+ spin_unlock(&partid_max_lock);
}
static char *mpam_errcode_names[16] = {
diff --git a/drivers/platform/arm64/mpam/mpam_internal.h b/drivers/platform/arm64/mpam/mpam_internal.h
index 965f4fea3012..6e10e4789506 100644
--- a/drivers/platform/arm64/mpam/mpam_internal.h
+++ b/drivers/platform/arm64/mpam/mpam_internal.h
@@ -215,6 +215,7 @@ enum mpam_device_features {
mpam_feat_msmon_mbwu_hw_nrdy,
mpam_feat_msmon_capt,
mpam_feat_partid_nrw,
+ mpam_feat_partid_aliasing, /* derived feature */
MPAM_FEATURE_LAST,
};
#define MPAM_ALL_FEATURES ((1<<MPAM_FEATURE_LAST) - 1)
@@ -474,6 +475,7 @@ static inline void mpam_assert_srcu_read_lock_held(void)
/* System wide partid/pmg values */
extern u16 mpam_partid_max;
extern u8 mpam_pmg_max;
+extern bool mpam_partid_aliasing;
/* Scheduled work callback to enable mpam once all MSC have been probed */
void mpam_enable(struct work_struct *work);
diff --git a/drivers/platform/arm64/mpam/mpam_resctrl.c b/drivers/platform/arm64/mpam/mpam_resctrl.c
index 1f31b1455f66..5b296e3e27c2 100644
--- a/drivers/platform/arm64/mpam/mpam_resctrl.c
+++ b/drivers/platform/arm64/mpam/mpam_resctrl.c
@@ -160,7 +160,8 @@ static bool mpam_resctrl_hide_cdp(enum resctrl_res_level rid)
return cdp_enabled && !resctrl_arch_get_cdp_enabled(rid);
}
-static unsigned int partid_per_closid = 1;
+static unsigned int bootparam_partid_per_closid = 1;
+static unsigned int partid_per_closid;
static int mpam_resctrl_partid_per_closid_set(const char *val,
const struct kernel_param *kp)
@@ -178,7 +179,7 @@ static const struct kernel_param_ops mpam_resctrl_partid_per_closid_ops = {
};
device_param_cb(partid_per_closid, &mpam_resctrl_partid_per_closid_ops,
- &partid_per_closid, 0444);
+ &bootparam_partid_per_closid, 0444);
static unsigned int mpam_num_pmg(void)
{
@@ -1113,6 +1114,13 @@ int mpam_resctrl_setup(void)
if (!err && !exposed_alloc_capable && !exposed_mon_capable)
err = -EOPNOTSUPP;
+ partid_per_closid = bootparam_partid_per_closid;
+ if ((!mpam_partid_aliasing && bootparam_partid_per_closid > 1) ||
+ resctrl_arch_get_num_closid(NULL) < 1) {
+ pr_warn("Hardware incompatible with PARTID aliasing, limiting monitoring groups\n");
+ partid_per_closid = 1;
+ }
+
if (!err) {
err = resctrl_init();
if (!err)
--
2.34.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* [RFC PATCH v2 00/11] Introduce PARTID grouping support
2025-01-17 15:10 [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
` (10 preceding siblings ...)
2025-01-17 15:10 ` [RFC PATCH v2 11/11] arm_mpam: Detect whether the system can alias PARTIDs Dave Martin
@ 2025-01-17 15:10 ` Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 01/11] arm_mpam: Fix read-back of cloned resource controls under CDP emulation Dave Martin
` (10 subsequent siblings)
22 siblings, 0 replies; 26+ messages in thread
From: Dave Martin @ 2025-01-17 15:10 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-arm-kernel, Zeng Heng, Shaopeng Tan, James Morse
(Was: Introduce flexible CLOSID/RMID translation) [5]
This series introduces an ID remapping scheme for mapping multiple MPAM
PARTIDs to each resctrl control group, and programs the intPARTID
appropriately on MSCs that require and support PARTID Narrowing.
Rudimentary support is included for detecting whether this
configuration will work correctly on the detected hardware, but this
detection is currently inflexible and may disable this multiple PARTID
mapping behaviour unnecessarily on some platforms.
(Feedback would be appreciated from anyone who knows of hardware where
this approach is insufficient.)
On supported platforms, this allows the number of monitoring groups per
resctrl control group to be scaled up by the value of the (provisional)
mpam.partid_per_closid kernel command-line parameter, at the expense of
reducing the number of control groups that can be created.
See "Background" below, and the subsequent sections, for more details.
This series is not a complete implementation, but should be sufficient
to provide additional resctrl monitoring groups beyond the number of
PMGs supported by the hardware (with some limitations; see
"Limitations", below).
Based on:
git://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/snapshot/v6.12-rc1
4d17b37b670795954fd7a70c8ec83fb705a5e2ad ("arm64: mpam: Restore the expected MPAM sysregs on cpuhp")
Changes Since RFCv1: [5]
---
DROPPED:
* arm_mpam: Clean up config update checks in mpam_apply_config()
(present in the base branch)
CHANGED:
* arm_mpam: Introduce flexible CLOSID/RMID translation
* [bug fix] Add CDP index properly in mpam_resctrl_partid_range().
(Also reported independently by Shaopeng Tan.)
* [relaxation] Remove requirement that mpam_pmg_max + 1 is a power
of two.
(This constraint is redundant, due to the use of multiply/add
transformations of MPAM IDs in place of shift/OR.)
NEW:
* arm_mpam: Set INTERNAL as needed when setting MSC controls
(resurrected from my local MPAM fixes branch; previously posted,
but was not included in RFC v1 of this series)
* fs/resctrl: Add arch hook to set up the default CLOSID
* arm_mpam: resctrl: Implement arch hook to set up the default CLOSID
Add and wire up a hook to update reprogram intPARTID everywhere
when the resctrl<->MPAM ID mappings change. This is needed for
clean initialisation of the hardware, since the MPAM architecture
doesn't specify the default mappings -- purging everything to a
consistent state avoids the possibility of spurious aliasing.
* arm_mpam: Add control partition identifier to mpam_config
Add a mpam_config field to tell mpam_devices.c what to program
for intPARTID. (The idea is to keep the mapping logic in
mpam_resctrl.c so that mpam_devices.c does not need to understand
how this value is determined.)
* arm_mpam: Skip inapplicable cases when reprogramming a resctrl domain
To simplify programming of intPARTIDs, never try to program a
PARTID with controls for the wrong resctrl_conf_type. (If
resctrl passed any control changes for the wrong
resctrl_conf_type, that's a bug anyway...)
* arm_mpam: Detect whether the system can alias PARTIDs
Initial attempt to detect whether it is safe a map multiple
PARTIDs to the same resctrl control group: if not, the
user-specified mpam.partid_per_closid parameter is ignored, and
the driver reverts back to the default behaviour of mapping a
single PARTID to each CLOSID.
This is currently simplistic, and only considers all MPAM MSCs
and controls in the system (irrespective if they are going to be
exposed through resctrl or not).
Background
==========
An MPAM Memory System Component (MSC) that does not implement the
PARTID Narrowing feature (Narrowing) provides independently
programmable resource controls for each PARTID up to the maximum PARTID
that the MSC supports.
An MSC that _does_ implement Narrowing performs a remapping step to
convert the PARTID from the incoming memory transaction ("request
PARTID", or "reqPARTID") to an internal ID ("internal PARTID", or
"intPARTID") before looking up and applying resource controls. Such an
MSC does not necessarily need to provide the same number of
independently programmable resource control partitions (indexed by the
intPARTID) as that which the size of the supported PARTID ("reqPARTID")
space would otherwise require. Instead, independently programmable
resource controls are provided for each intPARTID, accompanied by a
software programmable mapping that allows each reqPARTID to be mapped
independently to a selected intPARTID.
Typically, the number of intPARTIDs supported by a Narrowing MSC is
smaller than the number of reqPARTIDs. Narrowing permits multiple
reqPARTIDs to be mapped to each intPARTID, so allowing the whole MPAM
PARTID space to be served.
A non-Narrowing MSC can be understood as implementing special case of
Narrowing, where the reqPARTID-to-intPARTID mapping is the identity
function and is not programmable.
The number of independently configurable control partitions at a given
MSC cannot exceed the number of intPARTIDs available, so the number of
resctrl CLOSIDs (which need to be independently controllable) must
typically be limited not to exceed the maximum number of intPARTIDs
available on any Narrowing MSC.
However, the presence of a programmable reqPARTID-to-intPARTID mapping
in each MSC creates an opportunity: because a group of multiple MPAM
PARTIDs can be mapped onto the same controls, they behave as a single
resource control partition. Because the PARTID is used as a match
criterion for monitoring purposes along with the MPAM Performance
Monitoring Group (PMG) identifier assigned each memory system
transaction, the specific PARTID in the group can be used to
distinguish transactions for monitoring purposes, even within the same
resctrl control group and when the PMGs of the transactions are the
same.
This means that it becomes possible to provide a larger number of
resctrl monitoring groups than the number of PMGs supported by the MPAM
hardware implementation.
In order to take advantage of this, a more flexible mapping scheme is
required for mapping resctrl CLOSIDs and RMIDs to MPAM PARTIDs and PMGs
than the current MPAM driver provides.
Similar approaches have been suggested by other people
(see Acknowledgements).
However, since I was already working on this, I'm posting my approach
for comparison.
This Series
===========
This series introduces a remapping scheme as follows:
1) The resctrl partition identifiers are combined into a single
identifier (formally the Cartesian product of the CLOSID, Code/Data
partition where applicable, and RMID ID spaces).
2) The resulting identifier is then broken down into the MPAM PARTID
and PMG, in such a way that each possible <CLOSID, CDP index, RMID>
maps to a unique <PARTID, PMG> pair and vice versa, and so that any two
<PARTID, PMG> that differ only in the PMG value, map onto the same
<CLOSID, CDP index> pair.
This transformation could be done in intuitive way using bitwise
concatenation, shift and masking operations, but since the size of the
MPAM hardware ID spaces may limited, I have tried to generalise this
approach using multiplication in place of bit-shifts, to permit ID
spaces whose size is not a power of two.
Limitations
===========
This series aims to contain all translations within the mpam_resctrl.c
code. This seems the most sensible place for it, since this is a
conversion scheme that will be used to interface the MPAM hardware with
the resctrl core code (but has nothing to do with the "pure" MPAM
architecture or the resctrl core code itself).
To simplify the interface between mpam_resctrl.c and mpam_devices.c,
the ID conversion scheme is arranged so that each resctrl CLOSID maps
to a contiguous sequence of MPAM PARTIDs. This means that programming
controls for a CLOSID can be achieved by iterating over the appropriate
range of PARTIDs.
This series provides a basic configuration interface that may be too
inflexible for real hardware.
Acknowledgements
================
Thanks to Shaopeng Tan for pointing out this opportunity and suggesting
a related idea, and to Zeng Heng who has also implemented something
similar, albeit with a slightly different mapping scheme [2], [3], [4];
this has continued to develop. [1]
References
==========
Zeng Heng <zengheng4@huawei.com>:
[1] (v4) arm_mpam: Introduce the Narrow-PARTID feature for MPAM driver
https://lore.kernel.org/linux-arm-kernel/20250104101224.873926-1-zengheng4@huawei.com/
[2] (v3) arm_mpam: Introduce the Narrow-PARTID feature for MPAM driver
https://lore.kernel.org/linux-arm-kernel/20241207092136.2488426-1-zengheng4@huawei.com/
[3] (v2) arm_mpam: Introduce the Narrow-PARTID feature for MPAM driver
https://lore.kernel.org/linux-arm-kernel/20241119135104.595630-1-zengheng4@huawei.com/
[4] (v1) arm_mpam: Introduce the definitions of intPARTID and reqPARTID
https://lore.kernel.org/linux-arm-kernel/20241114135037.918470-1-zengheng4@huawei.com/
Dave Martin <Dave.Martin@arm.com>:
[5] (previous post) [RFC PATCH 0/6] Introduce flexible CLOSID/RMID translation
https://lore.kernel.org/lkml/20241212154000.330467-1-Dave.Martin@arm.com/
Dave Martin (11):
arm_mpam: Fix read-back of cloned resource controls under CDP
emulation
arm_mpam: Delete unused function resctrl_arch_set_rmid()
arm_mpam: Set INTERNAL as needed when setting MSC controls
fs/resctrl: Add arch hook to set up the default CLOSID
arm_mpam: resctrl: Implement arch hook to set up the default CLOSID
arm_mpam: Introduce flexible CLOSID/RMID translation
arm_mpam: [NFU] Rework ID remapping to use a kernel command-line
argument
arm_mpam: [NFU] Development diagnostics for MPAM ID assignments
arm_mpam: Add control partition identifier to mpam_config
arm_mpam: Skip inapplicable cases when reprogramming a resctrl domain
arm_mpam: Detect whether the system can alias PARTIDs
arch/arm64/include/asm/mpam.h | 21 +-
arch/x86/include/asm/resctrl.h | 12 +
drivers/platform/arm64/mpam/mpam_devices.c | 88 +++++--
drivers/platform/arm64/mpam/mpam_internal.h | 9 +
drivers/platform/arm64/mpam/mpam_resctrl.c | 260 +++++++++++++-------
fs/resctrl/rdtgroup.c | 26 +-
include/linux/arm_mpam.h | 5 +
include/linux/resctrl.h | 2 +
8 files changed, 299 insertions(+), 124 deletions(-)
base-commit: 4d17b37b670795954fd7a70c8ec83fb705a5e2ad
--
2.34.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* [RFC PATCH v2 01/11] arm_mpam: Fix read-back of cloned resource controls under CDP emulation
2025-01-17 15:10 [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
` (11 preceding siblings ...)
2025-01-17 15:10 ` [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
@ 2025-01-17 15:10 ` Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 02/11] arm_mpam: Delete unused function resctrl_arch_set_rmid() Dave Martin
` (9 subsequent siblings)
22 siblings, 0 replies; 26+ messages in thread
From: Dave Martin @ 2025-01-17 15:10 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-arm-kernel, Zeng Heng, Shaopeng Tan, James Morse
When reading resource control values through a resctrl schemata
file, the index into the configuration array in the MPAM driver
is currently translated incorrectly.
This means that when resctrl is mounted with the "cdp" option, the
wrong array entry may be returned or an out-of-bounds array access
may occur when reading back of control values for the MB resource.
Fix it by translating the index for affected resources as if they
were "DATA" resources. ("CODE" would also work, but because
resctrl_arch_update_one() clones schemata writes across "CODE" and
"DATA" for affected resources, it does not matter which one is
read.)
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
This issue was introduced by the non-upstream commit
"arm_mpam: resctrl: Improve CDP emulation"
(commit b097a6193546dbbd27cd19d7e16e1e750ebb030f in
git://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/snapshot/v6.12-rc1 )
---
drivers/platform/arm64/mpam/mpam_resctrl.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/drivers/platform/arm64/mpam/mpam_resctrl.c b/drivers/platform/arm64/mpam/mpam_resctrl.c
index 76ddbce0ea9c..30f2caec11d7 100644
--- a/drivers/platform/arm64/mpam/mpam_resctrl.c
+++ b/drivers/platform/arm64/mpam/mpam_resctrl.c
@@ -1097,6 +1097,14 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_ctrl_domain *d,
dom = container_of(d, struct mpam_resctrl_dom, resctrl_ctrl_dom);
cprops = &res->class->props;
+ /*
+ * When CDP is enabled, but the resource doesn't support it,
+ * the control is cloned across both partids.
+ * Pick one at random to read:
+ */
+ if (mpam_resctrl_hide_cdp(r->rid))
+ type = CDP_DATA;
+
partid = resctrl_get_config_index(closid, type);
cfg = &dom->comp->cfg[partid];
--
2.34.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* [RFC PATCH v2 02/11] arm_mpam: Delete unused function resctrl_arch_set_rmid()
2025-01-17 15:10 [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
` (12 preceding siblings ...)
2025-01-17 15:10 ` [RFC PATCH v2 01/11] arm_mpam: Fix read-back of cloned resource controls under CDP emulation Dave Martin
@ 2025-01-17 15:10 ` Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 03/11] arm_mpam: Set INTERNAL as needed when setting MSC controls Dave Martin
` (8 subsequent siblings)
22 siblings, 0 replies; 26+ messages in thread
From: Dave Martin @ 2025-01-17 15:10 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-arm-kernel, Zeng Heng, Shaopeng Tan, James Morse
The function resctrl_arch_set_rmid() appears to have been left
behind after earlier refactoring and is now unused.
To avoid needing to answer awkward questions about how this
function should apply partition ID namespace remappings, just get
rid of it.
No functional change.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
I'm assuming here that there are no pending branches that implement /
use this function.
When moving tasks between monitoring groups, resctrlfs currently
extracts the CLOSID from struct rdtgroup and passes it explicitly
to resctrl_arch_set_closid_rmid(), so there seems to be no need for
a separate helper that doesn't overwrite the arch's control
partition identifier.
---
arch/arm64/include/asm/mpam.h | 14 --------------
1 file changed, 14 deletions(-)
diff --git a/arch/arm64/include/asm/mpam.h b/arch/arm64/include/asm/mpam.h
index e5f385767174..df725d2d9d05 100644
--- a/arch/arm64/include/asm/mpam.h
+++ b/arch/arm64/include/asm/mpam.h
@@ -93,20 +93,6 @@ static inline u64 mpam_get_regval(struct task_struct *tsk)
#endif
}
-static inline void resctrl_arch_set_rmid(struct task_struct *tsk, u32 rmid)
-{
-#ifdef CONFIG_ARM64_MPAM
- u64 regval = mpam_get_regval(tsk);
-
- regval &= ~MPAM1_EL1_PMG_D;
- regval &= ~MPAM1_EL1_PMG_I;
- regval |= FIELD_PREP(MPAM1_EL1_PMG_D, rmid);
- regval |= FIELD_PREP(MPAM1_EL1_PMG_I, rmid);
-
- WRITE_ONCE(task_thread_info(tsk)->mpam_partid_pmg, regval);
-#endif
-}
-
static inline void mpam_thread_switch(struct task_struct *tsk)
{
u64 oldregval;
--
2.34.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* [RFC PATCH v2 03/11] arm_mpam: Set INTERNAL as needed when setting MSC controls
2025-01-17 15:10 [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
` (13 preceding siblings ...)
2025-01-17 15:10 ` [RFC PATCH v2 02/11] arm_mpam: Delete unused function resctrl_arch_set_rmid() Dave Martin
@ 2025-01-17 15:10 ` Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 04/11] fs/resctrl: Add arch hook to set up the default CLOSID Dave Martin
` (7 subsequent siblings)
22 siblings, 0 replies; 26+ messages in thread
From: Dave Martin @ 2025-01-17 15:10 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-arm-kernel, Zeng Heng, Shaopeng Tan, James Morse
Currently, when an MSC implements PARTID narrowing, MPAMCFG_PART_SEL is
left set to the reqPARTID while programming resource controls in
an MSC.
The MPAM architecture does not guarantee that any particular resource
controls will be updated correctly in this scenario. Instead,
MPAMCFG_PART_SEL must be written with the corresponding intPARTID and
with the INTERNAL bit set before attempting to program resource controls.
Only the PARTID->intPARTID mappings can be written without the INTERNAL
bit set in MPAMCFG_PART_SEL.
Fix it, by rewriting MPAMCFG_PART_SEL appropriately after setting the
intPARTID mapping.
The MPAMCFG_INTPARTID_INTERNAL flag is not currently accepted as
input to the __mpam_part_sel() helper. In the interest of keeping
the code clean, break this helper up.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Signed-off-by: James Morse <james.morse@arm.com>
---
drivers/platform/arm64/mpam/mpam_devices.c | 26 +++++++++++++++++-----
1 file changed, 20 insertions(+), 6 deletions(-)
diff --git a/drivers/platform/arm64/mpam/mpam_devices.c b/drivers/platform/arm64/mpam/mpam_devices.c
index 9463045c0011..f965e6639e0d 100644
--- a/drivers/platform/arm64/mpam/mpam_devices.c
+++ b/drivers/platform/arm64/mpam/mpam_devices.c
@@ -204,15 +204,27 @@ static u64 mpam_msc_read_esr(struct mpam_msc *msc)
return (esr_high << 32) | esr_low;
}
+static void __mpam_part_sel_raw(u32 partsel, struct mpam_msc *msc)
+{
+ lockdep_assert_held(&msc->part_sel_lock);
+ mpam_write_partsel_reg(msc, PART_SEL, partsel);
+}
+
static void __mpam_part_sel(u8 ris_idx, u16 partid, struct mpam_msc *msc)
{
- u32 partsel;
+ u32 partsel = FIELD_PREP(MPAMCFG_PART_SEL_RIS, ris_idx) |
+ FIELD_PREP(MPAMCFG_PART_SEL_PARTID_SEL, partid);
- lockdep_assert_held(&msc->part_sel_lock);
+ __mpam_part_sel_raw(partsel, msc);
+}
- partsel = FIELD_PREP(MPAMCFG_PART_SEL_RIS, ris_idx) |
- FIELD_PREP(MPAMCFG_PART_SEL_PARTID_SEL, partid);
- mpam_write_partsel_reg(msc, PART_SEL, partsel);
+static void __mpam_intpart_sel(u8 ris_idx, u16 intpartid, struct mpam_msc *msc)
+{
+ u32 partsel = FIELD_PREP(MPAMCFG_PART_SEL_RIS, ris_idx) |
+ FIELD_PREP(MPAMCFG_PART_SEL_PARTID_SEL, intpartid) |
+ MPAMCFG_PART_SEL_INTERNAL;
+
+ __mpam_part_sel_raw(partsel, msc);
}
int mpam_register_requestor(u16 partid_max, u8 pmg_max)
@@ -1530,9 +1542,11 @@ static void mpam_reprogram_ris_partid(struct mpam_msc_ris *ris, u16 partid,
mutex_lock(&msc->part_sel_lock);
__mpam_part_sel(ris->ris_idx, partid, msc);
- if (mpam_has_feature(mpam_feat_partid_nrw, rprops))
+ if (mpam_has_feature(mpam_feat_partid_nrw, rprops)) {
mpam_write_partsel_reg(msc, INTPARTID,
MPAMCFG_INTPARTID_INTERNAL | partid);
+ __mpam_intpart_sel(ris->ris_idx, partid, msc);
+ }
if (mpam_has_feature(mpam_feat_cpor_part, rprops)) {
if (mpam_has_feature(mpam_feat_cpor_part, cfg))
--
2.34.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* [RFC PATCH v2 04/11] fs/resctrl: Add arch hook to set up the default CLOSID
2025-01-17 15:10 [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
` (14 preceding siblings ...)
2025-01-17 15:10 ` [RFC PATCH v2 03/11] arm_mpam: Set INTERNAL as needed when setting MSC controls Dave Martin
@ 2025-01-17 15:10 ` Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 05/11] arm_mpam: resctrl: Implement " Dave Martin
` (6 subsequent siblings)
22 siblings, 0 replies; 26+ messages in thread
From: Dave Martin @ 2025-01-17 15:10 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-arm-kernel, Zeng Heng, Shaopeng Tan, James Morse
Currently, the resctrl core code assumes that CLOSID 0 requires no
special initialisation before use.
This is true on x86, but may not be true on other architectures
(including Arm MPAM, in certain configurations).
Since the allocation of CLOSIDs can depend on parameters such as
resctrl filesystem mount options that are not available at arch driver
init time, an arch hook is needed to set up the default CLOSID when
the resctrl filesystem is mounted.
Add a suitable arch hook resctrl_arch_init_domains().
Wire it up to arch stubs that do nothing, for now.
(This is also the right thing for x86.)
No functional change.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
arch/x86/include/asm/resctrl.h | 12 ++++++++++++
fs/resctrl/rdtgroup.c | 26 ++++++++++++++++++++++++--
include/linux/arm_mpam.h | 10 ++++++++++
include/linux/resctrl.h | 2 ++
4 files changed, 48 insertions(+), 2 deletions(-)
diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h
index 332b0617d728..b5fbb94e756a 100644
--- a/arch/x86/include/asm/resctrl.h
+++ b/arch/x86/include/asm/resctrl.h
@@ -50,6 +50,18 @@ DECLARE_STATIC_KEY_FALSE(rdt_enable_key);
DECLARE_STATIC_KEY_FALSE(rdt_alloc_enable_key);
DECLARE_STATIC_KEY_FALSE(rdt_mon_enable_key);
+/* The default CLOSID needs no special setup on x86: */
+static inline bool resctrl_arch_default_closid_needs_init(void)
+{
+ return false;
+}
+
+/* ... so this won't be needed either: */
+static inline int resctrl_arch_init_domains(struct rdt_resource *r, u32 closid)
+{
+ return 0;
+}
+
static inline bool resctrl_arch_alloc_capable(void)
{
return rdt_alloc_capable;
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 30f1dd05e10b..200f92e8f619 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -65,6 +65,8 @@ static struct seq_buf last_cmd_status;
static char last_cmd_status_buf[512];
+static int rdtgroup_init_first_alloc(struct rdtgroup *rdtgrp);
+
static int rdtgroup_setup_root(struct rdt_fs_context *ctx);
static void rdtgroup_destroy_root(void);
@@ -2542,6 +2544,9 @@ static int rdt_get_tree(struct fs_context *fc)
}
closid_init();
+ ret = rdtgroup_init_first_alloc(&rdtgroup_default);
+ if (ret)
+ goto out_schemata_free;
if (resctrl_arch_mon_capable())
flags |= RFTYPE_MON;
@@ -3230,7 +3235,7 @@ static void rdtgroup_init_mba(struct rdt_resource *r, u32 closid)
}
/* Initialize the RDT group's allocations. */
-static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp)
+static int __rdtgroup_init_alloc(struct rdtgroup *rdtgrp, bool first_time_init)
{
struct resctrl_schema *s;
struct rdt_resource *r;
@@ -3251,7 +3256,11 @@ static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp)
goto out;
}
- ret = resctrl_arch_update_domains(r, rdtgrp->closid);
+ if (first_time_init)
+ ret = resctrl_arch_init_domains(r, rdtgrp->closid);
+ else
+ ret = resctrl_arch_update_domains(r, rdtgrp->closid);
+
if (ret < 0) {
rdt_last_cmd_puts("Failed to initialize allocations\n");
goto out;
@@ -3265,6 +3274,19 @@ static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp)
return ret;
}
+static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp)
+{
+ return __rdtgroup_init_alloc(rdtgrp, false);
+}
+
+static int rdtgroup_init_first_alloc(struct rdtgroup *rdtgrp)
+{
+ if (!resctrl_arch_default_closid_needs_init())
+ return 0;
+
+ return __rdtgroup_init_alloc(rdtgrp, true);
+}
+
static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp)
{
int ret;
diff --git a/include/linux/arm_mpam.h b/include/linux/arm_mpam.h
index c00b5fc1d235..dd5f18474846 100644
--- a/include/linux/arm_mpam.h
+++ b/include/linux/arm_mpam.h
@@ -45,6 +45,16 @@ static inline unsigned int resctrl_arch_round_mon_val(unsigned int val)
return val;
}
+static inline bool resctrl_arch_default_closid_needs_init(void)
+{
+ return false;
+}
+
+static inline int resctrl_arch_init_domains(struct rdt_resource *r, u32 closid)
+{
+ return 0;
+}
+
bool resctrl_arch_alloc_capable(void);
bool resctrl_arch_mon_capable(void);
bool resctrl_arch_is_llc_occupancy_enabled(void);
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index be9d9f4c7b81..fc00b080f1a1 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -371,6 +371,8 @@ u32 resctrl_arch_system_num_rmid_idx(void);
struct rdt_domain_hdr *resctrl_arch_find_domain(struct list_head *domain_list,
int id);
+bool resctrl_arch_default_closid_needs_init(void);
+int resctrl_arch_init_domains(struct rdt_resource *r, u32 closid);
int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid);
bool resctrl_arch_is_evt_configurable(enum resctrl_event_id evt);
--
2.34.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* [RFC PATCH v2 05/11] arm_mpam: resctrl: Implement arch hook to set up the default CLOSID
2025-01-17 15:10 [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
` (15 preceding siblings ...)
2025-01-17 15:10 ` [RFC PATCH v2 04/11] fs/resctrl: Add arch hook to set up the default CLOSID Dave Martin
@ 2025-01-17 15:10 ` Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 06/11] arm_mpam: Introduce flexible CLOSID/RMID translation Dave Martin
` (5 subsequent siblings)
22 siblings, 0 replies; 26+ messages in thread
From: Dave Martin @ 2025-01-17 15:10 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-arm-kernel, Zeng Heng, Shaopeng Tan, James Morse
Since the allocation of CLOSIDs can now depend on parameters such
as resctrl filesystem mount options that are not available at arch
driver init time, all MPAM resource allocations need to be
reprogrammed when resctrl is mounted.
Implement resctrl_arch_init_domains() so that resctrl will now
reprogram the default CLOSID's resource allocations at filesystem
mount time, in the same way as when creating a new resctrl control
group.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
drivers/platform/arm64/mpam/mpam_resctrl.c | 15 +++++++++++++--
include/linux/arm_mpam.h | 7 +------
2 files changed, 14 insertions(+), 8 deletions(-)
diff --git a/drivers/platform/arm64/mpam/mpam_resctrl.c b/drivers/platform/arm64/mpam/mpam_resctrl.c
index 30f2caec11d7..aa8a6c077918 100644
--- a/drivers/platform/arm64/mpam/mpam_resctrl.c
+++ b/drivers/platform/arm64/mpam/mpam_resctrl.c
@@ -1209,7 +1209,8 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_ctrl_domain *d,
}
/* TODO: this is IPI heavy */
-int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)
+static int __resctrl_arch_update_domains(struct rdt_resource *r, u32 closid,
+ bool force)
{
int err = 0;
enum resctrl_conf_type t;
@@ -1222,7 +1223,7 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)
list_for_each_entry(d, &r->ctrl_domains, hdr.list) {
for (t = 0; t < CDP_NUM_TYPES; t++) {
cfg = &d->staged_config[t];
- if (!cfg->have_new_ctrl)
+ if (!force && !cfg->have_new_ctrl)
continue;
err = resctrl_arch_update_one(r, d, closid, t,
@@ -1235,6 +1236,16 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)
return err;
}
+int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)
+{
+ return __resctrl_arch_update_domains(r, closid, false);
+}
+
+int resctrl_arch_init_domains(struct rdt_resource *r, u32 closid)
+{
+ return __resctrl_arch_update_domains(r, closid, true);
+}
+
void resctrl_arch_reset_resources(void)
{
int i, idx;
diff --git a/include/linux/arm_mpam.h b/include/linux/arm_mpam.h
index dd5f18474846..e6b188ab735b 100644
--- a/include/linux/arm_mpam.h
+++ b/include/linux/arm_mpam.h
@@ -47,12 +47,7 @@ static inline unsigned int resctrl_arch_round_mon_val(unsigned int val)
static inline bool resctrl_arch_default_closid_needs_init(void)
{
- return false;
-}
-
-static inline int resctrl_arch_init_domains(struct rdt_resource *r, u32 closid)
-{
- return 0;
+ return true;
}
bool resctrl_arch_alloc_capable(void);
--
2.34.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* [RFC PATCH v2 06/11] arm_mpam: Introduce flexible CLOSID/RMID translation
2025-01-17 15:10 [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
` (16 preceding siblings ...)
2025-01-17 15:10 ` [RFC PATCH v2 05/11] arm_mpam: resctrl: Implement " Dave Martin
@ 2025-01-17 15:10 ` Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 07/11] arm_mpam: [NFU] Rework ID remapping to use a kernel command-line argument Dave Martin
` (4 subsequent siblings)
22 siblings, 0 replies; 26+ messages in thread
From: Dave Martin @ 2025-01-17 15:10 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-arm-kernel, Zeng Heng, Shaopeng Tan, James Morse
Currently, the MPAM driver uses the resctrl CLOSID directly as the
MPAM Partition Identifier (PARTID; possibly modified to give
different PARTIDs to code and data when Code/Data Partitioning /
CDP is enabled), and uses the resctrl RMID directly as the MPAM
Performance Monitoring Group identifier (PMG).
In preparation for using the MPAM PARTID Narrowing feature to allow
more resctrl monitoring groups to be provided than the number of
PMG values supported by the hardware, a more flexible remapping
scheme is needed.
Factor out ID translation operations in the MPAM resctrl glue code
into a couple of self-contained helpers, and call them as
appropriate.
The translation scheme has a single parameter, partid_per_closid,
which is currently hard-wired to 1.
As a result, this patch should have no effect on functionality.
Logic to determine / control the mapping parameters can be
introduced later.
The ID transformation may be visualised as follows:
without CDP:
+---------------------------------------------------+-----------+
| CLOSID | RMID |
+---------------------------------------------------+-------+---+
| PARTID : |PMG|
+-----------------------------------------------------------+---+
and with CDP, where the "CDP" field is 0 for the data sub-
partition and 1 for the code sub-partition:
+-----------------------------------------------+---+-----------+
| CLOSID |CDP| RMID |
+-----------------------------------------------+---+-------+---+
| PARTID : |PMG|
+-----------------------------------------------------------+---+
where each box represents a non-negative integer spanning some
fixed range starting from zero, and horizontal concatenation
denotes multiplying the value denoted by the left-hand box by the
span of the right-hand box, and adding the result to the value
denoted by the right-hand box; thus, most-significant fields are on
the left.
(Except for the span of each field not necessarily being a power of
two, this is conceptually identical to concatenation and cutting of
bitfields.)
The dotted segment indicates the least significant part of PARTID,
which spans partid_per_closid PARTIDs and discriminates monitoring
groups that use the same MPAM PMG value.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
Changes since RFCv1:
* [bugfix] Add CDP index properly in mpam_resctrl_partid_range().
(Also reported by Shaopeng Tan.)
* [relaxation] Remove requirement that mpam_pmg_max + 1 is a power
of two.
(This constraint is redundant, due to the use of multiply/add
transformations of MPAM IDs in place of shift/OR.)
---
drivers/platform/arm64/mpam/mpam_resctrl.c | 211 +++++++++++----------
1 file changed, 112 insertions(+), 99 deletions(-)
diff --git a/drivers/platform/arm64/mpam/mpam_resctrl.c b/drivers/platform/arm64/mpam/mpam_resctrl.c
index aa8a6c077918..f834753a3274 100644
--- a/drivers/platform/arm64/mpam/mpam_resctrl.c
+++ b/drivers/platform/arm64/mpam/mpam_resctrl.c
@@ -156,6 +156,18 @@ static bool mpam_resctrl_hide_cdp(enum resctrl_res_level rid)
return cdp_enabled && !resctrl_arch_get_cdp_enabled(rid);
}
+static unsigned int partid_per_closid = 1;
+
+static unsigned int mpam_num_pmg(void)
+{
+ return mpam_pmg_max + 1;
+}
+
+static unsigned int mpam_num_rmid(void)
+{
+ return mpam_num_pmg() * partid_per_closid;
+}
+
/*
* MSC may raise an error interrupt if it sees an out or range partid/pmg,
* and go on to truncate the value. Regardless of what the hardware supports,
@@ -163,35 +175,73 @@ static bool mpam_resctrl_hide_cdp(enum resctrl_res_level rid)
*/
u32 resctrl_arch_get_num_closid(struct rdt_resource *ignored)
{
- return mpam_partid_max + 1;
+ u32 res = (mpam_partid_max + 1) / partid_per_closid;
+
+ WARN_ON(res < 1);
+ return res;
}
-u32 resctrl_arch_system_num_rmid_idx(void)
+static void mpam_resctrl_partid_range(u32 closid, enum resctrl_conf_type type,
+ const struct rdt_resource *r,
+ u16 *min_partid, u16 *max_partid)
{
- u8 closid_shift = fls(mpam_pmg_max);
- u32 num_partid = resctrl_arch_get_num_closid(NULL);
+ u16 base_partid = closid;
+ u16 span = 1;
- return num_partid << closid_shift;
+ if (cdp_enabled) {
+ base_partid *= 2;
+ if (mpam_resctrl_hide_cdp(r->rid) ||
+ type == CDP_NONE)
+ span *= 2;
+ else if (type == CDP_CODE)
+ base_partid++;
+ }
+
+ *min_partid = base_partid * partid_per_closid;
+ if (max_partid)
+ *max_partid = *min_partid + (span * partid_per_closid - 1);
}
-u32 resctrl_arch_rmid_idx_encode(u32 closid, u32 rmid)
+static void mpam_resctrl_hwid(u32 closid, u32 rmid,
+ u16 *partid_d, u16 *partid_i, u8 *pmg)
{
- u8 closid_shift = fls(mpam_pmg_max);
+ const u16 pmg_hi = rmid / mpam_num_pmg();
+ const u16 pmg_lo = rmid % mpam_num_pmg();
+ u16 base_partid = closid;
+ u16 partid_stride = 0;
+
+ if (cdp_enabled) {
+ base_partid *= 2;
+ partid_stride = 1;
+ }
+
+ partid_stride *= partid_per_closid;
+ base_partid *= partid_per_closid;
+ base_partid += pmg_hi;
- BUG_ON(closid_shift > 8);
+ *partid_d = base_partid;
+ if (partid_i)
+ *partid_i = base_partid + partid_stride;
- return (closid << closid_shift) | rmid;
+ *pmg = pmg_lo;
}
-void resctrl_arch_rmid_idx_decode(u32 idx, u32 *closid, u32 *rmid)
+u32 resctrl_arch_rmid_idx_encode(u32 closid, u32 rmid)
+{
+ return closid * mpam_num_rmid() + rmid;
+}
+
+u32 resctrl_arch_system_num_rmid_idx(void)
{
- u8 closid_shift = fls(mpam_pmg_max);
- u32 pmg_mask = ~(~0 << closid_shift);
+ u32 num_closid = resctrl_arch_get_num_closid(NULL);
- BUG_ON(closid_shift > 8);
+ return resctrl_arch_rmid_idx_encode(num_closid, 0);
+}
- *closid = idx >> closid_shift;
- *rmid = idx & pmg_mask;
+void resctrl_arch_rmid_idx_decode(u32 idx, u32 *closid, u32 *rmid)
+{
+ *closid = idx / mpam_num_rmid();
+ *rmid = idx % mpam_num_rmid();
}
void resctrl_arch_sched_in(struct task_struct *tsk)
@@ -203,21 +253,14 @@ void resctrl_arch_sched_in(struct task_struct *tsk)
void resctrl_arch_set_cpu_default_closid_rmid(int cpu, u32 closid, u32 rmid)
{
+ u16 partid_d, partid_i;
+ u8 pmg;
+
BUG_ON(closid > U16_MAX);
BUG_ON(rmid > U8_MAX);
- if (!cdp_enabled) {
- mpam_set_cpu_defaults(cpu, closid, closid, rmid, rmid);
- } else {
- /*
- * When CDP is enabled, resctrl halves the closid range and we
- * use odd/even partid for one closid.
- */
- u32 partid_d = resctrl_get_config_index(closid, CDP_DATA);
- u32 partid_i = resctrl_get_config_index(closid, CDP_CODE);
-
- mpam_set_cpu_defaults(cpu, partid_d, partid_i, rmid, rmid);
- }
+ mpam_resctrl_hwid(closid, rmid, &partid_d, &partid_i, &pmg);
+ mpam_set_cpu_defaults(cpu, partid_d, partid_i, pmg, pmg);
}
void resctrl_arch_sync_cpu_closid_rmid(void *info)
@@ -236,41 +279,38 @@ void resctrl_arch_sync_cpu_closid_rmid(void *info)
void resctrl_arch_set_closid_rmid(struct task_struct *tsk, u32 closid, u32 rmid)
{
+ u16 partid_d, partid_i;
+ u8 pmg;
+
BUG_ON(closid > U16_MAX);
BUG_ON(rmid > U8_MAX);
- if (!cdp_enabled) {
- mpam_set_task_partid_pmg(tsk, closid, closid, rmid, rmid);
- } else {
- u32 partid_d = resctrl_get_config_index(closid, CDP_DATA);
- u32 partid_i = resctrl_get_config_index(closid, CDP_CODE);
-
- mpam_set_task_partid_pmg(tsk, partid_d, partid_i, rmid, rmid);
- }
+ mpam_resctrl_hwid(closid, rmid, &partid_d, &partid_i, &pmg);
+ mpam_set_task_partid_pmg(tsk, partid_d, partid_i, pmg, pmg);
}
bool resctrl_arch_match_closid(struct task_struct *tsk, u32 closid)
{
+ u16 min_partid, max_partid;
u64 regval = mpam_get_regval(tsk);
u32 tsk_closid = FIELD_GET(MPAM1_EL1_PARTID_D, regval);
- if (cdp_enabled)
- tsk_closid >>= 1;
-
- return tsk_closid == closid;
+ mpam_resctrl_partid_range(closid, CDP_NONE, NULL,
+ &min_partid, &max_partid);
+ return tsk_closid >= min_partid && tsk_closid <= max_partid;
}
/* The task's pmg is not unique, the partid must be considered too */
bool resctrl_arch_match_rmid(struct task_struct *tsk, u32 closid, u32 rmid)
{
+ u16 partid_d;
+ u8 pmg;
u64 regval = mpam_get_regval(tsk);
- u32 tsk_closid = FIELD_GET(MPAM1_EL1_PARTID_D, regval);
- u32 tsk_rmid = FIELD_GET(MPAM1_EL1_PMG_D, regval);
-
- if (cdp_enabled)
- tsk_closid >>= 1;
+ u16 tsk_partid_d = FIELD_GET(MPAM1_EL1_PARTID_D, regval);
+ u8 tsk_pmg = FIELD_GET(MPAM1_EL1_PMG_D, regval);
- return (tsk_closid == closid) && (tsk_rmid == rmid);
+ mpam_resctrl_hwid(closid, rmid, &partid_d, NULL, &pmg);
+ return (tsk_partid_d == partid_d) && (tsk_pmg == pmg);
}
struct rdt_resource *resctrl_arch_get_resource(enum resctrl_res_level l)
@@ -370,6 +410,7 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_mon_domain *d,
struct mpam_resctrl_dom *dom;
u32 mon = *(u32 *)arch_mon_ctx;
enum mpam_device_features type;
+ u16 partid_d, partid_i, pmg;
resctrl_arch_rmid_read_context_check();
@@ -391,27 +432,23 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_mon_domain *d,
if (cfg.mon == USE_RMID_IDX)
cfg.mon = resctrl_arch_rmid_idx_encode(closid, rmid);
+ mpam_resctrl_hwid(closid, rmid, &partid_d, &partid_i, &cfg.pmg);
cfg.match_pmg = true;
- cfg.pmg = rmid;
+ cfg.pmg = pmg;
cfg.opts = resctrl_evt_config_to_mpam(dom->mbm_local_evt_cfg);
if (irqs_disabled()) {
/* Check if we can access this domain without an IPI */
err = -EIO;
} else {
- if (cdp_enabled) {
- cfg.partid = closid << 1;
- err = mpam_msmon_read(dom->comp, &cfg, type, val);
- if (err)
- return err;
+ cfg.partid = partid_d;
+ err = mpam_msmon_read(dom->comp, &cfg, type, val);
- cfg.partid += 1;
+ if (partid_i != partid_d) {
+ cfg.partid = partid_i;
err = mpam_msmon_read(dom->comp, &cfg, type, &cdp_val);
if (!err)
*val += cdp_val;
- } else {
- cfg.partid = closid;
- err = mpam_msmon_read(dom->comp, &cfg, type, val);
}
}
@@ -423,6 +460,7 @@ void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_mon_domain *d,
{
struct mon_cfg cfg;
struct mpam_resctrl_dom *dom;
+ u16 partid_d, partid_i;
if (eventid != QOS_L3_MBM_LOCAL_EVENT_ID)
return;
@@ -433,14 +471,13 @@ void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_mon_domain *d,
dom = container_of(d, struct mpam_resctrl_dom, resctrl_mon_dom);
- if (cdp_enabled) {
- cfg.partid = closid << 1;
- mpam_msmon_reset_mbwu(dom->comp, &cfg);
+ mpam_resctrl_hwid(closid, rmid, &partid_d, &partid_i, &cfg.pmg);
- cfg.partid += 1;
- mpam_msmon_reset_mbwu(dom->comp, &cfg);
- } else {
- cfg.partid = closid;
+ cfg.partid = partid_d;
+ mpam_msmon_reset_mbwu(dom->comp, &cfg);
+
+ if (partid_i != partid_d) {
+ cfg.partid = partid_i;
mpam_msmon_reset_mbwu(dom->comp, &cfg);
}
}
@@ -1051,15 +1088,6 @@ int mpam_resctrl_setup(void)
err = -EOPNOTSUPP;
if (!err) {
- if (!is_power_of_2(mpam_pmg_max + 1)) {
- /*
- * If not all the partid*pmg values are valid indexes,
- * resctrl may allocate pmg that don't exist. This
- * should cause an error interrupt.
- */
- pr_warn("Number of PMG is not a power of 2! resctrl may misbehave");
- }
-
err = resctrl_init();
if (!err)
WRITE_ONCE(resctrl_enabled, true);
@@ -1081,7 +1109,7 @@ static void mpam_resctrl_exit(void)
u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_ctrl_domain *d,
u32 closid, enum resctrl_conf_type type)
{
- u32 partid;
+ u16 partid;
struct mpam_config *cfg;
struct mpam_props *cprops;
struct mpam_resctrl_res *res;
@@ -1097,15 +1125,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_ctrl_domain *d,
dom = container_of(d, struct mpam_resctrl_dom, resctrl_ctrl_dom);
cprops = &res->class->props;
- /*
- * When CDP is enabled, but the resource doesn't support it,
- * the control is cloned across both partids.
- * Pick one at random to read:
- */
- if (mpam_resctrl_hide_cdp(r->rid))
- type = CDP_DATA;
-
- partid = resctrl_get_config_index(closid, type);
+ mpam_resctrl_partid_range(closid, type, r, &partid, NULL);
cfg = &dom->comp->cfg[partid];
switch (r->rid) {
@@ -1126,7 +1146,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_ctrl_domain *d,
return -EINVAL;
}
- if (!r->alloc_capable || partid >= resctrl_arch_get_num_closid(r) ||
+ if (!r->alloc_capable || partid > mpam_partid_max ||
!mpam_has_feature(configured_by, cfg))
return resctrl_get_default_ctrl(r);
@@ -1147,8 +1167,8 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_ctrl_domain *d,
int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_ctrl_domain *d,
u32 closid, enum resctrl_conf_type t, u32 cfg_val)
{
- int err;
- u32 partid;
+ int err = 0;
+ u16 partid, min_partid, max_partid;
struct mpam_config cfg;
struct mpam_props *cprops;
struct mpam_resctrl_res *res;
@@ -1163,8 +1183,10 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_ctrl_domain *d,
dom = container_of(d, struct mpam_resctrl_dom, resctrl_ctrl_dom);
cprops = &res->class->props;
- partid = resctrl_get_config_index(closid, t);
- if (!r->alloc_capable || partid >= resctrl_arch_get_num_closid(r))
+ mpam_resctrl_partid_range(closid, t, r, &min_partid, &max_partid);
+ if (!r->alloc_capable ||
+ min_partid > mpam_partid_max ||
+ max_partid > mpam_partid_max)
return -EINVAL;
cfg.features = 0;
@@ -1190,22 +1212,13 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_ctrl_domain *d,
return -EINVAL;
}
- /*
- * When CDP is enabled, but the resource doesn't support it, we need to
- * apply the same configuration to the other partid.
- */
- if (mpam_resctrl_hide_cdp(r->rid)) {
- partid = resctrl_get_config_index(closid, CDP_CODE);
+ for (partid = min_partid; partid <= max_partid; partid++) {
err = mpam_apply_config(dom->comp, partid, &cfg);
if (err)
- return err;
-
- partid = resctrl_get_config_index(closid, CDP_DATA);
- return mpam_apply_config(dom->comp, partid, &cfg);
-
- } else {
- return mpam_apply_config(dom->comp, partid, &cfg);
+ break;
}
+
+ return err;
}
/* TODO: this is IPI heavy */
--
2.34.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* [RFC PATCH v2 07/11] arm_mpam: [NFU] Rework ID remapping to use a kernel command-line argument
2025-01-17 15:10 [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
` (17 preceding siblings ...)
2025-01-17 15:10 ` [RFC PATCH v2 06/11] arm_mpam: Introduce flexible CLOSID/RMID translation Dave Martin
@ 2025-01-17 15:10 ` Dave Martin
2025-01-29 10:57 ` Shaopeng Tan (Fujitsu)
2025-01-17 15:10 ` [RFC PATCH v2 08/11] arm_mpam: [NFU] Development diagnostics for MPAM ID assignments Dave Martin
` (3 subsequent siblings)
22 siblings, 1 reply; 26+ messages in thread
From: Dave Martin @ 2025-01-17 15:10 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-arm-kernel, Zeng Heng, Shaopeng Tan, James Morse
Hacking the source is an acceptable interface for experimentation,
but not very convenient.
Instead, add a kernel command-line parameter
mpam_partid_per_closid=<n> to specify how many PARTIDs are used to
provide monitoring groups for each resctrl CLOSID.
This change is not intended for upstream.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
drivers/platform/arm64/mpam/mpam_resctrl.c | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)
diff --git a/drivers/platform/arm64/mpam/mpam_resctrl.c b/drivers/platform/arm64/mpam/mpam_resctrl.c
index f834753a3274..eb1ef5d2bc57 100644
--- a/drivers/platform/arm64/mpam/mpam_resctrl.c
+++ b/drivers/platform/arm64/mpam/mpam_resctrl.c
@@ -3,6 +3,9 @@
#define pr_fmt(fmt) "mpam: resctrl: " fmt
+#undef KBUILD_MODNAME
+#define KBUILD_MODNAME "mpam"
+
#include <linux/arm_mpam.h>
#include <linux/cacheinfo.h>
#include <linux/cpu.h>
@@ -10,6 +13,7 @@
#include <linux/errno.h>
#include <linux/limits.h>
#include <linux/list.h>
+#include <linux/moduleparam.h>
#include <linux/printk.h>
#include <linux/rculist.h>
#include <linux/resctrl.h>
@@ -158,6 +162,24 @@ static bool mpam_resctrl_hide_cdp(enum resctrl_res_level rid)
static unsigned int partid_per_closid = 1;
+static int mpam_resctrl_partid_per_closid_set(const char *val,
+ const struct kernel_param *kp)
+{
+ /*
+ * 16 in an arbitrary maximum, sufficient for experimentation
+ * but not ridiculously large:
+ */
+ return param_set_uint_minmax(val, kp, 1, 16);
+}
+
+static const struct kernel_param_ops mpam_resctrl_partid_per_closid_ops = {
+ .set = mpam_resctrl_partid_per_closid_set,
+ .get = param_get_uint,
+};
+
+device_param_cb(partid_per_closid, &mpam_resctrl_partid_per_closid_ops,
+ &partid_per_closid, 0444);
+
static unsigned int mpam_num_pmg(void)
{
return mpam_pmg_max + 1;
--
2.34.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* [RFC PATCH v2 08/11] arm_mpam: [NFU] Development diagnostics for MPAM ID assignments
2025-01-17 15:10 [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
` (18 preceding siblings ...)
2025-01-17 15:10 ` [RFC PATCH v2 07/11] arm_mpam: [NFU] Rework ID remapping to use a kernel command-line argument Dave Martin
@ 2025-01-17 15:10 ` Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 09/11] arm_mpam: Add control partition identifier to mpam_config Dave Martin
` (2 subsequent siblings)
22 siblings, 0 replies; 26+ messages in thread
From: Dave Martin @ 2025-01-17 15:10 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-arm-kernel, Zeng Heng, Shaopeng Tan, James Morse
Purely for development purposes, add some printks to show what MPAM
IDs actually get assigned and configured.
This would otherwise be observable only though statistical
measurements of performance (or not observable at all).
This change is not intended for upstream.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
arch/arm64/include/asm/mpam.h | 7 +++++++
drivers/platform/arm64/mpam/mpam_devices.c | 6 ++++++
2 files changed, 13 insertions(+)
diff --git a/arch/arm64/include/asm/mpam.h b/arch/arm64/include/asm/mpam.h
index df725d2d9d05..4ae2e635e261 100644
--- a/arch/arm64/include/asm/mpam.h
+++ b/arch/arm64/include/asm/mpam.h
@@ -66,6 +66,9 @@ static inline void mpam_set_cpu_defaults(int cpu, u16 partid_d, u16 partid_i,
default_val |= FIELD_PREP(MPAM1_EL1_PMG_I, pmg_i);
WRITE_ONCE(per_cpu(arm64_mpam_default, cpu), default_val);
+
+ pr_info("MPAM: CPU%d -> (D=%d:%d, I=%d:%d)\n",
+ cpu, partid_d, pmg_d, partid_i, pmg_i);
}
static inline void mpam_set_task_partid_pmg(struct task_struct *tsk,
@@ -81,6 +84,10 @@ static inline void mpam_set_task_partid_pmg(struct task_struct *tsk,
regval |= FIELD_PREP(MPAM1_EL1_PMG_I, pmg_i);
WRITE_ONCE(task_thread_info(tsk)->mpam_partid_pmg, regval);
+
+ pr_info("MPAM: task %s[%d/%d] -> (D=%d:%d, I=%d:%d)\n",
+ current->comm, current->pid, current->tgid,
+ partid_d, pmg_d, partid_i, pmg_i);
#endif
}
diff --git a/drivers/platform/arm64/mpam/mpam_devices.c b/drivers/platform/arm64/mpam/mpam_devices.c
index f965e6639e0d..6b326506c359 100644
--- a/drivers/platform/arm64/mpam/mpam_devices.c
+++ b/drivers/platform/arm64/mpam/mpam_devices.c
@@ -3149,6 +3149,12 @@ int mpam_apply_config(struct mpam_component *comp, u16 partid,
mpam_extend_config(comp->class, cfg);
+ pr_info("mpam_apply_config(): comp %d partid %d cfg={0x%x: 0x%lx, 0x%lx, %u, %u}\n",
+ comp->comp_id, partid, cfg->features,
+ (unsigned long)cfg->cpbm,
+ (unsigned long)cfg->mbw_pbm,
+ cfg->mbw_min, cfg->mbw_max);
+
if (!mpam_update_config(&comp->cfg[partid], cfg))
return 0;
--
2.34.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* [RFC PATCH v2 09/11] arm_mpam: Add control partition identifier to mpam_config
2025-01-17 15:10 [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
` (19 preceding siblings ...)
2025-01-17 15:10 ` [RFC PATCH v2 08/11] arm_mpam: [NFU] Development diagnostics for MPAM ID assignments Dave Martin
@ 2025-01-17 15:10 ` Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 10/11] arm_mpam: Skip inapplicable cases when reprogramming a resctrl domain Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 11/11] arm_mpam: Detect whether the system can alias PARTIDs Dave Martin
22 siblings, 0 replies; 26+ messages in thread
From: Dave Martin @ 2025-01-17 15:10 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-arm-kernel, Zeng Heng, Shaopeng Tan, James Morse
Currently, MPAM PARTIDs are assigned 1:1 to resctrl control groups.
This means that an MSC that implements PARTID Narrowing can simply
program the PARTID value for intPARTID where needed.
In order to enable multiple PARTIDs to be assigned to a single
control group in a flexible manner, the PARTID value cannot be used
directly for the intPARTID, so in the mpam_devices backend,
Narrowing MSCs need an explicit identifier for the control /
resource regulation group.
Add a field cpartid to struct mpam_config to hold this identifier,
and wire up the resctrl glue code to pass a suitable value.
In common with the handling of other parameters, make programming
of MPAMCFG_INTPARTID unconditional when applying control updates.
When no cpartid is supplied, e.g., during driver startup, intPARTID
is programmed with 0. (There is no architectural reset value for
MPAMCFG_INTPARTID even for PARTID 0, but the architecture can be
interpreted as suggesting intPARTID 0 as the default, so this is as
good a choice as any.) Control updates through resctrl will always
supply a cpartid, identifying the affected resctrl control
partition.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
drivers/platform/arm64/mpam/mpam_devices.c | 10 ++++++++--
drivers/platform/arm64/mpam/mpam_internal.h | 7 +++++++
drivers/platform/arm64/mpam/mpam_resctrl.c | 16 +++++++++++-----
3 files changed, 26 insertions(+), 7 deletions(-)
diff --git a/drivers/platform/arm64/mpam/mpam_devices.c b/drivers/platform/arm64/mpam/mpam_devices.c
index 6b326506c359..4b43dfef1cdc 100644
--- a/drivers/platform/arm64/mpam/mpam_devices.c
+++ b/drivers/platform/arm64/mpam/mpam_devices.c
@@ -1531,6 +1531,7 @@ static void mpam_quirk_post_config_change(struct mpam_msc_ris *ris, u16 partid,
static void mpam_reprogram_ris_partid(struct mpam_msc_ris *ris, u16 partid,
struct mpam_config *cfg)
{
+ u16 intpartid = 0;
u32 pri_val = 0;
u16 cmax = MPAMCFG_CMAX_CMAX;
u16 bwa_fract = MPAMCFG_MBW_MAX_MAX;
@@ -1543,9 +1544,13 @@ static void mpam_reprogram_ris_partid(struct mpam_msc_ris *ris, u16 partid,
__mpam_part_sel(ris->ris_idx, partid, msc);
if (mpam_has_feature(mpam_feat_partid_nrw, rprops)) {
+ if (mpam_has_feature(mpam_feat_partid_nrw, cfg))
+ intpartid = cfg->cpartid;
+
mpam_write_partsel_reg(msc, INTPARTID,
- MPAMCFG_INTPARTID_INTERNAL | partid);
- __mpam_intpart_sel(ris->ris_idx, partid, msc);
+ MPAMCFG_INTPARTID_INTERNAL | intpartid);
+
+ __mpam_intpart_sel(ris->ris_idx, intpartid, msc);
}
if (mpam_has_feature(mpam_feat_cpor_part, rprops)) {
@@ -3084,6 +3089,7 @@ static mpam_features_t mpam_update_config(struct mpam_config *cfg,
{
mpam_features_t changes = 0;
+ maybe_update_config(cfg, mpam_feat_partid_nrw, newcfg, cpartid, changes);
maybe_update_config(cfg, mpam_feat_cpor_part, newcfg, cpbm, changes);
maybe_update_config(cfg, mpam_feat_mbw_part, newcfg, mbw_pbm, changes);
maybe_update_config(cfg, mpam_feat_mbw_max, newcfg, mbw_max, changes);
diff --git a/drivers/platform/arm64/mpam/mpam_internal.h b/drivers/platform/arm64/mpam/mpam_internal.h
index 5af6ed60272e..965f4fea3012 100644
--- a/drivers/platform/arm64/mpam/mpam_internal.h
+++ b/drivers/platform/arm64/mpam/mpam_internal.h
@@ -300,6 +300,13 @@ struct mpam_config {
u16 mbw_max;
u16 mbw_min;
+ /*
+ * Control partition; resource regulation context for stateful controls
+ * Valid if features has mpam_feat_partid_nrw.
+ * For MSCs that implement PARTID Narrowing, this is intPARTID.
+ */
+ u16 cpartid;
+
struct mpam_garbage garbage;
};
diff --git a/drivers/platform/arm64/mpam/mpam_resctrl.c b/drivers/platform/arm64/mpam/mpam_resctrl.c
index eb1ef5d2bc57..f1d0d9f59771 100644
--- a/drivers/platform/arm64/mpam/mpam_resctrl.c
+++ b/drivers/platform/arm64/mpam/mpam_resctrl.c
@@ -205,7 +205,8 @@ u32 resctrl_arch_get_num_closid(struct rdt_resource *ignored)
static void mpam_resctrl_partid_range(u32 closid, enum resctrl_conf_type type,
const struct rdt_resource *r,
- u16 *min_partid, u16 *max_partid)
+ u16 *min_partid, u16 *max_partid,
+ u16 *cpartid)
{
u16 base_partid = closid;
u16 span = 1;
@@ -222,6 +223,9 @@ static void mpam_resctrl_partid_range(u32 closid, enum resctrl_conf_type type,
*min_partid = base_partid * partid_per_closid;
if (max_partid)
*max_partid = *min_partid + (span * partid_per_closid - 1);
+
+ if (cpartid)
+ *cpartid = base_partid;
}
static void mpam_resctrl_hwid(u32 closid, u32 rmid,
@@ -318,7 +322,7 @@ bool resctrl_arch_match_closid(struct task_struct *tsk, u32 closid)
u32 tsk_closid = FIELD_GET(MPAM1_EL1_PARTID_D, regval);
mpam_resctrl_partid_range(closid, CDP_NONE, NULL,
- &min_partid, &max_partid);
+ &min_partid, &max_partid, NULL);
return tsk_closid >= min_partid && tsk_closid <= max_partid;
}
@@ -1147,7 +1151,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_ctrl_domain *d,
dom = container_of(d, struct mpam_resctrl_dom, resctrl_ctrl_dom);
cprops = &res->class->props;
- mpam_resctrl_partid_range(closid, type, r, &partid, NULL);
+ mpam_resctrl_partid_range(closid, type, r, &partid, NULL, NULL);
cfg = &dom->comp->cfg[partid];
switch (r->rid) {
@@ -1205,13 +1209,15 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_ctrl_domain *d,
dom = container_of(d, struct mpam_resctrl_dom, resctrl_ctrl_dom);
cprops = &res->class->props;
- mpam_resctrl_partid_range(closid, t, r, &min_partid, &max_partid);
+ cfg.features = 0;
+ mpam_set_feature(mpam_feat_partid_nrw, &cfg);
+ mpam_resctrl_partid_range(closid, t, r, &min_partid, &max_partid,
+ &cfg.cpartid);
if (!r->alloc_capable ||
min_partid > mpam_partid_max ||
max_partid > mpam_partid_max)
return -EINVAL;
- cfg.features = 0;
switch (r->rid) {
case RDT_RESOURCE_L2:
case RDT_RESOURCE_L3:
--
2.34.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* [RFC PATCH v2 10/11] arm_mpam: Skip inapplicable cases when reprogramming a resctrl domain
2025-01-17 15:10 [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
` (20 preceding siblings ...)
2025-01-17 15:10 ` [RFC PATCH v2 09/11] arm_mpam: Add control partition identifier to mpam_config Dave Martin
@ 2025-01-17 15:10 ` Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 11/11] arm_mpam: Detect whether the system can alias PARTIDs Dave Martin
22 siblings, 0 replies; 26+ messages in thread
From: Dave Martin @ 2025-01-17 15:10 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-arm-kernel, Zeng Heng, Shaopeng Tan, James Morse
The implementation of resctrl_arch_update_domains() currently
iterates over all types of staged configuration changes passed from
resctrl, irrespective of whether a given type (CDP_CODE, CDP_DATA
or CDP_NONE) makes sense for the affected MPAM resource. Since
resctrl will not stage any updates of inappropriate type for the
resource, this is mostly harmless.
When doing a forced update as a result of a call via
resctrl_arch_init_domains() however, there is a need to update the
cpartid (intPARTID) even if no staged configuration changes are
supplied. This can result in intPARTID being repeatedly
reprogrammed with different values. As well as being inefficient,
the final value may not be the correct one.
To program just once with the correct intPARTID, only apply the
update types that actually make sense for the resource.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
drivers/platform/arm64/mpam/mpam_resctrl.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/drivers/platform/arm64/mpam/mpam_resctrl.c b/drivers/platform/arm64/mpam/mpam_resctrl.c
index f1d0d9f59771..1f31b1455f66 100644
--- a/drivers/platform/arm64/mpam/mpam_resctrl.c
+++ b/drivers/platform/arm64/mpam/mpam_resctrl.c
@@ -1263,6 +1263,10 @@ static int __resctrl_arch_update_domains(struct rdt_resource *r, u32 closid,
list_for_each_entry(d, &r->ctrl_domains, hdr.list) {
for (t = 0; t < CDP_NUM_TYPES; t++) {
+ if ((t == CDP_NONE) !=
+ (!cdp_enabled || mpam_resctrl_hide_cdp(r->rid)))
+ continue;
+
cfg = &d->staged_config[t];
if (!force && !cfg->have_new_ctrl)
continue;
--
2.34.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* [RFC PATCH v2 11/11] arm_mpam: Detect whether the system can alias PARTIDs
2025-01-17 15:10 [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
` (21 preceding siblings ...)
2025-01-17 15:10 ` [RFC PATCH v2 10/11] arm_mpam: Skip inapplicable cases when reprogramming a resctrl domain Dave Martin
@ 2025-01-17 15:10 ` Dave Martin
22 siblings, 0 replies; 26+ messages in thread
From: Dave Martin @ 2025-01-17 15:10 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-arm-kernel, Zeng Heng, Shaopeng Tan, James Morse
In order to be able to assign multiple PARTIDs freely to resource
control groups, it is necessary that all MSCs that are used by resctrl
either implement PARTID Narrowing (so that PARTIDs can be remapped
explicitly) or otherwise only have controls whose resource regulation
is stateless and so whose behaviour would not be affected by splitting
a control group across multiple PARTIDs.
Detect this property at the RIS level and merge it up the MPAM devices
tree.
Export the common global result as mpam_partid_aliasing, and apply
it to the PARTIDs-per-CLOSID configuration.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
Currently, this the logic is simple and may forbid PARTID aliasing
on some systems that can support it as a result of the precise set
of MSCs and controls that are exposed through resctrl.
The aim is to make this logic more flexible later.
---
drivers/platform/arm64/mpam/mpam_devices.c | 48 ++++++++++++++++++---
drivers/platform/arm64/mpam/mpam_internal.h | 2 +
drivers/platform/arm64/mpam/mpam_resctrl.c | 12 +++++-
3 files changed, 53 insertions(+), 9 deletions(-)
diff --git a/drivers/platform/arm64/mpam/mpam_devices.c b/drivers/platform/arm64/mpam/mpam_devices.c
index 4b43dfef1cdc..0182c922638f 100644
--- a/drivers/platform/arm64/mpam/mpam_devices.c
+++ b/drivers/platform/arm64/mpam/mpam_devices.c
@@ -68,6 +68,7 @@ static DEFINE_MUTEX(mpam_cpuhp_state_lock);
*/
u16 mpam_partid_max;
u8 mpam_pmg_max;
+bool mpam_partid_aliasing; /* PARTID aliasing supported */
static bool partid_max_init, partid_max_published;
static DEFINE_SPINLOCK(partid_max_lock);
@@ -828,6 +829,7 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)
struct mpam_msc *msc = ris->vmsc->msc;
struct mpam_props *props = &ris->props;
struct mpam_class *class = ris->vmsc->comp->class;
+ bool partid_can_alias = true;
lockdep_assert_held(&msc->probe_lock);
lockdep_assert_held(&msc->part_sel_lock);
@@ -836,6 +838,8 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)
/* Cache Capacity Partitioning */
if (FIELD_GET(MPAMF_IDR_HAS_CCAP_PART, ris->idr)) {
+ partid_can_alias = false;
+
ris->ccap_idr = mpam_read_partsel_reg(msc, CCAP_IDR);
props->cmax_wd = FIELD_GET(MPAMF_CCAP_IDR_CMAX_WD, ris->ccap_idr);
@@ -884,14 +888,20 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)
*/
props->bwa_wd = min(props->bwa_wd, 16);
- if (props->bwa_wd && FIELD_GET(MPAMF_MBW_IDR_HAS_MAX, mbw_features))
+ if (props->bwa_wd && FIELD_GET(MPAMF_MBW_IDR_HAS_MAX, mbw_features)) {
mpam_set_feature(mpam_feat_mbw_max, props);
+ partid_can_alias = false;
+ }
- if (props->bwa_wd && FIELD_GET(MPAMF_MBW_IDR_HAS_MIN, mbw_features))
+ if (props->bwa_wd && FIELD_GET(MPAMF_MBW_IDR_HAS_MIN, mbw_features)) {
mpam_set_feature(mpam_feat_mbw_min, props);
+ partid_can_alias = false;
+ }
- if (props->bwa_wd && FIELD_GET(MPAMF_MBW_IDR_HAS_PROP, mbw_features))
+ if (props->bwa_wd && FIELD_GET(MPAMF_MBW_IDR_HAS_PROP, mbw_features)) {
mpam_set_feature(mpam_feat_mbw_prop, props);
+ partid_can_alias = false;
+ }
}
/* Priority partitioning */
@@ -988,17 +998,26 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)
* RIS with PARTID narrowing don't have enough storage for one
* configuration per PARTID. If these are in a class we could use,
* reduce the supported partid_max to match the numer of intpartid.
- * If the class is unknown, just ignore it.
+ * If the MSC won't be used for anything or implements PARTID
+ * Narrowing then PARTIDs can be assigned freely, so PARTID
+ * aliasing can be supported regardless of the resource
+ * controls present.
*/
- if (FIELD_GET(MPAMF_IDR_HAS_PARTID_NRW, ris->idr) &&
- class->type != MPAM_CLASS_UNKNOWN) {
+ if (class->type == MPAM_CLASS_UNKNOWN)
+ partid_can_alias = true;
+ else if (FIELD_GET(MPAMF_IDR_HAS_PARTID_NRW, ris->idr)) {
u32 nrwidr = mpam_read_partsel_reg(msc, PARTID_NRW_IDR);
u16 partid_max = FIELD_GET(MPAMF_PARTID_NRW_IDR_INTPARTID_MAX, nrwidr);
mpam_set_feature(mpam_feat_partid_nrw, props);
msc->partid_max = min(msc->partid_max, partid_max);
+
+ partid_can_alias = true;
}
+ if (partid_can_alias)
+ mpam_set_feature(mpam_feat_partid_aliasing, props);
+
mpam_mon_sel_outer_unlock(msc);
}
@@ -2368,6 +2387,10 @@ static void __props_mismatch(struct mpam_props *parent,
mpam_clear_feature(mpam_feat_dspri_part_0_low, &parent->features);
}
+ /* PARTID aliasing must be supported everywhere, or not at all: */
+ if (!mpam_has_feature(mpam_feat_partid_aliasing, parent))
+ mpam_clear_feature(mpam_feat_partid_aliasing, &parent->features);
+
if (alias) {
/* Merge features for aliased resources */
parent->features |= child->features;
@@ -2486,6 +2509,7 @@ static void mpam_enable_merge_features(struct list_head *all_classes_list)
{
struct mpam_class *class;
struct mpam_component *comp;
+ bool partid_can_alias = true, comp_partid_aliasing;
lockdep_assert_held(&mpam_list_lock);
@@ -2495,9 +2519,19 @@ static void mpam_enable_merge_features(struct list_head *all_classes_list)
mpam_enable_init_class_features(class);
- list_for_each_entry(comp, &class->components, class_list)
+ list_for_each_entry(comp, &class->components, class_list) {
mpam_enable_merge_class_features(comp);
+
+ comp_partid_aliasing = mpam_has_feature(
+ mpam_feat_partid_aliasing, &class->props);
+ partid_can_alias = partid_can_alias &&
+ comp_partid_aliasing;
+ }
}
+
+ spin_lock(&partid_max_lock);
+ mpam_partid_aliasing = partid_can_alias;
+ spin_unlock(&partid_max_lock);
}
static char *mpam_errcode_names[16] = {
diff --git a/drivers/platform/arm64/mpam/mpam_internal.h b/drivers/platform/arm64/mpam/mpam_internal.h
index 965f4fea3012..6e10e4789506 100644
--- a/drivers/platform/arm64/mpam/mpam_internal.h
+++ b/drivers/platform/arm64/mpam/mpam_internal.h
@@ -215,6 +215,7 @@ enum mpam_device_features {
mpam_feat_msmon_mbwu_hw_nrdy,
mpam_feat_msmon_capt,
mpam_feat_partid_nrw,
+ mpam_feat_partid_aliasing, /* derived feature */
MPAM_FEATURE_LAST,
};
#define MPAM_ALL_FEATURES ((1<<MPAM_FEATURE_LAST) - 1)
@@ -474,6 +475,7 @@ static inline void mpam_assert_srcu_read_lock_held(void)
/* System wide partid/pmg values */
extern u16 mpam_partid_max;
extern u8 mpam_pmg_max;
+extern bool mpam_partid_aliasing;
/* Scheduled work callback to enable mpam once all MSC have been probed */
void mpam_enable(struct work_struct *work);
diff --git a/drivers/platform/arm64/mpam/mpam_resctrl.c b/drivers/platform/arm64/mpam/mpam_resctrl.c
index 1f31b1455f66..5b296e3e27c2 100644
--- a/drivers/platform/arm64/mpam/mpam_resctrl.c
+++ b/drivers/platform/arm64/mpam/mpam_resctrl.c
@@ -160,7 +160,8 @@ static bool mpam_resctrl_hide_cdp(enum resctrl_res_level rid)
return cdp_enabled && !resctrl_arch_get_cdp_enabled(rid);
}
-static unsigned int partid_per_closid = 1;
+static unsigned int bootparam_partid_per_closid = 1;
+static unsigned int partid_per_closid;
static int mpam_resctrl_partid_per_closid_set(const char *val,
const struct kernel_param *kp)
@@ -178,7 +179,7 @@ static const struct kernel_param_ops mpam_resctrl_partid_per_closid_ops = {
};
device_param_cb(partid_per_closid, &mpam_resctrl_partid_per_closid_ops,
- &partid_per_closid, 0444);
+ &bootparam_partid_per_closid, 0444);
static unsigned int mpam_num_pmg(void)
{
@@ -1113,6 +1114,13 @@ int mpam_resctrl_setup(void)
if (!err && !exposed_alloc_capable && !exposed_mon_capable)
err = -EOPNOTSUPP;
+ partid_per_closid = bootparam_partid_per_closid;
+ if ((!mpam_partid_aliasing && bootparam_partid_per_closid > 1) ||
+ resctrl_arch_get_num_closid(NULL) < 1) {
+ pr_warn("Hardware incompatible with PARTID aliasing, limiting monitoring groups\n");
+ partid_per_closid = 1;
+ }
+
if (!err) {
err = resctrl_init();
if (!err)
--
2.34.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* RE: [RFC PATCH v2 06/11] arm_mpam: Introduce flexible CLOSID/RMID translation
2025-01-17 15:10 ` [RFC PATCH v2 06/11] arm_mpam: Introduce flexible CLOSID/RMID translation Dave Martin
@ 2025-01-29 10:40 ` Shaopeng Tan (Fujitsu)
0 siblings, 0 replies; 26+ messages in thread
From: Shaopeng Tan (Fujitsu) @ 2025-01-29 10:40 UTC (permalink / raw)
To: 'Dave Martin', linux-kernel
Cc: linux-arm-kernel, Zeng Heng, James Morse
Hello
> Currently, the MPAM driver uses the resctrl CLOSID directly as the MPAM
> Partition Identifier (PARTID; possibly modified to give different PARTIDs to
> code and data when Code/Data Partitioning / CDP is enabled), and uses the
> resctrl RMID directly as the MPAM Performance Monitoring Group identifier
> (PMG).
>
> In preparation for using the MPAM PARTID Narrowing feature to allow more
> resctrl monitoring groups to be provided than the number of PMG values
> supported by the hardware, a more flexible remapping scheme is needed.
>
> Factor out ID translation operations in the MPAM resctrl glue code into a couple
> of self-contained helpers, and call them as appropriate.
>
> The translation scheme has a single parameter, partid_per_closid, which is
> currently hard-wired to 1.
>
> As a result, this patch should have no effect on functionality.
>
> Logic to determine / control the mapping parameters can be introduced later.
>
> The ID transformation may be visualised as follows:
>
> without CDP:
> +---------------------------------------------------+-----------+
> | CLOSID | RMID |
> +---------------------------------------------------+-------+---+
> | PARTID : |PMG|
> +-----------------------------------------------------------+---+
>
> and with CDP, where the "CDP" field is 0 for the data sub- partition and 1 for
> the code sub-partition:
> +-----------------------------------------------+---+-----------+
> | CLOSID |CDP| RMID |
> +-----------------------------------------------+---+-------+---+
> | PARTID : |PMG|
> +-----------------------------------------------------------+---+
>
> where each box represents a non-negative integer spanning some fixed range
> starting from zero, and horizontal concatenation denotes multiplying the value
> denoted by the left-hand box by the span of the right-hand box, and adding the
> result to the value denoted by the right-hand box; thus, most-significant fields
> are on the left.
>
> (Except for the span of each field not necessarily being a power of two, this is
> conceptually identical to concatenation and cutting of
> bitfields.)
>
> The dotted segment indicates the least significant part of PARTID, which spans
> partid_per_closid PARTIDs and discriminates monitoring groups that use the
> same MPAM PMG value.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
>
> ---
>
> Changes since RFCv1:
>
> * [bugfix] Add CDP index properly in mpam_resctrl_partid_range().
>
> (Also reported by Shaopeng Tan.)
>
> * [relaxation] Remove requirement that mpam_pmg_max + 1 is a power
> of two.
>
> (This constraint is redundant, due to the use of multiply/add
> transformations of MPAM IDs in place of shift/OR.)
> ---
> drivers/platform/arm64/mpam/mpam_resctrl.c | 211
> +++++++++++----------
> 1 file changed, 112 insertions(+), 99 deletions(-)
>
> diff --git a/drivers/platform/arm64/mpam/mpam_resctrl.c
> b/drivers/platform/arm64/mpam/mpam_resctrl.c
> index aa8a6c077918..f834753a3274 100644
> --- a/drivers/platform/arm64/mpam/mpam_resctrl.c
> +++ b/drivers/platform/arm64/mpam/mpam_resctrl.c
> @@ -156,6 +156,18 @@ static bool mpam_resctrl_hide_cdp(enum
> resctrl_res_level rid)
> return cdp_enabled && !resctrl_arch_get_cdp_enabled(rid);
> }
>
> +static unsigned int partid_per_closid = 1;
> +
> +static unsigned int mpam_num_pmg(void)
> +{
> + return mpam_pmg_max + 1;
> +}
> +
> +static unsigned int mpam_num_rmid(void) {
> + return mpam_num_pmg() * partid_per_closid; }
> +
> /*
> * MSC may raise an error interrupt if it sees an out or range partid/pmg,
> * and go on to truncate the value. Regardless of what the hardware supports,
> @@ -163,35 +175,73 @@ static bool mpam_resctrl_hide_cdp(enum
> resctrl_res_level rid)
> */
> u32 resctrl_arch_get_num_closid(struct rdt_resource *ignored) {
> - return mpam_partid_max + 1;
> + u32 res = (mpam_partid_max + 1) / partid_per_closid;
> +
> + WARN_ON(res < 1);
> + return res;
> }
>
> -u32 resctrl_arch_system_num_rmid_idx(void)
> +static void mpam_resctrl_partid_range(u32 closid, enum resctrl_conf_type
> type,
> + const struct rdt_resource *r,
> + u16 *min_partid, u16 *max_partid)
> {
> - u8 closid_shift = fls(mpam_pmg_max);
> - u32 num_partid = resctrl_arch_get_num_closid(NULL);
> + u16 base_partid = closid;
> + u16 span = 1;
>
> - return num_partid << closid_shift;
> + if (cdp_enabled) {
> + base_partid *= 2;
> + if (mpam_resctrl_hide_cdp(r->rid) ||
> + type == CDP_NONE)
> + span *= 2;
> + else if (type == CDP_CODE)
> + base_partid++;
> + }
> +
> + *min_partid = base_partid * partid_per_closid;
> + if (max_partid)
> + *max_partid = *min_partid + (span * partid_per_closid - 1);
> }
>
> -u32 resctrl_arch_rmid_idx_encode(u32 closid, u32 rmid)
> +static void mpam_resctrl_hwid(u32 closid, u32 rmid,
> + u16 *partid_d, u16 *partid_i, u8 *pmg)
> {
> - u8 closid_shift = fls(mpam_pmg_max);
> + const u16 pmg_hi = rmid / mpam_num_pmg();
> + const u16 pmg_lo = rmid % mpam_num_pmg();
> + u16 base_partid = closid;
> + u16 partid_stride = 0;
> +
> + if (cdp_enabled) {
> + base_partid *= 2;
> + partid_stride = 1;
> + }
> +
> + partid_stride *= partid_per_closid;
> + base_partid *= partid_per_closid;
> + base_partid += pmg_hi;
>
> - BUG_ON(closid_shift > 8);
> + *partid_d = base_partid;
> + if (partid_i)
> + *partid_i = base_partid + partid_stride;
>
> - return (closid << closid_shift) | rmid;
> + *pmg = pmg_lo;
> }
>
> -void resctrl_arch_rmid_idx_decode(u32 idx, u32 *closid, u32 *rmid)
> +u32 resctrl_arch_rmid_idx_encode(u32 closid, u32 rmid) {
> + return closid * mpam_num_rmid() + rmid; }
> +
> +u32 resctrl_arch_system_num_rmid_idx(void)
> {
> - u8 closid_shift = fls(mpam_pmg_max);
> - u32 pmg_mask = ~(~0 << closid_shift);
> + u32 num_closid = resctrl_arch_get_num_closid(NULL);
>
> - BUG_ON(closid_shift > 8);
> + return resctrl_arch_rmid_idx_encode(num_closid, 0); }
>
> - *closid = idx >> closid_shift;
> - *rmid = idx & pmg_mask;
> +void resctrl_arch_rmid_idx_decode(u32 idx, u32 *closid, u32 *rmid) {
> + *closid = idx / mpam_num_rmid();
> + *rmid = idx % mpam_num_rmid();
> }
>
> void resctrl_arch_sched_in(struct task_struct *tsk) @@ -203,21 +253,14 @@
> void resctrl_arch_sched_in(struct task_struct *tsk)
>
> void resctrl_arch_set_cpu_default_closid_rmid(int cpu, u32 closid, u32 rmid)
> {
> + u16 partid_d, partid_i;
> + u8 pmg;
> +
> BUG_ON(closid > U16_MAX);
> BUG_ON(rmid > U8_MAX);
>
> - if (!cdp_enabled) {
> - mpam_set_cpu_defaults(cpu, closid, closid, rmid, rmid);
> - } else {
> - /*
> - * When CDP is enabled, resctrl halves the closid range and
> we
> - * use odd/even partid for one closid.
> - */
> - u32 partid_d = resctrl_get_config_index(closid, CDP_DATA);
> - u32 partid_i = resctrl_get_config_index(closid, CDP_CODE);
> -
> - mpam_set_cpu_defaults(cpu, partid_d, partid_i, rmid, rmid);
> - }
> + mpam_resctrl_hwid(closid, rmid, &partid_d, &partid_i, &pmg);
> + mpam_set_cpu_defaults(cpu, partid_d, partid_i, pmg, pmg);
> }
>
> void resctrl_arch_sync_cpu_closid_rmid(void *info) @@ -236,41 +279,38 @@
> void resctrl_arch_sync_cpu_closid_rmid(void *info)
>
> void resctrl_arch_set_closid_rmid(struct task_struct *tsk, u32 closid, u32 rmid)
> {
> + u16 partid_d, partid_i;
> + u8 pmg;
> +
> BUG_ON(closid > U16_MAX);
> BUG_ON(rmid > U8_MAX);
>
> - if (!cdp_enabled) {
> - mpam_set_task_partid_pmg(tsk, closid, closid, rmid, rmid);
> - } else {
> - u32 partid_d = resctrl_get_config_index(closid, CDP_DATA);
> - u32 partid_i = resctrl_get_config_index(closid, CDP_CODE);
> -
> - mpam_set_task_partid_pmg(tsk, partid_d, partid_i, rmid,
> rmid);
> - }
> + mpam_resctrl_hwid(closid, rmid, &partid_d, &partid_i, &pmg);
> + mpam_set_task_partid_pmg(tsk, partid_d, partid_i, pmg, pmg);
> }
>
> bool resctrl_arch_match_closid(struct task_struct *tsk, u32 closid) {
> + u16 min_partid, max_partid;
> u64 regval = mpam_get_regval(tsk);
> u32 tsk_closid = FIELD_GET(MPAM1_EL1_PARTID_D, regval);
>
> - if (cdp_enabled)
> - tsk_closid >>= 1;
> -
> - return tsk_closid == closid;
> + mpam_resctrl_partid_range(closid, CDP_NONE, NULL,
> + &min_partid, &max_partid);
> + return tsk_closid >= min_partid && tsk_closid <= max_partid;
> }
>
> /* The task's pmg is not unique, the partid must be considered too */ bool
> resctrl_arch_match_rmid(struct task_struct *tsk, u32 closid, u32 rmid) {
> + u16 partid_d;
> + u8 pmg;
> u64 regval = mpam_get_regval(tsk);
> - u32 tsk_closid = FIELD_GET(MPAM1_EL1_PARTID_D, regval);
> - u32 tsk_rmid = FIELD_GET(MPAM1_EL1_PMG_D, regval);
> -
> - if (cdp_enabled)
> - tsk_closid >>= 1;
> + u16 tsk_partid_d = FIELD_GET(MPAM1_EL1_PARTID_D, regval);
> + u8 tsk_pmg = FIELD_GET(MPAM1_EL1_PMG_D, regval);
>
> - return (tsk_closid == closid) && (tsk_rmid == rmid);
> + mpam_resctrl_hwid(closid, rmid, &partid_d, NULL, &pmg);
> + return (tsk_partid_d == partid_d) && (tsk_pmg == pmg);
> }
>
> struct rdt_resource *resctrl_arch_get_resource(enum resctrl_res_level l)
> @@ -370,6 +410,7 @@ int resctrl_arch_rmid_read(struct rdt_resource *r,
> struct rdt_mon_domain *d,
> struct mpam_resctrl_dom *dom;
> u32 mon = *(u32 *)arch_mon_ctx;
> enum mpam_device_features type;
> + u16 partid_d, partid_i, pmg;
>
> resctrl_arch_rmid_read_context_check();
>
> @@ -391,27 +432,23 @@ int resctrl_arch_rmid_read(struct rdt_resource *r,
> struct rdt_mon_domain *d,
> if (cfg.mon == USE_RMID_IDX)
> cfg.mon = resctrl_arch_rmid_idx_encode(closid, rmid);
>
> + mpam_resctrl_hwid(closid, rmid, &partid_d, &partid_i, &cfg.pmg);
> cfg.match_pmg = true;
> - cfg.pmg = rmid;
> + cfg.pmg = pmg;
> cfg.opts = resctrl_evt_config_to_mpam(dom->mbm_local_evt_cfg);
>
> if (irqs_disabled()) {
> /* Check if we can access this domain without an IPI */
> err = -EIO;
> } else {
> - if (cdp_enabled) {
> - cfg.partid = closid << 1;
> - err = mpam_msmon_read(dom->comp, &cfg, type,
> val);
> - if (err)
> - return err;
> + cfg.partid = partid_d;
> + err = mpam_msmon_read(dom->comp, &cfg, type, val);
>
> - cfg.partid += 1;
> + if (partid_i != partid_d) {
> + cfg.partid = partid_i;
> err = mpam_msmon_read(dom->comp, &cfg, type,
> &cdp_val);
> if (!err)
> *val += cdp_val;
> - } else {
> - cfg.partid = closid;
> - err = mpam_msmon_read(dom->comp, &cfg, type,
> val);
> }
> }
>
> @@ -423,6 +460,7 @@ void resctrl_arch_reset_rmid(struct rdt_resource *r,
> struct rdt_mon_domain *d, {
> struct mon_cfg cfg;
> struct mpam_resctrl_dom *dom;
> + u16 partid_d, partid_i;
>
> if (eventid != QOS_L3_MBM_LOCAL_EVENT_ID)
> return;
> @@ -433,14 +471,13 @@ void resctrl_arch_reset_rmid(struct rdt_resource *r,
> struct rdt_mon_domain *d,
>
> dom = container_of(d, struct mpam_resctrl_dom, resctrl_mon_dom);
>
> - if (cdp_enabled) {
> - cfg.partid = closid << 1;
> - mpam_msmon_reset_mbwu(dom->comp, &cfg);
> + mpam_resctrl_hwid(closid, rmid, &partid_d, &partid_i, &cfg.pmg);
>
> - cfg.partid += 1;
> - mpam_msmon_reset_mbwu(dom->comp, &cfg);
> - } else {
> - cfg.partid = closid;
> + cfg.partid = partid_d;
> + mpam_msmon_reset_mbwu(dom->comp, &cfg);
> +
> + if (partid_i != partid_d) {
> + cfg.partid = partid_i;
> mpam_msmon_reset_mbwu(dom->comp, &cfg);
> }
> }
> @@ -1051,15 +1088,6 @@ int mpam_resctrl_setup(void)
> err = -EOPNOTSUPP;
>
> if (!err) {
> - if (!is_power_of_2(mpam_pmg_max + 1)) {
> - /*
> - * If not all the partid*pmg values are valid indexes,
> - * resctrl may allocate pmg that don't exist. This
> - * should cause an error interrupt.
> - */
> - pr_warn("Number of PMG is not a power of 2! resctrl
> may misbehave");
> - }
> -
> err = resctrl_init();
> if (!err)
> WRITE_ONCE(resctrl_enabled, true);
> @@ -1081,7 +1109,7 @@ static void mpam_resctrl_exit(void)
> u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_ctrl_domain *d,
> u32 closid, enum resctrl_conf_type type) {
> - u32 partid;
> + u16 partid;
> struct mpam_config *cfg;
> struct mpam_props *cprops;
> struct mpam_resctrl_res *res;
> @@ -1097,15 +1125,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r,
> struct rdt_ctrl_domain *d,
> dom = container_of(d, struct mpam_resctrl_dom, resctrl_ctrl_dom);
> cprops = &res->class->props;
>
> - /*
> - * When CDP is enabled, but the resource doesn't support it,
> - * the control is cloned across both partids.
> - * Pick one at random to read:
> - */
> - if (mpam_resctrl_hide_cdp(r->rid))
> - type = CDP_DATA;
> -
> - partid = resctrl_get_config_index(closid, type);
> + mpam_resctrl_partid_range(closid, type, r, &partid, NULL);
> cfg = &dom->comp->cfg[partid];
>
> switch (r->rid) {
> @@ -1126,7 +1146,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r,
> struct rdt_ctrl_domain *d,
> return -EINVAL;
> }
>
> - if (!r->alloc_capable || partid >= resctrl_arch_get_num_closid(r) ||
> + if (!r->alloc_capable || partid > mpam_partid_max ||
> !mpam_has_feature(configured_by, cfg))
> return resctrl_get_default_ctrl(r);
>
> @@ -1147,8 +1167,8 @@ u32 resctrl_arch_get_config(struct rdt_resource *r,
> struct rdt_ctrl_domain *d, int resctrl_arch_update_one(struct rdt_resource *r,
> struct rdt_ctrl_domain *d,
> u32 closid, enum resctrl_conf_type t, u32 cfg_val)
> {
> - int err;
> - u32 partid;
> + int err = 0;
> + u16 partid, min_partid, max_partid;
> struct mpam_config cfg;
> struct mpam_props *cprops;
> struct mpam_resctrl_res *res;
> @@ -1163,8 +1183,10 @@ int resctrl_arch_update_one(struct rdt_resource *r,
> struct rdt_ctrl_domain *d,
> dom = container_of(d, struct mpam_resctrl_dom, resctrl_ctrl_dom);
> cprops = &res->class->props;
>
> - partid = resctrl_get_config_index(closid, t);
> - if (!r->alloc_capable || partid >= resctrl_arch_get_num_closid(r))
> + mpam_resctrl_partid_range(closid, t, r, &min_partid, &max_partid);
> + if (!r->alloc_capable ||
> + min_partid > mpam_partid_max ||
> + max_partid > mpam_partid_max)
> return -EINVAL;
"min_partid > mpam_partid_max" seems unnesscessary.
Best regards,
Shaopeng TAN
> cfg.features = 0;
> @@ -1190,22 +1212,13 @@ int resctrl_arch_update_one(struct rdt_resource *r,
> struct rdt_ctrl_domain *d,
> return -EINVAL;
> }
>
> - /*
> - * When CDP is enabled, but the resource doesn't support it, we need
> to
> - * apply the same configuration to the other partid.
> - */
> - if (mpam_resctrl_hide_cdp(r->rid)) {
> - partid = resctrl_get_config_index(closid, CDP_CODE);
> + for (partid = min_partid; partid <= max_partid; partid++) {
> err = mpam_apply_config(dom->comp, partid, &cfg);
> if (err)
> - return err;
> -
> - partid = resctrl_get_config_index(closid, CDP_DATA);
> - return mpam_apply_config(dom->comp, partid, &cfg);
> -
> - } else {
> - return mpam_apply_config(dom->comp, partid, &cfg);
> + break;
> }
> +
> + return err;
> }
>
> /* TODO: this is IPI heavy */
> --
> 2.34.1
^ permalink raw reply [flat|nested] 26+ messages in thread
* RE: [RFC PATCH v2 07/11] arm_mpam: [NFU] Rework ID remapping to use a kernel command-line argument
2025-01-17 15:10 ` [RFC PATCH v2 07/11] arm_mpam: [NFU] Rework ID remapping to use a kernel command-line argument Dave Martin
@ 2025-01-29 10:57 ` Shaopeng Tan (Fujitsu)
0 siblings, 0 replies; 26+ messages in thread
From: Shaopeng Tan (Fujitsu) @ 2025-01-29 10:57 UTC (permalink / raw)
To: 'Dave Martin', linux-kernel
Cc: linux-arm-kernel, Zeng Heng, James Morse
Hello,
> Hacking the source is an acceptable interface for experimentation, but not very
> convenient.
>
> Instead, add a kernel command-line parameter mpam_partid_per_closid=<n>
> to specify how many PARTIDs are used to provide monitoring groups for each
> resctrl CLOSID.
>
> This change is not intended for upstream.
Are you looking for an alternative to using kernel command-line parameter?
Specifying the number of monitoring groups under one resource control group might make it easier for users to understand. For example, mongrps_per_ctrlgrp.
Best regards,
Shaopeng TAN
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> ---
> drivers/platform/arm64/mpam/mpam_resctrl.c | 22
> ++++++++++++++++++++++
> 1 file changed, 22 insertions(+)
>
> diff --git a/drivers/platform/arm64/mpam/mpam_resctrl.c
> b/drivers/platform/arm64/mpam/mpam_resctrl.c
> index f834753a3274..eb1ef5d2bc57 100644
> --- a/drivers/platform/arm64/mpam/mpam_resctrl.c
> +++ b/drivers/platform/arm64/mpam/mpam_resctrl.c
> @@ -3,6 +3,9 @@
>
> #define pr_fmt(fmt) "mpam: resctrl: " fmt
>
> +#undef KBUILD_MODNAME
> +#define KBUILD_MODNAME "mpam"
> +
> #include <linux/arm_mpam.h>
> #include <linux/cacheinfo.h>
> #include <linux/cpu.h>
> @@ -10,6 +13,7 @@
> #include <linux/errno.h>
> #include <linux/limits.h>
> #include <linux/list.h>
> +#include <linux/moduleparam.h>
> #include <linux/printk.h>
> #include <linux/rculist.h>
> #include <linux/resctrl.h>
> @@ -158,6 +162,24 @@ static bool mpam_resctrl_hide_cdp(enum
> resctrl_res_level rid)
>
> static unsigned int partid_per_closid = 1;
>
> +static int mpam_resctrl_partid_per_closid_set(const char *val,
> + const struct kernel_param *kp)
> {
> + /*
> + * 16 in an arbitrary maximum, sufficient for experimentation
> + * but not ridiculously large:
> + */
> + return param_set_uint_minmax(val, kp, 1, 16); }
> +
> +static const struct kernel_param_ops mpam_resctrl_partid_per_closid_ops
> = {
> + .set = mpam_resctrl_partid_per_closid_set,
> + .get = param_get_uint,
> +};
> +
> +device_param_cb(partid_per_closid, &mpam_resctrl_partid_per_closid_ops,
> + &partid_per_closid, 0444);
> +
> static unsigned int mpam_num_pmg(void)
> {
> return mpam_pmg_max + 1;
> --
> 2.34.1
^ permalink raw reply [flat|nested] 26+ messages in thread
end of thread, other threads:[~2025-01-29 10:58 UTC | newest]
Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2025-01-17 15:10 [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 01/11] arm_mpam: Fix read-back of cloned resource controls under CDP emulation Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 02/11] arm_mpam: Delete unused function resctrl_arch_set_rmid() Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 03/11] arm_mpam: Set INTERNAL as needed when setting MSC controls Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 04/11] fs/resctrl: Add arch hook to set up the default CLOSID Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 05/11] arm_mpam: resctrl: Implement " Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 06/11] arm_mpam: Introduce flexible CLOSID/RMID translation Dave Martin
2025-01-29 10:40 ` Shaopeng Tan (Fujitsu)
2025-01-17 15:10 ` [RFC PATCH v2 07/11] arm_mpam: [NFU] Rework ID remapping to use a kernel command-line argument Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 08/11] arm_mpam: [NFU] Development diagnostics for MPAM ID assignments Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 09/11] arm_mpam: Add control partition identifier to mpam_config Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 10/11] arm_mpam: Skip inapplicable cases when reprogramming a resctrl domain Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 11/11] arm_mpam: Detect whether the system can alias PARTIDs Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 00/11] Introduce PARTID grouping support Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 01/11] arm_mpam: Fix read-back of cloned resource controls under CDP emulation Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 02/11] arm_mpam: Delete unused function resctrl_arch_set_rmid() Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 03/11] arm_mpam: Set INTERNAL as needed when setting MSC controls Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 04/11] fs/resctrl: Add arch hook to set up the default CLOSID Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 05/11] arm_mpam: resctrl: Implement " Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 06/11] arm_mpam: Introduce flexible CLOSID/RMID translation Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 07/11] arm_mpam: [NFU] Rework ID remapping to use a kernel command-line argument Dave Martin
2025-01-29 10:57 ` Shaopeng Tan (Fujitsu)
2025-01-17 15:10 ` [RFC PATCH v2 08/11] arm_mpam: [NFU] Development diagnostics for MPAM ID assignments Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 09/11] arm_mpam: Add control partition identifier to mpam_config Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 10/11] arm_mpam: Skip inapplicable cases when reprogramming a resctrl domain Dave Martin
2025-01-17 15:10 ` [RFC PATCH v2 11/11] arm_mpam: Detect whether the system can alias PARTIDs Dave Martin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®