[RFC PATCH v2 00/11] x86/resctrl: Support asymmetric L2 CAT on hybrid CPUs

[RFC PATCH v2 00/11] x86/resctrl: Support asymmetric L2 CAT on hybrid CPUs

[Tony Luck]

Stil RFC as it builds on Reinette's multiple control patches[0].

Some hybrid CPUs (e.g. Panther Lake) have L2 caches whose cache
allocation capabilities differ between P-core and E-core domains.
Today these show up as symmetric L2 CAT, on just the P-core. New CPUID
leaves enumerate asymmetric allocation separately, including per-domain
capacity bitmask (CBM) lengths.

This series teaches resctrl to use that enumeration:

 - Define the four Intel RDT CPUID leaves, and add the asymmetric
   L2 CAT/CDP feature bits, CPUID dependencies and rdt= boot options
   (l2cat_asym, l2cdp_asym).
 - Disable symmetric L2 CAT on Panther Lake, where it is misreported.
 - Clean up rdt_get_cache_alloc_cfg() (failure handling, leaf argument)
   and the RDT allocation feature checks to prepare for a second leaf.
 - Move the CBM length from the shared control to each control domain
   (touches fs/resctrl, x86 and MPAM) since lengths can now differ.
 - Enumerate asymmetric L2 CAT/CDP from CPUID leaf 0x28 when no
   shareable bits are reported (resctrl cannot represent them per
   domain). info/L2/cbm_mask reports the minimum CBM across domains.
 - Change the "max" file to show per-domain values, matching the
   schemata format, so the UI needs no change for asymmetric controls.
 - Document the new options and the cbm_mask semantics.

Changes since first RFC[1]:

*) No new/renamed resource or control. Just keep this as "L2" for both.

*) Keep the info/L2/cbm_mask but make it display the minimum mask across
   all domains. Naive legacy s/w may use this and waste some cache.

*) Chen Yu pointed out that I don't need to search the resource for the
   control to update the domain. Drop the helper and just update the
   bitmask length in domain_add_cpu_ctrl()

*) Find per-domain cbm_len for L3 as well as L2 cache (though still only
   look for asymmetric L2

*) Intel L2 caches don't have any shareable bits. Check that the
   shareable mask is zero (as this series doesn't update file system
   code to handle asymmetric shareable bits).

[0] Link: git://git.kernel.org/pub/scm/linux/kernel/git/reinette/linux.git resctrl/controls_rfc_v2.4
[1] Link: https://lore.kernel.org/all/auDiLEujZ5BGCACj@agluck-desk3/

Tony Luck (11):
  fs/resctrl: Change UI of "max" file (merge back into earlier patches)
  x86/resctrl: Disable symmetric L2 CAT on Panther Lake
  x86/cpuid: Define Intel RDT enumeration leaves
  x86/resctrl: Add asymmetric L2 CAT feature bits and boot options
  x86/resctrl: Handle cache allocation setup failures
  x86/resctrl: Pass the CPUID leaf to rdt_get_cache_alloc_cfg()
  arm,x86,fs/resctrl: Store cache bitmask length per control domain
  x86/resctrl: Use cpu_feature_enabled() for RDT allocation
  x86/resctrl: Accept asymmetric RDT allocation features
  x86/resctrl: Enable asymmetric CAT from CPUID leaf 0x28
  fs/resctrl: Document asymmetric L2 allocation options

 .../admin-guide/kernel-parameters.txt         |   3 +-
 Documentation/filesystems/resctrl.rst         |   8 +-
 include/linux/resctrl.h                       |  12 +-
 arch/x86/include/asm/cpufeatures.h            |   3 +
 arch/x86/include/asm/cpuid/types.h            |   4 +
 arch/x86/kernel/cpu/cpuid-deps.c              |   2 +
 arch/x86/kernel/cpu/resctrl/core.c            | 138 +++++++++++++-----
 arch/x86/kernel/cpu/resctrl/rdtgroup.c        |   2 +-
 arch/x86/kernel/cpu/scattered.c               |   3 +
 drivers/resctrl/mpam_resctrl.c                |  14 +-
 fs/resctrl/ctrlmondata.c                      |   9 +-
 fs/resctrl/pseudo_lock.c                      |   2 +-
 fs/resctrl/rdtgroup.c                         |  56 ++++---
 13 files changed, 184 insertions(+), 72 deletions(-)

-- 
2.56.0

[RFC PATCH v2 01/11] fs/resctrl: Change UI of "max" file (merge back into earlier patches)

[Tony Luck]

Make the UI of the "max" file show values per-domain so no changes
in UI are needed when support for asymmetric controls (L2 on hybrid
CPUs) is added later.

Format chosen to match schemata file.

Signed-off-by: Tony Luck <tony.luck@intel.com>
---
 Documentation/filesystems/resctrl.rst |  5 ++++-
 fs/resctrl/rdtgroup.c                 | 27 ++++++++++++++++++++-------
 2 files changed, 24 insertions(+), 8 deletions(-)

diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
index 264a44001003..524ad062a6d9 100644
--- a/Documentation/filesystems/resctrl.rst
+++ b/Documentation/filesystems/resctrl.rst
@@ -397,12 +397,15 @@ with individual files:
 		the schemata file.
 
 "max":
+	Maximum values per domain in the same format as the schemata file.
+	Format of values depends on the type of the control:
+
 	When "type" == "scalar":
 		The maximum value that can be written to the control when writing
 		the schemata file.
 
 	When "type" == "bitmap":
-		The supported bitmap (in hexadecimal with 0x prefix) with
+		The supported bitmap (in hexadecimal without 0x prefix) with
 		all bits set.
 
 "tolerance":
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index feaaec18b835..57c050b05798 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -2465,15 +2465,28 @@ static int resctrl_ctrl_max_show(struct kernfs_open_file *of,
 				 struct seq_file *seq, void *v)
 {
 	struct resctrl_ctrl *ctrl = rdt_kn_parent_priv(of->kn);
+	struct rdt_ctrl_domain *d;
+	bool sep = false;
 
-	switch (ctrl->type) {
-	case RESCTRL_CTRL_SCALAR:
-		seq_printf(seq, "%u\n", ctrl->scalar.max);
-		break;
-	case RESCTRL_CTRL_BITMAP:
-		seq_printf(seq, "0x%lx\n", BIT_MASK(ctrl->bitmap.cbm_len) - 1);
-		break;
+	if (!info_kn_lock(of->kn))
+		return -ENOENT;
+
+	list_for_each_entry_rcu(d, &ctrl->domains, hdr.list, lockdep_is_cpus_held()) {
+		if (sep)
+			seq_putc(seq, ';');
+		switch (ctrl->type) {
+		case RESCTRL_CTRL_SCALAR:
+			seq_printf(seq, "%d=%u", d->hdr.id, ctrl->scalar.max);
+			break;
+		case RESCTRL_CTRL_BITMAP:
+			seq_printf(seq, "%d=%lx", d->hdr.id, BIT_MASK(ctrl->bitmap.cbm_len) - 1);
+			break;
+		}
+		sep = true;
 	}
+	seq_putc(seq, '\n');
+
+	info_kn_unlock(of->kn);
 
 	return 0;
 }
-- 
2.56.0

[RFC PATCH v2 02/11] x86/resctrl: Disable symmetric L2 CAT on Panther Lake

[Tony Luck]

Panther Lake reports L2 CAT in the symmetric CPUID leaf even though the
feature differs between P-cores and E-cores.

Disable the symmetric l2cat option for Panther Lake so resctrl does not
enumerate this asymmetric feature as symmetric.

Signed-off-by: Tony Luck <tony.luck@intel.com>
---
 arch/x86/kernel/cpu/resctrl/core.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index 7951cfd20da4..709075ffc706 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -1128,6 +1128,9 @@ static __init void __check_quirks_intel(void)
 	case INTEL_BROADWELL_X:
 		intel_rdt_mbm_apply_quirk();
 		break;
+	case INTEL_PANTHERLAKE_L:
+		set_rdt_options("!l2cat");
+		break;
 	}
 }
 
-- 
2.56.0

[RFC PATCH v2 03/11] x86/cpuid: Define Intel RDT enumeration leaves

[Tony Luck]

Intel RDT monitoring and allocation use separate symmetric and asymmetric
CPUID leaves. The symmetric leaf numbers are also hard-coded in resctrl.

Define constants for all four leaves and use the symmetric monitoring and
allocation constants instead of numeric leaf values.

Signed-off-by: Tony Luck <tony.luck@intel.com>
---
 arch/x86/include/asm/cpuid/types.h | 4 ++++
 arch/x86/kernel/cpu/resctrl/core.c | 8 ++++----
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/arch/x86/include/asm/cpuid/types.h b/arch/x86/include/asm/cpuid/types.h
index 3d0e611c97ba..9594965a54e7 100644
--- a/arch/x86/include/asm/cpuid/types.h
+++ b/arch/x86/include/asm/cpuid/types.h
@@ -28,9 +28,13 @@ enum cpuid_regs_idx {
 #define CPUID_LEAF_MWAIT	0x05
 #define CPUID_LEAF_DCA		0x09
 #define CPUID_LEAF_XSTATE	0x0d
+#define CPUID_LEAF_RDT_M	0x0f
+#define CPUID_LEAF_RDT_A	0x10
 #define CPUID_LEAF_TSC		0x15
 #define CPUID_LEAF_FREQ		0x16
 #define CPUID_LEAF_TILE		0x1d
+#define CPUID_LEAF_RDT_M_ASYM	0x27
+#define CPUID_LEAF_RDT_A_ASYM	0x28
 
 #define CPUID_RANGE(idx)	((idx) & 0xffff0000)
 #define CPUID_RANGE_MAX(idx)	(CPUID_RANGE(idx) + 0xffff)
diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index 709075ffc706..c66ff4cdd84b 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -235,7 +235,7 @@ static __init bool __get_mem_config_intel(struct rdt_resource *r)
 	union cpuid_0x10_x_edx edx;
 	u32 ebx, ecx, max_delay;
 
-	cpuid_count(0x00000010, 3, &eax.full, &ebx, &ecx, &edx.full);
+	cpuid_count(CPUID_LEAF_RDT_A, 3, &eax.full, &ebx, &ecx, &edx.full);
 	hw_res->num_closid = edx.split.cos_max + 1;
 	max_delay = eax.split.max_delay + 1;
 
@@ -357,7 +357,7 @@ static void rdt_get_cache_alloc_cfg(int idx, struct rdt_resource *r)
 	union cpuid_0x10_x_edx edx;
 	u32 ebx, default_ctrl;
 
-	cpuid_count(0x00000010, idx, &eax.full, &ebx, &ecx.full, &edx.full);
+	cpuid_count(CPUID_LEAF_RDT_A, idx, &eax.full, &ebx, &ecx.full, &edx.full);
 	hw_res->num_closid = edx.split.cos_max + 1;
 	hw_ctrl = kzalloc_obj(*hw_ctrl);
 	if (!hw_ctrl)
@@ -1196,7 +1196,7 @@ void resctrl_cpu_detect(struct cpuinfo_x86 *c)
 	}
 
 	/* will be overridden if occupancy monitoring exists */
-	c->x86_cache_max_rmid = cpuid_ebx(0xf);
+	c->x86_cache_max_rmid = cpuid_ebx(CPUID_LEAF_RDT_M);
 
 	if (cpu_has(c, X86_FEATURE_CQM_OCCUP_LLC) ||
 	    cpu_has(c, X86_FEATURE_CQM_MBM_TOTAL) ||
@@ -1205,7 +1205,7 @@ void resctrl_cpu_detect(struct cpuinfo_x86 *c)
 		u32 eax, ebx, ecx, edx;
 
 		/* QoS sub-leaf, EAX=0Fh, ECX=1 */
-		cpuid_count(0xf, 1, &eax, &ebx, &ecx, &edx);
+		cpuid_count(CPUID_LEAF_RDT_M, 1, &eax, &ebx, &ecx, &edx);
 
 		c->x86_cache_max_rmid  = ecx;
 		c->x86_cache_occ_scale = ebx;
-- 
2.56.0

[RFC PATCH v2 04/11] x86/resctrl: Add asymmetric L2 CAT feature bits and boot options

[Tony Luck]

CPUID enumerates asymmetric allocation support separately from symmetric
RDT, with L2 CAT and CDP capabilities reported through the asymmetric leaves.

Define and detect the asymmetric feature bits, add their CPUID dependencies,
and provide rdt= options to control asymmetric L2 CAT and CDP.

Signed-off-by: Tony Luck <tony.luck@intel.com>
---
 arch/x86/include/asm/cpufeatures.h |  3 +++
 arch/x86/kernel/cpu/cpuid-deps.c   |  2 ++
 arch/x86/kernel/cpu/resctrl/core.c | 28 ++++++++++++++++------------
 arch/x86/kernel/cpu/scattered.c    |  3 +++
 4 files changed, 24 insertions(+), 12 deletions(-)

diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index 61930f0d4578..8f8296eb7d83 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -432,6 +432,9 @@
 #define X86_FEATURE_CPPC_PERF_PRIO	(17*32+ 2) /* CPPC Floor Perf support */
 #define X86_FEATURE_SMCA		(17*32+ 3) /* "smca" Scalable MCA */
 #define X86_FEATURE_BTB_CTX_ISOLATION	(17*32+ 4) /* AMD: Branch predictions contexts isolated */
+#define X86_FEATURE_RDT_A_ASYM		(17*32+ 5) /* Asymmetric allocation features */
+#define X86_FEATURE_CAT_L2_ASYM		(17*32+ 6) /* Asymmetric Cache Allocation Technology L2 */
+#define X86_FEATURE_CDP_L2_ASYM		(17*32+ 7) /* Asymmetric Code and Data Prioritization L2 */
 
 /* Intel-defined CPU features, CPUID level 0x00000007:0 (EDX), word 18 */
 #define X86_FEATURE_AVX512_4VNNIW	(18*32+ 2) /* "avx512_4vnniw" AVX-512 Neural Network Instructions */
diff --git a/arch/x86/kernel/cpu/cpuid-deps.c b/arch/x86/kernel/cpu/cpuid-deps.c
index 99801e844b30..aeed13296233 100644
--- a/arch/x86/kernel/cpu/cpuid-deps.c
+++ b/arch/x86/kernel/cpu/cpuid-deps.c
@@ -73,6 +73,8 @@ static const struct cpuid_dep cpuid_deps[] = {
 	{ X86_FEATURE_BMEC,			X86_FEATURE_CQM_MBM_TOTAL   },
 	{ X86_FEATURE_BMEC,			X86_FEATURE_CQM_MBM_LOCAL   },
 	{ X86_FEATURE_SDCIAE,			X86_FEATURE_CAT_L3    },
+	{ X86_FEATURE_CAT_L2_ASYM,		X86_FEATURE_RDT_A_ASYM },
+	{ X86_FEATURE_CDP_L2_ASYM,		X86_FEATURE_RDT_A_ASYM },
 	{ X86_FEATURE_AVX512_BF16,		X86_FEATURE_AVX512VL  },
 	{ X86_FEATURE_AVX512_FP16,		X86_FEATURE_AVX512BW  },
 	{ X86_FEATURE_ENQCMD,			X86_FEATURE_XSAVES    },
diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index c66ff4cdd84b..0b1de97af17d 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -921,6 +921,8 @@ enum {
 	RDT_FLAG_L3_CDP,
 	RDT_FLAG_L2_CAT,
 	RDT_FLAG_L2_CDP,
+	RDT_FLAG_L2_CAT_ASYM,
+	RDT_FLAG_L2_CDP_ASYM,
 	RDT_FLAG_MBA,
 	RDT_FLAG_SMBA,
 	RDT_FLAG_BMEC,
@@ -941,18 +943,20 @@ struct rdt_options {
 };
 
 static struct rdt_options rdt_options[]  __ro_after_init = {
-	RDT_OPT(RDT_FLAG_CMT,	    "cmt",	X86_FEATURE_CQM_OCCUP_LLC),
-	RDT_OPT(RDT_FLAG_MBM_TOTAL, "mbmtotal", X86_FEATURE_CQM_MBM_TOTAL),
-	RDT_OPT(RDT_FLAG_MBM_LOCAL, "mbmlocal", X86_FEATURE_CQM_MBM_LOCAL),
-	RDT_OPT(RDT_FLAG_L3_CAT,    "l3cat",	X86_FEATURE_CAT_L3),
-	RDT_OPT(RDT_FLAG_L3_CDP,    "l3cdp",	X86_FEATURE_CDP_L3),
-	RDT_OPT(RDT_FLAG_L2_CAT,    "l2cat",	X86_FEATURE_CAT_L2),
-	RDT_OPT(RDT_FLAG_L2_CDP,    "l2cdp",	X86_FEATURE_CDP_L2),
-	RDT_OPT(RDT_FLAG_MBA,	    "mba",	X86_FEATURE_MBA),
-	RDT_OPT(RDT_FLAG_SMBA,	    "smba",	X86_FEATURE_SMBA),
-	RDT_OPT(RDT_FLAG_BMEC,	    "bmec",	X86_FEATURE_BMEC),
-	RDT_OPT(RDT_FLAG_ABMC,	    "abmc",	X86_FEATURE_ABMC),
-	RDT_OPT(RDT_FLAG_SDCIAE,    "sdciae",	X86_FEATURE_SDCIAE),
+	RDT_OPT(RDT_FLAG_CMT,		"cmt",		X86_FEATURE_CQM_OCCUP_LLC),
+	RDT_OPT(RDT_FLAG_MBM_TOTAL,	"mbmtotal",	X86_FEATURE_CQM_MBM_TOTAL),
+	RDT_OPT(RDT_FLAG_MBM_LOCAL,	"mbmlocal",	X86_FEATURE_CQM_MBM_LOCAL),
+	RDT_OPT(RDT_FLAG_L3_CAT,	"l3cat",	X86_FEATURE_CAT_L3),
+	RDT_OPT(RDT_FLAG_L3_CDP,	"l3cdp",	X86_FEATURE_CDP_L3),
+	RDT_OPT(RDT_FLAG_L2_CAT,	"l2cat",	X86_FEATURE_CAT_L2),
+	RDT_OPT(RDT_FLAG_L2_CDP,	"l2cdp",	X86_FEATURE_CDP_L2),
+	RDT_OPT(RDT_FLAG_L2_CAT_ASYM,	"l2cat_asym",	X86_FEATURE_CAT_L2_ASYM),
+	RDT_OPT(RDT_FLAG_L2_CDP_ASYM,	"l2cdp_asym",	X86_FEATURE_CDP_L2_ASYM),
+	RDT_OPT(RDT_FLAG_MBA,		"mba",		X86_FEATURE_MBA),
+	RDT_OPT(RDT_FLAG_SMBA,		"smba",		X86_FEATURE_SMBA),
+	RDT_OPT(RDT_FLAG_BMEC,		"bmec",		X86_FEATURE_BMEC),
+	RDT_OPT(RDT_FLAG_ABMC,		"abmc",		X86_FEATURE_ABMC),
+	RDT_OPT(RDT_FLAG_SDCIAE,	"sdciae",	X86_FEATURE_SDCIAE),
 };
 #define NUM_RDT_OPTIONS ARRAY_SIZE(rdt_options)
 
diff --git a/arch/x86/kernel/cpu/scattered.c b/arch/x86/kernel/cpu/scattered.c
index d867ab90d42a..3db914c170f9 100644
--- a/arch/x86/kernel/cpu/scattered.c
+++ b/arch/x86/kernel/cpu/scattered.c
@@ -28,6 +28,7 @@ static const struct cpuid_bit cpuid_bits[] = {
 	{ X86_FEATURE_APERFMPERF,		CPUID_ECX,  0, 0x00000006, 0 },
 	{ X86_FEATURE_EPB,			CPUID_ECX,  3, 0x00000006, 0 },
 	{ X86_FEATURE_INTEL_PPIN,		CPUID_EBX,  0, 0x00000007, 1 },
+	{ X86_FEATURE_RDT_A_ASYM,		CPUID_ECX,  1, 0x00000007, 1 },
 	{ X86_FEATURE_MSR_IMM,			CPUID_ECX,  5, 0x00000007, 1 },
 	{ X86_FEATURE_APX,			CPUID_EDX, 21, 0x00000007, 1 },
 	{ X86_FEATURE_RRSBA_CTRL,		CPUID_EDX,  2, 0x00000007, 2 },
@@ -46,6 +47,8 @@ static const struct cpuid_bit cpuid_bits[] = {
 	{ X86_FEATURE_SGX2,			CPUID_EAX,  1, 0x00000012, 0 },
 	{ X86_FEATURE_SGX_EUPDATESVN,		CPUID_EAX, 10, 0x00000012, 0 },
 	{ X86_FEATURE_SGX_EDECCSSA,		CPUID_EAX, 11, 0x00000012, 0 },
+	{ X86_FEATURE_CAT_L2_ASYM,		CPUID_EBX,  2, 0x00000028, 0 },
+	{ X86_FEATURE_CDP_L2_ASYM,		CPUID_ECX,  2, 0x00000028, 2 },
 	{ X86_FEATURE_OVERFLOW_RECOV,		CPUID_EBX,  0, 0x80000007, 0 },
 	{ X86_FEATURE_SUCCOR,			CPUID_EBX,  1, 0x80000007, 0 },
 	{ X86_FEATURE_SMCA,			CPUID_EBX,  3, 0x80000007, 0 },
-- 
2.56.0

[RFC PATCH v2 05/11] x86/resctrl: Handle cache allocation setup failures

[Tony Luck]

rdt_get_cache_alloc_cfg() can fail when allocation of its control structure
fails or the CPU vendor is unsupported. The caller currently marks the
resource allocation-capable even when no control was created.

Return the control from rdt_get_cache_alloc_cfg() and mark the resource
allocation-capable only after successful setup. Free the allocated control
when the vendor is unsupported.

Signed-off-by: Tony Luck <tony.luck@intel.com>
---
 arch/x86/kernel/cpu/resctrl/core.c | 42 ++++++++++++++++++------------
 1 file changed, 25 insertions(+), 17 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index 0b1de97af17d..6df8e3c03289 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -348,7 +348,7 @@ static __init bool __rdt_get_mem_config_amd(struct rdt_resource *r)
 	return true;
 }
 
-static void rdt_get_cache_alloc_cfg(int idx, struct rdt_resource *r)
+static struct resctrl_ctrl *rdt_get_cache_alloc_cfg(int idx, struct rdt_resource *r)
 {
 	struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r);
 	struct resctrl_hw_ctrl *hw_ctrl;
@@ -361,7 +361,7 @@ static void rdt_get_cache_alloc_cfg(int idx, struct rdt_resource *r)
 	hw_res->num_closid = edx.split.cos_max + 1;
 	hw_ctrl = kzalloc_obj(*hw_ctrl);
 	if (!hw_ctrl)
-		return;
+		return NULL;
 
 	hw_ctrl->r_ctrl.type = RESCTRL_CTRL_BITMAP;
 	hw_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF;
@@ -380,15 +380,14 @@ static void rdt_get_cache_alloc_cfg(int idx, struct rdt_resource *r)
 		__set_bit(RESCTRL_BITMAP_FLAG_SPARSE, hw_ctrl->r_ctrl.bitmap.flags);
 		hw_ctrl->r_ctrl.bitmap.min_cbm_bits = 0;
 	} else {
-		return;
+		kfree(hw_ctrl);
+		return NULL;
 	}
 
 	hw_ctrl->msr_base = idx == 1 ? MSR_IA32_L3_CBM_BASE: MSR_IA32_L2_CBM_BASE;
 	hw_ctrl->msr_update = cat_wrmsr;
 
-	list_add(&hw_ctrl->r_ctrl.entry, &r->controls);
-
-	r->alloc_capable = true;
+	return &hw_ctrl->r_ctrl;
 }
 
 static void rdt_get_cdp_config(int level)
@@ -1054,6 +1053,7 @@ static __init bool get_slow_mem_config(void)
 
 static __init bool get_rdt_alloc_resources(void)
 {
+	struct resctrl_ctrl *ctrl;
 	struct rdt_resource *r;
 	bool ret = false;
 
@@ -1065,20 +1065,28 @@ static __init bool get_rdt_alloc_resources(void)
 
 	if (rdt_cpu_has(X86_FEATURE_CAT_L3)) {
 		r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
-		rdt_get_cache_alloc_cfg(1, r);
-		if (rdt_cpu_has(X86_FEATURE_CDP_L3))
-			rdt_get_cdp_l3_config();
-		if (rdt_cpu_has(X86_FEATURE_SDCIAE))
-			rdt_set_io_alloc_capable(r);
-		ret = true;
+		ctrl = rdt_get_cache_alloc_cfg(1, r);
+		if (ctrl) {
+			if (rdt_cpu_has(X86_FEATURE_CDP_L3))
+				rdt_get_cdp_l3_config();
+			if (rdt_cpu_has(X86_FEATURE_SDCIAE))
+				rdt_set_io_alloc_capable(r);
+			list_add(&ctrl->entry, &r->controls);
+			r->alloc_capable = true;
+			ret = true;
+		}
 	}
 	if (rdt_cpu_has(X86_FEATURE_CAT_L2)) {
-		/* CPUID 0x10.2 fields are same format at 0x10.1 */
+		/* CPUID 0x10.2 fields are same format as 0x10.1 */
 		r = &rdt_resources_all[RDT_RESOURCE_L2].r_resctrl;
-		rdt_get_cache_alloc_cfg(2, r);
-		if (rdt_cpu_has(X86_FEATURE_CDP_L2))
-			rdt_get_cdp_l2_config();
-		ret = true;
+		ctrl = rdt_get_cache_alloc_cfg(2, r);
+		if (ctrl) {
+			if (rdt_cpu_has(X86_FEATURE_CDP_L2))
+				rdt_get_cdp_l2_config();
+			list_add(&ctrl->entry, &r->controls);
+			r->alloc_capable = true;
+			ret = true;
+		}
 	}
 
 	if (get_mem_config())
-- 
2.56.0

[RFC PATCH v2 06/11] x86/resctrl: Pass the CPUID leaf to rdt_get_cache_alloc_cfg()

[Tony Luck]

Symmetric and asymmetric cache allocation parameters use the same CPUID
subleaf format but different leaves.

Pass the leaf number to rdt_get_cache_alloc_cfg() so callers can select the
enumeration leaf. This prepares the helper for asymmetric allocation without
changing current behavior.

Signed-off-by: Tony Luck <tony.luck@intel.com>
---
 arch/x86/kernel/cpu/resctrl/core.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index 6df8e3c03289..7981ddbde8c9 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -348,7 +348,7 @@ static __init bool __rdt_get_mem_config_amd(struct rdt_resource *r)
 	return true;
 }
 
-static struct resctrl_ctrl *rdt_get_cache_alloc_cfg(int idx, struct rdt_resource *r)
+static struct resctrl_ctrl *rdt_get_cache_alloc_cfg(int leaf, int idx, struct rdt_resource *r)
 {
 	struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r);
 	struct resctrl_hw_ctrl *hw_ctrl;
@@ -357,7 +357,7 @@ static struct resctrl_ctrl *rdt_get_cache_alloc_cfg(int idx, struct rdt_resource
 	union cpuid_0x10_x_edx edx;
 	u32 ebx, default_ctrl;
 
-	cpuid_count(CPUID_LEAF_RDT_A, idx, &eax.full, &ebx, &ecx.full, &edx.full);
+	cpuid_count(leaf, idx, &eax.full, &ebx, &ecx.full, &edx.full);
 	hw_res->num_closid = edx.split.cos_max + 1;
 	hw_ctrl = kzalloc_obj(*hw_ctrl);
 	if (!hw_ctrl)
@@ -1065,7 +1065,7 @@ static __init bool get_rdt_alloc_resources(void)
 
 	if (rdt_cpu_has(X86_FEATURE_CAT_L3)) {
 		r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
-		ctrl = rdt_get_cache_alloc_cfg(1, r);
+		ctrl = rdt_get_cache_alloc_cfg(CPUID_LEAF_RDT_A, 1, r);
 		if (ctrl) {
 			if (rdt_cpu_has(X86_FEATURE_CDP_L3))
 				rdt_get_cdp_l3_config();
@@ -1079,7 +1079,7 @@ static __init bool get_rdt_alloc_resources(void)
 	if (rdt_cpu_has(X86_FEATURE_CAT_L2)) {
 		/* CPUID 0x10.2 fields are same format as 0x10.1 */
 		r = &rdt_resources_all[RDT_RESOURCE_L2].r_resctrl;
-		ctrl = rdt_get_cache_alloc_cfg(2, r);
+		ctrl = rdt_get_cache_alloc_cfg(CPUID_LEAF_RDT_A, 2, r);
 		if (ctrl) {
 			if (rdt_cpu_has(X86_FEATURE_CDP_L2))
 				rdt_get_cdp_l2_config();
-- 
2.56.0

[RFC PATCH v2 07/11] arm,x86,fs/resctrl: Store cache bitmask length per control domain

[Tony Luck]

Cache allocation bitmask properties currently live on the shared control,
which assumes that every domain has the same CBM length. Asymmetric cache
domains can have different lengths.

Store CBM length on each control domain and use that value when initializing
default controls, validating and displaying masks, and calculating cache
allocation sizes. Initialize the per-domain value in the x86 and MPAM
implementations.

Signed-off-by: Tony Luck <tony.luck@intel.com>
---
 include/linux/resctrl.h                | 10 +++++++--
 arch/x86/kernel/cpu/resctrl/core.c     |  9 +++++---
 arch/x86/kernel/cpu/resctrl/rdtgroup.c |  2 +-
 drivers/resctrl/mpam_resctrl.c         | 14 +++++++-----
 fs/resctrl/ctrlmondata.c               |  9 ++++----
 fs/resctrl/pseudo_lock.c               |  2 +-
 fs/resctrl/rdtgroup.c                  | 31 ++++++++++++++------------
 7 files changed, 47 insertions(+), 30 deletions(-)

diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index f58a9e773ff6..8ac799d1feff 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -184,12 +184,15 @@ static inline bool domain_header_is_valid(struct rdt_domain_hdr *hdr,
  * @mbps_val:		When mba_sc is enabled, this holds the array of user
  *			specified control values for mba_sc in MBps, indexed
  *			by closid
+ * @cbm_len:		Length of cache bitmask for a cache control in this
+ *			domain
  */
 struct rdt_ctrl_domain {
 	struct rdt_domain_hdr		hdr;
 	struct pseudo_lock_region	*plr;
 	struct resctrl_staged_config	staged_config[CDP_NUM_TYPES];
 	u32				*mbps_val;
+	unsigned int			cbm_len;
 };
 
 /**
@@ -597,8 +600,11 @@ void resctrl_arch_sync_cpu_closid_rmid(void *info);
  * resctrl_get_default_ctrlval() - Return the default control value for this
  *                                 control.
  * @ctrl:	The control whose default control type is queried.
+ * @d:		The control domain to query, or NULL if @ctrl's properties
+ *		should be used instead (e.g. when no domain exists yet).
  */
-static inline u32 resctrl_get_default_ctrlval(struct resctrl_ctrl *ctrl)
+static inline u32 resctrl_get_default_ctrlval(struct resctrl_ctrl *ctrl,
+					      struct rdt_ctrl_domain *d)
 {
 	switch (ctrl->type) {
 	case RESCTRL_CTRL_BITMAP:
@@ -606,7 +612,7 @@ static inline u32 resctrl_get_default_ctrlval(struct resctrl_ctrl *ctrl)
 		 * Should map to 100% allocation as exposed to user space
 		 * via "cbm_mask".
 		 */
-		return BIT_MASK(ctrl->bitmap.cbm_len) - 1;
+		return BIT_MASK(d ? d->cbm_len : ctrl->bitmap.cbm_len) - 1;
 	case RESCTRL_CTRL_SCALAR:
 		return ctrl->scalar.reset_val;
 	}
diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index 7981ddbde8c9..cd3bcd71f475 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -484,14 +484,14 @@ void rdt_ctrl_update(void *arg)
 }
 
 static void setup_default_ctrlval(struct rdt_resource *r, struct resctrl_ctrl *ctrl,
-				  u32 *dc)
+				  u32 *dc, struct rdt_ctrl_domain *d)
 {
 	struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r);
 	int i;
 
 	/* Initialize the control MSRs to their reset values. */
 	for (i = 0; i < hw_res->num_closid; i++, dc++)
-		*dc = resctrl_get_default_ctrlval(ctrl);
+		*dc = resctrl_get_default_ctrlval(ctrl, d);
 }
 
 static void ctrl_domain_free(struct rdt_hw_ctrl_domain *hw_dom)
@@ -524,7 +524,7 @@ static int domain_setup_ctrlval(struct rdt_resource *r, struct resctrl_ctrl *ctr
 		return -ENOMEM;
 
 	hw_dom->ctrl_val = dc;
-	setup_default_ctrlval(r, ctrl, dc);
+	setup_default_ctrlval(r, ctrl, dc, d);
 
 	m.res = r;
 	m.ctrl = ctrl;
@@ -625,6 +625,9 @@ static void domain_add_cpu_ctrl(int cpu, struct rdt_resource *r,
 	d->hdr.rid = r->rid;
 	cpumask_set_cpu(cpu, &d->hdr.cpu_mask);
 
+	if (ctrl->type == RESCTRL_CTRL_BITMAP)
+		d->cbm_len = ctrl->bitmap.cbm_len;
+
 	rdt_domain_reconfigure_cdp(r);
 
 	if (domain_setup_ctrlval(r, ctrl, d)) {
diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
index edf42c59d7c1..691cd9fb1687 100644
--- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
+++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
@@ -257,7 +257,7 @@ void resctrl_arch_reset_all_ctrls(struct rdt_resource *r)
 			hw_dom = resctrl_to_arch_ctrl_dom(d);
 
 			for (i = 0; i < hw_res->num_closid; i++)
-				hw_dom->ctrl_val[i] = resctrl_get_default_ctrlval(ctrl);
+				hw_dom->ctrl_val[i] = resctrl_get_default_ctrlval(ctrl, d);
 			msr_param.dom = d;
 			smp_call_function_any(&d->hdr.cpu_mask, rdt_ctrl_update, &msr_param, 1);
 		}
diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c
index 12315cc7dec6..d2890b0d1cd8 100644
--- a/drivers/resctrl/mpam_resctrl.c
+++ b/drivers/resctrl/mpam_resctrl.c
@@ -1246,7 +1246,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res)
 		 * we have configured the SMMU and GIC not to do this 'all the
 		 * bits' is the correct answer here.
 		 */
-		mpam_ctrl->r_ctrl.bitmap.shareable_bits = resctrl_get_default_ctrlval(&mpam_ctrl->r_ctrl);
+		mpam_ctrl->r_ctrl.bitmap.shareable_bits = resctrl_get_default_ctrlval(&mpam_ctrl->r_ctrl, NULL);
 		list_add(&mpam_ctrl->r_ctrl.entry, &r->controls);
 		r->alloc_capable = true;
 		break;
@@ -1411,7 +1411,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct resctrl_ctrl *ctrl,
 	lockdep_assert_cpus_held();
 
 	if (!mpam_is_enabled())
-		return resctrl_get_default_ctrlval(ctrl);
+		return resctrl_get_default_ctrlval(ctrl, d);
 
 	res = container_of(r, struct mpam_resctrl_res, resctrl_res);
 	dom = container_of(d, struct mpam_resctrl_dom, resctrl_ctrl_dom);
@@ -1440,12 +1440,12 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct resctrl_ctrl *ctrl,
 		}
 		fallthrough;
 	default:
-		return resctrl_get_default_ctrlval(ctrl);
+		return resctrl_get_default_ctrlval(ctrl, d);
 	}
 
 	if (!r->alloc_capable || partid >= resctrl_arch_get_num_closid(r) ||
 	    !mpam_has_feature(configured_by, cfg))
-		return resctrl_get_default_ctrlval(ctrl);
+		return resctrl_get_default_ctrlval(ctrl, d);
 
 	switch (configured_by) {
 	case mpam_feat_cpor_part:
@@ -1453,7 +1453,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct resctrl_ctrl *ctrl,
 	case mpam_feat_mbw_max:
 		return mbw_max_to_percent(cfg->mbw_max, cprops);
 	default:
-		return resctrl_get_default_ctrlval(ctrl);
+		return resctrl_get_default_ctrlval(ctrl, d);
 	}
 }
 
@@ -1704,6 +1704,10 @@ mpam_resctrl_alloc_ctrl_domain(unsigned int cpu, struct mpam_resctrl_res *res,
 	dom->ctrl_comp = ctrl_comp;
 
 	ctrl_d = &dom->resctrl_ctrl_dom;
+
+	if (ctrl->type == RESCTRL_CTRL_BITMAP)
+		ctrl_d->cbm_len = ctrl->bitmap.cbm_len;
+
 	mpam_resctrl_domain_hdr_init(cpu, ctrl_comp, r->rid, &ctrl_d->hdr);
 	ctrl_d->hdr.type = RESCTRL_CTRL_DOMAIN;
 	err = resctrl_online_ctrl_domain(r, ctrl, ctrl_d);
diff --git a/fs/resctrl/ctrlmondata.c b/fs/resctrl/ctrlmondata.c
index 627998fed555..90274fdb0364 100644
--- a/fs/resctrl/ctrlmondata.c
+++ b/fs/resctrl/ctrlmondata.c
@@ -189,10 +189,11 @@ static int parse_bw(struct rdt_parse_data *data, struct rdt_resource_final *f,
  * requires at least two bits set.
  * AMD allows non-contiguous bitmasks.
  */
-static bool cbm_validate(char *buf, u32 *data, struct resctrl_ctrl *ctrl)
+static bool cbm_validate(char *buf, u32 *data, struct rdt_ctrl_domain *d,
+			 struct resctrl_ctrl *ctrl)
 {
-	u32 supported_bits = BIT_MASK(ctrl->bitmap.cbm_len) - 1;
-	unsigned int cbm_len = ctrl->bitmap.cbm_len;
+	u32 supported_bits = BIT_MASK(d->cbm_len) - 1;
+	unsigned int cbm_len = d->cbm_len;
 	unsigned long first_bit, zero_bit, val;
 	int ret;
 
@@ -255,7 +256,7 @@ static int parse_cbm(struct rdt_parse_data *data, struct rdt_resource_final *f,
 		return -EINVAL;
 	}
 
-	if (!cbm_validate(data->buf, &cbm_val, ctrl))
+	if (!cbm_validate(data->buf, &cbm_val, d, ctrl))
 		return -EINVAL;
 
 	if ((mode == RDT_MODE_EXCLUSIVE || mode == RDT_MODE_SHAREABLE) &&
diff --git a/fs/resctrl/pseudo_lock.c b/fs/resctrl/pseudo_lock.c
index 836a9b93a587..043a704aa8c9 100644
--- a/fs/resctrl/pseudo_lock.c
+++ b/fs/resctrl/pseudo_lock.c
@@ -637,7 +637,7 @@ bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_ctrl_domain *d, unsigned lon
 		return false;
 	}
 
-	cbm_len = ctrl->bitmap.cbm_len;
+	cbm_len = d->cbm_len;
 	cbm_b = d->plr->cbm;
 	if (bitmap_intersects(&cbm, &cbm_b, cbm_len))
 		return true;
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 57c050b05798..02636fee9dfd 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -1106,7 +1106,7 @@ static int rdt_default_ctrl_show(struct kernfs_open_file *of,
 	if (!ctrl)
 		goto out_unlock;
 
-	seq_printf(seq, "%x\n", resctrl_get_default_ctrlval(ctrl));
+	seq_printf(seq, "%x\n", resctrl_get_default_ctrlval(ctrl, NULL));
 
 out_unlock:
 	info_kn_unlock(of->kn);
@@ -1250,7 +1250,7 @@ static int rdt_bit_usage_show(struct kernfs_open_file *of,
 			hw_shareable |= ctrl_val;
 		}
 
-		for (i = ctrl->bitmap.cbm_len - 1; i >= 0; i--) {
+		for (i = dom->cbm_len - 1; i >= 0; i--) {
 			pseudo_locked = dom->plr ? dom->plr->cbm : 0;
 			hwb = test_bit(i, &hw_shareable);
 			swb = test_bit(i, &sw_shareable);
@@ -1546,7 +1546,7 @@ static bool __rdtgroup_cbm_overlaps(struct rdt_resource *r, struct rdt_ctrl_doma
 	/* Check for any overlap with regions used by hardware directly */
 	if (!exclusive) {
 		ctrl_b = ctrl->bitmap.shareable_bits;
-		if (bitmap_intersects(&cbm, &ctrl_b, ctrl->bitmap.cbm_len))
+		if (bitmap_intersects(&cbm, &ctrl_b, d->cbm_len))
 			return true;
 	}
 
@@ -1556,7 +1556,7 @@ static bool __rdtgroup_cbm_overlaps(struct rdt_resource *r, struct rdt_ctrl_doma
 		mode = rdtgroup_mode_by_closid(i);
 		if (closid_allocated(i) && i != closid &&
 		    mode != RDT_MODE_PSEUDO_LOCKSETUP) {
-			if (bitmap_intersects(&cbm, &ctrl_b, ctrl->bitmap.cbm_len)) {
+			if (bitmap_intersects(&cbm, &ctrl_b, d->cbm_len)) {
 				if (exclusive) {
 					if (mode == RDT_MODE_EXCLUSIVE)
 						return true;
@@ -1761,10 +1761,10 @@ unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r, struct resctrl_ctrl *c
 	if (WARN_ON_ONCE(r->ctrl_scope != RESCTRL_L2_CACHE && r->ctrl_scope != RESCTRL_L3_CACHE))
 		return size;
 
-	num_b = bitmap_weight(&cbm, ctrl->bitmap.cbm_len);
+	num_b = bitmap_weight(&cbm, d->cbm_len);
 	ci = get_cpu_cacheinfo_level(cpumask_any(&d->hdr.cpu_mask), r->ctrl_scope);
 	if (ci)
-		size = ci->size / ctrl->bitmap.cbm_len * num_b;
+		size = ci->size / d->cbm_len * num_b;
 
 	return size;
 }
@@ -2479,7 +2479,7 @@ static int resctrl_ctrl_max_show(struct kernfs_open_file *of,
 			seq_printf(seq, "%d=%u", d->hdr.id, ctrl->scalar.max);
 			break;
 		case RESCTRL_CTRL_BITMAP:
-			seq_printf(seq, "%d=%lx", d->hdr.id, BIT_MASK(ctrl->bitmap.cbm_len) - 1);
+			seq_printf(seq, "%d=%lx", d->hdr.id, BIT_MASK(d->cbm_len) - 1);
 			break;
 		}
 		sep = true;
@@ -4227,6 +4227,9 @@ static int mkdir_mondata_all(struct kernfs_node *parent_kn,
  * cbm_ensure_valid - Enforce validity on provided CBM
  * @_val:	Candidate CBM
  * @r:		RDT resource to which the CBM belongs
+ * @d:		RDT domain to which the CBM belongs. Its bitmap properties
+ *		(e.g. @cbm_len) are used since these may differ from
+ *		@ctrl's on asymmetric systems.
  * @ctrl:	Properties of the bitmap control CBM should be tested against.
  *
  * The provided CBM represents all cache portions available for use. This
@@ -4241,10 +4244,10 @@ static int mkdir_mondata_all(struct kernfs_node *parent_kn,
  *
  * Return: A CBM that is valid for resource @r.
  */
-static u32 cbm_ensure_valid(u32 _val, struct rdt_resource *r,
+static u32 cbm_ensure_valid(u32 _val, struct rdt_resource *r, struct rdt_ctrl_domain *d,
 			    struct resctrl_ctrl *ctrl)
 {
-	unsigned int cbm_len = ctrl->bitmap.cbm_len;
+	unsigned int cbm_len = d->cbm_len;
 	unsigned long first_bit, zero_bit;
 	unsigned long val;
 
@@ -4312,20 +4315,20 @@ static int __init_one_rdt_domain(struct rdt_ctrl_domain *d, struct rdt_resource_
 	}
 	if (d->plr && d->plr->cbm > 0)
 		used_b |= d->plr->cbm;
-	unused_b = used_b ^ (BIT_MASK(ctrl->bitmap.cbm_len) - 1);
-	unused_b &= BIT_MASK(ctrl->bitmap.cbm_len) - 1;
+	unused_b = used_b ^ (BIT_MASK(d->cbm_len) - 1);
+	unused_b &= BIT_MASK(d->cbm_len) - 1;
 	cfg->new_ctrl |= unused_b;
 	/*
 	 * Force the initial CBM to be valid, user can
 	 * modify the CBM based on system availability.
 	 */
-	cfg->new_ctrl = cbm_ensure_valid(cfg->new_ctrl, r, ctrl);
+	cfg->new_ctrl = cbm_ensure_valid(cfg->new_ctrl, r, d, ctrl);
 	/*
 	 * Assign the u32 CBM to an unsigned long to ensure that
 	 * bitmap_weight() does not access out-of-bound memory.
 	 */
 	tmp_cbm = cfg->new_ctrl;
-	if (bitmap_weight(&tmp_cbm, ctrl->bitmap.cbm_len) < ctrl->bitmap.min_cbm_bits) {
+	if (bitmap_weight(&tmp_cbm, d->cbm_len) < ctrl->bitmap.min_cbm_bits) {
 		rdt_last_cmd_printf("No space on %s:%d\n", f->name, d->hdr.id);
 		return -ENOSPC;
 	}
@@ -4373,7 +4376,7 @@ static void rdtgroup_init_mba(struct rdt_resource *r, struct resctrl_ctrl *ctrl,
 		}
 
 		cfg = &d->staged_config[CDP_NONE];
-		cfg->new_ctrl = resctrl_get_default_ctrlval(ctrl);
+		cfg->new_ctrl = resctrl_get_default_ctrlval(ctrl, d);
 		cfg->have_new_ctrl = true;
 	}
 }
-- 
2.56.0

[RFC PATCH v2 08/11] x86/resctrl: Use cpu_feature_enabled() for RDT allocation

[Tony Luck]

The RDT allocation probe uses boot_cpu_has() to test X86_FEATURE_RDT_A.

Use cpu_feature_enabled() for this feature check.

Signed-off-by: Tony Luck <tony.luck@intel.com>
---
 arch/x86/kernel/cpu/resctrl/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index cd3bcd71f475..197d50b795bc 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -1063,7 +1063,7 @@ static __init bool get_rdt_alloc_resources(void)
 	if (rdt_alloc_capable)
 		return true;
 
-	if (!boot_cpu_has(X86_FEATURE_RDT_A))
+	if (!cpu_feature_enabled(X86_FEATURE_RDT_A))
 		return false;
 
 	if (rdt_cpu_has(X86_FEATURE_CAT_L3)) {
-- 
2.56.0

[RFC PATCH v2 09/11] x86/resctrl: Accept asymmetric RDT allocation features

[Tony Luck]

Some systems expose asymmetric allocation features without setting the
X86_FEATURE_RDT_A bit for symmetric allocation.

Allow resctrl to probe allocation resources when either the symmetric or
asymmetric allocation feature is enabled.

Signed-off-by: Tony Luck <tony.luck@intel.com>
---
 arch/x86/kernel/cpu/resctrl/core.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index 197d50b795bc..aa0bb32d191e 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -1063,7 +1063,8 @@ static __init bool get_rdt_alloc_resources(void)
 	if (rdt_alloc_capable)
 		return true;
 
-	if (!cpu_feature_enabled(X86_FEATURE_RDT_A))
+	if (!cpu_feature_enabled(X86_FEATURE_RDT_A) &&
+	    !cpu_feature_enabled(X86_FEATURE_RDT_A_ASYM))
 		return false;
 
 	if (rdt_cpu_has(X86_FEATURE_CAT_L3)) {
-- 
2.56.0

[RFC PATCH v2 10/11] x86/resctrl: Enable asymmetric CAT from CPUID leaf 0x28

[Tony Luck]

Asymmetric CAT parameters are enumerated by CPUID leaf 0x28.
resctrl cannot represent per-domain shareable bits, so it cannot use this
enumeration when shareable bits are reported.

Enumerate the per-domain CBM lengths and enable asymmetric L2 CAT when the
feature is supported and has no shareable bits. Use the minimum domain CBM
length for the control's common view, and enable asymmetric CDP when available.

Signed-off-by: Tony Luck <tony.luck@intel.com>
---
 include/linux/resctrl.h            |  2 ++
 arch/x86/kernel/cpu/resctrl/core.c | 49 ++++++++++++++++++++++++++++--
 2 files changed, 49 insertions(+), 2 deletions(-)

diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index 8ac799d1feff..efaddd7c5270 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -420,6 +420,7 @@ enum resctrl_ctrl_name {
  *		Specifically, "rdt_resource_final::name"_"resctrl_ctrl::name".
  *		For example, with resource name "MB" and control name "MAX" the
  *		schema entry will be "MB_MAX".
+ * @asymmetric:	This control may have different per-domain parameters.
  * @bitmap:	Bitmap control properties. Used by cache allocation.
  * @scalar:	Scalar control properties. Valid when @type == RESCTRL_CTRL_SCALAR.
  *		Used by memory bandwidth allocation.
@@ -436,6 +437,7 @@ struct resctrl_ctrl {
 	struct list_head	domains;
 	enum resctrl_ctrl_type	type;
 	enum resctrl_ctrl_name	name;
+	bool			asymmetric;
 	union {
 		struct resctrl_ctrl_bitmap	bitmap;
 		struct resctrl_ctrl_scalar	scalar;
diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index aa0bb32d191e..93d1b153614f 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -348,6 +348,28 @@ static __init bool __rdt_get_mem_config_amd(struct rdt_resource *r)
 	return true;
 }
 
+static int rdt_get_cache_asym_cbm_len(enum resctrl_res_level rid)
+{
+	union cpuid_0x10_1_eax eax;
+	u32 ebx, ecx, edx;
+	int subleaf;
+
+	switch (rid) {
+	case RDT_RESOURCE_L3:
+		subleaf = 1;
+		break;
+	case RDT_RESOURCE_L2:
+		subleaf = 2;
+		break;
+	default:
+		pr_info("Unexpected resource %d\n", rid);
+		return 0;
+	}
+	cpuid_count(CPUID_LEAF_RDT_A_ASYM, subleaf, &eax.full, &ebx, &ecx, &edx);
+
+	return eax.split.cbm_len + 1;
+}
+
 static struct resctrl_ctrl *rdt_get_cache_alloc_cfg(int leaf, int idx, struct rdt_resource *r)
 {
 	struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r);
@@ -625,8 +647,14 @@ static void domain_add_cpu_ctrl(int cpu, struct rdt_resource *r,
 	d->hdr.rid = r->rid;
 	cpumask_set_cpu(cpu, &d->hdr.cpu_mask);
 
-	if (ctrl->type == RESCTRL_CTRL_BITMAP)
-		d->cbm_len = ctrl->bitmap.cbm_len;
+	if (ctrl->type == RESCTRL_CTRL_BITMAP) {
+		if (ctrl->asymmetric) {
+			d->cbm_len = rdt_get_cache_asym_cbm_len(r->rid);
+			ctrl->bitmap.cbm_len = min(ctrl->bitmap.cbm_len, d->cbm_len);
+		} else {
+			d->cbm_len = ctrl->bitmap.cbm_len;
+		}
+	}
 
 	rdt_domain_reconfigure_cdp(r);
 
@@ -1091,6 +1119,23 @@ static __init bool get_rdt_alloc_resources(void)
 			r->alloc_capable = true;
 			ret = true;
 		}
+	} else if (rdt_cpu_has(X86_FEATURE_CAT_L2_ASYM)) {
+		/* CPUID 0x28.2 fields are same format as 0x10.1 */
+		r = &rdt_resources_all[RDT_RESOURCE_L2].r_resctrl;
+		ctrl = rdt_get_cache_alloc_cfg(CPUID_LEAF_RDT_A_ASYM, 2, r);
+		if (ctrl) {
+			if (ctrl->bitmap.shareable_bits) {
+				pr_info("No support for asymmetric L2 with shareable bits\n");
+				kfree(container_of(ctrl, struct resctrl_hw_ctrl, r_ctrl));
+			} else {
+				if (rdt_cpu_has(X86_FEATURE_CDP_L2_ASYM))
+					rdt_get_cdp_l2_config();
+				list_add(&ctrl->entry, &r->controls);
+				r->alloc_capable = true;
+				ctrl->asymmetric = true;
+				ret = true;
+			}
+		}
 	}
 
 	if (get_mem_config())
-- 
2.56.0

[RFC PATCH v2 11/11] fs/resctrl: Document asymmetric L2 allocation options

[Tony Luck]

The rdt= kernel parameter accepts separate options for asymmetric L2 CAT and
CDP. On asymmetric systems, info/L2/cbm_mask represents the minimum cache
bitmask across domains.

Document the l2cat_asym and l2cdp_asym options and clarify the meaning of
cbm_mask.

Signed-off-by: Tony Luck <tony.luck@intel.com>
---
 Documentation/admin-guide/kernel-parameters.txt | 3 ++-
 Documentation/filesystems/resctrl.rst           | 3 +++
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 0b3c140babd9..a16f84bed8f9 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -6460,7 +6460,8 @@ Kernel parameters
 	rdt=		[HW,X86,RDT]
 			Turn on/off individual RDT features. List is:
 			cmt, mbmtotal, mbmlocal, l3cat, l3cdp, l2cat, l2cdp,
-			mba, smba, bmec, abmc, sdciae, energy[:guid],
+			l2cat_asym, l2cdp_asym, mba, smba, bmec, abmc, sdciae,
+			energy[:guid],
 			perf[:guid].
 			E.g. to turn on cmt and turn off mba use:
 				rdt=cmt,!mba
diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
index 524ad062a6d9..241461e0612b 100644
--- a/Documentation/filesystems/resctrl.rst
+++ b/Documentation/filesystems/resctrl.rst
@@ -91,6 +91,9 @@ related to allocation:
 "cbm_mask":
 		The bitmask which is valid for this resource.
 		This mask is equivalent to 100%.
+		On systems that support asymmetric, per-domain,
+		cache allocation bitmasks this is the minimum
+		mask across all domains.
 "min_cbm_bits":
 		The minimum number of consecutive bits which
 		must be set when writing a mask.
-- 
2.56.0