[PATCH v6 00/18] x86,fs/resctrl: Add kernel-mode (e.g., PLZA) support to the resctrl subsystem

[PATCH v6 00/18] x86,fs/resctrl: Add kernel-mode (e.g., PLZA) support to the resctrl subsystem

[Babu Moger]

Hi All,

This series adds support for AMD's Privilege-Level Zero Association
(PLZA) so kernel work can use a separate resource allocation and/or
monitoring association from the user task, and wires it up through a
small generic "kernel mode" (kmode) layer in fs/resctrl so future
architectures can plug in without touching (hopefully) core resctrl.

The features are documented in:

   AMD64 Zen6 Platform Quality of Service (PQOS) Extensions,
   Publication # 69193 Revision 1.00, Issue Date March 2026

available at https://bugzilla.kernel.org/show_bug.cgi?id=206537

The patches are based on top of commit (tip/master):

  8cf07e484290 Merge branch into tip/master: 'x86/tdx'

Any feedback is appreciated.

Background
==========

When memory bandwidth associated with a CLOSID is aggressively throttled,
and a task with that CLOSID moves into kernel mode, the kernel operations
are also aggressively throttled. This can stall forward progress and
eventually degrade overall system performance.

Privilege-Level Zero Association (PLZA) allows the user to specify a CLOSID
and/or RMID for execution at Privilege Level Zero. When PLZA is enabled on
a CPU, kernel work at PL0 uses the CLOSID and/or RMID from MSR
PQR_PLZA_ASSOC; otherwise, the CPU uses the CLOSID and RMID from PQR_ASSOC.

Design
======

A new sysfs file, info/kernel_mode, selects the kernel mode for resource
allocation and monitoring of kernel work and, when the mode needs one, the
resource group that backs it. Reads list the supported modes and the
currently active association; writes select the mode or the group. Look
at the thread below for design discussion.
https://lore.kernel.org/lkml/14a8ad0a-e842-4268-871a-0762f1169e03@intel.com/
https://lore.kernel.org/lkml/e4c95002-ae8a-48d0-bedd-772db58b937e@intel.com/

Two kernel modes are exposed:

- inherit_user: kernel work uses the allocation and monitoring
  associations of the user-space task on whose behalf it runs.

- global_enable_per_cpu: kernel work uses the allocation and/or
  monitoring association of a backing CTRL_MON or MON group. ctrl= and
  mon= select assign or inherit for each, and group= names the group
  with <CTRL_MON>/<MON>/ path syntax. An omitted parameter keeps the
  value the file currently shows. A monitor group requires mon=assign.
  A failed write leaves the current association in place.

Per-rdtgroup files kmode_cpus and kmode_cpus_list scope the association
to a subset of online CPUs. They are visible only on the group that
backs the active association. A write enables the association on CPUs
added to the mask and disables it on CPUs removed from the mask. A new
global_enable_per_cpu association sets the mask to every online CPU. A
CPU that comes online later joins the association. A CPU that goes
offline leaves it only when that CPU is in the mask.

A group that backs the association, or a control group whose monitor
group does, cannot change mode, including entering pseudo-lock setup.
The backing group cannot be renamed until the association is cleared.

The arch hook, resctrl_arch_configure_kmode_global(), keeps the
fs/resctrl layer arch-neutral.

resctrl_set_kmode_support() lets architecture code register each extra
mode during resctrl initialization. inherit_user is already registered
when the platform can allocate or monitor. On AMD, PLZA registers
global_enable_per_cpu when the feature is available.

Only AMD PLZA is wired up here; Intel and ARM can add their own support
later by implementing the hooks.

Examples
========

(See Documentation/filesystems/resctrl.rst, "kernel_mode", "kmode_cpus",
and "Examples on working with kernel_mode", for the full UAPI.)

  # Mount resctrl
  # mount -t resctrl resctrl /sys/fs/resctrl
  # cd /sys/fs/resctrl

  # Read the supported modes.  The active mode is bracketed for display
  # only; do not include brackets when writing.
  # cat info/kernel_mode
  [inherit_user]
  global_enable_per_cpu:ctrl=assign;mon=assign;group=//

  # Create a CTRL_MON group and back kernel-mode allocation with it.
  # mkdir ctrl1
  # echo "global_enable_per_cpu:ctrl=assign;mon=inherit;group=ctrl1//" \
          > info/kernel_mode
  # cat info/kernel_mode
  inherit_user
  [global_enable_per_cpu:ctrl=assign;mon=inherit;group=ctrl1//]

  # kmode_cpus and kmode_cpus_list are visible only on the backing group.
  # ls ctrl1/kmode_cpus*
  ctrl1/kmode_cpus  ctrl1/kmode_cpus_list

  # Restrict the association to a CPU subset. The association is enabled
  # on CPUs added to the mask and disabled on CPUs removed from it.
  # echo 0-3 > ctrl1/kmode_cpus_list
  # cat ctrl1/kmode_cpus
  f
  # cat ctrl1/kmode_cpus_list
  0-3

  # Return to inherit_user.
  # echo "inherit_user" > info/kernel_mode
  # cat info/kernel_mode
  [inherit_user]
  global_enable_per_cpu:ctrl=assign;mon=assign;group=//

Tested on AMD with PLZA; builds on x86 without PLZA and does not expose
global_enable_per_cpu unless the feature is available.

Layout
======

  01-03  x86: PLZA CPU feature, command-line option, and MSR definitions.
  04-07  fs/resctrl: kernel mode enum, arch hook, kmode state, and
         resctrl_set_kmode_support().
  08     x86/resctrl: register global_enable_per_cpu when PLZA is
         available.
  09     fs/resctrl: info/kernel_mode read interface.
  10     fs/resctrl: hidden rdtgroup files.
  11     fs/resctrl: per-rdtgroup kmode_cpus/kmode_cpus_list.
  12     fs/resctrl: CPU hotplug for the active association.
  13     fs/resctrl: deactivate the association when a group is removed.
  14     fs/resctrl: show or hide kmode_cpus on the backing group.
  15     fs/resctrl: reject mode changes while a group backs the
         association.
  16     fs/resctrl: info/kernel_mode write interface.
  17     fs/resctrl: kmode_cpus/kmode_cpus_list writes.
  18     fs/resctrl: documentation and examples.


Changelog
=========
v6:
  - Addressed all the comments as discussed in v5. Anything missing
    is not intentional. Please feel to comment on any mistake.
  - Rename assign_global_enable_per_cpu to global_enable_per_cpu and
    describe the interface as a kernel mode rather than a policy.
  - Flatten kernel mode state into resctrl_kmode. The arch hook is
    resctrl_arch_configure_kmode_global().
  - An omitted ctrl=, mon=, or group= keeps the value shown by reading
    info/kernel_mode. inherit_user takes no parameters.
    ctrl=inherit;mon=inherit remains global_enable_per_cpu.
  - Selecting a new global_enable_per_cpu association sets that group's
    kmode_cpus to every online CPU. A CPU that comes online later joins
    the association even if user space had removed it from the mask.
  - kmode_cpus files are created hidden and shown only while the group
    backs the active association.
  - A monitor group requires mon=assign. Removing a control group
    detaches a child that backs the association before the parent's
    CLOSID is freed.
  - Reject a mode change when the group or one of its monitor groups
    backs the association.
  - Split kmode_cpus visibility and the mode-change rejection into
    their own patches.
  - Use the parsing similar to ctrlmondata.c:parse_line(),
    ctrlmondata.c:resctrl_io_alloc_parse_line(),
    monitor.c:resctrl_parse_mbm_assignment().
  - Re-wrote the changelog as a feature and not as a bug-fix.
  - Remove implementation details in function headers.

v5:
  - Collapse the two v4 global-assign modes into a single
    assign_global_enable_per_cpu policy with ctrl= and mon= options.
  - Use "association" terminology consistently in code, errors, and
    documentation.
  - Register assign_global_enable_per_cpu once during x86 resource
    discovery when PLZA is available.
  - Reject mon=inherit when group= selects a monitor group.
  - Split hidden-file support, deactivation-on-teardown, and PLZA mode
    registration into separate patches for easier review.
  - Fix msr_pqr_plza_assoc truncation on 32-bit systems by using u64.
  - Refresh documentation and add a readable end-to-end example walkthrough.
  - Block pseudo-lock setup on kernel-mode-associated groups.
  - Reject mode changes and rename while a group backs the active
    association.

v4:
  - Reorder and split the series into 15 patches: separate read-only
    info/kernel_mode display from the write path; add hotplug support
    when a CPU comes online; add an end-to-end documentation/examples
    patch.
  - Introduced resctrl_set_kmode_support() so architecture code can
    register supported kernel-mode policies during resctrl initialization.
  - info/kernel_mode write: validate group type; run fail paths before
    tearing down the active association so errors retain the old state.
  - kmode_cpus / kmode_cpus_list: writable with incremental enable/disable
    deltas; empty masks allowed; offline CPUs rejected.
  - Hotplug: newly online CPUs are added to the associated group's
    kmode_cpu_mask and programmed when assign_global_enable_per_cpu is
    active.

v3:
  - Generalise the layer beyond AMD: rename "PLZA mode" to "kernel mode"
    (kmode) in code, sysfs, and Documentation.
  - Reset the association when the associated rdtgroup is removed,
    instead of leaving stale state.

v2:
  - Similar to RFC with a new proposal; interface names were not final.
  - Separated Global Bandwidth Enforcement (GLBE) from PLZA; this series
    only adds PLZA support.
  - Used "kmode" instead of "PLZA" in the generic layer.

Previous versions:
v5: https://lore.kernel.org/lkml/cover.1787772750.git.babu.moger@amd.com/
v4: https://lore.kernel.org/lkml/cover.1783461016.git.babu.moger@amd.com/
v3: https://lore.kernel.org/lkml/cover.1777591496.git.babu.moger@amd.com/
v2: https://lore.kernel.org/lkml/cover.1773347820.git.babu.moger@amd.com/
v1: https://lore.kernel.org/lkml/cover.1769029977.git.babu.moger@amd.com/


Babu Moger (18):
  x86/cpufeatures: Support Privilege Level Zero Association (PLZA)
  x86/resctrl: Add PLZA support to command-line options
  x86/resctrl: Add PLZA configuration definitions and data structures
  fs/resctrl: Introduce kernel mode enum
  x86,arm,fs/resctrl: Introduce architecture hook to program global
    kernel mode
  fs/resctrl: Introduce kernel mode states for resctrl
  fs/resctrl: Introduce resctrl_set_kmode_support() to register
    supported modes
  x86/resctrl: Register RESCTRL_GLOBAL_ENABLE_PER_CPU when PLZA is
    available
  fs/resctrl: Add interface to display kernel mode status
  fs/resctrl: Add support for hidden resource group files
  fs/resctrl: Introduce kmode_cpus/kmode_cpus_list per rdtgroup
  fs/resctrl: Add CPU hotplug support for kernel mode associations
  fs/resctrl: Deactivate kernel mode associations when a group is
    removed
  fs/resctrl: Control visibility of global per-CPU kernel mode group
    files
  fs/resctrl: Reject mode changes for groups backing kernel mode
  fs/resctrl: Add interface to modify kernel mode via info/kernel_mode
  fs/resctrl: Allow user space to write kmode_cpus/kmode_cpus_list
  fs/resctrl: Add kernel mode documentation and examples

 .../admin-guide/kernel-parameters.txt         |   2 +-
 Documentation/filesystems/resctrl.rst         | 209 +++++
 arch/x86/include/asm/cpufeatures.h            |   1 +
 arch/x86/include/asm/msr-index.h              |   1 +
 arch/x86/kernel/cpu/resctrl/core.c            |   6 +
 arch/x86/kernel/cpu/resctrl/ctrlmondata.c     |  38 +
 arch/x86/kernel/cpu/resctrl/internal.h        |  49 ++
 arch/x86/kernel/cpu/scattered.c               |   1 +
 drivers/resctrl/mpam_resctrl.c                |   7 +
 fs/resctrl/internal.h                         |  35 +
 fs/resctrl/rdtgroup.c                         | 713 ++++++++++++++++++
 include/linux/resctrl.h                       |  81 ++
 12 files changed, 1142 insertions(+), 1 deletion(-)

-- 
2.43.0

[PATCH v6 01/18] x86/cpufeatures: Support Privilege Level Zero Association (PLZA)

[Babu Moger]

When memory bandwidth associated with a CLOSID is aggressively throttled,
and a task with that CLOSID moves into kernel mode, the kernel operations
are also aggressively throttled. This can stall forward progress and
eventually degrade overall system performance.

AMD hardware supports a feature Privilege Level Zero Association (PLZA),
which allows the CPU's CLOSID and/or RMID association to be changed during
the transition from user mode to kernel mode. This allows the kernel to run
using a different CLOSID and/or RMID than user space, which can improve
system performance and/or allow kernel work to be monitored separately.

Note that PLZA feature associates CLOSID and/or RMID to kernel work, and
therefore influences resource allocation and/or monitoring for all
resources, not only memory bandwidth.

The feature is detected via CPUID_Fn80000020_EBX_x00 [Bit 9]: Privilege
Level Zero Association (PLZA).

The PLZA feature details are documented in [1] available from [2].

[1] AMD64 Zen6 Platform Quality of Service (PQOS) Extensions:
    Publication # 69193 Revision: 1.00, Issue Date: March 2026

Signed-off-by: Babu Moger <babu.moger@amd.com>
Acked-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://bugzilla.kernel.org/show_bug.cgi?id=206537 # [2]
---
v6: Highlight that CLOSID/RMID association affects all resources,
    not just memory bandwidth.

v5: Added Acked-by from Boris.
    Updated the changelog to include cache allocation and monitoring.
    Moved the Link tag last.
    Updated subject line for cpufeatures.

v4: Split the patch into 2. This patch only handles x86 changes.
    Re-wrote the changelog along the ABMC changes.

v3: Code did not change. Patch order changed.
    Added documentation link.

v2: Rebased on top of the latest tip.
---
 arch/x86/include/asm/cpufeatures.h | 1 +
 arch/x86/kernel/cpu/scattered.c    | 1 +
 2 files changed, 2 insertions(+)

diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index bce3cc6bbadd..64ad09084766 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -532,6 +532,7 @@
 						      * and purposes if CLEAR_CPU_BUF_VM is set).
 						      */
 #define X86_FEATURE_X2AVIC_EXT		(21*32+20) /* AMD SVM x2AVIC support for 4k vCPUs */
+#define X86_FEATURE_PLZA		(21*32+21) /* Privilege Level Zero Association */
 
 /*
  * BUG word(s)
diff --git a/arch/x86/kernel/cpu/scattered.c b/arch/x86/kernel/cpu/scattered.c
index 580b252e07cc..e103ea8b62df 100644
--- a/arch/x86/kernel/cpu/scattered.c
+++ b/arch/x86/kernel/cpu/scattered.c
@@ -61,6 +61,7 @@ static const struct cpuid_bit cpuid_bits[] = {
 	{ X86_FEATURE_BMEC,			CPUID_EBX,  3, 0x80000020, 0 },
 	{ X86_FEATURE_ABMC,			CPUID_EBX,  5, 0x80000020, 0 },
 	{ X86_FEATURE_SDCIAE,			CPUID_EBX,  6, 0x80000020, 0 },
+	{ X86_FEATURE_PLZA,			CPUID_EBX,  9, 0x80000020, 0 },
 	{ X86_FEATURE_AMD_WORKLOAD_CLASS,	CPUID_EAX, 22, 0x80000021, 0 },
 	{ X86_FEATURE_TSA_SQ_NO,		CPUID_ECX,  1, 0x80000021, 0 },
 	{ X86_FEATURE_TSA_L1_NO,		CPUID_ECX,  2, 0x80000021, 0 },
-- 
2.43.0

[PATCH v6 02/18] x86/resctrl: Add PLZA support to command-line options

[Babu Moger]

Add a kernel command-line parameter to enable or disable the exposure of
the PLZA (Privilege Level Zero Association) feature to resctrl.

This allows administrators to control PLZA visibility at boot time via the
rdt= option (e.g. rdt=plza or rdt=!plza).

Add "plza" to the rdt= command-line parameter so PLZA exposure to resctrl
can be enabled or disabled at boot. PLZA is exposed by default when
supported; rdt=!plza suppresses it.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v6: Updated the changelog.

v5: Updated the changelog to mention about rdt= options.

v4: Split the patch 1 from v3 into 2 patches.
    This patch contains changes for resctrl subsystem.
---
 Documentation/admin-guide/kernel-parameters.txt | 2 +-
 Documentation/filesystems/resctrl.rst           | 1 +
 arch/x86/kernel/cpu/resctrl/core.c              | 2 ++
 3 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 33cd30996e47..ee39a39ee9cd 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -6449,7 +6449,7 @@ Kernel parameters
 	rdt=		[HW,X86,RDT]
 			Turn on/off individual RDT features. List is:
 			cmt, mbmtotal, mbmlocal, l3cat, l3cdp, l2cat, l2cdp,
-			mba, smba, bmec, abmc, sdciae, energy[:guid],
+			mba, smba, bmec, abmc, sdciae, plza, energy[:guid],
 			perf[:guid].
 			E.g. to turn on cmt and turn off mba use:
 				rdt=cmt,!mba
diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
index dd4f80da419c..73bf89d0ac83 100644
--- a/Documentation/filesystems/resctrl.rst
+++ b/Documentation/filesystems/resctrl.rst
@@ -28,6 +28,7 @@ SMBA (Slow Memory Bandwidth Allocation)				""
 BMEC (Bandwidth Monitoring Event Configuration)			""
 ABMC (Assignable Bandwidth Monitoring Counters)			""
 SDCIAE (Smart Data Cache Injection Allocation Enforcement)	""
+PLZA (Privilege Level Zero Association)				""
 =============================================================== ================================
 
 Historically, new features were made visible by default in /proc/cpuinfo. This
diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index 55214d6fdc49..4ca8bedeb995 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -803,6 +803,7 @@ enum {
 	RDT_FLAG_BMEC,
 	RDT_FLAG_ABMC,
 	RDT_FLAG_SDCIAE,
+	RDT_FLAG_PLZA,
 };
 
 #define RDT_OPT(idx, n, f)	\
@@ -830,6 +831,7 @@ static struct rdt_options rdt_options[]  __ro_after_init = {
 	RDT_OPT(RDT_FLAG_BMEC,	    "bmec",	X86_FEATURE_BMEC),
 	RDT_OPT(RDT_FLAG_ABMC,	    "abmc",	X86_FEATURE_ABMC),
 	RDT_OPT(RDT_FLAG_SDCIAE,    "sdciae",	X86_FEATURE_SDCIAE),
+	RDT_OPT(RDT_FLAG_PLZA,	    "plza",	X86_FEATURE_PLZA),
 };
 #define NUM_RDT_OPTIONS ARRAY_SIZE(rdt_options)
 
-- 
2.43.0

[PATCH v6 03/18] x86/resctrl: Add PLZA configuration definitions and data structures

[Babu Moger]

Privilege Level Zero Association (PLZA) enables the kernel to switch to
a different CLOSID and/or RMID than those used in user mode when entering
kernel mode. CLOSID and RMID selection are independent and can be
configured separately.

PLZA associations are programmed on a per-CPU basis through
MSR_IA32_PQR_PLZA_ASSOC (0xc00003fc), which contains the kernel RMID and
CLOSID values along with their corresponding enable bits.

Make the register and its field layout available so kernel mode allocation
and monitoring associations can be configured independently.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v6: Updated the code comments to match general format (members first and
    details after).
    changelog update.

v5: Sashiko reported:
    https://sashiko.dev/#/patchset/cover.1783461016.git.babu.moger%40amd.com?part=3
    Fix the union truncate issue on 32 bit systems by using u64.
    Updated the changelog and code comments.

v4: Re-wrote the changelog and code comment.

v3: No code changes. Patch order changed. Improved changelog.

v2: No changes. Just rebasing on top of the latest tip branch.
---
 arch/x86/include/asm/msr-index.h       |  1 +
 arch/x86/kernel/cpu/resctrl/internal.h | 49 ++++++++++++++++++++++++++
 2 files changed, 50 insertions(+)

diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index 82ca6356dc62..1331291f7a36 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -1304,6 +1304,7 @@
 /* - AMD: */
 #define MSR_IA32_MBA_BW_BASE		0xc0000200
 #define MSR_IA32_SMBA_BW_BASE		0xc0000280
+#define MSR_IA32_PQR_PLZA_ASSOC		0xc00003fc
 #define MSR_IA32_L3_QOS_ABMC_CFG	0xc00003fd
 #define MSR_IA32_L3_QOS_EXT_CFG		0xc00003ff
 #define MSR_IA32_EVT_CFG_BASE		0xc0000400
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index e3cfa0c10e92..44ecde20b1b4 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -222,6 +222,55 @@ union l3_qos_abmc_cfg {
 	unsigned long full;
 };
 
+/*
+ * Privilege Level Zero Association (PLZA) is configured via
+ * MSR_IA32_PQR_PLZA_ASSOC.
+ *
+ * @rmid		: RMID programmed for PLZA.
+ * @reserved1		: Reserved.
+ * @rmid_en		: Enable RMID for PLZA.
+ * @closid		: CLOSID programmed for PLZA.
+ * @reserved2		: Reserved.
+ * @closid_en		: Enable CLOSID for PLZA.
+ * @reserved3		: Reserved.
+ * @plza_en		: Enable PLZA. When enabled, PLZA applies to the
+ *			  given CPU.
+ *
+ * The MSR_IA32_PQR_PLZA_ASSOC must be programmed on every CPU in a QoS
+ * domain. Each domain supports a single PLZA CLOSID and/or RMID
+ * association, and all fields except @plza_en must be programmed
+ * identically across the domain to ensure consistent behavior.
+ *
+ * CLOSID and RMID selections are controlled independently. When PLZA is
+ * enabled (@plza_en = 1), CPL0 uses CLOSID and RMID values from
+ * MSR_IA32_PQR_PLZA_ASSOC depending on the corresponding enable bits. When
+ * PLZA is disabled (@plza_en = 0), the CLOSID and RMID are always taken
+ * from MSR_IA32_PQR_ASSOC regardless of privilege level.
+ *
+ * - @rmid_en = 1: use the RMID from MSR_IA32_PQR_PLZA_ASSOC if @plza_en = 1.
+ * - @rmid_en = 0: use the RMID from MSR_IA32_PQR_ASSOC.
+ * - @closid_en = 1: use the CLOSID from MSR_IA32_PQR_PLZA_ASSOC if @plza_en = 1.
+ * - @closid_en = 0: use the CLOSID from MSR_IA32_PQR_ASSOC.
+ *
+ * This allows PLZA to override only RMID, only CLOSID, or both. If both
+ * rmid_en and closid_en are clear, the effective behavior is the same as
+ * when PLZA is disabled, with RMID and CLOSID used from MSR_IA32_PQR_ASSOC
+ * irrespective of privilege level.
+ */
+union msr_pqr_plza_assoc {
+	struct {
+		u64	rmid		:12,
+			reserved1	:19,
+			rmid_en		: 1,
+			closid		: 4,
+			reserved2	:11,
+			closid_en	: 1,
+			reserved3	:15,
+			plza_en		: 1;
+	} split;
+	u64	full;
+};
+
 void rdt_ctrl_update(void *arg);
 
 int rdt_get_l3_mon_config(struct rdt_resource *r);
-- 
2.43.0

[PATCH v6 04/18] fs/resctrl: Introduce kernel mode enum

[Babu Moger]

Kernel mode work can use different allocation and monitoring associations
than the originating user task. On x86, Privilege Level Zero Association
(PLZA) enables the kernel to switch to a different CLOSID and/or RMID when
entering kernel mode.

Architectures need a common way to name kernel modes before resctrl
can report what is active or what the platform supports.

Introduce enum resctrl_kernel_mode:
  - RESCTRL_INHERIT_USER: Kernel mode work inherits allocation and
    monitoring from the user task.
  - RESCTRL_GLOBAL_ENABLE_PER_CPU: Kernel mode work uses allocation and/or
    monitoring associations that differ from the user task.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v6: changelog update and code comment update.
    Renamed RESCTRL_ASSIGN_GLOBAL_ENABLE_PER_CPU to RESCTRL_GLOBAL_ENABLE_PER_CPU.

v5: Collapse the two global-assign modes into a single
    RESCTRL_ASSIGN_GLOBAL_ENABLE_PER_CPU value. Update the changelog to
    match the enum names and descriptions in the code.

v4: Updated the changelog to be generic as possible.
    Moved the enum resctrl_kernel_mode to include/linux/resctrl.h.
    Updated the code comments to be generic.
    Removed resctrl_kmode_cfg from the code. This definition can be fs
    specific only and architectures dont need to know.
    Changed enum name to resctrl_kernel_mode from resctrl_kernel_modes.

v3: Removed resctrl_kmode definition.
    Changed the kernel mode definitions to enum resctrl_kernel_modes.
    Used BIT() to set/test the features.
    Added details to changelog.

v2: New patch to handle PLZA interfaces with /sys/fs/resctrl/info/ directory.
    https://lore.kernel.org/lkml/2ab556af-095b-422b-9396-f845c6fd0342@intel.com/
---
 include/linux/resctrl.h | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index 10dfdca7f4bf..f6adff7f4e1f 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -723,6 +723,35 @@ int resctrl_arch_io_alloc_enable(struct rdt_resource *r, bool enable);
  */
 bool resctrl_arch_get_io_alloc_enabled(struct rdt_resource *r);
 
+/**
+ * enum resctrl_kernel_mode - Kernel mode control and monitoring association.
+ *
+ * @RESCTRL_INHERIT_USER:
+ *     Kernel mode inherits both allocation and monitoring associations
+ *     from the user-space task. On x86, the kernel uses the same CLOSID and
+ *     RMID as the user-space task.
+ *
+ * @RESCTRL_GLOBAL_ENABLE_PER_CPU:
+ *     All kernel mode work uses the same allocation and/or monitoring
+ *     association via an assigned CTRL_MON or MON group. The mode can
+ *     be enabled/disabled per-CPU. When the mode is disabled on a CPU
+ *     then the kernel work on that CPU inherits the allocation and
+ *     monitoring association from the user-space task, effectively the
+ *     kernel mode becomes RESCTRL_INHERIT_USER on that CPU.
+ *
+ *     All online CPUs are included by default. A subset may be selected
+ *     through the resctrl group interface, and a CTRL_MON or MON group
+ *     may be assigned to this mode. On x86, CLOSID and RMID are configured
+ *     independently; either or both may differ from the user-space task.
+ */
+enum resctrl_kernel_mode {
+	RESCTRL_INHERIT_USER,
+	RESCTRL_GLOBAL_ENABLE_PER_CPU,
+	RESCTRL_KMODE_LAST = RESCTRL_GLOBAL_ENABLE_PER_CPU
+};
+
+#define RESCTRL_NUM_KERNEL_MODES (RESCTRL_KMODE_LAST + 1)
+
 extern unsigned int resctrl_rmid_realloc_threshold;
 extern unsigned int resctrl_rmid_realloc_limit;
 
-- 
2.43.0

[PATCH v6 05/18] x86,arm,fs/resctrl: Introduce architecture hook to program global kernel mode

[Babu Moger]

RESCTRL_GLOBAL_ENABLE_PER_CPU mode enables kernel work to use different
allocation and monitoring associations than the originating user task.

The mode is enabled independently on each CPU, so the filesystem must
program hardware when the mode is activated, when CPUs are added or
removed, and when it is deactivated. Allocation and monitoring associations
can be done independently, so the architecture callback takes separate
control and monitoring options.

Add resctrl_arch_configure_kmode_global() with an x86 PLZA implementation
and an empty MPAM stub for ARM systems that do not support this mode.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v6: Rename to resctrl_arch_configure_kmode_global(). Rewrite changelog
    for RESCTRL_GLOBAL_ENABLE_PER_CPU only (not a bugfix, not a code
    restatement). Add arm to the subject. Document parameters first;
    closid/rmid are always programmed. Split fs/arch contract from
    call scenarios and name the mode explicitly.

v5: Fix kernel-doc for resctrl_arch_configure_kmode(), wire assign_ctrl
    through to PLZA closid_en, and update the changelog.
    Add calling context in API doc.

v4: Added assign_mon parameter in resctrl_arch_configure_kmode() to program the RMID
    as discussed in below.
    https://lore.kernel.org/lkml/20260605100642.1103628-1-qinyuntan@linux.alibaba.com/
    Reintroduce independent monitoring assignment via assign_mon after removing
    the task-based PLZA approach in v3.
    Changed cpumask type to "const struct cpumask *cpu_mask".
    Added MPAM stub to avoid any linking issues when resctrl_arch_configure_kmode()
    is called from FS layer. Thanks to Qinyun.
    Re-wrote the changelog to be generic.
    Updated code comments.
v3: Removed task based PLZA implementation so related changes are removed.
    Per-task RMID handling is replaced by group type selection in the fs layer.
    Updated the change log with details.
    Removed resctrl_arch_set_kmode() as arch only provides the modes supported.
    It is FS which decided which mode to apply.
v2: Updated the commit message to include the sequence of steps to enable PLZA.
    Added mode code comments for clarity.
    Added kmode to function names to be generic.
---
 arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 38 +++++++++++++++++++++++
 drivers/resctrl/mpam_resctrl.c            |  7 +++++
 include/linux/resctrl.h                   | 37 ++++++++++++++++++++++
 3 files changed, 82 insertions(+)

diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
index 62044489b052..a343dc0c71c5 100644
--- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
+++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
@@ -137,3 +137,41 @@ int resctrl_arch_io_alloc_enable(struct rdt_resource *r, bool enable)
 
 	return 0;
 }
+
+static void resctrl_kmode_set_one_amd(void *arg)
+{
+	union msr_pqr_plza_assoc *plza = arg;
+
+	wrmsrq(MSR_IA32_PQR_PLZA_ASSOC, plza->full);
+}
+
+/*
+ * resctrl_arch_configure_kmode_global - Program Privilege Level Zero
+ *					 Association (PLZA).
+ * @cpu_mask:	CPUs whose PLZA MSR should be updated.
+ * @closid:	CLOSID programmed into MSR_IA32_PQR_PLZA_ASSOC. Always
+ *		written; required even when @assign_ctrl is false.
+ * @assign_ctrl: Whether PLZA should program the kernel-mode CLOSID.
+ * @rmid:	RMID programmed into MSR_IA32_PQR_PLZA_ASSOC. Always
+ *		written; required even when @assign_mon is false.
+ * @assign_mon:	Whether PLZA should program the kernel-mode RMID.
+ * @enable:	Whether PLZA should program the kernel-mode association.
+ *
+ * Program PLZA on @cpu_mask for %RESCTRL_GLOBAL_ENABLE_PER_CPU. See
+ * union msr_pqr_plza_assoc for the hardware field meanings.
+ */
+void resctrl_arch_configure_kmode_global(const struct cpumask *cpu_mask,
+					 u32 closid, bool assign_ctrl,
+					 u32 rmid, bool assign_mon,
+					 bool enable)
+{
+	union msr_pqr_plza_assoc plza = { 0 };
+
+	plza.split.rmid = rmid;
+	plza.split.rmid_en = assign_mon;
+	plza.split.closid = closid;
+	plza.split.closid_en = assign_ctrl;
+	plza.split.plza_en = enable;
+
+	on_each_cpu_mask(cpu_mask, resctrl_kmode_set_one_amd, &plza, 1);
+}
diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c
index f2c651e9f6ea..0c8594e98526 100644
--- a/drivers/resctrl/mpam_resctrl.c
+++ b/drivers/resctrl/mpam_resctrl.c
@@ -139,6 +139,13 @@ bool resctrl_arch_get_io_alloc_enabled(struct rdt_resource *r)
 	return false;
 }
 
+void resctrl_arch_configure_kmode_global(const struct cpumask *cpu_mask,
+					 u32 closid, bool assign_ctrl,
+					 u32 rmid, bool assign_mon,
+					 bool enable)
+{
+}
+
 void resctrl_arch_pre_mount(void)
 {
 }
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index f6adff7f4e1f..dcb0beb847b6 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -752,6 +752,43 @@ enum resctrl_kernel_mode {
 
 #define RESCTRL_NUM_KERNEL_MODES (RESCTRL_KMODE_LAST + 1)
 
+/**
+ * resctrl_arch_configure_kmode_global() - Program RESCTRL_GLOBAL_ENABLE_PER_CPU
+ *
+ * @cpu_mask:	CPUs whose kernel-mode association should be updated.
+ * @closid:	CLOSID that matches the RMID to program kernel mode. Depending
+ *		on the architecture, the counter may match traffic of both
+ *		@closid and @rmid, or @rmid only. Always provided; a valid
+ *		CLOSID is required even when @assign_ctrl is false.
+ * @assign_ctrl: true to assign @closid for kernel mode; false to inherit
+ *		association from the user-space task.
+ * @rmid:	RMID to program the kernel mode. Some architectures may use
+ *		CLOSID/RMID separately, others will consider them together.
+ *		Always provided; a valid RMID is required even when
+ *		@assign_mon is false.
+ * @assign_mon:	true to assign @rmid for kernel mode; false to inherit
+ *		monitoring association from the user-space task.
+ * @enable:	true to enable kernel mode association on CPUs in @cpu_mask;
+ *		false to disable kernel mode.
+ *
+ * This callback programs %RESCTRL_GLOBAL_ENABLE_PER_CPU. It is called in
+ * the following scenarios:
+ * - If the mode is active when user space changes the configuration, it is
+ *   first called to deactivate the mode on all CPUs where it is active.
+ * - When user space activates the mode, it is called with cpu_online_mask.
+ * - When user space adds a CPU to the active mode.
+ * - When user space removes a CPU from the active mode.
+ *
+ * Contract between fs and arch: resctrl fs always provides the same
+ * closid, assign_ctrl, rmid, and assign_mon when activating
+ * %RESCTRL_GLOBAL_ENABLE_PER_CPU, during CPU add/remove while the mode is
+ * active, and when deactivating the mode.
+ */
+void resctrl_arch_configure_kmode_global(const struct cpumask *cpu_mask,
+					 u32 closid, bool assign_ctrl,
+					 u32 rmid, bool assign_mon,
+					 bool enable);
+
 extern unsigned int resctrl_rmid_realloc_threshold;
 extern unsigned int resctrl_rmid_realloc_limit;
 
-- 
2.43.0

[PATCH v6 06/18] fs/resctrl: Introduce kernel mode states for resctrl

[Babu Moger]

The filesystem tracks supported kernel modes, the active mode, and the
control and monitoring association for the assigned group.

Introduce struct resctrl_kmode_cfg and initialize the default configuration
from the platform's capabilities.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v6: Drop ctrl_en/mon_en and use resctrl_arch_alloc_capable() /
    resctrl_arch_mon_capable() directly. Flatten caps/active into
    resctrl_kmode_cfg. Rename resctrl_kcfg to resctrl_kmode, ctrl_mode
    to ctrl, mon_mode to mon, and k_rdtgrp to rdtgrp. Comments describe
    the objects, not a member list. Rewrite changelog.

v5: Replace per-mode registration helpers with a single resctrl_kmode_cfg
    structure. Derive ctrl_en and mon_en from resctrl_arch_alloc_capable()
    and resctrl_arch_mon_capable() during filesystem init. Update code
    comments and changelog.

v4: New patch to initialize supported kernel modes.
    https://lore.kernel.org/lkml/737a4228-52fb-4583-ac64-8efe79c107e6@intel.com/
    Changed the kmode_cur to enum resctrl_kernel_mode in resctrl_kmode_cfg.
    Moved the resctrl_kmode_cfg to filesystem code.
---
 fs/resctrl/internal.h | 29 +++++++++++++++++++++++++++++
 fs/resctrl/rdtgroup.c | 25 +++++++++++++++++++++++++
 2 files changed, 54 insertions(+)

diff --git a/fs/resctrl/internal.h b/fs/resctrl/internal.h
index e62a277dee85..98f04a2a762b 100644
--- a/fs/resctrl/internal.h
+++ b/fs/resctrl/internal.h
@@ -314,6 +314,35 @@ struct mbm_state {
 	u32	prev_bw;
 };
 
+/**
+ * enum kmode_state - Control or monitoring association
+ * @KMODE_INHERIT: Inherit from the user-space task.
+ * @KMODE_ASSIGN: Use the backing group's CLOSID or RMID.
+ */
+enum kmode_state {
+	KMODE_INHERIT,
+	KMODE_ASSIGN
+};
+
+/**
+ * struct resctrl_kmode_cfg - Kernel mode configuration
+ * @kmode_sup:	Bitmap of supported kernel modes.
+ * @kmode_cur:	Currently selected kernel mode.
+ * @ctrl:	Control association for %RESCTRL_GLOBAL_ENABLE_PER_CPU.
+ *		Kept while %RESCTRL_INHERIT_USER is selected.
+ * @mon:	Monitoring association for %RESCTRL_GLOBAL_ENABLE_PER_CPU.
+ *		Kept while %RESCTRL_INHERIT_USER is selected.
+ * @rdtgrp:	Resource group backing %RESCTRL_GLOBAL_ENABLE_PER_CPU.
+ *		%NULL while that mode is not selected.
+ */
+struct resctrl_kmode_cfg {
+	DECLARE_BITMAP(kmode_sup, RESCTRL_NUM_KERNEL_MODES);
+	enum resctrl_kernel_mode	kmode_cur;
+	enum kmode_state		ctrl;
+	enum kmode_state		mon;
+	struct rdtgroup			*rdtgrp;
+};
+
 extern struct mutex rdtgroup_mutex;
 
 static inline const char *rdt_kn_name(const struct kernfs_node *kn)
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 68be9b903ac6..7c39792be1fa 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -76,6 +76,9 @@ static void rdtgroup_destroy_root(void);
 
 struct dentry *debugfs_resctrl;
 
+/* Kernel mode for this system. */
+static struct resctrl_kmode_cfg resctrl_kmode;
+
 /*
  * Memory bandwidth monitoring event to use for the default CTRL_MON group
  * and each new CTRL_MON group created by the user.  Only relevant when
@@ -2297,6 +2300,26 @@ static void io_alloc_init(void)
 	}
 }
 
+/*
+ * Set RESCTRL_INHERIT_USER as the default when the platform supports
+ * allocation or monitoring. Otherwise leave kernel mode unsupported.
+ */
+static void resctrl_kmode_init(void)
+{
+	if (!resctrl_arch_alloc_capable() && !resctrl_arch_mon_capable())
+		return;
+
+	resctrl_kmode.kmode_cur = RESCTRL_INHERIT_USER;
+	__set_bit(RESCTRL_INHERIT_USER, resctrl_kmode.kmode_sup);
+
+	/*
+	 * Defaults used when RESCTRL_GLOBAL_ENABLE_PER_CPU mode is
+	 * activated without new values.
+	 */
+	resctrl_kmode.ctrl = KMODE_ASSIGN;
+	resctrl_kmode.mon = KMODE_ASSIGN;
+}
+
 void resctrl_file_fflags_init(const char *config, unsigned long fflags)
 {
 	struct rftype *rft;
@@ -4814,6 +4837,8 @@ int resctrl_init(void)
 	if (ret)
 		return ret;
 
+	resctrl_kmode_init();
+
 	ret = sysfs_create_mount_point(fs_kobj, "resctrl");
 	if (ret) {
 		resctrl_l3_mon_resource_exit();
-- 
2.43.0

[PATCH v6 07/18] fs/resctrl: Introduce resctrl_set_kmode_support() to register supported modes

[Babu Moger]

The resctrl subsystem tracks kernel mode configuration. Architectures need
to register the supported kernel modes.

Add resctrl_set_kmode_support() so an architecture can register supported
kernel modes.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v6: Use resctrl_kmode.kmode_sup after the struct flatten. Drop duplicate
    kernel-doc from the .c definition and place the function next to
    resctrl_kmode_init(). Rewrite changelog (not a bugfix).

v5: New patch split from resctrl_kmode_cfg initialization. Register modes
    in resctrl_kmode_cfg.caps.kmode_sup using RESCTRL_INHERIT_USER and
    RESCTRL_ASSIGN_GLOBAL_ENABLE_PER_CPU. Ignore out-of-range mode values.

v4: New patch to initialize supported kernel modes.
    https://lore.kernel.org/lkml/737a4228-52fb-4583-ac64-8efe79c107e6@intel.com/
---
 fs/resctrl/rdtgroup.c   |  6 ++++++
 include/linux/resctrl.h | 15 +++++++++++++++
 2 files changed, 21 insertions(+)

diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 7c39792be1fa..75b81d07a247 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -2320,6 +2320,12 @@ static void resctrl_kmode_init(void)
 	resctrl_kmode.mon = KMODE_ASSIGN;
 }
 
+void resctrl_set_kmode_support(enum resctrl_kernel_mode kmode)
+{
+	if (kmode < RESCTRL_NUM_KERNEL_MODES)
+		__set_bit(kmode, resctrl_kmode.kmode_sup);
+}
+
 void resctrl_file_fflags_init(const char *config, unsigned long fflags)
 {
 	struct rftype *rft;
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index dcb0beb847b6..5553f6301bd4 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -752,6 +752,21 @@ enum resctrl_kernel_mode {
 
 #define RESCTRL_NUM_KERNEL_MODES (RESCTRL_KMODE_LAST + 1)
 
+/**
+ * resctrl_set_kmode_support() - Register a supported kernel mode
+ * @kmode:	Kernel mode supported by the architecture.
+ *
+ * RESCTRL_INHERIT_USER is registered by resctrl_kmode_init() and does
+ * not need to be registered again.
+ *
+ * Architectures use this interface during resctrl initialization to
+ * advertise supported kernel modes. Registration of a mode indicates
+ * architectural support only. The final decision to expose and enable
+ * a mode is made by resctrl based on system capabilities and
+ * configuration.
+ */
+void resctrl_set_kmode_support(enum resctrl_kernel_mode kmode);
+
 /**
  * resctrl_arch_configure_kmode_global() - Program RESCTRL_GLOBAL_ENABLE_PER_CPU
  *
-- 
2.43.0

[PATCH v6 08/18] x86/resctrl: Register RESCTRL_GLOBAL_ENABLE_PER_CPU when PLZA is available

[Babu Moger]

PLZA (Privilege Level Zero Association) provides the global per-CPU kernel
mode, RESCTRL_GLOBAL_ENABLE_PER_CPU, allowing kernel work to use allocation
and monitoring associations that differ from user tasks.

Add x86 support for this kernel mode and make it available through the
resctrl kernel-mode infrastructure.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v6: Use RESCTRL_GLOBAL_ENABLE_PER_CPU.
    Rewrite changelog (not a bugfix).

v5: Update the patch to adapt new data structure and resctrl_set_kmode_support()
    changes.

v4: New patch to set the supported features during arch init.
---
 arch/x86/kernel/cpu/resctrl/core.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index 4ca8bedeb995..fc59d66f77de 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -1021,6 +1021,10 @@ static __init bool get_rdt_resources(void)
 	rdt_alloc_capable = get_rdt_alloc_resources();
 	rdt_mon_capable = get_rdt_mon_resources();
 
+	if (rdt_cpu_has(X86_FEATURE_PLZA) &&
+	    (rdt_alloc_capable || rdt_mon_capable))
+		resctrl_set_kmode_support(RESCTRL_GLOBAL_ENABLE_PER_CPU);
+
 	return (rdt_mon_capable || rdt_alloc_capable);
 }
 
-- 
2.43.0

[PATCH v6 09/18] fs/resctrl: Add interface to display kernel mode status

[Babu Moger]

Resctrl kernel modes control how allocation and monitoring associations
are applied in kernel mode relative to user mode.

Add info/kernel_mode to expose the supported and active kernel modes via
the resctrl filesystem. Supported modes are listed one per line, with the
active mode enclosed in square brackets.

Document the interface in Documentation/filesystems/resctrl.rst.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v6: Rewrote the changelog to make it as feature and not as a bug fix.
    Update the code to use the new name "global_enable_per_cpu".
    Display ctrl or mon only when allocation or monitoring is supportd.

v5: Use resctrl_kcfg.caps and resctrl_kcfg.active after the struct split
    from patch 0006. Rename resctrl_kmode_bind_str() to
    resctrl_kmode_state_str() and ctrl_bind/mon_bind to ctrl_state/mon_state.
    Used info_kn_lock()/info_kn_unlock().
    Add documentation update in resctrl.rst.

v4: Fixed the display of inherit_ctrl_and_mon policy. It is not
    associated with any group.
    Added "uninitialized" for the non-active group.
    Rewrote the changelog.

v3: New patch to handle the changed interface file info/kernel_mode.
    Changed the group name to "none" if kmode binding is not done.
    Reinette suggested "uninitialized". "none" seemed more relevent.
---
 Documentation/filesystems/resctrl.rst |  48 +++++++++++++
 fs/resctrl/rdtgroup.c                 | 100 ++++++++++++++++++++++++++
 2 files changed, 148 insertions(+)

diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
index 73bf89d0ac83..73ccbb9594f5 100644
--- a/Documentation/filesystems/resctrl.rst
+++ b/Documentation/filesystems/resctrl.rst
@@ -527,6 +527,54 @@ conveyed in the error returns from file operations. E.g.
 	# cat info/last_cmd_status
 	mask f7 has non-consecutive 1-bits
 
+"kernel_mode":
+	In the top level of the "info" directory, "kernel_mode" lists the
+	kernel modes supported by the system, one per line, with the active
+	mode enclosed in square brackets. A kernel mode determines the
+	allocation and monitoring associations when a task switches from
+	user mode to kernel mode.
+
+	Possible kernel modes are:
+
+	"inherit_user":
+		Kernel mode work uses the allocation and monitoring associations
+		of the user-space task on whose behalf it runs.
+
+	"global_enable_per_cpu":
+		Kernel mode work uses the allocation and/or monitoring
+		association of a backing CTRL_MON or MON group. The mode can
+		be activated or deactivated per CPU. The line contains group=
+		and whichever of ctrl= and mon= the platform supports::
+
+		  global_enable_per_cpu:ctrl=<assign|inherit>;mon=<assign|inherit>;group=<CTRL_MON>/<MON>/
+
+		"ctrl":
+			"assign" if kernel mode work uses the allocation
+			association of the resource group named by "group",
+			"inherit" if it uses the allocation association of
+			the user-space task. Present only on systems that
+			support allocation.
+		"mon":
+			"assign" if kernel mode work uses the monitoring
+			association of the resource group named by "group",
+			"inherit" if it uses the monitoring association of
+			the user-space task. Present only on systems that
+			support monitoring.
+		"group":
+			Path of the resource group, relative to the resctrl
+			mount point. A CTRL_MON group is shown as <CTRL_MON>//,
+			a MON group as <CTRL_MON>/<MON>/.
+
+		When the mode is not active, the parameters are the defaults
+		used if it is activated without new values.
+
+	Example of a system that supports allocation and monitoring, with
+	default mode inherit_user::
+
+	  # cat info/kernel_mode
+	  [inherit_user]
+	  global_enable_per_cpu:ctrl=assign;mon=assign;group=//
+
 Resource alloc and monitor groups
 =================================
 
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 75b81d07a247..e82709d64a00 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -993,6 +993,99 @@ static int rdt_last_cmd_status_show(struct kernfs_open_file *of,
 	return 0;
 }
 
+/* Names used in info/kernel_mode; indexed by enum resctrl_kernel_mode */
+static const char * const resctrl_kmode_str[] = {
+	[RESCTRL_INHERIT_USER]		= "inherit_user",
+	[RESCTRL_GLOBAL_ENABLE_PER_CPU]	= "global_enable_per_cpu"
+};
+
+static_assert(ARRAY_SIZE(resctrl_kmode_str) == RESCTRL_NUM_KERNEL_MODES);
+
+static const char *resctrl_kmode_state_str(enum kmode_state state)
+{
+	return state == KMODE_ASSIGN ? "assign" : "inherit";
+}
+
+static void resctrl_kmode_group_path(struct rdtgroup *rdtgrp,
+				     const char **ctrl, const char **mon)
+{
+	*ctrl = "";
+	*mon = "";
+
+	if (rdtgrp->type == RDTMON_GROUP) {
+		*ctrl = rdt_kn_name(rdtgrp->mon.parent->kn);
+		*mon = rdt_kn_name(rdtgrp->kn);
+	} else {
+		*ctrl = rdt_kn_name(rdtgrp->kn);
+	}
+}
+
+/**
+ * resctrl_kernel_mode_show() - Display supported and active kernel modes
+ * @of:		kernfs open file
+ * @seq:	output seq_file
+ * @v:		unused
+ *
+ * Lists one line per mode and brackets the active mode. inherit_user
+ * has no parameters. global_enable_per_cpu prints ctrl= only when
+ * allocation is supported and mon= only when monitoring is supported,
+ * then group=.
+ *
+ * Return: 0 on success, or -ENOENT on error.
+ */
+static int resctrl_kernel_mode_show(struct kernfs_open_file *of,
+				    struct seq_file *seq, void *v)
+{
+	const char *ctrl_grp, *mon_grp;
+	enum resctrl_kernel_mode mode;
+	bool active;
+	char sep;
+
+	if (!info_kn_lock(of->kn))
+		return -ENOENT;
+
+	for (mode = 0; mode < RESCTRL_NUM_KERNEL_MODES; mode++) {
+		if (!test_bit(mode, resctrl_kmode.kmode_sup))
+			continue;
+
+		active = resctrl_kmode.kmode_cur == mode;
+
+		seq_printf(seq, active ? "[%s" : "%s", resctrl_kmode_str[mode]);
+
+		if (mode == RESCTRL_GLOBAL_ENABLE_PER_CPU) {
+			ctrl_grp = "";
+			mon_grp = "";
+			if (active) {
+				if (resctrl_kmode.rdtgrp)
+					resctrl_kmode_group_path(resctrl_kmode.rdtgrp,
+								 &ctrl_grp, &mon_grp);
+				else
+					pr_warn("%s active without a resource group\n",
+						resctrl_kmode_str[mode]);
+			}
+
+			sep = ':';
+			if (resctrl_arch_alloc_capable()) {
+				seq_printf(seq, "%cctrl=%s", sep,
+					   resctrl_kmode_state_str(resctrl_kmode.ctrl));
+				sep = ';';
+			}
+			if (resctrl_arch_mon_capable()) {
+				seq_printf(seq, "%cmon=%s", sep,
+					   resctrl_kmode_state_str(resctrl_kmode.mon));
+				sep = ';';
+			}
+			seq_printf(seq, "%cgroup=%s/%s/", sep, ctrl_grp, mon_grp);
+		}
+
+		seq_puts(seq, active ? "]\n" : "\n");
+	}
+
+	info_kn_unlock(of->kn);
+
+	return 0;
+}
+
 void *rdt_kn_parent_priv(struct kernfs_node *kn)
 {
 	/*
@@ -1978,6 +2071,13 @@ static struct rftype res_common_files[] = {
 		.seq_show	= rdt_last_cmd_status_show,
 		.fflags		= RFTYPE_TOP_INFO,
 	},
+	{
+		.name		= "kernel_mode",
+		.mode		= 0444,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= resctrl_kernel_mode_show,
+		.fflags		= RFTYPE_TOP_INFO,
+	},
 	{
 		.name		= "mbm_assign_on_mkdir",
 		.mode		= 0644,
-- 
2.43.0

[PATCH v6 10/18] fs/resctrl: Add support for hidden resource group files

[Babu Moger]

Some per-group resctrl files are only meaningful once a resource group
has an active kernel mode association.

Creating them only at activation would complicate group setup and
teardown, while exposing them on every group would show empty files
with no clear purpose.

Add a hidden property to struct rftype to hide files at creation.

Suggested-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v6: Removed resctrl_hidden_files_set_visible() from the patch.
    It will be separate patch.
    Re-wrote the changelog.

v5: New patch to add support for creating hidden files.
---
 fs/resctrl/internal.h | 2 ++
 fs/resctrl/rdtgroup.c | 3 +++
 2 files changed, 5 insertions(+)

diff --git a/fs/resctrl/internal.h b/fs/resctrl/internal.h
index 98f04a2a762b..986b9c942fb2 100644
--- a/fs/resctrl/internal.h
+++ b/fs/resctrl/internal.h
@@ -283,6 +283,7 @@ extern int max_name_width;
  * @kf_ops:	File operations
  * @flags:	File specific RFTYPE_FLAGS_* flags
  * @fflags:	File specific RFTYPE_* flags
+ * @hidden:	Hide file at creation; visibility may be restored later
  * @seq_show:	Show content of the file
  * @write:	Write to the file
  */
@@ -292,6 +293,7 @@ struct rftype {
 	const struct kernfs_ops	*kf_ops;
 	unsigned long		flags;
 	unsigned long		fflags;
+	bool			hidden;
 
 	int (*seq_show)(struct kernfs_open_file *of,
 			struct seq_file *sf, void *v);
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index e82709d64a00..46b2cafd9411 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -308,6 +308,9 @@ static int rdtgroup_add_file(struct kernfs_node *parent_kn, struct rftype *rft)
 		return ret;
 	}
 
+	if (rft->hidden)
+		kernfs_show(kn, false);
+
 	return 0;
 }
 
-- 
2.43.0

[PATCH v6 11/18] fs/resctrl: Introduce kmode_cpus/kmode_cpus_list per rdtgroup

[Babu Moger]

The RESCTRL_GLOBAL_ENABLE_PER_CPU kernel mode allows tasks to use different
allocation and monitoring associations when transitioning from user mode to
kernel mode. The mode is enabled/disabled on a per-CPU basis.

Add kmode_cpus and kmode_cpus_list to display the CPUs associated with a
kernel mode. These files are exposed only when the global_enable_per_cpu
mode is active.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v6: Drop rdtgrp::kmode. Rewrite changelog (not a bugfix; association).
    https://lore.kernel.org/lkml/ef8804d5-f20c-49a1-87cf-ab39437a5f0d@intel.com/

v5: Updated the changelog.
    Removed the RDT_MODE_PSEUDO_LOCKED check and added rdtgroup->kmode check.
    Files are created hidden.

v4: Rewrote the change to be generic.

v3: New patch to add "kmode_cpus" and "kmode_cpus_list" to support
    kernel_modes.
---
 fs/resctrl/internal.h |  2 ++
 fs/resctrl/rdtgroup.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 44 insertions(+)

diff --git a/fs/resctrl/internal.h b/fs/resctrl/internal.h
index 986b9c942fb2..6b8d14c4bd83 100644
--- a/fs/resctrl/internal.h
+++ b/fs/resctrl/internal.h
@@ -216,6 +216,7 @@ struct mongroup {
  * @mon:			mongroup related data
  * @mode:			mode of resource group
  * @mba_mbps_event:		input monitoring event id when mba_sc is enabled
+ * @kmode_cpu_mask:		CPUs in this group's kernel-mode association
  * @plr:			pseudo-locked region
  */
 struct rdtgroup {
@@ -229,6 +230,7 @@ struct rdtgroup {
 	struct mongroup			mon;
 	enum rdtgrp_mode		mode;
 	enum resctrl_event_id		mba_mbps_event;
+	struct cpumask			kmode_cpu_mask;
 	struct pseudo_lock_region	*plr;
 };
 
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 46b2cafd9411..0e638dc654c1 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -385,6 +385,31 @@ static int rdtgroup_cpus_show(struct kernfs_open_file *of,
 	return ret;
 }
 
+/*
+ * Display CPU masks for the kernel-mode associated resource group.
+ * Supports both "kmode_cpus" (bitmap format) and "kmode_cpus_list"
+ * (range list format); the output format is selected accordingly.
+ *
+ * Returns -ENOENT on error.
+ */
+static int rdtgroup_kmode_cpus_show(struct kernfs_open_file *of,
+				    struct seq_file *s, void *v)
+{
+	struct rdtgroup *rdtgrp;
+	int ret = 0;
+
+	rdtgrp = rdtgroup_kn_lock_live(of->kn);
+
+	if (!rdtgrp || rdtgrp != resctrl_kmode.rdtgrp)
+		ret = -ENOENT;
+	else
+		seq_printf(s, is_cpu_list(of) ? "%*pbl\n" : "%*pb\n",
+			   cpumask_pr_args(&rdtgrp->kmode_cpu_mask));
+	rdtgroup_kn_unlock(of->kn);
+
+	return ret;
+}
+
 /*
  * Update the PGR_ASSOC MSR on all cpus in @cpu_mask,
  *
@@ -2256,6 +2281,23 @@ static struct rftype res_common_files[] = {
 		.flags		= RFTYPE_FLAGS_CPUS_LIST,
 		.fflags		= RFTYPE_BASE,
 	},
+	{
+		.name		= "kmode_cpus",
+		.mode		= 0444,
+		.hidden		= true,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdtgroup_kmode_cpus_show,
+		.fflags		= RFTYPE_BASE,
+	},
+	{
+		.name		= "kmode_cpus_list",
+		.mode		= 0444,
+		.hidden		= true,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdtgroup_kmode_cpus_show,
+		.flags		= RFTYPE_FLAGS_CPUS_LIST,
+		.fflags		= RFTYPE_BASE,
+	},
 	{
 		.name		= "tasks",
 		.mode		= 0644,
-- 
2.43.0

[PATCH v6 12/18] fs/resctrl: Add CPU hotplug support for kernel mode associations

[Babu Moger]

Kernel mode allocation and monitoring associations are maintained on a
per-CPU basis. Extend the kernel mode infrastructure to integrate with
CPU hotplug handling, allowing kernel mode state to be managed as CPUs
enter or leave the online set.

Introduce resctrl_kmode_global_online_cpu() and
resctrl_kmode_global_offline_cpu(), and hook them into the existing
resctrl_online_cpu() and resctrl_offline_cpu() paths.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v6: Update the changelog in non-bugfix form.
    Update the code to remove rdtgroup->kmode.
    Introduce new wrapper resctrl_configure_kmode_global() to consolidate
    architecture calls.

v5: Added resctrl_kmode_offline_cpu() to disable the kernel mode
    association and update kmode_cpu_mask when an associated CPU goes
    offline.
    This addresses sashiko comments:
    https://sashiko.dev/#/patchset/cover.1783461016.git.babu.moger%40amd.com?part=11
    https://sashiko.dev/#/patchset/cover.1783461016.git.babu.moger%40amd.com?part=10
    Also moved the patch earlier than patch 10.
    [RESEND PATCH v4 10/15] fs/resctrl: Reset the kernel mode binding when an rdtgroup is removed
    That seemed much more clear.

v4: New patch in the series. Patch taken from:
    https://lore.kernel.org/lkml/20260611111706.1981788-5-qinyuntan@linux.alibaba.com/
    Updated the code to enable kernel mode by default whenever a CPU comes
    online when one of global-assign mode is enabled.
---
 fs/resctrl/rdtgroup.c | 57 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 57 insertions(+)

diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 0e638dc654c1..f5c7790f7970 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -410,6 +410,17 @@ static int rdtgroup_kmode_cpus_show(struct kernfs_open_file *of,
 	return ret;
 }
 
+static void resctrl_configure_kmode_global(const struct cpumask *mask,
+					   struct rdtgroup *rdtgrp,
+					   bool enable)
+{
+	resctrl_arch_configure_kmode_global(mask, rdtgrp->closid,
+					    resctrl_kmode.ctrl == KMODE_ASSIGN,
+					    rdtgrp->mon.rmid,
+					    resctrl_kmode.mon == KMODE_ASSIGN,
+					    enable);
+}
+
 /*
  * Update the PGR_ASSOC MSR on all cpus in @cpu_mask,
  *
@@ -1114,6 +1125,48 @@ static int resctrl_kernel_mode_show(struct kernfs_open_file *of,
 	return 0;
 }
 
+/*
+ * resctrl_kmode_global_online_cpu() - Program kernel mode association for @cpu
+ * @cpu: CPU that has just been brought online
+ *
+ * When global_enable_per_cpu is active, @cpu joins that association and is
+ * programmed on.
+ */
+static void resctrl_kmode_global_online_cpu(unsigned int cpu)
+{
+	struct rdtgroup *rdtgrp = resctrl_kmode.rdtgrp;
+
+	if (resctrl_kmode.kmode_cur != RESCTRL_GLOBAL_ENABLE_PER_CPU || !rdtgrp)
+		return;
+
+	cpumask_set_cpu(cpu, &rdtgrp->kmode_cpu_mask);
+
+	resctrl_configure_kmode_global(cpumask_of(cpu), rdtgrp, true);
+}
+
+/*
+ * resctrl_kmode_global_offline_cpu() - Clear kernel mode association for @cpu
+ * @cpu: CPU being taken offline.
+ *
+ * When global_enable_per_cpu is active and @cpu is in kmode_cpu_mask, the
+ * association is disabled on @cpu and @cpu is removed from that mask. A CPU
+ * outside the mask is left unchanged.
+ */
+static void resctrl_kmode_global_offline_cpu(unsigned int cpu)
+{
+	struct rdtgroup *rdtgrp = resctrl_kmode.rdtgrp;
+
+	if (resctrl_kmode.kmode_cur != RESCTRL_GLOBAL_ENABLE_PER_CPU || !rdtgrp)
+		return;
+
+	if (!cpumask_test_cpu(cpu, &rdtgrp->kmode_cpu_mask))
+		return;
+
+	cpumask_clear_cpu(cpu, &rdtgrp->kmode_cpu_mask);
+
+	resctrl_configure_kmode_global(cpumask_of(cpu), rdtgrp, false);
+}
+
 void *rdt_kn_parent_priv(struct kernfs_node *kn)
 {
 	/*
@@ -4896,6 +4949,7 @@ void resctrl_online_cpu(unsigned int cpu)
 	mutex_lock(&rdtgroup_mutex);
 	/* The CPU is set in default rdtgroup after online. */
 	cpumask_set_cpu(cpu, &rdtgroup_default.cpu_mask);
+	resctrl_kmode_global_online_cpu(cpu);
 	mutex_unlock(&rdtgroup_mutex);
 }
 
@@ -4939,6 +4993,9 @@ void resctrl_offline_cpu(unsigned int cpu)
 		}
 	}
 
+	/* Clear any active kernel mode association on this CPU. */
+	resctrl_kmode_global_offline_cpu(cpu);
+
 	if (!l3->mon_capable)
 		goto out_unlock;
 
-- 
2.43.0

[PATCH v6 13/18] fs/resctrl: Deactivate kernel mode associations when a group is removed

[Babu Moger]

The active kernel mode association is backed by either a CTRL_MON or
MON group.

When the backing group is removed, or when resctrl is unmounted, clear
the programmed association and restore the default settings.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v6: Rewrite changelog (not a bugfix).
    Shorten the code comments.

v5: Took care of couple of Sashiko comments on previous patch.
    https://sashiko.dev/#/patchset/cover.1783461016.git.babu.moger@amd.com?part=10
    Re-wrote the changelog.
    Renamed rdtgroup_config_kmode_reset() to rdtgroup_kmode_deactivate().

v4: Re-wrote the changelog.
    Added the call free_all_child_rdtgrp() and rmdir_all_sub()
    Simplified the code comments.

v3: New patch to handle the kernel_mode clean up.
---
 fs/resctrl/rdtgroup.c | 44 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)

diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index f5c7790f7970..0cdc868a55b1 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -1167,6 +1167,36 @@ static void resctrl_kmode_global_offline_cpu(unsigned int cpu)
 	resctrl_configure_kmode_global(cpumask_of(cpu), rdtgrp, false);
 }
 
+/*
+ * Disable the association on @rdtgrp's kmode_cpu_mask, clear the mask, and
+ * hide the group's kmode_cpus files.
+ */
+static void rdtgroup_kmode_global_deactivate(struct rdtgroup *rdtgrp)
+{
+	resctrl_configure_kmode_global(&rdtgrp->kmode_cpu_mask, rdtgrp, false);
+	cpumask_clear(&rdtgrp->kmode_cpu_mask);
+}
+
+/*
+ * rdtgroup_kmode_global_detach() - Detach @rdtgrp from the global per-CPU mode
+ * @rdtgrp: Resctrl group being removed or torn down.
+ *
+ * If @rdtgrp backs the active association, disable it, clear kernel mode
+ * state, and restore RESCTRL_INHERIT_USER.
+ */
+static void rdtgroup_kmode_global_detach(struct rdtgroup *rdtgrp)
+{
+	if (resctrl_kmode.rdtgrp != rdtgrp)
+		return;
+
+	rdtgroup_kmode_global_deactivate(rdtgrp);
+
+	resctrl_kmode.rdtgrp = NULL;
+	resctrl_kmode.kmode_cur = RESCTRL_INHERIT_USER;
+	resctrl_kmode.ctrl = KMODE_ASSIGN;
+	resctrl_kmode.mon = KMODE_ASSIGN;
+}
+
 void *rdt_kn_parent_priv(struct kernfs_node *kn)
 {
 	/*
@@ -3228,6 +3258,7 @@ static void free_all_child_rdtgrp(struct rdtgroup *rdtgrp)
 
 	head = &rdtgrp->mon.crdtgrp_list;
 	list_for_each_entry_safe(sentry, stmp, head, mon.crdtgrp_list) {
+		rdtgroup_kmode_global_detach(sentry);
 		rdtgroup_unassign_cntrs(sentry);
 		free_rmid(sentry->closid, sentry->mon.rmid);
 		list_del(&sentry->mon.crdtgrp_list);
@@ -3265,6 +3296,7 @@ static void rmdir_all_sub(void)
 		cpumask_or(&rdtgroup_default.cpu_mask,
 			   &rdtgroup_default.cpu_mask, &rdtgrp->cpu_mask);
 
+		rdtgroup_kmode_global_detach(rdtgrp);
 		rdtgroup_unassign_cntrs(rdtgrp);
 
 		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP ||
@@ -3360,6 +3392,7 @@ static void resctrl_fs_teardown(void)
 		return;
 
 	rmdir_all_sub();
+	rdtgroup_kmode_global_detach(&rdtgroup_default);
 	rdtgroup_unassign_cntrs(&rdtgroup_default);
 	mon_put_kn_priv();
 	rdt_pseudo_lock_release();
@@ -4395,6 +4428,8 @@ static int rdtgroup_rmdir_mon(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)
 	u32 closid, rmid;
 	int cpu;
 
+	rdtgroup_kmode_global_detach(rdtgrp);
+
 	/* Give any tasks back to the parent group */
 	rdt_move_group_tasks(rdtgrp, prdtgrp, tmpmask);
 
@@ -4442,9 +4477,18 @@ static int rdtgroup_ctrl_remove(struct rdtgroup *rdtgrp)
 
 static int rdtgroup_rmdir_ctrl(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)
 {
+	struct rdtgroup *sentry;
 	u32 closid, rmid;
 	int cpu;
 
+	/*
+	 * A monitor group uses its parent's CLOSID. Detach a child that
+	 * backs the association before that CLOSID is freed.
+	 */
+	list_for_each_entry(sentry, &rdtgrp->mon.crdtgrp_list, mon.crdtgrp_list)
+		rdtgroup_kmode_global_detach(sentry);
+	rdtgroup_kmode_global_detach(rdtgrp);
+
 	/* Give any tasks back to the default group */
 	rdt_move_group_tasks(rdtgrp, &rdtgroup_default, tmpmask);
 
-- 
2.43.0

[PATCH v6 14/18] fs/resctrl: Control visibility of global per-CPU kernel mode group files

[Babu Moger]

The kmode_cpus and kmode_cpus_list files are specific to the
RESCTRL_GLOBAL_ENABLE_PER_CPU kernel mode. These files are created in a
hidden state when an rdtgroup is created.

Introduce resctrl_kmode_global_files_show() to manage the visibility of
these files based on whether the global per-CPU kernel mode is enabled or
disabled.

Suggested-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v6: Helper named for the global per-CPU files instead of unhiding every
    hidden file on the group.
    https://lore.kernel.org/lkml/34aa4824-810b-4c81-8188-f2f37e1dc277@intel.com/
---
 fs/resctrl/rdtgroup.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 0cdc868a55b1..98451b417fe3 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -1125,6 +1125,24 @@ static int resctrl_kernel_mode_show(struct kernfs_open_file *of,
 	return 0;
 }
 
+/* Show or hide kmode_cpus and kmode_cpus_list on @kn. */
+static void resctrl_kmode_global_files_show(struct kernfs_node *kn, bool show)
+{
+	struct kernfs_node *kn_file;
+
+	kn_file = kernfs_find_and_get(kn, "kmode_cpus");
+	if (kn_file) {
+		kernfs_show(kn_file, show);
+		kernfs_put(kn_file);
+	}
+
+	kn_file = kernfs_find_and_get(kn, "kmode_cpus_list");
+	if (kn_file) {
+		kernfs_show(kn_file, show);
+		kernfs_put(kn_file);
+	}
+}
+
 /*
  * resctrl_kmode_global_online_cpu() - Program kernel mode association for @cpu
  * @cpu: CPU that has just been brought online
@@ -1175,6 +1193,7 @@ static void rdtgroup_kmode_global_deactivate(struct rdtgroup *rdtgrp)
 {
 	resctrl_configure_kmode_global(&rdtgrp->kmode_cpu_mask, rdtgrp, false);
 	cpumask_clear(&rdtgrp->kmode_cpu_mask);
+	resctrl_kmode_global_files_show(rdtgrp->kn, false);
 }
 
 /*
-- 
2.43.0

[PATCH v6 15/18] fs/resctrl: Reject mode changes for groups backing kernel mode

[Babu Moger]

The RESCTRL_GLOBAL_ENABLE_PER_CPU kernel mode can be backed by only one
resource group at a time. The active kernel-mode association depends on
the group's mode and, for a monitor group, its parent control group.

Prevent mode changes for a resource group that backs the active kernel
mode. Also prevent mode changes to the parent control group of a backing
monitor group, and prevent a backing monitor group from being moved to a
different parent. These restrictions ensure that the active kernel-mode
association remains valid and consistent.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v6: New patch to fix sashiko feedback.
    https://sashiko.dev/#/patchset/cover.1787772750.git.babu.moger%40amd.com
---
 fs/resctrl/internal.h |  2 ++
 fs/resctrl/rdtgroup.c | 35 +++++++++++++++++++++++++++++++++++
 2 files changed, 37 insertions(+)

diff --git a/fs/resctrl/internal.h b/fs/resctrl/internal.h
index 6b8d14c4bd83..84238e9e8e15 100644
--- a/fs/resctrl/internal.h
+++ b/fs/resctrl/internal.h
@@ -462,6 +462,8 @@ ssize_t resctrl_mbm_assign_mode_write(struct kernfs_open_file *of, char *buf,
 void resctrl_bmec_files_show(struct rdt_resource *r, struct kernfs_node *l3_mon_kn,
 			     bool show);
 
+bool rdtgroup_has_kmode_global(struct rdtgroup *rdtgrp);
+
 int resctrl_num_mbm_cntrs_show(struct kernfs_open_file *of, struct seq_file *s, void *v);
 
 int resctrl_available_mbm_cntrs_show(struct kernfs_open_file *of, struct seq_file *s,
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 98451b417fe3..dcf9f26540d6 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -1738,6 +1738,27 @@ static bool rdtgroup_mode_test_exclusive(struct rdtgroup *rdtgrp)
 	return true;
 }
 
+/* True when @rdtgrp or one of its monitor groups backs the kernel mode. */
+bool rdtgroup_has_kmode_global(struct rdtgroup *rdtgrp)
+{
+	struct rdtgroup *child;
+
+	if (!resctrl_kmode.rdtgrp)
+		return false;
+
+	if (resctrl_kmode.rdtgrp == rdtgrp)
+		return true;
+
+	if (rdtgrp->type != RDTCTRL_GROUP)
+		return false;
+
+	list_for_each_entry(child, &rdtgrp->mon.crdtgrp_list, mon.crdtgrp_list)
+		if (child == resctrl_kmode.rdtgrp)
+			return true;
+
+	return false;
+}
+
 /*
  * rdtgroup_mode_write - Modify the resource group's mode
  */
@@ -1778,6 +1799,12 @@ static ssize_t rdtgroup_mode_write(struct kernfs_open_file *of,
 		goto out;
 	}
 
+	if (rdtgroup_has_kmode_global(rdtgrp)) {
+		rdt_last_cmd_puts("Cannot change mode for kernel-mode assigned group\n");
+		ret = -EINVAL;
+		goto out;
+	}
+
 	if (!strcmp(buf, "shareable")) {
 		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
 			ret = rdtgroup_locksetup_exit(rdtgrp);
@@ -4697,6 +4724,14 @@ static int rdtgroup_rename(struct kernfs_node *kn,
 		goto out;
 	}
 
+	/* Reject moving to different parent if group backs kernel-mode */
+	if (rdtgrp == resctrl_kmode.rdtgrp &&
+	    rdtgrp->mon.parent != new_prdtgrp) {
+		rdt_last_cmd_puts("Cannot move a group that backs kernel mode\n");
+		ret = -EPERM;
+		goto out;
+	}
+
 	/*
 	 * Allocate the cpumask for use in mongrp_reparent() to avoid the
 	 * possibility of failing to allocate it after kernfs_rename() has
-- 
2.43.0

[PATCH v6 16/18] fs/resctrl: Add interface to modify kernel mode via info/kernel_mode

[Babu Moger]

Kernel modes determine how resctrl allocation and monitoring associations
are applied when execution transitions between user mode and kernel mode.

Add write support to info/kernel_mode to allow user space to configure the
active kernel mode and any required resource group associations.

Document the interface in Documentation/filesystems/resctrl.rst.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v6: Use flattened resctrl_kmode and global_enable_per_cpu. Omitted
    parameters keep the values shown by reading the file. Drop the
    ctrl=inherit;mon=inherit no-op. Parse options with strsep(), as the
    other resctrl writers do, and split validation out of the write path.
    https://lore.kernel.org/lkml/3bc39dd7-b7b0-4203-a493-0bd872fdc644@intel.com/

v5: Adapt to the resctrl_kcfg caps and active split. Accept
    assign_global_enable_per_cpu:ctrl=;mon=;group=<ctrl>/<mon>/ writes.
    Treat both ctrl=inherit and mon=inherit as a no-op. Use group=// for
    the default control group. Show associated kmode files through
    resctrl_hidden_files_set_visible().
    Added kmode check in rdtgroup_locksetup_enter(), rdtgroup_mode_write()
    and rdtgroup_rename().
    Simplify kernel-doc for write and activate helpers.

v4: Validate input before tearing down the active association so failures
    retain the previous association. Allow assign_mon on CTRL_MON and MON
    groups; require a control group when control assignment is inherit.

v3: New patch to make info/kernel_mode writable.
---
 Documentation/filesystems/resctrl.rst |  28 +++
 fs/resctrl/rdtgroup.c                 | 274 +++++++++++++++++++++++++-
 2 files changed, 301 insertions(+), 1 deletion(-)

diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
index 73ccbb9594f5..bf9d34d6a335 100644
--- a/Documentation/filesystems/resctrl.rst
+++ b/Documentation/filesystems/resctrl.rst
@@ -575,6 +575,34 @@ conveyed in the error returns from file operations. E.g.
 	  [inherit_user]
 	  global_enable_per_cpu:ctrl=assign;mon=assign;group=//
 
+	Writes use the same line format as read, without square brackets and
+	with a trailing newline.
+
+	inherit_user is written without any parameters.
+
+	For global_enable_per_cpu, the parameters are optional. An omitted
+	ctrl=, mon=, or group= keeps the value currently shown. Parameters
+	may appear in any order. Empty values are rejected.
+
+	Selecting a new mode, parameter, or group replaces the active
+	association. A new global_enable_per_cpu association sets that
+	group's kmode_cpus to every online CPU. Errors are reported in
+	"info/last_cmd_status".
+	Example::
+
+	  # mkdir ctrl1
+	  # echo "global_enable_per_cpu:ctrl=assign;mon=assign;group=ctrl1//" \
+	         > info/kernel_mode
+
+	  # cat info/kernel_mode
+	  inherit_user
+	  [global_enable_per_cpu:ctrl=assign;mon=assign;group=ctrl1//]
+
+	  # echo "inherit_user" > info/kernel_mode
+	  # cat info/kernel_mode
+	  [inherit_user]
+	  global_enable_per_cpu:ctrl=assign;mon=assign;group=//
+
 Resource alloc and monitor groups
 =================================
 
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index dcf9f26540d6..73749e5a4ad0 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -1216,6 +1216,277 @@ static void rdtgroup_kmode_global_detach(struct rdtgroup *rdtgrp)
 	resctrl_kmode.mon = KMODE_ASSIGN;
 }
 
+/* Program @rdtgrp on every online CPU and show its kmode_cpus files. */
+static void rdtgroup_kmode_global_activate(struct rdtgroup *rdtgrp)
+{
+	cpumask_copy(&rdtgrp->kmode_cpu_mask, cpu_online_mask);
+	resctrl_configure_kmode_global(&rdtgrp->kmode_cpu_mask, rdtgrp, true);
+	resctrl_kmode_global_files_show(rdtgrp->kn, true);
+}
+
+/**
+ * rdtgroup_by_kmode_path() - Resolve a "<ctrl>/<mon>/" path to an rdtgroup
+ * @ctrl_name:	CTRL_MON group name, or "" for the default control group.
+ * @mon_name:	MON group name, or "" to select the control group itself.
+ *
+ * Matches the path syntax displayed by resctrl_kernel_mode_show():
+ *   "//"            - the default CTRL_MON group
+ *   "<ctrl>//"      - CTRL_MON group @ctrl_name
+ *   "/<mon>/"       - MON group @mon_name under the default control group
+ *   "<ctrl>/<mon>/" - MON group @mon_name under control group @ctrl_name
+ *
+ * Return: Pointer to the matching rdtgroup, or NULL if no such group exists.
+ */
+static struct rdtgroup *rdtgroup_by_kmode_path(const char *ctrl_name,
+					       const char *mon_name)
+{
+	struct rdtgroup *rdtg, *parent = &rdtgroup_default;
+
+	if (*ctrl_name) {
+		parent = NULL;
+		list_for_each_entry(rdtg, &rdt_all_groups, rdtgroup_list) {
+			if (rdtg->type != RDTCTRL_GROUP)
+				continue;
+			if (!strcmp(rdt_kn_name(rdtg->kn), ctrl_name)) {
+				parent = rdtg;
+				break;
+			}
+		}
+	}
+	if (!parent)
+		return NULL;
+
+	if (!*mon_name)
+		return parent;
+
+	list_for_each_entry(rdtg, &parent->mon.crdtgrp_list, mon.crdtgrp_list)
+		if (!strcmp(rdt_kn_name(rdtg->kn), mon_name))
+			return rdtg;
+	return NULL;
+}
+
+static int resctrl_kmode_parse_state(char *val, enum kmode_state *state)
+{
+	if (!strcmp(val, "assign")) {
+		*state = KMODE_ASSIGN;
+		return 0;
+	}
+	if (!strcmp(val, "inherit")) {
+		*state = KMODE_INHERIT;
+		return 0;
+	}
+	return -EINVAL;
+}
+
+static int resctrl_kmode_parse_group(char *group, struct rdtgroup **rdtgrp)
+{
+	struct rdtgroup *grp;
+	char *mon, *slash;
+
+	slash = strchr(group, '/');
+	if (!slash) {
+		rdt_last_cmd_puts("Group must be <CTRL_MON>/<MON>/\n");
+		return -EINVAL;
+	}
+	*slash = '\0';
+	mon = strim(slash + 1);
+
+	slash = strchr(mon, '/');
+	if (!slash || slash[1] != '\0') {
+		rdt_last_cmd_puts("Group must be <CTRL_MON>/<MON>/\n");
+		return -EINVAL;
+	}
+	*slash = '\0';
+
+	grp = rdtgroup_by_kmode_path(group, mon);
+	if (!grp) {
+		rdt_last_cmd_puts("Group not found\n");
+		return -EINVAL;
+	}
+	*rdtgrp = grp;
+	return 0;
+}
+
+static int resctrl_kmode_parse_parameters(char *params, enum kmode_state *ctrl,
+					  enum kmode_state *mon,
+					  struct rdtgroup **rdtgrp)
+{
+	char *key, *tok;
+
+	while ((tok = strsep(&params, ";"))) {
+		tok = strim(tok);
+		if (!*tok)
+			continue;
+
+		key = strsep(&tok, "=");
+		if (!tok) {
+			rdt_last_cmd_puts("Empty kernel mode parameters\n");
+			return -EINVAL;
+		}
+		key = strim(key);
+		tok = strim(tok);
+		if (!*key || !*tok) {
+			rdt_last_cmd_puts("Empty kernel mode parameters\n");
+			return -EINVAL;
+		}
+
+		if (!strcmp(key, "ctrl")) {
+			if (resctrl_kmode_parse_state(tok, ctrl)) {
+				rdt_last_cmd_puts("Invalid ctrl= parameter\n");
+				return -EINVAL;
+			}
+		} else if (!strcmp(key, "mon")) {
+			if (resctrl_kmode_parse_state(tok, mon)) {
+				rdt_last_cmd_puts("Invalid mon= parameter\n");
+				return -EINVAL;
+			}
+		} else if (!strcmp(key, "group")) {
+			if (resctrl_kmode_parse_group(tok, rdtgrp))
+				return -EINVAL;
+		} else {
+			rdt_last_cmd_puts("Unknown kernel mode parameter\n");
+			return -EINVAL;
+		}
+	}
+	return 0;
+}
+
+static int resctrl_kmode_global_validate(struct rdtgroup *rdtgrp,
+					 enum kmode_state ctrl,
+					 enum kmode_state mon)
+{
+	if (ctrl == KMODE_ASSIGN && !resctrl_arch_alloc_capable()) {
+		rdt_last_cmd_puts("ctrl=assign is not supported\n");
+		return -EINVAL;
+	}
+	if (mon == KMODE_ASSIGN && !resctrl_arch_mon_capable()) {
+		rdt_last_cmd_puts("mon=assign is not supported\n");
+		return -EINVAL;
+	}
+	if (mon == KMODE_INHERIT && rdtgrp->type == RDTMON_GROUP) {
+		rdt_last_cmd_puts("A monitor group requires mon=assign to use its RMID\n");
+		return -EINVAL;
+	}
+	if (rdtgrp->mode != RDT_MODE_SHAREABLE) {
+		rdt_last_cmd_puts("The group is not in shareable mode\n");
+		return -EINVAL;
+	}
+	if (rdtgrp->type == RDTMON_GROUP &&
+	    rdtgrp->mon.parent->mode != RDT_MODE_SHAREABLE) {
+		rdt_last_cmd_puts("The parent group is not in shareable mode\n");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static void resctrl_kmode_switch(enum resctrl_kernel_mode mode,
+				 struct rdtgroup *rdtgrp,
+				 enum kmode_state ctrl, enum kmode_state mon)
+{
+	if (resctrl_kmode.rdtgrp)
+		rdtgroup_kmode_global_deactivate(resctrl_kmode.rdtgrp);
+
+	resctrl_kmode.kmode_cur = mode;
+	if (mode != RESCTRL_GLOBAL_ENABLE_PER_CPU) {
+		resctrl_kmode.rdtgrp = NULL;
+		resctrl_kmode.ctrl = KMODE_ASSIGN;
+		resctrl_kmode.mon = KMODE_ASSIGN;
+		return;
+	}
+
+	resctrl_kmode.ctrl = ctrl;
+	resctrl_kmode.mon = mon;
+	resctrl_kmode.rdtgrp = rdtgrp;
+	rdtgroup_kmode_global_activate(rdtgrp);
+}
+
+/**
+ * resctrl_kernel_mode_write() - Set the active kernel mode
+ * @of:		kernfs open file
+ * @buf:	write buffer
+ * @nbytes:	length of @buf
+ * @off:	unused
+ *
+ * Return: @nbytes on success, negative errno on error.
+ */
+static ssize_t resctrl_kernel_mode_write(struct kernfs_open_file *of,
+					 char *buf, size_t nbytes, loff_t off)
+{
+	struct rdtgroup *rdtgrp = &rdtgroup_default;
+	enum resctrl_kernel_mode mode;
+	enum kmode_state ctrl, mon;
+	char *params;
+	int ret = 0;
+
+	if (!info_kn_lock(of->kn))
+		return -ENOENT;
+
+	rdt_last_cmd_clear();
+
+	if (nbytes == 0 || buf[nbytes - 1] != '\n') {
+		rdt_last_cmd_puts("kernel_mode: Invalid input\n");
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+	buf[nbytes - 1] = '\0';
+	buf = strim(buf);
+
+	params = strchr(buf, ':');
+	if (params)
+		*params++ = '\0';
+
+	for (mode = 0; mode < RESCTRL_NUM_KERNEL_MODES; mode++) {
+		if (!strcmp(buf, resctrl_kmode_str[mode]))
+			break;
+	}
+	if (mode == RESCTRL_NUM_KERNEL_MODES) {
+		rdt_last_cmd_puts("Unknown kernel mode\n");
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+	if (!test_bit(mode, resctrl_kmode.kmode_sup)) {
+		rdt_last_cmd_puts("Kernel mode not available\n");
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+
+	if (mode == RESCTRL_INHERIT_USER) {
+		if (params) {
+			rdt_last_cmd_puts("inherit_user has no parameters\n");
+			ret = -EINVAL;
+			goto out_unlock;
+		}
+		if (resctrl_kmode.kmode_cur != mode)
+			resctrl_kmode_switch(mode, NULL, KMODE_ASSIGN, KMODE_ASSIGN);
+		goto out_unlock;
+	}
+
+	/* Omitted parameters keep the values shown by reading the file. */
+	ctrl = resctrl_arch_alloc_capable() ? resctrl_kmode.ctrl : KMODE_INHERIT;
+	mon = resctrl_arch_mon_capable() ? resctrl_kmode.mon : KMODE_INHERIT;
+	if (resctrl_kmode.rdtgrp)
+		rdtgrp = resctrl_kmode.rdtgrp;
+
+	if (params) {
+		ret = resctrl_kmode_parse_parameters(params, &ctrl, &mon, &rdtgrp);
+		if (ret)
+			goto out_unlock;
+	}
+
+	ret = resctrl_kmode_global_validate(rdtgrp, ctrl, mon);
+	if (ret)
+		goto out_unlock;
+
+	if (resctrl_kmode.kmode_cur != mode || resctrl_kmode.rdtgrp != rdtgrp ||
+	    resctrl_kmode.ctrl != ctrl || resctrl_kmode.mon != mon)
+		resctrl_kmode_switch(mode, rdtgrp, ctrl, mon);
+
+out_unlock:
+	info_kn_unlock(of->kn);
+	return ret ?: nbytes;
+}
+
 void *rdt_kn_parent_priv(struct kernfs_node *kn)
 {
 	/*
@@ -2230,9 +2501,10 @@ static struct rftype res_common_files[] = {
 	},
 	{
 		.name		= "kernel_mode",
-		.mode		= 0444,
+		.mode		= 0644,
 		.kf_ops		= &rdtgroup_kf_single_ops,
 		.seq_show	= resctrl_kernel_mode_show,
+		.write		= resctrl_kernel_mode_write,
 		.fflags		= RFTYPE_TOP_INFO,
 	},
 	{
-- 
2.43.0

[PATCH v6 17/18] fs/resctrl: Allow user space to write kmode_cpus/kmode_cpus_list

[Babu Moger]

The RESCTRL_GLOBAL_ENABLE_PER_CPU kernel mode supports configuring
kernel-mode resource allocation and monitoring associations on a per-CPU
basis. The CPU scope of the association is exposed through kmode_cpus and
kmode_cpus_list.

Add write support for kmode_cpus and kmode_cpus_list to allow user space
to configure the CPUs associated with the active kernel-mode group.

Document the interface in Documentation/filesystems/resctrl.rst.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v6: Use flattened resctrl_kmode and global_enable_per_cpu.
    Fix the sashiko reported issue.
    https://sashiko.dev/#/patchset/cover.1787772750.git.babu.moger%40amd.com?part=15
    Update the changelog.

v5: Adapt to the resctrl_kcfg caps and active split. Use
    resctrl_kcfg.active.ctrl_mode and mon_mode in kmode_cpus_write().
    Use full ctrl=/mon= syntax in the kmode_cpus example.

v4: Empty masks are now allowed and updated masks are in rdtgroup->kmode_cpu_mask.
    Updated the changelog.

v3: New patch to add "kmode_cpus" and "kmode_cpus_list" to support
    kernel_modes.
---
 Documentation/filesystems/resctrl.rst |  30 +++++++
 fs/resctrl/rdtgroup.c                 | 114 +++++++++++++++++++++++++-
 2 files changed, 142 insertions(+), 2 deletions(-)

diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
index bf9d34d6a335..0d9dac38210c 100644
--- a/Documentation/filesystems/resctrl.rst
+++ b/Documentation/filesystems/resctrl.rst
@@ -671,6 +671,36 @@ All groups contain the following files:
 "cpus_list":
 	Just like "cpus", only using ranges of CPUs instead of bitmasks.
 
+"kmode_cpus":
+	Accessible only within the resource group currently associated with
+	the active kernel mode (see "info/kernel_mode").
+
+	Bitmask of the logical CPUs associated with the group's kernel mode.
+	When kernel mode is activated through info/kernel_mode, every currently
+	online CPU is included. CPUs that come online later are automatically
+	added to the association.
+
+	Writing a mask replaces the association's CPU mask. The association is
+	enabled on CPUs added to the mask and disabled on CPUs removed from
+	it. An empty mask disables the association on all currently online
+	CPUs, but CPUs that come online later are still automatically
+	associated. The mask must contain only online CPUs; masks naming offline
+	CPUs are rejected. Errors are reported in "info/last_cmd_status".
+	Example::
+
+	  # mkdir ctrl1
+	  # echo "global_enable_per_cpu:ctrl=assign;mon=assign;group=ctrl1//" \
+	        > info/kernel_mode
+	  # echo 0-3 > ctrl1/kmode_cpus_list
+	  # cat ctrl1/kmode_cpus
+	  f
+	  # cat ctrl1/kmode_cpus_list
+	  0-3
+
+"kmode_cpus_list":
+	Just like "kmode_cpus", only using ranges of CPUs instead of bitmasks.
+	Writable with the same semantics and restrictions as "kmode_cpus".
+
 
 When control is enabled all CTRL_MON groups will also contain:
 
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 73749e5a4ad0..5c812a9941cf 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -421,6 +421,114 @@ static void resctrl_configure_kmode_global(const struct cpumask *mask,
 					    enable);
 }
 
+/**
+ * kmode_cpus_write() - Update @rdtgrp's kmode_cpu_mask from @newmask
+ * @rdtgrp:	Resctrl group whose kmode_cpu_mask is being updated.
+ * @newmask:	New CPUs for @rdtgrp's kernel mode association.
+ * @tmpmask:	Caller-allocated scratch cpumask used to compute the
+ *		incremental enable/disable deltas.
+ *
+ * Enable the association on CPUs added to the mask and disable it on
+ * CPUs removed from the mask.
+ */
+static void kmode_cpus_write(struct rdtgroup *rdtgrp,
+			     cpumask_var_t newmask, cpumask_var_t tmpmask)
+{
+	/* CPUs dropped from this group: old & ~newmask. */
+	cpumask_andnot(tmpmask, &rdtgrp->kmode_cpu_mask, newmask);
+	if (!cpumask_empty(tmpmask))
+		resctrl_configure_kmode_global(tmpmask, rdtgrp, false);
+
+	/* CPUs newly added: newmask & ~old. */
+	cpumask_andnot(tmpmask, newmask, &rdtgrp->kmode_cpu_mask);
+	if (!cpumask_empty(tmpmask))
+		resctrl_configure_kmode_global(tmpmask, rdtgrp, true);
+
+	cpumask_copy(&rdtgrp->kmode_cpu_mask, newmask);
+}
+
+/**
+ * rdtgroup_kmode_cpus_write() - Write kmode_cpus or kmode_cpus_list
+ * @of: kernfs open file
+ * @buf: CPU bitmask or list from userspace
+ * @nbytes: length of @buf
+ * @off: unused
+ *
+ * Update the CPU mask of the group that backs the kernel mode.
+ *
+ * Return: @nbytes on success, negative errno on error.
+ */
+static ssize_t rdtgroup_kmode_cpus_write(struct kernfs_open_file *of,
+					 char *buf, size_t nbytes, loff_t off)
+{
+	cpumask_var_t tmpmask = CPUMASK_VAR_NULL, newmask = CPUMASK_VAR_NULL;
+	struct rdtgroup *rdtgrp;
+	int ret;
+
+	rdtgrp = rdtgroup_kn_lock_live(of->kn);
+	if (!rdtgrp) {
+		ret = -ENOENT;
+		goto out_unlock;
+	}
+
+	if (!buf) {
+		rdt_last_cmd_printf("%s: Invalid input\n",
+				    is_cpu_list(of) ? "kmode_cpus_list" :
+						      "kmode_cpus");
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+
+	if (!zalloc_cpumask_var(&tmpmask, GFP_KERNEL) ||
+	    !zalloc_cpumask_var(&newmask, GFP_KERNEL)) {
+		rdt_last_cmd_printf("%s: Kernel allocation failure\n",
+				    is_cpu_list(of) ? "kmode_cpus_list" :
+						      "kmode_cpus");
+		ret = -ENOMEM;
+		goto out_free;
+	}
+
+	if (resctrl_kmode.kmode_cur == RESCTRL_INHERIT_USER) {
+		rdt_last_cmd_puts("No active kernel-mode association\n");
+		ret = -EINVAL;
+		goto out_free;
+	}
+
+	if (resctrl_kmode.rdtgrp != rdtgrp) {
+		rdt_last_cmd_puts("Group not assigned to kernel-mode\n");
+		ret = -EINVAL;
+		goto out_free;
+	}
+
+	if (is_cpu_list(of))
+		ret = cpulist_parse(buf, newmask);
+	else
+		ret = cpumask_parse(buf, newmask);
+
+	if (ret) {
+		rdt_last_cmd_puts("Bad CPU list/mask\n");
+		goto out_free;
+	}
+
+	/* kernel-mode association is only programmed on online CPUs. */
+	cpumask_andnot(tmpmask, newmask, cpu_online_mask);
+	if (!cpumask_empty(tmpmask)) {
+		rdt_last_cmd_puts("Can only assign online CPUs\n");
+		ret = -EINVAL;
+		goto out_free;
+	}
+
+	kmode_cpus_write(rdtgrp, newmask, tmpmask);
+
+out_free:
+	free_cpumask_var(tmpmask);
+	free_cpumask_var(newmask);
+out_unlock:
+	rdtgroup_kn_unlock(of->kn);
+
+	return ret ?: nbytes;
+}
+
 /*
  * Update the PGR_ASSOC MSR on all cpus in @cpu_mask,
  *
@@ -2684,17 +2792,19 @@ static struct rftype res_common_files[] = {
 	},
 	{
 		.name		= "kmode_cpus",
-		.mode		= 0444,
+		.mode		= 0644,
 		.hidden		= true,
 		.kf_ops		= &rdtgroup_kf_single_ops,
+		.write		= rdtgroup_kmode_cpus_write,
 		.seq_show	= rdtgroup_kmode_cpus_show,
 		.fflags		= RFTYPE_BASE,
 	},
 	{
 		.name		= "kmode_cpus_list",
-		.mode		= 0444,
+		.mode		= 0644,
 		.hidden		= true,
 		.kf_ops		= &rdtgroup_kf_single_ops,
+		.write		= rdtgroup_kmode_cpus_write,
 		.seq_show	= rdtgroup_kmode_cpus_show,
 		.flags		= RFTYPE_FLAGS_CPUS_LIST,
 		.fflags		= RFTYPE_BASE,
-- 
2.43.0

[PATCH v6 18/18] fs/resctrl: Add kernel mode documentation and examples

[Babu Moger]

Kernel-mode configuration is exposed through info/kernel_mode,
kmode_cpus, and kmode_cpus_list.

Provide examples showing how to query supported modes, configure kernel
mode associations, and manage per-CPU kernel-mode associations.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v6: Renamed assign_global_enable_per_cpu to global_enable_per_cpu in the
    examples. Replaced "policy" with "mode" throughout.

v5: Update the example for the collapsed kernel mode model:
    inherit_user and assign_global_enable_per_cpu with ctrl=/mon=
    association options, and <CTRL_MON group>/<MON group>/ paths.

v4: New patch. Added examples in the documentation.
---
 Documentation/filesystems/resctrl.rst | 102 ++++++++++++++++++++++++++
 1 file changed, 102 insertions(+)

diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
index 0d9dac38210c..c0999d4d96d6 100644
--- a/Documentation/filesystems/resctrl.rst
+++ b/Documentation/filesystems/resctrl.rst
@@ -1888,6 +1888,108 @@ View the llc occupancy snapshot::
   11234000
 
 
+Examples on working with kernel_mode
+====================================
+The kernel-mode interface spans three files:
+
+- ``info/kernel_mode`` - select the global mode and associated group
+- ``kmode_cpus`` / ``kmode_cpus_list`` - view or adjust the CPU scope of the
+  active association (visible only on the associated group)
+
+The walkthrough below covers a typical workflow on a platform with PLZA
+(Privilege Level Zero Association) support:
+
+1. Discover supported modes.
+2. Associate ``global_enable_per_cpu`` with a control group.
+3. Narrow the CPU scope through ``kmode_cpus_list``.
+4. Move the association to a monitor group.
+5. Return to ``inherit_user``.
+
+a. Discover supported kernel modes
+::
+
+  # mount -t resctrl resctrl /sys/fs/resctrl/
+
+  # cat /sys/fs/resctrl/info/kernel_mode
+  [inherit_user]
+  global_enable_per_cpu:ctrl=assign;mon=assign;group=//
+
+``inherit_user`` is the active mode. Kernel work inherits the resource
+allocation and monitoring of the current user-space task. The second line
+shows ``global_enable_per_cpu`` is available; square brackets mark
+the active entry when read back after later steps.
+
+b. Associate kernel mode allocation with a control group
+::
+
+  # mkdir /sys/fs/resctrl/ctrl1
+  # echo "global_enable_per_cpu:ctrl=assign;mon=inherit;group=ctrl1//" > \
+        /sys/fs/resctrl/info/kernel_mode
+
+  # cat /sys/fs/resctrl/info/kernel_mode
+  inherit_user
+  [global_enable_per_cpu:ctrl=assign;mon=inherit;group=ctrl1//]
+
+``ctrl=assign`` uses ctrl1's allocation association. ``mon=inherit`` uses
+the monitoring association of the user-space task. The group path uses
+``<CTRL_MON group>/<MON group>/`` syntax; a control group alone is written
+as ``ctrl1//``.
+
+When the mode is active:
+
+- ``ctrl1/kmode_cpus`` and ``ctrl1/kmode_cpus_list`` become visible.
+- All currently online CPUs are associated initially.
+- CPUs that come online later are added to the association automatically.
+
+c. Narrow the association to CPUs 0-3
+::
+
+  # echo 0-3 > /sys/fs/resctrl/ctrl1/kmode_cpus_list
+  # cat /sys/fs/resctrl/ctrl1/kmode_cpus
+  f
+  # cat /sys/fs/resctrl/ctrl1/kmode_cpus_list
+  0-3
+
+The write replaces the association's CPU mask:
+
+- The association is enabled on CPUs added to the mask.
+- The association is disabled on CPUs removed from the mask.
+- Offline CPUs in the mask are rejected.
+- An empty mask disables the association on all currently online CPUs, but
+  later hotplug CPUs are still associated automatically.
+
+d. Move the association to a monitor group
+::
+
+  # mkdir /sys/fs/resctrl/ctrl1/mon_groups/mon1
+  # echo "global_enable_per_cpu:ctrl=assign;mon=assign;group=ctrl1/mon1/" > \
+        /sys/fs/resctrl/info/kernel_mode
+
+  # cat /sys/fs/resctrl/info/kernel_mode
+  inherit_user
+  [global_enable_per_cpu:ctrl=assign;mon=assign;group=ctrl1/mon1/]
+
+Kernel mode work uses ctrl1's allocation association (``ctrl=assign``) and
+mon1's monitoring association (``mon=assign``). A monitor group requires
+``mon=assign``; ``mon=inherit`` does not use that group's RMID and is
+rejected. Changing the associated group through ``info/kernel_mode``:
+
+- Resets the CPU scope to all currently online CPUs.
+- Hides ``kmode_cpus`` and ``kmode_cpus_list`` on the previous group.
+- Shows them under ``ctrl1/mon_groups/mon1/``.
+
+e. Return to inherit_user
+::
+
+  # echo "inherit_user" > /sys/fs/resctrl/info/kernel_mode
+  # cat /sys/fs/resctrl/info/kernel_mode
+  [inherit_user]
+  global_enable_per_cpu:ctrl=assign;mon=assign;group=//
+
+Writing ``inherit_user`` clears the active kernel-mode association and
+hides ``kmode_cpus`` and ``kmode_cpus_list`` because no group is
+associated with kernel mode.
+
 Examples on working with mbm_assign_mode
 ========================================
 
-- 
2.43.0