[PATCH v4 0/7] cgroup/cpuset: Fix partition transitions and invalidation

[PATCH v4 0/7] cgroup/cpuset: Fix partition transitions and invalidation

[Guopeng Zhang]

Several partition type-change and validation paths use effective_xcpus,
which also includes CPUs assigned to valid child partitions. As a result,
changing a parent between root and isolated can alter a child's isolation
accounting or fail housekeeping validation because of CPUs the parent does
not own.

A proposed parent CPU mask needs different treatment for children which
would become invalid under that mask. CPUs they can no longer retain must
be accounted for under their new owner, including in the parent's
housekeeping validation. Otherwise a parent can pass validation with a
boot-isolated CPU hidden behind a child which is about to become invalid.

There are related problems in the paths which apply CPU mask changes and
propagate partition invalidation. Child validation can use an old parent
exclusive mask, CPUs released by an invalid child can retain their former
isolation state, and a later hierarchy walk can incorrectly make an
out-of-mask child valid again. Configured masks are also published too late
for partition update paths which inspect or revisit the cpuset.

Factor the child invalidation checks into cs_partition_error() and keep the
complete parent exclusive mask and the remaining active mask explicit at
its call sites. Use directly owned CPUs for root/isolated type changes and
housekeeping validation, invalidate the appropriate isolated ancestor when
the last housekeeping CPU is returned, and release partition CPUs after a
failed type-change validation.

Use newly computed masks when invalidating children, update isolation
accounting according to the partition receiving each CPU, prevent
out-of-mask children from being revalidated, and publish configured masks
before partition updates can inspect or revisit the cpuset.

This v4 contains only the seven cpuset implementation patches. Patches 1-3
correspond to the three implementation patches in v3; patches 4-7 add the
related fixes for child invalidation, propagated isolation accounting and
publication of configured CPU masks. The selftests from v3 and the
additional regression tests are kept locally for validation. I plan to
submit them as a separate series after these fixes are accepted.

Testing:

The previously reported local results, using the selftests retained
outside this series, are:

- Partition type-change tests with isolcpus=domain,15: all applicable
  tests passed.
- Housekeeping CPU-return tests with
  nohz_full=1-14 isolcpus=domain,15: both cases passed.

Changes in v4:

- temporarily drop all selftest patches, including those posted in v3,
  as the current test design needs rework and some cases are redundant;
  retain them locally for validation and submit them as a separate series
  after the fixes are accepted;
- rename child_partition_error() to cs_partition_error(), document its
  inputs and keep the complete and remaining CPU masks explicit;
- add the related child-invalidation, propagated isolation-accounting and
  configured-mask publication fixes as patches 4-7;
- keep child containment, invalid-partition recovery and direct isolation
  accounting in the same patch around
  compute_partition_effective_cpumask();
- use the newly computed, unfiltered exclusive mask to validate children
  and account offline CPUs released by an invalid child;
- prevent a later hierarchy walk from revalidating an out-of-mask child;
- update isolation accounting when invalidation propagates through a
  member or invalid parent;
- publish cpus_allowed and exclusive_cpus before partition update paths
  inspect or revisit the cpuset.

Link: https://lore.kernel.org/all/20260902102615.79189-1-guopeng.zhang@linux.dev/

Changes in v3:

- factor the child partition validity rules into a preparatory patch and
  reuse them for trial ownership calculation;
- during trial validation, subtract only children which remain valid under
  the proposed parent CPU mask, covering both PERR_INVCPUS and PERR_NOCPUS;
- keep cs->effective_xcpus as the default isolation-accounting mask and
  use the directly owned mask only for a successful root/isolated type
  change;
- use an isolated child for the boot-isolated trial-validation tests, as a
  root child containing that CPU fails immediately with PERR_HKEEPING;
- make the isolated-ancestor walk safe when it reaches the top cpuset and
  clarify the member-transition handling;
- force hierarchy propagation after invalidating an isolated ancestor and
  verify task CPU masks and descendant partition states;
- move the test process back to the root cgroup and clean up nested
  housekeeping-return test cgroups on failure.

Link: https://lore.kernel.org/all/20260828095643.13395-1-guopeng.zhang@linux.dev/

Changes in v2:

- split the type-transition fixes from the original series;
- merge the child-owned accounting and validation changes;
- validate trial CPU masks against CPUs owned directly by the partition;
- handle a root child returning the last housekeeping CPU, including
  nested isolated ancestors;
- use the common partition-disable path after type-change validation
  failure;
- initialize boot-isolated CPU data during selftest setup;
- split the accounting, child-owned validation and housekeeping-return
  tests.

Link: https://lore.kernel.org/all/20260820124202.517160-1-guopeng.zhang@linux.dev/

Guopeng Zhang (7):
  cgroup/cpuset: Factor out child partition validation
  cgroup/cpuset: Account for child CPU ownership in partition changes
  cgroup/cpuset: Release CPUs when type-change validation fails
  cgroup/cpuset: Fix child invalidation after parent CPU changes
  cgroup/cpuset: Fix isolation accounting on propagated invalidation
  cgroup/cpuset: Publish cpus_allowed before partition updates
  cgroup/cpuset: Publish exclusive_cpus before partition updates

 kernel/cgroup/cpuset.c | 277 ++++++++++++++++++++++++++++++++++++-----
 1 file changed, 247 insertions(+), 30 deletions(-)

-- 
2.43.0

[PATCH v4 1/7] cgroup/cpuset: Factor out child partition validation

[Guopeng Zhang]

From: Guopeng Zhang <zhangguopeng@kylinos.cn>

compute_partition_effective_cpumask() checks whether each valid child
partition remains covered by the parent exclusive CPU mask and whether it
would consume all remaining active CPUs of a populated parent.

Factor these two checks into cs_partition_error() so the same rules can be
reused when evaluating a proposed parent configuration. This is a
preparatory refactoring with no intended functional change.

Signed-off-by: Guopeng Zhang <zhangguopeng@kylinos.cn>
---
 kernel/cgroup/cpuset.c | 47 ++++++++++++++++++++++++++++++++++--------
 1 file changed, 38 insertions(+), 9 deletions(-)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 8f24171b6055..a2514fcb1144 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -2085,6 +2085,37 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,
 	return 0;
 }
 
+/**
+ * cs_partition_error - Return a cpuset partition invalidation error
+ * @cs: Partition being evaluated
+ * @parent_xcpus: Parent's complete effective exclusive CPU mask, including
+ *                offline CPUs
+ * @remaining_ecpus: Parent's active effective CPUs remaining before @cs is
+ *                   evaluated
+ * @parent_populated: Whether the parent partition contains tasks
+ *
+ * @remaining_ecpus may still contain active CPUs assigned to @cs and to
+ * children that have not yet been evaluated. It excludes only CPUs assigned
+ * to previously evaluated children that remain valid.
+ *
+ * Return: The error that would invalidate @cs, or PERR_NONE
+ */
+static enum prs_errcode
+cs_partition_error(struct cpuset *cs,
+		   const struct cpumask *parent_xcpus,
+		   const struct cpumask *remaining_ecpus,
+		   bool parent_populated)
+{
+	if (!cpumask_subset(cs->effective_xcpus, parent_xcpus))
+		return PERR_INVCPUS;
+
+	if (parent_populated &&
+	    cpumask_subset(remaining_ecpus, cs->effective_xcpus))
+		return PERR_NOCPUS;
+
+	return PERR_NONE;
+}
+
 /**
  * compute_partition_effective_cpumask - compute effective_cpus for partition
  * @cs: partition root cpuset
@@ -2121,6 +2152,8 @@ static void compute_partition_effective_cpumask(struct cpuset *cs,
 
 	rcu_read_lock();
 	cpuset_for_each_child(child, css, cs) {
+		enum prs_errcode child_err;
+
 		if (!is_partition_valid(child))
 			continue;
 
@@ -2129,15 +2162,11 @@ static void compute_partition_effective_cpumask(struct cpuset *cs,
 		 * partition root.
 		 */
 		WARN_ON_ONCE(is_remote_partition(child));
-		WRITE_ONCE(child->prs_err, 0);
-		if (!cpumask_subset(child->effective_xcpus,
-				    cs->effective_xcpus))
-			WRITE_ONCE(child->prs_err, PERR_INVCPUS);
-		else if (populated &&
-			 cpumask_subset(new_ecpus, child->effective_xcpus))
-			WRITE_ONCE(child->prs_err, PERR_NOCPUS);
-
-		if (child->prs_err) {
+		child_err = cs_partition_error(child, cs->effective_xcpus,
+					       new_ecpus, populated);
+		WRITE_ONCE(child->prs_err, child_err);
+
+		if (child_err) {
 			int old_prs = child->partition_root_state;
 
 			/*
-- 
2.43.0

[PATCH v4 2/7] cgroup/cpuset: Account for child CPU ownership in partition changes

[Guopeng Zhang]

From: Guopeng Zhang <zhangguopeng@kylinos.cn>

effective_xcpus includes CPUs granted to valid child partitions. A change
to the parent must not apply its isolation state or housekeeping checks to
CPUs which remain owned by those children.

For example, on a cgroup v2 system with CPUs 0-3 online:

    cd /sys/fs/cgroup
    echo +cpuset > cgroup.subtree_control
    mkdir type-repro
    echo 1-3 > type-repro/cpuset.cpus
    echo isolated > type-repro/cpuset.cpus.partition
    echo +cpuset > type-repro/cgroup.subtree_control
    mkdir type-repro/child
    echo 2-3 > type-repro/child/cpuset.cpus
    echo isolated > type-repro/child/cpuset.cpus.partition
    echo root > type-repro/cpuset.cpus.partition
    cat cpuset.cpus.isolated

The isolated mask should still contain CPUs 2-3 after the parent becomes a
root partition. Without this change, those CPUs are removed even though
the child remains isolated.

Compute the CPUs owned directly by a partition by excluding CPUs granted
to valid children. When validating a trial parent mask, exclude only CPUs
granted to children that will remain valid under that mask. Reuse the child
validation rules so PERR_INVCPUS and PERR_NOCPUS are handled consistently.

Use the directly owned mask for root/isolated type changes and housekeeping
validation. If a root child returns the last housekeeping CPU to an
isolated parent, invalidate the outermost isolated ancestor so the child
can still become a member without violating the housekeeping constraint.

Force the hierarchy update from the invalidated ancestor. Otherwise an
unchanged member cpuset can cause its subtree to be skipped, leaving task
CPU masks or descendant partition states stale.

Link: https://sashiko.dev/#/patchset/20260820124202.517160-1-guopeng.zhang%40linux.dev?part=6
Link: https://sashiko.dev/#/patchset/20260828095643.13395-1-guopeng.zhang@linux.dev?part=1
Link: https://sashiko.dev/#/patchset/20260902102615.79189-1-guopeng.zhang@linux.dev?part=2
Fixes: 4a74e418881f ("cgroup/cpuset: Check partition conflict with housekeeping setup")
Fixes: 11e5f407b64a ("cgroup/cpuset: Keep track of CPUs in isolated partitions")
Fixes: 103b08709e8a ("cgroup/cpuset: Fail if isolated and nohz_full don't leave any housekeeping")
Fixes: b1034a690129 ("cgroup/cpuset: Ensure domain isolated CPUs stay in root or isolated partition")
Signed-off-by: Guopeng Zhang <zhangguopeng@kylinos.cn>
---
 kernel/cgroup/cpuset.c | 142 ++++++++++++++++++++++++++++++++++++++---
 1 file changed, 132 insertions(+), 10 deletions(-)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index a2514fcb1144..994ddb79272d 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -2184,6 +2184,44 @@ static void compute_partition_effective_cpumask(struct cpuset *cs,
 	rcu_read_unlock();
 }
 
+/*
+ * Compute CPUs owned directly by a partition under @parent_xcpus by excluding
+ * CPUs granted to children that remain valid under that mask.
+ */
+static void compute_partition_owned_cpumask(struct cpuset *cs,
+					    const struct cpumask *parent_xcpus,
+					    struct cpumask *owned_cpus,
+					    struct cpumask *remaining_ecpus)
+{
+	struct cgroup_subsys_state *css;
+	struct cpuset *child;
+	bool populated = partition_is_populated(cs, NULL);
+
+	lockdep_assert_held(&cpuset_mutex);
+	cpumask_copy(owned_cpus, parent_xcpus);
+	cpumask_and(remaining_ecpus, parent_xcpus, cpu_active_mask);
+
+	rcu_read_lock();
+	cpuset_for_each_child(child, css, cs) {
+		if (!is_partition_valid(child))
+			continue;
+
+		/*
+		 * A child that would become invalid under the proposed
+		 * configuration cannot retain ownership of its CPUs.
+		 */
+		if (cs_partition_error(child, parent_xcpus,
+				       remaining_ecpus, populated))
+			continue;
+
+		cpumask_andnot(owned_cpus, owned_cpus,
+			       child->effective_xcpus);
+		cpumask_andnot(remaining_ecpus, remaining_ecpus,
+			       child->effective_xcpus);
+	}
+	rcu_read_unlock();
+}
+
 /*
  * update_cpumasks_hier - Update effective cpumasks and tasks in the subtree
  * @cs:  the cpuset to consider
@@ -2424,13 +2462,18 @@ static int parse_cpuset_cpulist(const char *buf, struct cpumask *out_mask)
  * validate_partition - Validate a cpuset partition configuration
  * @cs: The cpuset to validate
  * @trialcs: The trial cpuset containing proposed configuration changes
+ * @owned_cpus: Scratch mask for CPUs owned directly by the trial partition
+ * @remaining_ecpus: Scratch mask used to predict valid child partitions
  *
  * If any validation check fails, the appropriate error code is set in the
  * cpuset's prs_err field.
  *
  * Return: PRS error code (0 if valid, non-zero error code if invalid)
  */
-static enum prs_errcode validate_partition(struct cpuset *cs, struct cpuset *trialcs)
+static enum prs_errcode validate_partition(struct cpuset *cs,
+					   struct cpuset *trialcs,
+					   struct cpumask *owned_cpus,
+					   struct cpumask *remaining_ecpus)
 {
 	struct cpuset *parent = parent_cs(cs);
 
@@ -2440,8 +2483,10 @@ static enum prs_errcode validate_partition(struct cpuset *cs, struct cpuset *tri
 	if (cpumask_empty(trialcs->effective_xcpus))
 		return PERR_INVCPUS;
 
+	compute_partition_owned_cpumask(cs, trialcs->effective_xcpus,
+					owned_cpus, remaining_ecpus);
 	if (prstate_housekeeping_conflict(trialcs->partition_root_state,
-					  trialcs->effective_xcpus))
+					  owned_cpus))
 		return PERR_HKEEPING;
 
 	if (tasks_nocpu_error(parent, cs, trialcs->effective_xcpus))
@@ -2467,7 +2512,8 @@ static void partition_cpus_change(struct cpuset *cs, struct cpuset *trialcs,
 	if (cs_is_member(cs))
 		return;
 
-	prs_err = validate_partition(cs, trialcs);
+	prs_err = validate_partition(cs, trialcs, tmp->new_cpus,
+				     tmp->addmask);
 	if (prs_err) {
 		WRITE_ONCE(cs->prs_err, prs_err);
 		trialcs->prs_err = prs_err;
@@ -2946,6 +2992,49 @@ int cpuset_update_flag(cpuset_flagbits_t bit, struct cpuset *cs,
 	return err;
 }
 
+/*
+ * Invalidate the highest isolated partition that contains @cs.
+ *
+ * A root partition returning CPUs to an isolated parent can consume the last
+ * housekeeping CPU. Invalidating the highest isolated ancestor lets the
+ * subsequent hierarchy update propagate invalidation down the chain and
+ * return the CPUs to a root partition.
+ */
+static struct cpuset *invalidate_isolated_ancestor(struct cpuset *cs,
+						   struct tmpmasks *tmp)
+{
+	struct cpuset *ancestor = parent_cs(cs);
+	struct cpuset *parent;
+	int err;
+
+	lockdep_assert_held(&cpuset_mutex);
+	if (WARN_ON_ONCE(!ancestor))
+		return NULL;
+
+	while ((ancestor != &top_cpuset) &&
+	       !is_remote_partition(ancestor)) {
+		parent = parent_cs(ancestor);
+		if (!parent ||
+		    parent->partition_root_state != PRS_ISOLATED)
+			break;
+		ancestor = parent;
+	}
+
+	if (WARN_ON_ONCE(ancestor == &top_cpuset))
+		return NULL;
+
+	WRITE_ONCE(ancestor->prs_err, PERR_HKEEPING);
+	if (is_remote_partition(ancestor)) {
+		remote_partition_disable(ancestor, tmp);
+	} else {
+		err = update_parent_effective_cpumask(ancestor,
+						      partcmd_invalidate, NULL, tmp);
+		WARN_ON_ONCE(err);
+	}
+
+	return ancestor;
+}
+
 /**
  * update_prstate - update partition_root_state
  * @cs: the cpuset to update
@@ -2958,6 +3047,8 @@ static int update_prstate(struct cpuset *cs, int new_prs)
 {
 	int err = PERR_NONE, old_prs = cs->partition_root_state;
 	struct cpuset *parent = parent_cs(cs);
+	struct cpuset *invalidated = NULL;
+	struct cpumask *isolcpus_update_cpus = cs->effective_xcpus;
 	struct tmpmasks tmpmask;
 	bool isolcpus_updated = false;
 
@@ -3014,19 +3105,38 @@ static int update_prstate(struct cpuset *cs, int new_prs)
 	} else if (old_prs && new_prs) {
 		/*
 		 * A change in load balance state only, no change in cpumasks.
-		 * Need to update isolated_cpus.
+		 * Need to update isolated_cpus for CPUs owned by this partition,
+		 * excluding CPUs distributed to valid child partitions.
 		 */
+		compute_partition_owned_cpumask(cs, cs->effective_xcpus,
+						tmpmask.new_cpus,
+						tmpmask.addmask);
 		if (((new_prs == PRS_ISOLATED) &&
-		     !isolated_cpus_can_update(cs->effective_xcpus, NULL)) ||
-		    prstate_housekeeping_conflict(new_prs, cs->effective_xcpus))
+		     !isolated_cpus_can_update(tmpmask.new_cpus, NULL)) ||
+		    prstate_housekeeping_conflict(new_prs, tmpmask.new_cpus)) {
 			err = PERR_HKEEPING;
-		else
+		} else {
+			/*
+			 * Only directly owned CPUs change isolation state for a
+			 * successful root <-> isolated type change.
+			 */
+			isolcpus_update_cpus = tmpmask.new_cpus;
 			isolcpus_updated = true;
+		}
 	} else {
 		/*
 		 * Switching back to member is always allowed even if it
-		 * disables child partitions.
+		 * disables child partitions. If returning CPUs to an isolated
+		 * parent would consume the last housekeeping CPU, invalidate
+		 * the outermost isolated ancestor and return its CPUs instead.
 		 */
+		if (old_prs == PRS_ROOT &&
+		    parent->partition_root_state == PRS_ISOLATED &&
+		    !isolated_cpus_can_update(cs->effective_xcpus, NULL))
+			invalidated = invalidate_isolated_ancestor(cs, &tmpmask);
+		if (invalidated)
+			goto out;
+
 		if (is_remote_partition(cs))
 			remote_partition_disable(cs, &tmpmask);
 		else
@@ -3054,11 +3164,23 @@ static int update_prstate(struct cpuset *cs, int new_prs)
 	if (!is_partition_valid(cs))
 		reset_partition_data(cs);
 	else if (isolcpus_updated)
-		isolated_cpus_update(old_prs, new_prs, cs->effective_xcpus);
+		isolated_cpus_update(old_prs, new_prs,
+				     isolcpus_update_cpus);
 	spin_unlock_irq(&callback_lock);
 
 	/* Force update if switching back to member & update effective_xcpus */
-	update_cpumasks_hier(cs, &tmpmask, !new_prs);
+	if (invalidated) {
+		/*
+		 * Ancestor invalidation changes the partition hierarchy. Force the
+		 * traversal so an unchanged member cpuset does not cause its subtree
+		 * to be skipped.
+		 */
+		update_cpumasks_hier(invalidated, &tmpmask, true);
+		update_partition_sd_lb(invalidated, PRS_ISOLATED);
+		notify_partition_change(invalidated, PRS_ISOLATED);
+	} else {
+		update_cpumasks_hier(cs, &tmpmask, !new_prs);
+	}
 
 	/* A newly created partition must have effective_xcpus set */
 	WARN_ON_ONCE(!old_prs && (new_prs > 0)
-- 
2.43.0

[PATCH v4 3/7] cgroup/cpuset: Release CPUs when type-change validation fails

[Guopeng Zhang]

From: Guopeng Zhang <zhangguopeng@kylinos.cn>

When housekeeping validation fails during a root/isolated type change,
update_prstate() marks the requested state invalid without running the
partition-disable path. The failed partition's effective_xcpus may be
cleared, but its CPUs remain unavailable to the partition which owns the
invalidated subtree.

This can be reproduced on a cgroup v2 system booted with
isolcpus=domain,15:

    cd /sys/fs/cgroup
    echo +cpuset > cgroup.subtree_control
    mkdir type-fail-repro
    echo 15 > type-fail-repro/cpuset.cpus
    echo isolated > type-fail-repro/cpuset.cpus.partition
    echo root > type-fail-repro/cpuset.cpus.partition
    cat type-fail-repro/cpuset.cpus.partition
    cat cpuset.cpus.effective

The requested root state is recorded as invalid, but CPU 15 remains
unavailable to the top cpuset.

Run the common partition-disable path when housekeeping validation fails.
Disable remote partitions with remote_partition_disable() and return local
partition CPUs to their parent. If that would consume the last housekeeping
CPU, invalidate the outermost isolated ancestor instead.

Fixes: 103b08709e8a ("cgroup/cpuset: Fail if isolated and nohz_full don't leave any housekeeping")
Fixes: b1034a690129 ("cgroup/cpuset: Ensure domain isolated CPUs stay in root or isolated partition")
Signed-off-by: Guopeng Zhang <zhangguopeng@kylinos.cn>
---
 kernel/cgroup/cpuset.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 994ddb79272d..7e8b167a29ed 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -3050,6 +3050,7 @@ static int update_prstate(struct cpuset *cs, int new_prs)
 	struct cpuset *invalidated = NULL;
 	struct cpumask *isolcpus_update_cpus = cs->effective_xcpus;
 	struct tmpmasks tmpmask;
+	bool disable_partition = false;
 	bool isolcpus_updated = false;
 
 	if (old_prs == new_prs)
@@ -3115,6 +3116,7 @@ static int update_prstate(struct cpuset *cs, int new_prs)
 		     !isolated_cpus_can_update(tmpmask.new_cpus, NULL)) ||
 		    prstate_housekeeping_conflict(new_prs, tmpmask.new_cpus)) {
 			err = PERR_HKEEPING;
+			disable_partition = true;
 		} else {
 			/*
 			 * Only directly owned CPUs change isolation state for a
@@ -3130,6 +3132,10 @@ static int update_prstate(struct cpuset *cs, int new_prs)
 		 * parent would consume the last housekeeping CPU, invalidate
 		 * the outermost isolated ancestor and return its CPUs instead.
 		 */
+		disable_partition = true;
+	}
+
+	if (disable_partition) {
 		if (old_prs == PRS_ROOT &&
 		    parent->partition_root_state == PRS_ISOLATED &&
 		    !isolated_cpus_can_update(cs->effective_xcpus, NULL))
-- 
2.43.0

[PATCH v4 4/7] cgroup/cpuset: Fix child invalidation after parent CPU changes

[Guopeng Zhang]

From: Guopeng Zhang <zhangguopeng@kylinos.cn>

compute_partition_effective_cpumask() recomputes a partition's exclusive
CPU mask before walking its children, but checks child containment against
cs->effective_xcpus. During an update, that field can still describe an
earlier point in the update, allowing a child outside the newly computed
mask to remain valid.

Use the newly computed exclusive mask for the containment check. Keep this
mask separate from the active-only effective mask because offline CPUs
remain part of the partition's CPU ownership.

After a child is invalidated, update_cpumasks_hier() can revisit it by
calling update_parent_effective_cpumask() with partcmd_update. The
invalid-partition recovery path is currently entered only when the child's
CPUs are already a subset of its parent's effective exclusive mask.
Otherwise part_error remains clear and the subsequent state transition
makes the child valid again. Enter the recovery path for every non-empty
CPU mask and report PERR_INVCPUS when the child is still outside the parent
mask.

The invalidation path also calls make_partition_invalid() without updating
isolated_cpus for the CPUs released by the child. Account each released CPU
according to its new owner. When the parent remains valid and owns the CPU,
use its partition state; otherwise use the state of the nearest valid
partition ancestor. Use the unfiltered exclusive mask so offline CPUs are
included in the accounting.

Fixes: 11e5f407b64a ("cgroup/cpuset: Keep track of CPUs in isolated partitions")
Fixes: 0c7f293efc87 ("cgroup/cpuset: Add cpuset.cpus.exclusive.effective for v2")
Signed-off-by: Guopeng Zhang <zhangguopeng@kylinos.cn>
---
 kernel/cgroup/cpuset.c | 51 +++++++++++++++++++++++++++++++++---------
 1 file changed, 41 insertions(+), 10 deletions(-)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 7e8b167a29ed..7ba26b924086 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -1281,6 +1281,17 @@ static bool isolated_cpu_update(int new_prs, int cpu)
 	return true;
 }
 
+/* Return the nearest valid partition ancestor of @cs. */
+static struct cpuset *partition_owner(struct cpuset *cs)
+{
+	struct cpuset *owner = parent_cs(cs);
+
+	lockdep_assert_held(&cpuset_mutex);
+	while (!is_partition_valid(owner))
+		owner = parent_cs(owner);
+	return owner;
+}
+
 /*
  * isolated_cpus_update - Update the isolated_cpus mask
  * @old_prs: old partition_root_state
@@ -1976,12 +1987,16 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,
 				adding = cpumask_and(tmp->addmask,
 						     cs->effective_xcpus,
 						     parent->effective_xcpus);
-		} else if (is_partition_invalid(cs) && !cpumask_empty(xcpus) &&
-			   cpumask_subset(xcpus, parent->effective_xcpus)) {
+		} else if (is_partition_invalid(cs) && !cpumask_empty(xcpus)) {
 			struct cgroup_subsys_state *css;
 			struct cpuset *child;
 			bool exclusive = true;
 
+			if (!cpumask_subset(xcpus, parent->effective_xcpus)) {
+				part_error = PERR_INVCPUS;
+				goto write_error;
+			}
+
 			/*
 			 * Convert invalid partition to valid has to
 			 * pass the cpu exclusivity test.
@@ -2120,6 +2135,7 @@ cs_partition_error(struct cpuset *cs,
  * compute_partition_effective_cpumask - compute effective_cpus for partition
  * @cs: partition root cpuset
  * @new_ecpus: previously computed effective_cpus to be updated
+ * @new_xcpus: scratch mask for the new effective_xcpus
  *
  * Compute the effective_cpus of a partition root by scanning effective_xcpus
  * of child partition roots and excluding their effective_xcpus.
@@ -2133,7 +2149,8 @@ cs_partition_error(struct cpuset *cs,
  * Note that rcu_read_lock() is assumed to be held.
  */
 static void compute_partition_effective_cpumask(struct cpuset *cs,
-						struct cpumask *new_ecpus)
+						struct cpumask *new_ecpus,
+						struct cpumask *new_xcpus)
 {
 	struct cgroup_subsys_state *css;
 	struct cpuset *child;
@@ -2147,8 +2164,8 @@ static void compute_partition_effective_cpumask(struct cpuset *cs,
 	 *  2) All the effective_cpus will be used up and cp
 	 *     has tasks
 	 */
-	compute_excpus(cs, new_ecpus);
-	cpumask_and(new_ecpus, new_ecpus, cpu_active_mask);
+	compute_excpus(cs, new_xcpus);
+	cpumask_and(new_ecpus, new_xcpus, cpu_active_mask);
 
 	rcu_read_lock();
 	cpuset_for_each_child(child, css, cs) {
@@ -2162,17 +2179,31 @@ static void compute_partition_effective_cpumask(struct cpuset *cs,
 		 * partition root.
 		 */
 		WARN_ON_ONCE(is_remote_partition(child));
-		child_err = cs_partition_error(child, cs->effective_xcpus,
+		child_err = cs_partition_error(child, new_xcpus,
 					       new_ecpus, populated);
 		WRITE_ONCE(child->prs_err, child_err);
 
 		if (child_err) {
 			int old_prs = child->partition_root_state;
+			int parent_prs = cs->partition_root_state;
+			int owner_prs = partition_owner(cs)->partition_root_state;
+			int cpu;
 
 			/*
-			 * Invalidate child partition
+			 * Account each released CPU according to whether it is now
+			 * owned by the parent or by the partition that owns the parent.
 			 */
 			spin_lock_irq(&callback_lock);
+			for_each_cpu(cpu, child->effective_xcpus) {
+				int new_prs = parent_prs > 0 &&
+					      cpumask_test_cpu(cpu, new_xcpus)
+					      ? parent_prs : owner_prs;
+
+				if (old_prs == new_prs)
+					continue;
+				if (isolated_cpu_update(new_prs, cpu))
+					update_housekeeping = true;
+			}
 			make_partition_invalid(child);
 			spin_unlock_irq(&callback_lock);
 			notify_partition_change(child, old_prs);
@@ -2274,7 +2305,7 @@ static void update_cpumasks_hier(struct cpuset *cs, struct tmpmasks *tmp,
 		}
 
 		if (remote || (is_partition_valid(parent) && is_partition_valid(cp)))
-			compute_partition_effective_cpumask(cp, tmp->new_cpus);
+			compute_partition_effective_cpumask(cp, tmp->new_cpus, tmp->addmask);
 		else
 			compute_effective_cpumask(tmp->new_cpus, cp, parent);
 
@@ -4111,7 +4142,7 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs, struct tmpmasks *tmp)
 	 */
 	remote = is_remote_partition(cs);
 	if (remote || (is_partition_valid(cs) && is_partition_valid(parent)))
-		compute_partition_effective_cpumask(cs, &new_cpus);
+		compute_partition_effective_cpumask(cs, &new_cpus, tmp->addmask);
 
 	if (remote && (cpumask_empty(subpartitions_cpus) ||
 			(cpumask_empty(&new_cpus) &&
@@ -4146,7 +4177,7 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs, struct tmpmasks *tmp)
 	if (partcmd >= 0) {
 		update_parent_effective_cpumask(cs, partcmd, NULL, tmp);
 		if ((partcmd == partcmd_invalidate) || is_partition_valid(cs)) {
-			compute_partition_effective_cpumask(cs, &new_cpus);
+			compute_partition_effective_cpumask(cs, &new_cpus, tmp->addmask);
 			cpuset_force_rebuild();
 		}
 	}
-- 
2.43.0

[PATCH v4 5/7] cgroup/cpuset: Fix isolation accounting on propagated invalidation

[Guopeng Zhang]

From: Guopeng Zhang <zhangguopeng@kylinos.cn>

update_cpumasks_hier() invalidates a local partition when its parent
becomes a member or an invalid partition. Its CPUs return to the nearest
valid partition ancestor, but isolated_cpus still reflects the old
partition type.

This can be reproduced on a cgroup v2 system with CPUs 0-3 online:

    cd /sys/fs/cgroup
    echo +cpuset > cgroup.subtree_control
    mkdir propagation-repro
    echo 0-3 > propagation-repro/cpuset.cpus
    echo root > propagation-repro/cpuset.cpus.partition
    echo +cpuset > propagation-repro/cgroup.subtree_control
    mkdir propagation-repro/child
    echo 2-3 > propagation-repro/child/cpuset.cpus
    echo isolated > propagation-repro/child/cpuset.cpus.partition
    echo member > propagation-repro/cpuset.cpus.partition
    cat cpuset.cpus.isolated

Without this fix, CPUs 2-3 remain isolated even though the child is
invalid and its CPUs have returned to the top partition. The expected
dynamic isolated mask is empty.

Update isolated_cpus before resetting a local partition whose state
changes from valid to invalid below an invalid parent. Remote partitions
are excluded because remote_partition_disable() already updates their
isolated CPU accounting.

Fixes: 11e5f407b64a ("cgroup/cpuset: Keep track of CPUs in isolated partitions")
Signed-off-by: Guopeng Zhang <zhangguopeng@kylinos.cn>
---
 kernel/cgroup/cpuset.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 7ba26b924086..bbc4868f026b 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -2277,9 +2277,12 @@ static void update_cpumasks_hier(struct cpuset *cs, struct tmpmasks *tmp,
 	cpuset_for_each_descendant_pre(cp, pos_css, cs) {
 		struct cpuset *parent = parent_cs(cp);
 		bool remote = is_remote_partition(cp);
+		bool was_remote = remote;
 		bool update_parent = false;
+		int owner_prs;
 
 		old_prs = new_prs = cp->partition_root_state;
+		owner_prs = old_prs;
 
 		/*
 		 * For child remote partition root (!= cs), we need to call
@@ -2379,7 +2382,18 @@ static void update_cpumasks_hier(struct cpuset *cs, struct tmpmasks *tmp,
 			new_prs = cp->partition_root_state;
 		}
 
+		/*
+		 * With no valid parent partition left, this partition's CPUs
+		 * return to the nearest valid partition ancestor.
+		 */
+		if (!was_remote && old_prs > 0 && new_prs < 0 &&
+		    !is_partition_valid(parent))
+			owner_prs = partition_owner(cp)->partition_root_state;
+
 		spin_lock_irq(&callback_lock);
+		if (old_prs != owner_prs)
+			isolated_cpus_update(old_prs, owner_prs,
+					     cp->effective_xcpus);
 		cpumask_copy(cp->effective_cpus, tmp->new_cpus);
 		cp->partition_root_state = new_prs;
 		/*
-- 
2.43.0

[PATCH v4 6/7] cgroup/cpuset: Publish cpus_allowed before partition updates

[Guopeng Zhang]

From: Guopeng Zhang <zhangguopeng@kylinos.cn>

update_cpumask() calls partition_cpus_change() before copying the new
cpus_allowed mask. A remote partition update can propagate through an
ancestor and revisit the cpuset while the old mask is still visible. The
second visit then adds back CPUs that the first visit released.

This can be reproduced on a cgroup v2 system with CPUs 1-7 online:

    cd /sys/fs/cgroup
    echo +cpuset > cgroup.subtree_control
    mkdir remote-repro
    echo 1-7 > remote-repro/cpuset.cpus
    echo 1-7 > remote-repro/cpuset.cpus.exclusive
    echo +cpuset > remote-repro/cgroup.subtree_control
    mkdir remote-repro/part
    echo 1-4 > remote-repro/part/cpuset.cpus
    echo root > remote-repro/part/cpuset.cpus.partition
    echo 1-3 > remote-repro/part/cpuset.cpus
    cat cpuset.cpus.effective

Without this fix, CPU 4 remains missing from the top cpuset after the
remote partition is narrowed.

Copy cpus_allowed before partition_cpus_change(). All checks and
allocations that can fail have completed by this point, and cpuset_mutex
remains held for the rest of the update. Keep effective_xcpus unchanged
until afterward so the partition code can calculate the old-to-new
difference.

Fixes: f62a5d39368e ("cgroup/cpuset: Remove remote_partition_check() & make update_cpumasks_hier() handle remote partition")
Signed-off-by: Guopeng Zhang <zhangguopeng@kylinos.cn>
---
 kernel/cgroup/cpuset.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index bbc4868f026b..80a709bfa4b7 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -2618,10 +2618,17 @@ static int update_cpumask(struct cpuset *cs, struct cpuset *trialcs,
 	 */
 	force = !cpumask_equal(cs->effective_xcpus, trialcs->effective_xcpus);
 
+	/*
+	 * remote_cpus_update() can propagate through an ancestor and revisit
+	 * this cpuset. Make sure that it sees the new configured CPU mask.
+	 */
+	spin_lock_irq(&callback_lock);
+	cpumask_copy(cs->cpus_allowed, trialcs->cpus_allowed);
+	spin_unlock_irq(&callback_lock);
+
 	partition_cpus_change(cs, trialcs, &tmp);
 
 	spin_lock_irq(&callback_lock);
-	cpumask_copy(cs->cpus_allowed, trialcs->cpus_allowed);
 	cpumask_copy(cs->effective_xcpus, trialcs->effective_xcpus);
 	if ((old_prs > 0) && !is_partition_valid(cs))
 		reset_partition_data(cs);
-- 
2.43.0

[PATCH v4 7/7] cgroup/cpuset: Publish exclusive_cpus before partition updates

[Guopeng Zhang]

From: Guopeng Zhang <zhangguopeng@kylinos.cn>

update_exclusive_cpumask() calls partition_cpus_change() before publishing
the new exclusive_cpus mask. When an invalid partition is recovered,
update_parent_effective_cpumask() therefore gets the old mask from
user_xcpus(). If that mask contains only offline CPUs and the partition is
populated, the valid replacement is rejected with PERR_NOCPUS.

This can be reproduced on a cgroup v2 system with CPUs 1-4 online:

    cd /sys/fs/cgroup
    echo +cpuset > cgroup.subtree_control
    mkdir exclusive-repro
    echo 1-4 > exclusive-repro/cpuset.cpus
    echo isolated > exclusive-repro/cpuset.cpus.partition
    echo +cpuset > exclusive-repro/cgroup.subtree_control
    mkdir exclusive-repro/child
    echo 4 > exclusive-repro/child/cpuset.cpus.exclusive
    echo isolated > exclusive-repro/child/cpuset.cpus.partition
    sleep 1000 & task=$!
    echo $task > exclusive-repro/child/cgroup.procs
    echo 0 > /sys/devices/system/cpu/cpu4/online
    echo 3 > exclusive-repro/child/cpuset.cpus.exclusive
    cat exclusive-repro/child/cpuset.cpus.partition
    echo 1 > /sys/devices/system/cpu/cpu4/online
    kill $task

Without this fix, the child remains invalid after CPU 3 is written because
the recovery path still validates the old CPU 4 mask.

Publish exclusive_cpus after all fallible preparation and before
partition_cpus_change(), as update_cpumask() already does for cpus_allowed.
Keep effective_xcpus unchanged until the partition update has computed the
old-to-new difference.

Fixes: c49b5e89c45f ("cpuset: use partition_cpus_change for setting exclusive cpus")
Signed-off-by: Guopeng Zhang <zhangguopeng@kylinos.cn>
---
 kernel/cgroup/cpuset.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 80a709bfa4b7..94c56ec8d4f8 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -2690,10 +2690,18 @@ static int update_exclusive_cpumask(struct cpuset *cs, struct cpuset *trialcs,
 		return -ENOMEM;
 
 	trialcs->prs_err = PERR_NONE;
-	partition_cpus_change(cs, trialcs, &tmp);
 
+	/*
+	 * partition_cpus_change() may inspect user_xcpus(cs). Publish the
+	 * configured mask that was used to compute trialcs->effective_xcpus.
+	 */
 	spin_lock_irq(&callback_lock);
 	cpumask_copy(cs->exclusive_cpus, trialcs->exclusive_cpus);
+	spin_unlock_irq(&callback_lock);
+
+	partition_cpus_change(cs, trialcs, &tmp);
+
+	spin_lock_irq(&callback_lock);
 	cpumask_copy(cs->effective_xcpus, trialcs->effective_xcpus);
 	if ((old_prs > 0) && !is_partition_valid(cs))
 		reset_partition_data(cs);
-- 
2.43.0