[PATCH-next v2 0/6] cgroup/cpuset: Fix various housekeeping check problems

[PATCH-next v2 0/6] cgroup/cpuset: Fix various housekeeping check problems

[Waiman Long]

 v2:
  - Update patches 2 and 3 to address bugs reported by sashiko.
  - Add a new test_cpuset_prs.sh patch to skip the test if some of the
    required CPUs are offline.

Commit 4a74e418881f ("cgroup/cpuset: Check partition conflict with
housekeeping setup") and commit 103b08709e8a ("cgroup/cpuset: Fail if
isolated and nohz_full don't leave any housekeeping") add housekeeping
checks to fail partition changes that will violate the housekeeping
rules. However there are several issues that make the check incomplete
and missing in some cases. This series fixes the various known
housekeeing check issues and adds additional test cases in
test_cpuset_prs.sh to test for some of those issues.

Waiman Long (6):
  cgroup/cpuset: Consolidate isolated_cpus_can_update() into
    prstate_housekeeping_conflict()
  cgroup/cpuset: Consider all the exclusive CPUs when doing housekeeping
    check
  cgroup/cpuset: Do housekeeping check before converting invalid
    partition to valid
  cgroup/cpuset: Properly disabling partition when partition state
    switching fails
  selftests/cgroup: Add tests for housekeeping check
  selftests/cgroup: Skip test_cpuset_prs.sh test if some required CPUs
    are offline

 kernel/cgroup/cpuset.c                        | 170 +++++++++---------
 .../selftests/cgroup/test_cpuset_prs.sh       | 125 ++++++++++---
 2 files changed, 195 insertions(+), 100 deletions(-)

-- 
2.55.0

[PATCH-next v2 1/6] cgroup/cpuset: Consolidate isolated_cpus_can_update() into prstate_housekeeping_conflict()

[Waiman Long]

The isolated_cpus_can_update() and prstate_housekeeping_conflict() are
checking different aspects of upcoming cpumask changes that may conflict
with the current setting of the housekeeping cpumasks. There are places
where both are called together. There are also places where only one
of them is called. That inconsistency can contribute to missing check
where invalid cpumask changes may be allowed to move forward.

Fix that by consolidating isolated_cpus_can_update() into
prstate_housekeeping_conflict() and call prstate_housekeeping_conflict()
in all the places where either one of them or both are called. The
exception is the validate_partition() function where the
prstate_housekeeping_conflict() call is removed. It is because
validate_partition() is called only from partition_cpus_change()
where prstate_housekeeping_conflict() will be called from either
remote_cpus_update() or update_parent_effective_cpumask() with
partcmd_update if not for partition invalidatation or disablement.

Now prstate_housekeeping_conflict() will be called in the following
locations:
 - when a partition is enabled in remote_partition_enable() or in
   update_parent_effective_cpumask() with partcmd_enable*.
 - when a cpumask is updated in remote_cpus_update() or in
   update_parent_effective_cpumask() with partcmd_update.
 - when a partition state changes from root to isolated or vice versa
   in update_prstate().

Fixes: 4a74e418881f ("cgroup/cpuset: Check partition conflict with housekeeping setup")
Fixes: 103b08709e8a ("cgroup/cpuset: Fail if isolated and nohz_full don't leave any housekeeping")
Signed-off-by: Waiman Long <longman@redhat.com>
---
 kernel/cgroup/cpuset.c | 123 +++++++++++++++++------------------------
 1 file changed, 52 insertions(+), 71 deletions(-)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 0ebba2d646c3..f3cebb277a68 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -1353,65 +1353,62 @@ static void partition_xcpus_del(int old_prs, struct cpuset *parent,
 }
 
 /*
- * isolated_cpus_can_update - check for isolated & nohz_full conflicts
- * @add_cpus: cpu mask for cpus that are going to be isolated
- * @del_cpus: cpu mask for cpus that are no longer isolated, can be NULL
- * Return: false if there is conflict, true otherwise
- *
- * If nohz_full is enabled and we have isolated CPUs, their combination must
- * still leave housekeeping CPUs.
+ * prstate_housekeeping_conflict - check for partition & housekeeping conflicts
+ * @new_prs: new partition root state to be checked
+ * @parent_prs: Parent partition root state
+ * @add_cpus: additional CPUs to be added to current cpuset
+ * @del_cpus: CPUs to be removed from current cpuset, can be NULL
+ * Return: true if there is conflict, false otherwise
  *
- * TBD: Should consider merging this function into
- *      prstate_housekeeping_conflict().
+ * There are two different housekeeping conflicts to be checked:
+ * 1) If new_prs is PRS_ROOT, none of the @add_cpus can be a boot-time isolated
+ *    CPU. IOW, the whole @add_cpus must be a subset of HK_TYPE_DOMAIN_BOOT.
+ * 2) If nohz_full is enabled and we have isolated CPUs, their combination must
+ *    still leave housekeeping CPUs. This check is only needed if new_prs
+ *    differs from parent_prs and one of them is PRS_ISOLATED.
  */
-static bool isolated_cpus_can_update(struct cpumask *add_cpus,
-				     struct cpumask *del_cpus)
+static bool prstate_housekeeping_conflict(int new_prs, int parent_prs,
+					  struct cpumask *add_cpus,
+					  struct cpumask *del_cpus)
 {
 	cpumask_var_t full_hk_cpus;
-	int res = true;
+	int res;
 
-	if (!housekeeping_enabled(HK_TYPE_KERNEL_NOISE))
+	if (housekeeping_enabled(HK_TYPE_DOMAIN_BOOT) && (new_prs == PRS_ROOT) &&
+	    !cpumask_subset(add_cpus, housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT)))
 		return true;
 
-	if (del_cpus && cpumask_weight_and(del_cpus,
-			housekeeping_cpumask(HK_TYPE_KERNEL_NOISE)))
-		return true;
+	if (!housekeeping_enabled(HK_TYPE_KERNEL_NOISE) ||
+	    (new_prs == parent_prs) ||
+	    (!del_cpus && (new_prs != PRS_ISOLATED)) ||
+	    ((new_prs != PRS_ISOLATED) && (parent_prs != PRS_ISOLATED)))
+		return false;
 
-	if (!alloc_cpumask_var(&full_hk_cpus, GFP_KERNEL))
+	/*
+	 * Make sure that @add_cpus contains new CPUs to be isolated and
+	 * @del_cpus contains isolated CPUs to be un-isolated.
+	 */
+	if (parent_prs == PRS_ISOLATED)
+		swap(add_cpus, del_cpus);
+
+	if (del_cpus &&
+	   (cpumask_first_and_and(del_cpus, housekeeping_cpumask(HK_TYPE_KERNEL_NOISE),
+				   cpu_active_mask) < nr_cpu_ids))
 		return false;
 
+	if (!alloc_cpumask_var(&full_hk_cpus, GFP_KERNEL))
+		return true;
+
 	cpumask_and(full_hk_cpus, housekeeping_cpumask(HK_TYPE_KERNEL_NOISE),
 		    housekeeping_cpumask(HK_TYPE_DOMAIN));
 	cpumask_andnot(full_hk_cpus, full_hk_cpus, isolated_cpus);
 	cpumask_and(full_hk_cpus, full_hk_cpus, cpu_active_mask);
-	if (!cpumask_weight_andnot(full_hk_cpus, add_cpus))
-		res = false;
+	res = !cpumask_weight_andnot(full_hk_cpus, add_cpus);
 
 	free_cpumask_var(full_hk_cpus);
 	return res;
 }
 
-/*
- * prstate_housekeeping_conflict - check for partition & housekeeping conflicts
- * @prstate: partition root state to be checked
- * @new_cpus: cpu mask
- * Return: true if there is conflict, false otherwise
- *
- * CPUs outside of HK_TYPE_DOMAIN_BOOT, if defined, can only be used in an
- * isolated partition.
- */
-static bool prstate_housekeeping_conflict(int prstate, struct cpumask *new_cpus)
-{
-	if (!housekeeping_enabled(HK_TYPE_DOMAIN_BOOT))
-		return false;
-
-	if ((prstate != PRS_ISOLATED) &&
-	    !cpumask_subset(new_cpus, housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT)))
-		return true;
-
-	return false;
-}
-
 /*
  * cpuset_update_sd_hk_unlock - Rebuild sched domains, update HK & unlock
  *
@@ -1593,9 +1590,7 @@ static int remote_partition_enable(struct cpuset *cs, int new_prs,
 		return PERR_INVCPUS;
 	if (cpumask_intersects(tmp->new_cpus, subpartitions_cpus))
 		return PERR_NOCPUS;
-	if (((new_prs == PRS_ISOLATED) &&
-	     !isolated_cpus_can_update(tmp->new_cpus, NULL)) ||
-	    prstate_housekeeping_conflict(new_prs, tmp->new_cpus))
+	if (prstate_housekeeping_conflict(new_prs, PRS_ROOT, tmp->new_cpus, NULL))
 		return PERR_HKEEPING;
 
 	spin_lock_irq(&callback_lock);
@@ -1697,8 +1692,8 @@ static void remote_cpus_update(struct cpuset *cs, struct cpumask *xcpus,
 		else if (cpumask_intersects(tmp->addmask, subpartitions_cpus) ||
 			 cpumask_subset(top_cpuset.effective_cpus, tmp->addmask))
 			WRITE_ONCE(cs->prs_err, PERR_NOCPUS);
-		else if ((prs == PRS_ISOLATED) &&
-			 !isolated_cpus_can_update(tmp->addmask, tmp->delmask))
+		else if (prstate_housekeeping_conflict(prs, PRS_ROOT,
+						       tmp->addmask, tmp->delmask))
 			WRITE_ONCE(cs->prs_err, PERR_HKEEPING);
 		if (cs->prs_err)
 			goto invalidate;
@@ -1840,11 +1835,7 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,
 		if (cpumask_empty(xcpus))
 			return PERR_INVCPUS;
 
-		if (prstate_housekeeping_conflict(new_prs, xcpus))
-			return PERR_HKEEPING;
-
-		if ((new_prs == PRS_ISOLATED) && (new_prs != parent_prs) &&
-		    !isolated_cpus_can_update(xcpus, NULL))
+		if (prstate_housekeeping_conflict(new_prs, parent_prs, xcpus, NULL))
 			return PERR_HKEEPING;
 
 		if (tasks_nocpu_error(parent, cs, xcpus))
@@ -1917,20 +1908,12 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,
 		}
 
 		/*
-		 * TBD: Invalidate a currently valid child root partition may
-		 * still break isolated_cpus_can_update() rule if parent is an
-		 * isolated partition.
+		 * Check for housekeeping conflicts
 		 */
-		if (is_partition_valid(cs) && (old_prs != parent_prs)) {
-			if ((parent_prs == PRS_ROOT) &&
-			    /* Adding to parent means removing isolated CPUs */
-			    !isolated_cpus_can_update(tmp->delmask, tmp->addmask))
-				part_error = PERR_HKEEPING;
-			if ((parent_prs == PRS_ISOLATED) &&
-			    /* Adding to parent means adding isolated CPUs */
-			    !isolated_cpus_can_update(tmp->addmask, tmp->delmask))
-				part_error = PERR_HKEEPING;
-		}
+		if (is_partition_valid(cs) &&
+		    prstate_housekeeping_conflict(old_prs, parent_prs,
+						  tmp->delmask, tmp->addmask))
+			part_error = PERR_HKEEPING;
 
 		/*
 		 * The new CPUs to be removed from parent's effective CPUs
@@ -2391,10 +2374,6 @@ static enum prs_errcode validate_partition(struct cpuset *cs, struct cpuset *tri
 	if (cpumask_empty(trialcs->effective_xcpus))
 		return PERR_INVCPUS;
 
-	if (prstate_housekeeping_conflict(trialcs->partition_root_state,
-					  trialcs->effective_xcpus))
-		return PERR_HKEEPING;
-
 	if (tasks_nocpu_error(parent, cs, trialcs->effective_xcpus))
 		return PERR_NOCPUS;
 
@@ -2411,7 +2390,7 @@ static enum prs_errcode validate_partition(struct cpuset *cs, struct cpuset *tri
  * CPU modifications may cause a partition to be disabled or require state updates.
  */
 static void partition_cpus_change(struct cpuset *cs, struct cpuset *trialcs,
-					struct tmpmasks *tmp)
+				  struct tmpmasks *tmp)
 {
 	enum prs_errcode prs_err;
 
@@ -2911,13 +2890,15 @@ static int update_prstate(struct cpuset *cs, int new_prs)
 			err = remote_partition_enable(cs, new_prs, &tmpmask);
 		}
 	} else if (old_prs && new_prs) {
+		int parent_prs = is_remote_partition(cs)
+				 ? PRS_ROOT : parent->partition_root_state;
+
 		/*
 		 * A change in load balance state only, no change in cpumasks.
 		 * Need to update isolated_cpus.
 		 */
-		if (((new_prs == PRS_ISOLATED) &&
-		     !isolated_cpus_can_update(cs->effective_xcpus, NULL)) ||
-		    prstate_housekeeping_conflict(new_prs, cs->effective_xcpus))
+		if (prstate_housekeeping_conflict(new_prs, parent_prs,
+						  cs->effective_xcpus, NULL))
 			err = PERR_HKEEPING;
 		else
 			isolcpus_updated = true;
-- 
2.55.0

Re: [PATCH-next v2 1/6] cgroup/cpuset: Consolidate isolated_cpus_can_update() into prstate_housekeeping_conflict()

[Ridong Chen]

On 10/11/2026 6:19 AM, Waiman Long wrote:
> The isolated_cpus_can_update() and prstate_housekeeping_conflict() are
> checking different aspects of upcoming cpumask changes that may conflict
> with the current setting of the housekeeping cpumasks. There are places
> where both are called together. There are also places where only one
> of them is called. That inconsistency can contribute to missing check
> where invalid cpumask changes may be allowed to move forward.
> 
> Fix that by consolidating isolated_cpus_can_update() into
> prstate_housekeeping_conflict() and call prstate_housekeeping_conflict()
> in all the places where either one of them or both are called. The
> exception is the validate_partition() function where the
> prstate_housekeeping_conflict() call is removed. It is because
> validate_partition() is called only from partition_cpus_change()
> where prstate_housekeeping_conflict() will be called from either
> remote_cpus_update() or update_parent_effective_cpumask() with
> partcmd_update if not for partition invalidatation or disablement.
> 
> Now prstate_housekeeping_conflict() will be called in the following
> locations:
>   - when a partition is enabled in remote_partition_enable() or in
>     update_parent_effective_cpumask() with partcmd_enable*.
>   - when a cpumask is updated in remote_cpus_update() or in
>     update_parent_effective_cpumask() with partcmd_update.
>   - when a partition state changes from root to isolated or vice versa
>     in update_prstate().
> 

This will fix the issue reported by Guopeng [1]. However, I believe Guopeng's 
patch is still necessary, whether as a fix patch or a cleanup patch.

[1] 
https://lore.kernel.org/cgroups/585bc1e0-f538-444e-b8b4-28185ee6b19c@linux.dev/T/#t

> Fixes: 4a74e418881f ("cgroup/cpuset: Check partition conflict with housekeeping setup")
> Fixes: 103b08709e8a ("cgroup/cpuset: Fail if isolated and nohz_full don't leave any housekeeping")
> Signed-off-by: Waiman Long <longman@redhat.com>
> ---
>   kernel/cgroup/cpuset.c | 123 +++++++++++++++++------------------------
>   1 file changed, 52 insertions(+), 71 deletions(-)
> 
> diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
> index 0ebba2d646c3..f3cebb277a68 100644
> --- a/kernel/cgroup/cpuset.c
> +++ b/kernel/cgroup/cpuset.c
> @@ -1353,65 +1353,62 @@ static void partition_xcpus_del(int old_prs, struct cpuset *parent,
>   }
>   
>   /*
> - * isolated_cpus_can_update - check for isolated & nohz_full conflicts
> - * @add_cpus: cpu mask for cpus that are going to be isolated
> - * @del_cpus: cpu mask for cpus that are no longer isolated, can be NULL
> - * Return: false if there is conflict, true otherwise
> - *
> - * If nohz_full is enabled and we have isolated CPUs, their combination must
> - * still leave housekeeping CPUs.
> + * prstate_housekeeping_conflict - check for partition & housekeeping conflicts
> + * @new_prs: new partition root state to be checked
> + * @parent_prs: Parent partition root state
> + * @add_cpus: additional CPUs to be added to current cpuset
> + * @del_cpus: CPUs to be removed from current cpuset, can be NULL
> + * Return: true if there is conflict, false otherwise
>    *
> - * TBD: Should consider merging this function into
> - *      prstate_housekeeping_conflict().
> + * There are two different housekeeping conflicts to be checked:
> + * 1) If new_prs is PRS_ROOT, none of the @add_cpus can be a boot-time isolated
> + *    CPU. IOW, the whole @add_cpus must be a subset of HK_TYPE_DOMAIN_BOOT.
> + * 2) If nohz_full is enabled and we have isolated CPUs, their combination must
> + *    still leave housekeeping CPUs. This check is only needed if new_prs
> + *    differs from parent_prs and one of them is PRS_ISOLATED.
>    */
> -static bool isolated_cpus_can_update(struct cpumask *add_cpus,
> -				     struct cpumask *del_cpus)
> +static bool prstate_housekeeping_conflict(int new_prs, int parent_prs,
> +					  struct cpumask *add_cpus,
> +					  struct cpumask *del_cpus)
>   {
>   	cpumask_var_t full_hk_cpus;
> -	int res = true;
> +	int res;
>   
> -	if (!housekeeping_enabled(HK_TYPE_KERNEL_NOISE))
> +	if (housekeeping_enabled(HK_TYPE_DOMAIN_BOOT) && (new_prs == PRS_ROOT) &&
> +	    !cpumask_subset(add_cpus, housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT)))
>   		return true;
>   
> -	if (del_cpus && cpumask_weight_and(del_cpus,
> -			housekeeping_cpumask(HK_TYPE_KERNEL_NOISE)))
> -		return true;
> +	if (!housekeeping_enabled(HK_TYPE_KERNEL_NOISE) ||
> +	    (new_prs == parent_prs) ||
> +	    (!del_cpus && (new_prs != PRS_ISOLATED)) ||
> +	    ((new_prs != PRS_ISOLATED) && (parent_prs != PRS_ISOLATED)))
> +		return false;
>   
> -	if (!alloc_cpumask_var(&full_hk_cpus, GFP_KERNEL))
> +	/*
> +	 * Make sure that @add_cpus contains new CPUs to be isolated and
> +	 * @del_cpus contains isolated CPUs to be un-isolated.
> +	 */
> +	if (parent_prs == PRS_ISOLATED)
> +		swap(add_cpus, del_cpus);
> +
> +	if (del_cpus &&
> +	   (cpumask_first_and_and(del_cpus, housekeeping_cpumask(HK_TYPE_KERNEL_NOISE),
> +				   cpu_active_mask) < nr_cpu_ids))
>   		return false;
>   
> +	if (!alloc_cpumask_var(&full_hk_cpus, GFP_KERNEL))
> +		return true;
> +
>   	cpumask_and(full_hk_cpus, housekeeping_cpumask(HK_TYPE_KERNEL_NOISE),
>   		    housekeeping_cpumask(HK_TYPE_DOMAIN));
>   	cpumask_andnot(full_hk_cpus, full_hk_cpus, isolated_cpus);
>   	cpumask_and(full_hk_cpus, full_hk_cpus, cpu_active_mask);
> -	if (!cpumask_weight_andnot(full_hk_cpus, add_cpus))
> -		res = false;
> +	res = !cpumask_weight_andnot(full_hk_cpus, add_cpus);
>   
>   	free_cpumask_var(full_hk_cpus);
>   	return res;
>   }
>   
> -/*
> - * prstate_housekeeping_conflict - check for partition & housekeeping conflicts
> - * @prstate: partition root state to be checked
> - * @new_cpus: cpu mask
> - * Return: true if there is conflict, false otherwise
> - *
> - * CPUs outside of HK_TYPE_DOMAIN_BOOT, if defined, can only be used in an
> - * isolated partition.
> - */
> -static bool prstate_housekeeping_conflict(int prstate, struct cpumask *new_cpus)
> -{
> -	if (!housekeeping_enabled(HK_TYPE_DOMAIN_BOOT))
> -		return false;
> -
> -	if ((prstate != PRS_ISOLATED) &&
> -	    !cpumask_subset(new_cpus, housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT)))
> -		return true;
> -
> -	return false;

Can we simply return isolated_cpus_can_update(...) here?

> -}
> -
>   /*
>    * cpuset_update_sd_hk_unlock - Rebuild sched domains, update HK & unlock
>    *
> @@ -1593,9 +1590,7 @@ static int remote_partition_enable(struct cpuset *cs, int new_prs,
>   		return PERR_INVCPUS;
>   	if (cpumask_intersects(tmp->new_cpus, subpartitions_cpus))
>   		return PERR_NOCPUS;
> -	if (((new_prs == PRS_ISOLATED) &&
> -	     !isolated_cpus_can_update(tmp->new_cpus, NULL)) ||
> -	    prstate_housekeeping_conflict(new_prs, tmp->new_cpus))
> +	if (prstate_housekeeping_conflict(new_prs, PRS_ROOT, tmp->new_cpus, NULL))
>   		return PERR_HKEEPING;
>   
>   	spin_lock_irq(&callback_lock);
> @@ -1697,8 +1692,8 @@ static void remote_cpus_update(struct cpuset *cs, struct cpumask *xcpus,
>   		else if (cpumask_intersects(tmp->addmask, subpartitions_cpus) ||
>   			 cpumask_subset(top_cpuset.effective_cpus, tmp->addmask))
>   			WRITE_ONCE(cs->prs_err, PERR_NOCPUS);
> -		else if ((prs == PRS_ISOLATED) &&
> -			 !isolated_cpus_can_update(tmp->addmask, tmp->delmask))
> +		else if (prstate_housekeeping_conflict(prs, PRS_ROOT,
> +						       tmp->addmask, tmp->delmask))
>   			WRITE_ONCE(cs->prs_err, PERR_HKEEPING);
>   		if (cs->prs_err)
>   			goto invalidate;
> @@ -1840,11 +1835,7 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,
>   		if (cpumask_empty(xcpus))
>   			return PERR_INVCPUS;
>   
> -		if (prstate_housekeeping_conflict(new_prs, xcpus))
> -			return PERR_HKEEPING;
> -
> -		if ((new_prs == PRS_ISOLATED) && (new_prs != parent_prs) &&
> -		    !isolated_cpus_can_update(xcpus, NULL))
> +		if (prstate_housekeeping_conflict(new_prs, parent_prs, xcpus, NULL))
>   			return PERR_HKEEPING;
>   
>   		if (tasks_nocpu_error(parent, cs, xcpus))
> @@ -1917,20 +1908,12 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,
>   		}
>   
>   		/*
> -		 * TBD: Invalidate a currently valid child root partition may
> -		 * still break isolated_cpus_can_update() rule if parent is an
> -		 * isolated partition.
> +		 * Check for housekeeping conflicts
>   		 */
> -		if (is_partition_valid(cs) && (old_prs != parent_prs)) {
> -			if ((parent_prs == PRS_ROOT) &&
> -			    /* Adding to parent means removing isolated CPUs */
> -			    !isolated_cpus_can_update(tmp->delmask, tmp->addmask))
> -				part_error = PERR_HKEEPING;
> -			if ((parent_prs == PRS_ISOLATED) &&
> -			    /* Adding to parent means adding isolated CPUs */
> -			    !isolated_cpus_can_update(tmp->addmask, tmp->delmask))
> -				part_error = PERR_HKEEPING;
> -		}
> +		if (is_partition_valid(cs) &&
> +		    prstate_housekeeping_conflict(old_prs, parent_prs,
> +						  tmp->delmask, tmp->addmask))
> +			part_error = PERR_HKEEPING;
>   
>   		/*
>   		 * The new CPUs to be removed from parent's effective CPUs
> @@ -2391,10 +2374,6 @@ static enum prs_errcode validate_partition(struct cpuset *cs, struct cpuset *tri
>   	if (cpumask_empty(trialcs->effective_xcpus))
>   		return PERR_INVCPUS;
>   
> -	if (prstate_housekeeping_conflict(trialcs->partition_root_state,
> -					  trialcs->effective_xcpus))
> -		return PERR_HKEEPING;
> -
>   	if (tasks_nocpu_error(parent, cs, trialcs->effective_xcpus))
>   		return PERR_NOCPUS;
>   
> @@ -2411,7 +2390,7 @@ static enum prs_errcode validate_partition(struct cpuset *cs, struct cpuset *tri
>    * CPU modifications may cause a partition to be disabled or require state updates.
>    */
>   static void partition_cpus_change(struct cpuset *cs, struct cpuset *trialcs,
> -					struct tmpmasks *tmp)
> +				  struct tmpmasks *tmp)
>   {
>   	enum prs_errcode prs_err;
>   
> @@ -2911,13 +2890,15 @@ static int update_prstate(struct cpuset *cs, int new_prs)
>   			err = remote_partition_enable(cs, new_prs, &tmpmask);
>   		}
>   	} else if (old_prs && new_prs) {
> +		int parent_prs = is_remote_partition(cs)
> +				 ? PRS_ROOT : parent->partition_root_state;
> +
>   		/*
>   		 * A change in load balance state only, no change in cpumasks.
>   		 * Need to update isolated_cpus.
>   		 */
> -		if (((new_prs == PRS_ISOLATED) &&
> -		     !isolated_cpus_can_update(cs->effective_xcpus, NULL)) ||
> -		    prstate_housekeeping_conflict(new_prs, cs->effective_xcpus))
> +		if (prstate_housekeeping_conflict(new_prs, parent_prs,
> +						  cs->effective_xcpus, NULL))
>   			err = PERR_HKEEPING;
>   		else
>   			isolcpus_updated = true;

-- 
Best regards
Ridong

Re: [PATCH-next v2 1/6] cgroup/cpuset: Consolidate isolated_cpus_can_update() into prstate_housekeeping_conflict()

[Guopeng Zhang]

Hi，Longman,
在 2026/10/11 06:19, Waiman Long 写道:
> The isolated_cpus_can_update() and prstate_housekeeping_conflict() are
> checking different aspects of upcoming cpumask changes that may conflict
> with the current setting of the housekeeping cpumasks. There are places
> where both are called together. There are also places where only one
> of them is called. That inconsistency can contribute to missing check
> where invalid cpumask changes may be allowed to move forward.
> 
> Fix that by consolidating isolated_cpus_can_update() into
> prstate_housekeeping_conflict() and call prstate_housekeeping_conflict()
> in all the places where either one of them or both are called. The
> exception is the validate_partition() function where the
> prstate_housekeeping_conflict() call is removed. It is because
> validate_partition() is called only from partition_cpus_change()
> where prstate_housekeeping_conflict() will be called from either
> remote_cpus_update() or update_parent_effective_cpumask() with
> partcmd_update if not for partition invalidatation or disablement.
> 
> Now prstate_housekeeping_conflict() will be called in the following
> locations:
>  - when a partition is enabled in remote_partition_enable() or in
>    update_parent_effective_cpumask() with partcmd_enable*.
>  - when a cpumask is updated in remote_cpus_update() or in
>    update_parent_effective_cpumask() with partcmd_update.
>  - when a partition state changes from root to isolated or vice versa
>    in update_prstate().
> 
> Fixes: 4a74e418881f ("cgroup/cpuset: Check partition conflict with housekeeping setup")
> Fixes: 103b08709e8a ("cgroup/cpuset: Fail if isolated and nohz_full don't leave any housekeeping")
> Signed-off-by: Waiman Long <longman@redhat.com>
> ---
>  kernel/cgroup/cpuset.c | 123 +++++++++++++++++------------------------
>  1 file changed, 52 insertions(+), 71 deletions(-)
> 
> diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
> index 0ebba2d646c3..f3cebb277a68 100644
> --- a/kernel/cgroup/cpuset.c
> +++ b/kernel/cgroup/cpuset.c
> @@ -1353,65 +1353,62 @@ static void partition_xcpus_del(int old_prs, struct cpuset *parent,
>  }
>  
>  /*
> - * isolated_cpus_can_update - check for isolated & nohz_full conflicts
> - * @add_cpus: cpu mask for cpus that are going to be isolated
> - * @del_cpus: cpu mask for cpus that are no longer isolated, can be NULL
> - * Return: false if there is conflict, true otherwise
> - *
> - * If nohz_full is enabled and we have isolated CPUs, their combination must
> - * still leave housekeeping CPUs.
> + * prstate_housekeeping_conflict - check for partition & housekeeping conflicts
> + * @new_prs: new partition root state to be checked
> + * @parent_prs: Parent partition root state
> + * @add_cpus: additional CPUs to be added to current cpuset
> + * @del_cpus: CPUs to be removed from current cpuset, can be NULL
> + * Return: true if there is conflict, false otherwise
>   *
> - * TBD: Should consider merging this function into
> - *      prstate_housekeeping_conflict().
> + * There are two different housekeeping conflicts to be checked:
> + * 1) If new_prs is PRS_ROOT, none of the @add_cpus can be a boot-time isolated
> + *    CPU. IOW, the whole @add_cpus must be a subset of HK_TYPE_DOMAIN_BOOT.
> + * 2) If nohz_full is enabled and we have isolated CPUs, their combination must
> + *    still leave housekeeping CPUs. This check is only needed if new_prs
> + *    differs from parent_prs and one of them is PRS_ISOLATED.
>   */
> -static bool isolated_cpus_can_update(struct cpumask *add_cpus,
> -				     struct cpumask *del_cpus)
> +static bool prstate_housekeeping_conflict(int new_prs, int parent_prs,
> +					  struct cpumask *add_cpus,
> +					  struct cpumask *del_cpus)
>  {
>  	cpumask_var_t full_hk_cpus;
> -	int res = true;
> +	int res;
>  
> -	if (!housekeeping_enabled(HK_TYPE_KERNEL_NOISE))
> +	if (housekeeping_enabled(HK_TYPE_DOMAIN_BOOT) && (new_prs == PRS_ROOT) &&
> +	    !cpumask_subset(add_cpus, housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT)))
>  		return true;
>  
> -	if (del_cpus && cpumask_weight_and(del_cpus,
> -			housekeeping_cpumask(HK_TYPE_KERNEL_NOISE)))
> -		return true;
> +	if (!housekeeping_enabled(HK_TYPE_KERNEL_NOISE) ||
> +	    (new_prs == parent_prs) ||
> +	    (!del_cpus && (new_prs != PRS_ISOLATED)) ||
> +	    ((new_prs != PRS_ISOLATED) && (parent_prs != PRS_ISOLATED)))
> +		return false;
>  

When a child partition switches from root to isolated under an isolated
parent, this returns false even though the child's CPUs were not isolated
before the switch.

On a fresh boot with CPUs 0-15 online and nohz_full=2-15:

    # cd /sys/fs/cgroup
    # echo +cpuset > cgroup.subtree_control
    # mkdir A B
    # echo 1-2 > A/cpuset.cpus
    # echo isolated > A/cpuset.cpus.partition
    # echo +cpuset > A/cgroup.subtree_control
    # mkdir A/C
    # echo 1 > A/C/cpuset.cpus
    # echo root > A/C/cpuset.cpus.partition
    # echo 0 > B/cpuset.cpus
    # echo isolated > B/cpuset.cpus.partition
    # cat cpuset.cpus.isolated
    0,2
    # echo isolated > A/C/cpuset.cpus.partition
    # cat A/C/cpuset.cpus.partition
    isolated
    # cat cpuset.cpus.isolated
    0-2

CPU 1 was the last remaining housekeeping CPU: it was in neither the
nohz_full mask nor the cpuset.cpus.isolated mask. The final write isolates
CPU 1 as well, leaving no housekeeping CPU.

Should this check also take the child's previous partition state into
account, rather than just comparing its new state with the parent's state?

Thanks,
Guopeng
> -	if (!alloc_cpumask_var(&full_hk_cpus, GFP_KERNEL))
> +	/*
> +	 * Make sure that @add_cpus contains new CPUs to be isolated and
> +	 * @del_cpus contains isolated CPUs to be un-isolated.
> +	 */
> +	if (parent_prs == PRS_ISOLATED)
> +		swap(add_cpus, del_cpus);
> +
> +	if (del_cpus &&
> +	   (cpumask_first_and_and(del_cpus, housekeeping_cpumask(HK_TYPE_KERNEL_NOISE),
> +				   cpu_active_mask) < nr_cpu_ids))
>  		return false;
>  
> +	if (!alloc_cpumask_var(&full_hk_cpus, GFP_KERNEL))
> +		return true;
> +
>  	cpumask_and(full_hk_cpus, housekeeping_cpumask(HK_TYPE_KERNEL_NOISE),
>  		    housekeeping_cpumask(HK_TYPE_DOMAIN));
>  	cpumask_andnot(full_hk_cpus, full_hk_cpus, isolated_cpus);
>  	cpumask_and(full_hk_cpus, full_hk_cpus, cpu_active_mask);
> -	if (!cpumask_weight_andnot(full_hk_cpus, add_cpus))
> -		res = false;
> +	res = !cpumask_weight_andnot(full_hk_cpus, add_cpus);
>  
>  	free_cpumask_var(full_hk_cpus);
>  	return res;
>  }
>  
> -/*
> - * prstate_housekeeping_conflict - check for partition & housekeeping conflicts
> - * @prstate: partition root state to be checked
> - * @new_cpus: cpu mask
> - * Return: true if there is conflict, false otherwise
> - *
> - * CPUs outside of HK_TYPE_DOMAIN_BOOT, if defined, can only be used in an
> - * isolated partition.
> - */
> -static bool prstate_housekeeping_conflict(int prstate, struct cpumask *new_cpus)
> -{
> -	if (!housekeeping_enabled(HK_TYPE_DOMAIN_BOOT))
> -		return false;
> -
> -	if ((prstate != PRS_ISOLATED) &&
> -	    !cpumask_subset(new_cpus, housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT)))
> -		return true;
> -
> -	return false;
> -}
> -
>  /*
>   * cpuset_update_sd_hk_unlock - Rebuild sched domains, update HK & unlock
>   *
> @@ -1593,9 +1590,7 @@ static int remote_partition_enable(struct cpuset *cs, int new_prs,
>  		return PERR_INVCPUS;
>  	if (cpumask_intersects(tmp->new_cpus, subpartitions_cpus))
>  		return PERR_NOCPUS;
> -	if (((new_prs == PRS_ISOLATED) &&
> -	     !isolated_cpus_can_update(tmp->new_cpus, NULL)) ||
> -	    prstate_housekeeping_conflict(new_prs, tmp->new_cpus))
> +	if (prstate_housekeeping_conflict(new_prs, PRS_ROOT, tmp->new_cpus, NULL))
>  		return PERR_HKEEPING;
>  
>  	spin_lock_irq(&callback_lock);
> @@ -1697,8 +1692,8 @@ static void remote_cpus_update(struct cpuset *cs, struct cpumask *xcpus,
>  		else if (cpumask_intersects(tmp->addmask, subpartitions_cpus) ||
>  			 cpumask_subset(top_cpuset.effective_cpus, tmp->addmask))
>  			WRITE_ONCE(cs->prs_err, PERR_NOCPUS);
> -		else if ((prs == PRS_ISOLATED) &&
> -			 !isolated_cpus_can_update(tmp->addmask, tmp->delmask))
> +		else if (prstate_housekeeping_conflict(prs, PRS_ROOT,
> +						       tmp->addmask, tmp->delmask))
>  			WRITE_ONCE(cs->prs_err, PERR_HKEEPING);
>  		if (cs->prs_err)
>  			goto invalidate;
> @@ -1840,11 +1835,7 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,
>  		if (cpumask_empty(xcpus))
>  			return PERR_INVCPUS;
>  
> -		if (prstate_housekeeping_conflict(new_prs, xcpus))
> -			return PERR_HKEEPING;
> -
> -		if ((new_prs == PRS_ISOLATED) && (new_prs != parent_prs) &&
> -		    !isolated_cpus_can_update(xcpus, NULL))
> +		if (prstate_housekeeping_conflict(new_prs, parent_prs, xcpus, NULL))
>  			return PERR_HKEEPING;
>  
>  		if (tasks_nocpu_error(parent, cs, xcpus))
> @@ -1917,20 +1908,12 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,
>  		}
>  
>  		/*
> -		 * TBD: Invalidate a currently valid child root partition may
> -		 * still break isolated_cpus_can_update() rule if parent is an
> -		 * isolated partition.
> +		 * Check for housekeeping conflicts
>  		 */
> -		if (is_partition_valid(cs) && (old_prs != parent_prs)) {
> -			if ((parent_prs == PRS_ROOT) &&
> -			    /* Adding to parent means removing isolated CPUs */
> -			    !isolated_cpus_can_update(tmp->delmask, tmp->addmask))
> -				part_error = PERR_HKEEPING;
> -			if ((parent_prs == PRS_ISOLATED) &&
> -			    /* Adding to parent means adding isolated CPUs */
> -			    !isolated_cpus_can_update(tmp->addmask, tmp->delmask))
> -				part_error = PERR_HKEEPING;
> -		}
> +		if (is_partition_valid(cs) &&
> +		    prstate_housekeeping_conflict(old_prs, parent_prs,
> +						  tmp->delmask, tmp->addmask))
> +			part_error = PERR_HKEEPING;
>  
>  		/*
>  		 * The new CPUs to be removed from parent's effective CPUs
> @@ -2391,10 +2374,6 @@ static enum prs_errcode validate_partition(struct cpuset *cs, struct cpuset *tri
>  	if (cpumask_empty(trialcs->effective_xcpus))
>  		return PERR_INVCPUS;
>  
> -	if (prstate_housekeeping_conflict(trialcs->partition_root_state,
> -					  trialcs->effective_xcpus))
> -		return PERR_HKEEPING;
> -
>  	if (tasks_nocpu_error(parent, cs, trialcs->effective_xcpus))
>  		return PERR_NOCPUS;
>  
> @@ -2411,7 +2390,7 @@ static enum prs_errcode validate_partition(struct cpuset *cs, struct cpuset *tri
>   * CPU modifications may cause a partition to be disabled or require state updates.
>   */
>  static void partition_cpus_change(struct cpuset *cs, struct cpuset *trialcs,
> -					struct tmpmasks *tmp)
> +				  struct tmpmasks *tmp)
>  {
>  	enum prs_errcode prs_err;
>  
> @@ -2911,13 +2890,15 @@ static int update_prstate(struct cpuset *cs, int new_prs)
>  			err = remote_partition_enable(cs, new_prs, &tmpmask);
>  		}
>  	} else if (old_prs && new_prs) {
> +		int parent_prs = is_remote_partition(cs)
> +				 ? PRS_ROOT : parent->partition_root_state;
> +
>  		/*
>  		 * A change in load balance state only, no change in cpumasks.
>  		 * Need to update isolated_cpus.
>  		 */
> -		if (((new_prs == PRS_ISOLATED) &&
> -		     !isolated_cpus_can_update(cs->effective_xcpus, NULL)) ||
> -		    prstate_housekeeping_conflict(new_prs, cs->effective_xcpus))
> +		if (prstate_housekeeping_conflict(new_prs, parent_prs,
> +						  cs->effective_xcpus, NULL))
>  			err = PERR_HKEEPING;
>  		else
>  			isolcpus_updated = true;

[PATCH-next v2 2/6] cgroup/cpuset: Consider all the exclusive CPUs when doing housekeeping check

[Waiman Long]

When prstate_housekeeping_conflict() is called to perform housekeeping
check with cpumask changes, the exclusive CPUs owned by child partitions
are excluded. If the current cpuset is an isolated partition with root
partition children. It is possible the cpumask change will still leave
active housekeeping CPUs but then no housekeeping CPUs will be left when
the child partitions are disabled or invalidated.

To protect against this possibility, all the exclusive CPUs of the
cpuset should be considered to be owned by the current cpuset when doing
housekeeping check. Do that by calling prstate_housekeeping_conflict()
in partition_cpus_change() and passing in the add_cpus and del_cpus
parameters as if the cpuset owns all the exclusive CPUs.

With this change, the prstate_housekeeping_conflict() call in
update_parent_effective_cpumask() with partcmd_update and newmask
is now duplicative and can be removed. The housekeeping check in
remote_cpus_update(), however, will still be useful as this function can
be called directly from update_cpumasks_hier() without going through
partition_cpus_change(). The update_parent_effective_cpumask() call
from update_cpumasks_hier() is partcmd_update with no cpumask which
still have the housekeeping check and so is covered.

In the case of partition state switch from isolated to root and vice
versa, the set of exclusive CPUs are recomputed to include those
transferred to child paritions as well.

Fixes: 4a74e418881f ("cgroup/cpuset: Check partition conflict with housekeeping setup")
Signed-off-by: Waiman Long <longman@redhat.com>
---
 kernel/cgroup/cpuset.c | 26 +++++++++++++++++---------
 1 file changed, 17 insertions(+), 9 deletions(-)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index f3cebb277a68..1c0441fa3bea 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -1907,14 +1907,6 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,
 					       parent->effective_xcpus);
 		}
 
-		/*
-		 * Check for housekeeping conflicts
-		 */
-		if (is_partition_valid(cs) &&
-		    prstate_housekeeping_conflict(old_prs, parent_prs,
-						  tmp->delmask, tmp->addmask))
-			part_error = PERR_HKEEPING;
-
 		/*
 		 * The new CPUs to be removed from parent's effective CPUs
 		 * must be present.
@@ -2398,6 +2390,21 @@ static void partition_cpus_change(struct cpuset *cs, struct cpuset *trialcs,
 		return;
 
 	prs_err = validate_partition(cs, trialcs);
+	if (!prs_err) {
+		int parent_prs = is_remote_partition(cs)
+			       ? PRS_ROOT : parent_cs(cs)->partition_root_state;
+		/*
+		 * Check for housekeeping CPUs conflict assuming that all the
+		 * exclusive CPUs belong to the current cpuset.
+		 */
+		compute_excpus(trialcs, tmp->new_cpus);
+		cpumask_andnot(tmp->addmask, tmp->new_cpus, cs->effective_xcpus);
+		cpumask_andnot(tmp->delmask, cs->effective_xcpus, tmp->new_cpus);
+		if (prstate_housekeeping_conflict(cs->partition_root_state,
+						  parent_prs, tmp->addmask,
+						  tmp->delmask))
+			prs_err = PERR_HKEEPING;
+	}
 	if (prs_err) {
 		WRITE_ONCE(cs->prs_err, prs_err);
 		trialcs->prs_err = prs_err;
@@ -2897,8 +2904,9 @@ static int update_prstate(struct cpuset *cs, int new_prs)
 		 * A change in load balance state only, no change in cpumasks.
 		 * Need to update isolated_cpus.
 		 */
+		compute_excpus(cs, tmpmask.new_cpus);
 		if (prstate_housekeeping_conflict(new_prs, parent_prs,
-						  cs->effective_xcpus, NULL))
+						  tmpmask.new_cpus, NULL))
 			err = PERR_HKEEPING;
 		else
 			isolcpus_updated = true;
-- 
2.55.0

Re: [PATCH-next v2 2/6] cgroup/cpuset: Consider all the exclusive CPUs when doing housekeeping check

[Guopeng Zhang]

在 2026/10/11 06:19, Waiman Long 写道:
> When prstate_housekeeping_conflict() is called to perform housekeeping
> check with cpumask changes, the exclusive CPUs owned by child partitions
> are excluded. If the current cpuset is an isolated partition with root
> partition children. It is possible the cpumask change will still leave
> active housekeeping CPUs but then no housekeeping CPUs will be left when
> the child partitions are disabled or invalidated.
> 
> To protect against this possibility, all the exclusive CPUs of the
> cpuset should be considered to be owned by the current cpuset when doing
> housekeeping check. Do that by calling prstate_housekeeping_conflict()
> in partition_cpus_change() and passing in the add_cpus and del_cpus
> parameters as if the cpuset owns all the exclusive CPUs.
> 
> With this change, the prstate_housekeeping_conflict() call in
> update_parent_effective_cpumask() with partcmd_update and newmask
> is now duplicative and can be removed. The housekeeping check in
> remote_cpus_update(), however, will still be useful as this function can
> be called directly from update_cpumasks_hier() without going through
> partition_cpus_change(). The update_parent_effective_cpumask() call
> from update_cpumasks_hier() is partcmd_update with no cpumask which
> still have the housekeeping check and so is covered.
> 
> In the case of partition state switch from isolated to root and vice
> versa, the set of exclusive CPUs are recomputed to include those
> transferred to child paritions as well.
> 
> Fixes: 4a74e418881f ("cgroup/cpuset: Check partition conflict with housekeeping setup")
> Signed-off-by: Waiman Long <longman@redhat.com>
> ---
>  kernel/cgroup/cpuset.c | 26 +++++++++++++++++---------
>  1 file changed, 17 insertions(+), 9 deletions(-)
> 
> diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
> index f3cebb277a68..1c0441fa3bea 100644
> --- a/kernel/cgroup/cpuset.c
> +++ b/kernel/cgroup/cpuset.c
> @@ -1907,14 +1907,6 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,
>  					       parent->effective_xcpus);
>  		}
>  
> -		/*
> -		 * Check for housekeeping conflicts
> -		 */
> -		if (is_partition_valid(cs) &&
> -		    prstate_housekeeping_conflict(old_prs, parent_prs,
> -						  tmp->delmask, tmp->addmask))
> -			part_error = PERR_HKEEPING;
> -
>  		/*
>  		 * The new CPUs to be removed from parent's effective CPUs
>  		 * must be present.
> @@ -2398,6 +2390,21 @@ static void partition_cpus_change(struct cpuset *cs, struct cpuset *trialcs,
>  		return;
>  
>  	prs_err = validate_partition(cs, trialcs);
> +	if (!prs_err) {
> +		int parent_prs = is_remote_partition(cs)
> +			       ? PRS_ROOT : parent_cs(cs)->partition_root_state;
> +		/*
> +		 * Check for housekeeping CPUs conflict assuming that all the
> +		 * exclusive CPUs belong to the current cpuset.
> +		 */
> +		compute_excpus(trialcs, tmp->new_cpus);
> +		cpumask_andnot(tmp->addmask, tmp->new_cpus, cs->effective_xcpus);
> +		cpumask_andnot(tmp->delmask, cs->effective_xcpus, tmp->new_cpus);

I ran a few local tests on v2 and still found the following gaps in the
housekeeping checks.

With CPUs 0-15 online and nohz_full=2-15:

    # cd /sys/fs/cgroup
    # echo +cpuset > cgroup.subtree_control
    # mkdir A
    # echo 1 > A/cpuset.cpus
    # echo isolated > A/cpuset.cpus.partition
    # echo 0-1 > A/cpuset.cpus
    # cat A/cpuset.cpus.partition
    isolated
    # cat A/cpuset.cpus.exclusive.effective
    0-1
    # cat cpuset.cpus.isolated
    0-1

The check computes delmask={1}, although CPU 1 is still isolated after the
update. That lets it isolate CPU 0, the last remaining housekeeping CPU.

On a fresh boot with the same nohz_full setting:

    # cd /sys/fs/cgroup
    # echo +cpuset > cgroup.subtree_control
    # mkdir A B
    # echo 1 > A/cpuset.cpus
    # echo 1 > A/cpuset.cpus.exclusive
    # echo isolated > A/cpuset.cpus.partition
    # echo +cpuset > A/cgroup.subtree_control
    # mkdir A/C
    # echo 1 > A/C/cpuset.cpus
    # echo root > A/C/cpuset.cpus.partition
    # echo 0 > B/cpuset.cpus
    # echo isolated > B/cpuset.cpus.partition
    # echo 1-2 > A/cpuset.cpus
    # echo 1-2 > A/cpuset.cpus.exclusive
    # cat cpuset.cpus.isolated
    0,2
    # echo member > A/C/cpuset.cpus.partition
    # cat A/cpuset.cpus.partition
    isolated
    # cat cpuset.cpus.isolated
    0-2

CPU 1 is present in both masks and cancels out of addmask while the root
child owns it. With nohz_full=2-15 and B isolating CPU 0, CPU 1 was the last
remaining housekeeping CPU; disabling the child leaves none.
This also reproduces before the series, so it is an existing issue.

> +		if (prstate_housekeeping_conflict(cs->partition_root_state,
> +						  parent_prs, tmp->addmask,
> +						  tmp->delmask))
> +			prs_err = PERR_HKEEPING;
> +	}

An invalid root with an explicit exclusive mask can retain effective_xcpus
without owning those CPUs. Subtracting this mask skips checking them when
the partition becomes valid again.

With CPUs 0-15 online and isolcpus=domain,10-11, on a fresh boot:

    # cd /sys/fs/cgroup
    # echo +cpuset > cgroup.subtree_control
    # mkdir A
    # echo 10 > A/cpuset.cpus
    # echo 10 > A/cpuset.cpus.exclusive
    # echo root > A/cpuset.cpus.partition
    # cat A/cpuset.cpus.partition
    root invalid (partition config conflicts with housekeeping setup)
    # cat A/cpuset.cpus.exclusive.effective
    10
    # echo 10,12 > A/cpuset.cpus
    # cat A/cpuset.cpus.partition
    root
    # cat A/cpuset.cpus.effective
    10
    # cat cpuset.cpus.effective
    0-9,11-15

addmask is empty here, so boot-isolated CPU 10 is allocated to the now-valid
root without being checked.

Should an invalid partition be checked against the full mask it will
acquire, rather than these deltas?

Thanks,
Guopeng
>  	if (prs_err) {
>  		WRITE_ONCE(cs->prs_err, prs_err);
>  		trialcs->prs_err = prs_err;
> @@ -2897,8 +2904,9 @@ static int update_prstate(struct cpuset *cs, int new_prs)
>  		 * A change in load balance state only, no change in cpumasks.
>  		 * Need to update isolated_cpus.
>  		 */
> +		compute_excpus(cs, tmpmask.new_cpus);
>  		if (prstate_housekeeping_conflict(new_prs, parent_prs,
> -						  cs->effective_xcpus, NULL))
> +						  tmpmask.new_cpus, NULL))
>  			err = PERR_HKEEPING;
>  		else
>  			isolcpus_updated = true;

[PATCH-next v2 3/6] cgroup/cpuset: Do housekeeping check before converting invalid partition to valid

[Waiman Long]

When a hotplug event happens or when update_cpumasks_hier() is called, an
invalid partition may be converted back to valid. However check against
housekeeping conflict isn't being done which can lead to a partition
state that violates the housekeeping rules. Update the code to allow
reviving an invalid partition only if it can pass the housekeeping check.

Fixes: 4a74e418881f ("cgroup/cpuset: Check partition conflict with housekeeping setup")
Signed-off-by: Waiman Long <longman@redhat.com>
---
 kernel/cgroup/cpuset.c | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 1c0441fa3bea..fb41cd76add6 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -1374,6 +1374,13 @@ static bool prstate_housekeeping_conflict(int new_prs, int parent_prs,
 	cpumask_var_t full_hk_cpus;
 	int res;
 
+	/*
+	 * For a currently invalid partition state, do the check assuming that
+	 * it may be converted back to valid.
+	 */
+	if (new_prs < 0)
+		new_prs = -new_prs;
+
 	if (housekeeping_enabled(HK_TYPE_DOMAIN_BOOT) && (new_prs == PRS_ROOT) &&
 	    !cpumask_subset(add_cpus, housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT)))
 		return true;
@@ -1956,9 +1963,16 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,
 			bool exclusive = true;
 
 			/*
-			 * Convert invalid partition to valid has to
-			 * pass the cpu exclusivity test.
+			 * Convert invalid partition to valid has to pass the
+			 * cpu exclusivity test as well as the housekeeping
+			 * check.
 			 */
+			if (prstate_housekeeping_conflict(cs->partition_root_state,
+							  parent->partition_root_state,
+							  xcpus, NULL)) {
+				part_error = PERR_HKEEPING;
+				goto write_error;
+			}
 			rcu_read_lock();
 			cpuset_for_each_child(child, css, parent) {
 				if (child == cs)
-- 
2.55.0

Re: [PATCH-next v2 3/6] cgroup/cpuset: Do housekeeping check before converting invalid partition to valid

[Ridong Chen]

On 10/11/2026 6:19 AM, Waiman Long wrote:
> When a hotplug event happens or when update_cpumasks_hier() is called, an
> invalid partition may be converted back to valid. However check against
> housekeeping conflict isn't being done which can lead to a partition
> state that violates the housekeeping rules. Update the code to allow
> reviving an invalid partition only if it can pass the housekeeping check.
> 
> Fixes: 4a74e418881f ("cgroup/cpuset: Check partition conflict with housekeeping setup")
> Signed-off-by: Waiman Long <longman@redhat.com>
> ---
>   kernel/cgroup/cpuset.c | 18 ++++++++++++++++--
>   1 file changed, 16 insertions(+), 2 deletions(-)
> 
> diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
> index 1c0441fa3bea..fb41cd76add6 100644
> --- a/kernel/cgroup/cpuset.c
> +++ b/kernel/cgroup/cpuset.c
> @@ -1374,6 +1374,13 @@ static bool prstate_housekeeping_conflict(int new_prs, int parent_prs,
>   	cpumask_var_t full_hk_cpus;
>   	int res;
>   
> +	/*
> +	 * For a currently invalid partition state, do the check assuming that
> +	 * it may be converted back to valid.
> +	 */
> +	if (new_prs < 0)
> +		new_prs = -new_prs;
> +

Personally, I don't like this. I think it should be certain what new_prs really 
is when prstate_housekeeping_conflict() is called; otherwise, it makes the code 
unreadable.

So I think when new_prs < 0, we should return false.

>   	if (housekeeping_enabled(HK_TYPE_DOMAIN_BOOT) && (new_prs == PRS_ROOT) &&
>   	    !cpumask_subset(add_cpus, housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT)))
>   		return true;
> @@ -1956,9 +1963,16 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,
>   			bool exclusive = true;
>   
>   			/*
> -			 * Convert invalid partition to valid has to
> -			 * pass the cpu exclusivity test.
> +			 * Convert invalid partition to valid has to pass the
> +			 * cpu exclusivity test as well as the housekeeping
> +			 * check.
>   			 */
> +			if (prstate_housekeeping_conflict(cs->partition_root_state,
> +							  parent->partition_root_state,
> +							  xcpus, NULL)) {
> +				part_error = PERR_HKEEPING;
> +				goto write_error;
> +			}
>   			rcu_read_lock();
>   			cpuset_for_each_child(child, css, parent) {
>   				if (child == cs)

-- 
Best regards
Ridong

Re: [PATCH-next v2 3/6] cgroup/cpuset: Do housekeeping check before converting invalid partition to valid

[Guopeng Zhang]

在 2026/10/11 06:19, Waiman Long 写道:
> When a hotplug event happens or when update_cpumasks_hier() is called, an
> invalid partition may be converted back to valid. However check against
> housekeeping conflict isn't being done which can lead to a partition
> state that violates the housekeeping rules. Update the code to allow
> reviving an invalid partition only if it can pass the housekeeping check.
> 
> Fixes: 4a74e418881f ("cgroup/cpuset: Check partition conflict with housekeeping setup")
> Signed-off-by: Waiman Long <longman@redhat.com>
> ---
>  kernel/cgroup/cpuset.c | 18 ++++++++++++++++--
>  1 file changed, 16 insertions(+), 2 deletions(-)
> 
> diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
> index 1c0441fa3bea..fb41cd76add6 100644
> --- a/kernel/cgroup/cpuset.c
> +++ b/kernel/cgroup/cpuset.c
> @@ -1374,6 +1374,13 @@ static bool prstate_housekeeping_conflict(int new_prs, int parent_prs,
>  	cpumask_var_t full_hk_cpus;
>  	int res;
>  
> +	/*
> +	 * For a currently invalid partition state, do the check assuming that
> +	 * it may be converted back to valid.
> +	 */
> +	if (new_prs < 0)
> +		new_prs = -new_prs;
> +
>  	if (housekeeping_enabled(HK_TYPE_DOMAIN_BOOT) && (new_prs == PRS_ROOT) &&
>  	    !cpumask_subset(add_cpus, housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT)))
>  		return true;
> @@ -1956,9 +1963,16 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,
>  			bool exclusive = true;
>  
>  			/*
> -			 * Convert invalid partition to valid has to
> -			 * pass the cpu exclusivity test.
> +			 * Convert invalid partition to valid has to pass the
> +			 * cpu exclusivity test as well as the housekeeping
> +			 * check.
>  			 */
> +			if (prstate_housekeeping_conflict(cs->partition_root_state,
> +							  parent->partition_root_state,
> +							  xcpus, NULL)) {
> +				part_error = PERR_HKEEPING;
> +				goto write_error;
> +			}
>  			rcu_read_lock();
>  			cpuset_for_each_child(child, css, parent) {
>  				if (child == cs)

Reviewed-by: Guopeng Zhang <zhangguopeng@kylinos.cn>

Thanks,
Guopeng

[PATCH-next v2 4/6] cgroup/cpuset: Properly disabling partition when partition state switching fails

[Waiman Long]

Before commit 103b08709e8a ("cgroup/cpuset: Fail if isolated and nohz_full
don't leave any housekeeping"), the partition state can be freely switched
from "root" to "isolated" and vice versa. After that commit, the switch
from "root" to "isolated" can fail if it exhausts all the housekeeping
CPUs. Later on, the switch from "isolated" to "root" can also fail if
some of the partition CPUs are boot-time isolated by "isolcpus".

A local or remote partition is made invalid when the switch fails.
However there are 2 problems with the existing invalidation code.

 1) Both the subpartitions_cpus and isolated_cpus are not properly
    updated.
 2) In the case of remote partition with remote_partition flag set, it
    is not cleared.

Fix this by calling the proper partition disabling function in this case.

On an x86 test system with boot option "isolcpus=10 cgroup_debug" set
and more than 16 cores, the following commands were executed after boot.

  # cd /sys/fs/cgroup
  # echo +cpuset > cgroup.subtree_control
  # cat cpuset.cpus.isolated
  10
  # mkdir A
  # echo 10-12 > A/cpuset.cpus
  # echo isolated > A/cpuset.cpus.partition
  # cat A/cpuset.cpus.partition
  isolated

Before the patch:

  # echo root > A/cpuset.cpus.partition
  # cat A/cpuset.cpus.partition
  root invalid (partition config conflicts with housekeeping setup)
  # cat cpuset.cpus.isolated .__DEBUG__.cpuset.cpus.subpartitions
  10-12
  10-12

After the patch:

  # echo root > A/cpuset.cpus.partition
  # cat A/cpuset.cpus.partition
  root invalid (partition config conflicts with housekeeping setup)
  # cat cpuset.cpus.isolated .__DEBUG__.cpuset.cpus.subpartitions
  10

Reported-by: Tejun Heo <tj@kernel.org>
Link: https://lore.kernel.org/lkml/7f4c57b26ad120ab30adf35653f1dd94@kernel.org/
Fixes: 4a74e418881f ("cgroup/cpuset: Check partition conflict with housekeeping setup")
Fixes: 103b08709e8a ("cgroup/cpuset: Fail if isolated and nohz_full don't leave any housekeeping")
Signed-off-by: Waiman Long <longman@redhat.com>
---
 kernel/cgroup/cpuset.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index fb41cd76add6..df1619d7a701 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -2863,6 +2863,7 @@ static int update_prstate(struct cpuset *cs, int new_prs)
 	struct cpuset *parent = parent_cs(cs);
 	struct tmpmasks tmpmask;
 	bool isolcpus_updated = false;
+	bool disable_partition = false;
 
 	if (old_prs == new_prs)
 		return 0;
@@ -2920,15 +2921,21 @@ static int update_prstate(struct cpuset *cs, int new_prs)
 		 */
 		compute_excpus(cs, tmpmask.new_cpus);
 		if (prstate_housekeeping_conflict(new_prs, parent_prs,
-						  tmpmask.new_cpus, NULL))
+						  tmpmask.new_cpus, NULL)) {
+			disable_partition = true;
 			err = PERR_HKEEPING;
-		else
+		} else {
 			isolcpus_updated = true;
+		}
 	} else {
 		/*
 		 * Switching back to member is always allowed even if it
 		 * disables child partitions.
 		 */
+		disable_partition = true;
+	}
+
+	if (disable_partition) {
 		if (is_remote_partition(cs))
 			remote_partition_disable(cs, &tmpmask);
 		else
@@ -2937,7 +2944,7 @@ static int update_prstate(struct cpuset *cs, int new_prs)
 
 		/*
 		 * Invalidation of child partitions will be done in
-		 * update_cpumasks_hier().
+		 * update_cpumasks_hier() below.
 		 */
 	}
 out:
@@ -2956,8 +2963,8 @@ static int update_prstate(struct cpuset *cs, int new_prs)
 		isolated_cpus_update(old_prs, new_prs, cs->effective_xcpus);
 	spin_unlock_irq(&callback_lock);
 
-	/* Force update if switching back to member & update effective_xcpus */
-	update_cpumasks_hier(cs, &tmpmask, !new_prs);
+	/* Force update if partition is disabled & update effective_xcpus */
+	update_cpumasks_hier(cs, &tmpmask, disable_partition);
 
 	/* A newly created partition must have effective_xcpus set */
 	WARN_ON_ONCE(!old_prs && (new_prs > 0)
-- 
2.55.0

Re: [PATCH-next v2 4/6] cgroup/cpuset: Properly disabling partition when partition state switching fails

[Guopeng Zhang]

在 2026/10/11 06:19, Waiman Long 写道:
> Before commit 103b08709e8a ("cgroup/cpuset: Fail if isolated and nohz_full
> don't leave any housekeeping"), the partition state can be freely switched
> from "root" to "isolated" and vice versa. After that commit, the switch
> from "root" to "isolated" can fail if it exhausts all the housekeeping
> CPUs. Later on, the switch from "isolated" to "root" can also fail if
> some of the partition CPUs are boot-time isolated by "isolcpus".
> 
> A local or remote partition is made invalid when the switch fails.
> However there are 2 problems with the existing invalidation code.
> 
>  1) Both the subpartitions_cpus and isolated_cpus are not properly
>     updated.
>  2) In the case of remote partition with remote_partition flag set, it
>     is not cleared.
> 
> Fix this by calling the proper partition disabling function in this case.
> 
> On an x86 test system with boot option "isolcpus=10 cgroup_debug" set
> and more than 16 cores, the following commands were executed after boot.
> 
>   # cd /sys/fs/cgroup
>   # echo +cpuset > cgroup.subtree_control
>   # cat cpuset.cpus.isolated
>   10
>   # mkdir A
>   # echo 10-12 > A/cpuset.cpus
>   # echo isolated > A/cpuset.cpus.partition
>   # cat A/cpuset.cpus.partition
>   isolated
> 
> Before the patch:
> 
>   # echo root > A/cpuset.cpus.partition
>   # cat A/cpuset.cpus.partition
>   root invalid (partition config conflicts with housekeeping setup)
>   # cat cpuset.cpus.isolated .__DEBUG__.cpuset.cpus.subpartitions
>   10-12
>   10-12
> 
> After the patch:
> 
>   # echo root > A/cpuset.cpus.partition
>   # cat A/cpuset.cpus.partition
>   root invalid (partition config conflicts with housekeeping setup)
>   # cat cpuset.cpus.isolated .__DEBUG__.cpuset.cpus.subpartitions
>   10
> 
> Reported-by: Tejun Heo <tj@kernel.org>
> Link: https://lore.kernel.org/lkml/7f4c57b26ad120ab30adf35653f1dd94@kernel.org/
> Fixes: 4a74e418881f ("cgroup/cpuset: Check partition conflict with housekeeping setup")
> Fixes: 103b08709e8a ("cgroup/cpuset: Fail if isolated and nohz_full don't leave any housekeeping")
> Signed-off-by: Waiman Long <longman@redhat.com>
> ---
>  kernel/cgroup/cpuset.c | 17 ++++++++++++-----
>  1 file changed, 12 insertions(+), 5 deletions(-)
> 
> diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
> index fb41cd76add6..df1619d7a701 100644
> --- a/kernel/cgroup/cpuset.c
> +++ b/kernel/cgroup/cpuset.c
> @@ -2863,6 +2863,7 @@ static int update_prstate(struct cpuset *cs, int new_prs)
>  	struct cpuset *parent = parent_cs(cs);
>  	struct tmpmasks tmpmask;
>  	bool isolcpus_updated = false;
> +	bool disable_partition = false;
>  
>  	if (old_prs == new_prs)
>  		return 0;
> @@ -2920,15 +2921,21 @@ static int update_prstate(struct cpuset *cs, int new_prs)
>  		 */
>  		compute_excpus(cs, tmpmask.new_cpus);
>  		if (prstate_housekeeping_conflict(new_prs, parent_prs,
> -						  tmpmask.new_cpus, NULL))
> +						  tmpmask.new_cpus, NULL)) {
> +			disable_partition = true;
>  			err = PERR_HKEEPING;
> -		else
> +		} else {
>  			isolcpus_updated = true;
> +		}
>  	} else {
>  		/*
>  		 * Switching back to member is always allowed even if it
>  		 * disables child partitions.
>  		 */
> +		disable_partition = true;
> +	}
> +
> +	if (disable_partition) {
>  		if (is_remote_partition(cs))
>  			remote_partition_disable(cs, &tmpmask);
>  		else
> @@ -2937,7 +2944,7 @@ static int update_prstate(struct cpuset *cs, int new_prs)
>  
>  		/*
>  		 * Invalidation of child partitions will be done in
> -		 * update_cpumasks_hier().
> +		 * update_cpumasks_hier() below.
>  		 */
>  	}
>  out:
> @@ -2956,8 +2963,8 @@ static int update_prstate(struct cpuset *cs, int new_prs)
>  		isolated_cpus_update(old_prs, new_prs, cs->effective_xcpus);
>  	spin_unlock_irq(&callback_lock);
>  
> -	/* Force update if switching back to member & update effective_xcpus */
> -	update_cpumasks_hier(cs, &tmpmask, !new_prs);
> +	/* Force update if partition is disabled & update effective_xcpus */
> +	update_cpumasks_hier(cs, &tmpmask, disable_partition);
>  
>  	/* A newly created partition must have effective_xcpus set */
>  	WARN_ON_ONCE(!old_prs && (new_prs > 0)


Reviewed-by: Guopeng Zhang <zhangguopeng@kylinos.cn>

Thanks,
Guopeng

[PATCH-next v2 5/6] selftests/cgroup: Add tests for housekeeping check

[Waiman Long]

Whenever a partition is enabled or the cpumask of a valid partition
changes, a housekeeping check will be performed to ensure that the
change won't violate the imposed housekeeping constraints. One of the
constraints is that a boot-time isolated CPU will not be used to form
a non-isolated root partition.  This requires the presence of boot-time
isolated CPUs when a test is run.

The test_cpuset_prs.sh test script is now updated to run a set a
special housekeeping check tests that will only be run if a suitable
boot-time isolated CPU is present. This will enable us to detect bugs
or regressions in the housekeeping checking code though a "isolcpus="
boot parameter with CPUs beyond CPU 8 must be present.

Adding test to check for exhaustion of all the housekeeping CPUs is
much harder and so will not be attempted at this time.

This patch also includes a minor fix to the REMOTE_TEST_MATRIX to avoid
test failure when the -v option is used.

Signed-off-by: Waiman Long <longman@redhat.com>
---
 .../selftests/cgroup/test_cpuset_prs.sh       | 109 ++++++++++++++----
 1 file changed, 87 insertions(+), 22 deletions(-)

diff --git a/tools/testing/selftests/cgroup/test_cpuset_prs.sh b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
index 7efd5e645767..3d392e8c66aa 100755
--- a/tools/testing/selftests/cgroup/test_cpuset_prs.sh
+++ b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
@@ -86,26 +86,37 @@ echo "" > test/cpuset.cpus
 
 #
 # If isolated CPUs have been reserved at boot time (as shown in
-# cpuset.cpus.isolated), these isolated CPUs should be outside of CPUs 0-8
-# that will be used by this script for testing purpose. If not, some of
-# the tests may fail incorrectly. Wait a bit and retry again just in case
-# these isolated CPUs are leftover from previous run and have just been
-# cleaned up earlier in this script.
+# /sys/devices/system/cpu/isolated), these isolated CPUs should be outside of
+# CPUs 0-8 that will be used by this script for testing purpose. If not, some
+# of the tests may fail incorrectly.
 #
-# These pre-isolated CPUs should stay in an isolated state throughout the
+# The current set of isolated CPUs (as shown in cpuset.cpus.isolated) should
+# be the same as the boot value. If not, wait a bit and retry again just in
+# case these isolated CPUs are leftover from previous run and have just been
+# cleaned up earlier in this script. If they still don't match, we report an
+# warning and skip the test.
+#
+# These boot isolated CPUs should stay in an isolated state throughout the
 # testing process for now.
 #
-BOOT_ISOLCPUS=$(cat $CGROUP2/cpuset.cpus.isolated)
+BOOT_ISOLCPUS=$(cat /sys/devices/system/cpu/isolated)
+CURRENT_ISOLCPUS=$(cat $CGROUP2/cpuset.cpus.isolated)
+FIRST_ISOLCPUS=
+TEST_ISOLCPUS=
+
 [[ -n "$BOOT_ISOLCPUS" ]] && {
+	FIRST_ISOLCPUS=$(echo $BOOT_ISOLCPUS | sed -e "s/[,-].*//")
+	[[ $FIRST_ISOLCPUS -le 8 ]] &&
+		skip_test "Boot-isolated CPUs ($BOOT_ISOLCPUS) overlap CPUs to be tested"
+	echo "Boot-isolated CPUs: $BOOT_ISOLCPUS"
+}
+
+[[ "$BOOT_ISOLCPUS" != "$CURRENT_ISOLCPUS" ]] && {
 	sleep 0.5
-	BOOT_ISOLCPUS=$(cat $CGROUP2/cpuset.cpus.isolated)
+	CURRENT_ISOLCPUS=$(cat $CGROUP2/cpuset.cpus.isolated)
+	[[ "$BOOT_ISOLCPUS" != "$CURRENT_ISOLCPUS" ]] &&
+		skip_test "Current isolated CPUs ($CURRENT_ISOLCPUS) don't match boot isolated CPUs!"
 }
-if [[ -n "$BOOT_ISOLCPUS" ]]
-then
-	[[ $(echo $BOOT_ISOLCPUS | sed -e "s/[,-].*//") -le 8 ]] &&
-		skip_test "Pre-isolated CPUs ($BOOT_ISOLCPUS) overlap CPUs to be tested"
-	echo "Pre-isolated CPUs: $BOOT_ISOLCPUS"
-fi
 
 cleanup()
 {
@@ -202,7 +213,7 @@ test_add_proc()
 #
 # ECPUs    - effective CPUs of cpusets
 # Pstate   - partition root state
-# ISOLCPUS - isolated CPUs (<icpus>[,<icpus2>])
+# ISOLCPUS - isolated CPUs (<icpus>[,<icpus2>] or .)
 #
 # Note that if there are 2 fields in ISOLCPUS, the first one is for
 # sched-debug matching which includes offline CPUs and single-CPU partitions
@@ -444,6 +455,35 @@ TEST_MATRIX=(
 	"   C0-3     .      .    C4-5   X3-5     .      .      .     1 A1:0-3|B1:4-5"
 )
 
+# Test matrix with boot time isolated CPUs present for housekeeping check testing
+ISOLCPUS_TEST_MATRIX=(
+	#  old-A1 old-A2 old-A3 old-B1 new-A1 new-A2 new-A3 new-B1 fail ECPUs Pstate ISOLCPUS
+	#  ------ ------ ------ ------ ------ ------ ------ ------ ---- ----- ------ --------
+	# A local isolated partition containing boot isolated CPU cannot be
+	# switched to root partition.
+	" C1-2,I:P2  .      .      .      P1     .      .      .     0 A1:1-2,I A1:P-1 ."
+
+	# A remote isolated partition containing boot isolated CPU cannot be
+	# switched to root partition.
+	" X1-2,I CX1-2,I:P2 .      .       .    P1      .      .     0 A2:1-2,I A2:P-1 ."
+
+	# A local isolated partition with a boot isolated CPU distributed to a
+	# child isolated partition also cannot be switched to a root partition.
+	" C1-3,I:P2 C1,I:P2 .      .      P1     .      .      .     0 A1:1-3,I|A2:1,I A1:P-1|A2:P-2 ."
+
+	# A remote isolated partition with a boot isolated CPU distributed to a
+	# child isolated partition also cannot be switched to a root partition.
+	" X1-3,I C1-3,I:P2 C1,I:P2 .       .    P1      .      .     0 A2:1-3,I|A3:1,I A2:P-1|A3:P-2 ."
+
+	# The cpumask of a local root partition cannot be changed to include a
+	# boot isolated CPU.
+	" C1-2:P1    .      .      .     C2,I    .      .      .     0 A1:2,I A1:P-1 ."
+
+	# The cpumask of a remote root partition cannot be changed to include a
+	# boot isolated CPU.
+	" CX1-2,I  CX1-2:P1 .      .       .   CX2,I    .      .     0 A1:1-2,I|A2:2,I A2:P-1 ."
+)
+
 #
 # Cpuset controller remote partition test matrix.
 #
@@ -496,8 +536,8 @@ REMOTE_TEST_MATRIX=(
 							 p1:P1|c11:P1|c12:P-1"
 	# Narrowing cpuset.cpus to previously sibling-excluded CPUs should
 	# not return CPUs that were never actually owned.
-	"  C1-4:P1   .   C1-2:P1  C1-3:P2  .       .  \
-	      .      .     .       C3      .       .     p1:4|c11:1-2|c12:3 \
+	"  C1-5:P1   .   C1-2:P1  C1-3:P2  .       .  \
+	      .      .     .       C3      .       .     p1:4-5|c11:1-2|c12:3 \
 							 p1:P1|c11:P1|c12:P2 3"
 	# Expanding cpuset.cpus to include a previously sibling-excluded CPU
 	# after the sibling has become a member should correctly request it.
@@ -541,6 +581,27 @@ write_cpu_online()
 	pause 0.05
 }
 
+#
+# Set the CPUS values
+#  $1 - the passed in cpu parameter
+#
+# The special ",I" suffix if present is being replaced by the first boot CPU
+# and assigned to the "CPUS" variable when the TEST_ISOLCPUS flag is set.
+# If not, the program will exit with an error.
+#
+set_cpus()
+{
+	local cpus=$1
+	if [[ $(expr "$cpus" : .*,I) -gt 0 ]]
+	then
+		[[ -z "$TEST_ISOLCPUS" ]] &&
+			skip_test "',I' added to non-isolcpus testing matrix!"
+		CPUS=$(echo $cpus | sed -e "s/,I/,$FIRST_ISOLCPUS/")
+	else
+		CPUS=$cpus
+	fi
+}
+
 #
 # Set controller state
 #  $1 - cgroup directory
@@ -575,16 +636,16 @@ set_ctrl_state()
 		}
 		case $CMD in
 		    X*)
-			CPUS=${CMD#?}
+			set_cpus ${CMD#?}
 			COMM="echo $CPUS > $XFILE"
 			eval $COMM $REDIRECT
 			;;
 		    CX*)
-			CPUS=${CMD#??}
+			set_cpus ${CMD#??}
 			COMM="echo $CPUS > $CFILE; echo $CPUS > $XFILE"
 			eval $COMM $REDIRECT
 			;;
-		    C*) CPUS=${CMD#?}
+		    C*) set_cpus ${CMD#?}
 			COMM="echo $CPUS > $CFILE"
 			eval $COMM $REDIRECT
 			;;
@@ -978,8 +1039,11 @@ run_state_test()
 	I=0
 	eval CNT="\${#$TEST[@]}"
 
+	# Skip isolcpus specific test matrix if FIRST_ISOLCPUS not defined
+	[[ -n "$TEST_ISOLCPUS" && -z "$FIRST_ISOLCPUS" ]] && return
+
 	reset_cgroup_states
-	console_msg "Running state transition test ..."
+	console_msg "Running state transition test for $TEST ..."
 
 	while [[ $I -lt $CNT ]]
 	do
@@ -998,7 +1062,7 @@ run_state_test()
 		NEW_A3=$7
 		NEW_B1=$8
 		RESULT=$9
-		ECPUS=${10}
+		ECPUS=$(echo ${10} | sed -e "s/,I/,${FIRST_ISOLCPUS}/g")
 		STATES=${11}
 		ICPUS=${12}
 
@@ -1284,6 +1348,7 @@ test_inotify()
 trap cleanup 0 2 3 6
 run_state_test TEST_MATRIX
 run_remote_state_test REMOTE_TEST_MATRIX
+TEST_ISOLCPUS=1 run_state_test ISOLCPUS_TEST_MATRIX
 test_isolated
 test_boot_isolated
 test_inotify
-- 
2.55.0

Re: [PATCH-next v2 5/6] selftests/cgroup: Add tests for housekeeping check

[Guopeng Zhang]

在 2026/10/11 06:19, Waiman Long 写道:
> Whenever a partition is enabled or the cpumask of a valid partition
> changes, a housekeeping check will be performed to ensure that the
> change won't violate the imposed housekeeping constraints. One of the
> constraints is that a boot-time isolated CPU will not be used to form
> a non-isolated root partition.  This requires the presence of boot-time
> isolated CPUs when a test is run.
> 
> The test_cpuset_prs.sh test script is now updated to run a set a
> special housekeeping check tests that will only be run if a suitable
> boot-time isolated CPU is present. This will enable us to detect bugs
> or regressions in the housekeeping checking code though a "isolcpus="
> boot parameter with CPUs beyond CPU 8 must be present.
> 
> Adding test to check for exhaustion of all the housekeeping CPUs is
> much harder and so will not be attempted at this time.
> 
> This patch also includes a minor fix to the REMOTE_TEST_MATRIX to avoid
> test failure when the -v option is used.
> 
> Signed-off-by: Waiman Long <longman@redhat.com>
> ---
>  .../selftests/cgroup/test_cpuset_prs.sh       | 109 ++++++++++++++----
>  1 file changed, 87 insertions(+), 22 deletions(-)
> 
> diff --git a/tools/testing/selftests/cgroup/test_cpuset_prs.sh b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
> index 7efd5e645767..3d392e8c66aa 100755
> --- a/tools/testing/selftests/cgroup/test_cpuset_prs.sh
> +++ b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
> @@ -86,26 +86,37 @@ echo "" > test/cpuset.cpus
>  
>  #
>  # If isolated CPUs have been reserved at boot time (as shown in
> -# cpuset.cpus.isolated), these isolated CPUs should be outside of CPUs 0-8
> -# that will be used by this script for testing purpose. If not, some of
> -# the tests may fail incorrectly. Wait a bit and retry again just in case
> -# these isolated CPUs are leftover from previous run and have just been
> -# cleaned up earlier in this script.
> +# /sys/devices/system/cpu/isolated), these isolated CPUs should be outside of
> +# CPUs 0-8 that will be used by this script for testing purpose. If not, some
> +# of the tests may fail incorrectly.
>  #
> -# These pre-isolated CPUs should stay in an isolated state throughout the
> +# The current set of isolated CPUs (as shown in cpuset.cpus.isolated) should
> +# be the same as the boot value. If not, wait a bit and retry again just in
> +# case these isolated CPUs are leftover from previous run and have just been
> +# cleaned up earlier in this script. If they still don't match, we report an
> +# warning and skip the test.
> +#
> +# These boot isolated CPUs should stay in an isolated state throughout the
>  # testing process for now.
>  #
> -BOOT_ISOLCPUS=$(cat $CGROUP2/cpuset.cpus.isolated)
> +BOOT_ISOLCPUS=$(cat /sys/devices/system/cpu/isolated)
> +CURRENT_ISOLCPUS=$(cat $CGROUP2/cpuset.cpus.isolated)
> +FIRST_ISOLCPUS=
> +TEST_ISOLCPUS=
> +
>  [[ -n "$BOOT_ISOLCPUS" ]] && {
> +	FIRST_ISOLCPUS=$(echo $BOOT_ISOLCPUS | sed -e "s/[,-].*//")
> +	[[ $FIRST_ISOLCPUS -le 8 ]] &&
> +		skip_test "Boot-isolated CPUs ($BOOT_ISOLCPUS) overlap CPUs to be tested"
> +	echo "Boot-isolated CPUs: $BOOT_ISOLCPUS"
> +}
> +
> +[[ "$BOOT_ISOLCPUS" != "$CURRENT_ISOLCPUS" ]] && {
>  	sleep 0.5
> -	BOOT_ISOLCPUS=$(cat $CGROUP2/cpuset.cpus.isolated)
> +	CURRENT_ISOLCPUS=$(cat $CGROUP2/cpuset.cpus.isolated)
> +	[[ "$BOOT_ISOLCPUS" != "$CURRENT_ISOLCPUS" ]] &&
> +		skip_test "Current isolated CPUs ($CURRENT_ISOLCPUS) don't match boot isolated CPUs!"
>  }
> -if [[ -n "$BOOT_ISOLCPUS" ]]
> -then
> -	[[ $(echo $BOOT_ISOLCPUS | sed -e "s/[,-].*//") -le 8 ]] &&
> -		skip_test "Pre-isolated CPUs ($BOOT_ISOLCPUS) overlap CPUs to be tested"
> -	echo "Pre-isolated CPUs: $BOOT_ISOLCPUS"
> -fi
>  
>  cleanup()
>  {
> @@ -202,7 +213,7 @@ test_add_proc()
>  #
>  # ECPUs    - effective CPUs of cpusets
>  # Pstate   - partition root state
> -# ISOLCPUS - isolated CPUs (<icpus>[,<icpus2>])
> +# ISOLCPUS - isolated CPUs (<icpus>[,<icpus2>] or .)
>  #
>  # Note that if there are 2 fields in ISOLCPUS, the first one is for
>  # sched-debug matching which includes offline CPUs and single-CPU partitions
> @@ -444,6 +455,35 @@ TEST_MATRIX=(
>  	"   C0-3     .      .    C4-5   X3-5     .      .      .     1 A1:0-3|B1:4-5"
>  )
>  
> +# Test matrix with boot time isolated CPUs present for housekeeping check testing
> +ISOLCPUS_TEST_MATRIX=(
> +	#  old-A1 old-A2 old-A3 old-B1 new-A1 new-A2 new-A3 new-B1 fail ECPUs Pstate ISOLCPUS
> +	#  ------ ------ ------ ------ ------ ------ ------ ------ ---- ----- ------ --------
> +	# A local isolated partition containing boot isolated CPU cannot be
> +	# switched to root partition.
> +	" C1-2,I:P2  .      .      .      P1     .      .      .     0 A1:1-2,I A1:P-1 ."
> +
> +	# A remote isolated partition containing boot isolated CPU cannot be
> +	# switched to root partition.
> +	" X1-2,I CX1-2,I:P2 .      .       .    P1      .      .     0 A2:1-2,I A2:P-1 ."
> +
> +	# A local isolated partition with a boot isolated CPU distributed to a
> +	# child isolated partition also cannot be switched to a root partition.
> +	" C1-3,I:P2 C1,I:P2 .      .      P1     .      .      .     0 A1:1-3,I|A2:1,I A1:P-1|A2:P-2 ."
> +
> +	# A remote isolated partition with a boot isolated CPU distributed to a
> +	# child isolated partition also cannot be switched to a root partition.
> +	" X1-3,I C1-3,I:P2 C1,I:P2 .       .    P1      .      .     0 A2:1-3,I|A3:1,I A2:P-1|A3:P-2 ."
> +
> +	# The cpumask of a local root partition cannot be changed to include a
> +	# boot isolated CPU.
> +	" C1-2:P1    .      .      .     C2,I    .      .      .     0 A1:2,I A1:P-1 ."
> +
> +	# The cpumask of a remote root partition cannot be changed to include a
> +	# boot isolated CPU.
> +	" CX1-2,I  CX1-2:P1 .      .       .   CX2,I    .      .     0 A1:1-2,I|A2:2,I A2:P-1 ."
> +)

Could we add two invalid-root revival cases with an explicit exclusive
mask here? For example:

        # An invalid root must stay invalid when cpus grows but exclusive
        # still contains a boot isolated CPU.
        " CX1,I:P1        .  .  .  C1-2,I   .  .  .  0 A1:1-2,I A1:P-1 ."

        # Growing exclusive must not revive it either.
        " C1-2,I:X1,I:P1 .  .  .  X1-2,I   .  .  .  0 A1:1-2,I A1:P-1 ."

Thanks,
Guopeng

> +
>  #
>  # Cpuset controller remote partition test matrix.
>  #
> @@ -496,8 +536,8 @@ REMOTE_TEST_MATRIX=(
>  							 p1:P1|c11:P1|c12:P-1"
>  	# Narrowing cpuset.cpus to previously sibling-excluded CPUs should
>  	# not return CPUs that were never actually owned.
> -	"  C1-4:P1   .   C1-2:P1  C1-3:P2  .       .  \
> -	      .      .     .       C3      .       .     p1:4|c11:1-2|c12:3 \
> +	"  C1-5:P1   .   C1-2:P1  C1-3:P2  .       .  \
> +	      .      .     .       C3      .       .     p1:4-5|c11:1-2|c12:3 \
>  							 p1:P1|c11:P1|c12:P2 3"
>  	# Expanding cpuset.cpus to include a previously sibling-excluded CPU
>  	# after the sibling has become a member should correctly request it.
> @@ -541,6 +581,27 @@ write_cpu_online()
>  	pause 0.05
>  }
>  
> +#
> +# Set the CPUS values
> +#  $1 - the passed in cpu parameter
> +#
> +# The special ",I" suffix if present is being replaced by the first boot CPU
> +# and assigned to the "CPUS" variable when the TEST_ISOLCPUS flag is set.
> +# If not, the program will exit with an error.
> +#
> +set_cpus()
> +{
> +	local cpus=$1
> +	if [[ $(expr "$cpus" : .*,I) -gt 0 ]]
> +	then
> +		[[ -z "$TEST_ISOLCPUS" ]] &&
> +			skip_test "',I' added to non-isolcpus testing matrix!"
> +		CPUS=$(echo $cpus | sed -e "s/,I/,$FIRST_ISOLCPUS/")
> +	else
> +		CPUS=$cpus
> +	fi
> +}
> +
>  #
>  # Set controller state
>  #  $1 - cgroup directory
> @@ -575,16 +636,16 @@ set_ctrl_state()
>  		}
>  		case $CMD in
>  		    X*)
> -			CPUS=${CMD#?}
> +			set_cpus ${CMD#?}
>  			COMM="echo $CPUS > $XFILE"
>  			eval $COMM $REDIRECT
>  			;;
>  		    CX*)
> -			CPUS=${CMD#??}
> +			set_cpus ${CMD#??}
>  			COMM="echo $CPUS > $CFILE; echo $CPUS > $XFILE"
>  			eval $COMM $REDIRECT
>  			;;
> -		    C*) CPUS=${CMD#?}
> +		    C*) set_cpus ${CMD#?}
>  			COMM="echo $CPUS > $CFILE"
>  			eval $COMM $REDIRECT
>  			;;
> @@ -978,8 +1039,11 @@ run_state_test()
>  	I=0
>  	eval CNT="\${#$TEST[@]}"
>  
> +	# Skip isolcpus specific test matrix if FIRST_ISOLCPUS not defined
> +	[[ -n "$TEST_ISOLCPUS" && -z "$FIRST_ISOLCPUS" ]] && return
> +
>  	reset_cgroup_states
> -	console_msg "Running state transition test ..."
> +	console_msg "Running state transition test for $TEST ..."
>  
>  	while [[ $I -lt $CNT ]]
>  	do
> @@ -998,7 +1062,7 @@ run_state_test()
>  		NEW_A3=$7
>  		NEW_B1=$8
>  		RESULT=$9
> -		ECPUS=${10}
> +		ECPUS=$(echo ${10} | sed -e "s/,I/,${FIRST_ISOLCPUS}/g")
>  		STATES=${11}
>  		ICPUS=${12}
>  
> @@ -1284,6 +1348,7 @@ test_inotify()
>  trap cleanup 0 2 3 6
>  run_state_test TEST_MATRIX
>  run_remote_state_test REMOTE_TEST_MATRIX
> +TEST_ISOLCPUS=1 run_state_test ISOLCPUS_TEST_MATRIX
>  test_isolated
>  test_boot_isolated
>  test_inotify

[PATCH-next v2 6/6] selftests/cgroup: Skip test_cpuset_prs.sh test if some required CPUs are offline

[Waiman Long]

The test_cpuset_prs.sh test assumes that all the relevant CPUs being
tested are online. If some of them are offline, some of the test cases
may fail. So skip the test if the required CPUs are either offline or
not present.

Signed-off-by: Waiman Long <longman@redhat.com>
---
 .../selftests/cgroup/test_cpuset_prs.sh       | 20 +++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/tools/testing/selftests/cgroup/test_cpuset_prs.sh b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
index 3d392e8c66aa..f57971ebb90e 100755
--- a/tools/testing/selftests/cgroup/test_cpuset_prs.sh
+++ b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
@@ -118,6 +118,26 @@ TEST_ISOLCPUS=
 		skip_test "Current isolated CPUs ($CURRENT_ISOLCPUS) don't match boot isolated CPUs!"
 }
 
+#
+# This test assumes that all the relevant CPUs being tested are online.
+# If some of them are offline, some of the tests may fail. So we are going to
+# skip running this test script if some of the required CPUs may be offline.
+# The required CPUs include CPUs 0-7 and the first boot time isolated CPU if
+# present. However the check below is not exhaustive and can be a false
+# positive, but users should not run this test if some CPUs are offline.
+#
+ONLINE_CPUS=$(cat /sys/devices/system/cpu/online | sed -e "s/,.*//")
+ONLINE_FIRST=$(echo $ONLINE_CPUS | sed -e "s/-.*//")
+ONLINE_LAST=$(echo $ONLINE_CPUS | sed -e "s/.*-//")
+if [[ -n "$FIRST_ISOLCPUS" ]]
+then
+	LAST_REQUIRED_CPU=$FIRST_ISOLCPUS
+else
+	LAST_REQUIRED_CPU=7
+fi
+[[ $ONLINE_FIRST != 0 || $ONLINE_LAST -lt $LAST_REQUIRED_CPU ]] &&
+	skip_test "Some of the required CPUs are offline or not present!"
+
 cleanup()
 {
 	online_cpus
-- 
2.55.0

Re: [PATCH-next v2 6/6] selftests/cgroup: Skip test_cpuset_prs.sh test if some required CPUs are offline

[Guopeng Zhang]

在 2026/10/11 06:19, Waiman Long 写道:
> The test_cpuset_prs.sh test assumes that all the relevant CPUs being
> tested are online. If some of them are offline, some of the test cases
> may fail. So skip the test if the required CPUs are either offline or
> not present.
> 
> Signed-off-by: Waiman Long <longman@redhat.com>
> ---
>  .../selftests/cgroup/test_cpuset_prs.sh       | 20 +++++++++++++++++++
>  1 file changed, 20 insertions(+)
> 
> diff --git a/tools/testing/selftests/cgroup/test_cpuset_prs.sh b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
> index 3d392e8c66aa..f57971ebb90e 100755
> --- a/tools/testing/selftests/cgroup/test_cpuset_prs.sh
> +++ b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
> @@ -118,6 +118,26 @@ TEST_ISOLCPUS=
>  		skip_test "Current isolated CPUs ($CURRENT_ISOLCPUS) don't match boot isolated CPUs!"
>  }
>  
> +#
> +# This test assumes that all the relevant CPUs being tested are online.
> +# If some of them are offline, some of the tests may fail. So we are going to
> +# skip running this test script if some of the required CPUs may be offline.
> +# The required CPUs include CPUs 0-7 and the first boot time isolated CPU if
> +# present. However the check below is not exhaustive and can be a false
> +# positive, but users should not run this test if some CPUs are offline.
> +#
> +ONLINE_CPUS=$(cat /sys/devices/system/cpu/online | sed -e "s/,.*//")
> +ONLINE_FIRST=$(echo $ONLINE_CPUS | sed -e "s/-.*//")
> +ONLINE_LAST=$(echo $ONLINE_CPUS | sed -e "s/.*-//")
> +if [[ -n "$FIRST_ISOLCPUS" ]]
> +then
> +	LAST_REQUIRED_CPU=$FIRST_ISOLCPUS
> +else
> +	LAST_REQUIRED_CPU=7
> +fi
> +[[ $ONLINE_FIRST != 0 || $ONLINE_LAST -lt $LAST_REQUIRED_CPU ]] &&
> +	skip_test "Some of the required CPUs are offline or not present!"
> +

Could this check run before the setup that creates the test cgroup and
changes sched/verbose?

skip_test() exits directly, and the cleanup trap hasn't been installed
here. We have already created test; with -v, sched/verbose has also been
set to Y. This exit leaves the cgroup behind and bypasses restoring
verbose.

Thanks,
Guopeng
>  cleanup()
>  {
>  	online_cpus