mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Waiman Long <longman@redhat.com>
To: "Ridong Chen" <ridong.chen@linux.dev>,
	"Tejun Heo" <tj@kernel.org>,
	"Johannes Weiner" <hannes@cmpxchg.org>,
	"Michal Koutný" <mkoutny@suse.com>,
	"Shuah Khan" <shuah@kernel.org>
Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-kselftest@vger.kernel.org,
	Guopeng Zhang <guopeng.zhang@linux.dev>,
	Waiman Long <longman@redhat.com>,
	Sashiko <sashiko-bot@kernel.org>
Subject: [PATCH-next v4 5/6] cgroup/cpuset: Handle remote partitions before local partitions in cpuset_handle_hotplug()
Date: Sun, 15 Nov 2026 20:23:08 -0500	[thread overview]
Message-ID: <20261116012330.2271010-6-longman@redhat.com> (raw)
In-Reply-To: <20261116012330.2271010-1-longman@redhat.com>

When there is a valid remote partition underneath an invalid local
partition with subpartitions_cpus cleared because of missing active
CPUs in the top_cpuset partition, the current iteration logic in
cpuset_handle_hotplug() will handle the invalid local partition first
and potentially make the local partition valid. The remote partition
will be handled later with the potential that both the local and remote
partitions will be valid with overlapping exclusive CPUs violating the
cpu exclusivity rule.

Fix this problem by adding a new remote_partition_scan() helper to
iterate the existing remote partitions first before iterating down
the hierarchy to handle the local partitions. That also means that
when subpartitions_cpus is cleared, remote partitions have precedence
to acquire the exclusive CPUs first before local partitions. It also
gives remote partition the ability to recover from the clearing of
subpartitions_cpus as all the remote partitions will be disabled before
this commit.

A remote partition can only be created if there is at least one active
CPU in its effective_xcpus. So remote_partition_scan() will also
invalidate the remote partition if there is no active CPU left in its
effective_xcpus. This requires a minor change in a test_cpuset_prs.sh
test case where a remote partition with no active CPU left will now
be invalidated.

With the addition of the remote_partition_scan() call in
cpuset_handle_hotplug(), there is no longer a need to do a remote
partition check in cpuset_hotplug_update_tasks() and the corresponding
code can be removed.

While at it, the subpartition_cpus clearing logic is enhanced to
clear it when no domain housekeeping CPUs are left in the top_cpuset
partition. When the remote partition is disabled, the effective_cpus
is modified in a such a way that cpus_updated will be set true later
in cpuset_hotplug_update_tasks() so that the CPU affinity of the
tasks within the partition will be updated. This will also fix a
bug reported by sashiko that CPU affinity will not be updated when
remote_partition_disable() is called in such circumstance.

When efffective_cpus is read during this interim period, the content
will be invalid before and after this commit.

Fixes: 181c8e091aae ("cgroup/cpuset: Introduce remote partition")
Reported-by: Guopeng Zhang <guopeng.zhang@linux.dev>
Link: https://lore.kernel.org/lkml/fe7aa630-d323-4981-9f89-0958cb6204a2@linux.dev
Reported-by: Sashiko <sashiko-bot@kernel.org>
Link: https://sashiko.dev/#/bug/linux-ab90f7e8-0d60-40ca-83e9-e405075b24d2
Signed-off-by: Waiman Long <longman@redhat.com>
---
 kernel/cgroup/cpuset.c                        | 97 ++++++++++++++++---
 .../selftests/cgroup/test_cpuset_prs.sh       |  2 +-
 2 files changed, 84 insertions(+), 15 deletions(-)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 0fe9f2731fee..64cbcbdbf6da 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -153,6 +153,12 @@ static cpumask_var_t	isolated_cpus;		/* CSCB */
  */
 static struct list_head remote_children;	/* RWCS */
 
+/*
+ * Temporary cpumask for local cpumask computation
+ * The cpumask may be changed when a cpuset helper is being called.
+ */
+static cpumask_var_t	tmp_cpus;		/* RWCS */
+
 /*
  * Set if housekeeping cpumasks are to be updated.
  */
@@ -1873,6 +1879,67 @@ static void remote_cpus_update(struct cpuset *cs, struct cpumask *xcpus,
 	remote_partition_disable(cs, tmp);
 }
 
+/**
+ * remote_partition_scan: scan remote partitions to check exclusive CPUs
+ * @tmpmask: ptr to tmpmasks structure
+ *
+ * This function is called in cpuset_handle_hotplug() after the update on
+ * top_cpuset.effective_cpus when there is cpu online/offline operation.
+ * As subpartitions_cpus may have been cleared, we need to check if the remote
+ * partition should acquire the exclusive CPUs again without impacting
+ * effective_cpus or it needs to be disabled when one or more of the following
+ * conditions is true.
+ * 1) All the domain housekeeping CPUs in the top_cpuset will be taken away
+ * 2) A populated partition with no active CPU or no active CPU in
+ *    effective_xcpus
+ */
+static void remote_partition_scan(struct tmpmasks *tmp)
+{
+	struct cpuset *cs, *next;
+
+	rcu_read_lock();
+	list_for_each_entry_safe(cs, next, &remote_children, remote_sibling) {
+		bool noxcpus = !cpumask_subset(cs->effective_xcpus, subpartitions_cpus);
+		bool nocpus = !cpumask_intersects(cs->effective_xcpus, cpu_active_mask) ||
+			      (!cpumask_intersects(cs->effective_cpus, cpu_active_mask) &&
+			       partition_is_populated(cs, NULL));
+		int cpu;
+
+		cpumask_and(tmp_cpus, top_cpuset.effective_cpus,
+			    housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT));
+		if (noxcpus && !nocpus && !cpumask_subset(tmp_cpus, cs->effective_xcpus)) {
+			/*
+			 * Reacquire exclusive CPUs & update top_cpuset.effective_cpus
+			 */
+			spin_lock_irq(&callback_lock);
+			cpumask_andnot(subpartitions_cpus, subpartitions_cpus,
+				       cs->effective_xcpus);
+			cpumask_andnot(top_cpuset.effective_cpus, top_cpuset.effective_xcpus,
+				       cs->effective_xcpus);
+			spin_unlock_irq(&callback_lock);
+			continue;
+		}
+
+		if (noxcpus || nocpus || cpumask_empty(tmp_cpus)) {
+			/*
+			 * Disable remote partition & modifiy effective_cpus
+			 * so that cpus_updated will be set true in
+			 * cpuset_hotplug_update_tasks().
+			 */
+			WRITE_ONCE(cs->prs_err, PERR_HOTPLUG);
+			remote_partition_disable(cs, tmp);
+			cpu = cpumask_first(cs->effective_cpus);
+			spin_lock_irq(&callback_lock);
+			if (cpu < nr_cpu_ids)
+				cpumask_clear_cpu(cpu, cs->effective_cpus);
+			else
+				cpumask_set_cpu(0, cs->effective_cpus);
+			spin_unlock_irq(&callback_lock);
+		}
+	}
+	rcu_read_unlock();
+}
+
 /**
  * update_parent_effective_cpumask - update effective_cpus mask of parent cpuset
  * @cs:      The cpuset that requests change in partition root state
@@ -3919,6 +3986,7 @@ int __init cpuset_init(void)
 	BUG_ON(!zalloc_cpumask_var(&subpartitions_cpus, GFP_KERNEL));
 	BUG_ON(!zalloc_cpumask_var(&isolated_cpus, GFP_KERNEL));
 	BUG_ON(!zalloc_cpumask_var(&isolated_hk_cpus, GFP_KERNEL));
+	BUG_ON(!alloc_cpumask_var(&tmp_cpus, GFP_KERNEL));
 
 	cpumask_setall(top_cpuset.cpus_allowed);
 	nodes_setall(top_cpuset.mems_allowed);
@@ -3993,14 +4061,6 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs, struct tmpmasks *tmp)
 	if (!tmp || !cs->partition_root_state)
 		goto update_tasks;
 
-	if (is_remote_partition(cs) &&
-	    (!cpumask_subset(cs->effective_xcpus, subpartitions_cpus) ||
-	     (cpumask_empty(&new_cpus) && partition_is_populated(cs, NULL)))) {
-		WRITE_ONCE(cs->prs_err, PERR_HOTPLUG);
-		remote_partition_disable(cs, tmp);
-		compute_effective_cpumask(&new_cpus, cs, parent);
-	}
-
 	/*
 	 * Force the partition to become invalid if either one of
 	 * the following conditions hold:
@@ -4104,22 +4164,31 @@ static void cpuset_handle_hotplug(void)
 		if (!on_dfl)
 			cpumask_copy(top_cpuset.cpus_allowed, &new_cpus);
 		/*
-		 * Make sure that CPUs allocated to child partitions
-		 * do not show up in effective_cpus. If no CPU is left,
-		 * we clear the subpartitions_cpus & let the child partitions
-		 * fight for the CPUs again.
+		 * Make sure that CPUs allocated to child partitions do not
+		 * show up in effective_cpus. If no housekeeping CPU is left,
+		 * we clear the subpartitions_cpus, scan the remote partitions
+		 * to determine if they can remain valid & let the descendant
+		 * local partitions fight for the CPUs again.
 		 */
 		if (!cpumask_empty(subpartitions_cpus)) {
+			cpumask_andnot(&new_cpus, cpu_active_mask,
+				       housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT));
 			if (cpumask_subset(&new_cpus, subpartitions_cpus)) {
 				cpumask_clear(subpartitions_cpus);
+				cpumask_copy(&new_cpus, cpu_active_mask);
 			} else {
-				cpumask_andnot(&new_cpus, &new_cpus,
+				cpumask_andnot(&new_cpus, cpu_active_mask,
 					       subpartitions_cpus);
 			}
 		}
 		cpumask_copy(top_cpuset.effective_cpus, &new_cpus);
 		spin_unlock_irq(&callback_lock);
-		/* we don't mess with cpumasks of tasks in top_cpuset */
+		/*
+		 * We don't mess with cpumasks of tasks in top_cpuset, but we
+		 * have to examine impact on remote partition roots if present.
+		 */
+		if (cpuset_v2() && ptmp)
+			remote_partition_scan(ptmp);
 	}
 
 	/* synchronize mems_allowed to N_MEMORY */
diff --git a/tools/testing/selftests/cgroup/test_cpuset_prs.sh b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
index 81f0eecaca80..5c7b583cba18 100755
--- a/tools/testing/selftests/cgroup/test_cpuset_prs.sh
+++ b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
@@ -378,7 +378,7 @@ TEST_MATRIX=(
 	# Remote partition offline tests
 	"   C0-3    C1-3  C2-3     .    X2-3   X2-3 X2-3:P2:O2=0 .   0 A1:0-1|A2:1|A3:3 A1:P0|A3:P2 2-3"
 	"   C0-3    C1-3  C2-3     .    X2-3   X2-3 X2-3:P2:O2=0 O2=1 0 A1:0-1|A2:1|A3:2-3 A1:P0|A3:P2 2-3"
-	"   C0-3    C1-3   C3      .    X2-3   X2-3    P2:O3=0   .   0 A1:0-2|A2:1-2|A3: A1:P0|A3:P2 3"
+	"   C0-3    C1-3   C3      .    X2-3   X2-3    P2:O3=0   .   0 A1:0-2|A2:1-2|A3:1-2 A1:P0|A3:P-2 3|"
 	"   C0-3    C1-3   C3      .    X2-3   X2-3   T:P2:O3=0  .   0 A1:0-2|A2:1-2|A3:1-2 A1:P0|A3:P-2 3|"
 
 	# An invalidated remote partition cannot self-recover from hotplug
-- 
2.56.0


  parent reply	other threads:[~2026-11-16  1:24 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-16  1:23 [PATCH-next v4 0/6] cgroup/cpuset: More fix patches Waiman Long
2026-11-16  1:23 ` [PATCH-next v4 1/6] cgroup/cpuset: Fix incorrect partition invalidation logic in cpuset_hotplug_update_tasks() Waiman Long
2026-11-16  1:23 ` [PATCH-next v4 2/6] cgroup/cpuset: excpus parameter of remote_cpus_update() must have at least 1 active CPU Waiman Long
2026-11-16  1:23 ` [PATCH-next v4 3/6] cgroup/cpuset: Check remote partition underneath before making an invalid partition valid Waiman Long
2026-11-16  1:23 ` [PATCH-next v4 4/6] cgroup/cpuset: Revert "remove global remote_children list" Waiman Long
2026-11-16  1:23 ` Waiman Long [this message]
2026-11-16  1:23 ` [PATCH-next v4 6/6] cgroup/cpuset: Fix lockdep_is_cpuset_held() Waiman Long

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261116012330.2271010-6-longman@redhat.com \
    --to=longman@redhat.com \
    --cc=cgroups@vger.kernel.org \
    --cc=guopeng.zhang@linux.dev \
    --cc=hannes@cmpxchg.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=mkoutny@suse.com \
    --cc=ridong.chen@linux.dev \
    --cc=sashiko-bot@kernel.org \
    --cc=shuah@kernel.org \
    --cc=tj@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®