[PATCH -next RFC 0/2] cpuset: Add cpuset.mems.spread_page to cgroup v2

[PATCH -next RFC 0/2] cpuset: Add cpuset.mems.spread_page to cgroup v2

[Cai Xinchen]

I encountered a scenario where a machine with 1.5TB of memory,
while testing the Spark TPCDS 3TB dataset, experienced a significant
concentration of page cache usage on one of the NUMA nodes.
I discovered that the DataNode process had requested a large amount
of page cache. most of the page cache was concentrated in one NUMA node,
ultimately leading to the exhaustion of memory in that NUMA node.
At this point, all other processes in that NUMA node have to alloc
memory across NUMA nodes, or even across sockets. This eventually
caused a degradation in the end-to-end performance of the Spark test.

I do not want to restart the Spark DataNode service during business
operations. This issue can be resolved by migrating the DataNode into
a cpuset, dropping the cache, and setting cpuset.memory_spread_page to
allow it to evenly request memory. The core business threads could still
allocate local numa memory. After using cpuset.memory_spread_page, the
performance in the tpcds-99 test is improved by 2%.

The key point is that the even distribution of page cache within the
DataNode process (rather than the current NUMA distribution) does not
significantly affect end-to-end performance. However, the allocation
of core business processes, such as Executors, to the same NUMA node
does have a noticeable impact on end-to-end performance.

However, I found that cgroup v2 does not provide this interface. I
believe this interface still holds value in addressing issues caused
by uneven distribution of page cache allocation among process groups.

Thus I add cpuset.mems.spread_page to cpuset v2 interface.

Cai Xinchen (2):
  cpuset: Move cpuset1_update_spread_flag to cpuset
  cpuset: Add spread_page interface to cpuset v2

 kernel/cgroup/cpuset-internal.h |  6 ++--
 kernel/cgroup/cpuset-v1.c       | 25 +----------------
 kernel/cgroup/cpuset.c          | 49 ++++++++++++++++++++++++++++++++-
 3 files changed, 51 insertions(+), 29 deletions(-)

-- 
2.34.1

[PATCH -next RFC 1/2] cpuset: Move cpuset1_update_spread_flag to cpuset

[Cai Xinchen]

To add cpuset.mems.spread_flag to cgroup v2
Move cpuset1_update_spread_flag to cpuset.c
Remove check for cgroup v2 and remove memory_spread_slab update
because it is useless now.

Signed-off-by: Cai Xinchen <caixinchen1@huawei.com>
---
 kernel/cgroup/cpuset-internal.h |  6 ++----
 kernel/cgroup/cpuset-v1.c       | 25 +------------------------
 kernel/cgroup/cpuset.c          | 16 +++++++++++++++-
 3 files changed, 18 insertions(+), 29 deletions(-)

diff --git a/kernel/cgroup/cpuset-internal.h b/kernel/cgroup/cpuset-internal.h
index 337608f408ce..1cd1795729c7 100644
--- a/kernel/cgroup/cpuset-internal.h
+++ b/kernel/cgroup/cpuset-internal.h
@@ -273,6 +273,8 @@ void cpuset_callback_unlock_irq(void);
 void cpuset_update_tasks_cpumask(struct cpuset *cs, struct cpumask *new_cpus);
 void cpuset_update_tasks_nodemask(struct cpuset *cs);
 int cpuset_update_flag(cpuset_flagbits_t bit, struct cpuset *cs, int turning_on);
+void cpuset_update_task_spread_flags(struct cpuset *cs,
+					struct task_struct *tsk);
 ssize_t cpuset_write_resmask(struct kernfs_open_file *of,
 				    char *buf, size_t nbytes, loff_t off);
 int cpuset_common_seq_show(struct seq_file *sf, void *v);
@@ -285,8 +287,6 @@ void cpuset_full_unlock(void);
 #ifdef CONFIG_CPUSETS_V1
 extern struct cftype cpuset1_files[];
 void fmeter_init(struct fmeter *fmp);
-void cpuset1_update_task_spread_flags(struct cpuset *cs,
-					struct task_struct *tsk);
 void cpuset1_update_tasks_flags(struct cpuset *cs);
 void cpuset1_hotplug_update_tasks(struct cpuset *cs,
 			    struct cpumask *new_cpus, nodemask_t *new_mems,
@@ -294,8 +294,6 @@ void cpuset1_hotplug_update_tasks(struct cpuset *cs,
 int cpuset1_validate_change(struct cpuset *cur, struct cpuset *trial);
 #else
 static inline void fmeter_init(struct fmeter *fmp) {}
-static inline void cpuset1_update_task_spread_flags(struct cpuset *cs,
-					struct task_struct *tsk) {}
 static inline void cpuset1_update_tasks_flags(struct cpuset *cs) {}
 static inline void cpuset1_hotplug_update_tasks(struct cpuset *cs,
 			    struct cpumask *new_cpus, nodemask_t *new_mems,
diff --git a/kernel/cgroup/cpuset-v1.c b/kernel/cgroup/cpuset-v1.c
index 12e76774c75b..bcd20f198a0d 100644
--- a/kernel/cgroup/cpuset-v1.c
+++ b/kernel/cgroup/cpuset-v1.c
@@ -203,29 +203,6 @@ static s64 cpuset_read_s64(struct cgroup_subsys_state *css, struct cftype *cft)
 	return 0;
 }
 
-/*
- * update task's spread flag if cpuset's page/slab spread flag is set
- *
- * Call with callback_lock or cpuset_mutex held. The check can be skipped
- * if on default hierarchy.
- */
-void cpuset1_update_task_spread_flags(struct cpuset *cs,
-					struct task_struct *tsk)
-{
-	if (cgroup_subsys_on_dfl(cpuset_cgrp_subsys))
-		return;
-
-	if (is_spread_page(cs))
-		task_set_spread_page(tsk);
-	else
-		task_clear_spread_page(tsk);
-
-	if (is_spread_slab(cs))
-		task_set_spread_slab(tsk);
-	else
-		task_clear_spread_slab(tsk);
-}
-
 /**
  * cpuset1_update_tasks_flags - update the spread flags of tasks in the cpuset.
  * @cs: the cpuset in which each task's spread flags needs to be changed
@@ -241,7 +218,7 @@ void cpuset1_update_tasks_flags(struct cpuset *cs)
 
 	css_task_iter_start(&cs->css, 0, &it);
 	while ((task = css_task_iter_next(&it)))
-		cpuset1_update_task_spread_flags(cs, task);
+		cpuset_update_task_spread_flags(cs, task);
 	css_task_iter_end(&it);
 }
 
diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 52468d2c178a..3a0d443180c6 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -3205,7 +3205,7 @@ static void cpuset_attach_task(struct cpuset *cs, struct task_struct *task)
 	WARN_ON_ONCE(set_cpus_allowed_ptr(task, cpus_attach));
 
 	cpuset_change_task_nodemask(task, &cpuset_attach_nodemask_to);
-	cpuset1_update_task_spread_flags(cs, task);
+	cpuset_update_task_spread_flags(cs, task);
 }
 
 static void cpuset_attach(struct cgroup_taskset *tset)
@@ -3447,6 +3447,20 @@ static ssize_t cpuset_partition_write(struct kernfs_open_file *of, char *buf,
 	return retval ?: nbytes;
 }
 
+/*
+ * update task's spread flag if cpuset's page spread flag is set
+ *
+ * Call with callback_lock or cpuset_mutex held.
+ */
+void cpuset_update_task_spread_flags(struct cpuset *cs,
+					struct task_struct *tsk)
+{
+	if (is_spread_page(cs))
+		task_set_spread_page(tsk);
+	else
+		task_clear_spread_page(tsk);
+}
+
 /*
  * This is currently a minimal set for the default hierarchy. It can be
  * expanded later on by migrating more features and control files from v1.
-- 
2.34.1

[PATCH -next RFC 2/2] cpuset: Add spread_page interface to cpuset v2

[Cai Xinchen]

I encountered a scenario where a machine with 1.5TB of memory,
while testing the Spark TPCDS 3TB dataset, experienced a significant
concentration of page cache usage on one of the NUMA nodes.
I discovered that the DataNode process had requested a large amount
of page cache. most of the page cache was concentrated in one NUMA node,
ultimately leading to the exhaustion of memory in that NUMA node.
At this point, all other processes in that NUMA node have to alloc
memory across NUMA nodes, or even across sockets. This eventually
caused a degradation in the end-to-end performance of the Spark test.

I do not want to restart the Spark DataNode service during business
operations. This issue can be resolved by migrating the DataNode into
a cpuset, dropping the cache, and setting cpuset.memory_spread_page to
allow it to evenly request memory. The core business threads could still
allocate local numa memory. After using cpuset.memory_spread_page, the
performance in the tpcds-99 test is improved by 2%.

The key point is that the even distribution of page cache within the
DataNode process (rather than the current NUMA distribution) does not
significantly affect end-to-end performance. However, the allocation
of core business processes, such as Executors, to the same NUMA node
does have a noticeable impact on end-to-end performance.

However, I found that cgroup v2 does not provide this interface. I
believe this interface still holds value in addressing issues caused
by uneven distribution of page cache allocation among process groups.

Thus I add cpuset.mems.spread_page to cpuset v2 interface.

Signed-off-by: Cai Xinchen <caixinchen1@huawei.com>
---
 kernel/cgroup/cpuset.c | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 3a0d443180c6..04a24642f490 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -3461,6 +3461,32 @@ void cpuset_update_task_spread_flags(struct cpuset *cs,
 		task_clear_spread_page(tsk);
 }
 
+static int cpuset_update_spread_page(struct cgroup_subsys_state *css, struct cftype *cft,
+			    u64 val)
+{
+	struct cpuset *cs = css_cs(css);
+	int retval = 0;
+
+	cpuset_full_lock();
+	if (!is_cpuset_online(cs)) {
+		retval = -ENODEV;
+		goto out_unlock;
+	}
+
+	retval = cpuset_update_flag(CS_SPREAD_PAGE, cs, val);
+
+out_unlock:
+	cpuset_full_unlock();
+	return retval;
+}
+
+static u64 cpuset_read_spread_page(struct cgroup_subsys_state *css, struct cftype *cft)
+{
+	struct cpuset *cs = css_cs(css);
+
+	return is_spread_page(cs);
+}
+
 /*
  * This is currently a minimal set for the default hierarchy. It can be
  * expanded later on by migrating more features and control files from v1.
@@ -3535,6 +3561,13 @@ static struct cftype dfl_files[] = {
 		.flags = CFTYPE_ONLY_ON_ROOT,
 	},
 
+	{
+		.name = "mems.spread_page",
+		.read_u64 = cpuset_read_spread_page,
+		.write_u64 = cpuset_update_spread_page,
+		.private = FILE_SPREAD_PAGE,
+	},
+
 	{ }	/* terminate */
 };
 
-- 
2.34.1

Re: [PATCH -next RFC 0/2] cpuset: Add cpuset.mems.spread_page to cgroup v2

[Michal Koutný]

[-- Attachment #1: Type: text/plain, Size: 804 bytes --]

Hello Xinchen.

On Tue, Sep 30, 2025 at 09:35:50AM +0000, Cai Xinchen <caixinchen1@huawei.com> wrote:
> I discovered that the DataNode process had requested a large amount
> of page cache. most of the page cache was concentrated in one NUMA node,
> ultimately leading to the exhaustion of memory in that NUMA node.
[...]
> This issue can be resolved by migrating the DataNode into
> a cpuset, dropping the cache, and setting cpuset.memory_spread_page to
> allow it to evenly request memory.

Would it work in your case instead to apply memory.max or apply
MPOL_INTERLEAVE to DataNode process?

In anyway, please see commit 012c419f8d248 ("cgroup/cpuset-v1: Add
deprecation messages to memory_spread_page and memory_spread_slab")
since your patchset would need to touch that place(s) too.

Thanks,
Michal

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 265 bytes --]

Re: [PATCH -next RFC 0/2] cpuset: Add cpuset.mems.spread_page to cgroup v2

[Cai Xinchen]

I tried using memory.max, but clearing the page cache caused performance 
degradation. The MPOL_INTERLEAVE setting requires restarting the 
DataNode service. In this scenario, the issue can be resolved by 
restarting the service, but I would prefer not to restart the DataNode 
service if possible, as it would cause a period of service interruption.

On 9/30/2025 8:05 PM, Michal Koutný wrote:
> Hello Xinchen.
>
> On Tue, Sep 30, 2025 at 09:35:50AM +0000, Cai Xinchen <caixinchen1@huawei.com> wrote:
>> I discovered that the DataNode process had requested a large amount
>> of page cache. most of the page cache was concentrated in one NUMA node,
>> ultimately leading to the exhaustion of memory in that NUMA node.
> [...]
>> This issue can be resolved by migrating the DataNode into
>> a cpuset, dropping the cache, and setting cpuset.memory_spread_page to
>> allow it to evenly request memory.
> Would it work in your case instead to apply memory.max or apply
> MPOL_INTERLEAVE to DataNode process?
>
> In anyway, please see commit 012c419f8d248 ("cgroup/cpuset-v1: Add
> deprecation messages to memory_spread_page and memory_spread_slab")
> since your patchset would need to touch that place(s) too.
>
> Thanks,
> Michal

Re: [PATCH -next RFC 0/2] cpuset: Add cpuset.mems.spread_page to cgroup v2

[Michal Koutný]

[-- Attachment #1: Type: text/plain, Size: 728 bytes --]

On Mon, Oct 20, 2025 at 02:20:30PM +0800, Cai Xinchen <caixinchen1@huawei.com> wrote:
> The MPOL_INTERLEAVE setting requires restarting the DataNode service.
> In this scenario, the issue can be resolved by restarting the service,

\o/

> but I would prefer not to restart the DataNode service if possible,
> as it would cause a period of service interruption.

AFAICS, the implementation of cpuset's spread page works only for new
allocations (filemap_alloc_folio_noprof/cpuset_do_page_mem_spread) and
there's no migraion in cpuset1_update_task_spread_flags(). So this
simple v1-like spreading would still require restart of the service.

(This challenge of dynamism also a reason why it's not ready for v2,
IMO.)

HTH,
Michal

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 265 bytes --]

Re: [PATCH -next RFC 0/2] cpuset: Add cpuset.mems.spread_page to cgroup v2

[Cai Xinchen]

Therefore, when setting memory_spread_page, I also need to perform a 
drop cache operation.

This way, there is no need to restart the service.


Some previous ideas, such as using procfs or set_mempolicy to change the 
mempolicy of

non-current processes, could also achieve the goal after dropping the 
cache. [1]


Of course, changing the current memory distribution does not necessarily 
require dropping

the cache, but it will inevitably lead to a series of memory migrations.


[1] 
https://lore.kernel.org/all/20231122211200.31620-1-gregory.price@memverge.com/

https://lore.kernel.org/all/ZWS19JFHm_LFSsFd@tiehlicka/

On 11/3/2025 9:39 PM, Michal Koutný wrote:
> On Mon, Oct 20, 2025 at 02:20:30PM +0800, Cai Xinchen <caixinchen1@huawei.com> wrote:
>> The MPOL_INTERLEAVE setting requires restarting the DataNode service.
>> In this scenario, the issue can be resolved by restarting the service,
> \o/
>
>> but I would prefer not to restart the DataNode service if possible,
>> as it would cause a period of service interruption.
> AFAICS, the implementation of cpuset's spread page works only for new
> allocations (filemap_alloc_folio_noprof/cpuset_do_page_mem_spread) and
> there's no migraion in cpuset1_update_task_spread_flags(). So this
> simple v1-like spreading would still require restart of the service.
>
> (This challenge of dynamism also a reason why it's not ready for v2,
> IMO.)
>
> HTH,
> Michal

Re: [PATCH -next RFC 0/2] cpuset: Add cpuset.mems.spread_page to cgroup v2

[Waiman Long]

On 9/30/25 5:35 AM, Cai Xinchen wrote:
> I encountered a scenario where a machine with 1.5TB of memory,
> while testing the Spark TPCDS 3TB dataset, experienced a significant
> concentration of page cache usage on one of the NUMA nodes.
> I discovered that the DataNode process had requested a large amount
> of page cache. most of the page cache was concentrated in one NUMA node,
> ultimately leading to the exhaustion of memory in that NUMA node.
> At this point, all other processes in that NUMA node have to alloc
> memory across NUMA nodes, or even across sockets. This eventually
> caused a degradation in the end-to-end performance of the Spark test.
>
> I do not want to restart the Spark DataNode service during business
> operations. This issue can be resolved by migrating the DataNode into
> a cpuset, dropping the cache, and setting cpuset.memory_spread_page to
> allow it to evenly request memory. The core business threads could still
> allocate local numa memory. After using cpuset.memory_spread_page, the
> performance in the tpcds-99 test is improved by 2%.
>
> The key point is that the even distribution of page cache within the
> DataNode process (rather than the current NUMA distribution) does not
> significantly affect end-to-end performance. However, the allocation
> of core business processes, such as Executors, to the same NUMA node
> does have a noticeable impact on end-to-end performance.
>
> However, I found that cgroup v2 does not provide this interface. I
> believe this interface still holds value in addressing issues caused
> by uneven distribution of page cache allocation among process groups.
>
> Thus I add cpuset.mems.spread_page to cpuset v2 interface.
>
> Cai Xinchen (2):
>    cpuset: Move cpuset1_update_spread_flag to cpuset
>    cpuset: Add spread_page interface to cpuset v2
>
>   kernel/cgroup/cpuset-internal.h |  6 ++--
>   kernel/cgroup/cpuset-v1.c       | 25 +----------------
>   kernel/cgroup/cpuset.c          | 49 ++++++++++++++++++++++++++++++++-
>   3 files changed, 51 insertions(+), 29 deletions(-)
>
The spread_page flag is only used in filemap_alloc_folio_noprof() of 
mm/filemap.c. By setting it, the code will attempt to spread the 
folio/page allocation across different nodes. As noted by Michal,  it is 
more or less equivalent to setting a MPOL_INTERLEAVE memory policy with 
set_mempolicy(2) with the node mask of cpuset.mems. Using 
set_mempolicy(2) has a finer task granularity instead of all the tasks 
within a cpuset. Of course, this requires making changes to the 
application instead of making change to external cgroup control file.

cpusets.mems.spread_page is a legacy interface, we need good 
justification if we want to enable it in cgroup v2.

Cheers,
Longman