[PATCH v2 1/3] sched/core: Clear the root_domain cpumasks in init_rootdomain()

[PATCH v2 1/3] sched/core: Clear the root_domain cpumasks in init_rootdomain()

[Xunlei Pang]

root_domain::rto_mask allocated through alloc_cpumask_var()
contains garbage data with CONFIG_CPUMASK_OFFSTACK set, this
may cause problems. For instance, When doing pull_rt_task(),
it may do useless iterations if rto_mask retains some extra
garbage bits. Worse still, this can violate the isolated domain
rule for clustered scheduling using cpuset, because the tasks
(with all the cpus allowed) which belongs to one root domain
can be pulled away into another root domain.

The patch cleans the garbage by passing alloc_cpumask_var()
with an extra __GFP_ZERO for root_domain::rto_mask allocation,
thereby addressing the issues.

Do the same thing for root_domain's other cpumask memembers:
dlo_mask, span, and online.

Signed-off-by: Xunlei Pang <xlpang@redhat.com>
---
v1->v2:
Use alloc_cpumask_var() with __GFP_ZERO instead of zalloc_cpumask_var()
to avoid duplicate clean for systems without CONFIG_CPUMASK_OFFSTACK set.

 kernel/sched/core.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 5b420d2..c11e11e 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -5858,13 +5858,13 @@ static int init_rootdomain(struct root_domain *rd)
 {
 	memset(rd, 0, sizeof(*rd));
 
-	if (!alloc_cpumask_var(&rd->span, GFP_KERNEL))
+	if (!alloc_cpumask_var(&rd->span, GFP_KERNEL | __GFP_ZERO))
 		goto out;
-	if (!alloc_cpumask_var(&rd->online, GFP_KERNEL))
+	if (!alloc_cpumask_var(&rd->online, GFP_KERNEL | __GFP_ZERO))
 		goto free_span;
-	if (!alloc_cpumask_var(&rd->dlo_mask, GFP_KERNEL))
+	if (!alloc_cpumask_var(&rd->dlo_mask, GFP_KERNEL | __GFP_ZERO))
 		goto free_online;
-	if (!alloc_cpumask_var(&rd->rto_mask, GFP_KERNEL))
+	if (!alloc_cpumask_var(&rd->rto_mask, GFP_KERNEL | __GFP_ZERO))
 		goto free_dlo_mask;
 
 	init_dl_bw(&rd->dl_bw);
-- 
2.5.0

[PATCH v2 2/3] sched/cpupri: Cleanup of duplicate memory initialization

[Xunlei Pang]

There is already a memset clear operation for '*cp', so we can use
alloc_cpumask_var() with __GFP_ZERO instead of zalloc_cpumask_var()
to avoid duplicate clear for systems without CONFIG_CPUMASK_OFFSTACK
set.

Also omit "atomic_set(&vec->count, 0);".

Signed-off-by: Xunlei Pang <xlpang@redhat.com>
---
 kernel/sched/cpupri.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/kernel/sched/cpupri.c b/kernel/sched/cpupri.c
index 981fcd7..b8f73a7 100644
--- a/kernel/sched/cpupri.c
+++ b/kernel/sched/cpupri.c
@@ -214,8 +214,7 @@ int cpupri_init(struct cpupri *cp)
 	for (i = 0; i < CPUPRI_NR_PRIORITIES; i++) {
 		struct cpupri_vec *vec = &cp->pri_to_cpu[i];
 
-		atomic_set(&vec->count, 0);
-		if (!zalloc_cpumask_var(&vec->mask, GFP_KERNEL))
+		if (!alloc_cpumask_var(&vec->mask, GFP_KERNEL | __GFP_ZERO))
 			goto cleanup;
 	}
 
-- 
2.5.0

[PATCH v2 3/3] sched/cpudeadline: Cleanup of duplicate memory initialization

[Xunlei Pang]

There is already a memset clear operation for '*cp', so we can use
alloc_cpumask_var() with __GFP_ZERO instead of zalloc_cpumask_var()
to avoid duplicate clear for systems without CONFIG_CPUMASK_OFFSTACK
set.

Also omit "cp->size = 0;".

Signed-off-by: Xunlei Pang <xlpang@redhat.com>
---
 kernel/sched/cpudeadline.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/kernel/sched/cpudeadline.c b/kernel/sched/cpudeadline.c
index 5a75b08..ca780ec 100644
--- a/kernel/sched/cpudeadline.c
+++ b/kernel/sched/cpudeadline.c
@@ -211,7 +211,6 @@ int cpudl_init(struct cpudl *cp)
 
 	memset(cp, 0, sizeof(*cp));
 	raw_spin_lock_init(&cp->lock);
-	cp->size = 0;
 
 	cp->elements = kcalloc(nr_cpu_ids,
 			       sizeof(struct cpudl_item),
@@ -219,7 +218,7 @@ int cpudl_init(struct cpudl *cp)
 	if (!cp->elements)
 		return -ENOMEM;
 
-	if (!zalloc_cpumask_var(&cp->free_cpus, GFP_KERNEL)) {
+	if (!alloc_cpumask_var(&cp->free_cpus, GFP_KERNEL | __GFP_ZERO)) {
 		kfree(cp->elements);
 		return -ENOMEM;
 	}
-- 
2.5.0

Re: [PATCH v2 1/3] sched/core: Clear the root_domain cpumasks in init_rootdomain()

[Steven Rostedt]

On Thu,  3 Dec 2015 12:44:59 +0800
Xunlei Pang <xlpang@redhat.com> wrote:

> root_domain::rto_mask allocated through alloc_cpumask_var()
> contains garbage data with CONFIG_CPUMASK_OFFSTACK set, this
> may cause problems. For instance, When doing pull_rt_task(),
> it may do useless iterations if rto_mask retains some extra
> garbage bits. Worse still, this can violate the isolated domain
> rule for clustered scheduling using cpuset, because the tasks
> (with all the cpus allowed) which belongs to one root domain
> can be pulled away into another root domain.

I really hate the fact that alloc_cpumask_var() behaves differently
depending on weather or not CONFIG_CPUMASK_OFFSTACK is defined or not.
I wonder if the better solution is to pass the __GFP_ZERO into
allocating the cpu mask when CPUMASK_OFFSTACK is defined. This will
prevent other areas from having this same bug. Also, I doubt any
allocations of cpumasks is in performance critical sections.

Or at least see if the address passed in is already null, which tells
us that something already cleared it. Like in this case the
memset(rd, 0, sizeof(*rd)).

Have something like:

bool alloc_cpumask_var_node(cpumask_var_t *mask, gfp_t flags, int node)
{
	if ((long)*mask == 0)
		flags |= __GFP_ZERO;

Maybe I'll post a patch to do this.

> 
> The patch cleans the garbage by passing alloc_cpumask_var()
> with an extra __GFP_ZERO for root_domain::rto_mask allocation,
> thereby addressing the issues.
> 
> Do the same thing for root_domain's other cpumask memembers:
> dlo_mask, span, and online.

Acked-by: Steven Rostedt <rostedt@goodmis.org>

-- Steve

> 
> Signed-off-by: Xunlei Pang <xlpang@redhat.com>
> ---