[PATCH] sched: fix warning in sched_setaffinity

[PATCH] sched: fix warning in sched_setaffinity

[Josh Don]

Commit 8f9ea86fdf99b added some logic to sched_setaffinity that included
a WARN when a per-task affinity assignment races with a cpuset update.

Specifically, we can have a race where a cpuset update results in the
task affinity no longer being a subset of the cpuset. That's fine; we
have a fallback to instead use the cpuset mask. However, we have a WARN
set up that will trigger if the cpuset mask has no overlap at all with
the requested task affinity. This shouldn't be a warning condition; its
trivial to create this condition.

Reproduced the warning by the following setup:

- $PID inside a cpuset cgroup
- another thread repeatedly switching the cpuset cpus from 1-2 to just 1
- another thread repeatedly setting the $PID affinity (via taskset) to 2

Fixes: 8f9ea86fdf99b ("sched: Always preserve the user requested cpumask")
Signed-off-by: Josh Don <joshdon@google.com>
Acked-by: Waiman Long <longman@redhat.com>
Tested-by: Madadi Vineeth Reddy <vineethr@linux.ibm.com>
Acked-and-tested-by: Vincent Guittot <vincent.guittot@linaro.org>
---
 kernel/sched/syscalls.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c
index 4fae3cf25a3a..3a88f7c0cb69 100644
--- a/kernel/sched/syscalls.c
+++ b/kernel/sched/syscalls.c
@@ -1321,7 +1321,7 @@ int __sched_setaffinity(struct task_struct *p, struct affinity_context *ctx)
 			bool empty = !cpumask_and(new_mask, new_mask,
 						  ctx->user_mask);
 
-			if (WARN_ON_ONCE(empty))
+			if (empty)
 				cpumask_copy(new_mask, cpus_allowed);
 		}
 		__set_cpus_allowed_ptr(p, ctx);
-- 
2.46.0.469.g59c65b2a67-goog

Re: [PATCH] sched: fix warning in sched_setaffinity

[Josh Don]

On Mon, Nov 11, 2024 at 10:27 AM Josh Don <joshdon@google.com> wrote:
>
> Commit 8f9ea86fdf99b added some logic to sched_setaffinity that included
> a WARN when a per-task affinity assignment races with a cpuset update.
>
> Specifically, we can have a race where a cpuset update results in the
> task affinity no longer being a subset of the cpuset. That's fine; we
> have a fallback to instead use the cpuset mask. However, we have a WARN
> set up that will trigger if the cpuset mask has no overlap at all with
> the requested task affinity. This shouldn't be a warning condition; its
> trivial to create this condition.
>
> Reproduced the warning by the following setup:
>
> - $PID inside a cpuset cgroup
> - another thread repeatedly switching the cpuset cpus from 1-2 to just 1
> - another thread repeatedly setting the $PID affinity (via taskset) to 2
>
> Fixes: 8f9ea86fdf99b ("sched: Always preserve the user requested cpumask")
> Signed-off-by: Josh Don <joshdon@google.com>
> Acked-by: Waiman Long <longman@redhat.com>
> Tested-by: Madadi Vineeth Reddy <vineethr@linux.ibm.com>
> Acked-and-tested-by: Vincent Guittot <vincent.guittot@linaro.org>
> ---
>  kernel/sched/syscalls.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c
> index 4fae3cf25a3a..3a88f7c0cb69 100644
> --- a/kernel/sched/syscalls.c
> +++ b/kernel/sched/syscalls.c
> @@ -1321,7 +1321,7 @@ int __sched_setaffinity(struct task_struct *p, struct affinity_context *ctx)
>                         bool empty = !cpumask_and(new_mask, new_mask,
>                                                   ctx->user_mask);
>
> -                       if (WARN_ON_ONCE(empty))
> +                       if (empty)
>                                 cpumask_copy(new_mask, cpus_allowed);
>                 }
>                 __set_cpus_allowed_ptr(p, ctx);
> --
> 2.46.0.469.g59c65b2a67-goog
>

Hey,

I wanted to bump this one last time. It's a pretty simple change that
already has ACK's from 3 other folks, but has seemed to fall through
the cracks [1].

Best,
Josh

[1] Timeline:
Original patch sent Aug 29 and quickly got ACK'd:
https://lkml.org/lkml/2024/8/29/1819
Pinged on Sep 30: https://lkml.org/lkml/2024/9/30/1528
Pinged on Oct 25: https://lkml.org/lkml/2024/10/25/1708
Resent as a new message on Nov 11: https://lkml.org/lkml/2024/11/11/1125

Re: [PATCH] sched: fix warning in sched_setaffinity

[Peter Zijlstra]

On Thu, Nov 21, 2024 at 04:09:16PM -0800, Josh Don wrote:
> On Mon, Nov 11, 2024 at 10:27 AM Josh Don <joshdon@google.com> wrote:
> >
> > Commit 8f9ea86fdf99b added some logic to sched_setaffinity that included
> > a WARN when a per-task affinity assignment races with a cpuset update.
> >
> > Specifically, we can have a race where a cpuset update results in the
> > task affinity no longer being a subset of the cpuset. That's fine; we
> > have a fallback to instead use the cpuset mask. However, we have a WARN
> > set up that will trigger if the cpuset mask has no overlap at all with
> > the requested task affinity. This shouldn't be a warning condition; its
> > trivial to create this condition.
> >
> > Reproduced the warning by the following setup:
> >
> > - $PID inside a cpuset cgroup
> > - another thread repeatedly switching the cpuset cpus from 1-2 to just 1
> > - another thread repeatedly setting the $PID affinity (via taskset) to 2
> >
> > Fixes: 8f9ea86fdf99b ("sched: Always preserve the user requested cpumask")
> > Signed-off-by: Josh Don <joshdon@google.com>
> > Acked-by: Waiman Long <longman@redhat.com>
> > Tested-by: Madadi Vineeth Reddy <vineethr@linux.ibm.com>
> > Acked-and-tested-by: Vincent Guittot <vincent.guittot@linaro.org>
> > ---
> >  kernel/sched/syscalls.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c
> > index 4fae3cf25a3a..3a88f7c0cb69 100644
> > --- a/kernel/sched/syscalls.c
> > +++ b/kernel/sched/syscalls.c
> > @@ -1321,7 +1321,7 @@ int __sched_setaffinity(struct task_struct *p, struct affinity_context *ctx)
> >                         bool empty = !cpumask_and(new_mask, new_mask,
> >                                                   ctx->user_mask);
> >
> > -                       if (WARN_ON_ONCE(empty))
> > +                       if (empty)
> >                                 cpumask_copy(new_mask, cpus_allowed);
> >                 }
> >                 __set_cpus_allowed_ptr(p, ctx);
> > --
> > 2.46.0.469.g59c65b2a67-goog
> >
> 
> Hey,
> 
> I wanted to bump this one last time. It's a pretty simple change that
> already has ACK's from 3 other folks, but has seemed to fall through
> the cracks [1].

Got it, I'll stick it in tip/sched/urgent post -rc1 or so.

[tip: sched/core] sched: fix warning in sched_setaffinity

[tip-bot2 for Josh Don]

The following commit has been merged into the sched/core branch of tip:

Commit-ID:     70ee7947a29029736a1a06c73a48ff37674a851b
Gitweb:        https://git.kernel.org/tip/70ee7947a29029736a1a06c73a48ff37674a851b
Author:        Josh Don <joshdon@google.com>
AuthorDate:    Mon, 11 Nov 2024 10:27:38 -08:00
Committer:     Peter Zijlstra <peterz@infradead.org>
CommitterDate: Mon, 02 Dec 2024 12:01:27 +01:00

sched: fix warning in sched_setaffinity

Commit 8f9ea86fdf99b added some logic to sched_setaffinity that included
a WARN when a per-task affinity assignment races with a cpuset update.

Specifically, we can have a race where a cpuset update results in the
task affinity no longer being a subset of the cpuset. That's fine; we
have a fallback to instead use the cpuset mask. However, we have a WARN
set up that will trigger if the cpuset mask has no overlap at all with
the requested task affinity. This shouldn't be a warning condition; its
trivial to create this condition.

Reproduced the warning by the following setup:

- $PID inside a cpuset cgroup
- another thread repeatedly switching the cpuset cpus from 1-2 to just 1
- another thread repeatedly setting the $PID affinity (via taskset) to 2

Fixes: 8f9ea86fdf99b ("sched: Always preserve the user requested cpumask")
Signed-off-by: Josh Don <joshdon@google.com>
Acked-and-tested-by: Vincent Guittot <vincent.guittot@linaro.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Waiman Long <longman@redhat.com>
Tested-by: Madadi Vineeth Reddy <vineethr@linux.ibm.com>
Link: https://lkml.kernel.org/r/20241111182738.1832953-1-joshdon@google.com
---
 kernel/sched/syscalls.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c
index 0d71fcb..ff0e5ab 100644
--- a/kernel/sched/syscalls.c
+++ b/kernel/sched/syscalls.c
@@ -1200,7 +1200,7 @@ int __sched_setaffinity(struct task_struct *p, struct affinity_context *ctx)
 			bool empty = !cpumask_and(new_mask, new_mask,
 						  ctx->user_mask);
 
-			if (WARN_ON_ONCE(empty))
+			if (empty)
 				cpumask_copy(new_mask, cpus_allowed);
 		}
 		__set_cpus_allowed_ptr(p, ctx);

[PATCH] sched: fix warning in sched_setaffinity

[Josh Don]

Commit 8f9ea86fdf99b added some logic to sched_setaffinity that included
a WARN when a per-task affinity assignment races with a cpuset update.

Specifically, we can have a race where a cpuset update results in the
task affinity no longer being a subset of the cpuset. That's fine; we
have a fallback to instead use the cpuset mask. However, we have a WARN
set up that will trigger if the cpuset mask has no overlap at all with
the requested task affinity. This shouldn't be a warning condition; its
trivial to create this condition.

Reproduced the warning by the following setup:

- $PID inside a cpuset cgroup
- another thread repeatedly switching the cpuset cpus from 1-2 to just 1
- another thread repeatedly setting the $PID affinity (via taskset) to 2

Fixes: 8f9ea86fdf99b ("sched: Always preserve the user requested cpumask")
Signed-off-by: Josh Don <joshdon@google.com>
---
 kernel/sched/syscalls.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c
index 4fae3cf25a3a..3a88f7c0cb69 100644
--- a/kernel/sched/syscalls.c
+++ b/kernel/sched/syscalls.c
@@ -1321,7 +1321,7 @@ int __sched_setaffinity(struct task_struct *p, struct affinity_context *ctx)
 			bool empty = !cpumask_and(new_mask, new_mask,
 						  ctx->user_mask);
 
-			if (WARN_ON_ONCE(empty))
+			if (empty)
 				cpumask_copy(new_mask, cpus_allowed);
 		}
 		__set_cpus_allowed_ptr(p, ctx);
-- 
2.46.0.469.g59c65b2a67-goog

Re: [PATCH] sched: fix warning in sched_setaffinity

[Waiman Long]

On 8/29/24 18:04, Josh Don wrote:
> Commit 8f9ea86fdf99b added some logic to sched_setaffinity that included
> a WARN when a per-task affinity assignment races with a cpuset update.
>
> Specifically, we can have a race where a cpuset update results in the
> task affinity no longer being a subset of the cpuset. That's fine; we
> have a fallback to instead use the cpuset mask. However, we have a WARN
> set up that will trigger if the cpuset mask has no overlap at all with
> the requested task affinity. This shouldn't be a warning condition; its
> trivial to create this condition.
>
> Reproduced the warning by the following setup:
>
> - $PID inside a cpuset cgroup
> - another thread repeatedly switching the cpuset cpus from 1-2 to just 1
> - another thread repeatedly setting the $PID affinity (via taskset) to 2
>
> Fixes: 8f9ea86fdf99b ("sched: Always preserve the user requested cpumask")
> Signed-off-by: Josh Don <joshdon@google.com>
> ---
>   kernel/sched/syscalls.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c
> index 4fae3cf25a3a..3a88f7c0cb69 100644
> --- a/kernel/sched/syscalls.c
> +++ b/kernel/sched/syscalls.c
> @@ -1321,7 +1321,7 @@ int __sched_setaffinity(struct task_struct *p, struct affinity_context *ctx)
>   			bool empty = !cpumask_and(new_mask, new_mask,
>   						  ctx->user_mask);
>   
> -			if (WARN_ON_ONCE(empty))
> +			if (empty)
>   				cpumask_copy(new_mask, cpus_allowed);
>   		}
>   		__set_cpus_allowed_ptr(p, ctx);

Taking out the WARN_ON_ONCE() should be fine.

Acked-by: Waiman Long <longman@redhat.com>

Re: [PATCH] sched: fix warning in sched_setaffinity

[Madadi Vineeth Reddy]

Hi Josh Don,

On 30/08/24 03:34, Josh Don wrote:
> Commit 8f9ea86fdf99b added some logic to sched_setaffinity that included
> a WARN when a per-task affinity assignment races with a cpuset update.
> 
> Specifically, we can have a race where a cpuset update results in the
> task affinity no longer being a subset of the cpuset. That's fine; we
> have a fallback to instead use the cpuset mask. However, we have a WARN
> set up that will trigger if the cpuset mask has no overlap at all with
> the requested task affinity. This shouldn't be a warning condition; its
> trivial to create this condition.
> 
> Reproduced the warning by the following setup:
> 
> - $PID inside a cpuset cgroup
> - another thread repeatedly switching the cpuset cpus from 1-2 to just 1
> - another thread repeatedly setting the $PID affinity (via taskset) to 2
> 

I was testing the patch using the following two scripts run concurrently:

Script 1:
while true; do
    echo 1 > /sys/fs/cgroup/test_group/cpuset.cpus;
    echo 1-2 > /sys/fs/cgroup/test_group/cpuset.cpus;
done

Script 2:
while true; do
    sudo taskset -p 0x2 $$;
done

However, I am unable to trigger the warning in dmesg on the unpatched kernel.
I was expecting to see the warning as described, but it doesn't seem to appear.

Additionally, I also tried the following script to increase the chances of
triggering the race condition:

while true; do
    echo 1 > /sys/fs/cgroup/test_group/cpuset.cpus;
    sudo taskset -p 0x2 $$;
    sleep 0.1;
    echo 1-2 > /sys/fs/cgroup/test_group/cpuset.cpus;
done

Despite this, the warning still does not appear in dmesg.

Am I missing something in my testing approach, or is there a different setup
required to reproduce the issue?

Thanks and Regards
Madadi Vineeth Reddy

> Fixes: 8f9ea86fdf99b ("sched: Always preserve the user requested cpumask")
> Signed-off-by: Josh Don <joshdon@google.com>

Re: [PATCH] sched: fix warning in sched_setaffinity

[Josh Don]

Hi Madadi,

On Sun, Sep 1, 2024 at 7:25 AM Madadi Vineeth Reddy
<vineethr@linux.ibm.com> wrote:
>
> Hi Josh Don,
>
> On 30/08/24 03:34, Josh Don wrote:
> > Commit 8f9ea86fdf99b added some logic to sched_setaffinity that included
> > a WARN when a per-task affinity assignment races with a cpuset update.
> >
> > Specifically, we can have a race where a cpuset update results in the
> > task affinity no longer being a subset of the cpuset. That's fine; we
> > have a fallback to instead use the cpuset mask. However, we have a WARN
> > set up that will trigger if the cpuset mask has no overlap at all with
> > the requested task affinity. This shouldn't be a warning condition; its
> > trivial to create this condition.
> >
> > Reproduced the warning by the following setup:
> >
> > - $PID inside a cpuset cgroup
> > - another thread repeatedly switching the cpuset cpus from 1-2 to just 1
> > - another thread repeatedly setting the $PID affinity (via taskset) to 2
> >
>
> I was testing the patch using the following two scripts run concurrently:
>
> Script 1:
> while true; do
>     echo 1 > /sys/fs/cgroup/test_group/cpuset.cpus;
>     echo 1-2 > /sys/fs/cgroup/test_group/cpuset.cpus;
> done
>
> Script 2:
> while true; do
>     sudo taskset -p 0x2 $$;
> done
>
> However, I am unable to trigger the warning in dmesg on the unpatched kernel.
> I was expecting to see the warning as described, but it doesn't seem to appear.
>
> Additionally, I also tried the following script to increase the chances of
> triggering the race condition:
>
> while true; do
>     echo 1 > /sys/fs/cgroup/test_group/cpuset.cpus;
>     sudo taskset -p 0x2 $$;
>     sleep 0.1;
>     echo 1-2 > /sys/fs/cgroup/test_group/cpuset.cpus;
> done
>
> Despite this, the warning still does not appear in dmesg.
>
> Am I missing something in my testing approach, or is there a different setup
> required to reproduce the issue?

taskset -p 0x2 $$ will affine to cpu 1 :)

I'd recommend using the '-c' arg to specify the mask as a cpulist, as
it is easier to validate.

taskset -c -p 2 $$

>
> Thanks and Regards
> Madadi Vineeth Reddy
>
> > Fixes: 8f9ea86fdf99b ("sched: Always preserve the user requested cpumask")
> > Signed-off-by: Josh Don <joshdon@google.com>
>

Best,
Josh

Re: [PATCH] sched: fix warning in sched_setaffinity

[Madadi Vineeth Reddy]

Hi Josh Don,

On 04/09/24 03:03, Josh Don wrote:
> Hi Madadi,
> 
> On Sun, Sep 1, 2024 at 7:25 AM Madadi Vineeth Reddy
> <vineethr@linux.ibm.com> wrote:
>>
>> Hi Josh Don,
>>
>> On 30/08/24 03:34, Josh Don wrote:
>>> Commit 8f9ea86fdf99b added some logic to sched_setaffinity that included
>>> a WARN when a per-task affinity assignment races with a cpuset update.
>>>
>>> Specifically, we can have a race where a cpuset update results in the
>>> task affinity no longer being a subset of the cpuset. That's fine; we
>>> have a fallback to instead use the cpuset mask. However, we have a WARN
>>> set up that will trigger if the cpuset mask has no overlap at all with
>>> the requested task affinity. This shouldn't be a warning condition; its
>>> trivial to create this condition.
>>>
>>> Reproduced the warning by the following setup:
>>>
>>> - $PID inside a cpuset cgroup
>>> - another thread repeatedly switching the cpuset cpus from 1-2 to just 1
>>> - another thread repeatedly setting the $PID affinity (via taskset) to 2
>>>
>>
>> I was testing the patch using the following two scripts run concurrently:
>>
>> Script 1:
>> while true; do
>>     echo 1 > /sys/fs/cgroup/test_group/cpuset.cpus;
>>     echo 1-2 > /sys/fs/cgroup/test_group/cpuset.cpus;
>> done
>>
>> Script 2:
>> while true; do
>>     sudo taskset -p 0x2 $$;
>> done
>>
>> However, I am unable to trigger the warning in dmesg on the unpatched kernel.
>> I was expecting to see the warning as described, but it doesn't seem to appear.
>>
>> Additionally, I also tried the following script to increase the chances of
>> triggering the race condition:
>>
>> while true; do
>>     echo 1 > /sys/fs/cgroup/test_group/cpuset.cpus;
>>     sudo taskset -p 0x2 $$;
>>     sleep 0.1;
>>     echo 1-2 > /sys/fs/cgroup/test_group/cpuset.cpus;
>> done
>>
>> Despite this, the warning still does not appear in dmesg.
>>
>> Am I missing something in my testing approach, or is there a different setup
>> required to reproduce the issue?
> 
> taskset -p 0x2 $$ will affine to cpu 1 :)
> 
> I'd recommend using the '-c' arg to specify the mask as a cpulist, as
> it is easier to validate.
> 
> taskset -c -p 2 $$
> 

Thanks for the clarification.

Tested-by: Madadi Vineeth Reddy <vineethr@linux.ibm.com>

Thanks and Regards
Madadi Vineeth Reddy

>>
>> Thanks and Regards
>> Madadi Vineeth Reddy
>>
>>> Fixes: 8f9ea86fdf99b ("sched: Always preserve the user requested cpumask")
>>> Signed-off-by: Josh Don <joshdon@google.com>
>>
> 
> Best,
> Josh

Re: [PATCH] sched: fix warning in sched_setaffinity

[Josh Don]

On Thu, Aug 29, 2024 at 3:04 PM Josh Don <joshdon@google.com> wrote:
>
> Commit 8f9ea86fdf99b added some logic to sched_setaffinity that included
> a WARN when a per-task affinity assignment races with a cpuset update.
>
> Specifically, we can have a race where a cpuset update results in the
> task affinity no longer being a subset of the cpuset. That's fine; we
> have a fallback to instead use the cpuset mask. However, we have a WARN
> set up that will trigger if the cpuset mask has no overlap at all with
> the requested task affinity. This shouldn't be a warning condition; its
> trivial to create this condition.
>
> Reproduced the warning by the following setup:
>
> - $PID inside a cpuset cgroup
> - another thread repeatedly switching the cpuset cpus from 1-2 to just 1
> - another thread repeatedly setting the $PID affinity (via taskset) to 2
>
> Fixes: 8f9ea86fdf99b ("sched: Always preserve the user requested cpumask")
> Signed-off-by: Josh Don <joshdon@google.com>

Gentle ping to bump this in case it got lost.

I've also collected the following:

Acked-by: Waiman Long <longman@redhat.com>
Tested-by: Madadi Vineeth Reddy <vineethr@linux.ibm.com>

Best,
Josh

Re: [PATCH] sched: fix warning in sched_setaffinity

[Vincent Guittot]

On Mon, 30 Sept 2024 at 22:26, Josh Don <joshdon@google.com> wrote:
>
> On Thu, Aug 29, 2024 at 3:04 PM Josh Don <joshdon@google.com> wrote:
> >
> > Commit 8f9ea86fdf99b added some logic to sched_setaffinity that included
> > a WARN when a per-task affinity assignment races with a cpuset update.
> >
> > Specifically, we can have a race where a cpuset update results in the
> > task affinity no longer being a subset of the cpuset. That's fine; we
> > have a fallback to instead use the cpuset mask. However, we have a WARN
> > set up that will trigger if the cpuset mask has no overlap at all with
> > the requested task affinity. This shouldn't be a warning condition; its
> > trivial to create this condition.
> >
> > Reproduced the warning by the following setup:
> >
> > - $PID inside a cpuset cgroup
> > - another thread repeatedly switching the cpuset cpus from 1-2 to just 1
> > - another thread repeatedly setting the $PID affinity (via taskset) to 2
> >
> > Fixes: 8f9ea86fdf99b ("sched: Always preserve the user requested cpumask")
> > Signed-off-by: Josh Don <joshdon@google.com>
>
> Gentle ping to bump this in case it got lost.

I have also been able to reproduce the race and the WARN with the
steps described in the commit description

Acked-and-tested-by: Vincent Guittot <vincent.guittot@linaro.org>

>
> I've also collected the following:
>
> Acked-by: Waiman Long <longman@redhat.com>
> Tested-by: Madadi Vineeth Reddy <vineethr@linux.ibm.com>
>
> Best,
> Josh

Re: [PATCH] sched: fix warning in sched_setaffinity

[Josh Don]

> Gentle ping to bump this in case it got lost.

Nudge :)

> I've also collected the following:
>
> Acked-by: Waiman Long <longman@redhat.com>
> Tested-by: Madadi Vineeth Reddy <vineethr@linux.ibm.com>

Also adding

Acked-and-tested-by: Vincent Guittot <vincent.guittot@linaro.org>

Best,
Josh