From: Waiman Long <longman@redhat.com>
To: Valentin Schneider <vschneid@redhat.com>, Tejun Heo <tj@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>,
Peter Zijlstra <peterz@infradead.org>,
Juri Lelli <juri.lelli@redhat.com>,
Vincent Guittot <vincent.guittot@linaro.org>,
Dietmar Eggemann <dietmar.eggemann@arm.com>,
Steven Rostedt <rostedt@goodmis.org>,
Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
Daniel Bristot de Oliveira <bristot@redhat.com>,
Zefan Li <lizefan.x@bytedance.com>,
Johannes Weiner <hannes@cmpxchg.org>,
cgroups@vger.kernel.org, linux-kernel@vger.kernel.org,
Will Deacon <will@kernel.org>
Subject: Re: [PATCH 1/2] cgroup/cpuset: Keep current cpus list if cpus affinity was explicitly set
Date: Fri, 29 Jul 2022 14:31:00 -0400 [thread overview]
Message-ID: <c9d82d78-b47d-0f4a-f1ca-81df78e1baa6@redhat.com> (raw)
In-Reply-To: <92f68392-12d2-f64a-9bb9-1a3a15f99d02@redhat.com>
On 7/29/22 10:50, Waiman Long wrote:
> On 7/29/22 10:15, Valentin Schneider wrote:
>> On 28/07/22 11:39, Tejun Heo wrote:
>>> Hello, Waiman.
>>>
>>> On Thu, Jul 28, 2022 at 05:04:19PM -0400, Waiman Long wrote:
>>>>> So, the patch you proposed is making the code remember one special
>>>>> aspect of
>>>>> user requested configuration - whether it configured it or not,
>>>>> and trying
>>>>> to preserve that particular state as cpuset state changes. It
>>>>> addresses the
>>>>> immediate problem but it is a very partial approach. Let's say a
>>>>> task wanna
>>>>> be affined to one logical thread of each core and set its mask to
>>>>> 0x5555.
>>>>> Now, let's say cpuset got enabled and enforced 0xff and affined
>>>>> the task to
>>>>> 0xff. After a while, the cgroup got more cpus allocated and its
>>>>> cpuset now
>>>>> has 0xfff. Ideally, what should happen is the task now having the
>>>>> effective
>>>>> mask of 0x555. In practice, tho, it either would get 0xf55 or 0x55
>>>>> depending
>>>>> on which way we decide to misbehave.
>>>> OK, I see what you want to accomplish. To fully address this issue,
>>>> we will
>>>> need to have a new cpumask variable in the the task structure which
>>>> will be
>>>> allocated if sched_setaffinity() is ever called. I can rework my
>>>> patch to
>>>> use this approach.
>>> Yeah, we'd need to track what user requested separately from the
>>> currently
>>> effective cpumask. Let's make sure that the scheduler folks are on
>>> board
>>> before committing to the idea tho. Peter, Ingo, what do you guys think?
>>>
>> FWIW on a runtime overhead side of things I think it'll be OK as that
>> should be just an extra mask copy in sched_setaffinity() and a subset
>> check / cpumask_and() in set_cpus_allowed_ptr(). The policy side is a
>> bit
>> less clear (when, if ever, do we clear the user-defined mask? Will it
>> keep
>> haunting us even after moving a task to a disjoint cpuset partition?).
>
> The runtime overhead should be minimal. It is the behavioral side that
> we should be careful about. It is a change in existing behavior and we
> don't want to cause surprise to the users. Currently, a task that set
> its cpu affinity explicitly will have its affinity reset whenever
> there is any change to the cpuset it belongs to or a hotplug event
> touch any cpu in the current cpuset. The new behavior we are proposing
> here is that it will try its best to keep the cpu affinity that the
> user requested within the constraint of the current cpuset as well as
> the cpu hotplug state.
>
>
>>
>> There's also if/how that new mask should be exposed, because attaching a
>> task to a cpuset will now yield a not-necessarily-obvious affinity -
>> e.g. in the thread affinity example above, if the initial affinity
>> setting
>> was done ages ago by some system tool, IMO the user needs a way to be
>> able
>> to expect/understand the result of 0x555 rather than 0xfff.
>
> Users can use sched_getaffinity(2) to retrieve the current cpu
> affinity. It is up to users to set another one if they don't like the
> current one. I don't think we need to return what the previous
> requested cpu affinity is. They are suppose to know that or they can
> set their own if they don't like it. \
Looking at Will's series that introduced user_cpus_ptr, I think we can
overlay our proposal on top of that. So calling sched_setaffinity() will
also update user_cpus_ptr. We may still need a flag to indicate whether
user_cpus_ptr is set up because of sched_setaffinity() or due to a call
to force_compatible_cpus_allowed_ptr() from arm64 arch code. That will
make our work easier as some of the infrastructure is already there. I
am looking forward for your feedback.
Thanks,
Longman
prev parent reply other threads:[~2022-07-29 18:31 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-07-28 0:58 Waiman Long
2022-07-28 0:58 ` [PATCH 2/2] cgroup: Skip subtree root in cgroup_update_dfl_csses() Waiman Long
2022-07-28 14:44 ` Michal Koutný
2022-07-28 14:49 ` Waiman Long
2022-07-28 17:26 ` Tejun Heo
2022-07-28 17:27 ` Tejun Heo
2022-07-28 14:44 ` [PATCH 1/2] cgroup/cpuset: Keep current cpus list if cpus affinity was explicitly set Michal Koutný
2022-07-28 14:59 ` Waiman Long
2022-07-28 15:23 ` Michal Koutný
2022-07-28 15:35 ` Waiman Long
2022-07-28 16:50 ` Valentin Schneider
2022-07-28 17:42 ` Waiman Long
2022-07-28 17:23 ` Tejun Heo
2022-07-28 18:57 ` Waiman Long
2022-07-28 19:02 ` Tejun Heo
2022-07-28 19:21 ` Waiman Long
2022-07-28 20:44 ` Tejun Heo
2022-07-28 21:04 ` Waiman Long
2022-07-28 21:39 ` Tejun Heo
2022-07-29 14:15 ` Valentin Schneider
2022-07-29 14:50 ` Waiman Long
2022-07-29 18:31 ` Waiman Long [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=c9d82d78-b47d-0f4a-f1ca-81df78e1baa6@redhat.com \
--to=longman@redhat.com \
--cc=bristot@redhat.com \
--cc=bsegall@google.com \
--cc=cgroups@vger.kernel.org \
--cc=dietmar.eggemann@arm.com \
--cc=hannes@cmpxchg.org \
--cc=juri.lelli@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=lizefan.x@bytedance.com \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tj@kernel.org \
--cc=vincent.guittot@linaro.org \
--cc=vschneid@redhat.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®