Re: [PATCH v4 2/2] cpusets,isolcpus: add file to show isolated cpus in cpuset

[Mike Galbraith <umgwanakikbuti@gmail.com>]

On Mon, 2015-03-02 at 10:09 +0100, Peter Zijlstra wrote:
> On Thu, Feb 26, 2015 at 12:12:31PM -0500, Rik van Riel wrote:
> > Subject: cpusets,isolcpus: add file to show isolated cpus in cpuset
> > 
> > The previous patch makes it so the code skips over isolcpus when
> > building scheduler load balancing domains. This makes it hard to
> > see for a user which of the CPUs in a cpuset are participating in
> > load balancing, and which ones are isolated cpus.
> > 
> > Add a cpuset.isolcpus file with info on which cpus in a cpuset are
> > isolated CPUs.
> > 
> > This file is read-only for now. In the future we could extend things
> > so isolcpus can be changed at run time, for the root (system wide)
> > cpuset only.
> > 
> > Acked-by: David Rientjes <rientjes@google.com>
> > Cc: Peter Zijlstra <peterz@infradead.org>
> > Cc: Clark Williams <williams@redhat.com>
> > Cc: Li Zefan <lizefan@huawei.com>
> > Cc: Ingo Molnar <mingo@redhat.com>
> > Cc: Luiz Capitulino <lcapitulino@redhat.com>
> > Cc: David Rientjes <rientjes@google.com>
> > Cc: Mike Galbraith <umgwanakikbuti@gmail.com>
> > Cc: cgroups@vger.kernel.org
> > Signed-off-by: Rik van Riel <riel@redhat.com>
> 
> So let me start off by saying I hate isolcpus ;-)
> 
> Let me further state that I had hopes we could extend cpusets to
> natively provide the functionality isolcpus has, and kill isolcpus.

+1

That's where nohz_full goop belongs too.

> The 'normal' way would be to create 2 cgroups with disjoint cpus,
> disable sched_load_balance on root and one of the siblings, while moving
> everything into the other group.

That's what cset shield does, works fine.

> The 'problem' is that we cannot move everything that is affected by
> isolcpus, workqueues have grown a horrible 'new' interface outside of
> the regular task interfaces and things like kthreadd are non-movable for
> mostly good reasons.
> 
> Furthermore it appears that software like system-disease and libvirt
> hard assume they're lord and master of the cgroup hierarchy and do not
> expect things like this.
> 
> So while I mostly hate all of this it might be the best we can do :-(

Hm, I'm now all system-disease-ified now (still hate the bloody thing),
and have no problem isolating cpus via cpusets, modulo workqueues
wanting a bat upside the head.

	-Mike