[proposal] delegating cgroup manager to non-PID1

[proposal] delegating cgroup manager to non-PID1

[Luke Kenneth Casson Leighton]

(i am subscribed nomail to lkml, please cc thx)

in recent discussions about PID-1 alternatives (sysvinit, openrc,
systemd, depinit) i was alerted to the idea that PID1 is to become the
sole exlcusive process permitted to manage cgroups.  given that, just
as one specific example, depinit is only around 2,300 lines of c code,
adding extra code to manage cgroups is of some concern especially in
light of the general UNIX philosophy "do one thing and do it well".

to allow the general UNIX philosophy to be honoured, may i
respectfully propose an additional linux kernel systemcall which
permits delegation - solely and exclusively by PID1 - of the
management of cgroups to one (and only one) other process, and that
furthermore that the process must be an immediate child of PID1?

l.

Re: [proposal] delegating cgroup manager to non-PID1

[David Lang]

On Sun, 1 Mar 2015, Luke Kenneth Casson Leighton wrote:

> in recent discussions about PID-1 alternatives (sysvinit, openrc,
> systemd, depinit) i was alerted to the idea that PID1 is to become the
> sole exlcusive process permitted to manage cgroups.  given that, just
> as one specific example, depinit is only around 2,300 lines of c code,
> adding extra code to manage cgroups is of some concern especially in
> light of the general UNIX philosophy "do one thing and do it well".
>
> to allow the general UNIX philosophy to be honoured, may i
> respectfully propose an additional linux kernel systemcall which
> permits delegation - solely and exclusively by PID1 - of the
> management of cgroups to one (and only one) other process, and that
> furthermore that the process must be an immediate child of PID1?

There is less agreement on the idea that PID1 will have exclusive control over 
cgroups than some of the posts make it seem. There are many people who use 
cgroups for things that PID1 (and systemd) aren't dealing with. The issue is 
that the people working to revamp cgroups are saying that allowing other 
processed to affect cgroups brings up hard problems that they don't want to deal 
with right now, so they want to make cgroups exclusive to PID1 as a 'temporary' 
measure, and then look at solving the problems that are needed to let other 
processes manage parts or all of the cgroups config.

David Lang

Re: [proposal] delegating cgroup manager to non-PID1

[Luke Kenneth Casson Leighton]

On Mon, Mar 2, 2015 at 12:13 AM, David Lang <david@lang.hm> wrote:
> On Sun, 1 Mar 2015, Luke Kenneth Casson Leighton wrote:
>
>> in recent discussions about PID-1 alternatives (sysvinit, openrc,
>> systemd, depinit) i was alerted to the idea that PID1 is to become the
>> sole exlcusive process permitted to manage cgroups.  given that, just
>> as one specific example, depinit is only around 2,300 lines of c code,
>> adding extra code to manage cgroups is of some concern especially in
>> light of the general UNIX philosophy "do one thing and do it well".
>>
>> to allow the general UNIX philosophy to be honoured, may i
>> respectfully propose an additional linux kernel systemcall which
>> permits delegation - solely and exclusively by PID1 - of the
>> management of cgroups to one (and only one) other process, and that
>> furthermore that the process must be an immediate child of PID1?
>
>
> There is less agreement on the idea that PID1 will have exclusive control
> over cgroups than some of the posts make it seem. There are many people who
> use cgroups for things that PID1 (and systemd) aren't dealing with.

 such as lxc, yes - i use it to run 5 virtual machines for clients,
having got absolutely fed up with xen crashing because i was using a
slightly different swap/VM arrangement from usual, and every time a
xen guest hit maximum available memory it would go into 100% cpu
usage, lock up the guest entirely, and sometimes take the entire xen
host with it.  this being bad i ripped it out and replaced it with
lxc, which, despite the xen guests being on LVM partitions, worked
very well once i worked out how to specify LVM partitions in the lxc
cgroups device control file.

 so... as an end-user i would be rather upset if lxc suddenly got
really awkward to use, or complex because lxc had to be part of PID1.
or was forced to become more complex by having (god forbid) d-bus in
it as it was forced to communicate with PID1.

 however i would probably grumble if PID1 had to be responsible for
exec'ing a special lxc-cgroup-management program, might find it a bit
weird, but would otherwise tolerate it.

 where i _would_ get upset would be if say the PID1 decided "i'm
sorry, i control cgroups, i'm delegating to this executable, i can't
cope if you try to run a different one, so the fact that you're
running lxc is something i completely disregard, fuck off and die".
that would... err... be bad :)

> The
> issue is that the people working to revamp cgroups are saying that allowing
> other processed to affect cgroups brings up hard problems that they don't
> want to deal with right now, so they want to make cgroups exclusive to PID1
> as a 'temporary' measure, and then look at solving the problems that are
> needed to let other processes manage parts or all of the cgroups config.

 well then, controlled and explicit delegation would appear to be
quite a reasonable compromise.  and would also make SE/Linux advocates
happy as well.  [SE/Linux advocates _really_ don't like the idea of
one process doing multiple jobs, it attracts attention as a
high-priority target for concerted efforts to exploit possible
weaknesses on the basis that doing so is a higher payoff].

the actual issue i feel is that the plans for cgroups (such as solving
the console vt issue [1], and using cgroups to add labels to processes
and their children so that both a parent and all children may be
killed atomically) are, regardless of whether they're controlled
exclusively by PID1, mutually exclusively incompatible with whatever
other uses cgroups are put to (such as lxc).

 if you use cgroups for lxc, for example, chances are pretty high that
it'll massively interfere with, or require some careful (i.e.
completely inappropriate) negotiation, with *all* the other uses not
just PID1.

 the thought therefore occurs to me - and this is just a first
immediate thought that i haven't had much time to mull over - that it
might make sense to follow the whole "VMS" idea where within one
virtual machine you could create any number of VMS OS instances, any
one of which could create any number of VMS virtual machines, repeat
until no resources.

 a simpler expression of that concept: allow cgroups management of
devices and responsibilities to be hierarchical (just like a file
system).

 sooo... you would saaay:

"i am PID1, i control all cgroups (by default).  *but*, i delegate
responsibility for this this and this device node, and this this and
this functionality tooo.... PID1234"

 in this way it would be possible to have PID1 delegate responsibility
for lxc-style (or other) device nodes etc. etc. out to lxc.  it would
be possible to delegate the things that are needed for console
management out to a separate process, and to delegate the
responsibility for putting labels onto parents-plus-children out to
yet another.

 i'd *recommend* that the delegation be permitted to be recursive
(i.e. that there be a permission "can-further-subdelegate").

 the only other possible refinement to this would be to follow the
SE/Linux model instead of a "PID1 is god, can do anything, has
everything, and delegates parts out that fan out to progressively less
and less".  SE/LInux has the wonderful (i.e. much better) security
model of "even God has limits", i.e. no process is *ever* given more
than it actually needs.

so, for example, using the SE/Linux style security model as
inspiration, PID1 would *NOT* be given the full set of cgroup
management rights: it would instead be allocated merely the *right to
delegate* management rights to other (specific) processes. i.e. if
PID1 actually tried to manage cgroup permissions itself, those
systemcalls would *rightly fail*.

 l.

[1] depinit makes the console vt issue - recovering a system where x11
has crashed - a non-issue by using SysRq as the means to open another
console login.  whilst i "Get" the whole idea of why effectively
creating "virtual machines" - similar to how lxc operates - to manage
and separate console sessions from virtual seats, i do see it as
complete overkill.  if people *want* separate console sessions and
separate virtual seats, i feel that they should do so by deploying...
lxc!!  "reinventing" what lxc has achieved, putting it directly into
say systemd, seems to be going a bit too far.  especially as the only
real serious problem is x11 crashing and taking out the keyboard, but
depinit shows a way to deal with that.  but i digress...

Re: [proposal] delegating cgroup manager to non-PID1

[Luke Kenneth Casson Leighton]

On Mon, Mar 2, 2015 at 12:13 AM, David Lang <david@lang.hm> wrote:
> On Sun, 1 Mar 2015, Luke Kenneth Casson Leighton wrote:
>
>> in recent discussions about PID-1 alternatives (sysvinit, openrc,
>> systemd, depinit) i was alerted to the idea that PID1 is to become the
>> sole exlcusive process permitted to manage cgroups.  given that, just

> There is less agreement on the idea that PID1 will have exclusive control
> over cgroups than some of the posts make it seem.

david, small favour to ask you (for archival purposes as well as
convenience) - finding posts, or even the mailing lists themselves, on
which the discussions you mention may be found, turns out to be um
rather challenging if you happen not to have subscribed to those
lists, such that every day you've seen all the discussions and know
what to look for, and where.

 would you therefore be so kind as to publish some hints - specific
keyword phrases more general than "PID1 cgroups" which is about the
limit of what i can think might apply (which ironically comes up with
every archive linking to *this* thread.... *sigh*), or better perhaps
some specific references to specific mailing lists.

 many many thanks,

l.

Re: [proposal] delegating cgroup manager to non-PID1

[David Lang]

On Tue, 3 Mar 2015, Luke Kenneth Casson Leighton wrote:

> On Mon, Mar 2, 2015 at 12:13 AM, David Lang <david@lang.hm> wrote:
>> On Sun, 1 Mar 2015, Luke Kenneth Casson Leighton wrote:
>>
>>> in recent discussions about PID-1 alternatives (sysvinit, openrc,
>>> systemd, depinit) i was alerted to the idea that PID1 is to become the
>>> sole exlcusive process permitted to manage cgroups.  given that, just
>
>> There is less agreement on the idea that PID1 will have exclusive control
>> over cgroups than some of the posts make it seem.
>
> david, small favour to ask you (for archival purposes as well as
> convenience) - finding posts, or even the mailing lists themselves, on
> which the discussions you mention may be found, turns out to be um
> rather challenging if you happen not to have subscribed to those
> lists, such that every day you've seen all the discussions and know
> what to look for, and where.
>
> would you therefore be so kind as to publish some hints - specific
> keyword phrases more general than "PID1 cgroups" which is about the
> limit of what i can think might apply (which ironically comes up with
> every archive linking to *this* thread.... *sigh*), or better perhaps
> some specific references to specific mailing lists.

I was tryng to avoid fanning the flames, but there has been some discussion of 
it here on linux-kernel, if you search for systemd and cgroups you should find 
some of the discussions. This came up and got quite heated when the changes were 
first being proposed. If you look at the later discussions regarding the changes 
to cgroups you should be able to find the posts where the part about enabling 
wider access to cgroups has been said. There have also been long threads on 
lwn.net on the topic.

sorry I can't give you better pointers from memory.

David Lang

Re: [proposal] delegating cgroup manager to non-PID1

[Luke Kenneth Casson Leighton]

On Tue, Mar 3, 2015 at 1:45 AM, David Lang <david@lang.hm> wrote:
> On Tue, 3 Mar 2015, Luke Kenneth Casson Leighton wrote:
>
>> On Mon, Mar 2, 2015 at 12:13 AM, David Lang <david@lang.hm> wrote:
>>>
>>> On Sun, 1 Mar 2015, Luke Kenneth Casson Leighton wrote:
>>>
>>>> in recent discussions about PID-1 alternatives (sysvinit, openrc,
>>>> systemd, depinit) i was alerted to the idea that PID1 is to become the
>>>> sole exlcusive process permitted to manage cgroups.  given that, just
>>
>>
>>> There is less agreement on the idea that PID1 will have exclusive control
>>> over cgroups than some of the posts make it seem.
>>
>>
>> david, small favour to ask you (for archival purposes as well as
>> convenience) - finding posts, or even the mailing lists themselves, on
>> which the discussions you mention may be found, turns out to be um
>> rather challenging if you happen not to have subscribed to those
>> lists, such that every day you've seen all the discussions and know
>> what to look for, and where.
>>
>> would you therefore be so kind as to publish some hints - specific
>> keyword phrases more general than "PID1 cgroups" which is about the
>> limit of what i can think might apply (which ironically comes up with
>> every archive linking to *this* thread.... *sigh*), or better perhaps
>> some specific references to specific mailing lists.
>
>
> I was tryng to avoid fanning the flames, but there has been some discussion
> of it here on linux-kernel, if you search for systemd and cgroups you should
> find some of the discussions. This came up and got quite heated when the
> changes were first being proposed.

 *cringe* yehh i suspected as much.  i've been thinking about this for
quite some time: there is something very _very_ strange going on here,
where everyone gets on with their chosen role of dealing (technically)
with their specific area of expertise, in an extremely competent and
perfectly logical and acceptable way, which nobody can (or should)
fault them for or criticise...

 ... yet the changes being spearheaded by the systemd team, which by
coincidence and correlation happens to have funnelled into its remit
the fixing of a large number of inherent long-standing problems, is
causing *massive* amounts of tension and ire, causing in some cases
long-standing associates of over ten to fifteen years to abandon the
work that they love.

> If you look at the later discussions
> regarding the changes to cgroups you should be able to find the posts where
> the part about enabling wider access to cgroups has been said. There have
> also been long threads on lwn.net on the topic.

 ok i'll have another go.

> sorry I can't give you better pointers from memory.

 no problem david: if anyone else recalls the discussions quicker,
greatly appreciated the [published] references.

l.

Re: [proposal] delegating cgroup manager to non-PID1

[Luke Kenneth Casson Leighton]

https://lkml.org/lkml/2013/4/5/535

ok found one: i'll read up and see if there's evidence that the idea
mentioned a couple of days ago has occurred to the cgroups devs
(tejun) - i think it has, but i'll need to check to be sure.

l.