Re: [PATCH 01/13] kdbus: add documentation

[ebiederm@xmission.com (Eric W. Biederman)]

Andy Lutomirski <luto@amacapital.net> writes:

> On Tue, Feb 3, 2015 at 2:09 AM, Daniel Mack <daniel@zonque.org> wrote:
>> Hi Andy,
>>
>> On 02/02/2015 09:12 PM, Andy Lutomirski wrote:
>>> On Feb 2, 2015 1:34 AM, "Daniel Mack" <daniel@zonque.org> wrote:
>>
>>>> That's right, but again - if an application wants to gather this kind of
>>>> information about tasks it interacts with, it can do so today by looking
>>>> at /proc or similar sources. Desktop machines do exactly that already,
>>>> and the kernel code executed in such cases very much resembles that in
>>>> metadata.c, and is certainly not cheaper. kdbus just makes such
>>>> information more accessible when requested. Which information is
>>>> collected is defined by bit-masks on both the sender and the receiver
>>>> connection, and most applications will effectively only use a very
>>>> limited set by default if they go through one of the more high-level
>>>> libraries.
>>>
>>> I should rephrase a bit.  Kdbus doesn't require use of send-time
>>> metadata.  It does, however, strongly encourage it, and it sounds like
>>
>> On the kernel level, kdbus just *offers* that, just like sockets offer
>> SO_PASSCRED. On the userland level, kdbus helps applications get that
>> information race-free, easier and faster than they would otherwise.
>>
>>> systemd and other major users will use send-time metadata.  Once that
>>> happens, it's ABI (even if it's purely in userspace), and changing it
>>> is asking for security holes to pop up.  So you'll be mostly stuck
>>> with it.
>>
>> We know we can't break the ABI. At most, we could deprecate item types
>> and introduce new ones, but we want to avoid that by all means of
>> course. However, I fail to see how that is related to send time
>> metadata, or even to kdbus in general, as all ABIs have to be kept stable.
>
> I should have said it differently.  ABI is the wrong term -- it's more
> of a protocol issue.
>
> It looks like, with the current code, the kernel will provide
> (optional) send-time metadata, and the sd-bus library will use it.
> The result will be that the communication protocol between clients and
> udev, systemd, systemd-logind, g-s-d, etc, will likely involve
> send-time metadata.  This may end up being a bottleneck.

A quick note on a couple of things I have seen in this conversation.

- The reason for kdbus is performance.

- pipes rather than unix domain sockets are likely the standard to meet.
  If you can't equal unix domain sockets for simple things you are
  likely leaving a lot of stops in.  Last I looked pipes in general were
  notiably faster than unix domain sockets.

  The performance numbers I saw posted up-thread were horrible.  I have
  seen faster numbers across a network of machines.  If your ping-pong
  latency isn't measured in nano-seconds you are probably doing
  something wrong.

- syscalls remove overhead.  So since performance is kdbus's reason for existence
  let's remove some ridiculous stops, and get a fast path into the kernel.

- send-time metadata is a performance nightmare.  SO_PASSCRED is hard
  to implement in a fast performant way, especially when namespaces
  get involved.  Over the long term if you use send-time metadata
  you will grow the kind of compatibility hacks that the user
  namespace and the pid namespace have on SO_PASSCRED and things will
  slow down.

  A similar effect that is more performant in general is to enforce that
  the sender has the expected attributes.

> Once this happens, changing the protocol will be very hard without
> introducing security bugs.  If people start switching to
> connection-time metadata to gain performance, then they'll break both
> the communication protocol and the expectations of client code.  (In
> fact, it'll break twice, sort of, since I think that the current
> protocols are connect-time.)
>
> To me, this seems like a down-side of using send-time metadata, albeit
> possibly not a huge downside at least in the near term.  I don't see a
> corresponding benefit, though.

I think send-time metadata verification is less bad in this regard.

Eric