Re: [Patch][RFC] Disabling per-tgid stats on task exit in taskstats

[Andrew Morton <akpm@osdl.org>]

On Thu, 29 Jun 2006 15:10:31 -0400
Shailabh Nagar <nagar@watson.ibm.com> wrote:

> >I agree, and I'm viewing this as blocking the taskstats merge.  Because if
> >this _is_ a problem then it's a big one because fixing it will be
> >intrusive, and might well involve userspace-visible changes.
> >  
> >
> First off, just a reminder that this is inherently a netlink flow 
> control issue...which was being exacerbated
> earlier by taskstats decision to send per-tgid data (no longer the case).
> 
> But I'd like to know whats our target here ? How many messages per 
> second do we want to be able to be sent
> and received without risking any loss of data ? Netlink will lose 
> messages at a high enough rate so the design point
> will need to be known a bit.
> 
> For statistics type usage of the genetlink/netlink, I would have thought 
> that userspace, provided it is reliably informed
> about the loss of data through ENOBUFS, could take measures to just 
> account for the missing data and carry on ?

Could be so.  But we need to understand how significant the impact of this
will be in practice.

We could find, once this is deployed is real production environments on
large machines that the data loss is sufficiently common and sufficiently
serious that the feature needs a lot of rework.

Now there's always a risk of that sort of thing happening with all
features, but it's usually not this evident so early in the development
process.  We need to get a better understanding of the risk before
proceeding too far.

And there's always a 100% reliable fix for this: throttling.  Make the
sender of the messages block until the consumer can catch up.  In some
situations, that is what people will want to be able to do.  I suspect a
good implementation would be to run a collection daemon on each CPU and
make the delivery be cpu-local.  That's sounding more like relayfs than
netlink.