RE: IP Acounting Idea for 2.5

RE: IP Acounting Idea for 2.5

[Leif Sawyer]

Manfred Bartz responded to
> Russell King <rmk@arm.linux.org.uk> who writes:
> 
> > On Mon, Apr 16, 2001 at 12:07:31PM +1000, Manfred Bartz wrote:
> > > There is another issue with logging in general:
> > > 
> > >                 *COUNTERS MUST NOT BE RESETABLE!!!*
> > 
> > Umm, no.  Counters can be resetable - you just specify that 
> > accounting programs should not reset them, ever.
> > 
> > The ability to reset counters is extremely useful if you're a human
> > looking at the output of iptables -L -v.  (I thus far know of no one
> > who can memorise the counter values for around 40 rules).
> 
> You just illustrated my point.  While there is a reset capability
> people will use it and accounting/logging programs will get wrong
> data.  Resetable counters might be a minor convenience when debugging
> but the price is unreliable programs and the loss of the ability of
> several programs to use the same counters.

You of course, are commenting from the fact that your applications are
stupid, written poorly, and cannot handle 'wrapped' data.  Take MRTG
as one example of a good application, for it can handle counters that
have been cleared between reads.
 
> Also, do you always reset the date and time everytime you make time 
> measurements?

No, but i realize that if my time has incremented, and my date has not
changed, then it's later.  If my time has _decremented_ and my date has not,
then I've time-warped.  If my time has decremented and my date has
incremented,
then it's simply the next-date-increment.

Similarly, if my InPackets are at 102345 at one read, and 2345 the next
read,
and I know that my counter is 32 bits, then I know i've wrapped and can do
my
own math.

Applications need to get smarter.  Don't place arbitrary rules where they
don't
belong.

Re: IP Acounting Idea for 2.5

[Ian Stirling]

> 
> Manfred Bartz responded to
> > Russell King <rmk@arm.linux.org.uk> who writes:
<snip>
> > You just illustrated my point.  While there is a reset capability
> > people will use it and accounting/logging programs will get wrong
> > data.  Resetable counters might be a minor convenience when debugging
> > but the price is unreliable programs and the loss of the ability of
> > several programs to use the same counters.
> 
> You of course, are commenting from the fact that your applications are
> stupid, written poorly, and cannot handle 'wrapped' data.  Take MRTG
<snip>
> Similarly, if my InPackets are at 102345 at one read, and 2345 the next
> read,
> and I know that my counter is 32 bits, then I know i've wrapped and can do

I think the point being made is that if InPackets are at 102345 at one read,
and 2345 the next, and you know it's a 32 bit counter, it's completely
unreliable to assume that you have in fact recieved 4294867295
packets, if the counter can be zeroed.
You can say nothing other than at least 2345 packets, at most 
2345+n*2^32 have been got since you last checked.

Re: IP Acounting Idea for 2.5

[Manfred Bartz]

Leif Sawyer <lsawyer@gci.com> writes:

> Manfred Bartz responded to
> > Russell King <rmk@arm.linux.org.uk> who writes:
> > 
> > > On Mon, Apr 16, 2001 at 12:07:31PM +1000, Manfred Bartz wrote:
> > > > There is another issue with logging in general:
> > > > 
> > > >                 *COUNTERS MUST NOT BE RESETABLE!!!*
> > > 
> > > Umm, no.  Counters can be resetable - you just specify that 
> > > accounting programs should not reset them, ever.
> > > 
> > > The ability to reset counters is extremely useful if you're a human
> > > looking at the output of iptables -L -v.  (I thus far know of no one
> > > who can memorise the counter values for around 40 rules).
> > 
> > You just illustrated my point.  While there is a reset capability
> > people will use it and accounting/logging programs will get wrong
> > data.  Resetable counters might be a minor convenience when debugging
> > but the price is unreliable programs and the loss of the ability of
> > several programs to use the same counters.
> 
> You of course, are commenting from the fact that your applications are
> stupid, written poorly, and cannot handle 'wrapped' data.  

Well, apps written by me *can handle* wrapped data; it is actually far
easier that resetting at each read.

> Take MRTG as one example of a good application, for it can handle
> counters that have been cleared between reads.

Regardless of how well a logging application is written, there is no
way that it does not loose data when someone or something resets its
counter.  F.e. if an app reads a counter every 10 minutes and a
counter reset occurs from elsewhere, then anywhere between 0 and 10
minutes worth of data is lost.

I had a brief look at MRTG.  It seems to be a well written app and
while it can handle counter reset (with potential loss of an unknown 
amount of data), it does not actively reset counters.  It also doesn't
use iptables.

> Applications need to get smarter.  

Agreed.

> Don't place arbitrary rules where they don't belong.

Agreed too.  Counters should not arbitrarily be equipped with a reset
capability, there is hardly any benefit in that and it causes nothing
but problems.

As far as I can see, the counters in /proc/net/snmp don't have a
reset, same with /proc/net/dev and possibly other counters elsewhere.

Ideally iptables would fall in line with that.  Rules can still be
unloaded and reloaded, also causing counter reset and loss of data,
but since that is a lot more involved, application authors would have
an incentive to handle counters properly.

-- 
Manfred Bartz

[Counters] Re: IP Acounting Idea for 2.5

[Harald Welte]

On Tue, Apr 17, 2001 at 11:13:19AM +1000, Manfred Bartz wrote:

> I had a brief look at MRTG.  It seems to be a well written app and
> while it can handle counter reset (with potential loss of an unknown 
> amount of data), it does not actively reset counters.  It also doesn't
> use iptables.

Yes, that's the whole point. If you want to have some kind of per-ip / 
per-network accounting, I'd rather recommend using something else
(i.e. an accounting module attaching to the netfilter hook itself,
something using the ULOG target to do the logging in userspace, ...)

Just reading out the per-rule counters always introduces problems as soon
as your ruleset changes. And when do you have a really static ruleset?
There are always new services/... to configure.

> Agreed too.  Counters should not arbitrarily be equipped with a reset
> capability, there is hardly any benefit in that and it causes nothing
> but problems.

so what about iptables-save at shutdown time and iptables-restore at 
bootup time? Then you can have your counters persist even after kernel
upgrades / reboots / crashes / ... 

> As far as I can see, the counters in /proc/net/snmp don't have a
> reset, same with /proc/net/dev and possibly other counters elsewhere.

Yes, because it is per network device, not per some arbitrarily inserted
rule which can be changed all the time. As I've stated more than once
in this thread, and as you just continue to say: just delete and re-insert
the rule, and you have your counter reset.

> Ideally iptables would fall in line with that.  Rules can still be
> unloaded and reloaded, also causing counter reset and loss of data,
> but since that is a lot more involved, application authors would have
> an incentive to handle counters properly.

I don't think that the iptables kernel part should remove some feature
just because there are application programmers wrongly designing their
applications.


> Manfred Bartz

-- 
Live long and prosper
- Harald Welte / laforge@gnumonks.org                http://www.gnumonks.org
============================================================================
GCS/E/IT d- s-: a-- C+++ UL++++$ P+++ L++++$ E--- W- N++ o? K- w--- O- M- 
V-- PS+ PE-- Y+ PGP++ t++ 5-- !X !R tv-- b+++ DI? !D G+ e* h+ r% y+(*)

Re: IP Acounting Idea for 2.5

[Olaf Titz]

> Similarly, if my InPackets are at 102345 at one read, and 2345 the
> next read, and I know that my counter is 32 bits, then I know i've
> wrapped and can do my own math.

No. When you have resettable counters, you don't know if the counter
has wrapped or been reset. Either you have received 2345 packets in
between, or 2^32-102345. The difference is not negligible. ;-)

Unless you have a second counter which is incremented with every wrap
(_or_, perhaps better, with every reset) and can _not_ be reset
manually. This would be the date in your date/time example.

Olaf