2.2 networking, NET_BH latency

2.2 networking, NET_BH latency

[Stelian Pop]

[ 
  I posted this on lkml two weeks ago, and got no responses. 
  Am I asking something too trivial or nobody knows the answers ? :-)
  
  Thanks.
]

Hi,

I experience some odd behaviour when routing some network packets
on a 2.2(.18) kernel (with Ingo's low latency patch in case it 
matters).

Although there are probably bugs in the modifications we made
(a network card driver, some tweaks in the network core to deal
with several packet priorities etc), I'm not sure the behaviour
is directly due to a bug in our modifications or some synchronisation
issue we overlooked.

So, the network driver receives a packet, pushes it to the upper
layers (netif_rx), the packet does all its job in the network
layers (in the NET_BH bottom-half), it gets routed to another 
interface, and get send.

The problem is that the time of the treatment (measured as time
between the moments when the packet enters the box and exits it)
_always_ exceeds HZ (in fact it is between 1*HZ and 2*HZ). 

Is this normal ? 

Is this related to the scheduling of NET_BH ? In this case, is it
possible to schedule the bottom-half more often ?

It should be noted that, each time a packet is received by the
network card, the driver wakes up a process waiting in ioctl(), 
making it eligible. Could this have any influence on the above ?

In order to respect some minimum timing requirements, we took the
approach of increasing HZ. Since the net latency (at least in our
case) is directly related to the value of the tick, it works. But
maybe there is a better solution.

Thanks,

Stelian.
-- 
Stelian Pop <stelian.pop@fr.alcove.com>
Alcove - http://www.alcove.com

Re: 2.2 networking, NET_BH latency

[James Morris]

On Tue, 10 Dec 2002, Stelian Pop wrote:

> I experience some odd behaviour when routing some network packets
> on a 2.2(.18) kernel (with Ingo's low latency patch in case it 
> matters).
> 
> Although there are probably bugs in the modifications we made
> (a network card driver, some tweaks in the network core to deal
> with several packet priorities etc), I'm not sure the behaviour
> is directly due to a bug in our modifications or some synchronisation
> issue we overlooked.

Can you reproduce the problem with a vanilla 2.2.23 kernel?


- James
-- 
James Morris
<jmorris@intercode.com.au>

Re: 2.2 networking, NET_BH latency

[Stelian Pop]

On Wed, Dec 11, 2002 at 02:46:11AM +1100, James Morris wrote:

> On Tue, 10 Dec 2002, Stelian Pop wrote:
> 
> > I experience some odd behaviour when routing some network packets
> > on a 2.2(.18) kernel (with Ingo's low latency patch in case it 
> > matters).
> > 
> > Although there are probably bugs in the modifications we made
> > (a network card driver, some tweaks in the network core to deal
> > with several packet priorities etc), I'm not sure the behaviour
> > is directly due to a bug in our modifications or some synchronisation
> > issue we overlooked.
> 
> Can you reproduce the problem with a vanilla 2.2.23 kernel?

I didn't try yet, but it is on my list.

Should I interpret your message as some changes between 2.2.18 and 2.2.23
could be responsible for that behaviour or you are just shooting in the
dark ? :-)

Stelian.
-- 
Stelian Pop <stelian.pop@fr.alcove.com>
Alcove - http://www.alcove.com

Re: 2.2 networking, NET_BH latency

[James Morris]

On Tue, 10 Dec 2002, Stelian Pop wrote:

> > Can you reproduce the problem with a vanilla 2.2.23 kernel?
> 
> I didn't try yet, but it is on my list.
> 
> Should I interpret your message as some changes between 2.2.18 and 2.2.23
> could be responsible for that behaviour or you are just shooting in the
> dark ? :-)
> 

No.


- James
-- 
James Morris
<jmorris@intercode.com.au>

Re: 2.2 networking, NET_BH latency

[Stelian Pop]

On Wed, Dec 11, 2002 at 02:57:32AM +1100, James Morris wrote:

> On Tue, 10 Dec 2002, Stelian Pop wrote:
> 
> > > Can you reproduce the problem with a vanilla 2.2.23 kernel?
> > 
> > I didn't try yet, but it is on my list.
> > 
> > Should I interpret your message as some changes between 2.2.18 and 2.2.23
> > could be responsible for that behaviour or you are just shooting in the
> > dark ? :-)
> > 
> 
> No.

I take it as 'no, this was not a shot in the dark' then... 

Could you please explain a bit more the problem ? Maybe point me to
some networking changes in particular ? (in my particular case, it
may be simpler to just try a specific patch instead of going directly
to 2.2.23...).

Thanks,

Stelian.
-- 
Stelian Pop <stelian.pop@fr.alcove.com>
Alcove - http://www.alcove.com

Re: 2.2 networking, NET_BH latency

[Stelian Pop]

On Tue, Dec 10, 2002 at 01:02:04PM -0800, David S. Miller wrote:

> > [ 
> >   I posted this on lkml two weeks ago, and got no responses. 
> >   Am I asking something too trivial or nobody knows the answers ? :-)
> >   
> >   Thanks.
> > ]
> 
> Most of us have forgotten even how the 2.2.x networking works
> it's been so long since we even glanced at the code.

Well, that's an easy excuse, especially coming from you :-)

Stelian.
-- 
Stelian Pop <stelian.pop@fr.alcove.com>
Alcove - http://www.alcove.com

Re: 2.2 networking, NET_BH latency

[David S. Miller]

On Tue, 2002-12-10 at 07:31, Stelian Pop wrote:
> [ 
>   I posted this on lkml two weeks ago, and got no responses. 
>   Am I asking something too trivial or nobody knows the answers ? :-)
>   
>   Thanks.
> ]

Most of us have forgotten even how the 2.2.x networking works
it's been so long since we even glanced at the code.

2.2 networking, NET_BH latency

[Stelian Pop]

Hi,

I experience some odd behaviour when routing some network packets
on a 2.2(.18) kernel (with Ingo's low latency patch in case it 
matters).

Although there are probably bugs in the modifications we made
(a network card driver, some tweaks in the network core to deal
with several packet priorities etc), I'm not sure the behaviour
is directly due to a bug in our modifications or some synchronisation
issue we overlooked.

So, the network driver receives a packet, pushes it to the upper
layers (netif_rx), the packet does all its job in the network
layers (in the NET_BH bottom-half), it gets routed to another 
interface, and get send.

The problem is that the time of the treatment (measured as time
between the moments when the packet enters the box and exits it)
_always_ exceeds HZ (in fact it is between 1*HZ and 2*HZ). 

Is this normal ? 

Is this related to the scheduling of NET_BH ? In this case, is it
possible to schedule the bottom-half more often ?

It should be noted that, each time a packet is received by the
network card, the driver wakes up a process waiting in ioctl(), 
making it eligible. Could this have any influence on the above ?

In order to respect some minimum timing requirements, we took the
approach of increasing HZ. Since the net latency (at least in our
case) is directly related to the value of the tick, it works. But
maybe there is a better solution.

Thanks,

Stelian.
-- 
Stelian Pop <stelian.pop@fr.alcove.com>
Alcove - http://www.alcove.com