Re: skb->truesize > sk->rcvbuf == Dropped packets

Re: skb->truesize > sk->rcvbuf == Dropped packets

[mike_phillips]

>> I can implement one solution by copying the received packets into skb's 

>> with the correct length, but that eliminates the performance gains from 

>> simply swapping buffers around (and would definately mean no 
zero-copy). 

> Generally it is a win to copy rather than swap buffers when the frame 
does
> not occupy most of the buffer. Most traffic is very small or MTU sized
> frames (and on TR of course ethernet not TR mtu frames are popular)

Any suggestions on heuristics for this ? 
Say maybe copy if skb->len <= eth_max_mtu, skb->len <= skb->truesize * .5, 
or just copy the packets no matter what size. 

Mike

Re: skb->truesize > sk->rcvbuf == Dropped packets

[David S. Miller]

mike_phillips@urscorp.com writes:
 > Any suggestions on heuristics for this ? 
 > Say maybe copy if skb->len <= eth_max_mtu, skb->len <= skb->truesize * .5, 
 > or just copy the packets no matter what size. 

Pick a percentage of "acceptable waste" (ie. ratio of truesize to
len).

We know that cutoff values ~200 work for ethernet, for example.

On the topic of these checks that drop the packets, just so there
there is no misunderstanding.  We can't remove them else users
could consume memory in a basically uncontrolled manner.

Later,
David S. Miller
davem@redhat.com

Re: skb->truesize > sk->rcvbuf == Dropped packets

[kuznet]

Hello!

>  > Any suggestions on heuristics for this ? 

Not to set rcvbuf to ridiculously low values. The best variant is not
to touch SO_*BUF options at all.

Alexey

Re: skb->truesize > sk->rcvbuf == Dropped packets

[David S. Miller]

kuznet@ms2.inr.ac.ru writes:
 > >  > Any suggestions on heuristics for this ? 
 > 
 > Not to set rcvbuf to ridiculously low values. The best variant is not
 > to touch SO_*BUF options at all.

Hmmm... I don't see how not touching buffer values can solve his
problem at all.  His MTU is really HUGE, and in this case 300 byte
packet eats 10k or so space in receive buffer.

I doubt our buffer size tuning algorithms can cope with this.

Really, copy threshold in driver just must be choosen carefully.

Later,
David S. Miller
davem@redhat.com

Re: skb->truesize > sk->rcvbuf == Dropped packets

[kuznet]

Hello!

> Hmmm... I don't see how not touching buffer values can solve his
> problem at all.  His MTU is really HUGE, and in this case 300 byte
> packet eats 10k or so space in receive buffer.

Default rcvbuf is ~64K, it is enough to receive up to mtu of a bit less 64K.
When application says rcvbuf=2K it appearently does not expect to hold
more then one packet.



> I doubt our buffer size tuning algorithms can cope with this.

At least TCP will tune itself nicely under the most extremal conditions.
What's about this case, no chances to tune. rcvbuf just should be large.


> Really, copy threshold in driver just must be choosen carefully.

rx copybreak has no chances to work in the case of large mtus...
F.e. it does not with gigabit cards, which use 9K mtu, but need
to talk to 1.5K world. Blind copybreak at 1.5K is a non-sense,
meaning "copy all". Though acenic with its three rx rings is
a lucky exception. 8)

Alexey

Re: skb->truesize > sk->rcvbuf == Dropped packets

[mike_phillips]

> > >  > Any suggestions on heuristics for this ? 
> > 
> > Not to set rcvbuf to ridiculously low values. The best variant is not
> > to touch SO_*BUF options at all.

> Hmmm... I don't see how not touching buffer values can solve his
> problem at all.  His MTU is really HUGE, and in this case 300 byte
> packet eats 10k or so space in receive buffer.

> I doubt our buffer size tuning algorithms can cope with this.

Yep, it's no big thing to make the change in the driver, the copy is not 
that expensive and compared to the speed of the physical layer its 
virtually a non-impact. The most I've ever got out of 16 mbps t/r is just 
over 2 mb/second and memory copies can easily keep up with that. (That's 
with 8192 byte mtu's and ftp transfers which wouldn't get copied anyway). 

Mike

Re: skb->truesize > sk->rcvbuf == Dropped packets

[Alan Cox]

> I can implement one solution by copying the received packets into skb's 
> with the correct length, but that eliminates the performance gains from 
> simply swapping buffers around (and would definately mean no zero-copy). 

Generally it is a win to copy rather than swap buffers when the frame does
not occupy most of the buffer. Most traffic is very small or MTU sized
frames (and on TR of course ethernet not TR mtu frames are popular)

skb->truesize > sk->rcvbuf == Dropped packets

[mike_phillips]

Under certain conditions (e.g. nfs) the socket receive buffer is set to 
2048, but in the dma token ring drivers we have the receive skb's set to 
mtu size, i.e. anything up to 18200. The default for these drivers is 
4096. 

So, when any packets are received, even though the skb->len is less than 
sk->rcvbuf, these packets are getting dropping in sock_queue_rcv_skb 
causing massive timeout problems with nfs. 

I can implement one solution by copying the received packets into skb's 
with the correct length, but that eliminates the performance gains from 
simply swapping buffers around (and would definately mean no zero-copy). 

Is there a better way to do this, or can any changes be made in the socket 
handling functions ?

Mike