Re: New net features for added performance

["David S. Miller" <davem@redhat.com>]

Jeff Garzik writes:
 > 1) Rx Skb recycling.
 ...
 > Advantages:  A de-allocation immediately followed by a reallocation is
 > eliminated, less L1 cache pollution during interrupt handling. 
 > Potentially less DMA traffic between card and host.
 ...
 > Disadvantages?

It simply cannot work, as Alexey stated, in normal circumstances
netif_rx() queues until the user reads the data.  This is the whole
basis of our receive packet processing model within softint/user
context.

Secondly, I can argue that skb recycling can give _worse_ cache
performance.  If the next use and access by the card to the
skb data is deferred, this gives the cpu a chance to displace those
lines in it's cache naturally via displacement instead of being forced
quickly to do so when the device touches that data.

If the device forces the cache displacement, those cache lines become
empty until filled with something later (smaller utilization of total
cache contents) whereas natural displacement puts useful data into
the cache at the time of the displacement (larger utilization of total
cache contents).

It is an NT/windows driver API rubbish idea, and it is full crap.

 > 2) Tx packet grouping.
 ...
 > Disadvantages?

See Torvalds vs. world discussion on this list about API entry points
which pass multiple pages at a time versus simpler ones which pass
only a single page at a time. :-)

 > 3) Slabbier packet allocation.
 ...
 > Disadvantages?  Doing this might increase cache pollution due to
 > increased code and data size, but I think the hot path is much improved
 > (dequeue a properly sized, initialized, skb-reserved'd skb off a list)
 > and would help mitigate the impact of sudden bursts of traffic.

I don't know what I think about this one, but my hunch is that it will
lead to worse data packing via such an allocator.

Later,
David S. Miller
davem@redhat.com