Accelerated TCP/IP support from kernel

Accelerated TCP/IP support from kernel

[Bharath Madhavan]

Hello all,
	I am looking into a scenario where we have a NIC which performs 
all the TCP/IP processing and basically the core CPU offloads all data from
the socket level interface onwards to this NIC. 
Can Linux do this as of now. I saw some limited support like TCP/IP
checksumming
being done in HW (ex: 3c905c) and linux supports it. I also saw the
ZERO_COPY
feature in linux code. 
My question is can there be support for entire (almost) TCP/IP to be
offloaded 
to a NIC card. I noticed some NIC cards by Alacritec and Applicom who have 
such NICs but it looks like the support from Linux would be quite a bit of
work.
Basically Linux needs to support all socket calls as it will usually but
the socket interface must be intelligent enough to call different drivers(!)
for those NIC cards and thus bypassing the whole of TCP/IP stack of Linux.
Is this possible at all and if so, is anyone doing this???
I talked to a vendor who does this NIC and he said that they provide a
library
for linux which means that they have proprietery functions for using their
NIC
and thus this cannot be used in the standard socket interface way. That is a
big
limitation and not of much use. Basically, I am looking into a case where 
the kernel code needs to be changed to handle this
Looking eagerly for some help/advice from you folks
Thanks a lot
Bharath

Re: Accelerated TCP/IP support from kernel

[David S. Miller]

Bharath Madhavan writes:
 > 	I am looking into a scenario where we have a NIC which performs 
 > all the TCP/IP processing and basically the core CPU offloads all data from
 > the socket level interface onwards to this NIC. 

Why would you ever want to do this?

Point 1: Support for new TCP techniques and bug fixes are hard enough
to propagate to user's systems as it is with the implementation being
done in software.

Point 2: If I find a bug in the cards TCP implementation, will I be
able to get at the source for the firmware and fix it?  Likely the
answer to this is no.

Every couple years a few vendors make cards like this, yet ignore
these core issues.  It is currently impractical to use these kinds of
cards today except in a few very special case situations.

Furthermore, the actual protocol processing overhead is quite small
and almost neglible especially on today's gigahertz beasts.  The data
copy is where the time is spent, and checksum offload takes care of
that.

Later,
David S. Miller
davem@redhat.com

RE: Accelerated TCP/IP support from kernel

[Bharath Madhavan]

Thanks a lot. That was useful info especially your last point
where you are saying that most of the area we can save is in
data processing and not in protocol processing.
So, if we use the ZERO_COPY feature, we should gain quite a bit.
I guess 3c905c NIC supports HW checksumming. Is this true?
In this case, do we have any benchmarking for this card 
with and without ZERO_COPY (and HW checksumming). I am eager to
know by how many times did the system throughput increase?
Thanks a lot
Bharath




-----Original Message-----
From: David S. Miller [mailto:davem@redhat.com]
Sent: Thursday, May 24, 2001 6:29 PM
To: Bharath Madhavan
Cc: linux-kernel@vger.kernel.org
Subject: Re: Accelerated TCP/IP support from kernel



Bharath Madhavan writes:
 > 	I am looking into a scenario where we have a NIC which performs 
 > all the TCP/IP processing and basically the core CPU offloads all data
from
 > the socket level interface onwards to this NIC. 

Why would you ever want to do this?

Point 1: Support for new TCP techniques and bug fixes are hard enough
to propagate to user's systems as it is with the implementation being
done in software.

Point 2: If I find a bug in the cards TCP implementation, will I be
able to get at the source for the firmware and fix it?  Likely the
answer to this is no.

Every couple years a few vendors make cards like this, yet ignore
these core issues.  It is currently impractical to use these kinds of
cards today except in a few very special case situations.

Furthermore, the actual protocol processing overhead is quite small
and almost neglible especially on today's gigahertz beasts.  The data
copy is where the time is spent, and checksum offload takes care of
that.

Later,
David S. Miller
davem@redhat.com
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

RE: Accelerated TCP/IP support from kernel

[David S. Miller]

Bharath Madhavan writes:
 > I guess 3c905c NIC supports HW checksumming. Is this true?

Yes.

 > In this case, do we have any benchmarking for this card 
 > with and without ZERO_COPY (and HW checksumming). I am eager to
 > know by how many times did the system throughput increase?

It doesn't matter with 100baseT cards, they are slow enough that even
with the cpu doing the data copies the link may be easily saturated.
What you will get is decreased CPU utilization.

You need to go to gigabit or faster link speeds to see any real
throughput improvement.

Later,
David S. Miller
davem@redhat.com

RE: Accelerated TCP/IP support from kernel

[Gerhard Mack]

I'm curious.. do your cards support IPv6 and ECN ?

	Gerhard


On Thu, 24 May 2001, Bharath Madhavan wrote:

> Thanks a lot. That was useful info especially your last point
> where you are saying that most of the area we can save is in
> data processing and not in protocol processing.
> So, if we use the ZERO_COPY feature, we should gain quite a bit.
> I guess 3c905c NIC supports HW checksumming. Is this true?
> In this case, do we have any benchmarking for this card 
> with and without ZERO_COPY (and HW checksumming). I am eager to
> know by how many times did the system throughput increase?
> Thanks a lot
> Bharath
> 
> 
> 
> 
> -----Original Message-----
> From: David S. Miller [mailto:davem@redhat.com]
> Sent: Thursday, May 24, 2001 6:29 PM
> To: Bharath Madhavan
> Cc: linux-kernel@vger.kernel.org
> Subject: Re: Accelerated TCP/IP support from kernel
> 
> 
> 
> Bharath Madhavan writes:
>  > 	I am looking into a scenario where we have a NIC which performs 
>  > all the TCP/IP processing and basically the core CPU offloads all data
> from
>  > the socket level interface onwards to this NIC. 
> 
> Why would you ever want to do this?
> 
> Point 1: Support for new TCP techniques and bug fixes are hard enough
> to propagate to user's systems as it is with the implementation being
> done in software.
> 
> Point 2: If I find a bug in the cards TCP implementation, will I be
> able to get at the source for the firmware and fix it?  Likely the
> answer to this is no.
> 
> Every couple years a few vendors make cards like this, yet ignore
> these core issues.  It is currently impractical to use these kinds of
> cards today except in a few very special case situations.
> 
> Furthermore, the actual protocol processing overhead is quite small
> and almost neglible especially on today's gigahertz beasts.  The data
> copy is where the time is spent, and checksum offload takes care of
> that.
> 
> Later,
> David S. Miller
> davem@redhat.com
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

--
Gerhard Mack

gmack@innerfire.net

<>< As a computer I find your faith in technology amusing.

RE: Accelerated TCP/IP support from kernel

[Bharath Madhavan]

Hello all,
	We will most probably be looking into supporting Gig MACs and that
is
the reason why I was wondering if HW assist will help.
We are talking of Gigs of network data (dont ask me the application!)
I dont know about IPv6 support, we havent thought of that.
But the point is, We can visualize a scenario where the processor
is overworked and we want to relieve it off some of its load.
To summarize the info I got from the mailing list, there are two
conflicting views about this topic.
1. Protocol processing does not take much time compared to data processing 
and so the ZERO_COPY/HW_CHECKSUM feature in Linux currently is good enough.
2. The application maybe unique and heavy load that it may save the
processor
some bandwidth especially when several TCP connections needs to be
maintained.

I am mainly looking into case 2, and just trying to see the amount of work
that needs to go into this area. I am asking your expert opinions and find
out if this is an easy job, if so can it be done to maintain compatability 
with older NICs too and able to move along easily with standard linux
distributions. If the kernel can provide some hooks to "branch off" to
a different set of drivers at the socket level, that will do the trick I
think.
Is this easy?
Thanks a lot
Bharath






-----Original Message-----
From: David S. Miller [mailto:davem@redhat.com]
Sent: Thursday, May 24, 2001 7:08 PM
To: Bharath Madhavan
Cc: 'linux-kernel@vger.kernel.org'
Subject: RE: Accelerated TCP/IP support from kernel



Bharath Madhavan writes:
 > I guess 3c905c NIC supports HW checksumming. Is this true?

Yes.

 > In this case, do we have any benchmarking for this card 
 > with and without ZERO_COPY (and HW checksumming). I am eager to
 > know by how many times did the system throughput increase?

It doesn't matter with 100baseT cards, they are slow enough that even
with the cpu doing the data copies the link may be easily saturated.
What you will get is decreased CPU utilization.

You need to go to gigabit or faster link speeds to see any real
throughput improvement.

Later,
David S. Miller
davem@redhat.com

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/