* [ANNOUNCE] Linux hits 20 Gb/s stream to disk levels over Infiniband
@ 2012-03-31 17:28 Jeffrey Merkey
2012-03-31 19:15 ` David Dillow
0 siblings, 1 reply; 5+ messages in thread
From: Jeffrey Merkey @ 2012-03-31 17:28 UTC (permalink / raw)
To: linux-kernel
Made it to 20 Gb/s network packet capture stream to disk performance
levels over Infiniband fabrics on Linux. Will be trying this on 3.0
after I work through the OFED issues (long list). Post the results
from 3.0. These are from 2.6.XX series kernels.
Linux f_cking ROCKS!!!
Jeff
http://www.prlog.org/11838605-lighthorse-networks-achieves-20-gbs-network-packet-capture-stream-to-disk-performance-on-infiniband.html
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [ANNOUNCE] Linux hits 20 Gb/s stream to disk levels over Infiniband
2012-03-31 17:28 [ANNOUNCE] Linux hits 20 Gb/s stream to disk levels over Infiniband Jeffrey Merkey
@ 2012-03-31 19:15 ` David Dillow
2012-03-31 19:23 ` Jeffrey Merkey
0 siblings, 1 reply; 5+ messages in thread
From: David Dillow @ 2012-03-31 19:15 UTC (permalink / raw)
To: Jeffrey Merkey; +Cc: linux-kernel
On Sat, 2012-03-31 at 11:28 -0600, Jeffrey Merkey wrote:
> Made it to 20 Gb/s network packet capture stream to disk performance
> levels over Infiniband fabrics on Linux.
I'm assuming that the interesting part is the 20 Gbits/s capture? And at
minimum packet sizes, I'd hope. Linux has been able to stream more than
5 GBytes/s (40 Gbits/s) to disk for several years now over SRP.
> Will be trying this on 3.0
> after I work through the OFED issues (long list). Post the results
> from 3.0. These are from 2.6.XX series kernels.
You may find it easier to use a recent RedHat or SUSE with an upstream
kernel. I know that RHEL6 userland is compatible with the upstream
kernel's Infiniband, and OFED is moving its kernel work to a "backport
upstream" model for OFED 3.2.
> Linux f_cking ROCKS!!!
Indeed it does.
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [ANNOUNCE] Linux hits 20 Gb/s stream to disk levels over Infiniband
2012-03-31 19:15 ` David Dillow
@ 2012-03-31 19:23 ` Jeffrey Merkey
2012-04-01 3:51 ` Joseph Glanville
0 siblings, 1 reply; 5+ messages in thread
From: Jeffrey Merkey @ 2012-03-31 19:23 UTC (permalink / raw)
To: David Dillow; +Cc: linux-kernel
On 3/31/12, David Dillow <dave@thedillows.org> wrote:
> On Sat, 2012-03-31 at 11:28 -0600, Jeffrey Merkey wrote:
>> Made it to 20 Gb/s network packet capture stream to disk performance
>> levels over Infiniband fabrics on Linux.
>
> I'm assuming that the interesting part is the 20 Gbits/s capture? And at
> minimum packet sizes, I'd hope. Linux has been able to stream more than
> 5 GBytes/s (40 Gbits/s) to disk for several years now over SRP.
Network Capture while being routed and regenerated and piped through
apps at the same time from live network traffic. Most of the other
vendors tout 10 Gb/s stream to disk but only Solera really can do it.
All of them except Niksun are Linux and I think they moved some of
their stuff to it. So hitting 20 Gb/s is a big deal right now. With
Intel's push in IB/Ethernet, this is where the next thing is going.
3D Torus stuff is totally cool. You can actually store data in the
network itself. Talk about a challenge for Network security.
The next class of malicious programs may be adapted to live within
fabrics and require new methods to detect and remove them.
>
>> Will be trying this on 3.0
>> after I work through the OFED issues (long list). Post the results
>> from 3.0. These are from 2.6.XX series kernels.
>
> You may find it easier to use a recent RedHat or SUSE with an upstream
> kernel. I know that RHEL6 userland is compatible with the upstream
> kernel's Infiniband, and OFED is moving its kernel work to a "backport
> upstream" model for OFED 3.2.
>
I've got RH6 done, but OFED was down on the list and SLES was ahead
of.it. No rest for the weary.
Jeff
>> Linux f_cking ROCKS!!!
>
> Indeed it does.
>
>
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [ANNOUNCE] Linux hits 20 Gb/s stream to disk levels over Infiniband
2012-03-31 19:23 ` Jeffrey Merkey
@ 2012-04-01 3:51 ` Joseph Glanville
2012-04-01 5:18 ` Jeffrey Merkey
0 siblings, 1 reply; 5+ messages in thread
From: Joseph Glanville @ 2012-04-01 3:51 UTC (permalink / raw)
To: Jeffrey Merkey; +Cc: David Dillow, linux-kernel
On 1 April 2012 05:23, Jeffrey Merkey <jeffmerkey@gmail.com> wrote:
> On 3/31/12, David Dillow <dave@thedillows.org> wrote:
>> On Sat, 2012-03-31 at 11:28 -0600, Jeffrey Merkey wrote:
>>> Made it to 20 Gb/s network packet capture stream to disk performance
>>> levels over Infiniband fabrics on Linux.
>>
>> I'm assuming that the interesting part is the 20 Gbits/s capture? And at
>> minimum packet sizes, I'd hope. Linux has been able to stream more than
>> 5 GBytes/s (40 Gbits/s) to disk for several years now over SRP.
>
> Network Capture while being routed and regenerated and piped through
> apps at the same time from live network traffic. Most of the other
> vendors tout 10 Gb/s stream to disk but only Solera really can do it.
> All of them except Niksun are Linux and I think they moved some of
> their stuff to it. So hitting 20 Gb/s is a big deal right now. With
> Intel's push in IB/Ethernet, this is where the next thing is going.
> 3D Torus stuff is totally cool. You can actually store data in the
> network itself. Talk about a challenge for Network security.
You can do Infiniband routing in software? This is indeed very interesting.
Pinged your contact page. :)
>
> The next class of malicious programs may be adapted to live within
> fabrics and require new methods to detect and remove them.
>
>>
>>> Will be trying this on 3.0
>>> after I work through the OFED issues (long list). Post the results
>>> from 3.0. These are from 2.6.XX series kernels.
>>
>> You may find it easier to use a recent RedHat or SUSE with an upstream
>> kernel. I know that RHEL6 userland is compatible with the upstream
>> kernel's Infiniband, and OFED is moving its kernel work to a "backport
>> upstream" model for OFED 3.2.
>>
>
>
> I've got RH6 done, but OFED was down on the list and SLES was ahead
> of.it. No rest for the weary.
>
> Jeff
>
>>> Linux f_cking ROCKS!!!
>>
>> Indeed it does.
>>
>>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
Joseph.
--
Founder | Director | VP Research
Orion Virtualisation Solutions | www.orionvm.com.au | Phone: 1300 56
99 52 | Mobile: 0428 754 846
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [ANNOUNCE] Linux hits 20 Gb/s stream to disk levels over Infiniband
2012-04-01 3:51 ` Joseph Glanville
@ 2012-04-01 5:18 ` Jeffrey Merkey
0 siblings, 0 replies; 5+ messages in thread
From: Jeffrey Merkey @ 2012-04-01 5:18 UTC (permalink / raw)
To: Joseph Glanville; +Cc: David Dillow, linux-kernel
On 3/31/12, Joseph Glanville <joseph.glanville@orionvm.com.au> wrote:
> On 1 April 2012 05:23, Jeffrey Merkey <jeffmerkey@gmail.com> wrote:
>> On 3/31/12, David Dillow <dave@thedillows.org> wrote:
>>> On Sat, 2012-03-31 at 11:28 -0600, Jeffrey Merkey wrote:
>>>> Made it to 20 Gb/s network packet capture stream to disk performance
>>>> levels over Infiniband fabrics on Linux.
>>>
>>> I'm assuming that the interesting part is the 20 Gbits/s capture? And at
>>> minimum packet sizes, I'd hope. Linux has been able to stream more than
>>> 5 GBytes/s (40 Gbits/s) to disk for several years now over SRP.
>>
>> Network Capture while being routed and regenerated and piped through
>> apps at the same time from live network traffic. Most of the other
>> vendors tout 10 Gb/s stream to disk but only Solera really can do it.
>> All of them except Niksun are Linux and I think they moved some of
>> their stuff to it. So hitting 20 Gb/s is a big deal right now. With
>> Intel's push in IB/Ethernet, this is where the next thing is going.
>> 3D Torus stuff is totally cool. You can actually store data in the
>> network itself. Talk about a challenge for Network security.
>
> You can do Infiniband routing in software? This is indeed very interesting.
> Pinged your contact page. :)
Got your email. Was catching up on SLES testing today, so sorry for
the late response. Yes, we can support IP routing and regeneration
over IB fabrics. If your question is can we also support native
routing of some IB functionality, yes. But RDMA stuff is better left
direct without local store and forward in software. I will email
you off line and you can better describe what you want done on IB.
The api and ib verbs are tedious and detailed to
program, but the technology rocks. We use IB primarily as an IP and
storage fabric, but I have worked on several commercial products with
IB, notably, this one last year which does routing -- a ton of work.
www.kove.com
Jeff
>
>>
>> The next class of malicious programs may be adapted to live within
>> fabrics and require new methods to detect and remove them.
>>
>>>
>>>> Will be trying this on 3.0
>>>> after I work through the OFED issues (long list). Post the results
>>>> from 3.0. These are from 2.6.XX series kernels.
>>>
>>> You may find it easier to use a recent RedHat or SUSE with an upstream
>>> kernel. I know that RHEL6 userland is compatible with the upstream
>>> kernel's Infiniband, and OFED is moving its kernel work to a "backport
>>> upstream" model for OFED 3.2.
>>>
>>
>>
>> I've got RH6 done, but OFED was down on the list and SLES was ahead
>> of.it. No rest for the weary.
>>
>> Jeff
>>
>>>> Linux f_cking ROCKS!!!
>>>
>>> Indeed it does.
>>>
>>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>> Please read the FAQ at http://www.tux.org/lkml/
>
> Joseph.
>
> --
> Founder | Director | VP Research
> Orion Virtualisation Solutions | www.orionvm.com.au | Phone: 1300 56
> 99 52 | Mobile: 0428 754 846
>
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2012-04-01 5:18 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-03-31 17:28 [ANNOUNCE] Linux hits 20 Gb/s stream to disk levels over Infiniband Jeffrey Merkey
2012-03-31 19:15 ` David Dillow
2012-03-31 19:23 ` Jeffrey Merkey
2012-04-01 3:51 ` Joseph Glanville
2012-04-01 5:18 ` Jeffrey Merkey
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®