* [gentoo-user] 10 G eth (10000) on Gentoo
@ 2011-08-31 13:08 James
2011-08-31 23:58 ` Adam Carter
0 siblings, 1 reply; 7+ messages in thread
From: James @ 2011-08-31 13:08 UTC (permalink / raw
To: gentoo-user
Hello,
I just might have an opportunity to setup a Gentoo
System using 10 G ethernet (fiber). The .39
kernel lists this hardware [1].
Does anyone have any experience with this hardware?
Does it work? Did you make any bandwidth measurements?
What type of fiber (mulimode/singlemode) (ST/SC) did you
use?
Any comments? Is the kernel a bottleneck or your application?
--- Ethernet (10000 Mbit)
Chelsio 10Gb Ethernet support
Chelsio Communications T3 10Gb Ethernet support
Chelsio Communications T4 Ethernet support
Chelsio Communications T4 Virtual Function Ethernet support
Cisco VIC Ethernet NIC Support
Intel(R) 10GbE PCI Express adapters support
Intel(R) 82599 Virtual Function Ethernet support
Intel(R) PRO/10GbE support
Exar Xframe 10Gb Ethernet Adapter
Exar X3100 Series 10GbE PCIe Server Adapter
Myricom Myri-10G Ethernet support
NetXen Multi port (1/10) Gigabit Ethernet NIC
Sun Neptune 10Gbit Ethernet support
Mellanox Technologies 10Gbit Ethernet support
Tehuti Networks 10G Ethernet
Broadcom NetXtremeII 10Gb support
QLOGIC QLCNIC 1/10Gb Converged Ethernet NIC Support
QLogic QLGE 10Gb Ethernet Driver Support
Brocade 1010/1020 10Gb Ethernet Driver support
Solarflare Solarstorm SFC4000/SFC9000-family support
ServerEngines' 10Gbps NIC - BladeEngine
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [gentoo-user] 10 G eth (10000) on Gentoo
2011-08-31 13:08 [gentoo-user] 10 G eth (10000) on Gentoo James
@ 2011-08-31 23:58 ` Adam Carter
2011-09-01 1:43 ` Michael Mol
0 siblings, 1 reply; 7+ messages in thread
From: Adam Carter @ 2011-08-31 23:58 UTC (permalink / raw
To: gentoo-user
> I just might have an opportunity to setup a Gentoo
> System using 10 G ethernet (fiber). The .39
> kernel lists this hardware [1].
>
> Does anyone have any experience with this hardware?
> Does it work? Did you make any bandwidth measurements?
> What type of fiber (mulimode/singlemode) (ST/SC) did you
> use?
>
> Any comments? Is the kernel a bottleneck or your application?
Intel are active in linux kernel development and their linux drivers
seem to be very good.
When testing Intel copper gig interfaces on an intel firewall (HP
DL380G5 8 core box), I was able to send a core to 100% with ~330Mb of
small packets. The limiting factor appears to be packet rate, and the
consequent processing of interrupts (1 irq to 1 core). I don't think
that a 10Gig interface would pass any more than that due to similar
limitations.
Tweaking the e1000 driver options RxDescriptors, TxDescriptors and
RxIntDelay pushed it up to ~350Mb. MSI was enabled so no interrupt
sharing.
So if you're running normal sized packets, you should be ok. Otherwise
you way want to look at what irqbalance can do for you (I didn't try
it at the time).
Also don't forget stuff like Large Receive Offload in your kernel.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [gentoo-user] 10 G eth (10000) on Gentoo
2011-08-31 23:58 ` Adam Carter
@ 2011-09-01 1:43 ` Michael Mol
2011-09-01 1:46 ` Adam Carter
0 siblings, 1 reply; 7+ messages in thread
From: Michael Mol @ 2011-09-01 1:43 UTC (permalink / raw
To: gentoo-user
On Wed, Aug 31, 2011 at 7:58 PM, Adam Carter <adamcarter3@gmail.com> wrote:
>> I just might have an opportunity to setup a Gentoo
>> System using 10 G ethernet (fiber). The .39
>> kernel lists this hardware [1].
>>
>> Does anyone have any experience with this hardware?
>> Does it work? Did you make any bandwidth measurements?
>> What type of fiber (mulimode/singlemode) (ST/SC) did you
>> use?
>>
>> Any comments? Is the kernel a bottleneck or your application?
>
> Intel are active in linux kernel development and their linux drivers
> seem to be very good.
>
> When testing Intel copper gig interfaces on an intel firewall (HP
> DL380G5 8 core box), I was able to send a core to 100% with ~330Mb of
> small packets. The limiting factor appears to be packet rate, and the
> consequent processing of interrupts (1 irq to 1 core). I don't think
> that a 10Gig interface would pass any more than that due to similar
> limitations.
>
> Tweaking the e1000 driver options RxDescriptors, TxDescriptors and
> RxIntDelay pushed it up to ~350Mb. MSI was enabled so no interrupt
> sharing.
>
> So if you're running normal sized packets, you should be ok. Otherwise
> you way want to look at what irqbalance can do for you (I didn't try
> it at the time).
>
> Also don't forget stuff like Large Receive Offload in your kernel.
I've overheard IRC conversations that discussed multi-queue network
cards in the context of multi-core systems. My educated guess, based
on what you mention, is that each queue in the card would ping a
different interrupt. Each interrupt might be handled by a different
core, so you'd see a scaleable improvement there.
--
:wq
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [gentoo-user] 10 G eth (10000) on Gentoo
2011-09-01 1:43 ` Michael Mol
@ 2011-09-01 1:46 ` Adam Carter
2011-09-01 6:48 ` Adam Carter
0 siblings, 1 reply; 7+ messages in thread
From: Adam Carter @ 2011-09-01 1:46 UTC (permalink / raw
To: gentoo-user
> I've overheard IRC conversations that discussed multi-queue network
> cards in the context of multi-core systems. My educated guess, based
> on what you mention, is that each queue in the card would ping a
> different interrupt. Each interrupt might be handled by a different
> core, so you'd see a scaleable improvement there.
Nice - multi-queue was vapourware when i last looked. It would be
worth checking the driver source/doco to see what's there.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [gentoo-user] 10 G eth (10000) on Gentoo
2011-09-01 1:46 ` Adam Carter
@ 2011-09-01 6:48 ` Adam Carter
2011-09-02 15:29 ` [gentoo-user] " James
0 siblings, 1 reply; 7+ messages in thread
From: Adam Carter @ 2011-09-01 6:48 UTC (permalink / raw
To: gentoo-user
> Nice - multi-queue was vapourware when i last looked. It would be
> worth checking the driver source/doco to see what's there.
Looks like intel made multiqueue available in Nov. 2010. You can set
up to 16 queues.
http://downloadmirror.intel.com/14687/eng/README.txt
^ permalink raw reply [flat|nested] 7+ messages in thread
* [gentoo-user] Re: 10 G eth (10000) on Gentoo
2011-09-01 6:48 ` Adam Carter
@ 2011-09-02 15:29 ` James
2011-09-02 18:09 ` Michael Mol
0 siblings, 1 reply; 7+ messages in thread
From: James @ 2011-09-02 15:29 UTC (permalink / raw
To: gentoo-user
Adam Carter <adamcarter3 <at> gmail.com> writes:
> Looks like intel made multiqueue available in Nov. 2010. You can set
> up to 16 queues.
> http://downloadmirror.intel.com/14687/eng/README.txt
Thanks guys, for the discussion. Kinda reminds me back in
in the day of "jumbo frames" and the arguments as to if/when
jumbo frame support helps in routing.....
Any anecdotal comments as to how poor-bad 10 G eth crawls/fails
on a Microsoft server, would give me ammo to shoot down
the opposition...
James
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [gentoo-user] Re: 10 G eth (10000) on Gentoo
2011-09-02 15:29 ` [gentoo-user] " James
@ 2011-09-02 18:09 ` Michael Mol
0 siblings, 0 replies; 7+ messages in thread
From: Michael Mol @ 2011-09-02 18:09 UTC (permalink / raw
To: gentoo-user
[-- Attachment #1: Type: text/plain, Size: 722 bytes --]
Honestly, I'd just test. There are so many factors that the only way to know
the appropriate choice is to try...
On Sep 2, 2011 11:35 AM, "James" <wireless@tampabay.rr.com> wrote:
> Adam Carter <adamcarter3 <at> gmail.com> writes:
>
>
>> Looks like intel made multiqueue available in Nov. 2010. You can set
>> up to 16 queues.
>> http://downloadmirror.intel.com/14687/eng/README.txt
>
>
> Thanks guys, for the discussion. Kinda reminds me back in
> in the day of "jumbo frames" and the arguments as to if/when
> jumbo frame support helps in routing.....
>
> Any anecdotal comments as to how poor-bad 10 G eth crawls/fails
> on a Microsoft server, would give me ammo to shoot down
> the opposition...
>
>
> James
>
>
>
>
>
[-- Attachment #2: Type: text/html, Size: 1138 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2011-09-02 18:10 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-08-31 13:08 [gentoo-user] 10 G eth (10000) on Gentoo James
2011-08-31 23:58 ` Adam Carter
2011-09-01 1:43 ` Michael Mol
2011-09-01 1:46 ` Adam Carter
2011-09-01 6:48 ` Adam Carter
2011-09-02 15:29 ` [gentoo-user] " James
2011-09-02 18:09 ` Michael Mol
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox