public inbox for gentoo-user@lists.gentoo.org
 help / color / mirror / Atom feed
* [gentoo-user] 10 G eth (10000) on Gentoo
@ 2011-08-31 13:08 James
  2011-08-31 23:58 ` Adam Carter
  0 siblings, 1 reply; 7+ messages in thread
From: James @ 2011-08-31 13:08 UTC (permalink / raw
  To: gentoo-user

Hello,

I just might have an opportunity to setup a Gentoo 
System using 10 G ethernet (fiber). The .39
kernel lists this hardware [1].

Does anyone have any experience with this hardware?
Does it work? Did you make any bandwidth measurements?
What type of fiber (mulimode/singlemode) (ST/SC) did you
use?

Any comments? Is the kernel a bottleneck or your application?

  --- Ethernet (10000 Mbit)                                            
  Chelsio 10Gb Ethernet support                                  
  Chelsio Communications T3 10Gb Ethernet support                
  Chelsio Communications T4 Ethernet support                    
  Chelsio Communications T4 Virtual Function Ethernet support   
  Cisco VIC Ethernet NIC Support                                
  Intel(R) 10GbE PCI Express adapters support                   
  Intel(R) 82599 Virtual Function Ethernet support               
  Intel(R) PRO/10GbE support                                    
  Exar Xframe 10Gb Ethernet Adapter                           
  Exar X3100 Series 10GbE PCIe Server Adapter                    
  Myricom Myri-10G Ethernet support                             
  NetXen Multi port (1/10) Gigabit Ethernet NIC                 
  Sun Neptune 10Gbit Ethernet support                    
  Mellanox Technologies 10Gbit Ethernet support               
  Tehuti Networks 10G Ethernet                                 
  Broadcom NetXtremeII 10Gb support                             
  QLOGIC QLCNIC 1/10Gb Converged Ethernet NIC Support          
  QLogic QLGE 10Gb Ethernet Driver Support                      
  Brocade 1010/1020 10Gb Ethernet Driver support                
  Solarflare Solarstorm SFC4000/SFC9000-family support         
  ServerEngines' 10Gbps NIC - BladeEngine       




^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [gentoo-user] 10 G eth (10000) on Gentoo
  2011-08-31 13:08 [gentoo-user] 10 G eth (10000) on Gentoo James
@ 2011-08-31 23:58 ` Adam Carter
  2011-09-01  1:43   ` Michael Mol
  0 siblings, 1 reply; 7+ messages in thread
From: Adam Carter @ 2011-08-31 23:58 UTC (permalink / raw
  To: gentoo-user

> I just might have an opportunity to setup a Gentoo
> System using 10 G ethernet (fiber). The .39
> kernel lists this hardware [1].
>
> Does anyone have any experience with this hardware?
> Does it work? Did you make any bandwidth measurements?
> What type of fiber (mulimode/singlemode) (ST/SC) did you
> use?
>
> Any comments? Is the kernel a bottleneck or your application?

Intel are active in linux kernel development and their linux drivers
seem to be very good.

When testing Intel copper gig interfaces on an intel firewall (HP
DL380G5 8 core box), I was able to send a core to 100% with ~330Mb of
small packets. The limiting factor appears to be packet rate, and the
consequent processing of interrupts (1 irq to 1 core). I don't think
that a 10Gig interface would pass any more than that due to similar
limitations.

Tweaking the e1000 driver options RxDescriptors, TxDescriptors and
RxIntDelay pushed it up to ~350Mb. MSI was enabled so no interrupt
sharing.

So if you're running normal sized packets, you should be ok. Otherwise
you way want to look at what irqbalance can do for you (I didn't try
it at the time).

Also don't forget stuff like Large Receive Offload in your kernel.



^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [gentoo-user] 10 G eth (10000) on Gentoo
  2011-08-31 23:58 ` Adam Carter
@ 2011-09-01  1:43   ` Michael Mol
  2011-09-01  1:46     ` Adam Carter
  0 siblings, 1 reply; 7+ messages in thread
From: Michael Mol @ 2011-09-01  1:43 UTC (permalink / raw
  To: gentoo-user

On Wed, Aug 31, 2011 at 7:58 PM, Adam Carter <adamcarter3@gmail.com> wrote:
>> I just might have an opportunity to setup a Gentoo
>> System using 10 G ethernet (fiber). The .39
>> kernel lists this hardware [1].
>>
>> Does anyone have any experience with this hardware?
>> Does it work? Did you make any bandwidth measurements?
>> What type of fiber (mulimode/singlemode) (ST/SC) did you
>> use?
>>
>> Any comments? Is the kernel a bottleneck or your application?
>
> Intel are active in linux kernel development and their linux drivers
> seem to be very good.
>
> When testing Intel copper gig interfaces on an intel firewall (HP
> DL380G5 8 core box), I was able to send a core to 100% with ~330Mb of
> small packets. The limiting factor appears to be packet rate, and the
> consequent processing of interrupts (1 irq to 1 core). I don't think
> that a 10Gig interface would pass any more than that due to similar
> limitations.
>
> Tweaking the e1000 driver options RxDescriptors, TxDescriptors and
> RxIntDelay pushed it up to ~350Mb. MSI was enabled so no interrupt
> sharing.
>
> So if you're running normal sized packets, you should be ok. Otherwise
> you way want to look at what irqbalance can do for you (I didn't try
> it at the time).
>
> Also don't forget stuff like Large Receive Offload in your kernel.

I've overheard IRC conversations that discussed multi-queue network
cards in the context of multi-core systems. My educated guess, based
on what you mention, is that each queue in the card would ping a
different interrupt. Each interrupt might be handled by a different
core, so you'd see a scaleable improvement there.


-- 
:wq



^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [gentoo-user] 10 G eth (10000) on Gentoo
  2011-09-01  1:43   ` Michael Mol
@ 2011-09-01  1:46     ` Adam Carter
  2011-09-01  6:48       ` Adam Carter
  0 siblings, 1 reply; 7+ messages in thread
From: Adam Carter @ 2011-09-01  1:46 UTC (permalink / raw
  To: gentoo-user

> I've overheard IRC conversations that discussed multi-queue network
> cards in the context of multi-core systems. My educated guess, based
> on what you mention, is that each queue in the card would ping a
> different interrupt. Each interrupt might be handled by a different
> core, so you'd see a scaleable improvement there.

Nice - multi-queue was vapourware when i last looked. It would be
worth checking the driver source/doco to see what's there.



^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [gentoo-user] 10 G eth (10000) on Gentoo
  2011-09-01  1:46     ` Adam Carter
@ 2011-09-01  6:48       ` Adam Carter
  2011-09-02 15:29         ` [gentoo-user] " James
  0 siblings, 1 reply; 7+ messages in thread
From: Adam Carter @ 2011-09-01  6:48 UTC (permalink / raw
  To: gentoo-user

> Nice - multi-queue was vapourware when i last looked. It would be
> worth checking the driver source/doco to see what's there.

Looks like intel made multiqueue available in Nov. 2010. You can set
up to 16 queues.
http://downloadmirror.intel.com/14687/eng/README.txt



^ permalink raw reply	[flat|nested] 7+ messages in thread

* [gentoo-user] Re: 10 G eth (10000) on Gentoo
  2011-09-01  6:48       ` Adam Carter
@ 2011-09-02 15:29         ` James
  2011-09-02 18:09           ` Michael Mol
  0 siblings, 1 reply; 7+ messages in thread
From: James @ 2011-09-02 15:29 UTC (permalink / raw
  To: gentoo-user

Adam Carter <adamcarter3 <at> gmail.com> writes:


> Looks like intel made multiqueue available in Nov. 2010. You can set
> up to 16 queues.
> http://downloadmirror.intel.com/14687/eng/README.txt


Thanks guys, for the discussion. Kinda reminds me back in
in the day of "jumbo frames" and the arguments as to if/when
jumbo frame support helps in routing.....

Any anecdotal comments as to how poor-bad 10 G eth crawls/fails
on a Microsoft server, would give me ammo to shoot down
the opposition...


James







^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [gentoo-user] Re: 10 G eth (10000) on Gentoo
  2011-09-02 15:29         ` [gentoo-user] " James
@ 2011-09-02 18:09           ` Michael Mol
  0 siblings, 0 replies; 7+ messages in thread
From: Michael Mol @ 2011-09-02 18:09 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: text/plain, Size: 722 bytes --]

Honestly, I'd just test. There are so many factors that the only way to know
the appropriate choice is to try...
On Sep 2, 2011 11:35 AM, "James" <wireless@tampabay.rr.com> wrote:
> Adam Carter <adamcarter3 <at> gmail.com> writes:
>
>
>> Looks like intel made multiqueue available in Nov. 2010. You can set
>> up to 16 queues.
>> http://downloadmirror.intel.com/14687/eng/README.txt
>
>
> Thanks guys, for the discussion. Kinda reminds me back in
> in the day of "jumbo frames" and the arguments as to if/when
> jumbo frame support helps in routing.....
>
> Any anecdotal comments as to how poor-bad 10 G eth crawls/fails
> on a Microsoft server, would give me ammo to shoot down
> the opposition...
>
>
> James
>
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 1138 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2011-09-02 18:10 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-08-31 13:08 [gentoo-user] 10 G eth (10000) on Gentoo James
2011-08-31 23:58 ` Adam Carter
2011-09-01  1:43   ` Michael Mol
2011-09-01  1:46     ` Adam Carter
2011-09-01  6:48       ` Adam Carter
2011-09-02 15:29         ` [gentoo-user] " James
2011-09-02 18:09           ` Michael Mol

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox