Re: [gentoo-user] 10 G eth (10000) on Gentoo

[Michael Mol <mikemol@gmail.com>]

On Wed, Aug 31, 2011 at 7:58 PM, Adam Carter <adamcarter3@gmail.com> wrote:
>> I just might have an opportunity to setup a Gentoo
>> System using 10 G ethernet (fiber). The .39
>> kernel lists this hardware [1].
>>
>> Does anyone have any experience with this hardware?
>> Does it work? Did you make any bandwidth measurements?
>> What type of fiber (mulimode/singlemode) (ST/SC) did you
>> use?
>>
>> Any comments? Is the kernel a bottleneck or your application?
>
> Intel are active in linux kernel development and their linux drivers
> seem to be very good.
>
> When testing Intel copper gig interfaces on an intel firewall (HP
> DL380G5 8 core box), I was able to send a core to 100% with ~330Mb of
> small packets. The limiting factor appears to be packet rate, and the
> consequent processing of interrupts (1 irq to 1 core). I don't think
> that a 10Gig interface would pass any more than that due to similar
> limitations.
>
> Tweaking the e1000 driver options RxDescriptors, TxDescriptors and
> RxIntDelay pushed it up to ~350Mb. MSI was enabled so no interrupt
> sharing.
>
> So if you're running normal sized packets, you should be ok. Otherwise
> you way want to look at what irqbalance can do for you (I didn't try
> it at the time).
>
> Also don't forget stuff like Large Receive Offload in your kernel.

I've overheard IRC conversations that discussed multi-queue network
cards in the context of multi-core systems. My educated guess, based
on what you mention, is that each queue in the card would ping a
different interrupt. Each interrupt might be handled by a different
core, so you'd see a scaleable improvement there.


-- 
:wq