[nylug-talk] SMB 10gigE (for Linux?)

[Alex Pilosov]

On Fri, 31 Aug 2007, jh wrote:

> Alex Pilosov wrote:
> > On Fri, 31 Aug 2007, jh wrote:
> > 
> >> Anyone using 10gigE with Linux at this point? And is any of the 10gigE
> >> gear remotely in the SMB price range?
> > Sure. 10GE is the new GE. You can buy ghetto 10G switches for ~2-3k$.
> 
> Really? It appears that D-Link's "ghetto" switches require an Uplink
> module, and then something called an XFP (some kind of transceiver?). So
> the hub may cost 2-3k, but the uplink module is $7k, and one XFP is
> ~2k... and I bet that you need those in pairs.


GSM7328S (24*GE + 4*10g open module) = 2k$
ax741 (converts the modular port to xfp) = 1k$

netgear xfp kinda pricy, but you can get elsewhere for 500-100$

> Am I missing something? Isn't there 10GB for copper yet?
cx4 (over infiniband wire) exists but is not available as xfp (only
xenpak) for complicated reasons. twisted pair copper 10g is not available
yet.

> > (netgear/dlink)
> > 
> >> I'd love to hear about performance, driver support on Linux, what's
> >> good, what's not, are jumbo packets required, etc.
> > what'd you expect basically.
> > 
> > you could saturate 10G if you work really hard at it.
> > 
> > please ask more detailed questions - what is your application etc.
> 
> Need to move (many, many small) files from certain machines to a central
> file server just about as fast as possible. Looks like GigE is
> bottlenecking here.
i doubt it, i'll bet OS or disk subsystem will get pegged before your
network is (many many small files).

> Other option is kicking up the bonding that I'm doing and getting a 48
> port GigE switch, but I'd bet that you get into declining returns with
> bonding rather quickly.
if you don't care about "peak stream bandwidth", it will scale just fine. 

in other words, if you have N clients moving files to 1 server, you will 
have close to linear scalability by doing M*GE bonding on the server. 

scalability will start to decrease as N/M ratio approaches 1. In which
case, just give each client a dedicated port.

-alex