My youngest daughter used to have trouble with her animals. Whenever she saw a giraffe, she would say…

In designing the network for Mozilla’s new data center build (http://blog.mozilla.com/mrz/2010/01/04/mozillas-new-phoenix-data-center/) we opted for all 10GE (fiber) at the core and as downlinks to the access layers (Cisco’s 3120X CBS). We felt it simplified wiring and added some future-proof to the design and except for the optics themselves, wasn’t significantly more expensive (and what difference there was erased itself with the reduction of cross-connects needed for same capacity).

I also changed our IP transit requirements – at a minimum I require anyone wanting to sell my IP transit to deliver 2x10GE handoffs and only bought hardware to support 10GE handoffs (Mozilla hardly pushes more than 3-4Gbps across a number of peers). Turns out this too is cost effective now.

We looked at Cisco but only focused on the C6500 – Nexus didn’t make sense for the price. The core in Phoenix is all Juniper. Had I really wanted to shave costs, I would have looked at Force10.

So to your point, 10GE is here for Mozilla.

It’s too early to predict the end of the copper, but I can see some of the signs.

We have been using 10Gbe (fiber) at the core and core and distribution layers for about a year now. We even went as far as to run CAT6A to the desktop because the historical network refresh cycles (particularly concerning cabling) of our organization have been extremly long.

It seems that 10Gbe over copper is even less mature than 10Gbe over fiber. We found that our network vendor (HP) has a cable length limitation of 15 meters when using CX4 cables. This would fit our current facility, however with expansion to a larger facility distance concerns pushed us to stick with fiber connections for the time being.

A biggish storage network might have a couple of hundred ports. Compare this with a Data Centre that has thousands, or a campus that has tens of thousands and you should have some idea of scale.

I agree the iSCSI will feed heavily on the perception of improved throughput of 10GbE, however, it’s not a huge driver because of the cost of 10GbE switches and NICs.

And yes, Infiniband is the REAL deal. Current 10Gb virtualisation technologies will take two or three years to catch up with Infiniband of today. But convincing people to buy Infiniband…. that’s hard.

10Gb iSCSI is a formidable option, particularly if the 10Gb switch ports are backward compatible to existing adapter speeds. Of course, even if that’s the case, the infiniband people still scoff. “Only 10Gb/s? Here’s a nickel. Buy yourself a REAL storage network”.