FCoE Isn’t a Replacement for Infiniband, It’s a Cheaper Copy That Customers Will Buy

Infiniband is some­thing of a for­got­ten pro­tocol these days but many of the mar­ket­ing fea­tures of FCoE are dir­ectly derived from Infiniband con­cepts and architecture.

Paradigms of Data Center Virtualization

For example, this page is taken from the “Paradigms of Data Center Virtualization” by Cisco at Networkers 2005. If you are fol­low­ing Cisco UCS then it should be very famil­iar, per­haps sur­pris­ing the the Unified Fabric idea is not new but either Infiniband or 10GB was going to be the net­work­ing fabric.

Unified Fabric Basics.(Click for a full size image)

If you are reas­on­ably new to Data Centres then you may not remem­ber that Cisco has a mod­er­ately suc­cess­ful Infiniband product, and in 2005, it was the centrepiece of Cisco’s mar­ket­ing strategy for a Unified Fabric for con­nectiv­ity in the Data Centre. As an aggreg­ator of con­nec­tions from the server down to an Infiniband back­bone it was the Unified Fabric that was going to unite the Data Centre for Grid Computing (the pre­cursor of what is now virtualization).

Caption Text.(Click for a full size image)

Unified Fabric Server Clusters

And you can see a Case Study of this using a pure Infiniband back­bone for a large scale server cluster in this slide (from the same present­a­tion)

Caption Text.(Click for a full size image)

So what changed ?

So why didn’t Cisco con­tinue with Infiniband ? I think it’s worth look­ing back and think­ing how we got FCoE instead and look­ing for les­sons to be learned. I can only spec­u­late on what happened.

Recap on Key Infiniband Features

1. InfiniBand is a high speed, low latency tech­no­logy used to inter­con­nect serv­ers, stor­age and net­works within the datacenter
2. Standards Based – InfiniBand Trade Association http://​www​.infin​i​bandta​.org and has been work­ing suc­cess­fully for more than ten years.
3. Scalable Interconnect speeds in mul­tiple of 2.5Gb/s, products are cur­rently ship­ping at 40Gb/​s and 120Gb/​s products have been announced.
4. Low latency net­work­ing with delays around 20 micro­seconds (end-​​to-​​end) is one thou­sand times less than 20 mil­li­seconds for a data centre eth­er­net network.

I also find it inter­est­ing that the sil­icon from QLogic and Mellanox has rap­idly been repur­posed for FCoE HBA’s. There must be strong sim­il­ar­it­ies is these products for this to happen.

Ethernet is easy to sell

People will buy and use what they know. In that sense, faster eth­er­net or ‘newer’ eth­er­net is an easier decision. I assume that offer­ing cus­tom­ers an easy choice, what they per­ceive as a simple upgrade path, is a much easier sale, than one that needs to teach cus­tom­ers new tech­no­lo­gies. People tend to lazi­ness, and this is a prob­able cause.

Cheap always wins

Ethernet is cheap and not very good at a lot of things but, there is a lot of it about and plenty of exist­ing tech­no­logy. And don’t for­get skills, lots of people under­stand Ethernet and that has a price tag.

And his­tory shows that cheap always wins.

Maybe Cisco wasn’t dominant

Cisco has a policy of being num­ber one or two in any mar­ket. If they can’t do that, they will walk away or do some­thing to kill that mar­ket. For example, in the early days of IPSec VPN, Cisco didn’t have a good story. They bought two or three com­pan­ies before con­ver­ging the code in the Cisco PIX (and later the ASA). They grew to num­ber one by char­ging no license fee for their solu­tion thus block­ing all their com­pet­it­ors from mak­ing a profit. They same sys­tem appears to be hap­pen­ing today with SSL VPN which is now a small license fee for most customers.

If Cisco couldn’t dom­in­ate the Infiniband mar­ket, then per­haps Cisco spun out the Nuova Systems (the com­pany that built FCoE and the Nexus fam­ily sil­icon as a star­tup) to give it a shot at col­lapsing that market.

FCoE cheaper but not better ?

If you spend some time with Infiniband, you real­ise that FCoE isn’t spe­cific­ally a world beat­ing tech­no­logy innov­a­tion. FCoE is the VHS to Infiniband Betamax. And you can see that many of the ideas the FCoE Unified Fabric pro­motes are the same as those pro­moted in the past. In that sense, FCoE isn’t new, just a rehash of old ideas mashed onto exist­ing technologies.

Now, I’m sure that FCoE and Cisco UCS strategy is work­ing just fine for cus­tom­ers and it’s going to do well in the mar­ket. Given that Cisco has spent any­thing up to US$1billion buy­ing, man­u­fac­tur­ing and mar­ket­ing the product, it’s a guar­an­teed suc­cess. But its worth look­ing back on older tech­no­lo­gies with regret and cyn­ic­ally view­ing these new devel­op­ments to determ­ine if they are really the best solu­tions for our networks.

So far, I’m not so sure. We could have had bet­ter, but FCoE will prob­ably work just fine for a while until we need to scale and get faster than Ethernet can ever go. Maybe Ethernet will work for us in the future but Infiniband will still be there wait­ing for us to reuse it. Just like all the other tech­no­lo­gies that we keep reusing.

Please rate this post:

(3 votes, average: 9.00 out of 10)

 Loading ...