Saturday, March 13, 2010

Blessay: FCoE Is JUST a Transition Technology

December 17, 2009 by Greg Ferro · 9 Comments 

It’s a com­mon fal­lacy of Storage Experts is that FCoE will “rule the world” and that Storage equip­ment has some spe­cial magical prop­er­ties that makes it dif­fer­ent from any other tech­no­logy. Chris Evans pos­ted the most com­mon fal­la­cies iSCSI is the home protocol

Let look at the most com­mon mis­takes that Storage people make.

The Four Common Falsehoods of Storage

Chris’s post out­lines them, and they are quite com­mon. Here is my list:

  • The Networking people don’t know what they are doing and aren’t reliable
  • FibreChannel net­works are a con­trolled environment
  • Networks use cheap and non-​​standard/​non-​​certified /​ open-​​standard equip­ment, Fibrechannel is expens­ive, cer­ti­fied and there­fore inher­ently better
  • FCoE har­mon­ises Storage and Networking at the Ethernet layer, and this is the ONLY pos­sible solu­tion that can work

OK. Let’s take these on. Lets start with an under­stand­ing of Good Enough versus Overly Perfect.

iSCSI is good enough, Customers buy good enough

The bulk of the Storage industry believes that iSCSI isn’t cap­able enough. Having been force fed this mar­ket­ing hype for the last ten years, it pretty well entrenched as mindset.

And yet, entire swathes of the Storage mar­ket are using iSCSI. Left Hand and EqualLogic, for example, have demon­strated that cus­tom­ers don’t always want FibreChannel. They don’t need stand­ards, and that good enough works.

Now, if you don’t believe that people will buy good enough then per­haps the most power­ful example of Good Enough in IT is Microsoft Windows. Even though it isn’t par­tic­u­larly good, or fast, or cheap cus­tom­ers con­tinue to buy and imple­ment because it’s good enough.

Networking is reli­able enough

It’s abso­lutely true in my opin­ion that today’s Networks are not ready, as a gen­eral concept, to carry stor­age data. There are good reas­ons for this.

  1. Data Network pro­to­cols are designed to be robust. Therefore the tol­er­ance of low cost net­work­ing is very high. Storage Protocols are not designed to be robust and there­fore tol­er­ance of the net­work­ing and open stor­ageis very low. The fault here is the FibreChannel stand­ards are quite poorly designed, or per­haps, nar­rowly designed with liim­ited flexibility.
  2. Data Networks spend­ing is about 30% the cost of an equi­val­ent Storage net­work. And if I had spent the same money, the Data Network would have the same per­form­ance cap­ab­il­it­ies. For example, in a Cisco envir­on­ment, I would imple­ment Nexus 7000 instead of Catalyst 6500, with all the fancy go-​​fast bits, at a price premium of 300%. Pointing out that Data Networks are not as reli­able as Storage net­works is the same as point­ing out the nose on your face.
  3. Storage net­work are really small. A few hun­dred ports, maybe a thou­sand or so in some cases. (and some more excep­tional cases). Data net­works have sev­eral orders of mag­nitude more ports, and vastly more com­plex per­form­ance require­ments. Scaling FC could be solved, but, why bother ?
  4. Data Networks have vastly more com­plex require­ments that a Storage net­work. A stor­age net­work with FC is focussed on a single task whereas a Data Network is soph­ist­ic­ated in it’s abil­ity to handle a wide range of vari­ably func­tioned loads, tasks, applic­a­tions and user needs. A stor­age net­work can only man­age a single thing.
  5. Data Networking people haven’t needed to deliver the cap­ab­il­it­ies and ser­vice levels that FC forced you to choose. But we are already train­ing and mov­ing to new designs that deliver the same out­come. Look at the Nexus 5000, which handles iSCSI, FC & FCoE equally as well as VoIP, SAP, Oracle and other mis­sion crit­ical services.

My point here is that Data Networks could be more reli­able and per­form to the same level as leg­acy FC Storage but his­tor­ic­ally, people have taken the “Good Enough” choice.

The good news is that Data Centre net­works will be designed and engin­eered to the same level as Storage net­works have been, and that will deliver the res­ults that are needed.

Standards, Certification and Unicorn Tears

Let remem­ber back to IBM main­frames and the after mar­ket addons that required IBM to cer­tify each and every products to be com­pli­ant with their hard­ware and soft­ware. It worked for a while, but cus­tom­ers rap­idly left Mainframes for new tech­no­lo­gies when they became viable.

Claims that Fibre Channel net­works are great because “they are expens­ive and approved” do not match with the real­ity of the mar­ket­place and his­tory. Customers will rap­idly move to other tech­no­lo­gies that show they work well enough. Because the Storage mar­ket is so early in the tech­no­logy cycle, the idea of ‘bless­ing products’ with val­id­a­tion or cer­ti­fy­ing that this product has been bathed in uni­corn tears, is just a passing phase. Eventually you will arrive truly open stand­ards that inter­op­er­ate prop­erly and cor­rectly without requir­ing spe­cial techniques.

How do I know this ? Because that’s what Data Networking did in the late 1980’s. Hark back to events such as ‘Interop’.

Note: IBM Mainframes still exist today for those use cases where they are excel­lent, and the future of stor­age will still have use cases where FC & FCoE make sense, but in the long run people will spend the bulk of their money elsewhere.

FCoE is not the ONLY choice

Read it and weep, IT ISN’T. There are sev­eral choices, includ­ing iSCSI, FibreChannel, Infiniband and new pro­to­cols that haven’t yet been developed. FCoE is the cur­rent choice, where pre­vi­ously the Storage industry had declared that sev­eral other protocols

Think of FICON, ESCON, iSCSI (in the early 2000’s this was THE stor­age pro­tocol), Infiniband and today it’s FibreChannel. Tomorrow’s choice is FCoE, and after that ?

After that it’s prob­ably iSCSI, because the 802.1 DCB stand­ards solves the net­work­ing prob­lems that stopped iSCSI from large scale accept­ance. Or pos­sibly a new pro­tocol that doesn’t use TCP and just encap­su­lates the data in an IP packet as a block thus remov­ing the pro­cessing load of the TCP header.

FCoE is a migra­tion technology

Because FCoE takes the exist­ing FC installed base, and neatly removes all the objec­tions of the stor­age industry to con­nect­ing to an Ethernet net­work, it sets up the Data Centre for a migra­tion away from FibreChannel. That, ulti­mately, is the primary pur­pose of FCoE. FCoE is not the end point, or the final pro­tocol for stor­age, it just the next step in a rap­idly matur­ing industry.

The Storage Industry appears to be stuck in the past. They have failed to adopt iSCSI in 2002/​2003, they failed to adopt Infiniband in 20052006. They kept banging away at mak­ing FibreChannel work, until it actu­ally did.

Networking isn’t exactly a saint here

Lets remem­ber that Networking has had the same pains and we’ve been through all this before. Token Ring and FDDI were also super­ior tech­no­lo­gies to Ethernet. I wouldn’t agree that Ethernet isn’t per­fect, far from it and many years of banging away at mak­ing it work have pro­duced a pro­tocol that works. But it’s flaws are many, it can­not read­ily scale and has a num­ber of inher­ent lim­it­a­tions that will take years for sil­icon and soft­ware to work around (RDMA over Ethernet is a case in point).

iSCSI works when cor­rectly designed

From a net­work­ing per­spect­ive, iSCSI has some quite spe­cific design factors that make it dif­fi­cult to per­form at high speed. However, it is much more robust and cap­able than FCoE and will work in a much wider vari­ety of Data Centre back­bone that FCoE ever will. This is what makes it espe­cially attract­ive to SME deployments.

iSCSI needs to be updated too

iSCSI will always be per­form­ance lim­ited since it uses the TCP pro­tocol as the trans­port mech­an­ism. At some point, someone is likely to develop a new ver­sion of iSCSI that run either over UDP (as NFS does) or dir­ectly encap­su­lated in IP as an IP-​​native protocol.

This type of change would improve the iSCSI per­form­ance match­ing FCoE. However, I think that devel­op­ment of this new pro­tocol won’t be pos­sible until the Cisco FCoE mar­ket­ing effort finally drives DCB into the data centre and actual deploy­ment. That is, any new devel­op­ments in stor­age will stalled until the cur­rent changes around FCoE either fail, or reach some level of acceptance.

FCoE is the only solution

It isn’t. Infiniband is sig­ni­fic­antly cheaper, and is much faster. Whether it can impact the mar­ket without a a bil­lion dol­lar mar­ket­ing pro­gram ‘a la Cisco’ remains to be seen.

And Storage over IP still hasn’t updated to mod­ern con­cepts. Choosing to move block stor­age data over TCP isn’t a great idea.

FCoE suits Cisco and Brocade

FCoE as a solu­tion is a mar­ket­ing move by Cisco that is now fol­lowed by Brocade. The net­work­ing industry has been mar­gin­al­ised with dumb switches and dumb net­work­ing for a num­ber of years. Cisco has con­stantly attemp­ted to develop products that pro­mote Smart Networks and failed thus leav­ing them in a com­mod­ity mar­ket. This is marked by the rise of HP ProCurve products and other mer­chant sil­icon plays such as Arista. Equally, it is also marked by the fail­ure of Nortel who were unable to compete.

Both Cisco and Brocade need a ‘smart net­work’ so that they can add value to their products and not be mar­gin­al­ised into a com­mod­ity mar­ket such as the Intel serv­ers mar­kets. Because FCoE requires the switches to have sig­ni­fic­ant par­ti­cip­a­tion in the stor­age layer, it’s a way of devel­op­ing another ‘Intelligent Network’.

What Networking still doesn’t have

The biggest miss­ing ele­ment of a mod­ern data net­work is decent, work­able man­age­ment tools. I con­stantly battle with the soft­ware that man­ages my net­work and no product today is up to the task of auto­ma­tion, ser­vice man­age­ment and oper­a­tional con­trol of a data net­work. But the focus on the Data Centre appears to be chan­ging this. FCoE man­dates the use of man­age­ment plat­forms that provide effect­ive tools. These tools are going to have a large impact on the net­work­ing industry.

Wrapping It Up

So FCoE is a trans­ition tech­no­logy that merges stor­age into an IP net­work. It may take a few years, but Storage over IP will even­tu­ally dom­in­ate. The data net­works of today are cap­able, but we need to develop new man­age­ment plat­forms and just a few new skills. The addi­tion of ser­i­ous cash to upscale our net­work products will see Data Networking deliver a ser­vice far bet­ter than a leg­acy FibreChannel sys­tem, one that is flex­ible, mul­ti­func­tional and bet­ter value for money.

So don’t pre-​​judge tech­no­lo­gies like iSCSI. Whether you like it or not, con­verged Storage and Data Networks will hap­pen. I don’t really mind that another ser­vice is com­ing onto my net­work, it’s not much dif­fer­ent from the ones that we already have in terms of reli­ab­il­ity and per­form­ance. A few changes here and there, and Data Networking will be up to the job.

Please rate this post:

  Why Rate Posts?
1 Star - It\\\'s Crud2 Stars - It\\\'s Tosh3 Stars - Something\\\'s missing4 Stars - Needs works5 Stars - Good Enough6 Stars - Good7 Stars - Excellent8 Stars - Brilliant9 Stars - Astonishing10 Stars - Awesomely Godlike? (9 votes, average: 7.11 out of 10)
Loading ... Loading ...

Comments

9 Responses to “Blessay: FCoE Is JUST a Transition Technology”
  1. John Dias says:

    I agree with much of what you have stated here, all cogent points but what’s the under­ly­ing premise? Are you try­ing to argue that iSCSI will ulti­mately be the de facto stor­age pro­tocol? I’ve never con­sidered that my net­work peers didn’t know what they were doing, but rather that they’re solving/​designing for dif­fer­ent solu­tions — this is why stor­age is, well, special.

    The prob­lem is seri­al­iz­ing SCSI, a very intol­er­ant pro­tocol. Taking some­thing that wasn’t really meant to be strung out about the data cen­ter floor and try­ing to make it per­form as if it’s still local to the host. Nasty. A kludge, really. But neces­sary due to back­wards com­pat­ib­il­ity requirements.

    FCoE, in my opin­ion, doesn’t move any­one closer to stor­age over IP. There’s no IP involved at all from what I can tell — simply a replace­ment of the layer 2 trans­port. Actually an encap­su­la­tion of one layer 2 trans­port within another.

    In the end, iSCSI and FCoE (and FC for that mat­ter) are use case solu­tions to the point of cent­ral­iz­ing stor­age for bet­ter util­iz­a­tion and over­com­ing the lim­it­a­tions of a pro­tocol designed for local attach. If one becomes pre­dom­in­ate it will be because the use case is more applic­able in the field. Not because it is inher­ently better.

    • Greg Ferro says:

      What both­ers me about this debate is the per­cep­tion that iSCSI is some­how inferior. The Storage industry is quite inbred in its think­ing. The fact that Storage con­tin­ues with the IDEA of using SCSI at all is aston­ish­ing. Is there no innov­a­tion to move bey­ond SCSI for I/​O sig­nalling ? Why not ?

      Why not use RDMA for stor­age, or a stream­ing pro­tocol such SCTP ?

      My guess ? Probably because the Storage busi­ness is still early phase and noth­ing is very mature.

      • Data stor­age industry is talk­ing about FC, iSCSI, and XY-​​variants over Ethernet. How would industry react to Ethernet with add-​​on ser­vice which allows TDM style com­mu­nic­a­tion and exact band­width par­ti­tion­ing using stand­ard IEEE802.3? so syn­chron­ous and asyn­chron­ous mes­sages can be exchanged via Ethernet. This would allow low-​​latency applic­a­tions to work in the com­plex net­work together with stand­ard LAN applications …

        • Greg Ferro says:

          That was done in the late 1990’s. Twice. One was called VG-​​Anylan and was inven­ted by HP, and the other was called Isochochronous Ethernet cre­ated by the IEEE.

          Didn’t get any­where though.

          • My impres­sion is VG-​​Anylan haven’t made it, because it was too far from IEEE802.3 switched Ethenret and designed to cover both TokenRing and FastEthernet. The prin­ciple they used is not really dif­fer­ent from exist­ing priority-​​based (e.g. round robin) schemes added to switched Ethernet few years later. This means it is still a best-​​effort Ethernet — and can­not acco­mod­ate iso­chron­ous or low-​​latency data streams.

            Isochronous Ethernet was not suc­cess­ful because it relies on ISDN and 10MBit Ethernet, so it was almost obsol­ete before the stand­ard completion.

            Existing con­ges­tion man­age­ment (CM) approaches do not solve needs of los­less and low-​​latency data trans­fers over Ethernet, and it is ques­tion­able if CM can do the job any time soon.

            The solu­tion could be in SAE AS6802 — this set of ser­vices (Layer 2) to IEEE802.3 allows los­less and low-​​latency data trans­fers in open or mixed applic­a­tion Ethernet net­works. And it works with 1GbE, 10GbE or 100GbE. This seems inter­est­ing for stor­age applic­a­tion, but it is just a guess …

  2. Interesting Perspective says:

    In my exper­i­ence I have never seen ISCSI work well for a wide amount of serv­ers push­ing tons of data, its a corner case for SMB. Speeds over gig are not the right thing for ISCSI, from what I have seen.

    FC is the right solu­tion, scale/​resiliency/​futures/​etc are best suited here.

    What FCOE offers is the abil­ity to ride the same wire and save space/​power/​cooling/​etc, that is the major hurtle today. Additionally ima­gine and envir­on­ment with 40g/​100g or higher net­works and hav­ing stor­age just ride the wire we use.

    The future is brighter with FCOE that ISCSI. Just have to accept it and move on. Juniper/​Brocade/​Cisco/​HP will have FCOE within the year so grab a hold and hang on.

    • Greg Ferro says:

      Your exper­i­ence must be lim­ited. In my exper­i­ence, I have seen entire data centre of hun­dreds of serv­ers (i.e. more than a thou­sand) includ­ing VM’s using iSCSI as the primary stor­age pro­tocol. Some FC was used for cer­tain corner cases. I have also seen the more tra­di­tional FC convention.

      The iSCSI price was about 40% that of the FC solu­tion and required a lot less main­ten­ance. Storage admins didn’t need to keep fid­dling with it.

      Now that iSCSI is becom­ing accep­ted, per­form­ance for iSCSI drivers is becom­ing import­ant. vSphere (VMware) has increased the iSCSI per­form­ance sig­ni­fic­antly with some tests show­ing a three times increase is through­put and speed.

  3. Greg Schulz says:

    Saying that stor­age folks are inbreed around SCSI would be akin to someone fool­ishly say­ing net­work­ing people are inbreed around IP, both of which are absurd asser­tions or per­cep­tions that IMHO are off base.

    iSCSI has its place, as does FCP, SAS as well as SCSI on IBA eg SRP (nat­ive) or iSCSI (mapped onto IP) either with or without RDMA for mov­ing blocks of stor­age, not to men­tion co-​​existence com­pli­ment­ary things such as FCIP for span­ning dis­tances when xWDM/​SONET/​SCH not viable.

    However keep in mind, if you don’t like SCSI, why iSCSI as it maps the SCSI com­mand set onto IP, sim­ilar to how FCP maps SCSI com­mand set onto Fibre Channel, or SAS using SCSI com­mand set on serial cabling, or tra­di­tional leg­acy par­al­lel SCSI (what many people think of as SCSI) or, SRP SCSI com­mand set nat­ively mapped without IP onto InfiniBand, or of course you can also map iSCSI onto IP onto InfiniBand how­ever that gets into a dif­fer­ent discussion.

    Im not clear if its SCSI or FC or block pro­to­cols that you are not a fan of which is fine, too each his own and that’s the cool thing about tech­no­logy options, you can use what you like to address dif­fer­ent issues and challenges.

    However, if you don’t like mov­ing blocks and want IP, then drop iSCSI and go straight to NFS/​CIFS or some other pro­tocol and trans­port. If on the other hand, you like iSCSI then you in some shape or form you like or at least util­ize the SCSI com­mand set.

    Show me some­thing that is not a trans­it­ory over some period of time and I will show you a truly revolu­tion­ary tech­no­logy. ;)

    Now to be fair, the key is what is the time span or timeline of the trans­ition?
    Is it months, years or decades?

    FC is a trans­ition tech­no­logy that star­ted with quarter speeds in the early 90s and will go into the next dec­ades with 16G per­haps even 32G as invest­ment pro­tec­tion for those on that path until they trans­ition to other tech­no­lo­gies. That makes for a multi-​​decade transition.

    Look at base Ethernet how that has transitioned from 10100 to 10÷100÷1000 to 100010000 to 100GbE and bey­ond not to men­tion dif­fer­ent medi­ums and func­tion­al­ity.
    iSCSI has its place as does SAS, FC, FCoE, IP, PCoIP, NFS/​CIFS, pNFS and so forth; they all play to dif­fer­ent value pro­pos­i­tions, price point, feature/​functionality and per­sonal pref­er­ences that with the excep­tion of SAS, a com­mon denom­in­ator trans­ition is towards Ethernet with the abil­ity for dif­fer­ent upper level pro­to­cols and trans­ports to co-​​exist.

    Sure things would be great if every­one could get to IP as the com­mon denom­in­ator how­ever that’s not going to hap­pen at least not over night, nor, prob­ably not over the course of a dec­ade, fur­ther out, perhaps.

    IMHO the trick is to have mul­tiple tools in your tech­no­logy tool box using what makes sense instead of only hav­ing a ham­mer thus everything looks like a nail, or, needs to addressed with the claw end.

    Cheers gs

  4. Jason Gurtz says:

    What about ATAoE? I’ve never under­stood why no one talks about it.

Speak Your Mind

Tell us what you're thinking...
and oh, if you want a pic to show with your comment, go get a gravatar!