It’s not a secret the that FibreChannel over Ethernet has caused a lot of change in the Storage industry. In short, the tedious world of slow moving and unexciting storage technologies has long been in need of change. The Cisco-led development of FibreChannel over Ethernet challenges a number of fundamental, and indeed religious tenets of FibreChannel that no loss in the network layer can ever be permitted.

The Challenge of Lossless Networking

Ordinary people have failed to comprehend the magnitude of money and technology wasted to deliver this promise. In the real world, there is no way to build a lossless network, the physical and logical criteria makes this impossible.

Of course, you can OVERENGINEER a network so that it looks lossless. And that’s what the Storage Industry did. In spades.

The elements of Loss

In a storage network loss can occur, in brief, at the following points:

• the software driver
• the network adapter
• the ingress buffer on the network switch port
• the egress buffer on the network switch port
• the module or silicon handling the switch port
• the backplane or module interconnect of the fibrechannel switch
• the module or silicon handling the switch port
• the egress buffer on the network switch port
• the ingress buffer on the network switch port
• the available bandwidth of the interconnect between switch
• and so until it arrives at the storage array
• the egress buffer on the network switch port
• the ingress buffer on the storage switch

and so on. At each of these points congestion can occur, for microseconds or, in the worst case, indefinitely. Addressing challenges such as Head of Line blocking, Ingress port drops for microburst performance are, at best, a work around.

Over-Over-Engineering

The FibreChannel industry managed to convince customers that lossless networking was a good idea. In the same way that the American Automotive Industry hoodwinked their customers into believing that American Cars were the best in the world. Which they weren’t and no one outside of America was buying them (which, naturally, those idiot customers continued to buy and are now congratulating themselves for their discovery on non-American cars. Golly, aren’t they clever). The same has been happening in the Ethernet Industry which has looked and laughed at the Storage industry. It’s so easy to declare that the network shall be lossless and hide the dirty little secret of making that happen.

The answer to the over-load problems is to build so much capacity, so much performance, so much buffers and spend so much money that the loss is highly unlikely to occur.

Thus the FibreChannel CNA was born with onboard CPUs and massive buffer memory. The SAN switching fabric that achieves an average of 1% utilisation in real life. Line cards with 48 ports and 48 silicon chips to cope with the rare instance that data actually occurs.

As SAN switches went from 12 ports to 100 ports the scaling became an enormous headache. And required even more massive overengineering, which was promptly dubbed “Director Class” (a classic case of lipstick on a pig). The “Director Class” FC switching had so much bandwidth, buffers and acceleration tricks that it was effectively impossible (except for the very largest of SAN networks) to use even a fraction of its capacity.

Ten Years Later, It’s not really LOSSLESS

Ten years later ad thousands of man-years of self interested brainwashing on the importance of lossless networking, lets make the statement. FC can withstand loss provided that it’s less than one packet in 100 million. That’s right, the FC standard states that loss IS ACCEPTABLE.

Oh dear. Oh FECKING dear.

Anyway, the vast bulk of the Storage Industry has no idea that the lossless networking is a sham to overcharge for overspecified equipment. They don’t appear to think about these things.

Enter: Lossless Ethernet

Of course, Ethernet is not lossless. But we can over-engineer that to buggery, and it’s real easy. You see, the FibreChannel PHY layer is a direct copy of the Gigabit Ethernet physical transport. The FC people couldn’t be bothered making their own so they nicked the IEEE proposals. So, the Ethernet folks lets take the over-engineered FC switch and make forward ethernet frames. Too fecking easy. Cisco knocked it up in a year or so.

But Ethernet isn’t lossless, I hear you cry. And cry you might, because Ethernet protocols are TOLERANT to loss. In fact, they are inherently better designed to accept loss and cope with it. That’s why mainframe used Ethernet for communications back in the day. But I digress.

The problem isn’t that Ethernet is lossless, it’s that it is shared with other data.

The importance of sharing

So now that we have our overpriced, overspecified uber-fancy Ethernet network that can transport FC, it’s the realisation that the networking people want to use that network for everything. TCP/IP for data, and Internet access, and desktop access.

May Crom defend us. HERESY.

We all know that Storage networks have special magical powers that make them different from other networks. I couldn’t possibly share my storage network with “common network traffic”. Well, guess what dipstick. You are screwed. The secret is out about your overpriced, overperforming and underused Storage network kit, and you lost the war.

Deal with it.

The new standards for Data Centre Bridging provide for lossless ethernet that exist in the real world.

Whither Fibre Channel over Token Ring

So for all those Storage Nut Jobs who can’t imagine their precious FibreChannel frames crossing an ethernet network, we are proposing the development of FibreChannel over Token Ring. That’s right, the second best networking protocol ever invented (after FDDI), offers everything you sad, attention deficit ridden, storage losers ever wanted in shared network. Deterministic delivery, over engineered cabling, layer 2 troubleshooting. We can even improve the FC protocol by isochronous transmission for serial clocking performance and guaranteed delivery.

The networking industry obviously wasn’t able to recognise the true value of this amazing network technology and now, it’s time has truly come. It’s for the ANSI standards organisation (yes, the same lunatics who lord it over the FibreChannel standard) to start discussion with IBM and demand the patents for Token Ring and get to work.

Last known Token Ring standards were developed to Gigabit performance, and it shouldn’t be too hard to dust them off and ramp them to 10Gigabit and more. The Madge technology exists even today Token Ring networks are still used (Crom knows I have them where I’m working). Surely, a vital testimony to this persistent and most amazing technology.

The EtherealMind View

It’s time for the Storage Industry people to open their minds, reject their fears of Ethernet networking and demand Token Ring networks. This will guarantee their miserly little jobs and position in the data centre. Instead of spending their days slotting more spinning rust into their oversized array, they can have a new technology to play with that will be all their own.

Because I don’t want you on MY network, I urge you to join me at the FibreChannel over Token Ring blog, and add your voice to chorus of storage people who want to defend their petty little existence in their ivory towers. Join in and the pretend that the world is not changing around you. Put down your backups, stop fiddling with your LUNs, and join with the movement.

FibreChannel over Token Ring. You know you want it.