The TRILLing brain split

The TRILLing brain split

The split personality Cisco has exposed at Cisco Live 2010 is amazing: on one hand you have the Data Center team touting the benefits of Routing at Layer 2 (an oxymoron if Iíve ever seen one), on the other hand you have Russ White extolling the virtues of good layer-3 design in the CCDE training (the quote I like most: ìIt all meets at Layer 3 … thatís why CCDE is layer-3 centricî). If youíre confused, youíre not the only one, so letís try to analyze whatís going on.

The clash of interests. Letís be perfectly clear: the best design of your data center network is not the focus of vendorsí activities. Having a well-designed and stable network is definitely in your best interest, it might be in the interest of your external consultants or your system integration partner (assuming they are able to focus beyond quarterly results), but what the vendors want most is to sell you more boxes and/or services. Cisco would love you to upgrade from Catalyst 6500 to Nexus. Introducing a new technology that supposedly brings world peace to data center networks but only runs on a Nexus 7000 could be an enticing motivation for a forklift upgrade.

Virtualization and convergence. This is no news. Servers are getting virtualized. Storage is moving from embedded drives or SAN into converged LAN. LAN network and servers are getting tightly coupled. However, Cisco almost owned the LAN market, Brocade was big in the SAN market and HP is a major player in the server market. After the three components converge, someone is bound to lose big. Thatís why Cisco has launched UCS, Brocade is preaching that the Earth is flat and HP is trying to sell you high-end switches.

They need large-scale bridging. You donít need large-scale bridging in your Data Center. Your server team might think they need it to support inter-site vMotion, but even that can be solved (assuming itís a good idea to start with). Vendors need large-scale bridging if they want to sell you FCoE (remember: itís bridged) or if they want to sell the server managers a vision of seamless private clouds. We all know the drawbacks and complexities of spanning tree, so theyíre introducing a magic technology that will solve all those problems. It doesnít matter that it hasnít been tested, it doesnít matter that it requires new hardware (even Nexus 7000 requires TRILL-enabled blades).

Who are they talking to? In most organizations, the ìserver+storageî budget is bigger than the ìLANî budget (and the server team is bigger than the networking team). If you want to sell unified solutions, you have to sell them to the server managers. Their view of the network is exceedingly simple: it should be transparent. Now go and read the Scaling Data Center with FabricPath white paper and tell me whose sore spots itís addressing.

What can you do? If you have a feud with the server team, dump it. You will have to work very closely with them or they will go over your head and install something youíll be forced to support anyway. Try to understand their concerns and priorities. And, most importantly, start from the business perspective: what is it that your company is trying to solve and what are the true business requirements.

Last but not least, if you need a comprehensive overview of data center, server and storage technologies, you might consider registering for my Data Center 3.0 for Networking Engineers webinar.

  • Omar Sultan


    So part of the point is that customers have a choice for data center network design without having to replace hardware until they are ready. In the long-term, we expect customers to move from Catalyst to Nexus in the data center, but at their own pace.

    As far as architectural directions, our perspective, either 2 or 3 tier are valid approaches–it should depend on the specific requirements in a particular customers’ data center. As you point out, some applications such as VMware clusters, federated servers, etc, will benefit from FabricPath, while others will quite happily exist in a traditional 3 tier architecture. I would be surprised if customers wholesale replace 3-tier with 2-tier without their being any tangible benefits to balance out the disruption–rather, I think customers will deploy in a more targeted manner and I think most data centers will end up being a blend of traditional deployments, traditional deployments + vPC and FabricPath.

    I think the salient point is that we give customers the choice and, more importantly, the choice to easily migrate between the two approaches as application requirements change. A single N7K chassis will happily, concurrently support both 3-tier and 2-tier deployments. The new F-series I/O modules allow customers deploy 2-tier where it makes sense and stick with tried-and-true 3-tier where it makes sense.


    Omar Sultan

  • Ivan Pepelnjak


    Thank you for your feedback. Iím encouraged to see such a balanced and neutral position and Iím positive your account teams are perfectly synchronized with your thinking. Then again, this position might be more credible if the marketing people writing DC white papers would stop bashing what the other parts of your company are doing very well (and have been doing well for the last 20+ years).

    Youíre absolutely correct that the DC architecture is in the end the customerís decision and that ìthe customer is always rightî. I know your biggest customers know exactly what they want and how they want to get there. The majority of them, however, look up to you for guidance and advice. Giving them totally opposing messages coming from different teams within the same company, even at the same time, will only confuse the customers, scare some of them away (potentially even toward other vendors) and result in lower success rate for all of us.

    Ivan Pepelnjak

  • Peter Welcher

    I pretty much agree. I’m very much concerned about designs in the next couple of years hitting the “sweet spot” for VLAN scaling as the new technologies mature. Betting the whole of a large data center on immature TRILL for example seems like a bad idea — and might well be a bad idea even with mature TRILL. (Perfect example of the “beer principle” — too much of a good thing leads to a headache. Or perhaps it’s a case of “how close to the edge of the cliff is still safe”?)I’d hate to see vendor marketing drive sites to risky designs.

    The one thing I’ve noticed that I’d add to the mix is that the key vendors also have virtual switch plans. If the N7K could somehow drive a virtual set of say 2000 to 4000 ports (substitute HP or Brocade here if you want), then you could have a much larger L2 pod of servers without risking the entire data center. I note in passing that the N5K FEX technology already does that at a smaller scale.