Network Fabric:TRILL for Server and Network People. Welcome RBridges

TRILL is a key network technology for enabling Cloud Computing by allowing for better migrations of VM’s, and better utilisation of the network switching fabric and much improved stability of the Data Centre Server Fabric.

TRILL(Transparent Interconnection of Lots of Links)

TRILL is a proposed technology that is intended for use in the data centre. It is a fundamental network technology enabler for Cloud Computing so that:

• increase in switching density,
• better path utilisation
• underlying support for highly mobile servers in a virtualised server environment
• faster convergence, at power on and in response to failures.

The support for highly mobile virtual servers within VLANs is big feature.

From the IETF Charter for TRILL


The design should have the following properties:


• Minimal or no configuration required
• Load-splitting among multiple paths
• Routing loop mitigation (possibly through a TTL field)
• Support of multiple points of attachment
• Support for broadcast and multicast
• No significant service delay after attachment
• No less secure than existing bridged solutions

It’s also very interesting that TRILL is an Ethernet standard being progressed through the IETF (and not the IEEE), and is led by the Radia Perlman, the creator of Spanning Tree.

Why have TRILL at all ?

In current Layer 2 networks we use Spanning Tree (in all its forms) to ensure that there is only a single path across the network. At the same time, we deliberately design one or more redundant paths for failover in the event that the primary path fails. This redundant paths are blocked and that there are many paths in the network that are not used to carry frames thus:

• Bandwidth is potentially underutilised, especially when more than two paths exist.
• the shortest optimal (and thus fastest) path through the Layer 2 network is not practically possible thus most path are inefficient
• because it’s not proprietary
• because 802.1aq isn’t your cup of tea.

Current State of Spanning Tree

There are several forms of Spanning Tree used today:

• Common Spanning Tree (CST)
• Per VLAN Spanning Tree (PVST/PVST+)
• Rapid Spanning Tree
• Multiple Instance Spanning Tree

All types of spanning tree have one fundamental idea: that a single tree, representing a single path through the switching network is established for a given VLAN and all alternate paths are blocked. Each of the above offers improvements in speed of convergence using various features, however, even in the best cases, it takes seconds for paths to converge on the final solution.

Suboptimal path

Consider the following server/switch connections where two servers areconnected to a switch (with one or more ports if using LACP). You can see that there are several possible paths between the two servers.

Typically, a switch backbone for a given VLAN will be configured to force the tree to be at the core of the network. As a result the path between the servers will look something like this (if the top left switch is configured as the primary root switch):

Now this might seem OK until you realise that the entire data centre network looks more like this:

Ok, so this isn’t absolutely perfect, but you should get the idea. There are very good operational reasons to build a data centre this way. Very large organisations with substantial resources will do it smarter and better using PVST+ and port/path costs to optimise and try to balance some load. in real life, this is standard Enterprise switching backbone for data centres. .

That link between the core switches need to be big, and many of the of the uplinks are also large. This has been a big driver for 10Gb Ethernet in the early stages.

But when you think about it, what we would really like to achieve is the following shortest path through the network:

As you can see, this means that core is now more lightly loaded and that, scaling this up, the core is less likely to be bottlenecked around Core Switch 1.

A common scenario for this is when using top-of-rack Ethernet switches and two servers are in adjacent racks, yet the Ethernet frames travel via the core, adding a small amount of latency.

Note also that a failure event at Core 1, will have less overall impact on the entire network.

VM migration / mobility

Now consider what happens when all of these servers are virtual servers and then you use a vserver mobility tool to move ServerA across the backbone when using spanning tree:

The problem here is that the path is still shared by all the other servers. The load across the backbone may drastically altered the available bandwdith.

In this case, there is less chance of impact since the frames are now taking an alternate path. Your backbone is much more likely to handle the changes and surges in Internet traffic, and more generally you would have better overall usage of the available infrastructure.

Of course, you can’t be certain that this path has available bandwidth but there are other standards being developed to help solve this problem.

Multiple Paths at Layer 2

The other improvement that TRILL make available is multiple paths at layer 2. Consider the following model:

This is a major breakthrough compared to Spanning Tree and will drastically improve the available bandwidth because many links can be used to the destination.

Although it will make packet capture in the network much more difficult since predicting the path through the Layer2 network is quite complicated.

Terminology:Frames and Packets

The OSI model has defined an abstraction of the communication protocols that uses seven layers and Layer 3 = Network Layer, Layer 2 = Data Link layer. Typically, Network Engineers consider the term Packets to refer to Layer 3 data encapsulations, and Frames refer to Layer 2 encapsulations. In the past, FDDI, Token Ring and Ethernet were all Frames, and AppleTalk, IPX and IP are all Packets. For many people today, only Ethernet and IP exist and it is easy to confuse the terms.

TRILL – a fast overview on how it works

TRILL uses a concept of a Routing Bridge, known as an RBridge, running IS-IS routing protocol((Older engineers will recall Bridging Routers or BRouters, I find it amusing that this is an obvious return to bridging and stepping away from routing)) . IS-IS is a link state routing protocol that is mostly used by Global Service Providers to carry IP routes within their networks (and then use BGP between Service Providers) but is actually protocol independent. IS-IS does not use IP to establish neighbour relationships, its uses OSI protocols which includes CLNS and PDU’s to perform the neighbour and protocol exchanges. This means that IS-IS works for IPv4 & IPv6 and can equally be used for other protocols such as is proposed by TRILL.

TRILL uses IS-IS to carry routing information about MAC Addresses devices connected to VLANs and to build a shorted path tree for each MAC address in the VLAN.

TRILL intends to maintain compatibility with existing Spanning Tree implementation and can co-exist and be progressively migrated.

Thoughts and Personal Opinions

One of the most fascinating things about TRILL is that it is conceptually identical to IEEE 802.1aq. Why would someone create a competing standard ? Because the IEEE is slow, ponderous and bad company. Many times we have seen the IEEE take far too long to develop standards, fail to resolve conflicts between competing interests.

Radia Perlman is the creator of Spanning Tree who currently works for SUN, and appears to be the lead author of TRILL. Other authors include Dinesh G. Dutt from Cisco, Silvano Gai from Nuova (now Cisco Nexus products). These people are at the core of the data centre fabric development and their employers Sun and Cisco are promoting their Cloud Computing credentials.

Radia Perlman has also criticised the IEEE in her outstanding and seminal textbook on bridging – Interconnections:Bridges, Routers, Switches, and Internetworking Protocols – Radia Perlman – Addison-Wesley 2000. (Do yourself a favour and buy this book, its brilliant!).

I can’t get access to the current drafts of the IEEE since they are not open to the public. However, the most recent draft 1.5 was released on 2008/12/18 and appears to have been stumbling along for more than two years. Reviewing the available material suggests that the IEEE is not reaching any agreement at this stage.

Conclusion

I hope I have done this topic justice. I have only the IETF document to read and some of my own experiences. If anyone has more information or can’t point out my mistakes, please leave a note in the comments and I will respond or you can contact me using my contact page http://etherealmind.com/contact. I look forward to hearing back.

References

Problem & Applicability Statement

http://www.ietf.org/interet-drafts/draft-ietf-trill-prob-06.txt

Please rate this post:

(2 votes, average: 7.50 out of 10)

 Loading ...