The Supervisor 720 contains a 720 gigabit per second crossbar switching fabric that providing for multiple conflict-free (or non blocking) paths between switching modules. At the back of the module you can actually see the chip that is the switching fabric:
Using the “supa klever” form of marketing math, a switching fabric is actually 360 GB/s but because it goes in at 360Gb/s and out at 360Gb/s and therefore you could count it twice. So that is seven hundred and twenty gigabits. That’s the number.
In fact, the switching fabric is eighteen inputs of 20Gb/s per input. Of these, the backplane in your Catalyst 6500 will decide how many are presented to your line cards. The C6503, C6506 and C6509 will present dual fabric connections while the C6513 is dependent on slot.
Per Slot bandwidth
Each slot in the backplane of the C6509-E chassis has two 20Gb/s backplane connections. There are nine slots. Nine slots at 40Gb/s is 360GB/s.
and the Switch Fabric connections look like this:
Note that backplane slots on a 6513 do not support dual 20Gb/s on all slots. Because the switching fabric on the Supervisor has eighteen 20Gb/s inputs, you must layout the inputs differently.
Therefore the slots at the top of 6513 are only 20GB/s and not 40Gb/s. This is what makes the C6513 signficantly different from the C6509 chassis. The 6513 was a popular choice when low performance line cards were used, but today, high density gigabit ethernet is common and much increased bandwidth of servers and desktops †means the C6513 is not commonly used, especially for data centres.
The Switch Fabric connections look like this:
Not All Modules are created equal
Consider the following modules that are commonly used today (2009):
? WS-X6724-SFP: a 24 port Gigabit Ethernet SFP based line card supporting a single 20 Gbps fabric channel to the Supervisor Engine 720 crossbar switch fabric. Also supports an optional Distributed Forwarding Card 3a (DFC3a, DFC3B or DFC3BXL)
? WS-X6748-SFP: a 48 port 1000Mb SFP based line card supporting 2 x 20-Gbps fabric channels to the Supervisor Engine 720 crossbar switch fabric. Also supports an optional Distributed Forwarding Card 3 (DFC3a, DFC3B or DFC3BXL)
? WS-X6704-10GE: a 4 port 10 Gigabit Ethernet Xenpak based line card supporting 2 x 20 Gbps fabric channels to the Supervisor Engine 720 crossbar switch fabric. Also supports an optional Distributed Forwarding Card 3a (DFC3a, DFC3B or DFC3BXL)
? WS-X6708-10GE: an 8 port 10 Gigabit Ethernet X2 optic based line card supporting 2 x 20 Gbps fabric channels to the Supervisor Engine 720 crossbar switch fabric with integrated Distributed Forwarding Card 3CXL.Not this card has 80 Gb/s of input (in one direction, 160 Gb/s in two directions) and could easily oversubscribe the backplane connection and switch fabric connection on those inputs.
An oversubscribed card means that packets may be delayed or dropped under heavy loads.
Cross Bar Connector on the module
You can easily tell if the blade has some form of connection to the Cross Bar fabric by looking at the connectors on the back of the cards. If the Cross Bar Connector is present, then the module has some form of the connection to the fabric.
Shared Bus Connector
There is also a Shared Bus on the C6500 that operates at 32Gb/s (yes, 16Gb/s in both directions).
This is what was used in the early days of the chassis with the first generation C61xx and C63xx modules which could easily oversubscribe the backplane connection. Most new installation are completely fabric switched (excepting for IP Telephony installations who use the oversubscribed model and like the QoS features of the C6500 and the high power density of up to 8500W in a single chassis).
This post is not a comprehensive look at the modules and supervisor architecture. This is covered a number of documents but check the White paper link just below this which has a lot more information and is a complete reference to the architecture of the C6500.
Observations and Comments
The reason for this article is to outline some of the issues that a Data Centre designer needs to understand. The architecture of the switch, and same logic applies to storage switches, can impact the design of the network. The decision on whether to purchase a WS-X6704-10GE or WS-X6708-10GE can be impacted by the predicted traffic load.
There are other factors that might affect a design, such as the Multicast performance or the QoS features, but this article points out some of the first factors for thinking about whether C6500 is the right switch. I hope you realise that knowing something about the guts of your switch, and how the insides look, is important to getting your network right.
1) Some images and concepts are drawn from the CISCO CATALYST 6500 SWITCH ARCHITECTURE SESSION RST-4501 from Cisco Networkers in 2004.
2) Cisco Catalyst 6500 Architecture White Paper https://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/prod_white_paper0900aecd80673385.html