Notes on Cisco Catalyst 6500 Architecture. (Or What Does 720 in Supervisor 720 Mean ?)

Switching Fabric

The Supervisor 720 con­tains a 720 gig­abit per second cross­bar switch­ing fab­ric that provid­ing for mul­tiple conflict-​​free (or non block­ing) paths between switch­ing mod­ules. At the back of the mod­ule you can actu­ally see the chip that is the switch­ing fabric:

Using the “supa klever” form of mar­ket­ing math, a switch­ing fab­ric is actu­ally 360 GB/​s but because it goes in at 360Gb/​s and out at 360Gb/​s and there­fore you could count it twice. So that is seven hun­dred and twenty gig­abits. That’s the number.

In fact, the switch­ing fab­ric is eight­een inputs of 20Gb/​s per input. Of these, the back­plane in your Catalyst 6500 will decide how many are presen­ted to your line cards. The C6503, C6506 and C6509 will present dual fab­ric con­nec­tions while the C6513 is depend­ent on slot.

Per Slot bandwidth

Each slot in the back­plane of the C6509-​​E chassis has two 20Gb/​s back­plane con­nec­tions. There are nine slots. Nine slots at 40Gb/​s is 360GB/​s.

and the Switch Fabric con­nec­tions look like this:

Catalyst 6513

Note that back­plane slots on a 6513 do not sup­port dual 20Gb/​s on all slots. Because the switch­ing fab­ric on the Supervisor has eight­een 20Gb/​s inputs, you must lay­out the inputs differently.

Therefore the slots at the top of 6513 are only 20GB/​s and not 40Gb/​s. This is what makes the C6513 sign­fic­antly dif­fer­ent from the C6509 chassis. The 6513 was a pop­u­lar choice when low per­form­ance line cards were used, but today, high dens­ity gig­abit eth­er­net is com­mon and much increased band­width of serv­ers and desktops  means the C6513 is not com­monly used, espe­cially for data centres.

The Switch Fabric con­nec­tions look like this:

Fabric Connections for C6515 chassis.(Click for a full size image)

Not All Modules are cre­ated equal

Consider the fol­low­ing mod­ules that are com­monly used today (2009):

● WS-​​X6724-​​SFP: a 24 port Gigabit Ethernet SFP based line card sup­port­ing a single 20 Gbps fab­ric chan­nel to the Supervisor Engine 720 cross­bar switch fab­ric. Also sup­ports an optional Distributed Forwarding Card 3a (DFC3a, DFC3B or DFC3BXL)

● WS-​​X6748-​​SFP: a 48 port 1000Mb SFP based line card sup­port­ing 2 x 20-​​Gbps fab­ric chan­nels to the Supervisor Engine 720 cross­bar switch fab­ric. Also sup­ports an optional Distributed Forwarding Card 3 (DFC3a, DFC3B or DFC3BXL)

● WS-​​X6704-​​10GE: a 4 port 10 Gigabit Ethernet Xenpak based line card sup­port­ing 2 x 20 Gbps fab­ric chan­nels to the Supervisor Engine 720 cross­bar switch fab­ric. Also sup­ports an optional Distributed Forwarding Card 3a (DFC3a, DFC3B or DFC3BXL)

● WS-​​X6708-​​10GE: an 8 port 10 Gigabit Ethernet X2 optic based line card sup­port­ing 2 x 20 Gbps fab­ric chan­nels to the Supervisor Engine 720 cross­bar switch fab­ric with integ­rated Distributed Forwarding Card 3CXL.Not this card has 80 Gb/​s of input (in one dir­ec­tion, 160 Gb/​s in two dir­ec­tions) and could eas­ily over­sub­scribe the back­plane con­nec­tion and switch fab­ric con­nec­tion on those inputs.

An over­sub­scribed card means that pack­ets may be delayed or dropped under heavy loads.

Cross Bar Connector on the module

You can eas­ily tell if the blade has some form of con­nec­tion to the Cross Bar fab­ric by look­ing at the con­nect­ors on the back of the cards. If the Cross Bar Connector is present, then the mod­ule has some form of the con­nec­tion to the fabric.

Shared Bus Connector

There is also a Shared Bus on the C6500 that oper­ates at 32Gb/​s (yes, 16Gb/​s in both directions).

Shared Bus Conenctor.(Click for a full size image)

This is what was used in the early days of the chassis with the first gen­er­a­tion C61xx and C63xx mod­ules which could eas­ily over­sub­scribe the back­plane con­nec­tion. Most new install­a­tion are com­pletely fab­ric switched (except­ing for IP Telephony install­a­tions who use the over­sub­scribed model and like the QoS fea­tures of the C6500 and the high power dens­ity of up to 8500W in a single chassis).

Not Comprehensive

This post is not a com­pre­hens­ive look at the mod­ules and super­visor archi­tec­ture. This is covered a num­ber of doc­u­ments but check the White paper link just below this which has a lot more inform­a­tion and is a com­plete ref­er­ence to the archi­tec­ture of the C6500.

Observations and Comments

The reason for this art­icle is to out­line some of the issues that a Data Centre designer needs to under­stand. The archi­tec­ture of the switch, and same logic applies to stor­age switches, can impact the design of the net­work. The decision on whether to pur­chase a WS-​​X6704-​​10GE or WS-​​X6708-​​10GE can be impacted by the pre­dicted traffic load.

There are other factors that might affect a design, such as the Multicast per­form­ance or the QoS fea­tures, but this art­icle points out some of the first factors for think­ing about whether C6500 is the right switch. I hope you real­ise that know­ing some­thing about the guts of your switch, and how the insides look, is import­ant to get­ting your net­work right.

Reference

1) Some images and con­cepts are drawn from the CISCO CATALYST 6500 SWITCH ARCHITECTURE SESSION RST-​​4501 from Cisco Networkers in 2004.

2) Cisco Catalyst 6500 Architecture White Paper https://​www​.cisco​.com/​e​n​/​U​S​/​p​r​o​d​/​c​o​l​l​a​t​e​r​a​l​/​s​w​i​t​c​h​e​s​/​p​s​5​7​1​8​/​p​s​7​0​8​/​p​r​o​d​_​w​h​i​t​e​_​p​a​p​e​r​0​9​0​0​a​e​c​d​8​0​6​73385.html

Please rate this post:

(8 votes, average: 8.25 out of 10)

 Loading ...