To be fair to HP, they have had chassis based switches from at least 1996/97. AdvanceStack 2000, HP ProCurve 4000/8000M, 4100gl, 4200, 5300xl, 5400, 8212, etc. 
Sure, until recently most of their chassis designs only offered redundant PSU’s and fans. But from a hardware reliability point of view, they very rarely failed. 

I know the method of deployment that you use and it’s a good design. fan of this myself.

all the best

Greg’s

For a new product line, we deployed stacked 3750 chassis are top of rack switches. The long stacking cables are perfect for doing an every other rack connection and then looping back on itself. This saves real estate and power vs homerun/patch panel/chassis switch deployment. Now, you’re correct that the 32 Gbps limitation is a design consideration.

3 years later and we have had no issues with either stacking or switch reliability. Is it possible, sure. It’s also possible for configuration on any flash to become corrupted. That’s life. is the flash/NVM on the 6500 more reliable? I’ve seen no data on that but I suspect there’s no difference. You’re correct that more parts = more failures. What you fail to mention is that, if the network is well designed, these failures will be limited to impacting the directly connected, single homed devices (and I can tell you that replacing a 3750 with a spare is a lot easier than replacing many components on the 6500. )

The 6500 can go faster and support more granular levels of control. The questions that always needs to be answered are: “How much control does this deployment need?” and “How much aggregated bandwidth capacity does this deployment need?” The cost differences for the device lifecycles are considerable. The 3750 has limited lifetime warranty and no cost code updates. i can spare multiple 3750′s for the cost of a single year of maintenance on the 6500. Stackables can be grown into and the chassis is mostly capitalized up front. Add in the cost of electrical power for the TCAM and saying they are comparably priced, is absurd. (Even moreso when looking at the 3750-X series)

When given a mission critical component, I’d rather have my own spares *in hand* than 4 hour depot service.

The vast majority of 6500′s I’ve seen deployed have been performing almost no advanced features. They have been purchased by enterprises with a lot of cash and who didn’t really question the recommendation. if you’re going to deploy less than 20 VLANs and your use of QOS is limited to voice, why buy a 6500 as anything other than a core?

The article implies chassis switches are always the best answer unless you’re poor. That’s not even a good generalization. Use the right tool for the right job. It’s intellectually lazy and dishonest to your customers/employer to overengineer solutions just because its simpler.

Use a hammer for nails and use a screw driver for screws. If people bought cars like companies bought network gear, we’d all be driving ferrari’s and hummers.

Most people don’t consider the increased maintenance cost over time on stackables vs a fixed chassis, nor the fact that the annual maintenance costs increase as more switches are added to the stack, while a chassis has a fixed maintenance cost no matter how many line cards are added.

That said, some times lower upfront capital cost is more important to the business.

I have not found that chassis switches fail as much as you say, and have not had problems with stack connector fail requiring reseating, nor bandwidth when appropriately used (appropriate use of a 3750 switch stack is as an access layer for end users, IMHO).

I have seen many instances of fools who built ‘core’ networks and collapsed ‘core’, ‘distribution’ and ‘access’ all on a single stack of 3750s. Even saw critical data center Layer2 interconnects built with 3750s. All eventually replaced to resolve limitations of 3750 (key limitations include: small TCAM, and maximum of 32 interfaces with HSRP on them).

Fair point but what about using ‘telco’ style cabling with a chassis based switch.

Fix the cables into the RJ45 switchports and run the cables via the sub-floor void into an adjacent rack. These fixed cables are presented into the top half of the cable rack and cables from the client/server end are presented into the bottom half of the rack. That way all patch cabling is within one rack.

Normally used in big data centres but can be efficiently scaled down to suit localised deployments.

The cost of an extra rack can always be mitigated against a clean, tidy, dependable and traceable cabling infrastructure where effective troubleshooting can be achieved without risking unplanned outages to other services.

Cheers
Dave

Re: Reliability
I have two wiring closets – one with a 4506 and one with Cat3750′s. If a power supply fails in each closet, how many ports go down?

Re: Shared bus
If I lose a switch of a stackport in my Cat3750 stack what backplane speed do I have now?

Re: Feature poverty
Netflow – although I admit Cisco can’t make up their mind whether the Cat4500′s should have this or not.

You wrote: The manufacturing quality of fixed format switches is lower than chassis.

I say:
There are many considerations here, and this ends up as a speculation. Just consider 2500 (or 2960 from Sasquatch family). Manufacturing quality is just that, and if it is somewhere higher or lower depends on many factors. Take Mars Rovers as an example, manufacturing quality of which was predicted to provide for 3 month. Sure, I understand what you are saying, but I just don’t see how can it be true. I’ve seen 4500 start burning, and 6500 start shorting, and 12000 that would not turn on. Described pattern of better manufacturing quality was not observed.

You wrote: More elements means less overall reliability.
I say:
Your second claim directly contradicts the first one. Chassis intrinsically have more elements than fixed chassis. Therefore in this case you claim that they are less reliable.
And yes, theoretically I agree. But then again, this is too simplistic statistic to be relied upon in practical environments. Or as Mark Twain had put it: “There are three kinds of lies: lies, damned lies, and statistics.”. Ironically, one of the most failure-prone device that I have seen was an O/E transciever, that was only converting optical to electrical signal. You can’t get much less elements than that, and yet there it was. One failure a week. There are many more practical examples like that.

You wrote: Bandwidth Limitations and Shared Bus Architectures
I say:
Here you just get it wrong, because you are only partically right. You are right about 16Gbps, but you completely miss the fact, that there are TWO counter rotating rings. It’s a two-ring system, and each of those has 16Gbps, therefore together they do really have 32Gbps. Want proof? Just take apart the 3750 and look at the chips. You’ll find there Maxim MAX3780 chips, which provide Quad bi-directional interface of 2.5 Gbps per channel. That’s 8G of clean bandwidth per chip. There are four of these in total. And they are bidirectional… No matter the way you look at it, it’s not 16Gbps.
And, besides, you are just being plain ignorant here. What about 8:1 oversubscription of 6148 cards? What about 6Gbps per slot of 4500? What about the simplistic SERDES that are not even pass as an excuse for any sort of intelligence?
The truth is, every platform has it’s bottle neck. And stack bandwidth is not really a good example.

You wrote: Reliability and Availability are NOT the same†thing
I say:
Yes, but then again, conceptually, there is no protection against a failure of one linecard. If LC fails, all connected ports will loose service. There are no tricks and no magic.
Software reliability (Failure in a†stack is typically /? often takes the entire stack†down)
How many time a dual-supervisor device failed in a way, that took down device as a whole, and switchover did not take place? Just as many as stack failures?

You wrote: The Stack Connector
I say:
That’s your experience. I found that if I screw the connector on very tightly, it works after 4 years just as well as the first day.

You wrote: Feature Poverty
I say:
Comparing anything to 6500 can be a sin. There are very few hardware platforms that can offer so many fatures. Why didn’t you compare the 3750 with 4500? 4500 offers worse features in some cases than 3750. Two-level shapers (shaper in shaper) is a sci-fi for 4500/SUPV, and so are the queue lengths, and drop thresholds. Sure they are a bit on the heavy side control wise, but 4500/SUPV cannot even shape properly.
And besides, 6500 is a chassis. It’s empty. It only has clock, mac addresses, connectors, and the wiring. There are no features in 6500 chassis. That what you should have written about – comparing it to line cards and supervisors – where the switch intelligence actually is.

As the final note, it’s just wrong to look at platforms this way. The zen master should have asked the student this question: What differentiates one switch from two?

Cheers!

I’ve utilized Nortel stable switches for the past 13 years and while they certainly have their issues, what equipment doesn’t, they are very reliable in my opinion and have served us well.

If anything thought I can’t understate the impact that having 1′ patch cables in the closet. We’ve had numerous cabling solutions from companies such as Panduit and Ortronics and in the end we always came back to a closet three weeks later to find a gigantic mess. While that’s been our solution it certainly won’t be everyone’s solution.

Cheers!

Upgraded to C6500, and problems went away. Broke the C3750 down into top-of-rack and they worked well too.

Stacking, it’s not so great in Cisco land.

However, I found the Nortel stackables unreliable and generally poor feature-wise. I moved away the entire company as a viable supplier at that point (combined with the poor tech support outside the USA) so I can’t really say much about them.

The cabling problem can easily be solved on chassis by using deeper racks and putting the patch panels at the rear of the switch and running cables from front to back. Much better than stackables.

Look for a bunch of new hardware to come out over the next 2 quarters across each of the Nexus hardware platforms. I think the combo of what will be available by the end of the year will really make the Nexus line the platform of choice in the data center.

Have you posted any thoughts or blessays on the Nexus line? Cisco is marketing them up harder than I’ve ever seen their sales force work. I’ve heard little on them as of yet from the customer base.

Rob

Using Avaya/Nortel Ethernet Routing Switch 5520s we deploy in stacks as large as 8 switches (384 ports). We place the stackable switches in between the 48 port patch panels and then utilize 1 foot patch cables between the panel and switch. This helps greatly in terms of cable management and cooling and also great speeds break-fix replacements. It’s worked wonders for us… especially where you have multiple departments/people working in the closets and they aren’t all equally motivated to keep the cabling plant neat and orderly.

In the old days you’d have 300+ lbs of cabling hanging in front of the chassis based switch. It would take you 60 minutes just to label all the cables to replace a defective blade/module forget about the logistical task of actaully geting the blade/module out of the chassis with 300+ cables draped in front of it.

Thanks for another great article Greg!

Cheers!

But for the datacenter, with the Nexus line, I think we’re going to see the virtual chassis becoming the norm. With the introduction of the 2K Nexus line, many of the negatives that came with the 3750 series stackables are eliminated. Port costs go way down and availability should be much higher due to them being able to be dual managed.

Once the new models come out later this year, I think the Nexus line and the virtual chassis model will start to usurp the 6500 as the dominant platform in the datacenter. That is assuming that the Nexus line lives up to its promises.

However. I have never received any briefings, technical documents or deep dive into the JunOS software and have no view on how the software architecture looks. I know that JunOS is distributed OS and may have features that inherently allow distributed processing (compared to Cisco IOS-SX software which does not). With those caveats, I still feel that stackable technology is always weaker than chassis.

While 6 3750s would list for about 110k a 4510 with the same port density is about 95k and still has two more slots. I would take the 4500 any day (I am not the biggest fan of that product as a 6500 isn’t that much more money).

The bigger challenge is convincing the bean counters that the initial investment is the right one and even though you don’t need it today the capacity is there for the future.