Outsourced Data Centre Hosting and Wasted Time

Over the last two years I’ve worked almost constantly with customers who have outsourced their data centers. That is, their physical IT infrastructure is hosted in a facility owned by external company. What ever the reasons that drove that decision, I can certainly point to a painful outcome – wasted time.

The hosted data centers must take action to ensure controlled access to all parts of the data centre and maintain the integrity of devices for each customer. As such, there are processes in these data centers to ensure physical security and system integrity. And these processes are supremely painful to experience. A few negative features of outsourced data centres

  • They aren’t local to the company and often require travel to remote places.
  • I require travel approval and process to go there ( i.e. hotels and transport).
  • I have to negotiate with a third party for DC resources.

For example, installing a new rack requires, at least, the following steps to occur:

  • negotiate with DC Sales to set the current pricing and availability for a request for some more space.
  • Wait for DC provider to complete their process.
  • Wait for DC provider to complete their process.
  • DC team will investigate power, space and cooling availability,
  • Wait for DC provider to complete their process.
  • Wait for DC provider to complete their process.
  • Complete internal purchase order process.
  • Wait for DC provider to complete their process – design
  • Wait for DC provider to complete their process – electrical
  • Wait for DC provider to complete their process – space planning
  • Wait for DC provider to complete their process – cooling check.
  • Wait for DC provider to complete their process – design sign off
  • Apply for and receive approval to ship equipment to DC.
  • Wait for DC provider to complete their process.
  • Attend DC
  • Ship my equipment to Data Centre loading dock.
  • DC has no storage area therefore must arrive with delivery to install immediately (thus impacting the time schedule of IT people).
  • Most DC have no space for storage, and you have forgotten a cable or screwdriver.
  • Revisit site two weeks later to complete the install.
  • etc, etc, etc

Since the networking team is usually first through the door for any data center build, it specifically seems to impact my workflow heavily. And regularly I arrive at the DC to find that committed work was not completed correctly. Even worse, because all services at the DC are outsourced/subcontracted, there are no people to help me to fix such problems no matter how small.

Add Eight Weeks to the Timeline

In my experience, DC provisioning adds six to eight weeks to any project. I could go on and on about why the process is so resource intensive. Having a remote DC is a significant time loss and productivity loss. And it’s a cost that doesn’t appear on anyone’s budget. The extra cost is carried by the project and therefore cannot be used as a driver to get management to change how things work.

Of course, there isn’t a solution here. But outsourcing your data centre has several significant costs that aren’t discussed in the early stages.

  1. the hidden cost to business by being slow to respond to internal needs (six weeks to get access to more space is ridiculous and never happened before).
  2. The cost of time lost waiting for an external party to complete provisioning and
  3. project management costs to ensure that the DC provider delivers on time (never happens)
  4. The cost of travel and accommodation at the site. Remember that a single project will usually require several trips to the DC by different team members.

The EtherealMind View

Before you shut down your data centre, take time to think hard about the hidden costs. It’s frustrating to have projects budgets and deadlines blow out because your DC provider cannot respond in a timely fashion. And lately, it has become much worse. Data Centre provider have cut back on staff and response times are worse than ever. Most importantly, consider the impact that an extra eight weeks of project time causes to infrastructure projects, and the subsequent impact to business mobility.

About Greg Ferro

Greg Ferro is a Network Engineer/Architect, mostly focussed on Data Centre, Security Infrastructure, and recently Virtualization. He has over 20 years in IT, in wide range of employers working as a freelance consultant including Finance, Service Providers and Online Companies. He is CCIE#6920 and has a few ideas about the world, but not enough to really count.

He is a host on the Packet Pushers Podcast, blogger at EtherealMind.com and on Twitter @etherealmind and Google Plus

  • meeple

    >six weeks to get access to more space…

    imho it shouldn’t happen that You need more space and You don’t have these few weeks to provide it. it usually takes few weeks to deliver servers and You should know even earlier that new servers will be ordered

    if You are surprised by new servers coming to your dc within few days, then the problem is somewhere else.. not in an outsourced dc

    ps
    sorry for my poor english

  • http://twitter.com/Vegaskid1973 Matt Thompson

    There seems to be an unnerving trend in recent months to expect things yesterday. And if that can’t be done, then the first things to get dropped from the project are documentation or testing, which ends up coming back to bite you on the arse 10 fold, later on.
    Moving from onsite to a data centre is not a task to be taken lightly and as part of the project, risk management, timescales, costs, 3rd party involvement should all be given serious consideration.
    As the previous commentor has stated, a move such as this would take time even if you were moving to your own building next door. New kit to buy, migration plan, politics to dance around, contingency. It all adds up. Add to that appalling communication skills and you can find yourself destined to fail.
    However, I dont think hosting your kit in a data centre is a waste of time per se. I do think that there are plenty of data centresproject managersengineers who aren’t up to the task but that is a different matter entirely as there are equally those that can make such a move more than worthwhile.

  • James Cape

    I tend to avoid expansion issues with either a ROFR on the adjacent cabinet(s) or a cage big enough to handle my expected growth over the next few years.

    Capacity planning is tough, but if you know (for example) that implementing Hadoop will take 10 racks for current capacity, and your growth rate is doubling every 4 years, then you should aim to support 40 cabinets, and get that into your service orders at a reasonable price.

    However, I work in finance, and the bulk of the footprint is a lot
    closer to a traditional telecom than an enterprise—a few cabinets
    apiece in a lot of geographically distributed datacenters and a
    comparatively high switch:server ratio—and I have no idea what the cost structure at that scale even looks like (how much is 40T of redundant cooling? What’s the OpEx on 200kW of dual-grid power?).

  • http://www.facebook.com/scott.goldenberg.9 Scott Goldenberg

    Hi,

    I wanted to put my 2 cents in on this. Back in the late 90′s and early 2000′s we did
    all of our own hosting. Around 2004, I
    started working for a company that needed to have the best latency to some
    exchanges. This meant going to a co-location company. About a year ago that
    company closed it’s doors and I started my own third party manages services
    company called IT Hosting Service.

    When we were hosting in our own facilities we had to deal
    with the following issues.

    1) Physical space in our room for another rack.

    2) Building power. (Thinks getting a DC to give you more
    power sucks, try talking a landlord into another feeder to the building.)

    3) HVAC

    4) Fire Suppression

    5) Security

    6) Network lines, usually with only one or maybe two
    companies servicing the building

    7) Getting servers delivered (often having to tip the
    building personnel well just to get the equipment brought up from the loading
    dock)

    Once we moved into a colocation site, Our challenges changed
    a bit.

    1) We no longer had to deal with a landlord who did not
    know much about hosting servers. We now deal with the data center who makes
    money by providing space and power for technology. The ‘get’ power and
    communications.

    2) HVAC/Fire/Security – all were provided by the DC. I
    did not have the fire department telling me that they need sprinklers in the
    computer room.

    3) We have at least 20 network providers we can work
    with.

    One the down side:

    1) Cross connects are costly (large monthly costs for a
    wire sitting in a tray!!)

    2) Dealing with the opaque process that our data center
    providers have.

    I would also like to address some points.

    1) Most quality data centers have tools and cables
    available on site. Unless you are looking for a 50′ infiniband cable the
    probably have it.

    2) Storage is definitely an issue. Most data centers want
    to have the most effective use of space possible. After all, that is where they
    make their money.

    Finially, having a good
    managed service provider (don’t mind me
    tooting my company’s horn) is essential to keeping your systems operational.
    The data
    center personel are fantasic at cabling, power and space but no one is
    able to know all the ins and outs for 1000s of customers.