The following table is from Peter Norvigís essay Teach Yourself Programming in Ten Years. All times are in units of nanoseconds.
| Computer Action | Time (in NANOSECONDS) |
|---|---|
| execute typical instruction | 1 |
| fetch from L1 cache memory | 0.5 |
| branch misprediction | 5 |
| fetch from L2 cache memory | 7 |
| Mutex lock/unlock | 25 |
| fetch from main memory | 100 |
| send 2K bytes over 1Gbps network | 20,000 |
| read 1MB sequentially from memory | 250,000 |
| fetch from new disk location (seek) | 8,000,000 |
| read 1MB sequentially from disk | 20,000,000 |
| send packet US to Europe and back | 150,000,000 |
The EtherealMind View
So this is a part of the circular argument FOR CLOUD COMPUTERING: by centralising as much computing as possible in a single place, I reduce the latency for most transactions. Thus, transactions within the data centre are so much faster. Lets not forget easier to organise: especially interactions between multiple systems. Therefore Cloud Computering is a good thing.
But, you see that last number there. Yep **150 000 000 nanoseconds**. Got it ? “That’s me losing my religion” because you can’t change the speed of light. If you put my hosting in another continent, that’s it, Cloud Computering is a fail. How can you solve the problem for data exchange when, on a very very good day (yeah right), I get lossless 150 millisecond delay ? Cause I’m not getting lossless, and I’m not getting 150ms at my house.
So, yes, the challenge of overcoming the bandwidth shortage is the big issue today and for the next ten years. If I am accessing a Cloudy system in China or the US then I’m facing an extraordinary delay. A delay that is thousands of order of magnitude bigger than compute power. And if I’m using Internet bandwidth, the delay, packet loss and round trip time is not guaranteed.
And its cumulative – lets say a hundred transactions are needed (not uncommon for reasonable client server data interaction), that *fifteen seconds* of total delay. And that’s what I experience every day because I don’t live the in the San Francisco Bay area less than 100 kilometres from the data centres that host this cloud computering stuff. I live ten thousand kilometres away and the cloud is SLOW. Unusably slow.
Because the global network delay is SO LARGE how are you going to address this ? WAN Acceleration ? Local Caching ? Client Side Caching ? Packets Carried by Angels in Magical Boxes ? Avian Carriers ?
If you want to claim these are the saviours, then tell me why we aren’t using them today.
Where are the answers to that ? Please queue up in the comments below.
Reference
Reference to the table via John D Cook – The Endeavour
Are we talking about latency or bandwidth here? The table’s about latency, then you talk about “overcoming the bandwidth shortage” but then later about the latency for a hundred (presumably serial) transactions. As you yourself say, you can’t change the speed of light. High latency at global scale will always be a problem no matter how much bandwidth we have, so even mentioning bandwidth only confuses the issue.
You ask why we aren’t using these solutions today, but in fact we are. A vast quantity of the data in the world is delivered via CDNs, or via caching and replication delivered privately. Applications and protocols are becoming ever more parallel and pipelined to reduce latency sensitivity. Eventual consistency and asynchronous queuing have become common approaches precisely because they decouple the user experience from the underlying network latency. Ditto for AJAX and such in the web world. All of these approaches exist outside of a cloud context, but when combined they enable most applications to run in the cloud with minimal effect on the user experience.
The real lesson of the table (which BTW is usually attributed to Google’s Jeff Dean) is that the very longest operations are things to avoid. You can’t do anything about how long it takes for a packet to go around the globe, so *don’t do it* unless you absolutely have to. Applications should be structured so that the local/remote protocol only comes into play at the point where the interactions are fewest, to reduce the total number of round trips.
The colocation argument you cite is a rare one; most cloud arguments are based on operational flexibility and cost granularity. It does work if your data are already in the cloud, as is the case e.g. for Netflix, so colocation within the cloud actually avoids a round trip instead of incurring one. Even when it doesn’t work, though, refuting one argument for a conclusion isn’t at all the same as disproving the conclusion itself. There are other reasons and situations that favor cloud computing, even if it doesn’t solve one particular curmudgeon’s problems for him.
I comprehend that software attempts to make as many efficiencies as possible. And expect no less. However, when you have a 1Mbit/256kbit internet connection nothing happens at speed. Not Youtube, not gmail, not cloud storage, not anything. To make things worse, 60-70% of all cloud services are US based and the latency from UK to US is bad.
You can’t change the speed of light.
I agree with Jeff. The idea behind the table is to give developers an understanding of what various types of data access cost as compared to a single instruction. I’ve seen similar tables before using units of time (memory access = 1s; going to tape = 5 years). The table shows that going to disk is very expensive (eg, I could have run 8,000,000 instructions in the time that it took to move the head to get at the data I want). This type of thinking is what gets us solutions like memcached, shared memory stores, etc.
The table has little bearing on cloud computing. Your argument about latency is a complete strawman. Many cloud computing environments have crappy internal latency. Amazon had to come out with a separate HPC service just to address this.
I think your points would be more applicable to a discussion of application performance in general. So many developers don’t understand the impact of network latency on application performance. Web developers and server administrators don’t understand how web browsers request files, and we get dozens of requests to serve a single page. Just as an example, your static assets don’t seem to be using expires headers, so everyone has to make several requests just to find out that you haven’t updated your stylesheet since the last page request. Lots of huge web sites have this problem, it’s not just a cloud thing.
Along with capitalizing bandwidth shortage, some of the transatlantic fiber providers (ex. Hibernia Atlantic) are marketing low latency transatlantic connections in the sub 60ms range. While shaving a few milliseconds off the RTT will benefit the financial trading folks it’s not an appreciable reduction in latency for cloud based services for the masses.
The point Greg is trying to make (I think) is that if you are connecting to a “cloud” that is 10K miles away your experience could be impacted by the propagation delay if design considerations are not made to negate the impact. “How far away is the datacenter” is just another one of those questions that need the be asked when performing due diligence.
What cloud are you accessing that’s 10,000km away?
Gmail is in the cloud. It’s not slow.
MobileMe is in the cloud. It’s not slow.
YouTube, eBay, the list goes on.
I don’t know anyone who advocates having a cloud data centre on the other side of the world from it’s users.
In fact they are all slow for me. Really slow. I rarely watch YouTube because of lag – it takes 20 secs just load the player, and don’t use any cloud storage because of bandwidth limits. I don’t use the gmail web interface because of lag, always a desktop program.
However, when I travel to US I notice the difference, it dramatic when the latency is smaller.
Must be that 3rd world country you live in 😉
I’m 10,000km from the US also and even on a 3G network loads Gmail and YouTube faster than that.
Maybe. But that experience isn’t uncommon for people who live outside of capital cities.
Two weeks ago I moved from London to the US. All those services you mentioned run just fine when accessed from ISPs in London and I haven’t noticed them being any faster or slower when accessed from the US.
So close to Wales, it could be possible that your particular ISP runs TCP/IP over Ovis Aries…
😉
This is why most of the cloud services people, anyone running a consumer web service, people using CDNs (if appropriate), or anyone looking at consolidating lots of datacenters end up with at least 6 datacenters (or colo facilities at the very least). For the exact reasons you point out, plus a host of disaster recovery issues, putting everything in one datacenter is a bad idea.
From a service consumption perspective, ideally the performance of the service should be good regardless of where you consume it. Delivery of such a beast, as you know, is a bit harder.