Google’s blogger system was down for a couple of days last week and the bloggerati are up in arms about it. They have a point too. A software system delivered as a service that loses vital data and is down for unspecified periods cannot be relied on and if you can’t rely on cloud computing — which is, after all the heart of this matter — why do it at all? But too often it seems that the response to a problem like this, especially when handled by those who are emotionally involved and not by cooler heads can lead to a larger problem.
People like Ed Bott are calling for business strategies that do not rely exclusively on the cloud for storage. Bott obliquely makes the case for some form of local storage but I suppose a virtual back up to another vendor might also be possible and could even spawn a new enterprise level industry. But think of the overhead.
At its heart this is a “belts and braces” approach. A little digression might be in order here. We know what belts are and braces are a British word for suspenders. Belts and braces describes a needlessly redundant strategy for holding up one’s pants and has been in the lexicon as shorthand for all similar strategies for a long time though it seems to have fallen into disuse due to advancements in high tech belt technology.
At any rate, the opposite of belts and braces is counter intuition. We speak of “thinking outside the box” strategies as shorthand for being counter intuitive. What if we did the opposite of what appears to be the common sense solution? It doesn’t always work. Jumping off a bridge with wings strapped to your back won’t materially alter the outcome, so you need to pick your spots and use a bit more common sense.
Perhaps the best example I can give of this yin and yang of belts and braces and counter intuitive thinking comes from economics. We’re in tough economic shape right now and many people who should know better are saying we need to lower our deficit. Spend what you take in and not a penny more and you’ll be fine. But the historic record tells us this won’t work. It’s been tried before and deficit cutting by itself makes matters worse.
A better solution would be to start by asking what we’re spending money on and trying to figure out if the funding is going to beneficial and productive uses. I think we know the answer to that one. The American Society of Civil Engineers gives our infrastructure — roads, bridges, water and sewage treatment systems and other similar things — a D+ so we know pretty well that the money isn’t going there. Meanwhile, last week the heads of the five biggest oil companies testified in the U.S. Senate about their need for government subsidies despite the fact that they are the most profitable corporations in the history of the known universe.
Let’s not get too far afield though. The impetus to cut the budget rather than looking for ways to better spend money that will boost economic activity is understandable but wrong headed. In the General Theory of Employment, Interest and Money and other writings, John Manard Keynes got it right by pointing out that government spending needed to be counter intuitive and counter cyclical and that in bad economic times, government needs to be the buyer of last resort, temporarily replacing demand that the private economy cannot sustain in other ways. I am a Keynesian but I don’t want this column to be about Keynes. I am only making a case for counter intuition.
Back in the cloud, the idea of storing or backing up your data to prevent the kind of outage that happened last week at Google is a non-starter for several reasons.
First, if you back up data without backing up applications and making them available to run locally, you haven’t solved the whole problem. You have saved your data but you are still dependent on the cloud vendor to supply the application. Take either one away and you’re dead in the water.
Second, if you take the belts and braces approach and store your data as well as your programs, haven’t you obviated the need for the cloud? Of course you have so the solution is no cloud at all if you take this argument to its logical end point. Alternatively you have both — belts and braces — and you pay twice. Not smart.
But if you do give up on the cloud, then you also give up on the economies you achieve from it and you are back at square one in your data center with more demand for IT services than you can deliver with your budget and staffing levels.
Hmmm, that’s a tough one. What to do?
Clearly, we’re too far down the road to cloud computing to consider pulling the plug. There’s no safety back there only a different set of problems including the fact that your IT organization would now be responsible for providing 100 percent up time. Care to take that one on? Thought so.
The solution isn’t backsliding into conventional IT or quasi-cloud solutions but redundancy might play a part. I think the real answer can be found in redundancy, planning and policy. I know nothing about Google’s approaches to these issues but as an outsider looking in, it seems like there was little or certainly insufficient sandbox activity on Google’s part before they went ahead and upgraded the blogger software.
Sandbox activity, despite its childish sounding name, is a super-important and hyper-rational approach to avoiding this kind of problem. Rather than calling it a sandbox we should call it what it is, a model. Cloud vendors might be guilty of insufficiently modeling their customers and their services. A realistic model of the system that went down and the upgrade that caused it would most likely have caught the problem before it was one.
This is the kind of thing that only gets figured out in live practice during a product’s early life. The need for modeling is not hard to discern but until there’s a problem its an issue that won’t get funding.
So this was the wakeup call. We need much better system modeling in the cloud and I expect that smart customers will soon skip over their vendors’ comforting SLAs and pledges of a specific number of nines and ask the much harder question of how those nines get delivered.
Mirrored data centers and intelligent modeling policies as a standard seem like the next big steps in cloud computing. Counter intuition or thinking out of the box got us to the point of this solution, but there’s nothing counter intuitive about it. It is an approach that relies on redundancy but appropriate redundancy that is very different from the case where we were thinking about going it alone back into the past. I hope the people who think about the national budget think this way soon.