Recent outages or slowdowns at Salesforce.com have brought to our attention that, as good as on demand computing is, it is still a creation of fallible humans and subject to the occasional mishap.
I was surprised to learn that the December slowdown was due to a database bug and that the impending upgrade to something called MirrorForce (mirrored data centers) might not have made any difference. There was a bug in the database software that caused the slowdown and that is a product that Salesforce.com sources outside.
So what do we take away from this?
Well, for starters, there is no such thing as perfection or 100% up time. That’s not to make excuses for Salesforce.com and truth in reporting requires me to say that the company is a client. If anything, this simply drives home the fact that we really are dealing with a new kind of utility — the information utility — and that it has all of the goodness and badness of other utilities. Like other utilities you become dependent on them and though they may try to be perfect, sometimes they let you down.
That gets me thinking about back up and redundancy. Many hospitals, for example, have back up generating capacity in case the electric service goes down. In such a case, a generator which is cranked by a natural gas-powered diesel kicks in when there’s an interruption. But that’s an expensive proposition and one not usually opted for by residential consumers.
In the on demand world what would the equivalent be? I guess you’d need both redundant power and a server on premises but with that you reduce or even eliminate the cost savings you got when you went with the hosted solution to begin with. That won’t work. What probably will work, and what’s already out there is the increasing redundancy inherent in wireless and handheld devices. Most on demand CRM providers I know, and many traditional providers too, offer some kind of ability to down load data to little multi-function devices that mobile users have come to depend on.
Perhaps it won’t be long before plugging in these devices becomes SOP (in many places it already is) when you are in the office to ensure hot backup of at least your most immediately needed data. A few hundred bucks for a PDA is beginning to look like cheap insurance. Also, there might be an emerging market here for devices that have extra memory or other features that make them more useful as redundant hot spares. In fact, its possible all this could result in a new category for PDA vendors. After all, the PDA is a lot more valuable as a tool than an investment in redundant power.
On another, related, matter, kudos to the Salesforce.com management team for taking ownership of the problem and clearly communicating to the customer base what went wrong and what was being done to fix the problem, as well as to institute safeguards against a future problem. They also said they were sorry and reiterated their goals for service and up time. To me that was pretty good crisis management.