Posted on 11 July 2012
I live in the Washington, DC metro area, specifically the 51st state – Northern VA or better known as NoVA. Fortunately, I was not affected by the storms and power outages of a couple weeks ago. However, Amazon and their Ashburn data center once again. Once again, many of us social media users found the pain of the outage.
How many outages do we have to endure before these folks put together better monitoring and DR plans? The outage begs the question of how important are these sites in the grand scheme of things as it relates to human life, power outages, tainted water and housing destruction during a storm. My response is all in how you look at the situation. One way to look at it is this is their business and livelihood, so DR should be relevant. Another is that many of us use these sites during times of destruction to communicate with the rest of the world, not just for fun or business.
As a marketer and business persoin, I find it interesting none of them have found it a competitive advantage to have the best plan and be alive when their competition is down as Okta exclaimed over this last outage, but security is their business and they must be alive at all times to retain customers.
I’m excited about all of the change with technology underfoot, but control and flexibility must be weighed.
How are you preparing to manage your mixed environment?
Travis just posted, “Unlike the Weather, You Can Choose Your Cloud”, discussing the recent power outages we experienced here in the Northern VA area last week. As a native of this area, thunder storms and hurricanes do pass our way and affect us during this time of year. Late May of 2008, we had a storm very similar to this one and this isn’t the first Amazon outage, “Amazon EC2 Outage Downs Reddit, Quora“ April 2011, nor will it be their last. Intuit’s SaaS QuickBooks was down 36 hours in June 2010, “Update: Intuit Sites Outage Strands Thousands of SMBs“. Or just this past Tuesday Salesforce.com was down, triggered by a power outage at an Equinix data center in Silicon Valley. Each time one of these outages occurs, it gains big headlines that the cloud has failed. (Read more here …..)
Posted on 24 April 2011
The Hub Commentary_
The news continued through the weekend on the Amazon outage this past week. While many who are commenting are using it as an opportunity to point the finger that the cloud has failed. I do not. The cloud is here to stay.
The role of the IT organization is changing to that of the service provider managing the services delivered to the business. Just because a public cloud becomes the deployment option of choice does not let IT off the hook for responsibility of the service. Think about it as your car and taking it in for service. Just because Ford, GM, etc. didn’t ship a part doesn’t make you any less happy when the dealer says, “I didn’t get the part from the manufacturer”. Did you take your car to the dealer or the manufacturer?
Agile and dynamic infrastructures are here to stay. Those that embrace them and take advantage of them early, will lead their industries. As one article stated that one of those affected said, “we wouldn’t be as far along without Amazon”. What this boils down to is what often lags with new technology – management.
I wrote a previous article on defining service value and then define deployment options, service levels, back-up and recovery. The natural market reaction will be to impose stringent service level requirements on the service providers, I’m here to tell you that you will pay for doing that. Define the appropriate service level for the criticality of the service. Then I suggest planning for the contingency of a major outage, don’t pay for the risk of the catastrophic every day for something that may happen once, never, etc. Have a back-up plan.
What is your back-up plan?
Despite restoring service for many customers, Amazon’s troubles with its Elastic Compute Cloud (EC2) and Relational Database Service continue this morning, meaning that debate over the wisdom of relying on such cloud services is certain to grow louder. (Read Full Article…)
Posted on 22 April 2011
Last week, Lori MacVittie had a blog post on DevCentral about earning your data center merit badge. The message was delivered up front and it was simple to understand: Be prepared. MacVittie is right of course, the best way to stay out of trouble is to put systems in place to prevent it from happening in the first place.
But today’s outage at Amazon EC2 showed us something else — that no matter how well prepared you are, stuff happens that’s totally out of your control and it can spiral out of your control pretty quickly. Lest you think just because you don’t use public Cloud Infrastructure as a Service (IaaS) like Amazon EC2 and hence have nothing to worry about, think again.
If it can happen to Amazon, it can happen to you because at its heart what is Amazon but a giant data center, whose core business is keeping other businesses going. That would suggest that Thursday’s outage was something extraordinary to bypass all of the fail-safes that a system like Amazon has to have in place to keep things going. Today it all fell apart, and it could just as easily happen to you because chances are, your data center doesn’t have nearly the number of contingencies in place that Amazon has.
That means that ultimately you’re probably closer to a disaster like yesterday morning than Amazon ever was (yet it happened anyway).
All of this is not to scare you because IT pros know the score about these things, but it is to remind you that having systems in place to monitor and alert you *before* that disaster strikes is more important than ever. Now, it may end up that it doesn’t matter how prepared you are if a disaster strikes that’s completely beyond the scope of anything you could possibly have imagined in a reasonable contingency plan.
All you can do is follow MacVittie’s simple advice and be prepared for whatever comes. It might not always be enough, but if you do your best, you’ll minimize those major outages and be ready to deal with them when they do happen. But remember disasters happen to everyone at some point, whether your in the cloud or in-house in a data center, and you need to be ready.
Photo by rbrwr on Flickr. Used under Creative Commons License.
Posted on 21 April 2011
The Hub Commentary_
Outages always make for big headlines and make for good examples of the cost of visibility and management. I read these articles with positive thoughts and still see cloud computing as the future and those who are bold took advantage of growing business earlier rather than later.
As noted in the first article, some of these organizations would not be where they are today with their business if they had not leveraged new deployment options as fast as they did. In a previous post, I discussed mapping your services based upon business value and cost and how to manage those services and selecting deployment options. It’s a balancing act of how bold and how much of a risk you take to grow your business with cutting edge technology. The balance to strike is how much you spend to have a DR or back-up plan for an event like this as you know they will happen.
Outages will occur and this shouldn’t push folks to write unwieldy service level agreements as you pay for those by pushing the risk back to the provider. I would suggest that spend would be more wisely spent on a back-up provider to account for hiccups in a cost effective manner.
My hat still goes off to those embracing the cloud and growing their business as a result with agile technologies and there are learnings to plan for moving forward.
How are using agile technologies?
A partial failure at Amazon Web Services’ cloud-computing infrastructure brought down some Internet operations today, including the Web sites of Quora and Reddit. (Read Full Article…)
Amazon reports this morning that it is making progress in restoring full service to customers of its Elastic Compute Cloud (EC2) and Relational Database Service in the eastern portion of the country after a rocky stretch of trouble that began sometime before midnight. (Read Full Article…)
Posted on 13 January 2011
The Hub Commentary_
“Visionaries have an innovative and disruptive approach to the market, but their services are new to the market and are unproven,” Gartner Yes, this does describe Amazon and EC2, but does that mean it is 2 steps behind the Leaders? Not often are there times to truly innovate and redefine a market. Amazon and EC2 are redefining an industry regardless of internal enterprise IT or the consumer market. Their customers are a mixture now over the traditional IaaS (Infrastructure-as-a-Service) IT service providers.
To the innovator go the initial spoils, revenue in the market. Will or can they sustain that leadership will play out over time. Given where we are in the life cycle of this disruptive technology (mostly development and test) in Enterprise IT shops, EC2 brings just the agility required for businesses to drive agile development of new products and services and move to market faster than ever before mimicking the consumer market.
In order to take these early development projects to production, yes, more formalized monitoring and management will no doubt need to be baked in. Now the question becomes, who bakes that in and supplies that information? Isn’t that agent part of the workload that is packaged and shipped out to the cloud to run on the subscribed to infrastructure? I call this service enabling your workload and injecting intelligence into it for purposes of monitoring, securing and communicating the performance of the workload the subscribers responsibility. Right? The provider is responsible for the infrastructure your workload runs on, not what’s going on in the workload, that remains the responsibility of the subscriber.
I applaud Amazon and say keep challenging the status quo. IT and the traditional proven providers need to think a little out of the box to meet the demands of market dynamics in market time! What do you think?
Gartner’s Magic Quadrant report has placed Amazon’s cloud computing service in one of its lower tiers, saying that for all of Amazon’s commercial success it is “visionary” but “unproven.” (Read Full Article…)