Learning from Amazon's cloud collapse - Moral of the story? Technology is just an enabler and fallible

A lot of people did not know about a major shock that happened last week that had a big implication for businesses going forward. This article from CNN, View article...,you will learn that Amazon's cloud computing service went offline.

The concept of cloud computing is good. I wrote a piece in this blog earlier, http://myoneacademy.blogspot.com/2011/03/why-cloud-computing-will-be-next-new.html. However, a lot of people forget that technology is only an enabler and not a panacea for businesses of the future. One shouldn't put all their eggs in one basket. You must always have redundancy in place.

Below is an excerpt of the original article.

-------------------------------------------------------------------

Learning from Amazon's cloud collapse

(Mashable) -- Call it Cloudgate, Cloudpocalyse or whatever you'd like, but the extended collapse of Amazon Elastic Cloud Compute (EC2) is both a setback for cloud computing and an opportunity for us to figure out how to stop it from happening again.

Amazon may be best-known for its online shopping site, but it also has a substantial cloud computing business. It provides a scalable, flexible and particularly efficient solution for companies to store and deliver massive amounts of content.

Its model of only paying for what you consume was a radical innovation when it launched in 2006.

In fact, Amazon Web Services has been so affordable and reliable that thousands of companies from Foursquare to Netflix utilize the company's cloud computing technology and servers to run their businesses.

They put their faith in Amazon's cloud because there was no reason to think that it would falter. One of cloud computing's key tenants is reliability through redundancy of both servers and data centers.

Then on Wednesday, Amazon's northern Virginia data center started experiencing problems that caused major latency and connectivity issues.

The trouble was apparently due to excessive re-mirroring of its Elastic Block Storage (EBS) volumes -- this essentially created countless new backups of the EBS volumes that took up Amazon's storage capacity and triggered a cascading effect that caused downtime on hundreds (or more likely thousands) of websites for almost 24 hours.

The collapse took its share of victims. Among the most prominent companies affected were Foursquare, Quora, Hootsuite, SCVNGR, Heroku, Reddit and Wildfire, though hundreds of other companies big and small were affected.

Luckily, one of Amazon's most prominent customers, Netflix, didn't experience problems because it's built for the loss of an entire data center, while companies relying on Amazon's four other global data centers didn't experience too many issues.

A learning moment

FathomDB founder Justin Santa Barbara has a detailed post on his blog about what may be the biggest problem to come out of this week's collapse: Amazon's cloud redundancies failed to stop a mass outage.

Its Availability Zones are supposed to be able to fail independently without bringing the whole system down. Instead, there was a single point of failure that shouldn't have been there.

This week's disaster in the cloud is a reminder to startups to build redundancy into their applications and their own systems, but as Santa Barbara points out, most startups don't have the time or resources to engineer for multiple cloud systems (each Amazon global region/data center has its own rules and features, making a simple "switch" to another center difficult).

These companies trusted Amazon to keep them online, and Amazon failed to deliver.

Catastrophic issues will always occur, but in the pre-cloud era, downtime only affected a single computer or website. Today, a catastrophic event takes down thousands of websites, causing millions or even billions of dollars in lost revenue and productivity.

This incident is no reason for us to shun cloud computing, though. Its benefits (scalability, cost reduction, device independence, performance and more) far outweigh its cons.

We do need to take a hard look at how we structure our cloud infrastructure though and find new ways to either prevent single points of failure or quickly move content off failing clouds faster, especially as the world's computing power is consolidated into fewer and fewer systems.

Cloud computing is still in its infancy, and today's events make it clear that we still have a lot of work to do. It could be a whole lot worse next time if we aren't prepared.

Why cloud computing will be the next new thing for all of us

I recently went for a conference organized by Oracle regarding cloud computing. In a nutshell, cloud computing is just the notion of aggregating everything to the internet. In other words, the current environment of individual operating systems, individual file storage and individual licensed software will become the thing of the past.

In cloud computing, the only item that is necessary to have is a simple terminal with internet accessibility. The calculating power no longer resides on a chip but will be handled by the cloud itself. These following factors introduces a few interesting potential scenarios. These are:

1. Upfront cost of ownership reduces substantially. Instead of getting the latest computers with the fastest chip, a simple basic computer is all that is needed to do anything.
2. Education to the masses. Every child can be provided a terminal for internet based learning. The classroom is no longer defined as a physical location. Everyone can attend “a” class, no matter where there are in the world. Age will also not be a factor. Adults that did not have the opportunity to go through the standard education process can now attend. As there will not be any requirement to be at a certain place and hence meet other people in class, it minimizes “embarrassment” and hence more adults will be willing to take such classes.
3. Reduces the cost of maintaining a large quality teaching staff. With webcam and audio interaction, one quality teacher can handle a multitude of students at any point of time.
4. Ease of review and feedback. Using the internet meant everything that is done through it will be recorded. Hence, the ability to review and provide feedback will be a good learning experience not only for the students but also for the teachers and facilitators.

A few downsides to the cloud computing concept should also be noted. These are:

1. Internet infrastructure is crucial. Without a good internet infrastructure in place, the power of the cloud computing cannot be utilized effectively. This factor makes less develop countries unable to maximize of this technology to leapfrog towards the more developed nations.
2. Discipline is paramount. Using the infrastructure for non-productive activities reduces the capacity for more productive proceedings. Moreover, learning via internet will require discipline by students to remain at the terminal to learn as there will not be any form of disciplinary figure present.

For me, the benefits outweigh the negatives. I am all for the transition to cloud computing. What about you?

Nuclear energy - a must have technology for advancement

The events unfolding in Japan regarding its nuclear power stations is disturbing. I hope they can contain any fallout from any failure of the various reactors.

This is not be used as an excuse to limit Malaysia's energy focus to purely non-nuclear. Although renewal energy is the best option, the current renewable energy technology is not sufficient to support our current standard of living. To make it a reality, we must change the very nature of our life.

Instead, we should look at what has happened as a case study and ensure that if it does happen, we can adequate tackle the problem head on.

I am all for nuclear technology.