Cloud computing: specialisation not generalisation

Some of you may have read UCLA Professor Jared Diamond’s 1997 Pulitzer Prize-winning book, Guns, Germs, and Steel. In it, Diamond answers a question from Yali, his aboriginal New Guinean friend. The question (and I’m paraphrasing) is: “Why do you guys have all the stuff?” That is, why did civilization — and all the good stuff that comes with it — prosper in Europe and Asia but not in New Guinea?

Diamond’s book is a 496-page answer to this question, and begins with the proposition that human civilization took an enormous leap forward when it moved from a hunter-gatherer culture to an agricultural one. Once humans could farm, they could settle down in one spot instead of following their food as it migrated from place to place. And once people could stay in one place, they began to specialize. One group would build shelters, while another would grow food. One group would forge weapons and still another group would make clothing.

Fast forward to the 21st century.  If our tribe had just 100 people, then each of us would have to be a generalist as there are so many jobs that need to be done. But in a world of seven billion increasingly interconnected people, the specialists will be valued over the generalists. What this mean in the context of cloud computing?

Last year the industry sold about 8,000,000 servers and 28,000,000 terabytes of storage. Now if all of this had been sold as a cloud service it would have been worth $231B per year.  And assuming flat demand, the total market would be $462B next year and over $600B the year after. Amazon has reported over $6B in revenue and is well on their way to hitting $10B.  So, as you can see we are at the very early days of compute and storage cloud service adoption.

Will there be just a couple of suppliers, with a few products? Just like we had a number of compute and storage on premises suppliers, my theory is we’re going to end up with a number of specialized compute and storage cloud services.  These could be specialized by location, performance, security and business models. After the Edward Snowden revelations, no government is interested in keeping all of their data in the United States.  And perhaps equally importantly, without low cost high performance networks throughout the globe, we’re going to have to locate compute and storage closer to the users of the service.  As a result at the very least, we’ll have compute and storage cloud services in all of the G20 countries.

It’s likely that compute and storage cloud services will also be specialized by performance.  Today, the absolute performance of single core computer can vary by 615% between cloud service providers. According to a recent report by Burstorm and Rice University, the Microsoft Cloud G5 compute cloud service is the fastest, followed by Amazon’s m4.10xlarge and c3.8xlarge. It’s surprising that compute cloud service performance can vary by over 6x. That’s more than the performance difference between a Porsche 918 sprinting to 60mph in 2.2 seconds versus the Ford Transit 350HD van getting to the same speed in 12.5 seconds.

Of course absolute performance isn’t everything – after all a Ford Transit 350HD costs about $50,000 and a Porsche 918 is over 15 times more expensive. Amazingly, when you factor in price, the price performance for a 4-core compute cloud service can also vary by 15x. According to the report the top three price-performance winners were Linode-4GB, Digital Ocean’s 8GB instance and Rackspace’s General 1-4.

The cloud services of the future will also be specialized by security features.  Today you can think of that as specialization by compliance features such as PCI or HIPPA, but tomorrow there will be specialization by hardening, identity and access management, auditing and security testing features. Finally, while AWS has created innovative business models, including on-demand, reserved instance as well as spot instance pricing, this will not be the end.  We’ve already seen some innovative new business models from Google.

So as a consumer of compute and storage cloud services you’re going to see even greater number of cloud services, some specialized by location, others by performance or security features and still others by new business models.  In less than ten years the idea that a server arrives at your loading dock will be as quaint as a floppy disk.

For more information, check out Timothy’s series of books on cloud computing, available from CrowdStory.

Do you think cloud providers will focus on specialized services? Let us know in the comments below.