Open Source Meets the Cloud

The enterprise, government, startups and the system integrators are keenly watching the space of Cloud Computing. While it is a fact that only companies with deep pockets like Microsoft, Google and Amazon can get into the business of providing the Cloud infrastructure, the contribution from other companies is absolutely critical for the Cloud to become viable and real. The biggest concern and risk involved in moving to the cloud is security, privacy and reliability. Most of the customers would want to know what lies beneath the Cloud platform before betting their business on it. Making the Cloud transparent, open and interoperable will enable better adoption by businesses. This is where the Free and Open Source Software (FOSS) will act as a catalyst. The CIOs would want their teams to setup and play with the same stack before actually moving to the Cloud. Knowing that it is the same software that powers their infrastructure at some unknown corner of world will bring a level of comfort to the skeptical decision makers. The Private Cloud Computing offerings are now available from VMWare, Microsoft, IBM, Sun and others. But it is still not clear whether these are the same offerings that power the commercial Public Cloud offered by the Cloud providers. For example, Microsoft categorically mentions that Windows Azure, their Cloud OS is not available as a retail OS that the customers can implement in their data centers. It is also unclear whether Microsoft’s Private Cloud offering based on Windows Server 2008 Hyper-V is the same that is powering its Public Cloud, the Azure platform.

Cloud Stack

Cloud & Virtualization

Before looking at the critical contribution of FOSS to the Cloud, let’s understand the typical Cloud Computing architecture. At the heart of the Cloud is Virtualization. To bring the elasticity nature to the Cloud, Virtual Machines (VMs) should be dynamically added and removed on-demand. To manage these VMs efficiently, we need a special piece of software that is called the Hypervisor. These Hypervisors can be added to the existing Operating Systems. Some of the modern Server OSs come with the Hypervisor built into them. Ubuntu Server, Red Hat Enterprise Linux ship with KVM and Microsoft’s Windows Server 2008 Hyper-V Edition has a Hypervisor built into it. VMWare, the pioneer in virtualization has some of its Hypervisors like ESX 3.x which are Open Source. The most popular Hypervisor is an Open Source implementation called Xen. Xen is already shipped with the Linux Server editions of SUSE, Debian and few flavors of Red Hat Enterprise Linux. Because the VMs are the workhorses of the Cloud and Virtualization, the OS is just limited to booting up and running the Hypervisor. To avoid the overhead of the OS, Hypervisors are now shipped as standalone layers which do not need a separate OS to boot. Most of these standalone Hypervisiors are wrapped within Embedded Linux. They make the OS completely redundant and can also boot from a USB flash disk. It is hard to imagine this architecture without Linux and OSS forming the core. Though Microsoft had made its Hypervisor, Hyper-V Server free, you still need to invest in costly management software, Microsoft System Center Virtual Machine Manager (SCVMM) to administer, manage and monitor the VMs. The most successful commercial Cloud implementation, Amazon Web Services (AWS) run by Amazon is powered by the OSS Virtualization platform based on Xen.

After looking at the OSS Hypervisors, let’s look at the next layer in the stack, the VMs. VMs are just the virtualized instances of the typical servers that run in the Data Center. These server VMs represent the messaging, database, collaboration & portal, web and application servers. LAMP is undoubtedly the most popular stack that is powering widely used applications on the web including Facebook. When we sign up with Amazon EC2 to run the server instances in the Cloud, there is a huge collection of Amazon Machine Images (AMIs) built on Linux and FOSS and that are available at a nominal price. Remember that you only pay Amazon for the computing power and storage that you consumed and you need not worry about the license fee of the software that you use within these AMIs. For other commercial software, Amazon assumes that you have a valid license to run the software and you are completely liable and accountable for the software licenses. When you got to quickly build a prototype to demonstrate a customized portal app to your customer and you do not have free machines and expertise to setup a LAMP based application, don’t fret! Pickup an AMI from the community images running Ubuntu, Apache, PHP and MySQL along with an Open Source framework of your choice including Drupal, Joomla or WordPress. Just customize these AMIs to your need and store your new AMI on Amazon S3. You are all set to impress your customer.

Apart from the custom Line-of-Business (LOB) applications on LAMP, there are some really powerful frameworks built on OSS. A significant part of the web today runs on Open Source Content Management System (CMS) frameworks like WordPress, Drupal and Joomla.  To really exploit the power of on-demand availability of computing power, Apache foundation has released Hadoop. Hadoop is an Open Source framework to process huge datasets leveraging the computing power from dozens of servers. Hadoop enabled The New York Times to convert 4TB of raw TIFF data to an indexed, search-able digital archive of 11 million PDFs in just 24 hours costing them only $240! That is just unimaginable and only demonstrates what Open Source and the Cloud can achieve together.

The other area that is gaining ground is the Private Cloud. Private Cloud promises the benefits of the Cloud Computing while running in the enterprise data centers that are secured behind the firewall. The most popular Private Cloud implementation comes from Eucalyptus Systems. This was started as a research project by the Computer Science Department at the University of California, Santa Barbara before it was distributed through Ubuntu Server by Canonical that promotes Ubuntu and other OSS. The best thing that I personally like about Eucalyptus is its compatibility with Amazon Web Services. It supports the same WSDL as Amazon EC2 and S3. The API and VMs are completely compatible with AWS. I could install Eucalyptus on my Intel i7 Quad-Core, 6GB RAM home machine in a couple of hours to emulate the complete Cloud Controller, Cloud Cluster and Cloud Front-end on the same machine. This was a great experience in understanding the concepts of the Cloud. The other implementation of Private Cloud is from OpenNebula. I am yet to get my hands on this.

When the enterprise and the government want to have a combination of Private Cloud and Public Cloud to form the Hybrid Cloud, these OSS implementations will be really handy. Sensitive customer / patient / citizen data can reside on the Private Cloud thus respecting the privacy and adhering to the local regulations but can leverage the power of Cloud by moving the non-sensitive, compute intensive tasks to a Public Cloud like Amazon Web Services.

I am just scratching the surface of the possible benefits of the marriage between OSS and Cloud. There are many more aspects that I want to share in the coming days.

To catch the demo of OSS on the Cloud, do attend my session on ‘Cloud Computing and Open Source‘ at the Business Technology Summit on November 4th at Indian Institute of Science, Bangalore.


Related Posts

Top 5 Articles

Download The eBook - Demystifying The Cloud

I wanted to write a technical book for a long time. Though some of the well known publishers approached me, I never mustered enough courage to sign up and commit to them. With a full time job and a busy schedule, I am not sure if I would be able ...

Read More

Data on the Cloud - Amazon S3

Storage is one of the key resources offered as a Cloud Service. By moving data to the Cloud, there are quite a few benefits. Some of them include: 1. Illusion of unlimited storage - Unlike hosting, Cloud storage will virtually give you access to unlimited storage. You will never run out ...

Read More

Cloud Personas and Their Concerns

As one of the stakeholders of the Cloud ecosystem, I persuade, motivate and encourage decision makers in my circle to take a serious look at the Cloud. I want to share my experiences of having these conversations with the audiences. Off late, I came across a few IT Managers and technology ...

Read More

Top 10 Myths of Cloud Computing

As we step into the new year of 2010, I want to discuss the top 10 myths of Cloud Computing. Quite a few experts tried to debunk the myths earlier. My attempt is based on what I have seen or heard in my community. The concepts that I am bringing ...

Read More

Who Will Win The Cloud Computing Battle in India?

Till now I have discussed various technical aspects of the Cloud. Going forward I will also cover certain business aspects related to the Cloud. In this article, I want to analyze the opportunity for the Cloud vendors in India. We will first take a look at the ecosystem, the opportunity ...

Read More


  • @Jani
    I have created a UI -- when the initial Hyper-V Server released -- to create Virtual machines from the same command-line interface. Check it out @ http://hvs20008ui.codeplex.com. This works on both Hyper-V Server 2008 and 2008 R2. This can be used to create/manage VMs but for guest OS console access you still need another machine. That is the only drawback
  • JohnP
    I love how you leave out the cost of people researching and "playing with" cloud infrastructure when they could be doing something else. $240 - is unrealistic. A FTE is between $400-$2000 per day.

    People time is REAL money too.

    Don't get me wrong, private clouds are where most businesses should head and researching the solutions is necessary.
  • @Ravikanth - I agree with you. But my problem with the free Hyper-V Server is that it doesn't even let me create a basic VM. I am expected to build it outside and only deploy it. I also couldn't figure out the API that I want to use from a different machine. This may be supported on WS08 Hyper-V but I couldn't find it on the free version.

    Microsoft should also enable the customers to run their Private Cloud on the same Hypervisor that runs Azure. Having two different implementations doesn't win the trust.
  • Pretty well written.! Great way of creating the context, as always.

    My two cents: While embedded Hypervisors like ESX, etc provide a great value in implementing a private/public cloud, they (retail hypervisors) are not a "perfect" solution. Good example is how MS is tweaking Hyper-V for Windows Azure. A couple of years down the line, we may even see a Hypervisor implemented in the platform firmware (uEFI/EFI) and a standard API abstraction that can be used to run application irrespective of the OS they are designed for.
blog comments powered by Disqus

Previous post:

Next post: