Submitted by Matthias Marschall on Thu, 08/21/2008 - 21:33
Cloud Computing is a Very Hyped Topic
1. The "Plumbing"
Renting physical "root" servers, you don't have to deal with the "plumbing" of your physical servers any more. Things like power, cooling, connectivity, physical access security, rack space, ordering and maintaining of servers, etc. the hosting company does for you. Operating the data center is already outsourced, but everything else has to be done by you. This is our current level of operations.
We have to deal with physical boxes: Make sure they can talk to each other by setting up routing between the boxes. And we use them as our physical infrastructure for running our application in virtual servers. That means that we have to build all our infrastructure for virtualization on our physical layer: Storage, connectivity, Xen enabled kernels and everything else required to run a Virtual Infrastructure (see below) on top of our physical servers. For example, we use disk images connected to loop devices stuffed into LVM volumes as storage for our Xen based virtual machines. And we use bridged routing over a VPN between the physical hosts to enable the Xen based virtual machines to talk to each other. All that has to be managed and to be kept running all the time.
To me, Virtual Infrastructure is everything concerned with dealing with your virtual servers. You need to start/stop/restart them. You need to migrate them to different physical hosts and you might want to even automatically scale up (a.k.a start new instances) under higher load. I came up with a set of capistrano tasks for managing our Virtual Infrastructure. I have one capistrano task for extending the storage for a virtual machine by adding a new disk image to the LVM volume. I have another capistrano task for migrating a virtual machine from one physical host to the other by stopping it, copying over all disk images and starting it again. Lacking real network storage, our setup is eventually destined to suffer from I/O issues. This makes me long for a solution which would help me deal with managing the Virtual Infrastructure e.g by providing me with shared storage, a VLAN for networking and some way of starting/stopping/restarting our virtual servers.
In our setup, every virtual server has a dedicated role: Web server, application server, memcached server, db server, etc. These dedicated virtual servers act as appliances for their specific tasks. Encapsulating every role like that enables us to e.g. switch from a single MySQL instance to a Master-Master Cluster without having to change anything on the other appliances. This greatly reduces the complexity and risk of upgrading any appliance. Our development team is also used to thinking about and working with appliances. They don't care about the virtual or underlying physical infrastructure. And they care even less about the "plumbing" of the physical servers. All they have to deal with are web, app, mem and db appliances.
At the uppermost level in our production stack lives the application. Included in this level are basic tasks such as deploying new releases, making sure the required rubygems are available and ensuring the proper running of cron jobs. All these things only have to know about the configuration of the appliances: where do I find the database, from where to where shall I synchronize images, etc.
About: Matthias Marschall (co-editor)
Having started with Intranet applications in the nineties, I moved on to developing B2B web applications at the turn of the millenium. Today, I'm building and running a community driven web site: autoplenum.de. My journey has gradually brought me closer to embracing more operational responsibilities in addition to development and team leading.