Cloud Computing-Amazone EC2 S3

by jagbir on April 18, 2008

So you need a new server to test your project, and for testing its scalability you wish to have 100 servers running under your control for 10 hours. You wish to pay money for these servers as long as you actually use them, not for their ideal time. After 10 hours, you would like to throw them out. Sounds Interesting? But with usual dedicated/co-location hosting providers, its not possible. They require you to place an order, pay monthly/weekly commitment, wait for 1-4 days irrespective of the fact that you need machines only for 10 hours.

Now enter in the world of Cloud computing. Hire a single server or 1000 servers instantly. Pay the charges per hour per server. Throw them out when your job done. No need for any hectic ordering. No commitment. Get them again instantly when need arise. Store Unlimited data (yes, truly Unlimited) with guaranteed reliability and scalability. Thats the beauty of virtual world.

An interesting fact is that Amazon, a company commonly known for E-Commerce, Online shopping is the front runner for providing cloud computing services. Amazon Web Services (AWS) incorporates several web services which are able to change the way you think about web applications. Here I’m going to provide an overview of only Amazon services, but be sure that there are lots of other companies which provide same sort of services, such as, etc.

Amazon Elastic Computing Cloud (EC2):
This service give you capability to get a new server machine and pay hourly for it, as low as $ 0.10 per hour. Just terminate this server when your task get completed. Every server is known as Instance. An instance is actually a virtual machine running in a powerful physical machine. This technology commonly known as virtualization. Depending on the type of instance, a single physical machine can run multiple instances but all instances will be independent to each other and run like a physical server. You get full access to your server to do anything with it as you do in your physical servers like running a web server, app server or db server etc. You can create your own scripts to get a new instance ready when you think that current instance is getting busy and load should be shared. You can manage your instance with easy interfaces such as through firefox extension. Though, some restrictions are still there, like when you terminate your instance, all the data/apps stored in instance will be destroyed. So means an instance does not have persistent storage then what happened if you just configured full flagged server after 3 hours of struggle and shut it down by mistake or it become unresponsive leaving single choice with you to terminate it? Not much problem, configure a server, create its Image (AMI) and upload it to persistent storage (provided by another service). Now every time you need a new server with exact same configurations, just create new instance from that Machine Image. Images are somewhat like old Norton Ghost images which enables you to replicate same contents to different machines very easily.

Simple Storage Service (S3):
A storage location with unlimited capacity and full proof reliability. Thats S3 service from Amazon. To start storing of files/objects, you need to create a bucket. A bucket is somewhat similar to a directory. Store everything in this bucket or create another bucket for different set of files. Number of files in a bucket is unlimited but at present you can only store a file upto 5 GB of size in it. Access these files directly from a URL. Generally, both EC2 and S3 services are used together. Store your permanent data from EC2 Instances to S3. The traffic between EC2 and S3 is free. Traffic between an instance to another instance is also free in case both are in same availability zone which is by default they would be. Traffic to outside world from S3, EC2 is charged with very affordable rates.

The main benefit of these services is affordable and flexibility without any compromise of quality. Even Though most of these services are still in beta, more and more companies are making use of these service. Just for example, I, myself using more than 5 instances daily and our several TB of data is stored in S3. Our plan is to move the entire setup to AWS is on its way.

Other useful services are also there, like SQS (simple Queue Service) and SDB (Simple DataBase). SDB is used to store data in a database like format. Its reliable and scalable. The structure is entirely flat one and can not be used as pure relational database but still its very useful in specific situations. All of the services can be integrated with your applications by using their powerful APIs available in almost all popular languages like Perl, PHP, Ruby, C#, Java etc.

Getting started is easy. Just register yourself with AWS and choose your services. Test them and I’m sure your mind will quickly calculates the possible savings and think towards making use of them in your business.

Next post: