Amazon Web Services provide a free usage tier to allow new users to trial many aspects of the offering for 12 months without charge. There are many benefits to this tier, and it is ideally suited to small-scale deployments or for just learning about AWS and EC2. At the time of writing, among the benefits of this tier are 750 hours of compute time for a t1.micro instance, along with 30GB of elastic block store (EBS) storage. So - I can run up 750 hours of CPU time using as many t1.micro instances as I want, as long as I stay within various other confines imposed by the tier.

30GB free EBS storage doesn’t go far if you use EBS-backed instances. EBS provides persistent storage, and with EBS-backed instances, the ability to shutdown and start up your instances. The alternative to the elastic block store is the instance store. Instance store-backed instances are launched, and must be left running. The storage is ephemeral and all changes will be lost (well - the entire instance is lost) when the instance is terminated. Note the terminology - terminated means just that - that the instance will be halted, and marked for deletion. The only way to have persistence is to use EBS-backed AMIs, which is obviously my preference.

This article will cover the provisioning of instances and volumes via the command line. For this, the Amazon EC2 API Tools can be used. These tools are Java-based, so an appropriate JDK must be used.

Essentially, the EC2 tools interact with the Amazon EC2 API using X.509 certificate-based authentication, and provide functionality much the same as the web-based EC2 Management Console. This allows for quick interaction with instances, automated monitoring, automation ability, and so on.

The article assumes that you have already signed up for Amazon Web Services, familiarised yourself with the concepts and functionality of the EC2 Management Console, and have generated and downloaded the following:

• X.509 Certificate Pair (Public Certificate, Private Key)
• SSH Private Key

You can create your X.509 certificate pair and download it via the AWS Management Console -> Your account name in menu bar -> Security Credentials. You can generate your SSH Private Key via the EC2 Management Console -> Network & Security -> Key Pairs. The SSH Public Key will automatically be installed on any instances provisioned so that you can log in.

Installing EC2 Tools

I’m using CentOS 6.3 x86_64 as my management/admin server and the host on which I’ll be installing the tools. Unfortunately, there isn’t a package available in the standard repositories but this isn’t an issue as I’d prefer to obtain the very latest release directly from Amazon anyway.

Prior to grabbing the API tools, however, you’ll need a JDK installed. I downloaded the latest Oracle 1.7 JDK to /usr/local/src, and installed it as follows:

Next, I downloaded the API tools to /usr/local/src and installed them:

For EC2 administration, I created a separate user. I’ve seen a few articles with people charging around as root doing this - do not be one of those people. All the EC2 API commands can be executed as a standard unprivileged user.

Switch user to ec2 and create a directory to hold your certificate files, and your .ssh directory, with appropriate permissions:

Copy your authentication files into place:

Edit your shell’s start up environment as follows (I’ll presume bash is being used):

Source .bashrc:

Run an ec2-api-tools command to ensure everything is working:

If, instead of seeing the wonders above, you see

check that the time and date on your machine is set correctly.

Running an Instance

Terminology again - we don’t create an instance, we run one. That makes sense for instance-store backed instances. I’ll use a Ubuntu 12.10 i386 EBS AMI. You can take a look at the Ubuntu Amazon EC2 AMI Locator - I selected ami-08df4961 which was correct for my intended t1.micro instance in availability zone us-east-1. I also specified the name of the key pair I created when generating my SSH keys as per the introduction.

You can see that the instance is in state pending, that the instance backing type is ebs (scroll across the INSTANCE row above) and that its instance ID is i-124d8661. After a few moments, the instance will be provisioned, and information regarding the instance obtained from the ec2-describe-instances command:

A lot of information can be seen in the INSTANCE row now that the instance is running - including public and private IP addresses, and a status of running. The block device is presented to the OS as /dev/sda1. As these are xen guests (check INSTANCE again) this actually presents as /dev/xvda1 to the OS. It’s worth noting here that EBS storage can be provisioned and attached to running instances without issues. No partprobe-ing either …

It also helps to add a tag to an instance so that each has a meaningful name. You’ll see, if you look via the EC2 Management Console at the Instances page, that your new instance doesn’t have a name. If you run multiple instances, without a human-readable name you’re heading for confusion.

Refresh the UI, and you’ll see that the instance now has a name of testserver. You can also confirm this with ec2-describe-instances:

Modifying Security Group

The default security group does not allow ingress from anywhere other than your instances:

In order to SSH in to instances, a new permission needs to be added. Once you provision whatever services you’re placing on the instance, you’d need to add further permissions to the group too. You can create a new security group (recommended for a production roll-out), but I’ll just modify the default for now:

Port 22/tcp is now open from 1.2.3.4/32 - substitute this for your real source IP.

Logging In

With the Ubuntu AMIs, the default username is ubuntu, which has full privileges over the instance via sudo. The first thing you’ll want to do is disable/remove that account after setting up a suitable replacement and ensuring the keys work as you’d expect. First, log in using the default account and specifying the path to your SSH private key, and the IP address / hostname from the ec2-describe-instances command above:

Once logged in, perform the following steps:

Before disabling the ubuntu account, check that the new account works correctly by SSH-ing in from your admin server:

If you get root privileges, delete the ubuntu account password:

Adding EBS Storage

Let’s add a 5GB EBS volume to our running instance. First, create the volume:

It is important that you create your volume in the same availability zone as your instances. My volume has been created with volume ID vol-fcbb898d. You can detach an EBS from one running instance, and attach it to another running instance.

Next, attach the volume to the instance:

I specify /dev/sdb as the OS device path, which will actually be /dev/xvdb under xen. If you run ec2-describe-instances now, you’ll see two block devices associated with the instance:

Log back into your instance via SSH, and run fdisk -l:

Excellent - the new device is there. You can go ahead and create a filesystem on it, place some files here and there, reboot the instance, etc. Everything will persist. Use apt-get to install some packages, configure your services, migrate your site over to it, etc. (There will be articles around all this coming soon.)

Starting/Stopping/Terminating Instances

With EBS-backed instances, you can stop and start them, as well as terminate. You should only terminate your instances if you are happy to lose all data from the root block device - i.e. the OS and anything installed to that device.

With instance-store backed instances, you can only terminate. If you wish to terminate your instance you can run:

The instance will then be marked for deletion, and the root EBS block device deleted. Once it’s gone, it’s gone. If you want to delete a volume (such as the additional EBS volume added above), use:

If your volume is already in use by an instance, unmount as usual from within the instance, and then run the following from your admin server:

Once detached, you can delete if required.

Conclusion

There is much more to EC2, and obviously a whole gamut of offerings under the entire AWS suite, but this article should have you running up instances and delving into cloud-based computing.

I have 12 months of free usage up my sleeve, so prepare for a lot more articles around AWS/EC2, particularly involving ec2-api-tools, and the provisioning of various services on Ubuntu-based instances.