Using AWS Route 53 to Keep Track of EC2 Instances

This article is a guest post by Guy Rosen, CEO of Onavo and author of the Jack of All Clouds blog. Guy was one of the first people to produce hard numbers on cloud adoption for site hosting, and he continues to publish regular updates to this research in his State of the Cloud series. These days he runs his startup Onavo which uses the cloud to offer smartphone users a way to slash overpriced data roaming costs.

In this article, Guy provides another technique to track changes to your dynamic cloud services automatically, possible now that AWS has released Route 53, DNS services. Take it away, Guy.

While one of the greatest things about EC2 is the way you can spin up, stop and start instances to your heart’s desire, things get sticky when it comes to actually connecting to an instance. When an instance boots (or comes up after being in the Stopped state), Amazon assigns a pair of unique IPs (and DNS names) that you can use to connect: a private IP used when connecting from another machine in EC2, and a public IP is used to connect from the outside. The thing is, when you start and stop dozens of machines daily you lose track of these constantly changing IPs. How many of you have found, like me, that each time you want to connect to a machine (or hook up a pair of machines that need to communicate with each other, such as a web and database server) you find yourself going back to your EC2 console to copy and paste the IP?

This morning I got fed up with this, and since Amazon launched their new Route 53 service I figured the time was ripe to make things right. Here’s what I came up with: a (really) small script that takes your EC2 instance list and plugs it into DNS. You can then refer to your machines not by their IP but by their instance ID (which is preserved across stops and starts of EBS-backed instances) or by a user-readable tag you assign to a machine (such as “webserver”).

Here’s what you do:

1. Sign up to Amazon Route 53.
2. Download and install cli53 from https://github.com/barnybug/cli53 (follow the instructions to download the latest Boto and dnspython)
3. Set up a domain/subdomain you want to use for the mapping (e.g., ec2farm.mycompany.com):
   1. Set it up on Route53 using cli53:
      ./cli53.py create ec2farm.mycompany.com
   2. Use your domain provider’s interface to set Amazon’s DNS servers (reported in the response to the create command)
   3. Run the following script (replace any details and paths, emphasized in bold, with your own):
      

      #!/bin/tcsh -f
set root=`dirname $0`
setenv EC2_HOME /usr/local/ec2-api-tools
setenv EC2_CERT $root/ec2_x509_cert.pem
setenv EC2_PRIVATE_KEY $root/ec2_x509_private.pem
setenv AWS_ACCESS_KEY_ID myawsaccesskeyid
setenv AWS_SECRET_ACCESS_KEY mysecretaccesskey

      $EC2_HOME/bin/ec2-describe-instances | \
perl -ne '/^INSTANCE\s+(i-\S+).*?(\S+\.amazonaws\.com)/ \
and do { $dns = $2; print "$1 $dns\n" }; /^TAG.+\sShortName\s+(\S+)/ \
and print "$1 $dns\n"' | \
perl -ane 'print "$F[0] CNAME $F[1] --replace\n"' | \
xargs -n 4 $root/cli53/cli53.py \
rrcreate -x 60 ec2farm.mycompany.com

Voila! You now have DNS names such as i-abcd1234.ec2farm.mycompany.com that point to your instances. To make things more helpful, if you add a tag called ShortName to your instances it will be picked up, letting you create names such as dbserver2.ec2farm.mycompany.com. The script creates CNAME records, which means that you will automatically get internal EC2 IPs when querying inside EC2 and public IPs from the outside.

Put this script somewhere, run it in a cron – and you’ll have an auto-updating DNS zone for your EC2 servers.

Short disclaimer: the script above is a horrendous one-liner that roughly works and uses many assumptions, it works for me but no guarantees.