Scalability and HA Limitations of AWS Marketplace AMIs

Reading AWS’s recent announcement of the AWS Marketplace you would think that it provides a catalog of click-to-deploy, highly-available, scalable applications running on EC2. You’d be partially right: the applications available in the AWS Marketplace are deployable in only a few clicks. But highly-available and scalable services will be difficult to build using Marketplace images. Here’s why. Essential Ingredients of HA and Scalability on AWS AWS makes it easy to run scalable, HA applications via several features. Not all applications use all of these features, but it would be very difficult to provide scalable and highly available service without using at least one of these: ...

April 25, 2012 · 10 min · shlomo

Storing AWS Credentials on an EBS Snapshot Securely

Thanks to reader Ewout and his comment on my article How to Keep Your AWS Credentials on an EC2 Instance Securely for suggesting an additional method of transferring credentials: via a snapshot. It’s similar to burning credentials into an AMI, but easier to do and less prone to accidental inclusion in the application’s AMI. Read on for a discussion of how to implement this technique. How to Store AWS Credentials on an EBS Snapshot This is how to store a secret on an EBS snapshot. You do this only once, or whenever you need to change the secret. ...

July 19, 2010 · 5 min · shlomo

Creating Consistent Snapshots of a Live Instance with XFS on a Boot-from-EBS AMI

Eric Hammond has taught us how to create consistent snapshots of EBS volumes. Amazon has allowed us to use EBS snapshots as AMIs, providing a persistent root filesystem. Wouldn’t it be great if you could use both of these techniques together, to take a consistent snapshot of the root filesystem without stopping the instance? Read on for my instructions how to create an XFS-formatted boot-from-EBS AMI, allowing consistent live snapshots of the root filesystem to be created. ...

January 17, 2010 · 27 min · shlomo

Cool Things You Can Do with Shared EBS Snapshots

I’ve been awaiting this feature for a long time: Shared EBS Snapshots. Here’s a brief intro to using the feature, and some cool things you can do with shared snapshots. I also offer predictions about things that will appear as this feature gains adoption among developers. How to Share an EBS Snapshot Really, it’s easy. The first thing you’ll need to know is the Account Number of the user with whom you want to share the snapshot. If you want to make the snapshot public then you don’t need this. The account number can be found in the Your Account > Account Activity page. It’s in small numbers in the top-right of the page (so small you may need to click on the image below to see it in full size): ...

September 24, 2009 · 6 min · shlomo

Mount an EBS Volume Created from Snapshot at Startup

There are many posts about how to mount an EBS volume to your EC2 instance during the startup process. But requiring an instance to use a specific EBS volume has limitations that make the technique unsuitable for large-scale use. In this article I present a more flexible technique that uses an EBS snapshot instead. Update December 2009: Amazon released native support for automatically mounting EBS volumes created from a snapshot, as part of supporting the boot-from-EBS feature. The techniques in this article are no longer necessary. But they’re cool anyway. ...

August 7, 2009 · 22 min · shlomo

Copying ElasticFox Tags from One Browser to Another

The ElasticFox Firefox extension allows you to tag EC2 instances, EBS volumes, EBS snapshots, Elastic IPs, and AMIs. ElasticFox’s tags are stored locally within Firefox, so if you use ElasticFox from more than one browser your tags from one browser are not visible in any other browser. In this article I demonstrate how to copy tags from one ElasticFox installation to another. Where Are ElasticFox Tags Stored? My article about Tagging EC2 Instances Using Security Groups shows a brief walkthrough of using ElasticFox to tag instances. Under the hood, ElasticFox stores tags in the prefs.js file within your Firefox Profile directory. But instead of poking around in there directly, take a look at Firefox’s built-in interface to these preferences. It’s the about:config screen, and you get there by typing about:config into the address bar: ...

July 15, 2009 · 8 min · shlomo

Boot EC2 Instances from EBS

EBS offers the ability to have a “virtual disk drive” whose lifetime is independent of any EC2 instance. You can attach EBS drives to any running instance, and any changes made to the drive will persist even after the instance is terminated. You can also set up an EBS drive to be the root filesystem for an instance - giving you the benefits of an always-on instance but without paying for it when it’s not in use - and this article shows you how to do that. I also explore how to estimate the cost savings you can achieve using this solution. ...

July 14, 2009 · 23 min · shlomo