Cool Things You Can Do with Shared EBS Snapshots

I’ve been awaiting this feature for a long time: Shared EBS Snapshots. Here’s a brief intro to using the feature, and some cool things you can do with shared snapshots. I also offer predictions about things that will appear as this feature gains adoption among developers.

How to Share an EBS Snapshot

Really, it’s easy. The first thing you’ll need to know is the Account Number of the user with whom you want to share the snapshot. If you want to make the snapshot public then you don’t need this. The account number can be found in the Your Account > Account Activity page. It’s in small numbers in the top-right of the page (so small you may need to click on the image below to see it in full size):

The person with whom you want to share the snapshot (you are the sharer, they are the “sharee”?) should tell you this 12-digit number. Don’t worry, sharee, it’s not a secret.

Once you have the sharee’s account number you, the sharer, go into the AWS Management Console and choose the Snapshots item. Find the snapshot you want to share and right-click on it, choosing “Snapshot Permissions”. You’ll get the following dialog:

Fill in the sharee’s account number, without the separating dashes, into the dialog, and hit “Save”. It should only take a few seconds and… presto! The snapshot should be visible in the sharee’s AWS Management Console Snapshots page.

Cool Things You Can Do with Shared Snapshots

Update 27 September 2009: Before you share snapshots publicly, read Eric Hammond’s warning about the dangers of doing so.

Easily move data between development, testing, and production

You’ve been keeping separate AWS accounts for your production environment, your testing environment, and your development environment, right? Right? Well, in case you haven’t, you no longer have any excuse not to do so. You can now share your database, your HDFS volumes (if you use Cloudera’s Hadoop distribution with EBS support), and anything else of significant size between these separate accounts. No more “tar, gzip, split into < 5GB chunks, upload to S3” and “download from S3, concatenate, untar-gzip”. Your data is ready to go with the newly-created volume.

Share entire setups for troubleshooting and support

If you support a product that is deployed in EC2 you no longer need to jump through hoops to get access to your customer’s files when there’s a problem. Simply have them put the relevant files into an EBS volume, snapshot it, and share the snapshot with you.

Deliver your application in a more granular manner

Until today you delivered your application as an AMI – perhaps even a DevPay AMI – and you may not have given your customers root access. But, if your application used less than 100% of an instance’s CPU, the customer was stuck paying for an entire CPU. Now, you can distribute your applications as a shared snapshot instead, and your customers will be free to use the rest of the instance’s CPU. You’ll just need to build a way to manage access, only allowing authorized customers to see the snapshot.

Deliver you customer’s results in a more usable format

If you run a service that provides large amounts of data, you no longer need to use S3 to share the results. Until today you had to store the results in S3, and your customer needed to retrieve the results from S3 in order to use them. No longer: now you can provide a shared snapshot of the results, and the customer can access them via their filesystem more simply. “The shared snapshot is the new bucket.”

Mount a volume created from a shared snapshot at startup

In a previous article I explained how to automatically mount an EBS volume created from a snapshot during the instance’s startup sequence. I provided a script that gets the snapshot ID via the user-data and does all the rest automatically. Now you can also use snapshots that have been shared.

Update 25 September 2009: Share entire machines

Reader Robert Staveley (Tom) comments below about his use for shared snapshots: Sharing entire machines – boot code and everything – between development, testing, and production accounts. Using the technique to boot an instance from an EBS volume he points out that the entire bootable hard drive and all applications (even beyond 10GB) can be shared between these accounts.

Things to Expect in the Future

Shared snapshots are still a very new feature, but here are some things I expect to happen now that this is possible.

• The AWS Management Console is the only UI that allows you to share a snapshot. ElasticFox will be adding this capability Real Soon Now, and I am sure others will as well.
• Alternatives to AMIs. AMIs have many limitations, such as the 10GB maximum size, that can be circumvented using a technique I described to boot from an EBS volume. I expect to see OS distributions packaged as a shared EBS snapshot. These distributions could all share a common AMI containing just enough code to create a volume from the shared distribution snapshot, mount it, and boot from it. No more headaches bundling an AMI – just share a new bootable EBS snapshot.
  Update 3 December 2009: This prediction has come true, with AWS’s release of EBS-backed AMIs.
• Payment gateway services for managing access to shared snapshots. Now that you’re distributing software as a shared snapshot you’ll need to manage access to the snapshot, limiting it to authorized customers. You might build that system yourself today, but soon we’ll see third-party services that do this for you.
  Update 19 April 2012: This prediction has come true, with AWS’s release of the AWS Marketplace.

Do you have other cool uses or predictions for shared snapshots? Please comment!