Cloud-Developer-Tips

This is a true story about a lot of data. The cast of characters is as follows: The Protagonist: Me. The Hero: Jets3t, a Java library for using Amazon S3. The Villain: Decisions made long ago, for forgotten reasons. Innocent Bystanders: My client. Once Upon a Time… Amazon S3 is a great place to store media files and allows these files to be served directly from S3, instead of from your web server, thereby saving your server’s network and CPU for more important tasks. There are some gotchas with serving files directly from S3, and it is these gotchas that had my client locked in to paying for bandwidth and CPU to serve media files directly from his web server. ...

At long last, the most oft-requested feature for EC2’s Elastic Load Balancer is here: session affinity, also known as “sticky sessions”. What is session affinity? Why is this feature in such high demand? How can it be used with existing applications? Let’s take a look at these questions. But first, let’s explore what a session is - then we’ll cover why we want it to be sticky, and what ELB’s sticky session limitations are. [To skip directly to an explanation of how to use ELB sticky sessions, go toward the bottom of the article.] ...

You may know that Amazon Web Services Reserved Instances have some gotchas. One of these gotchas is that the availability zone in which the reservation is purchased cannot be changed. So if you need to use an instance in a different availability zone (e.g. if you hit InsufficientCapacity errors*) than your reservation, you’re out of luck - and you end up paying the on-demand price. Today at CloudConnect I chatted with Jeremy Edberg of Reddit and Joe Arnold of Cloudscaling, and we had an insight: there’s a workaround. Read on for more details. ...

You’re a developer - or you want to be - writing software that uses cloud computing. Why should you care about the CloudConnect Conference? Here are the top ten reasons why it’s worth your while. Full disclosure: Nothing to disclose. I have no commercial connection to CloudConnect or its producers. #10: Snacks Enjoy short, sweet, thought-provoking keynotes about applications, metrics, SaaS, and more. I happen to know of three keynote presenters who will be particularly interesting: William Louth, William Vambenepe, and Guy Rosen. Come hear them challenge the way you think about cloud applications, environments, and infrastructure. ...

Eric Hammond has taught us how to create consistent snapshots of EBS volumes. Amazon has allowed us to use EBS snapshots as AMIs, providing a persistent root filesystem. Wouldn’t it be great if you could use both of these techniques together, to take a consistent snapshot of the root filesystem without stopping the instance? Read on for my instructions how to create an XFS-formatted boot-from-EBS AMI, allowing consistent live snapshots of the root filesystem to be created. ...

This is one of the coolest uses of Amazon’s ELB I’ve seen yet. Check out James Elwood’s article. You may know that you can’t serve more than one SSL-enabled domain on a single EC2 instance. Okay, you can but only via a wildcard certificate (limited) or a multi-domain certificate (hard to maintain). So you really can’t do it properly. Serving multiple SSL domains is one of the main use cases behind the popular request to support multiple IP addresses per instance. ...

S3 has an “eventual consistency” model, which presents certain limitations on how S3 can be used. Today, Amazon released an improvement called “read-after-write-consistency” in the EU and US-west regions (it’s there, hidden at the bottom of the blog post). Here’s an explanation of what this is, and why it’s cool. What is Eventual Consistency? Consistency is a key concept in data storage: it describes when changes committed to a system are visible to all participants. Classic transactional databases employ various levels of consistency, but the golden standard is that after a transaction commits the changes are guaranteed to be visible to all participants. A change committed at millisecond 1 is guaranteed to be available to all views of the system - all queries - immediately thereafter. ...

Today I participated in the Cloud Standards & Interoperability panel at the IGT2009 conference, together with Shahar Evron of Zend Technologies, and moderated by Reuven Cohen. Reuven gave an overview of his involvement with various governments on the efforts to define and standardize “cloud”, and Shahar presented an overview of the Zend Simple Cloud API (for PHP). I presented an overview of the Open Grid Forum’s Open Cloud Computing Interface (OCCI). The slides include a 20,000-foot view of the specification, a 5,000-foot view of the specification, and an eye-level view in which I illustrated the metadata travelling over the wire using the HTTP Header rendering. ...

Recently I answered a question on the EC2 forums about how to give third parties access to EC2 instances. I noticed there’s not a lot of info out there about how to work with contractors, consultants, or even internal groups to whom you want to grant access to your AWS account. Here’s how. First, a Caveat Please be very selective when you choose a contractor. You want to make sure you choose a candidate who can actually do the work you need - and unfortunately, not everyone who advertises as such can really deliver the goods. Reuven Cohen’s post about choosing a contractor/consultant for cloud projects examines six key factors to consider: ...

You’re a developer writing applications that use the cloud. Your code manipulates cloud resources, creating and destroying VMs, defining storage and networking, and gluing these resources together to create the infrastructure upon which your application runs. You use an API to perform these cloud operations - and this API is specific to the programming language and to the cloud provider you’re using: for example, for Java EC2 applications you’d use typica, for Python EC2 applications you’d use boto, etc. But what’s happening under the hood, when you call these APIs? How do these libraries communicate with the cloud? What language does the cloud speak? ...