Lessons Learned from Using Multiple Cloud APIs

Adrian Cole, author of the jClouds library, has an excellent writeup of the trajectory the library’s development followed as it added support for more cloud providers’ APIs.

[Update August 2011: Blogger.com has removed Adrian’s blog so that link no longer works.]

Some important takeaways for application developers:

• Your unit tests are as valuable as your code. The tests ensure the code works to spec, and they should be used as frequently as possible during development.
• Make your code easy to test, with sensible defaults that require no external dependencies: e.g. don’t require internet connectivity.
• Cloud  limitations, both general (such as eventual consistency) and cloud-specific (such as a limit on the number of buckets per S3 account), will require careful consideration in your code (and tests).
• Some things can only really be tested against a live cloud service. As Adrian points out, the only way to test that an instance launched with the desired customizations is to ssh in to that instance and explore it from the inside. This is not testable using an offline stub emulator.

But the key lesson developers can learn is: Whenever possible, use an existing library to interface with your clouds. As Adrian’s post makes patently clear, a lot of effort goes into ensuring the library works properly with the various supported APIs, and you can only benefit by leveraging those accomplishments.

On the other side of the fence, API developers can also learn from Adrian’s article. As William Vambenepe recently commented:

Rather than spending hours obsessing about the finer points of your API, spend the time writing love letters to [boto author] Mitch and Adrian so they support you in their libraries.

In fact, Adrian’s blog can be viewed as a TODO list for API creators who want to encourage adoption.

API authors should also refer to Steve Loughran’s Cloud Tools Manifesto for more great ideas on how to make life easy for developers.