Elastic Load Balancing with Sticky Sessions

At long last, the most oft-requested feature for EC2’s Elastic Load Balancer is here: session affinity, also known as “sticky sessions”. What is session affinity? Why is this feature in such high demand? How can it be used with existing applications? Let’s take a look at these questions. But first, let’s explore what a session is – then we’ll cover why we want it to be sticky, and what ELB’s sticky session limitations are. [To skip directly to an explanation of how to use ELB sticky sessions, go toward the bottom of the article.]

What is a Session?

A session is a way to get your application involved in a long-lasting conversation with a particular client. Without a session, a conversation between your application and a client would  look like something straight out of the movie Memento. It would look like this:

Life Without Sessions

Client: Hi, I’d like to see /products/awesomeDoohickey.html

Application: I don’t know who you are. Please go here to login first: /login

Client: OK, I’d like to see /login

Application: Here it is: “…”

Client: Thanks. Here’s the filled in login form.

Application: Thanks for logging in. Where do you want to go?

Client: I’d like to see /products/awesomeDoohickey.html

Application: I don’t know who you are. Please go here to login first: /login

Client: >Sigh< OK, I’d like to see /login

Application: Happily! Here it is: “…”

Client: Here’s the filled in login form.

Application: Thanks for logging in. Where do you want to go?

Client: Show me /products/awesomeDoohickey.html already!

Application: I don’t know who you are. Please go here to login first: /login

Client: *$#%& this!

The application can’t remember who the client is – it has no context to process each request as part of a conversation. The client gets so frustrated he starts thinking he’s living in an Adam Sandler movie.

On a technical level: Each HTTP request-response pair between the client and application happens (most often) on a different TCP connection. This is especially true when a load balancer sits between the client and the application. So the application can’t use the TCP connection as a way to remember the conversational context. And, HTTP itself is stateless: any request can be sent at any time, in any sequence, regardless of the preceding requests. Sure, the application may demand a particular pattern of interaction – like logging in before accessing certain resources – but that application-level state is enforced by the application, not by HTTP. So HTTP cannot be relied on to maintain conversational context between the client and the application.

There are two ways to solve this problem of forgetting the context. The first is for the client to remind the application of the context every time he requests something: “My name is Henry Whatsisface, I have these items in my shopping cart (…), I got here via this affiliate (…), yada yada yada… and I’d like to see /products/awesomeDoohickey.html”. No sane client would ever agree to interact with an application that needed to be sent the entire context at every stage of the conversation. Its burdensome for the client, it’s difficult to maintain for the application, and it’s expensive (in bandwidth) for both of them. Besides, the application usually maintains the conversational state, not the client. So it’s wrong to require the client to send the entire conversation context along with each request.

The accepted solution is to have the application remember the context by creating an associated memento. This memento is given to the client and returned to the application on subsequent requests. Upon receiving the memento the application looks for the associated context, and – voila – discovers it. Thus, the conversation is preserved.

One way of providing a memento is by putting it into the URL. It looks really ugly when you do this: http://www.example.com/products/awesomeDoohickey.html?sessionID=0123456789ABCDEFGH

More commonly, mementos are provided via cookies, which all browsers these days support. Cookies are placed within the HTTP request so they can be discovered by the application even if a load balancer intervenes.

Here’s what that conversation looks like with cookies:

Life With Sessions, Take 1

Client: Hi, I’d like to see /products/awesomeDoohickey.html

Application: I don’t know who you are. Please go here to login first: /login

Client: OK, I’d like to see /login

Application: Here it is: “…”

Client: Thanks. Here’s the filled in login form.

Application: Thanks for logging in. Here’s a cookie. Where do you want to go?

Client: I’d like to see /products/awesomeDoohickey.html and here’s my cookie.

Application: I know you – I’d recognize that cookie anywhere! Great, here’s that page: “…”

Client: I’d like to buy 5000 units. Here’s my cookie.

Much improved, yes?

A side point: most modern applications will provide a cookie earlier in the conversation. This allows the following more optimal conversation:

Life With Sessions, Take 2

Client: Hi, I’d like to see /products/awesomeDoohickey.html

Application: I don’t know who you are. Here’s a cookie. Take this login page and fill it out: “…”

Client: OK. Here’s the filled in login form. And here’s my cookie.

Application: I know you – I’d recognize that cookie anywhere! Thanks for logging in. I recall you wanted to see /products/awesomeDoohickey.html. Here it is: “…”

Client: I’d like to buy 5000 units. Here’s my cookie.

That’s about as optimized a conversation as you can have. Cookies make it possible.

What is Session Affinity (Sticky Sessions)? Why is it in High Demand?

When you only have one application server talking to your clients life is easy: all the session contexts can be stored in that application server’s memory for fast retrieval. But in the world of highly available and scalable applications there’s likely to be more than one application server fulfilling requests, behind a load balancer. The load balancer routes the first request to an application server, who stores the session context in its own memory and gives the client back a cookie. The next request from the same client will contain the cookie – and, if the same application server gets the request again, the application will rediscover the session context. But what happens if that client’s next request instead gets routed to a different application server? That application server will not have the session context in its memory – even though the request contains the cookie, the application can’t discover the context.

If you’re willing to modify your application you can overcome this problem. You can store the session context in a shared location, visible to all application servers: the database or memcached, for example. All application servers will then be able to lookup the cookie in the central, shared location and discover the context. Until now, this was the approach you needed to take in order to retain the session context behind an Elastic Load Balancer.

But not all applications can be modified in this way. And not all developers want to modify existing applications. Instead of modifying the application, you need the load balancer to route the same client to the same application server. Once the client’s request has been routed to the correct application server, that application server can lookup the session cookie in its own memory and recover the conversational context.

That’s what sticky sessions are: the load balancer routing the same client to the same application server. And that’s why they’re so important: If the load balancer supports sticky sessions then you don’t need to modify your application to remember client session context.

How to Use ELB with Sticky Sessions with Existing Applications

The key to managing ELB sticky sessions is the duration of the stickiness: how long the client should consistently be routed to the same back-end instance. Too short, and the session context will be lost, forcing the client to login again. Too long, and the load balancer will not be able to distribute requests equally across the application servers.

Controlling the ELB Stickiness Duration

ELB supports two ways of managing the stickiness’ duration: either by specifying the duration explicitly, or by indicating that the stickiness expiration should follow the expiration of the application server’s own session cookie.

If your application server has an existing session cookie already, the simplest way to get stickiness is to configure your ELB to use the existing application cookie for determining the stickiness duration. PHP applications usually have a session cookie called PHPSESSID. Java applications usually have a session cookie called JSESSIONID. The expiration of these cookies is controlled by your application, and the stickiness expiration can be set to match as follows. Assuming your load balancer is called myLoadBalancer and it has an HTTP listener on port 80:

elb-create-app-cookie-stickiness-policy myLoadBalancer --cookie-name PHPSESSID --policy-name followPHPPolicy
elb-set-lb-policies-of-listener myLoadBalancer --lb-port 80 --policy-names followPHPPolicy

The above commands create a stickiness policy that says “make the session stickiness last as long as the cookie PHPSESSID does” and sets the load balancer to use that stickiness policy. Behind the scenes, the ELB’s session cookie will have the same lifetime as the PHPSESSID cookie provided by your application.

If your application does not have its own session cookie already, set your own stickiness duration for the load balancer, as follows:

elb-create-lb-cookie-stickiness-policy myLoadBalancer --policy-name fifteenMinutesPolicy --expiration-period 900
elb-set-lb-policies-of-listener myLoadBalancer --lb-port 80 --policy-names fifteenMinutesPolicy

These commands create a stickiness policy that says “make the session stickiness last for fifteen minutes” and sets the load balancer to use that stickiness policy. Behind the scenes, the ELB’s session cookie will have a lifetime of fifteen minutes.

What Can’t ELB Sticky Session Do?

Life is not all roses with ELB’s sticky session support. Here are some things it can’t do.

Update October 2010: ELB now supports SSL termination, and it can provide sticky sessions over HTTPS as well.

HTTPS

Remember how sticky sessions are typically provided via cookies? The cookie is inserted into the HTTP request by the client’s browser, and any server or load balancer that can read the request can recover the cookie. This works great for plain old HTTP-based communications.

With HTTPS connections the entire communications stream is encrypted. Only servers that have the proper decryption credentials can decipher the stream and discover the cookies. If the load balancer has the server’s SSL certificate then it can decrypt the stream. Because it does not have your application’s SSL certificate (and there’s no way to give it your certificate) ELB does not support HTTPS communications. If you need to support sticky sessions and HTTPS in EC2 then you can’t use ELB today. You need to use HAProxy or aiCache or another product that provide load balancing with session affinity and SSL termination.

Scaling-down-proof stickiness

What happens when you add or remove an application server to/from the load balancer? Depending on the stickiness implementation the load balancer may or may not be able to route requests to the same application servers as it did before the scaling event (caused, for example, by an AutoScaling trigger).

When scaling up (adding more application servers) ELB maintains stickiness of existing sessions. Only new connections will be forwarded to the newly-added application servers.

When scaling down (removing application servers), you should expect some of your clients to lose their sessions and require logins again. This is because some of the stored sessions were on the application server that is no longer servicing requests.

If you really want your sessions to persist even through scaling-down events, you need to go back to basics: your application will need to store the sessions independently, as it did before sticky sessions were supported. In this case, sticky session support can provide an added optimization, allowing you to cache the session locally on each application server and only retrieve it from the central session store (the DB?) if it’s not in the local cache. Such cache misses would happen when application servers are removed from the load balancing pool, but otherwise would not impact performance. Note that this technique can be used equally well with ELB and with other load balancers.

With the introduction of sticky sessions for ELB, you – the application developer – can avoid modifying your application in order to retain session context behind a load balancer. The technical term for this is “a good thing”. Sticky sessions are, despite their limitations, a very welcome addition to ELB’s features.

Thanks to Jeff Barr of Amazon Web Services for providing feedback on drafts of this article.