≡ Menu

Elastic Load Balancing with Sticky Sessions

At long last, the most oft-requested feature for EC2′s Elastic Load Balancer is here: session affinity, also known as “sticky sessions”. What is session affinity? Why is this feature in such high demand? How can it be used with existing applications? Let’s take a look at these questions. But first, let’s explore what a session is – then we’ll cover why we want it to be sticky, and what ELB’s sticky session limitations are. [To skip directly to an explanation of how to use ELB sticky sessions, go toward the bottom of the article.]

What is a Session?

A session is a way to get your application involved in a long-lasting conversation with a particular client. Without a session, a conversation between your application and a client would  look like something straight out of the movie Memento. It would look like this:

Life Without Sessions

Client: Hi, I’d like to see /products/awesomeDoohickey.html

Application: I don’t know who you are. Please go here to login first: /login

Client: OK, I’d like to see /login

Application: Here it is: “…”

Client: Thanks. Here’s the filled in login form.

Application: Thanks for logging in. Where do you want to go?

Client: I’d like to see /products/awesomeDoohickey.html

Application: I don’t know who you are. Please go here to login first: /login

Client: >Sigh< OK, I’d like to see /login

Application: Happily! Here it is: “…”

Client: Here’s the filled in login form.

Application: Thanks for logging in. Where do you want to go?

Client: Show me /products/awesomeDoohickey.html already!

Application: I don’t know who you are. Please go here to login first: /login

Client: *$#%& this!

The application can’t remember who the client is – it has no context to process each request as part of a conversation. The client gets so frustrated he starts thinking he’s living in an Adam Sandler movie.

On a technical level: Each HTTP request-response pair between the client and application happens (most often) on a different TCP connection. This is especially true when a load balancer sits between the client and the application. So the application can’t use the TCP connection as a way to remember the conversational context. And, HTTP itself is stateless: any request can be sent at any time, in any sequence, regardless of the preceding requests. Sure, the application may demand a particular pattern of interaction – like logging in before accessing certain resources – but that application-level state is enforced by the application, not by HTTP. So HTTP cannot be relied on to maintain conversational context between the client and the application.

There are two ways to solve this problem of forgetting the context. The first is for the client to remind the application of the context every time he requests something: “My name is Henry Whatsisface, I have these items in my shopping cart (…), I got here via this affiliate (…), yada yada yada… and I’d like to see /products/awesomeDoohickey.html”. No sane client would ever agree to interact with an application that needed to be sent the entire context at every stage of the conversation. Its burdensome for the client, it’s difficult to maintain for the application, and it’s expensive (in bandwidth) for both of them. Besides, the application usually maintains the conversational state, not the client. So it’s wrong to require the client to send the entire conversation context along with each request.

The accepted solution is to have the application remember the context by creating an associated memento. This memento is given to the client and returned to the application on subsequent requests. Upon receiving the memento the application looks for the associated context, and – voila – discovers it. Thus, the conversation is preserved.

One way of providing a memento is by putting it into the URL. It looks really ugly when you do this: http://www.example.com/products/awesomeDoohickey.html?sessionID=0123456789ABCDEFGH

More commonly, mementos are provided via cookies, which all browsers these days support. Cookies are placed within the HTTP request so they can be discovered by the application even if a load balancer intervenes.

Here’s what that conversation looks like with cookies:

Life With Sessions, Take 1

Client: Hi, I’d like to see /products/awesomeDoohickey.html

Application: I don’t know who you are. Please go here to login first: /login

Client: OK, I’d like to see /login

Application: Here it is: “…”

Client: Thanks. Here’s the filled in login form.

Application: Thanks for logging in. Here’s a cookie. Where do you want to go?

Client: I’d like to see /products/awesomeDoohickey.html and here’s my cookie.

Application: I know you – I’d recognize that cookie anywhere! Great, here’s that page: “…”

Client: I’d like to buy 5000 units. Here’s my cookie.

Much improved, yes?

A side point: most modern applications will provide a cookie earlier in the conversation. This allows the following more optimal conversation:

Life With Sessions, Take 2

Client: Hi, I’d like to see /products/awesomeDoohickey.html

Application: I don’t know who you are. Here’s a cookie. Take this login page and fill it out: “…”

Client: OK. Here’s the filled in login form. And here’s my cookie.

Application: I know you – I’d recognize that cookie anywhere! Thanks for logging in. I recall you wanted to see /products/awesomeDoohickey.html. Here it is: “…”

Client: I’d like to buy 5000 units. Here’s my cookie.

That’s about as optimized a conversation as you can have. Cookies make it possible.

What is Session Affinity (Sticky Sessions)? Why is it in High Demand?

When you only have one application server talking to your clients life is easy: all the session contexts can be stored in that application server’s memory for fast retrieval. But in the world of highly available and scalable applications there’s likely to be more than one application server fulfilling requests, behind a load balancer. The load balancer routes the first request to an application server, who stores the session context in its own memory and gives the client back a cookie. The next request from the same client will contain the cookie – and, if the same application server gets the request again, the application will rediscover the session context. But what happens if that client’s next request instead gets routed to a different application server? That application server will not have the session context in its memory – even though the request contains the cookie, the application can’t discover the context.

If you’re willing to modify your application you can overcome this problem. You can store the session context in a shared location, visible to all application servers: the database or memcached, for example. All application servers will then be able to lookup the cookie in the central, shared location and discover the context. Until now, this was the approach you needed to take in order to retain the session context behind an Elastic Load Balancer.

But not all applications can be modified in this way. And not all developers want to modify existing applications. Instead of modifying the application, you need the load balancer to route the same client to the same application server. Once the client’s request has been routed to the correct application server, that application server can lookup the session cookie in its own memory and recover the conversational context.

That’s what sticky sessions are: the load balancer routing the same client to the same application server. And that’s why they’re so important: If the load balancer supports sticky sessions then you don’t need to modify your application to remember client session context.

How to Use ELB with Sticky Sessions with Existing Applications

The key to managing ELB sticky sessions is the duration of the stickiness: how long the client should consistently be routed to the same back-end instance. Too short, and the session context will be lost, forcing the client to login again. Too long, and the load balancer will not be able to distribute requests equally across the application servers.

Controlling the ELB Stickiness Duration

ELB supports two ways of managing the stickiness’ duration: either by specifying the duration explicitly, or by indicating that the stickiness expiration should follow the expiration of the application server’s own session cookie.

If your application server has an existing session cookie already, the simplest way to get stickiness is to configure your ELB to use the existing application cookie for determining the stickiness duration. PHP applications usually have a session cookie called PHPSESSID. Java applications usually have a session cookie called JSESSIONID. The expiration of these cookies is controlled by your application, and the stickiness expiration can be set to match as follows. Assuming your load balancer is called myLoadBalancer and it has an HTTP listener on port 80:

elb-create-app-cookie-stickiness-policy myLoadBalancer --cookie-name PHPSESSID --policy-name followPHPPolicy
elb-set-lb-policies-of-listener myLoadBalancer --lb-port 80 --policy-names followPHPPolicy

The above commands create a stickiness policy that says “make the session stickiness last as long as the cookie PHPSESSID does” and sets the load balancer to use that stickiness policy. Behind the scenes, the ELB’s session cookie will have the same lifetime as the PHPSESSID cookie provided by your application.

If your application does not have its own session cookie already, set your own stickiness duration for the load balancer, as follows:

elb-create-lb-cookie-stickiness-policy myLoadBalancer --policy-name fifteenMinutesPolicy --expiration-period 900
elb-set-lb-policies-of-listener myLoadBalancer --lb-port 80 --policy-names fifteenMinutesPolicy

These commands create a stickiness policy that says “make the session stickiness last for fifteen minutes” and sets the load balancer to use that stickiness policy. Behind the scenes, the ELB’s session cookie will have a lifetime of fifteen minutes.

What Can’t ELB Sticky Session Do?

Life is not all roses with ELB’s sticky session support. Here are some things it can’t do.

Update October 2010: ELB now supports SSL termination, and it can provide sticky sessions over HTTPS as well.

HTTPS

Remember how sticky sessions are typically provided via cookies? The cookie is inserted into the HTTP request by the client’s browser, and any server or load balancer that can read the request can recover the cookie. This works great for plain old HTTP-based communications.

With HTTPS connections the entire communications stream is encrypted. Only servers that have the proper decryption credentials can decipher the stream and discover the cookies. If the load balancer has the server’s SSL certificate then it can decrypt the stream. Because it does not have your application’s SSL certificate (and there’s no way to give it your certificate) ELB does not support HTTPS communications. If you need to support sticky sessions and HTTPS in EC2 then you can’t use ELB today. You need to use HAProxy or aiCache or another product that provide load balancing with session affinity and SSL termination.

Scaling-down-proof stickiness

What happens when you add or remove an application server to/from the load balancer? Depending on the stickiness implementation the load balancer may or may not be able to route requests to the same application servers as it did before the scaling event (caused, for example, by an AutoScaling trigger).

When scaling up (adding more application servers) ELB maintains stickiness of existing sessions. Only new connections will be forwarded to the newly-added application servers.

When scaling down (removing application servers), you should expect some of your clients to lose their sessions and require logins again. This is because some of the stored sessions were on the application server that is no longer servicing requests.

If you really want your sessions to persist even through scaling-down events, you need to go back to basics: your application will need to store the sessions independently, as it did before sticky sessions were supported. In this case, sticky session support can provide an added optimization, allowing you to cache the session locally on each application server and only retrieve it from the central session store (the DB?) if it’s not in the local cache. Such cache misses would happen when application servers are removed from the load balancing pool, but otherwise would not impact performance. Note that this technique can be used equally well with ELB and with other load balancers.

With the introduction of sticky sessions for ELB, you – the application developer – can avoid modifying your application in order to retain session context behind a load balancer. The technical term for this is “a good thing”. Sticky sessions are, despite their limitations, a very welcome addition to ELB’s features.

Thanks to Jeff Barr of Amazon Web Services for providing feedback on drafts of this article.

{ 25 comments… add one }

  • Ari Zilka April 8, 2010, 2:54 pm

    Shlomo,

    Have you seen Terracotta? We offer a cloud stack on top of EC2 (fully integrated). Jeff and I did a webcast on it together last month. Anyways, Terracotta can, among other things, transparently store sessions underneath web apps and make them available to other machines when load balancer failover/ hopping occurs. So, Sticky load balancing + Terracotta means scaling up or down will not cause session loss. Even round robin load balancing w/ ELB has worked for a long time w/ Terracotta.

    –Ari

  • SalusaSecondus April 8, 2010, 6:36 pm

    Excellent summary, however you should note that providing a session-cookie prior to authentication opens the website to the vulnerability known as Session Fixation. Upon successful authentication, the client should always be provided with a new session-id.

    • shlomo April 8, 2010, 6:58 pm

      @SalusaSecondus,

      Thank for the heads-up.

      • Mark Haase June 27, 2013, 10:18 am

        I know this is an old post, but as somebody who can’t let misinformation go unrebutted, I must rebut SalusaSecondus:

        The session fixation attack has nothing to do with handing out a cookie before or after authentication. If you transfer cookies in plain text (not encrypted), then any eavesdropper can observe the cookie and steal the user’s session. Obviously this is worse in cases where the user is authenticated, but the order of operations doesn’t matter. The attacker can steal the cookie before or after authentication and hijack the session equally well.

        The only reasonable way to protect against session fixation on the web is to use SSL/TLS. The cookie should be set with the “secure cookie” option so that the client knows never to send that cookie in plain text.

        Ideally, if you’re going to the effort to set up SSL for the entire authenticated session, then you might as well have SSL across your entire site (i.e. SSL before the user even authenticates). Otherwise the user can still be exploited by a skilled attacker using an sslstrip attack.

  • Zeev Ravid April 15, 2010, 5:27 am

    Very good, nicely written.
    Clear and understandable explanation of topics such as load balancing sessions, stickiness, SSL termination!

  • Rob Staveley July 20, 2010, 10:11 pm

    Is the following a result of an Amazon service outage, or am I missing something?

    >elb-create-lb c2 –availability-zones us-east-1a –listener “protocol=tcp, lb-port=80, instance-port=80″
    DNS_NAME c2-588861184.us-east-1.elb.amazonaws.com

    >elb-create-app-cookie-stickiness-policy c2 –cookie-name PHPSESSID –policy-name x
    OK-Creating App Stickiness Policy

    >elb-set-lb-policies-of-listener c2 –lb-port 80 –policy-names x
    elb-set-lb-policies-of-listener: Service error: Policy [x] cannot be enabled for load balancer port [80] with
    protocol [TCP]. AWSRequestId:0c460472-944a-11df-81dd-d14f18cab845

    • shlomo July 20, 2010, 10:44 pm

      @Rob Staveley,

      You need to use an HTTP listener in order to be able to set up a session stickiness policy. Your elb-create-lb command should have “http” instead of “tcp”:

      elb-create-lb c2 –availability-zones us-east-1a –listener “protocol=http, lb-port=80, instance-port=80″

      • Rob Staveley July 20, 2010, 11:15 pm

        Great catch, Schlomo. Many thanks.

  • shankar August 6, 2010, 8:35 pm

    Hello ,

    We are planning to setup a high availability setup for our App with ELB + Apache (web server ) + Tomcat ( app server ) + MySQL

    We are planning to setup multiple instances for both Apache and Tomcat . We have some questions regarding this setup

    1 ) How to setup persistent session with multiple Tomcat instances ( separate servers ) when there are multiple apache instances under ELB

    e.g . when a request comes to the ELB , it goes to APACHE1 ->TOMCAT1->DATABASE . Suppose that a session is created in that request . For the subsequent request does ELB send the request to the same webserver APACHE1 or will it go to APACHE2 ? Does the session stickiness with ELB send the request to the same webserver APACHE1 ? or how do we setup a persistent session for these kind of setups ?

    Please assist

    Thanks
    Shankar

  • shankar August 6, 2010, 8:50 pm

    Hello

    An update : How is the state maintained if one tomcat server goes down ? Suppose that we have maintained session using JSESSIONID = *******.tomcat1 and what if tomcat1 server goes down ?

    Thanks
    Shankar

  • Seth Caldwell November 8, 2010, 7:08 pm

    Nice article. I wish ELB supported IP address based stickiness, which would work over HTTPS.
    =(

  • Mitesh January 6, 2012, 1:37 pm

    Nice and Simple explaination.

  • Nilesh January 12, 2012, 10:34 am

    The basic concept of ELB are explained nicely…Simple to understand article
    Thanks.

  • Sudhakar February 8, 2012, 2:42 pm

    That was an excellent explanation on Sticky sessions for beginners like me.. Didn’t know about any of these untill i read it here.. Thank you very much !!!

  • Rodney SJ June 15, 2012, 12:46 pm

    Great explanation of ELB and I’ve just heard about it about 2 weeks ago.

  • Mahatab Uddin July 12, 2012, 9:06 am

    Very nice article. Thanks for the easy and informative explanation.

  • chakri August 6, 2012, 11:40 pm

    Really is was excellent explanation on Sticky sessions for beginners like me.. Didn’t know about any of these untill i read it here.. i have learned a lot from this document, Thank you ,Thank you very much !!!

  • Srinivas September 28, 2012, 6:57 am

    Great explanation. Thank you very much!

  • Tom February 21, 2013, 9:55 am

    Is this still true? The docs aren’t clear: http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/using-elb-listenerconfig-quickref.html

    Only https listeners allow sticky sessions?

  • Phil August 17, 2014, 5:14 am

    How would you deal with Varnish with a session cookie set by the load balancer?

    • Shlomo Swidler August 17, 2014, 8:58 am

      @Phil,

      Not sure what the question is. Can you please elaborate?

Leave a Comment