Categories
Cloud Developer Tips

Elastic Load Balancing with Sticky Sessions

At long last, the most oft-requested feature for EC2’s Elastic Load Balancer is here: session affinity, also known as “sticky sessions”. What is session affinity? Why is this feature in such high demand? How can it be used with existing applications? Let’s take a look at these questions. But first, let’s explore what a session is – then we’ll cover why we want it to be sticky, and what ELB’s sticky session limitations are. [To skip directly to an explanation of how to use ELB sticky sessions, go toward the bottom of the article.]

What is a Session?

A session is a way to get your application involved in a long-lasting conversation with a particular client. Without a session, a conversation between your application and a client would  look like something straight out of the movie Memento. It would look like this:

Life Without Sessions

Client: Hi, I’d like to see /products/awesomeDoohickey.html

Application: I don’t know who you are. Please go here to login first: /login

Client: OK, I’d like to see /login

Application: Here it is: “…”

Client: Thanks. Here’s the filled in login form.

Application: Thanks for logging in. Where do you want to go?

Client: I’d like to see /products/awesomeDoohickey.html

Application: I don’t know who you are. Please go here to login first: /login

Client: >Sigh< OK, I’d like to see /login

Application: Happily! Here it is: “…”

Client: Here’s the filled in login form.

Application: Thanks for logging in. Where do you want to go?

Client: Show me /products/awesomeDoohickey.html already!

Application: I don’t know who you are. Please go here to login first: /login

Client: *$#%& this!

The application can’t remember who the client is – it has no context to process each request as part of a conversation. The client gets so frustrated he starts thinking he’s living in an Adam Sandler movie.

On a technical level: Each HTTP request-response pair between the client and application happens (most often) on a different TCP connection. This is especially true when a load balancer sits between the client and the application. So the application can’t use the TCP connection as a way to remember the conversational context. And, HTTP itself is stateless: any request can be sent at any time, in any sequence, regardless of the preceding requests. Sure, the application may demand a particular pattern of interaction – like logging in before accessing certain resources – but that application-level state is enforced by the application, not by HTTP. So HTTP cannot be relied on to maintain conversational context between the client and the application.

There are two ways to solve this problem of forgetting the context. The first is for the client to remind the application of the context every time he requests something: “My name is Henry Whatsisface, I have these items in my shopping cart (…), I got here via this affiliate (…), yada yada yada… and I’d like to see /products/awesomeDoohickey.html”. No sane client would ever agree to interact with an application that needed to be sent the entire context at every stage of the conversation. Its burdensome for the client, it’s difficult to maintain for the application, and it’s expensive (in bandwidth) for both of them. Besides, the application usually maintains the conversational state, not the client. So it’s wrong to require the client to send the entire conversation context along with each request.

The accepted solution is to have the application remember the context by creating an associated memento. This memento is given to the client and returned to the application on subsequent requests. Upon receiving the memento the application looks for the associated context, and – voila – discovers it. Thus, the conversation is preserved.

One way of providing a memento is by putting it into the URL. It looks really ugly when you do this: http://www.example.com/products/awesomeDoohickey.html?sessionID=0123456789ABCDEFGH

More commonly, mementos are provided via cookies, which all browsers these days support. Cookies are placed within the HTTP request so they can be discovered by the application even if a load balancer intervenes.

Here’s what that conversation looks like with cookies:

Life With Sessions, Take 1

Client: Hi, I’d like to see /products/awesomeDoohickey.html

Application: I don’t know who you are. Please go here to login first: /login

Client: OK, I’d like to see /login

Application: Here it is: “…”

Client: Thanks. Here’s the filled in login form.

Application: Thanks for logging in. Here’s a cookie. Where do you want to go?

Client: I’d like to see /products/awesomeDoohickey.html and here’s my cookie.

Application: I know you – I’d recognize that cookie anywhere! Great, here’s that page: “…”

Client: I’d like to buy 5000 units. Here’s my cookie.

Much improved, yes?

A side point: most modern applications will provide a cookie earlier in the conversation. This allows the following more optimal conversation:

Life With Sessions, Take 2

Client: Hi, I’d like to see /products/awesomeDoohickey.html

Application: I don’t know who you are. Here’s a cookie. Take this login page and fill it out: “…”

Client: OK. Here’s the filled in login form. And here’s my cookie.

Application: I know you – I’d recognize that cookie anywhere! Thanks for logging in. I recall you wanted to see /products/awesomeDoohickey.html. Here it is: “…”

Client: I’d like to buy 5000 units. Here’s my cookie.

That’s about as optimized a conversation as you can have. Cookies make it possible.

What is Session Affinity (Sticky Sessions)? Why is it in High Demand?

When you only have one application server talking to your clients life is easy: all the session contexts can be stored in that application server’s memory for fast retrieval. But in the world of highly available and scalable applications there’s likely to be more than one application server fulfilling requests, behind a load balancer. The load balancer routes the first request to an application server, who stores the session context in its own memory and gives the client back a cookie. The next request from the same client will contain the cookie – and, if the same application server gets the request again, the application will rediscover the session context. But what happens if that client’s next request instead gets routed to a different application server? That application server will not have the session context in its memory – even though the request contains the cookie, the application can’t discover the context.

If you’re willing to modify your application you can overcome this problem. You can store the session context in a shared location, visible to all application servers: the database or memcached, for example. All application servers will then be able to lookup the cookie in the central, shared location and discover the context. Until now, this was the approach you needed to take in order to retain the session context behind an Elastic Load Balancer.

But not all applications can be modified in this way. And not all developers want to modify existing applications. Instead of modifying the application, you need the load balancer to route the same client to the same application server. Once the client’s request has been routed to the correct application server, that application server can lookup the session cookie in its own memory and recover the conversational context.

That’s what sticky sessions are: the load balancer routing the same client to the same application server. And that’s why they’re so important: If the load balancer supports sticky sessions then you don’t need to modify your application to remember client session context.

How to Use ELB with Sticky Sessions with Existing Applications

The key to managing ELB sticky sessions is the duration of the stickiness: how long the client should consistently be routed to the same back-end instance. Too short, and the session context will be lost, forcing the client to login again. Too long, and the load balancer will not be able to distribute requests equally across the application servers.

Controlling the ELB Stickiness Duration

ELB supports two ways of managing the stickiness’ duration: either by specifying the duration explicitly, or by indicating that the stickiness expiration should follow the expiration of the application server’s own session cookie.

If your application server has an existing session cookie already, the simplest way to get stickiness is to configure your ELB to use the existing application cookie for determining the stickiness duration. PHP applications usually have a session cookie called PHPSESSID. Java applications usually have a session cookie called JSESSIONID. The expiration of these cookies is controlled by your application, and the stickiness expiration can be set to match as follows. Assuming your load balancer is called myLoadBalancer and it has an HTTP listener on port 80:

elb-create-app-cookie-stickiness-policy myLoadBalancer --cookie-name PHPSESSID --policy-name followPHPPolicy
elb-set-lb-policies-of-listener myLoadBalancer --lb-port 80 --policy-names followPHPPolicy

The above commands create a stickiness policy that says “make the session stickiness last as long as the cookie PHPSESSID does” and sets the load balancer to use that stickiness policy. Behind the scenes, the ELB’s session cookie will have the same lifetime as the PHPSESSID cookie provided by your application.

If your application does not have its own session cookie already, set your own stickiness duration for the load balancer, as follows:

elb-create-lb-cookie-stickiness-policy myLoadBalancer --policy-name fifteenMinutesPolicy --expiration-period 900
elb-set-lb-policies-of-listener myLoadBalancer --lb-port 80 --policy-names fifteenMinutesPolicy

These commands create a stickiness policy that says “make the session stickiness last for fifteen minutes” and sets the load balancer to use that stickiness policy. Behind the scenes, the ELB’s session cookie will have a lifetime of fifteen minutes.

What Can’t ELB Sticky Session Do?

Life is not all roses with ELB’s sticky session support. Here are some things it can’t do.

Update October 2010: ELB now supports SSL termination, and it can provide sticky sessions over HTTPS as well.

HTTPS

Remember how sticky sessions are typically provided via cookies? The cookie is inserted into the HTTP request by the client’s browser, and any server or load balancer that can read the request can recover the cookie. This works great for plain old HTTP-based communications.

With HTTPS connections the entire communications stream is encrypted. Only servers that have the proper decryption credentials can decipher the stream and discover the cookies. If the load balancer has the server’s SSL certificate then it can decrypt the stream. Because it does not have your application’s SSL certificate (and there’s no way to give it your certificate) ELB does not support HTTPS communications. If you need to support sticky sessions and HTTPS in EC2 then you can’t use ELB today. You need to use HAProxy or aiCache or another product that provide load balancing with session affinity and SSL termination.

Scaling-down-proof stickiness

What happens when you add or remove an application server to/from the load balancer? Depending on the stickiness implementation the load balancer may or may not be able to route requests to the same application servers as it did before the scaling event (caused, for example, by an AutoScaling trigger).

When scaling up (adding more application servers) ELB maintains stickiness of existing sessions. Only new connections will be forwarded to the newly-added application servers.

When scaling down (removing application servers), you should expect some of your clients to lose their sessions and require logins again. This is because some of the stored sessions were on the application server that is no longer servicing requests.

If you really want your sessions to persist even through scaling-down events, you need to go back to basics: your application will need to store the sessions independently, as it did before sticky sessions were supported. In this case, sticky session support can provide an added optimization, allowing you to cache the session locally on each application server and only retrieve it from the central session store (the DB?) if it’s not in the local cache. Such cache misses would happen when application servers are removed from the load balancing pool, but otherwise would not impact performance. Note that this technique can be used equally well with ELB and with other load balancers.

With the introduction of sticky sessions for ELB, you – the application developer – can avoid modifying your application in order to retain session context behind a load balancer. The technical term for this is “a good thing”. Sticky sessions are, despite their limitations, a very welcome addition to ELB’s features.

Thanks to Jeff Barr of Amazon Web Services for providing feedback on drafts of this article.

64 replies on “Elastic Load Balancing with Sticky Sessions”

Shlomo,

Have you seen Terracotta? We offer a cloud stack on top of EC2 (fully integrated). Jeff and I did a webcast on it together last month. Anyways, Terracotta can, among other things, transparently store sessions underneath web apps and make them available to other machines when load balancer failover/ hopping occurs. So, Sticky load balancing + Terracotta means scaling up or down will not cause session loss. Even round robin load balancing w/ ELB has worked for a long time w/ Terracotta.

–Ari

Excellent summary, however you should note that providing a session-cookie prior to authentication opens the website to the vulnerability known as Session Fixation. Upon successful authentication, the client should always be provided with a new session-id.

I know this is an old post, but as somebody who can’t let misinformation go unrebutted, I must rebut SalusaSecondus:

The session fixation attack has nothing to do with handing out a cookie before or after authentication. If you transfer cookies in plain text (not encrypted), then any eavesdropper can observe the cookie and steal the user’s session. Obviously this is worse in cases where the user is authenticated, but the order of operations doesn’t matter. The attacker can steal the cookie before or after authentication and hijack the session equally well.

The only reasonable way to protect against session fixation on the web is to use SSL/TLS. The cookie should be set with the “secure cookie” option so that the client knows never to send that cookie in plain text.

Ideally, if you’re going to the effort to set up SSL for the entire authenticated session, then you might as well have SSL across your entire site (i.e. SSL before the user even authenticates). Otherwise the user can still be exploited by a skilled attacker using an sslstrip attack.

I realise this now an old thread, but there is one other point I would like to add regarding handing out cookies before authentication. Doing so potentially leaves you open to a denial of service attack by allowing an attacker to continue requesting new cookies from you, extremely rapidly, that your Session management system is going to need to keep track of.

For example, if you were using memcached in a web farm to store session tokens, all these new session tokens might start dropping out old “valid” tokens, or your cache might simply fill up. Neither is a great situation to be in.

Is the following a result of an Amazon service outage, or am I missing something?

>elb-create-lb c2 –availability-zones us-east-1a –listener “protocol=tcp, lb-port=80, instance-port=80”
DNS_NAME c2-588861184.us-east-1.elb.amazonaws.com

>elb-create-app-cookie-stickiness-policy c2 –cookie-name PHPSESSID –policy-name x
OK-Creating App Stickiness Policy

>elb-set-lb-policies-of-listener c2 –lb-port 80 –policy-names x
elb-set-lb-policies-of-listener: Service error: Policy [x] cannot be enabled for load balancer port [80] with
protocol [TCP]. AWSRequestId:0c460472-944a-11df-81dd-d14f18cab845

@Rob Staveley,

You need to use an HTTP listener in order to be able to set up a session stickiness policy. Your elb-create-lb command should have “http” instead of “tcp”:

elb-create-lb c2 –availability-zones us-east-1a –listener “protocol=http, lb-port=80, instance-port=80″

Hello ,

We are planning to setup a high availability setup for our App with ELB + Apache (web server ) + Tomcat ( app server ) + MySQL

We are planning to setup multiple instances for both Apache and Tomcat . We have some questions regarding this setup

1 ) How to setup persistent session with multiple Tomcat instances ( separate servers ) when there are multiple apache instances under ELB

e.g . when a request comes to the ELB , it goes to APACHE1 ->TOMCAT1->DATABASE . Suppose that a session is created in that request . For the subsequent request does ELB send the request to the same webserver APACHE1 or will it go to APACHE2 ? Does the session stickiness with ELB send the request to the same webserver APACHE1 ? or how do we setup a persistent session for these kind of setups ?

Please assist

Thanks
Shankar

Hi Shankar,

I know it’s been so many year since you asked the question.

And I hope you have achieved this by now.

I am having same type of scenarion. So could you please tell me how did you achieve it.
I have two questions basically..

Can we have stickyness of ELB to Apache1.(Because we already have setup to sticky session setup for Apache1 to all other tomcats i.e. App Server).
Can we access the ELB IP at tomcat level i.e. Application.

PS: please use rajat.sharma2606 at gmail dot com for communication.

Thanks.

Hello

An update : How is the state maintained if one tomcat server goes down ? Suppose that we have maintained session using JSESSIONID = *******.tomcat1 and what if tomcat1 server goes down ?

Thanks
Shankar

That was an excellent explanation on Sticky sessions for beginners like me.. Didn’t know about any of these untill i read it here.. Thank you very much !!!

Really is was excellent explanation on Sticky sessions for beginners like me.. Didn’t know about any of these untill i read it here.. i have learned a lot from this document, Thank you ,Thank you very much !!!

It was nice explanation at all, i really appreciate.
But
I have a query, if any one of you can reply please…

I am using IBM WAS 7* Network Deployment (WAS7* ND) and clustered on 02 separate IP’s.
I want to use Apache_2.0.52-Openssl_0.9.7e-Win32 for Load balancing for huge number of requests.

Any body please guide me how I can achieve it, and what all modules I can use for this particular purposes please

Your response will be highly appreciated please

Thats nice but not enough. I need to balance my API calls.
I have some READ api calls and write API calls and want to separate them based on the URL.
Or what is the best solution? content load balancing ot …. please advice Shlomo.
todaraba

@Vlad,

The “simplest” way to do that is to use different host names (e.g. ro.example.com and write.example.com) for the read vs. write API calls. Then you can set up separate ELBs for each host name.

Beyond that, each application container has its own way of routing requests, and some support rerouting to external endpoints. The answer really depends on the application container you are using.

AWS simply offering Layer 4 Load Balancing vs Layer 7
Company of that size should do better…. in my view.

@Vlad,

AWS offers Infrastructure as a Service, which stays out of your application (for the most part). If you want the service layer to take care of more than IaaS, look for Platform as a Service offerrings.

Hi,

This is helpful but I have a question about application sticky sessions, wanted to see if you knew the answer.

So if I specify an application based sticky session to use “JSESSIONID” for example, when a request is sent to my load balancer with just JSESSIONID, ELB will return a responses with a special cookie (AWSELB?)?

I am using an IBM product that seems to be discarding the response cookie details, and only providing the actual response object.

If I were to send another request, without the AWSELB cookie, would the go to the correct server or would it apply the “lowest usage” logic described, and potentially go to a new server?

@Brad,

So if I specify an application based sticky session to use “JSESSIONID” for example, when a request is sent to my load balancer with just JSESSIONID, ELB will return a responses with a special cookie (AWSELB?)?

Correct. And the lifetime of the AWSELB cookie would be set to the lifetime of the JSESSIONID cookie.

If I were to send another request, without the AWSELB cookie, would the go to the correct server or would it apply the “lowest usage” logic described, and potentially go to a new server?

A request received without an AWSELB cookie would be routed using ELB’s standard algorithm, “approximate-least-connections”.

Hi Shlomo Swidler,
We need your here on Amazon Load Balancer sticky sessions with ajp:8009

We have configured ELB with sticky sessions for the JSESSIONID cookie for two tomcats (tomcat1 and tomcat2)(Flow is – Apache Http Server – ELB – tomcats)

AJP protocol with port 8009 has been configured on tomcat side as from AWS ELB there is no AJP option, we have configured with tcp:8009

So the Apache httpd.conf entry is, (xxx.amazonaws.com is ELB name)

BalancerMember ajp://xxx.amazonaws.com:8009

Somehow the sticky session is not working, the http request is sent to both tomcat servers. Is it because of the protocol on ELB side (tcp:8009)? We are not sure what is missing here, Need help!!

Hi Shlomo Swider,
Thanks a lot, We have tried with ELB->(HTTPS)Apache->(AJP)Tomcat and that worked.

Thank You!!

Hello Shlomo Swidler and Folks,

I have a already built application in the Classic ASP and I can’t store the session context in a shared location. I do want to implement Sticky Sessions with AWSELB and have following queries:

If I enable the Sticky sessions as above mentioned procedure will ELB sends Cookie
in every response? or Do we have to manually set the Cookie content?
(Can we directly use the Session Cookie for ELB stickiness implementation so that It
can automatically recognize particular Client redirects to particular instance?)
If no then would you like to tell me where exactly we can put the code in a single file
of application that solves the problem of adding the cookies and it’s content

(It will be too helpful if provide a small example.)

Thanks..!!

@Vikas Tomar,

You don’t need to think about the cookie at all. ELB will add its own cookie “AWSELB” to each response. ELB will manage the content of this cookie.

It sounds like you are asking if you can preemptively assign clients, who may not have connected yet, to specific instances behind the ELB. If that is your question, the answer is no, that can’t be done.

For the basic setup procedure to enable sticky sessions, see the AWS ELB Sticky Session docs.

Thank You..!!

Basic setup procedure to enable sticky sessions has done and working perfectly fine.

We have deployed Web application (developed using MVC C#) on Amazon Cloud IIS including ELB. Initially we have tested application for UAT in same environment using InProc Session & it works fine. Later application deployed for production on two server on Amazon cloud with LB. But then the loosing of Session start appearing. We have now have only one server with LB still losing session randomly. Does need to do any specific setting on LB to avoid lost of Session?

Initially we have tested application for UAT in same environment using InProc Session & it works fine… without LB.

This is great information on the Amazon ELB, but I have a question in regards to a slightly different approach. Can the client side initiate the assignment of the sticky cookie? That is, can a client send an HTTP request to a server with the cookie identified in the Amazon ELB for application cookie stickiness.policy? I have control over the client application and what it sends, but not the server. I want to have certain HTTP requests from the client application go to the same server behind ELB.

@Doug Jordan,

I would not expect that to work. The ELB will most likely not even check the cookie if it was not set up for sticky sessions. And even if it is set up for sticky sessions, the contents of the cookie are changeable any time by the ELB implementation, so I wouldn’t rely on that either.

Hello Shlomo Swidler,

I working with Application load balancer and running to issue, where user will not be bale to authenticate to application home page with windows credentials or local application accounts. I was wondering is there something we need to configure specifically for this scenario to work?

Thanks for answering.

Mohit

Hi Shlomo,

Thanks for the great explanation!

I have a query: what happens when the cookie expires in case of duration based stickiness? In the example given by you after 900 seconds if the “previously stuck” user sends the next request in his session, will the ELB direct his request to another app server instance? If that happens, will the user’s session context be lost ? Can you please throw some light on the real life use cases for duration based stickiness and how the aforementioned situation will be catered to ?

Thanks and regards
Mahesh

@Mahesh,

Yes, if the session stickiness cookie expires then the ELB will choose again from available app server instances, and the request will, most likely, be directed to a different app server instance.
This is one reason why it’s not a good idea to store session-specific state on your app servers. Use memcache or your database to store client state, so your app servers do not have to be stateful.

Hi,
I have two smtp mail servers which communicates on tcp port:25, so how can we create sticky session for that because without sticky session my VM is unable to connect to ELB its loses connection.

These are its log:
Feb 14 09:52:39 sendmail postfix/smtpd[27658]: connect from ip-x-x-x-x.ec2.internal[x.x.x.x]
Feb 14 09:52:39 sendmail postfix/smtpd[27658]: lost connection after CONNECT from ip-x-x-x-x.ec2.internal[x.x.x.x]
Feb 14 09:52:39 sendmail postfix/smtpd[27658]: disconnect from ip-x-x-x-x.ec2.internal[x.x.x.x] commands=0/0
Feb 14 09:52:43 sendmail postfix/smtpd[27658]: connect from ip-y-y-y-y.ec2.internal[y.y.y.y]
Feb 14 09:52:43 sendmail postfix/smtpd[27658]: lost connection after CONNECT from ip-y-y-y-y.ec2.internal[y.y.y.y]
Feb 14 09:52:43 sendmail postfix/smtpd[27658]: disconnect from ip-y-y-y-y.ec2.internal[y.y.y.y] commands=0/0

@Vivek Suhanda,

Sticky sessions only work over HTTP/S but you are trying to “stick” connections over port 25. You would need to code your application to perform its own load balancing.

Is there IP based stickiness? Thousands of businesses hosts API today in cloud, which is accessible directly from programming languages instead traditional browser where cookies can be stored.

Or another option may be using query string or request header. For example client X send [X-api-key : abc] or in in url then “api.domain.com?apikey=abc” and so on for each client.

So can’t we tell the ELB to see particular header key or IP address or query string to stick with same instance instead cookie?

@Vikash,

ELB does not support IP-based stickiness.

The API library you are using should provide session support if the API provider requires it. If not, perhaps you can use an HTTP/S support library in your programming language to handle the cookies.

Hi Shlomo,
First thanks for your write-up on Sticky Session. Have a query:
– Is it a good practice to turn off an existing Sticky Session for some reason on my ELB?
– Are there any drawbacks in using Sticky Session?
– Do we have any other option apart from using Sticky Session?

Thanks

@Viswanath,

If your application stores its state (account info, transactions, caching, etc.) off of the EC2 instance then you probably do not need sticky sessions. Using sticky sessions unnecessarily shouldn’t have any drawbacks though.

I highly recommend using ALB or NLB instead of ELB wherever possible. They have much richer feature sets.

Leave a Reply

Your email address will not be published.