Paul Hammant's Blog: AppEngine's blind-spot

I'm not wishing to be ungrateful for the opportunity to test-drive GAE over the last while, but I have to get something off my chest. There is a hole in the service right now that's going to mean a change to Java web app development. Namely there is modest risk to storing things in the session for apps that you could characterize as Ajax-like and chatty over the wire. The risk is to items being out of date when retrieved from the map. If you are running a stateless app, then there is no problem, but if you are trying to store things in the session (and not push everything to cookies or down to BigTable) then you could see concurrency problems.

The Proof?

I have an app deployed to GAE that uses two iFrames on the client side, each concurrently issuing requests to a slow servlet to expose the bug. Check it out. It was in my launch-night blog post, but is worth calling out separately.

Sun's Servlet spec suggests that in a cluster design, concurrent requests from the same session should hit the same 'container'. The spec says JVM, but who knows how Google have carved up the seeming Cray that they are running this on. So we'll say 'container'. The idea is that the second request is given references to the same instances at the session level as the first request. The first one to finish, may issue a session.setAttribute(..) to indicate a write/serialization is needed, but it won't happen until the second request is completed too. That second request may have also issued a session.setAttribute(..), but it would be redundant. Without requests sharing instances, you're going to have merge issues..

As you play with the app I made, be aware that the first page impression from you is going to have two requests that are not associated with the same session. Thus, hit refresh then scrutinize the results. There is a handy button at the bottom you an hit to compare (in JavaScript) the two results. I'm using System.identityHashCode(..) so we humans can see the instance refs, which the AppEngine folks have left in a white-list. The GAE assigned session ID (things like nNc3E8ame5p-irDqQ2cX1Q) is the same for both, which is good. Nearly everything else that you might expect to be the same instance ID is not though. This is rooted in the key fact in the page:

the Java virtual machines are different, meaning different servlet containers.

AppEngine is forwarding requests to different machines. Maybe not randomly, as your app is only going to be on a small percentage. Random assignment of request to those three nodes, is random enough for the Servlet spec to be broken in respect of section 7.7.2 (thanks to John Hume for relevant detail from that spec). This is how they are scaling - they are doing the minimal analysis of each incoming request.

It is not that a session's requests have to be eternally directed to the same node. Google most likely manage instances like they are wet paint and could not bear a node remaining locked because it has an active user. If there is a gap between requests though, the affinity to a single node could be dropped. I'm hoping that its not too hard to do that last, and that Google change their minds on this issue (not that I have evidence that they have stance on this issue at all).

Here's a screen shot:

Here's one that is on a vanilla version of Jetty (running local host), and dev_appserver.sh would be the same:

The Work Around ?

1. Be stateless; you're going to get a ton more scale. I'll come back another day with costs for implicitly serializing/de-serializing of session per request.
2. Ensure that your web client (or non web client) is not issuing concurrent requests that might mutate the session stored objects. That's not as hard as you think.
   
   

Another consequence of a cluster.

Your servlets/filters/listeners are going to get initialized more than once. If you have a thirty second impediment to first request being satisfied, and AppEngine chooses to relocate or cluster your app often, then you are going to see that ultimately cost you money. Nothing from the servlet context attribute map is shared between nodes. Thus without using caching or BigTable there is nothing you can do to streamline the startup costs.

Meanwhile...

• Google have put up my short developer interview video. It is not that I'm special, it is just that I was more local to Mountain View than colleagues John Hume, Ola Bini, Sriram Narayan, and Philip Calçado. All four of them had used more fancy technologies for launch, but were too far away to get an interview with Fred Sauer, though I might suggest that PicoContainer Web Remoting's leveraging of method parameter names is a cool technology.
• AppEngine Java has a developers mail list now. Wow, there are many hundreds of messages a day. There is a wide range of experience with web app development demonstrated in the list, and I can't help but feel sorry for Google and the level of the interest while they are still in beta. I put the emails on a kitchen scale, and weighed them, and estimate that we could well have half a million deployments within eighteen months, if Google unblock the limited-beta aspect. Lots will be "Hello World" for sure, but Java developer and their dog is going to want to push something up if they can.
• We have already worked out that Apps can hold onto instances between requests (a normal web-app feature), but there is a realization that Google are not currently pricing the memory usage. It would need a more advanced JVM than is available I think. You could not just periodically serialize an app to see how big it was, as folks could use 'transient' to make an app look smaller in terms of memory footprint than it was.
• I guess Sun is scurrying towards their cloud rollout. I bet it looks more like Google's one than those of RackSpace or Amazon. I don't know if there are ThoughtWorks staff helping out, but would not be surprised if there were. I guess its appropriate that there would be some walls up inside TW. Sun's will have their own special sauce too I'm guessing. I'm guessing too that they will go for partnership to backfill the services that Google have already. Services like payments, for which Google have Checkout, and Sun might partner Paypal or more than one provider.
  
  

Deploy it yourself

Here is the WAR file containing source and the zipped Maven project for the same (12K and 8.5K respectively).

I would love to get some observations back from other app-servers and servlet containers, the WAR file should be deployable to any servlet container. Be aware though that I have not tried the app in Internet Explorer.