In a recent blog post, Terracotta War Story:How We Clustered RIFE, Jonas Bonér recounts how he and Geert Bevin clustered Bevin's open-source Web framework, RIFE, with Terracotta.

Terracotta is a JVM-level clustering solution, now available under an open-source license. Terracotta promises developers the ability to cluster a Java application with minimal code changes. Bonér's post is an interesting account of just what sorts of code changes may still be required for moving an enterprise Java framework to Terracotta.

A novel RIFE feature is an implementation of continuations. According to Bonér, the main challenge here was to make thread-safe RIFE's internal data structure for storing continuations:

[RIFE continuations ] break down the object graph into primitives and store the actual data on the execution stack (similar to the approach taken by Terracotta)... The continuations are linked together in a tree structure which allows you to traverse to the different execution steps the way you want. This means that you can actually step back (and forward) in time, which is a very neat way of solving the browser back button problem—if used in the context of web applications.

Internally RIFE is storing its continuations in a regular java.util.HashMap... This means that this map is a data structure that holds on to all the state that we want to cluster, something that makes it a perfect fit for becoming a Terracotta ‘root’, e.g. the top-level object in a shared/clustered object graph...The problem with clustering this Map was that in the original RIFE design, only a single thread could access this Map.

Bonér writes that Terracotta aims to maintain object semantics across a cluster and, as a consequence, access to this shared data structure must be guaranteed safely from any thread or VM:

In short, Terracotta requires you to write thread-safe code. What this boiled down to in practice was that we had to make the access and modifications to the map itself and all the continuations in the map guarded (using synchronized blocks).

The second challenge in clustering RIFE with Terracotta had to with RIFE's reliance on dynamically-generated byte code, especially in the RIFE template engine:

RIFE is relying heavily on on-the-fly class generation using bytecode instrumentation. This is extremely powerful, since it allows the user to to find what he wants to do in metadata (with heavy reliance on defaults) and then using that metadata together with templates to generate the actual classes dynamically on demand. This extreme dynamicity imposed a lot of interesting challenges in terms of clustering.

The main challenge is that since some classes are not loaded from Terracotta's classloader, but are instead generated on the fly inside a JVM, such dynamically generated classes are not automatically available on other JVMs in the cluster. Boner's post describes the techniques they used to ensure the cluster-wide availability of such dynamically created classes.

What do you think of JVM-level clustering? What is your experience with the open-source Terracotta so far?