Clustering web applications on Tomcat - Part 4

<- Part 3

Maintenance and Reliability

Tomcat:

Given that Tomcat shares the clustered state among all participating nodes, it is not unexpected that this architecture introduces bottlenecks at some point in the cluster. The overall throughput of the system drops as a function of the number of nodes added into the cluster. From the operations point of view, this can be a considerable overhead as IT cannot add capacity on demand.

Tomcat also supports a couple of persistent mechanisms which write the sessions on to disk or into a Database through JDBC. The caveat here is that the control over when these sessions are written to the persistent store is not very robust, and hence there can be some data loss and/or data integrity can be compromised.

Terracotta:

Terracotta functions as a dedicated state server and plugs into the JVM memory model to provide a highly transparent and efficient clustering implementation to resolve typical bottlenecks, as illustrated by the following diagram.

The Terracotta solution provides a persistent mode, where multiple instances of the Terracotta server share data through a shared disk space (e.g. SAN). This makes the solution more robust as it eliminates any single point of failure in the system.

A very interesting and powerful side-effect of the Terracotta solution is that the application is no longer constrained by the JVM heap size and has "virtual" memory available on demand. Typical 32 bit architectures allocate a maximum of 1.5-2 GB (approx.) heap to the JVM. If your application grows beyond this limit, the JVM will spend most of its time garbage collecting (very high performance hit) and will eventually throw an OOME (OutOfMemoryException). In the Terracotta world, the LRU (least recently used) objects are flushed from the local (client) cache to the Terracotta server, and the LRU objects on the server are flushed onto disk. Terracotta gives you "Network Attached RAM"; no more annoying OOMEs.

As mentioned before, Terracotta also ships with a very useful Admin Console Tool, which allows you to look at the shared sessions and the objects within the session at run time. From the operations perspective, it allows you to monitor the system cache activity, transaction rate and the cache hit ratio as shown in the following screenshots.

Conclusion

Pairing Tomcat with Terracotta delivers a very unique and powerful linear-scale and high-performance HTTP Session clustering solution. Your existing Tomcat environment becomes very powerful and scalable by introducing Terracotta Sessions into the picture. Terracotta allows you to use clustering as an infrastructure-level service by not introducing any clustering code in your application.

Because of its unique architecture, Terracotta delivers the following benefits:

• Linear scale-out
• Significantly reduced TCO
• High performance
• No application changes – it is not necessary to write to an API, implement serialization, or call setAttribute().
• Network efficiency – Without serialization, changes are detected and transmitted at the field level. Replication of data occurs only where necessary.

Terracotta Sessions can be downloaded here.

<- Part 3

Clustering web applications on Tomcat - Part 3

<- Part 2 Part 4 ->

Performance

We performed a benchmarking exercise to highlight the performance characteristics of the two solutions. Our test consisted of the following simulated real world actions:

1. Log in to the application
2. Add several items to the shopping cart (The JPetStore application stores shopping cart in session)
3. Modify the quantity of the items in the shopping cart.

To obtain quantifiable results, Step 3 was repeated for 60,000 iterations and the resulting numbers of transactions per second were measured.

Results:

The following graphs show the performance test results for both the Terracotta Sessions and Tomcat Pooled Mode replication clustering solutions. To test linear scalability, the throughput results were obtained for one node in the cluster, not cluster wide.

In the above graph, Tomcat with Terracotta scales linearly as more nodes are added with a flat curve (red curve - no decrease in throughput). Tomcat with its "Pooled Mode Replication" clustering solution, on the other hand, does not scale linearly with a curve that decreases as more nodes are added (blue curve).

As a result, as is evident in the following graph, the overall system performance increases (Terracotta Sessions + Tomcat - red curve) as against decreasing (Tomcat Clustering - blue curve) when more nodes are added into the cluster.

<- Part 2 Part 4 ->

Clustering web applications on Tomcat - Part 2

<- Part 1 Part 3 ->

Now that we have the background, let's go ahead with the test.

For this test, we used the JPestStore application, which is a widely accepted reference implementation of a typical web application, using various components of the J2EE stack. We used the JPetStore implementation which is bundled with the The Spring Framework (v1.2.8) and can be downloaded here.

Simplicity

Tomcat:

To cluster JPetStore with Tomcat, we had to do the following:

• All the session attributes and the objects in their reference graph must implement java.io.Serializable
• Uncomment the Cluster element in server.xml
• Uncomment the Valve(ReplicationValve) element in server.xml
• If the Tomcat instances are running on the same machine, make sure the tcpListenPort attribute is unique for each instance.
• Make sure the web.xml has the element or set at your
• Make sure that jvmRoute attribute is set at your Engine
• Make sure that all nodes have the same time and sync with NTP service!
• Make sure that the loadbalancer is configured for sticky session mode.

In addition, the application had to make an explicit call to setAttribute() on the session to reduce network chattiness.

Terracotta:

With Terracotta Sessions, we were able to cluster the JPetStore application in under a few minutes, thanks to the really helpful configurator tool that is shipped with the product. The entire process involved firing up the configurator tool, importing the JPetStore war file from within the configurator and firing up the server instances. Done.

Terracotta clusters the sessions declaratively, which means we did not make any code changes or implement java.io.Serializable on session attributes. The entire process involves:

• Declaring the application name in the Terracotta configuration xml file (tc-config.xml).
• Fire up the Terracotta Server.
• Make a call to each Tomcat node in the cluster through Terracotta client script.

The configurator tool did all of the above for us. The configurator has another very useful feature called the Admin-Console which allowed us to look at behavior of our clustered application at run time.

<- Part 1 Part 3 ->

Clustering web applications on Tomcat - Part 1

Part 2 ->

Does your application scale with your business?

In today's world of open-source (read "free") frameworks and increasingly cheap hardware, the limit of applications is being tested more than ever before. An idea is brewed over latte and chai in the neighborhood coffee-shop, is somehow translated into boxes-and-arrows on a whiteboard, and six months later, is receiving over a million hits. So, as it turns out, the idea was really good and is on its way, as was intended. The problem is most enterprise web applications are not ready to scale and handle the requirements of large volumes, high availability and performance without some refactoring (read "high cost" and "bug-prone") or using some off-the-shelf solution or both. It is a real-world challenge for IT professionals to deliver on the promise of always-on, highly performing web applications. Also, to combat this challenge, it is widely accepted that scaling out an application (moving it from one to many physical machines) is a lot more cost affective and flexible than scaling up (increasing the capacity of the single machine).

Let's look at this problem within the context of a web application, which is deployed on multiple machines (scale-out).

In typical non-static web applications, the persistent data is stored in a database and the transient (lasting only a short time; existing briefly; temporary) data is stored in a session. A session is the duration for which the user is actively interacting with the web application. Since this data only needs to live while the user is interacting with the application, it is transient in nature. Now, persistent data can be accessed by multiple application instances from multiple physical locations, as the application communicates with the database through a network connection. This is not true for transient data. The transient session data resides on the local memory and can only be accessed by the local single application instance. Needless to say, if one of these instances goes down due to any reason, all the sessions on that instance would be lost. Hence the need for a clustering solution that will make this transient session information available to multiple nodes in a cluster, without compromising the integrity of the data.

A good clustering solution should offer the following:

1. Simple. It should be as simple as possible, but not simpler.

2. Fast. It should be high-performing and hence linearly scalable (within the constraints of the network speed and bandwidth). In other words, moving an application from one to many nodes should not decrease the overall system performance.

3. Reliable. It should be easy to maintain and dependable.

In this article, we will highlight a couple of solutions available today for scaling web applications deployed on the Apache Tomcat Servlet Container (Tomcat) and compare these solutions based on simplicity, performance and reliability.

Tomcat:
Tomcat is one of the most widely used Servlet Containers available today and is one of the official Reference Implementations for the Java Servlet and JavaServer Pages technologies. It allows web applications to store transient data through the HttpSession interface, which it internally stores in a HashMap. It creates a unique session for each user, and allows the application to set and get attributes to and from the session. The basic requirement of clustering any web application with Tomcat is to distribute these sessions across multiple nodes in a cluster, for which we will look at the following options:

1. The Tomcat Sessions Clustering solution shipped with Apache Tomcat of the Apache Software Foundation.

2. Terracotta Sessions, which is one of the many clustering solutions offered by Terracotta, Inc.

Before we delve into the comparison, it is important to look at the underlying architecture of the two solutions and their approach towards clustering.

Tomcat Clustering:
Tomcat's homegrown clustering solution is built on a peer-to-peer architecture and replicates the session at the application level. In this approach, whenever the session changes, the entire session is serialized on disk and is sent to every other node in the cluster. The developer must be careful to call setAttribute() on the session to make sure the important changes are propagated across the cluster. How often and where the developer calls setAttribute() can be a little tricky (if you forget to call it, you won't know it's broken until a failover happens; if you call it too often, you have potential production performance-degradation on your hands that might take days or weeks to fix, provided you can find the root-cause).

Terracotta Sessions:
Terracotta Sessions uses Terracotta's Distributed Shared Objects (DSO) technology which is built on a hub-and-spoke architecture. This technology takes a radical approach of clustering at the JVM level, which has some powerful advantages. The solution drills down at the byte level and is able to figure out at run time the bytes that have changed, and only propagates those bytes. The hub-and-spoke architecture eradicates the need to send the changed bytes to every other node in the cluster. This also means that the session is clustered without writing any additional code (yes, no serialization required), which makes it very inviting with an operations perspective. Very cool.



Part 2 ->