Monthly Archives: October 2015

Where Do Applications Throttle

There are two areas where the application throttles namely: ‘Memory Space’ and Time to Process’ the data.

Here are some basic steps that you should do ensure that the java application doesn’t throttle for memory space. However, with the newer operating system like 64 bit operating systems you would find this issue would be reduced drastically as there would be ample area of memory pointer to point to. Currently in 32 bit operating system each pointer can point at max upto 32bit location, this means that the java application reference could map at max to 3.x GB (2^32=3.X*10^9) space only even though you may have 4GB ram in your machine.

Having said that the constraint of extending heap memory has been now extended because of the memory advantages that 64 bit operating system provides but this does not diminish the importance of using proper coding and understanding how the ‘out of memory’ exception works in general.

To understand the case better let me explain with few simple examples. I assume we (Reader and author as obects of Class ‘Human’) are reading this article (EXCLUDING FEW BOTS WHO VISITS OUR SITES FOR INDEXING PURPOSES). We are object level instances of class Human. So first a concept and behaviour of humans are created, which in JAVA parlance is called a class. The JVM stores every thing in terms of an Object so stores this object as Class (Note C letter of class is capital).This object is created before single instance of object is created for this class. This object resides in permanent memory and is not garbage collected (Should I say seldom garbage collected).

Now assume that we are using a third party software and patching code to enhance few functionalities or adding extra jars to incorporate another application with this third party software. Assume that both are using the same (say XYZ.jar) jar but they need two separate version of jar to work properly. In this case as the JVM can load only 1 class object per class so this strategy would fail. So in this case we use custom class loader to load the second jar now this class would be loaded two times and there wouldn’t be any class version mismatch issues.

Now in the above example we have understood how the classes are created before an object ever comes to existence, so now lets wonder what is static variable and where it resides. Assume that there is a static variable named avgHeight of Human, this holds the average heights of humans object in that JVM. So this data would also be stored in permanent memory. However, the height of my (human) object would contain my information and height containing in your object would contain your height. So where do my and your human objects reside? They reside in heap; these are runtime memory requirement of a program under execution. In case if your program uses lot of PERMANENT memory, it can be assumed that your programming style creates more static methods and static variables which is a violation of the basic OOPS design. Under this condition the application is expected to throw PERMGEN ERROR. Permgen exception is also a kind of exception that occurs if either a very low perm space is allocated or we are storing the application state in static variables. For example we would be storing all the employee details in a static hashMap. Permgen exception could either be recovered by raising the perm memory limit, or revisiting the code and fixing it. My suggestion as a performance engineer is to fix the root cause and identify and fix the issue by designing the application.

NOTE: -XX:MaxPermSize=256m .Parameter should be low to keep bad coding practices at bay.

Now how can I smartly create and ensure the objects (heap) are garbage collected so that the application can run smoothly without any problem. JVM would avoid collecting garbage objects which are no more in use when the GC runs or when the application is facing a throttling effect due to less heap size left to run the application. In these cases only the GC would collect the object and remove it from heap.

Now we can also keep our program in such a manner that the object moves out of scope and is not needed. Say for example, if you write a function and create a variable inside that function. When the application comes out of that function, the object created inside the function is now eligible for garbage collection. This is one of the strategies.

Another strategy is to keep business logic as singleton so that application don’t store the parameter and return results. For example productionLine Object would take an input and return an output.

Now how can I have 2 production line for plastic chair manufacturing and Wooden chair manufacturing. In this case, I would prefer to use an Enum and have named objects inside them performing similar work of reproducing chairs but their manufacturing methods remain different and unique. Now using this approach we can ensure not much objects are created even if it is created it would be garbage collected.

I also see one more area when the application starts throttling badly when we read a BIG stream and convert that in a collection of object. This is extremely interesting case where the JVM needs to hold the byte array that it has received from network/DB/Filesystem and then creates equivalent number of objects and then put all those in a collection. In taking this approach we are in difficult spot of having two times the required object; namely:
a. Byte array of objects that has arrived from network or other mediums.
b. Collection of objects inside the collection.

To overcome this issue I create my custom Collection which is extending externalizable interface. This interface would allow us to consume objects as and when we receive the objects from stream. This would avoid a spike in memory. So this kind of exception occurs while reading/ writing (collection of objects) to another medium.

One more area of concern is when stream reading/writing occurs with NIO. With NIO we dont wait for streaming to complete we rather wait in some queue and get notification when the IO has been successfully complete this way we increase the throughput of an application. However, we MAY loose on memory spike side so we have to design our thread pool (ExecutorService) in a way that it just creates optimal performance with out a huge spike in memory.

What else can be done from programming perspective that can help in making more scalable solution? The thing that comes to my mind is using of proper Reference Object for referring objects, for example using weak and soft reference depending on the scenario can help in many cases. Using WeakHashMap from various open source collection framework provider can also help at times.

a. Set -XX:+UseParallelGC parallel (throughput) garbage collector
b. initialHeap size settings: -Xms initial heap size is 1/64th of the machine’s physical memory.
c. -Xmx maximum heap size is 1/4th of the machine’s physical memory
d. XX:SurvivalRatio=[ratio integer] smaller means more overstay in heap.
e. -XX:TargetSurvivalRatio=[%] default 50% allow more usage of survival ratio.

a. Keep the application state, and purge state just by moving out of scope of the reference
b. If using Static variable think @ using WeakReferences.
c. Evaluate the application does it contains enough variables[non static],I have observed in applications that don’t have any variables so either the whole state is carried forward using arguments( this approach finally end ups in abusing overriding – for example your application may have 4 different checkPnL with variable arguments ) or used static variable state maintenance. Note: both approaches are bad.

FINALLY IF NOTHING WORKS THEN OPEN A CHECK IN YOUR PROFILING TOOL. So approach should be to design in such a way that performance bottleneck doesn’t occur at all.