There are all these wonderful tools for profiling Java programs. JVisualVM, JProfiler, Mission Control and the Flight recorder, jhat, just to name a few. Yet every once in a while, you can't use any of them, for one reason or another. Where do we go from here?
How come there's no profiler?
The situation is not quite as exotic as you may think. When I was asked recently to write programs in the middleware layer, the lack of tools was the one of the reasons why I denied. More precisely, I was asked to write Java programs running in the webMethods ESB. In theory, that's a great idea, but the lack of tools makes programming a pain. No debugger. No profiler.
In theory, you can debug such a program using remote debugging, but that requires many preconditions to be fulfilled. The administrator has to start the program with additional parameters, and they have to open the ports in the firewall. Sometimes that's possible in the development and test stages, but usually it's completely out of question in the production stage.
However, being a consultant, sometimes I'm asked to help our customer nonetheless. That was the situation I found myself today in. One of our customer's programs crashed due to an OutOfMemoryException. Thing is, the program to crash runs in a Talend server. I don't know many of the details, but I'm told we can't debug our programs. Actually, that's a situation familiar to me: more often than not, you aren't allowed to debug a program running in an ESB remotely. Especially, if the problem occurs only in the production environment.
Time for plan B.
The OutOfMemoryError indicates that our program creates a lot of objects without releasing them again. My first step to solve such a problem is to identify which of the many objects of the program causes the problem. That, in turn, boils down to simply counting the objects in memory and looking at the most frequent objects. Usually that's character arrays and strings, but usually one of the next top hit on the lists is the trouble-maker.
Write your profiler yourself
Googling for "How to count Java objects in memory" reveals ... well, many things, none of them useful. That's pretty surprising, because it's very simple to count the number of objects in main memory. Plus, it's a really useful technique if you're suffering from a memory leak.
Skipping the inevitable boiler plate code, it's as simple as that:
final VirtualMachine vm = ac.attach(connectArgs); final ListThere's a drawback: you need to configure your Java program properly. You have to add the tools.jar from the JDK. The annoying thing is that there isn't a standard way to do it using Maven. And you have to open a network port in the firewall. The API I'm presenting today works similarly to remote debugging. You have to deal with two separate programs: your payload program (that's the program already causing trouble) and a diagnosis program. Both communicate with each other via the network.
To run the program, you have to start the program under test in debug mode. In other words, you have to add some start parameters:
-Xdebug -Xrunjdwp:transport=dt_socket,address=3001,server=y,suspend=nUsing the Java Debug Interface (JDI) for introspection
The API I just described has been invented to enable a Java program to analyze another Java program. For instance, the debugger of Eclipse uses this API to debug your web application. However, Java programs can also use this API to analyze themselves. This way, you don't have to ask your operations department to drill a hole into the firewall. It suffices to allow the program to access the local debug port (i.e. 127.0.0.1:3001 in our example).
Alternative solutions
While using the JDI is the most efficient and most powerful approach, you're hardly ever allowed to use it. So I gathered two other approaches which may be useful, too.
When you're looking for "how to count Java objects in memory", many posts suggest to add a counter in the constructor. In my case, I'm not interested in how many objects have been created, but in how many objects have been created and haven't been removed by the garbage collector. So you have to add a finalizer, too. The most simple approach looks like so:
public class SimpleApproachBean { private volatile static int counter=0; public SimpleApproachBean() { counter++; } @Override protected void finalize() throws Throwable { counter--; super.finalize(); } }Unfortunately, this simple approach is a bit cumbersome if you're interested in more than one or two classes. When you start investigating a performance issue, you hardly ever know in advance which class runs havoc. If you're using AspectJ, you can add code to every constructor of your project:
@Aspect @Component public class CommonLogAspect implements ILogAspect { private static int counter=0; @Before("execution(*.new(..))") public void countingConstructor() { counter++; } }As far as I know, it's impossible to decorate a finalizer with AspectJ (or any other AOP tool I know of). After some investigation, I learned it's possible to register a Garbage collector callback using the ReferenceQueue class. The implementation is a bit cumbersome, but it works fine. It requires a ReferenceQueue to watch the object, a dedicated thread to watch the reference queue and a variable to store the ReferenceQueue. You know, the queue must be protected from being removed by the garbage collector prematurely. I've implemented a fairly generic solution: