Today i was trying to do memory analysis on a remote tomcat and I faced some issues while setting up the jConsole. Here I am documenting all the issues I faced and the solutions to these issues:-Note : This setup is for the simple development environment with one user. For more advanced case like enabling the security refer here
Enable and configure JMX agent on the remote VM:
Configuration
These are the properties i set in my tomcat setenv.sh and this file does not exist by default in the tomcat 7 so I had to create it:
export CATALINA_OPTS="$CATALINA_OPTS -Dcom.sun.management.jmxremote.port=8999 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=remotehostip"
Issues Faced
Here are the issues which I faced before I arrived to the above configuration:
Issue 1: Read Access must be restricted
This error I recived on that startup of tomcat.
Password file read access must be restricted: /opt/jdk1.7.0_51/jre/lib/management/jmxremote.password
Solution: Do as the error says, give only the read access to the password file
chmod 600 jmxremote.password
Issue 2: java.rmi.server.ExportException
On tomcat shutdown I used to get this exception
Error: Exception thrown by the agent : java.rmi.server.ExportException: Port already in use: 8999; nested exception is:  java.net.BindException: Address already in use
And I had to so
sudo kill tomcatpid
.Solution: The solution to this error was to set the jmx properties on CATALINA_OPTS and this way it only is executed on startup
Wrong way of doing it:
JAVA_OPTS="$JAVA_OPTS -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=8999"
Right way:
CATALINA_OPTS="$CATALINA_OPTS -Dcom.sun.management.jmxremote.port=8999 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Djava.rmi.server.hostname=79.30.130.92"
After this change tomcat shuts down without any errors
Issue 3 : jConsole on the client machine not connecting
Solution :Here is what you can do to debug the issue:
Start jConsole with this command that enables the logging in jconsole.
jconsole -J-Djava.util.logging.config.file=\temp\jmx\log\logging.properties remote-host-ip:8999
Here is the sample logging.properties that I used:
handlers = java.util.logging.ConsoleHandler .level = INFO java.util.logging.ConsoleHandler.level=FINEST java.util.logging.ConsoleHandler.formatter=java.util.logging.SimpleFormatter // Level FINEST is suitable for diagnosing SSL-related JMX remote // connection issues. javax.management.level=FINEST javax.management.remote.level=FINEST
So you will see a log console opening with jconsole and you will have better idea why it is not connection.After enabling the logging i found out that jconsole was not printing the correct ip for remote host, so I introduced this property
-Djava.rmi.server.hostname=remotehostip
where remotehostip is the ip of the system on which tomcat is running.
I had to add this property because the hostname was not correctly setup on the vm running tomcat. You can see it by running the following command in the console:
hostname -i
If this command does not return the actual ip of the system then the hostname is not setup correctly. You can either setup the hostname correctly or use the above property. After that change the jConsole started connecting.