We have a number of wars in one tomcat. And recently we upgraded from log4j1.x to log4j2.x. And since then we have faced a lot of issues. One of them is listed here.
After the above issue we took out the log4j2 jars and placed them in the tomcat lib folder. But we didn’t move the configuration out of the web.xml as we had defined them.The following configuration was in web.xml:
<!-- log4j2 configuration --> <context-param> <description>define the path of log4j2 config file</description> <param-name>log4jConfiguration</param-name> <param-value>${catalina.home}/conf/log4j2.xml</param-value> </context-param> <context-param> <description>Makes all loggers asynchronous</description> <param-name>Log4jContextSelector</param-name> <param-value>org.apache.logging.log4j.core.async.AsyncLoggerContextSelector</param-value> </context-param> <!-- end - log4j2 configuration -->
So as we went on, the application was working correctly without any errors. Soon we had an issue which forced us do the memory analysis of the application. And soon we found out that log4j2 thread was still hanging in the tomcat after the webapp was undeployed.
Eclipse MAT showed the following Leak suspect:
The classloader/component "org.apache.catalina.loader.WebappClassLoader @ 0x700300ee0" occupies 2'575'368 (20.14%) bytes. The memory is accumulated in one instance of "java.lang.Object[]" loaded by "<system class loader>". Keywords java.lang.Object[] org.apache.catalina.loader.WebappClassLoader @ 0x700300ee0
And then the path led directly to log4j2.
Class Name Shallow Heap Retained Heap java.lang.Object[10240] @ 0x700b65a38 40'976 2'219'552 elementData java.util.Vector @ 0x70031afd8 32 2'219'584 .classes org.apache.catalina.loader.WebappClassLoader @ 0x700300ee0 192 2'575'368 ..classLoader org.apache.logging.log4j.spi.Provider @ 0x70069d698 32 336 ...key java.util.HashMap$Entry @ 0x70069d678 32 368 ....[3] java.util.HashMap$Entry[16] @ 0x70069d420 80 968 .....table java.util.HashMap @ 0x70069d3e0 48 1'032 ......map java.util.HashSet @ 0x70069d3d0 16 1'048 .......PROVIDERS class org.apache.logging.log4j.util.ProviderUtil @ 0x70069d300 32 1'464 ........[506] java.lang.Object[2560] @ 0x7009dd498 10'256 369'592 .........elementData java.util.Vector @ 0x7002a68c0 32 369'624 ..........classes org.apache.catalina.loader.StandardClassLoader @ 0x7002934e8 80 466'792 ...........+contextClassLoader java.lang.Thread @ 0x7004100d0 main Native Stack, Thread 104 26'296
So the fix was to remove the log4j2 configuration out of the web.xml and put it in the catalina.properties.
After that taking the heap dump showed the below leak suspect which was expected as it was started by the StandardClassLoader.
One instance of "com.lmax.disruptor.RingBuffer" loaded by "org.apache.catalina.loader.StandardClassLoader @ 0x7000e5a48" occupies 19'923'288 (59.30%) bytes. The memory is accumulated in one instance of "java.lang.Object[]" loaded by "". Keywords java.lang.Object[] com.lmax.disruptor.RingBuffer org.apache.catalina.loader.StandardClassLoader @ 0x7000e5a48
Comments and suggestions are welcome.
Have you figured out any solution for this problem? We are also facing same issue One instance of One instance of “com.lmax.disruptor.RingBuffer” loaded by “sun.misc.Launcher$AppClassLoader @ 0xc0000000” occupies 28,311,968 (22.80%) bytes. The memory is accumulated in one instance of “java.lang.Object[]” loaded by “”., no solution yet.
So, once you load it with tomcat startup, it will always be there. That should be fine.
Great post. I am facing a couple of these problems.
I hope you have already the issues solved