Description
We have a memory leak in the application. If we stop and run Jetty in the loop in JUnit we get OOM.
The issue is in the following lines. I think Log4j attaches an object to the thread via the get method (because the field is defined like this:
private static final ThreadLocal<DefaultLogBuilder> logBuilder = ThreadLocal.withInitial(DefaultLogBuilder::new);
It means that Log4j attaches an object to a thread even Constants.ENABLE_THREADLOCALS is disabled. My thoughts:
1) Log4j code attaches DefaultLogBuilder object to threads
2) DefaultLogBuilder object has a link to DefaultLogBuilder.class that has a link to DefaultLogBuilder.class.getClassloader()
3) DefaultLogBuilder.class.getClassloader() is a Web App Classloader and during our JUnit testing many classes are loaded to the classloader
4) I suspect somewhere some Thread Pool is used and even if we clean all resources in the application, Log4j holds classloaders and all loaded classes via DefaultLogBuilder.class.getClassloader()
I suggest checking Constants.ENABLE_THREADLOCALS variable before calling logBuilder.get() in org.apache.logging.log4j.spi.AbstractLogger. After the following changes, the problem is gone. (except additionally I added log4j2.disable.jmx=true, because it has a similar issue, but I think it is not a log4j issue)
/** * Returns a log builder that logs at the specified level. * * @since 2.20.0 */ protected LogBuilder getLogBuilder(Level level) { if (Constants.ENABLE_THREADLOCALS) { DefaultLogBuilder builder = logBuilder.get(); if (!builder.isInUse()) { return builder.reset(this, level); } } return new DefaultLogBuilder(this, level); }
Original code:
/** * Returns a log builder that logs at the specified level. * * @since 2.20.0 */ protected LogBuilder getLogBuilder(Level level) { DefaultLogBuilder builder = logBuilder.get(); return Constants.ENABLE_THREADLOCALS && !builder.isInUse() ? builder.reset(this, level) : new DefaultLogBuilder(this, level); }
P.S. Please, if it is possible can you include the fix in an upcoming release, the issue randomly happens during our application build and it is a headache for us Thank you.
P.S.S. We use a free license of Yourkit for open-source projects. Please, look at the picture from a profiler.