Description
In an effort to speed up logging ThreadLocals have been introduced see LOG4J2-1125 however this does causes memory issues.
The problem of the ThreadLocal occurs when threads are re-used which is an absolutely valid way of using java. For example an executor service can re-use threads as well as Jetty.
Below I demonstrate a contrived example of the memory leak:
int stringSize = 1024*1024*10; //~10MB maybe 20MB for UTF-16 StringBuilder sb = new StringBuilder(stringSize); for(int i = 0; i < stringSize; i++) { sb.append('a' + i % 5); } String largeString = sb.toString(); sb = null; //Let it be GC'ed ExecutorService es = Executors.newFixedThreadPool(100); final CountDownLatch countDownLatch = new CountDownLatch(100); for(int i = 0; i < 100; i++) { es.execute(()-> { //Log the big string to demonstrate the issue. log.fatal(largeString); //Ensure we use all 100 of our threads by not releasing this thread yet. countDownLatch.countDown(); }); //We sleep for 2s so we more easily watch memory growth Thread.sleep(2000); }
I recommend that log4j2 immediately remove the ThreadLocal as a small gain in performance does not outweigh the problems associated with memory leaks. Finally other options for caching the StringBuilder with a ThreadLocal could be considered for example we might re-use StringBuilders that are no larger than 3k while removing the ones which are larger than 3k.
Attachments
Issue Links
- Blocked
-
LOG4J2-1125 Reuse StringBuilder to improve performance for PatternLayout
-
- Closed
-