Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
1.5.6
-
None
-
None
-
None
Description
As part of our effort to find classloader leaks in Apache Flink, we found the following issue coming from OrcFile.
private static synchronized MemoryManager getStaticMemoryManager( final Configuration conf) { if (memoryManager == null) { memoryManager = new ThreadLocal<MemoryManager>() { @Override protected MemoryManager initialValue() { return new MemoryManagerImpl(conf); } }; } return memoryManager.get(); }
Here the original conf will be used for all other memory managers in the future. If you close the classloader of that conf, future usages of any MemoryManager coming through that ThreadLocal will fail.
For our use case, where there is only one conf/MemoryManager per Thread, it would be sufficient, to explicitly initialize the thread locals.
private static synchronized MemoryManager getStaticMemoryManager( final Configuration conf) { MemoryManager manager = memoryManager.get(); if (manager == null) { manager = new MemoryManagerImpl(conf); memoryManager.set(manager); } return manager; }
I can provide a patch and additional information.
Attachments
Issue Links
- is fixed by
-
ORC-361 org.apache.orc.impl.MemoryManagerImpl : Owner thread expected Thread[main,5,main], got Thread[pool-15-thread-1,5,main]
- Closed