Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
3.1.0, 3.0.0, 4.0.0
Description
Hiveserver2 instances where crashing every 3-4 days and observed HS2 in on unresponsive state. Also, observed that the FGC collection happening regularly
From JXray report it is seen that pmCache(List of JDOPersistenceManager objects) is occupying 84% of the heap and there are around 16,000 references of UDFClassLoader.
10,759,230K (84.7%) Object tree for GC root(s) Java Static org.apache.hadoop.hive.metastore.ObjectStore.pmf - org.datanucleus.api.jdo.JDOPersistenceManagerFactory.pmCache ↘ 10,744,419K (84.6%), 1 reference(s) - j.u.Collections$SetFromMap.m ↘ 10,744,419K (84.6%), 1 reference(s) - {java.util.concurrent.ConcurrentHashMap}.keys ↘ 10,743,764K (84.5%), 16,872 reference(s) - org.datanucleus.api.jdo.JDOPersistenceManager.ec ↘ 10,738,831K (84.5%), 16,872 reference(s) ... 3 more references together retaining 4,933K (< 0.1%) - java.util.concurrent.ConcurrentHashMap self 655K (< 0.1%), 1 object(s) ... 2 more references together retaining 48b (< 0.1%) - org.datanucleus.api.jdo.JDOPersistenceManagerFactory.nucleusContext ↘ 14,810K (0.1%), 1 reference(s) ... 3 more references together retaining 96b (< 0.1%)
When the RawStore object is re-created, it is not allowed to be updated into the ThreadWithGarbageCleanup.threadRawStoreMap which leads to the new RawStore never gets cleaned-up when the thread exit.
Attachments
Attachments
Issue Links
- links to
