Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
Reviewed
Description
Our XiaoMi HBase team are evaluating the performence improvement of HBASE-21879, and we have few CPU flame graph & heap flame graph by using async-profiler, and find that there're some performence issues in DFSClient .
See the attached two flame graphs, we can conclude that the try catch block in ShortCircuitCache#trimEvictionMaps has some serious perf problem , we should remove the try catch from DFSClient.
/** * Trim the eviction lists. */ private void trimEvictionMaps() { long now = Time.monotonicNow(); demoteOldEvictableMmaped(now); while (true) { long evictableSize = evictable.size(); long evictableMmappedSize = evictableMmapped.size(); if (evictableSize + evictableMmappedSize <= maxTotalSize) { return; } ShortCircuitReplica replica; try { if (evictableSize == 0) { replica = (ShortCircuitReplica)evictableMmapped.get(evictableMmapped .firstKey()); } else { replica = (ShortCircuitReplica)evictable.get(evictable.firstKey()); } } catch (NoSuchElementException e) { break; } if (LOG.isTraceEnabled()) { LOG.trace(this + ": trimEvictionMaps is purging " + replica + StringUtils.getStackTrace(Thread.currentThread())); } purge(replica); } }
Our Xiaomi HDFS Team member leosun08 will prepare patch for this issue.
Attachments
Attachments
Issue Links
- is related to
-
HDFS-15161 When evictableMmapped or evictable size is zero, do not throw NoSuchElementException in ShortCircuitCache#close()
- Resolved
- relates to
-
HBASE-22387 Evaluate the get/scan performance after reading HFile block into offheap directly
- Closed
-
HBASE-21879 Read HFile's block to ByteBuffer directly instead of to byte for reducing young gc purpose
- Closed
- links to