Details
Description
the hlog roller thread and hlog syncer thread may occur dead lock with the 'flushLock' and 'updateLock', and then cause all 'IPC Server handler' thread blocked on hlog append. the jstack info is as follow :
"regionserver60020.logRoller":
at org.apache.hadoop.hbase.regionserver.wal.HLog.syncer(HLog.java:1305)
- waiting to lock <0x000000067bf88d58> (a java.lang.Object)
at org.apache.hadoop.hbase.regionserver.wal.HLog.syncer(HLog.java:1283)
at org.apache.hadoop.hbase.regionserver.wal.HLog.sync(HLog.java:1456)
at org.apache.hadoop.hbase.regionserver.wal.HLog.cleanupCurrentWriter(HLog.java:876)
at org.apache.hadoop.hbase.regionserver.wal.HLog.rollWriter(HLog.java:657) - locked <0x000000067d54ace0> (a java.lang.Object)
at org.apache.hadoop.hbase.regionserver.LogRoller.run(LogRoller.java:94)
at java.lang.Thread.run(Thread.java:662)
"regionserver60020.logSyncer":
at org.apache.hadoop.hbase.regionserver.wal.HLog.syncer(HLog.java:1314) - waiting to lock <0x000000067d54ace0> (a java.lang.Object)
- locked <0x000000067bf88d58> (a java.lang.Object)
at org.apache.hadoop.hbase.regionserver.wal.HLog.syncer(HLog.java:1283)
at org.apache.hadoop.hbase.regionserver.wal.HLog.sync(HLog.java:1456)
at org.apache.hadoop.hbase.regionserver.wal.HLog$LogSyncer.run(HLog.java:1235)
at java.lang.Thread.run(Thread.java:662)