Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20404

Ugly cleanerchore complaint that dir is not empty

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.4.4, 2.0.0
    • 1.3.3, 1.4.4, 2.0.0
    • master
    • None

    Description

      I see these big dirty exceptions in my master log during a long-run.... Lets clean them up (Are they exceptions I as an operator can actually do something about? Are they 'problems'? Should they be LOG.warn?)

      2018-04-12 16:02:09,911 WARN  [ForkJoinPool-1-worker-15] cleaner.CleanerChore: Could not delete dir under hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta; {}
      org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): `/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta is non empty': Directory is not empty
        at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2848)
        at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1048)
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:641)
        at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:447)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
        at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:847)
        at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:790)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2486)
      
        at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1489)
        at org.apache.hadoop.ipc.Client.call(Client.java:1435)
        at org.apache.hadoop.ipc.Client.call(Client.java:1345)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
        at com.sun.proxy.$Proxy26.delete(Unknown Source)
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:568)
        at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:409)
        at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163)
        at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155)
        at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:346)
        at com.sun.proxy.$Proxy27.delete(Unknown Source)
        at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source)
      
      ...
      

      Looks like log format is off too...

      Attachments

        1. HBASE-20404.0.patch
          1 kB
          Sean Busbey
        2. HBASE-20404.1.patch
          1 kB
          Sean Busbey
        3. HBASE-20404.2.patch
          8 kB
          Sean Busbey

        Activity

          People

            busbey Sean Busbey
            stack Michael Stack
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: