Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-16039

Incremental backup action failed with NPE when table in full backup is deleted in between

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • HBASE-7912
    • None
    • Reviewed

    Description

      Incremental backup action failed with NPE.
      Creating a full backup went fine but creating an incremental backup failed

      hbase@cluster_name:~$ hbase backup create incremental hdfs://cluster-name:8020/user/hbase "table_02uvzkggro"
      
      2016-06-15 06:38:28,605 INFO  [main] util.BackupClientUtil: Using existing backup root dir: hdfs://cluster-name:8020/user/hbase
      2016-06-15 06:38:30,483 ERROR [main] util.AbstractHBaseTool: Error running command-line tool
      org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException): 
      at org.apache.hadoop.hbase.backup.master.FullTableBackupProcedure.cleanupTargetDir(FullTableBackupProcedure.java:198)
      at org.apache.hadoop.hbase.backup.master.FullTableBackupProcedure.failBackup(FullTableBackupProcedure.java:276)
      at org.apache.hadoop.hbase.backup.master.IncrementalTableBackupProcedure.executeFromState(IncrementalTableBackupProcedure.java:186)
      at org.apache.hadoop.hbase.backup.master.IncrementalTableBackupProcedure.executeFromState(IncrementalTableBackupProcedure.java:54)
      at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:107)
      at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:443)
      at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:934)
      at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:736)
      at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:689)
      at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$200(ProcedureExecutor.java:73)
      at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$1.run(ProcedureExecutor.java:416)
      

      from Master log

      2016-06-15 06:38:29,875 ERROR [ProcedureExecutorThread-3] master.FullTableBackupProcedure: Unexpected exception in incremental-backup: incremental copy backup_1465972709112org.apache.hadoop.hbase.TableInfoMissingException: No table descriptor file under hdfs://cluster-name:8020/apps/hbase/data/data/default/table_pjtxpp3r74
      org.apache.hadoop.hbase.backup.impl.BackupException: org.apache.hadoop.hbase.TableInfoMissingException: No table descriptor file under hdfs://cluster-name:8020/apps/hbase/data/data/default/table_pjtxpp3r74
              at org.apache.hadoop.hbase.backup.util.BackupServerUtil.copyTableRegionInfo(BackupServerUtil.java:196)
              at org.apache.hadoop.hbase.backup.master.IncrementalTableBackupProcedure.executeFromState(IncrementalTableBackupProcedure.java:178)
              at org.apache.hadoop.hbase.backup.master.IncrementalTableBackupProcedure.executeFromState(IncrementalTableBackupProcedure.java:54)
              at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:107)
              at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:443)
              at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:934)
              at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:736)
              at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:689)
              at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$200(ProcedureExecutor.java:73)
              at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$1.run(ProcedureExecutor.java:416)
      Caused by: org.apache.hadoop.hbase.TableInfoMissingException: No table descriptor file under hdfs://cluster-name:8020/apps/hbase/data/data/default/table_pjtxpp3r74
              at org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:509)
              at org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:496)
              at org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:476)
              at org.apache.hadoop.hbase.backup.util.BackupServerUtil.copyTableRegionInfo(BackupServerUtil.java:172)
              ... 9 more
      2016-06-15 06:38:29,875 INFO  [ProcedureExecutorThread-3-EventThread] zookeeper.ClientCnxn: EventThread shut down
      2016-06-15 06:38:29,875 ERROR [ProcedureExecutorThread-3] master.FullTableBackupProcedure: BackupId=backup_1465972709112,startts=1465972709342,failedts=1465972709875,failedphase=null,failedmessage=org.apache.hadoop.hbase.TableInfoMissingException: No table descriptor file under hdfs://cluster-name:8020/apps/hbase/data/data/default/table_pjtxpp3r74
      2016-06-15 06:38:29,884 ERROR [ProcedureExecutorThread-3] procedure2.ProcedureExecutor: CODE-BUG: Uncatched runtime exception for procedure: IncrementalTableBackupProcedure (targetRootDir=hdfs://cluster-name:8020/user/hbase) id=100 state=FINISHED
      java.lang.NullPointerException
              at org.apache.hadoop.hbase.backup.master.FullTableBackupProcedure.cleanupTargetDir(FullTableBackupProcedure.java:198)
              at org.apache.hadoop.hbase.backup.master.FullTableBackupProcedure.failBackup(FullTableBackupProcedure.java:276)
              at org.apache.hadoop.hbase.backup.master.IncrementalTableBackupProcedure.executeFromState(IncrementalTableBackupProcedure.java:186)
              at org.apache.hadoop.hbase.backup.master.IncrementalTableBackupProcedure.executeFromState(IncrementalTableBackupProcedure.java:54)
              at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:107)
              at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:443)
              at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:934)
              at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:736)
              at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:689)
              at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$200(ProcedureExecutor.java:73)
              at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$1.run(ProcedureExecutor.java:416)
      

      Attachments

        1. HBASE-16039-v1.patch
          8 kB
          Vladimir Rodionov

        Issue Links

          Activity

            People

              vrodionov Vladimir Rodionov
              romil.choksi Romil Choksi
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: