Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20828 Finish-up AMv2 Design/List of Tenets/Specification of operation
  3. HBASE-21278

Do not rollback successful sub procedures when rolling back a procedure

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • None
    • 3.0.0-alpha-1, 2.2.0
    • proc-v2
    • None
    • Reviewed
    • Hide
      For the sub procedures which are successfully finished, do not do rollback. This is a change in rollback behavior.

      State changes which are done by sub procedures should be handled by parent procedures when rolling back. For example, when rolling back a MergeTableProcedure, we will schedule new procedures to bring the offline regions online instead of rolling back the original procedures which off-lined the regions (in fact these procedures can not be rolled back...).
      Show
      For the sub procedures which are successfully finished, do not do rollback. This is a change in rollback behavior. State changes which are done by sub procedures should be handled by parent procedures when rolling back. For example, when rolling back a MergeTableProcedure, we will schedule new procedures to bring the offline regions online instead of rolling back the original procedures which off-lined the regions (in fact these procedures can not be rolled back...).

    Description

      https://builds.apache.org/job/HBase-Flaky-Tests/job/master/1235/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt/*view*/

      I think the problem is

      2018-10-08 03:44:30,315 INFO  [PEWorker-1] procedure.MasterProcedureScheduler(689): pid=43, ppid=42, state=SUCCESS, hasLock=false; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN checking lock on 9bac7c539ac0cff6dc5706ed375a3bfb
2018-10-08 03:44:30,320 ERROR [PEWorker-1] helpers.MarkerIgnoringBase(159): CODE-BUG: Uncaught runtime exception for pid=43, ppid=42, state=SUCCESS, hasLock=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN
java.lang.UnsupportedOperationException
	at org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:458)
	at org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:97)
	at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.rollback(StateMachineProcedure.java:208)
	at org.apache.hadoop.hbase.procedure2.Procedure.doRollback(Procedure.java:957)
	at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1605)
	at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1567)
	at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1446)
	at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$900(ProcedureExecutor.java:76)

      Typically there is no rollback for TRSP. Need to dig more.

      Attachments

        Issue Links

          relates to

          Sub-task - The sub-task of the issue HBASE-21321 Backport HBASE-21278 to branch-2.1 and branch-2.0

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          links to

          Web Link Review Board

          Activity

            People

              zhangduo Duo Zhang
              zhangduo Duo Zhang
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: