Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-14350

Procedure V2 Phase 2: Assignment Manager

    XMLWordPrintableJSON

Details

    • Task
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 2.0.0
    • 2.0.0-beta-1, 2.0.0
    • master, proc-v2
    • None
    • Hide
      (Incomplete)

      = Incompatbiles

      == Coprocessor Incompatibilities

      Split/Merge have moved to the Master; it runs them now. Means hooks around Split/Merge are now noops. To intercept Split/Merge phases, CPs need to intercept on MasterObserver.
      Show
      (Incomplete) = Incompatbiles == Coprocessor Incompatibilities Split/Merge have moved to the Master; it runs them now. Means hooks around Split/Merge are now noops. To intercept Split/Merge phases, CPs need to intercept on MasterObserver.

    Description

      This is the second phase of Procedure V2 (HBASE-12439). Built on top of Phase 1 (HBASE-14336), implement a new Assignment Manager based on Proc-V2.

      Attachments

        Issue Links

          blocks

          Bug - A problem which impairs or prevents the functions of the product. HBASE-16537 Add tests to verify create/modify table region reopen with missing coprocessor

          • Major - Major loss of function.
          • Open
          Is contained by

          New Feature - A new feature of the product, which has yet to be developed. HBASE-12439 Procedure V2

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HBASE-10025 Region stuck in transition after failed RS_ZK_REGION_FAILED_OPEN

          • Major - Major loss of function.
          • Closed
          links to

          Web Link AMv2 for Devs

          Web Link AMv2 Overview

          Web Link Setup for Phase 2, the Assignment Manager

          1.
          		 Procedure v2: Core Assignment Manager Sub-task Closed Matteo Bertozzi
          2.
          		 Procedure v2: Replace the old AM with the new AM Sub-task Closed Matteo Bertozzi
          3.
          		 Procedure V2: Implement move shell command to use Proc-V2 assignment Sub-task Closed Unassigned
          4.
          		 Procedure V2: unnecessaery operations on AssignmentManager#recoverTableInDisablingState() and recoverTableInEnablingState() Sub-task Closed Stephen Yuan Jiang
          5.
          		 Procedure V2: Implement single region assignment procedure from Master-side Sub-task Closed Unassigned
          6.
          		 Procedure V2: Update ServerCrashProcedure to interact with assignment procedures Sub-task Closed Unassigned
          7.
          		 Remove unused LoadBalancer.immediateAssignment() Sub-task Closed Matteo Bertozzi
          8.
          		 Procedure V2 - Completed support parent-child procedure Sub-task Closed Stephen Yuan Jiang
          9.
          		 Replace master.am.getTableStateManager() with the direct master.getTableStateManager() Sub-task Closed Matteo Bertozzi
          10.
          		 replace master.am and master.sm direct access with getter calls Sub-task Closed Matteo Bertozzi
          11.
          		 Remove HMaster.assignRegion() Sub-task Closed Matteo Bertozzi
          12.
          		 Procedure V2 - Procedure Queue with Regions Sub-task Closed Matteo Bertozzi
          13.
          		 Procedure v2 - Improve cleaning up of proc wals Sub-task Closed Apekshit Sharma
          14.
          		 Procedure v2 - Reimplement split Sub-task Closed Stephen Yuan Jiang
          15.
          		 Procedure v2 - Reimplement merge Sub-task Closed Stephen Yuan Jiang
          16.
          		 Require only MasterServices to the ServerManager constructor Sub-task Closed Matteo Bertozzi
          17.
          		 Procedure V2: Reimplement DispatchMergingRegionHandler Sub-task Closed Stephen Yuan Jiang
          18.
          		 Move meta bootstrap out of HMaster Sub-task Closed Matteo Bertozzi
          19.
          		 Procedure V2: Support acquire/release shared table lock concurrently Sub-task Closed Stephen Yuan Jiang
          20.
          		 Procedure V2 - Add support to addChildProcedure() as last "step" in StateMachineProcedure Sub-task Closed Matteo Bertozzi
          21.
          		 Separate Create/Modify Table operations from open/reopen regions Sub-task Closed Matteo Bertozzi
          22.
          		 Procedure v2 - Add handling of split region transition to the new AM Sub-task Closed Matteo Bertozzi
          23.
          		 Procedure v2 - Add new AM metrics Sub-task Closed Umesh Agashe
          24.
          		 Procedure V2 - Recover 'updated' part of WAL tracker if trailer is corrupted. Sub-task Closed Apekshit Sharma
          25.
          		 Procedure v2 - Compute WALs cleanup on wal modification and not on every sync Sub-task Closed Matteo Bertozzi
          26.
          		 Procedure v2 - Support for parent holding locks Sub-task Closed Matteo Bertozzi
          27.
          		 Procedure V2 - Lock procedures to allow clients to acquire locks on tables/namespaces/regions Sub-task Closed Matteo Bertozzi
          28.
          		 Procedure V2 - Move ZK-lock's uses to Procedure framework locks (LockProcedure) Sub-task Closed Matteo Bertozzi
          29.
          		 Procedure V2 - Remove org.apache.hadoop.hbase.zookeeper.lock Sub-task Closed Apekshit Sharma
          30.
          		 Procedure v2 - Fix StateMachineProcedure support for child procs at last step Sub-task Closed Matteo Bertozzi
          31.
          		 Procedure v2 - Inherit lock from root proc Sub-task Closed Matteo Bertozzi
          32.
          		 Procedure V2 - Check ACLs for remote HBaseLock Sub-task Closed Apekshit Sharma
          33.
          		 Procedure v2 - add waiting procs back to the queue after restart Sub-task Closed Matteo Bertozzi
          34.
          		 Procedure v2 - A couple of tweaks to the SplitTableRegionProcedure Sub-task Closed Unassigned
          35.
          		 Procedure v2 - A couple of tweaks to the SplitTableRegionProcedure Sub-task Closed Matteo Bertozzi
          36.
          		 Procedure v2 - remove tryAcquire*Lock and use wait/wake to make framework event based Sub-task Closed Matteo Bertozzi
          37.
          		 Procedure v2 - inherit region locks Sub-task Closed Matteo Bertozzi
          38.
          		 Remove merge region code from region server Sub-task Closed Stephen Yuan Jiang
          39.
          		 Remove split region code from Region Server Sub-task Closed Stephen Yuan Jiang
          40.
          		 Procedure v2 - cleanup isSuspended from MasterProcedureScheduler#Queue Sub-task Closed Apekshit Sharma
          41.
          		 Procedure v2 - Add handling of merge region transition to the new AM Sub-task Closed Matteo Bertozzi
          42.
          		 Readd TestMergeTableRegionsProcedure mistakenly removed by HBASE-16786 Sub-task Closed Michael Stack
          43.
          		 Tooling to move table out of FAILED_OPEN state Sub-task Closed Unassigned
          44.
          		 Create LoadBalancer perf-tests (test balancer algorithm decoupled from workload) Sub-task Closed Umesh Agashe
          45.
          		 Subset of HBASE-14614, Procedure v2: Core Assignment Manager (non-critical related changes) Sub-task Closed Michael Stack
          46.
          		 Implement abort for TruncateTableProcedure Sub-task Closed Umesh Agashe
          47.
          		 Support abort for all procedures by default Sub-task Closed Umesh Agashe
          48.
          		 Fix unit tests for TestTableFavoredNodes Sub-task Closed Umesh Agashe
          49.
          		 Add API for who currently holds a lock on namespace/ table/ region and log when state is LOCK_EVENT_WAIT Sub-task Closed Umesh Agashe
          50.
          		 [SHELL] Purge close_region command that allows by-pass of Master Sub-task Closed Apekshit Sharma
          51.
          		 [AMv2] If Master gives OPEN to another, if original eventually succeeds, Master will kill it Sub-task Closed Umesh Agashe
          52.
          		 [AMv2] Enable aggregation of RPCs (assigns/unassigns, etc.) Sub-task Closed Umesh Agashe
          53.
          		 [AMv2] Split/Merge need cleanup; currently they diverge and do not fully embrace AMv2 world Sub-task Closed Yi Liang
          54.
          		 Redo ProcedureInfo and LockInfo Sub-task Closed Balazs Meszaros
          55.
          		 [AMv2] Remove DispatchMergingRegionsRequest & DispatchMergingRegions Sub-task Closed Yi Liang
          56.
          		 Procedure WALs are archived but not cleaned; fix Sub-task Closed Peter Somogyi
          57.
          		 Assign system tables first (priority) Sub-task Closed Yi Liang
          58.
          		 Verify Master carrying system-tables only with short-circuit RPC Sub-task Closed Unassigned
          59.
          		 create new Async Split API to embrace AM v2 Sub-task Closed Yi Liang
          60.
          		 Deprecate and throw unsupported operation when Admin#closeRegion is called. Sub-task Closed Apekshit Sharma
          61.
          		 Reduce ProcedureInfo usage Sub-task Closed Balazs Meszaros
          62.
          		 Add pv2 and amv2 chapters to refguide Sub-task Closed Michael Stack

          Activity

            People

              syuanjiang Stephen Yuan Jiang
              syuanjiang Stephen Yuan Jiang
              Votes:
              0 Vote for this issue
              Watchers:
              24 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: