Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-19397

Design procedures for ReplicationManager to notify peer change event from master

    XMLWordPrintableJSON

Details

    • Reviewed
    • Hide
      Introduce 5 procedures to do peer modifications:
      AddPeerProcedure
      RemovePeerProcedure
      UpdatePeerConfigProcedure
      EnablePeerProcedure
      DisablePeerProcedure

      The procedures are all executed with the following stage:
      1. Call pre CP hook, if an exception is thrown then give up
      2. Check whether the operation is valid, if not then give up
      3. Update peer storage. Notice that if we have entered this stage, then we can not rollback any more.
      4. Schedule sub procedures to refresh the peer config on every RS.
      5. Do post cleanup if any.
      6. Call post CP hook. The exception thrown will be ignored since we have already done the work.

      The procedure will hold an exclusive lock on the peer id, so now there is no concurrent modifications on a single peer.

      And now it is guaranteed that once the procedure is done, the peer modification has already taken effect on all RSes.

      Abstracte a storage layer for replication peer/queue manangement, and refactored the upper layer to remove zk related naming/code/comment.

      Add pre/postExecuteProcedures CP hooks to RegionServerObserver, and add permission check for executeProcedures method which requires the caller to be system user or super user.

      On rolling upgrade: just do not do any replication peer modifications during the rolling upgrading. There is no pb/layout changes on the peer/queue storage on zk.
      Show
      Introduce 5 procedures to do peer modifications: AddPeerProcedure RemovePeerProcedure UpdatePeerConfigProcedure EnablePeerProcedure DisablePeerProcedure The procedures are all executed with the following stage: 1. Call pre CP hook, if an exception is thrown then give up 2. Check whether the operation is valid, if not then give up 3. Update peer storage. Notice that if we have entered this stage, then we can not rollback any more. 4. Schedule sub procedures to refresh the peer config on every RS. 5. Do post cleanup if any. 6. Call post CP hook. The exception thrown will be ignored since we have already done the work. The procedure will hold an exclusive lock on the peer id, so now there is no concurrent modifications on a single peer. And now it is guaranteed that once the procedure is done, the peer modification has already taken effect on all RSes. Abstracte a storage layer for replication peer/queue manangement, and refactored the upper layer to remove zk related naming/code/comment. Add pre/postExecuteProcedures CP hooks to RegionServerObserver, and add permission check for executeProcedures method which requires the caller to be system user or super user. On rolling upgrade: just do not do any replication peer modifications during the rolling upgrading. There is no pb/layout changes on the peer/queue storage on zk.

    Description

      After we store peer states / peer queues information into hbase table, RS can not track peer config change by adding watcher znode.

      So we need design procedures for ReplicationManager to notify peer change event. the replication rpc interfaces which may be implemented by procedures are following: 

      1. addReplicationPeer
2. removeReplicationPeer
3. enableReplicationPeer
4. disableReplicationPeer
5. updateReplicationPeerConfig

      BTW, our RS states will still be store in zookeeper, so when RS crash, the tracker which will trigger to transfer queues of crashed RS will still be a Zookeeper Tracker. we need NOT implement that by procedures.

      As we will release 2.0 in next weeks, and the HBASE-15867 can not be resolved before the release, so I'd prefer to create a new feature branch for HBASE-15867.

      Attachments

        1. HBASE-19397-master.patch
          732 kB
          Duo Zhang
        2. HBASE-19397-master-v1.patch
          733 kB
          Duo Zhang
        3. HBASE-19397-master-v1.patch
          733 kB
          Duo Zhang
        4. HBASE-19397-branch-2.patch
          733 kB
          Duo Zhang
        5. HBASE-19397-master-v2.patch
          732 kB
          Duo Zhang

        Issue Links

          blocks

          New Feature - A new feature of the product, which has yet to be developed. HBASE-19064 Synchronous replication for HBase

          • Major - Major loss of function.
          • Resolved
          contains

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-13771 Replication clients should not access zookeeper directly

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          is a parent of

          Sub-task - The sub-task of the issue HBASE-17615 Use nonce and procedure v2 for add/remove replication peer

          • Major - Major loss of function.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue HBASE-20082 Fix findbugs errors only on master which are introduced by HBASE-19397

          • Major - Major loss of function.
          • Resolved

          Task - A task that needs to be done. HBASE-19737 Manage a HBASE-19397-branch-2 branch and merge it to branch-2

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link Review Board

          There are no Sub-Tasks for this issue.

          Activity

            People

              zhangduo Duo Zhang
              openinx Zheng Hu
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: