Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20046

Reconsider the implementation for serial replication

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.0.0-alpha-1, 2.1.0
    • Replication
    • None
    • Now in replication we can make sure the order of pushing logs is same as the order of requests from client. Set the serial flag to true for a replication peer to enable this feature.

    Description

      When implementing HBASE-9465 we added two new families to meta table, one is rep meta and the other is rep_position. In general I think rep_meta is OK to put into meta table since it records the open and close sequence id for a region, but for rep_position, I think it should be put into another system table instead of meta table.

      This should be done before we finally release 2.0.0 or 1.5.0.

      Attachments

        1. HBASE-20046.pdf
          31 kB
          Duo Zhang

        Issue Links

          is related to

          New Feature - A new feature of the product, which has yet to be developed. HBASE-9465 Push entries to peer clusters serially

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Umbrella - An overarching type made of sub-tasks HBASE-20360 Further optimization for serial replication

          • Major - Major loss of function.
          • Resolved
          relates to

          Task - A task that needs to be done. HBASE-17918 document serial replication

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          links to

          Web Link Design Doc

          1.
          		 Revert serial replication feature Sub-task Resolved Duo Zhang
          2.
          		 Reimplement updateReplicationPositions logic in serial replication based on the newly introduced replication storage layer Sub-task Resolved Zheng Hu
          3.
          		 Reimplement serial replication based on the new replication storage layer Sub-task Resolved Duo Zhang
          4.
          		 Optimize the region last pushed sequence id layout on zk Sub-task Resolved Zheng Hu
          5.
          		 Cleanup the unused replication barriers in meta table Sub-task Resolved Duo Zhang
          6.
          		 Add UT for serial replication after region split and merge Sub-task Resolved Duo Zhang
          7.
          		 Add UT for serial replication after region merge Sub-task Resolved Unassigned
          8.
          		 Add UT for serial replication after failover Sub-task Resolved Duo Zhang
          9.
          		 Add UT for serial replication checker Sub-task Resolved Duo Zhang
          10.
          		 Find a way to deal with the conflicts when updating replication position Sub-task Resolved Zheng Hu
          11.
          		 Serial replication will be stuck if we create a table with serial replication but add it to a peer after there are region moves Sub-task Resolved Duo Zhang
          12.
          		 Make serial replication as a option for a peer instead of a table Sub-task Resolved Duo Zhang
          13.
          		 Shell command to make a normal peer to be a serial replication peer Sub-task Resolved Zheng Hu
          14.
          		 Optimize the implementation of ReplicationSourceWALReader Sub-task Resolved Duo Zhang
          15.
          		 Cleanup replication barrier and last pushed sequence id when deleting table Sub-task Resolved Duo Zhang
          16.
          		 WALEntryStream should not switch WAL file silently Sub-task Resolved Duo Zhang
          17.
          		 NPE when running TestSerialReplication Sub-task Resolved Fabrice MONNIER
          18.
          		 Add UT for ReplicationUtils.contains method Sub-task Resolved Jingyun Tian
          19.
          		 The open sequence number will grow if we fail to open a region after writing the max sequence id file Sub-task Resolved Duo Zhang
          20.
          		 ReplicationSourceWALReader.switched should use the file name instead of the path object directly Sub-task Resolved Duo Zhang
          21.
          		 Delete all last pushed sequence ids when removing a peer or removing the serial flag for a peer Sub-task Resolved Duo Zhang
          22.
          		 Remove last pushed sequence ids when removing tables from a peer Sub-task Resolved Duo Zhang

          Activity

            People

              zhangduo Duo Zhang
              zhangduo Duo Zhang
              Votes:
              0 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: