Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-15087

RBF: Balance/Rename across federation namespaces

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      The Xiaomi storage team has developed a new feature called HFR(HDFS Federation Rename) that enables us to do balance/rename across federation namespaces. The idea is to first move the meta to the dst NameNode and then link all the replicas. It has been working in our largest production cluster for 2 months. We use it to balance the namespaces. It turns out HFR is fast and flexible. The detail could be found in the design doc. 

      Looking forward to a lively discussion.

      Attachments

        1. HFR_Rename Across Federation Namespaces.pdf
          234 kB
          Jinglun
        2. HDFS-15087.initial.patch
          356 kB
          Jinglun

        Issue Links

          is related to

          New Feature - A new feature of the product, which has yet to be developed. HDFS-15747 RBF: Rename across sub-namespaces.

          • Major - Major loss of function.
          • Open
          relates to

          New Feature - A new feature of the product, which has yet to be developed. HDFS-15294 Federation balance tool

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-13123 RBF: Add a balancer tool to move data across subcluster

          • Major - Major loss of function.
          • Patch Available

          Activity

            People

              LiJinglun Jinglun
              LiJinglun Jinglun
              Votes:
              0 Vote for this issue
              Watchers:
              28 Start watching this issue

              Dates

                Created:
                Updated: