Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14603 Über-JIRA: HDFS RBF stabilization phase II
  3. HDFS-13248

RBF: Namenode need to choose block location for the client

    XMLWordPrintableJSON

Details

    Description

      When execute a put operation via router, the NameNode will choose block location for the router, not for the real client. This will affect the file's locality.

      I think on both NameNode and Router, we should add a new addBlock method, or add a parameter for the current addBlock method, to pass the real client information.

      Attachments

        1. clientMachine-call-path.jpeg
          171 kB
          Wu Weiwei
        2. debug-info-1.jpeg
          266 kB
          Wu Weiwei
        3. debug-info-2.jpeg
          242 kB
          Wu Weiwei
        4. HDFS-13248.000.patch
          6 kB
          Íñigo Goiri
        5. HDFS-13248.001.patch
          8 kB
          Íñigo Goiri
        6. HDFS-13248.002.patch
          4 kB
          Hui Fei
        7. HDFS-13248.003.patch
          8 kB
          Hui Fei
        8. HDFS-13248.004.patch
          9 kB
          Hui Fei
        9. HDFS-13248.005.patch
          9 kB
          Hui Fei
        10. HDFS-Router-Data-Locality.odt
          26 kB
          Ayush Saxena
        11. RBF Data Locality Design.pdf
          91 kB
          Xiaoqiao He

        Issue Links

          duplicates

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-16015 RBF: Read and write data through router select dn according to real user ip

          • Major - Major loss of function.
          • Open
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-16756 RBF proxies the client's user by the login user to enable CacheEntry

          • Major - Major loss of function.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue HDFS-13293 RBF: The RouterRPCServer should transfer client IP via CallerContext to NamenodeRpcServer

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. HADOOP-16254 Add proxy address in IPC connection

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved
          links to

          Web Link GitHub Pull Request #4081

          Activity

            People

              omalley Owen O'Malley
              wuweiwei Wu Weiwei
              Votes:
              0 Vote for this issue
              Watchers:
              22 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 5h 10m
                  5h 10m