Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-2856

Fix block protocol so that Datanodes don't require root or jsvc

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.4.0, 3.0.0-alpha1
    • 2.6.0
    • datanode, security
    • None
    • Reviewed
    • Hide
      SASL now can be used to secure the DataTransferProtocol, which transfers file block content between HDFS clients and DataNodes. In this configuration, it is no longer required for secured clusters to start the DataNode as root and bind to privileged ports.
      Show
      SASL now can be used to secure the DataTransferProtocol, which transfers file block content between HDFS clients and DataNodes. In this configuration, it is no longer required for secured clusters to start the DataNode as root and bind to privileged ports.

    Description

      Since we send the block tokens unencrypted to the datanode, we currently start the datanode as root using jsvc and get a secure (< 1024) port.

      If we have the datanode generate a nonce and send it on the connection and the sends an hmac of the nonce back instead of the block token it won't reveal any secrets. Thus, we wouldn't require a secure port and would not require root or jsvc.

      Attachments

        1. Datanode-Security-Design.pdf
          93 kB
          Chris Nauroth
        2. Datanode-Security-Design.pdf
          95 kB
          Chris Nauroth
        3. Datanode-Security-Design.pdf
          95 kB
          Chris Nauroth
        4. HDFS-2856.prototype.patch
          32 kB
          Chris Nauroth
        5. HDFS-2856.1.patch
          137 kB
          Chris Nauroth
        6. HDFS-2856.2.patch
          143 kB
          Chris Nauroth
        7. HDFS-2856.3.patch
          147 kB
          Chris Nauroth
        8. HDFS-2856-Test-Plan-1.pdf
          154 kB
          Chris Nauroth
        9. HDFS-2856.4.patch
          149 kB
          Chris Nauroth
        10. HDFS-2856.5.patch
          150 kB
          Chris Nauroth
        11. HDFS-2856.6.patch
          150 kB
          Chris Nauroth
        12. HDFS-2856-branch-2.7.patch
          157 kB
          Chris Nauroth
        13. HDFS-2856.7.patch
          150 kB
          Chris Nauroth

        Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. HDFS-7382 DataNode in secure mode may throw NullPointerException if client connects before DataNode registers itself with NameNode.

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-6600 fsck -move fails in secured clusters.

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. HDFS-3749 Disable check for jsvc on windows

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-7073 Allow falling back to a non-SASL connection on DataTransferProtocol in several edge cases.

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HDFS-7367 HDFS short-circuit read cannot negotiate shared memory slot and file descriptors when SASL is enabled on DataTransferProtocol.

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-6859 Allow dfs.data.transfer.protection default to hadoop.rpc.protection

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-6606 Optimize HDFS Encrypted Transport performance

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-7195 Update user doc of secure mode about Datanodes don't require root or jsvc

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-6858 Allow dfs.data.transfer.saslproperties.resolver.class default to hadoop.security.saslproperties.resolver.class

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-7386 Replace check "port number < 1024" with shared isPrivilegedPort method

          • Trivial - Cosmetic problem like misspelt words or misaligned text.
          • Closed
          relates to

          New Feature - A new feature of the product, which has yet to be developed. HDFS-3637 Add support for encrypting the DataTransferProtocol

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-9671 Improve Hadoop security - Use cases, Threat Model and Problems

          • Major - Major loss of function.
          • Open

          Activity

            People

              cnauroth Chris Nauroth
              omalley Owen O'Malley
              Votes:
              0 Vote for this issue
              Watchers:
              38 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: