Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6100

DataNodeWebHdfsMethods does not failover in HA mode

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.4.0
    • 2.4.0
    • ha
    • None
    • Reviewed

    Description

      In DataNodeWebHdfsMethods, the code creates a DFSClient to connect to the NN, so that it can access the files in the cluster. DataNodeWebHdfsMethods relies on the address passed in the URL to locate the NN. This implementation has two problems:

      1. The DFSClient only knows about the current active NN, thus it does not support failover.
      2. The delegation token is based on the active NN, therefore the DFSClient will fail to authenticate of the standby NN in secure HA setup.

      Currently the parameter namenoderpcaddress in the URL stores the host-ip pair that corresponds to the active NN. To fix this bug, this jira proposes to store the name service id in the parameter in HA setup (yet the parameter stays the same in non-HA setup).

      Attachments

        1. HDFS-6100.001.patch
          37 kB
          Haohui Mai
        2. HDFS-6100.000.patch
          33 kB
          Haohui Mai

        Issue Links

          Activity

            People

              wheat9 Haohui Mai
              arpitgupta Arpit Gupta
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: