XMLWordPrintableJSON

Details

    Description

      The fields in DatanodeStorageInfo are updated from two distinct paths:

      1. block reports
      2. storage reports (via heartbeats)

      The state and storageType fields are updated via the Block Report. However, as seen in the code blow, these fields are populated from a "dummy" DatanodeStorage object constructed in the DataNode:

      BPServiceActor.blockReport() {
      //...
              // Dummy DatanodeStorage object just for sending the block report.
              DatanodeStorage dnStorage = new DatanodeStorage(storageID);
      //...
      }
      

      The net effect is that the state and storageType fields are always the default of NORMAL and DISK in the NameNode.

      The recommended fix is to change FsDatasetSpi.getBlockReports() from:

      public Map<String, BlockListAsLongs> getBlockReports(String bpid);
      

      to:

      public Map<DatanodeStorage, BlockListAsLongs> getBlockReports(String bpid);
      

      thereby allowing BPServiceActor to send the "real" DatanodeStorage object with the block report.

      Attachments

        1. HDFS-5484-HDFS-2832--2.patch
          16 kB
          Eric Sirianni
        2. HDFS-5484-HDFS-2832.patch
          11 kB
          Eric Sirianni

        Issue Links

          Activity

            People

              sirianni Eric Sirianni
              sirianni Eric Sirianni
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: