Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-2832 Enable support for heterogeneous storages in HDFS - DN as a collection of storages
  3. HDFS-5134

NameNode improvements to per-storage block processing

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      Few tasks out of the NameNode change for multiple storage (HDFS-4987):

      1. blockContentsStale should be per storage.
      2. DatanodeDescriptor#firstBlockReport should also be per storage.
      3. BlockManager#processReport can call processFirstBlockReport per storage now.
      4. Synchronization for DatanodeDescriptor#numBlocks (mentioned by Nicholas in 4987).

      Attachments

        1. h5134_20130905.patch
          17 kB
          Tsz-wo Sze
        2. h5134_20130906.patch
          19 kB
          Tsz-wo Sze

        Issue Links

          depends upon

          Sub-task - The sub-task of the issue HDFS-4987 Namenode changes to track multiple storages

          • Major - Major loss of function.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue HDFS-5377 Heartbeats from Datandode should include one storage report per storage directory

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HDFS-5390 Send one incremental block report per storage directory

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              szetszwo Tsz-wo Sze
              arp Arpit Agarwal
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: