Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-64

DataNode should be capable of managing multiple volumes

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 0.2.0
    • 0.6.0
    • None
    • None

    Description

      The dfs Datanode can only store data on a single filesystem volume. When a node runs its disks JBOD this means running a Datanode per disk on the machine. While the scheme works reasonably well on small clusters, on larger installations (several 100 nodes) it implies a very large number of Datanodes with associated management overhead in the Namenode.

      The Datanod should be enhanced to be able to handle multiple volumes on a single machine.

      Attachments

        1. multiple-volumes.patch
          46 kB
          Milind Barve

        Issue Links

          duplicates

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-74 hash blocks into dfs.data.dirs

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-257 starting one data node thread to manage multiple data directories

          • Major - Major loss of function.
          • Closed
          incorporates

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-50 dfs datanode should store blocks in multiple directories

          • Major - Major loss of function.
          • Closed

          Activity

            People

              milindb Milind Barve
              sameerp Sameer Paranjpye
              Votes:
              2 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: