Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-2832

Enable support for heterogeneous storages in HDFS - DN as a collection of storages

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.0.0-alpha
    • 2.3.0
    • datanode, namenode
    • None

    Description

      HDFS currently supports configuration where storages are a list of directories. Typically each of these directories correspond to a volume with its own file system. All these directories are homogeneous and therefore identified as a single storage at the namenode. I propose, change to the current model where Datanode * is a * storage, to Datanode * is a collection * of strorages.

      Attachments

        1. h2832_branch-2_20140103.patch
          566 kB
          Arpit Agarwal
        2. editsStored
          4 kB
          Arpit Agarwal
        3. h2832_branch-2_20131226.patch
          566 kB
          Arpit Agarwal
        4. h2832_20131211b.patch
          574 kB
          Arpit Agarwal
        5. h2832_20131211.patch
          574 kB
          Arpit Agarwal
        6. h2832_20131210.patch
          578 kB
          Arpit Agarwal
        7. h2832_20131203.patch
          577 kB
          Arpit Agarwal
        8. 20131203-HeterogeneousStorage-TestPlan.pdf
          44 kB
          Arpit Agarwal
        9. 20131202-HeterogeneousStorage-TestPlan.pdf
          44 kB
          Arpit Agarwal
        10. h2832_20131202.patch
          576 kB
          Arpit Agarwal
        11. editsStored
          5 kB
          Arpit Agarwal
        12. 20131125-HeterogeneousStorage-TestPlan.pdf
          44 kB
          Arpit Agarwal
        13. 20131125-HeterogeneousStorage.pdf
          80 kB
          Arpit Agarwal
        14. h2832_20131124.patch
          576 kB
          Arpit Agarwal
        15. h2832_20131123.patch
          568 kB
          Tsz-wo Sze
        16. h2832_20131122b.patch
          562 kB
          Tsz-wo Sze
        17. h2832_20131122.patch
          575 kB
          Arpit Agarwal
        18. h2832_20131121.patch
          564 kB
          Arpit Agarwal
        19. h2832_20131119b.patch
          565 kB
          Arpit Agarwal
        20. h2832_20131119.patch
          568 kB
          Arpit Agarwal
        21. h2832_20131118.patch
          569 kB
          Arpit Agarwal
        22. h2832_20131114.patch
          567 kB
          Arpit Agarwal
        23. h2832_20131112b.patch
          564 kB
          Arpit Agarwal
        24. h2832_20131112.patch
          563 kB
          Arpit Agarwal
        25. h2832_20131111.patch
          563 kB
          Arpit Agarwal
        26. h2832_20131110b.patch
          564 kB
          Arpit Agarwal
        27. h2832_20131110.patch
          541 kB
          Arpit Agarwal
        28. h2832_20131108.patch
          505 kB
          Tsz-wo Sze
        29. h2832_20131107b.patch
          519 kB
          Arpit Agarwal
        30. H2832_20131107.patch
          518 kB
          Arpit Agarwal
        31. h2832_20131105.patch
          517 kB
          Arpit Agarwal
        32. h2832_20131104.patch
          500 kB
          Tsz-wo Sze
        33. h2832_20131103.patch
          497 kB
          Tsz-wo Sze
        34. h2832_20131029.patch
          505 kB
          Arpit Agarwal
        35. h2832_20131028b.patch
          506 kB
          Junping Du
        36. h2832_20131028.patch
          506 kB
          Arpit Agarwal
        37. h2832_20131025.patch
          456 kB
          Tsz-wo Sze
        38. h2832_20131023b.patch
          452 kB
          Tsz-wo Sze
        39. h2832_20131023.patch
          455 kB
          Tsz-wo Sze
        40. 20130813-HeterogeneousStorage.pdf
          80 kB
          Arpit Agarwal

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-6702 DFSClient should create blocks using StorageType

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. HDFS-4949 Centralized cache management in HDFS

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. HDFS-6584 Support Archival Storage

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-7343 HDFS smart storage management

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-6700 BlockPlacementPolicy shoud choose storage but not datanode for deletion

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          relates to

          Sub-task - The sub-task of the issue HDFS-7390 Provide JMX metrics per storage type

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. HDFS-4672 Support tiered storage policies

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. HDFS-7285 Erasure Coding Support inside HDFS

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. HDFS-5682 Heterogeneous Storage phase 2 - APIs to expose Storage Types

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-3672 Expose disk-location information for blocks to enable better scheduling

          • Major - Major loss of function.
          • Closed

          Brainstorming - A place to record back and forth on notions not yet formed enough to make a 'New Feature' or 'Task' HBASE-6572 Tiered HFile storage

          • Major - Major loss of function.
          • Closed

          Activity

            People

              arp Arpit Agarwal
              sureshms Suresh Srinivas
              Votes:
              0 Vote for this issue
              Watchers:
              88 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: