Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-3816

Erasure Coding

    XMLWordPrintableJSON

Details

    Description

      We propose to implement Erasure Coding in Apache Ozone to provide efficient storage. With EC in place, Ozone can provide same or better tolerance by giving 50% or more storage space savings.
      In HDFS project, we already have native codecs(ISAL) and Java codecs implemented, we can leverage the same or similar codec design.

      However, the critical part of EC data layout design is in-progress, we will post the design doc soon.

      Also see HDDS-5351, which has a bunch of pre-requisites for the EC feature committed directly to the master branch.

      Attachments

        1. Ozone EC v3.pdf
          1.16 MB
          Marton Elek
        2. Ozone EC Container groups and instances.pdf
          205 kB
          Marton Elek
        3. EC-read-write-path.pdf
          170 kB
          Marton Elek
        4. Apache Ozone Erasure Coding-V2.pdf
          294 kB
          Uma Maheswara Rao G
        5. Erasure Coding in Apache Hadoop Ozone.pdf
          375 kB
          Marton Elek

        Issue Links

          incorporates

          Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. HDDS-6462 Phase II : Erasure Coding Offline Recovery & Read/Write Improvements

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDDS-7731 Phase III: Erasure Coding followup tasks

          • Major - Major loss of function.
          • Open
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-7003 Support erasure-coding in impala

          • Major - Major loss of function.
          • Resolved
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. HDDS-3755 Storage-class support for Ozone

          • Major - Major loss of function.
          • Open
          requires

          New Feature - A new feature of the product, which has yet to be developed. HDDS-5351 Erasure Coding Prerequisites

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDDS-6390 Facilities for compatibility - umbrella jira

          • Major - Major loss of function.
          • Open
          links to

          Web Link GitHub Pull Request #2868

          Web Link GitHub Pull Request #3301

          Web Link Upgrade framework integration - problems and possible solutions

          There are no Sub-Tasks for this issue.

          Activity

            People

              umamaheswararao Uma Maheswara Rao G
              umamaheswararao Uma Maheswara Rao G
              Votes:
              1 Vote for this issue
              Watchers:
              39 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: