Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8957

Consolidate client striping input stream codes for stateful read and positional read

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • 3.0.0-alpha2
    • None
    • erasure-coding
    • None

    Description

      Currently we have different implementations for client striping read, having both StatefulStripeReader and PositionStripeReader. I attempted to consolidate the two implementations into one, and it results in much simpler codes, and also better performance. Now in both read paths, it will:

      • Use pooled ByteBuffers, as currently stateful read does;
      • Read directly into application's buffer, as currently positional read does;
      • Try to align and merge multiple stripes, as currently positional read does;
      • Use ECChunk version decode API.

      The resultant StripeReader is approaching very near now to the ideal state desired by next step, employing ErasureCoder API instead of RawErasureCoder API.

      Will upload an initial patch to illustrate the rough change, even though it depends on other issues.

      Attachments

        1. HDFS-8957.v4.patch
          58 kB
          Youwei Wang
        2. HDFS-8957.v3.patch
          61 kB
          Youwei Wang
        3. HDFS-8957.v2.patch
          43 kB
          Youwei Wang
        4. HDFS-8957-v1.patch
          36 kB
          Kai Zheng

        Issue Links

          Activity

            People

              Sammi Sammi Chen
              drankye Kai Zheng
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated: