Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8907

Configurable striping read buffer threhold

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • erasure-coding
    • None

    Description

      In striping input stream, positional read merges all the possible strips together, while stateful read reads a strip a time. The former is efficient but may incur too large chunk buffers for a client to afford, the latter is simple good but can be improved for better throughput. This would consolidate the both and use a configurable (new or existing) buffer threshold to control how it goes. Fixed chunk buffers for the read will be allocated accordingly and reused time and time, as existing stateful read does. The aligned strips to read a time may be computed against the threshold.

      Attachments

        Issue Links

          Activity

            People

              drankye Kai Zheng
              drankye Kai Zheng
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: