Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
In striping input stream, positional read merges all the possible strips together, while stateful read reads a strip a time. The former is efficient but may incur too large chunk buffers for a client to afford, the latter is simple good but can be improved for better throughput. This would consolidate the both and use a configurable (new or existing) buffer threshold to control how it goes. Fixed chunk buffers for the read will be allocated accordingly and reused time and time, as existing stateful read does. The aligned strips to read a time may be computed against the threshold.
Attachments
Issue Links
- requires
-
HDFS-8957 Consolidate client striping input stream codes for stateful read and positional read
- Patch Available