Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4588

Map local segments as on-disk segments

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • task
    • None

    Description

      Local map segments should never be handled as though they were remote (i.e., copied through a servlet to local disk). This optimization is uniformally more efficient for the fetch, though it increases the number of on-disk segments. Each segment has its own overhead, which can exceed the cost of pulling it into memory (e.g., the decompressor overhead for an active segment exceeds the cost of decompression into memory). Some logic is required to handle a large number of such segments.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              cdouglas Christopher Douglas
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated: