Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.2.0
    • 3.3.0
    • fs/azure
    • None

    Description

      In current AbfsOutPutStream, new temporary buffers always get allocated when uploading the data to server. These big chunk creates huge pressure on the GC, and in some extreme cases that GC doesn't happen in time(observed in some customer's env), it also lead to OutOfMemory issue easily.
      Hence we need to replace the current implementation with a bufferpool.

      Attachments

        1. HADOOP-16242-001.patch
          3 kB
          Da Zhou
        2. WritePerformanceWithBufferPool.pdf
          159 kB
          Da Zhou

        Issue Links

          Activity

            People

              DanielZhou Da Zhou
              DanielZhou Da Zhou
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: