Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-4454 Ozone Streaming Write Pipeline
  3. HDDS-5946

The 'Cannot allocate Memory 'problem occurs in client while test write

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Workaround
    • None
    • None
    • None

    Description

      When I tested write performance with the new client, I got the following exception

      I found the problem started after we implemented retry in HDDS-5674. In the test, I used a single client to write 400 files concurrently. and each file 128MB. The client's memory will become much larger than before.

      The reason for this is that bufferList is used in Retry to maintain references to all byteBuffers currently being written. The client will hold those references to byteBuffer until commit the block. In the past, all Bytebuffers were released after they were sent, but now we need to wait for write successful before releasing. So now the client will accumulate more Bytebuffers and need more memory.

      Attachments

        1. screenshot-3.png
          46 kB
          mingchao zhao
        2. screenshot-2.png
          139 kB
          mingchao zhao
        3. BlockingQueue.patch
          9 kB
          Tsz-wo Sze

        Issue Links

          Activity

            People

              micahzhao mingchao zhao
              micahzhao mingchao zhao
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: