Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-25792

Async Sink Base is too being flushed too frequently resulting in backpressure even when buffer is near empty

    XMLWordPrintableJSON

Details

    Description

      Bug:

      Async Sink Base is too being flushed too frequently resulting in backpressure even when buffer is near empty

      Cause:

      During a write(), flushIfAble() is called, which checks if the number of buffered elements is greater than a batch size, and if so, insists that the sink flushes immediately, even if the number of inFlightRequests is greater than the maximum allowed number of inFlightRequests, resulting in a yield of the current mailbox thread, and hence blocks.

      Notice that this can occur even if the buffer is near empty, so the blocking behaviour is unnecessary and undesirable, since we would like the element to be written to the buffer and no blocking to occur.

      Attachments

        Issue Links

          is caused by

          New Feature - A new feature of the product, which has yet to be developed. FLINK-24041 [FLIP-171] Generic AsyncSinkBase

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. FLINK-24905 [FLIP-171] KDS implementation of Async Sink Table API

          • Major - Major loss of function.
          • Closed
          is cloned by

          Bug - A problem which impairs or prevents the functions of the product. FLINK-25976 Update the KDS and KDF Sink's defaults & update the docs

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. FLINK-25793 Fix high throughput rate of AsyncSinkWriter for throttled destination

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #18651

          Activity

            People

              CrynetLogistics Zichen Liu
              CrynetLogistics Zichen Liu
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: