Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-24905

[FLIP-171] KDS implementation of Async Sink Table API

    XMLWordPrintableJSON

Details

    Description

      Motivation

      User stories:
      As a Flink user, I’d like to use the Table API for the new Kinesis Data Streams  sink.

      Scope:

      • Introduce AsyncDynamicTableSink that enables Sinking Tables into Async Implementations.
      • Implement a new KinesisDynamicTableSink that uses KinesisDataStreamSink Async Implementation and implements AsyncDynamicTableSink.
      • The implementation introduces Async Sink configurations as optional options in the table definition, with default values derived from the KinesisDataStream default values.
      • Unit/Integration testing. modify KinesisTableAPI tests for the new implementation, add unit tests for AsyncDynamicTableSink and KinesisDynamicTableSink and KinesisDynamicTableSinkFactory.
      • Java / code-level docs.

      References

      More details to be found https://cwiki.apache.org/confluence/display/FLINK/FLIP-171%3A+Async+Sink

       

      Attachments

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. FLINK-25792 Async Sink Base is too being flushed too frequently resulting in backpressure even when buffer is near empty

          • Major - Major loss of function.
          • Resolved
          Dependent

          New Feature - A new feature of the product, which has yet to be developed. FLINK-24041 [FLIP-171] Generic AsyncSinkBase

          • Major - Major loss of function.
          • Resolved
          is blocked by

          New Feature - A new feature of the product, which has yet to be developed. FLINK-24234 [FLIP-171] Byte Based & Time Based Flushing for AsyncSinkBase

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. FLINK-24227 [FLIP-171] KDS implementation of Async Sink

          • Major - Major loss of function.
          • Closed
          is cloned by

          New Feature - A new feature of the product, which has yet to be developed. FLINK-26056 CLONE - [FLIP-171] KDS implementation of Async Sink Table API

          • Major - Major loss of function.
          • Closed
          links to

          Web Link GitHub Pull Request #17907

          Web Link GitHub Pull Request #18348

          Web Link GitHub Pull Request #18468

          There are no Sub-Tasks for this issue.

          Activity

            People

              chalixar Ahmed Hamdy
              CrynetLogistics Zichen Liu
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: