Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
Description
Add a FlinkKinesisProducer for the Flink Kinesis streaming connector. We will be using AWS SDK implementation for code consistency with the FlinkKinesisConsumer.
The features of FlinkKinesisProducer is rather straightforward:
1. Partition put records based on partition key.
2. Configurable put mode: Bulk put for higher throughput vs. sequential single record puts. Size of bulk should also be configurable.
3. For bulk put, user can also choose to enforce strict ordering of the result with the tradeoff of higher put latency. Ref: https://brandur.org/kinesis-order