Details
-
Improvement
-
Status: Resolved
-
Minor
-
Resolution: Fixed
-
kafka-3.2.0
Description
This issue is a follow-up to this mailing list discussion.
I'd like to propose letting the ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG be overridable by user in KafkaSourceBuilder, as shown in this DRAFT PR:
https://github.com/apache/flink-connector-kafka/pull/108
From the PR description:
This allows users to easily implement the claim check large message pattern without bringing any concerns into the Flink codebase otherwise, by specifying a value.deserializer that handles it, but otherwise passes through the bytes.
Note: overriding value.serializer is already supported on the Producer side:
Other Reading:
https://www.kai-waehner.de/blog/2020/08/07/apache-kafka-handling-large-messages-and-files-for-image-video-audio-processing/
What do folks think? If it seems reasonable I can follow the steps in the contribution guide.
Attachments
Issue Links
- links to