Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
2.6.0
-
None
Description
Aggregate stream consumers are not currently able to get the end timestamp for Time Windows from the Deserializer. This is due to the fact that the end timestamp is not present in the message key. The end timestamp is not currently added to the end of the key for Time Window aggregates.
Allowing the Consumer to set the window size in the Deserializer's constructor is not sufficient as an objective consumer would not know what window size was being used. It also stands to reason that the aggregator could choose to change the window without notifying the consumer: e.g., 1 minute windows during business hours and 1 hour off business hours. It would also be possible for the same consumer to be consuming several topics at once, each with a different window on each topic.
Update the code to add the end timestamp long onto the end of the key and update the Deserializer to extract and set the correct end on the Window.
This issue is peripherally related to KAFKA-4468, KAFKA-9649 and KAFKA-10366.