Details
-
Bug
-
Status: In Progress
-
Major
-
Resolution: Unresolved
-
None
Description
The update fetch positions flow in new consumer attempts to :
- retrieve committed offsets
- reset positions for partitions that may still require one.
In the case that a partition with committed offsets is assigned in-between 1 and 2, the new consumer mistakenly included the newly added partition in the LIST_OFFSETS request issued as part of 2 (so it updates the new partition position using the partition offset and not the committed offsets)
We should ensure that we preserve the initial set of partitions we're initializing, so that the reset excludes partitions that may require a positions but that are not in the initial set. Those partitions should get a position on the next poll iteration (going through steps 1 and 2)