Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-29509

Set correct subtaskId during recovery of committables

    XMLWordPrintableJSON

Details

    Description

      When we recover the `CheckpointCommittableManager` we ignore the subtaskId it is recovered on. https://github.com/apache/flink/blob/d191bda7e63a2c12416cba56090e5cd75426079b/flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/sink/committables/CheckpointCommittableManagerImpl.java#L58

      This becomes a problem when a sink uses a post-commit topology because multiple committer operators might forward committable summaries coming from the same subtaskId.

       

      It should be possible to use the subtaskId already present in the `CommittableCollector` when creating the `CheckpointCommittableManager`s.

      Attachments

        Issue Links

          Activity

            People

              KristoffSC Krzysztof Chmielewski
              fpaul Fabian Paul
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: