[FLINK-35549] FLIP-461: Synchronize rescaling with checkpoint creation to minimize reprocessing for the AdaptiveScheduler - ASF JIRA

XML

Word

Printable

JSON

Release Note:

Hide
FLIP-461 enables the user to synchronize checkpointing and rescaling in the AdaptiveScheduler. New configuration parameters were introduced for the maximum trigger delay and the number of acceptable failed checkpoints before triggering a rescale to make this behavior configurable. These parameters were updated in ~~FLINK-36015~~. See the ~~FLINK-36015~~ release notes and Flink's configuration documentation for further details.

Show
FLIP-461 enables the user to synchronize checkpointing and rescaling in the AdaptiveScheduler. New configuration parameters were introduced for the maximum trigger delay and the number of acceptable failed checkpoints before triggering a rescale to make this behavior configurable. These parameters were updated in FLINK-36015 . See the FLINK-36015 release notes and Flink's configuration documentation for further details.

This is the umbrella issue for implementing FLIP-461

1.	Introduce new component RescaleManager	Resolved	Matthias Pohl
2.	Introduces RescaleManager#onTrigger endpoint	Resolved	Matthias Pohl
3.	Move CheckpointStatsTracker out of ExecutionGraph into Scheduler	Resolved	Matthias Pohl
4.	Integrate newly added trigger interface with checkpointing	Resolved	Matthias Pohl