Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-34504

Avoid the parallelism adjustment when the upstream shuffle type doesn't have keyBy

    XMLWordPrintableJSON

Details

    Description

      JobVertexScaler#scale has a optimization: Try to adjust the parallelism such that it divides the number of key groups without a remainder =>  data is evenly spread across subtasks.

       

      It's only useful when the upstream shuffle type has keyBy. We should avoid this optimization when the upstream shuffle type doesn't have keyBy.

      Attachments

        Issue Links

          Activity

            People

              fanrui Rui Fan
              fanrui Rui Fan
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: