Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-12341

Use portable job submission for Go jobs to dataflow.

Details

    • Improvement
    • Status: Resolved
    • P2
    • Resolution: Fixed
    • None
    • 2.31.0
    • runner-dataflow, sdk-go
    • None

    Description

      1. Need to  set the "use_portable_job_submission" when sending jobs to Dataflow.
      2. The Go SDK's portable format doesn't conform to how other SDKs generate their graph, so simply switching the feature on will cause composite scopes to not be represented in the runner's job view.  Since the Go SDK graph runs through a separate sort and uniquifying process, it can't have these names done at construction time, but after converting the graph from SDK to proto. That represents a clear place to update all the names.

      Attachments

        Issue Links

          Activity

            People

              lostluck Robert Burke
              lostluck Robert Burke
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 20m
                  1h 20m