Details
-
Bug
-
Status: Resolved
-
P2
-
Resolution: Fixed
-
None
-
None
Description
- Bug: the show and collect commands under Interactive Beam would eventually call a PipelineFragment to execute the InteractiveRunner corresponding to the given PCollection. The PipelineFragment by default does not wait for pipeline to finish execution, and when a pipeline runs the FlinkRunner remotely, the Interactive Environment is not updated with the results of the pipeline run. Subsequent calls to ib.show/ib.collect would rerun the pipeline instead of directly retrieving the results from the Interactive Environment.
- Setting the blocking to be True when using the FlinkRunner inside the RecordingManager.record() method when calling the .run() method of the PipelineFragment properly saves completed pipeline results in the Interactive Environment and prevents a need to rerun the pipelines again on subsequent calls to ib.show/ib.collect.
Attachments
Issue Links
- links to
