Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-1097

Dataflow error message for non-existing gcpTempLocation is misleading

Details

    Description

      The error message for specifying a GCP tempLocation which doesn't exist is misleading. Rather than mentioning the given path doesn't exist, it says none ways specified.

      This is particularly frustrating because it's one of the few configuration values necessary to get started with an example or starter archetype, and it's easy to introduce a typo as it's specified on the commandline. In my case, I was specifying "gs://swegner-tmp" instead of "gs://swegner-test".

      Repro:

      1. Clone the starter archetype:

      mvn archetype:generate -DarchetypeGroupId=org.apache.beam -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-starter

      2. Add beam-runners-google-cloud-dataflow-java as a dependency in the generated pom.xml
      3. Build:

      mvn install

      4. Run:

      mvn exec:java -DmainClass=swegner.StarterPipeline -Dexec.args="--runner=DataflowRunner --tempLocation=gs://swegner-tmp"

      Expected: An error message along the lines of: "The specified GCP temp location 'gs://swegner-tmp' does not exist under project 'myGcpProject'"

      [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project counter-names-test: An exception occured while executing the Java class. null: InvocationTargetException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions): DataflowRunner requires gcpTempLocation, and it is missing in PipelineOptions. -> [Help 1]

      Attachments

        Issue Links

          Activity

            People

              swegner Scott Wegner
              swegner Scott Wegner
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: