Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-9452

PipelineResources algorithm is not working on Windows

Details

    Description

      Causing issue: BEAM-5495

      Deploying a pipeline using

      • Dataflow runner
      • 2.19 SDK
      • Windows OS
        fails with the following stack trace:
      Exception in thread "main" java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
      at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
      at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
      at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
      at org.apache.beam.sdk.Pipeline.create(Pipeline.java:147)
      ...
      Caused by: java.lang.reflect.InvocationTargetException
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:498)
      at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
      ... 8 more
      Caused by: java.lang.IllegalArgumentException: No files to stage has been found.
      at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:281)
      ... 13 more
      

      After digging into the code, the library seems to think that my local jars are remote jars (the remote jar regex matches to windows paths as well):

      1. nonapi.io.github.classgraph.fastzipfilereader.NestedJarHandler:257
      2. io.github.classgraph.ClasspathElementZip:163

      Attachments

        Issue Links

          Activity

            People

              lcwik Luke Cwik
              fpeter8 Peter Farkas
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 20m
                  1h 20m