Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-3889

Revise python streaming mobile gaming examples.

Details

    • Bug
    • Status: Resolved
    • P2
    • Resolution: Duplicate
    • None
    • 2.5.0
    • examples-python
    • None

    Description

      I spent some time trying to run streaming examples [1] on Direct runner and ran into the following issues:

      • Examples use deprecated code as per warnings 
        • DeprecationWarning: options is deprecated since First stable release.. References to <pipeline>.options will not be supported
          project = pipeline.options.view_as(GoogleCloudOptions).project
          /tmp/release_testing/r2.4.0_env/local/lib/python2.7/site-packages/apache_beam/runners/direct/direct_runner.py:337: DeprecationWarning: options is deprecated since First stable release.. References to <pipeline>.options will not be supported
          pipeline.replace_all(_get_transform_overrides(pipeline.options))
      • Path to Injector has a typo (java8 instead of java)
      • I was not able to use the injector to produce an output on a pubsub topic, but perhaps I missed some instructions. Injector would exit after:
        • Warning! You are not using service account credentials to authenticate.
          You need to use service account credentials for this example,
          since user-level credentials do not have enough pubsub quota,
          and so you will run out of PubSub quota very quickly.
        • Note that I set an environment variable GOOGLE_APPLICATION_CREDENTIALS and also tried passing -DGOOGLE_APPLICATION_CREDENTIALS=/path/to/credential.json.
      • Examples say they are not applicable for Dataflow runner, is it still the case?
      • Running injector requires mvn, which is being replaced with gradle and also requires familiarity with java toolchain, even though the example is for Python SDK.

       

      [1] https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/complete/game/game_stats.py.

      Attachments

        Issue Links

          Activity

            People

              altay Ahmet Altay
              tvalentyn Valentyn Tymofieiev
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: