Details
-
Bug
-
Status: Resolved
-
P2
-
Resolution: Duplicate
-
None
-
None
Description
I spent some time trying to run streaming examples [1] on Direct runner and ran into the following issues:
- Examples use deprecated code as per warnings
- DeprecationWarning: options is deprecated since First stable release.. References to <pipeline>.options will not be supported
project = pipeline.options.view_as(GoogleCloudOptions).project
/tmp/release_testing/r2.4.0_env/local/lib/python2.7/site-packages/apache_beam/runners/direct/direct_runner.py:337: DeprecationWarning: options is deprecated since First stable release.. References to <pipeline>.options will not be supported
pipeline.replace_all(_get_transform_overrides(pipeline.options))
- DeprecationWarning: options is deprecated since First stable release.. References to <pipeline>.options will not be supported
- Path to Injector has a typo (java8 instead of java)
- I was not able to use the injector to produce an output on a pubsub topic, but perhaps I missed some instructions. Injector would exit after:
- Warning! You are not using service account credentials to authenticate.
You need to use service account credentials for this example,
since user-level credentials do not have enough pubsub quota,
and so you will run out of PubSub quota very quickly. - Note that I set an environment variable GOOGLE_APPLICATION_CREDENTIALS and also tried passing -DGOOGLE_APPLICATION_CREDENTIALS=/path/to/credential.json.
- Warning! You are not using service account credentials to authenticate.
- Examples say they are not applicable for Dataflow runner, is it still the case?
- Running injector requires mvn, which is being replaced with gradle and also requires familiarity with java toolchain, even though the example is for Python SDK.
Attachments
Issue Links
- is related to
-
BEAM-2484 Add streaming examples for Python
- Open